Machine Learning: Natural Language Processing by Stas Bekman


My name is Stas Bekman and I'm a software engineer who enjoys tinkering, building reliable systems and excells at identifying and solving problems.

I have been writing software since 1994.

I have worked in multiple domains, for many years taught at major tech conferences and user groups, published several books, and currently I specialize in the natural language processing (NLP) domain of machine learning.

I currently work primarily with the Pytorch ecosystem.

Sequence to sequence NLP

Currently I work on ML translation and other sequence to sequence tasks.

Performance optimization and problem solving

While I can build a whole system from the ground up, I have a knack, intuition and an extended experience dealing with a variety of problems in software. In particular, I'm good at identifying and sorting out performance issues, such as memory leaks, speed bottlenecks, but also various other types of bugs in systems.

Most of the time ML users don't realize that they waste money and resources because their programs are inefficient. Surely you can always throw more money at your project to buy/rent more powerful hardware and waste more electricity, but very often you can instead make your software lean and efficient.

I am based in beautiful Lantzville, BC, Canada.

Current Work

I've been working at Hugging Face since January 2021. You can see my contributions to the HuggingFace Transformers library. I'm involved in many other projects like Deepspeed, PyTorch and others.

One of the most recent major accomplishments was the massive training on 384 A100 GPUs of the 176B parameters BLOOM model, which as of this moment is the Largest Open-Access Multilingual language model. I was the lead engineer on this project.



Note: if you have any questions with regards to HuggingFace projects, like HF Transformers, please use the corresponding Issue and we will discuss it there. You can use Issues to ask questions. But please do not email me with HuggingFace-related questions. If you do I will reply with this same request. Thank you.

Some samples of my problem solving in the open source projects