Howdy AI friends, 

Today, let’s picture a cook who reads through a recipe, looks in their cupboard, and realizes they want to personalize it. So, the cook is brave and starts picking up some new ingredients and adding them to the original recipe, starting to fine-tune it. I am confident that many of you did the same, too. So, fine-tuning LLMs is similar.  

Fine-tuning LLMs with model distillation

Imagine your AI model as a blank canvas. It’s capable of great things but needs a little guidance to become a masterpiece. That’s where fine-tuning comes in. Think of it as adding a secret ingredient to your AI recipe, transforming it into a dish perfectly tailored to your taste. 

So, what exactly is fine-tuning?  

It’s essentially teaching your AI model to perform a specific task by providing it with additional data and training. It’s like giving a chef a new cookbook and some exotic ingredients. With some experimentation and practice, the chef can create a dish that’s even better than the original recipe. 

Here’s a quick breakdown of the fine-tuning process: 

– Choose your base model: Start with a pre-trained model already trained on a massive dataset. It is like using a proven recipe as a starting point. 

– Prepare your data: Gather a dataset relevant to the task you want your model to perform. Think of this as collecting the ingredients you need for your dish. 

– Train your model: Feed your data to the model and let it learn. It is like cooking the dish according to the recipe. 

– Evaluate and adjust: Test your model’s performance and adjust as needed. It is like tasting the dish and tweaking the seasoning. 

I highly recommend this blog post from Meta about fine-tuning. It provides a comprehensive understanding of the technique within the broader topic of adapting LLMs to domain data. It’s a valuable resource that discusses opportunities (for example, emerging abilities) against drawbacks (choosing the proper fine-tuning method, costs, etc.)  

A pragmatic approach to fine-tuning LLMs with data anonymization

What are the typical use cases of fine-tuning? Fine-tuning is not just crucial, it’s indispensable for customizing AI models to specific tasks, addressing data scarcity, and ensuring data privacy. Its value is especially pronounced in regulated industries and in cases where on-premises deployment is required. Smaller models may struggle in these scenarios, making fine-tuning a vital and impactful tool. 

One example can help us understand the application of fine-tuning. DATEV processes thousands of customer feedbacks every day for its software. As a European company, it must comply with GDPR and anonymize sensitive data before using it for business purposes. It involves identifying and replacing sensitive information with anonymized words while preserving data value and regulatory compliance. Damir Kopljar and the team have implemented a transformer-based named-entity-recognition model to address this challenge using AWS and Databricks as tech stacks and then the Microsoft Azure ecosystem.

Model distillation in fine-tuning

One important step in fine-tuning is data curation. For example, the DATEV data anonymization project included transforming existing data sets, large-scale data collection and labeling, and synthetic data generation. In this blog, Ivan Križanić describes the state-of-the-art approach to address these issues: knowledge distillation.

Knowledge distillation is a technique to create smaller, more efficient versions of large, pre-trained models. It works by transferring the knowledge from the large model to the smaller model, allowing the smaller model to perform tasks almost as well as the larger model but with fewer resources. Ivan’s blog not only describes the state-of-the-art approach but also provides a very informative and complete guide, combining both the technical description with the complexity of knowledge distillation in a use case.

This blog from Datacamp is quite comprehensive for those interested in a primer, while here is a Pytorch tutorial for those wanting to familiarize themselves with the code.

Fine-tuning is one step in the journey, it is not the end

Remember that fine-tuning is just one step for LLM in production. In the DATEV use case, fine-tuning was necessary to analyze anonymized data and work towards a better understanding of thousands of daily customers’ feedback. I am very fascinated by how Dr. Jonas Rende and Mareike Hoffman explain the journey from fine-tuning to sentiment analysis in this QED video.  

Finally, while fine-tuning allows the model to be tailored specifically to the desired task, much like a dish crafted to suit the chef’s unique palate, this customization can sometimes result in a model that is less versatile and may not perform as well on more general tasks. This specialized tuning can limit the model’s broader applicability, making it less effective in scenarios outside its fine-tuned domain. 

This is my view of a pragmatic approach to fine-tuning. In the next issue, I will discuss generating synthetic data for improving LLM performance. Synthetic data generation is an essential topic within fine-tuning and relates to data curation for fine-tuning. With research contributions and emerging use cases for business applications, synthetic data generation has gained increasing attention. Let me stop here so as not to spoil the next edition of AI a la Carte.