Unsloth: A Fine-Tuning Guide for Developers

Introduction to Fine-Tuning

Fine-tuning is the process of adjusting a pre-trained model to fit a specific task or use case. It is an essential part of developing a successful model. Fine-tuning is great for developers who want to use an existing model but may need to make some adjustments to better suit their needs. It is much simpler and faster than creating a new model.

Fine-Tuning with Unsloth

In this article, we’ll take a look at how we can fine-tune the Meta Llama 3.1 8B LLM using Beam with Unsloth.

In the following demo, you will see how to use the alpaca-cleaned dataset from Hugging Face to fine-tune the Meta Llama LLM with Unsloth; thus improving its instruction-following capabilities.

Unsloth is a Python package that allows developers to quickly and efficiently fine-tune models like Llama. The advantages of using Unsloth for fine-turning include memory optimization, speed and efficiency, compatibility with different hardware, and its open-source nature.

Getting Started with Unsloth

First, let’s import the necessary packages and set up our environment in finetune.py with our Image container where we define our model and dependencies:

Let’s start our fine-tuning function by defining our model and tokenizer:

Here we have defined our model as a FastLanguageModel from the Unsloth package.

Now we’ll create some helper functions within the fine_tune_model function to format the alpaca-cleaned dataset for our model, then we’ll load the data and perform the transformation:

We loaded our model earlier, but now we’ll use FastLanguageModel.get_peft_model to attach adapters in order to perform the fine-tuning:

We’ll take a deeper look into some of the parameters used to fine-tune the model in a later section.

Now we’ll use the supervised fine-tuning training (SFT Trainer) which is a specialized tool within Hugging Face’s ecosystem designed to streamline the fine-tuning process for language models like the one we are using. We’ll pass our model, tokenizer, and training data to the trainer along with some more parameters to complete the model training:

Lastly, we will train the model and save the results in our output_dir:

You can check out the full code repository here.

Deploying the Fine-Tuned Model

After executing the fine-tuning script, you should verify that the files are saved in your Beam Volume:

The expected output with the fine-tuned files should look like:

You then need to run the interference script which you can find here. The script (inference.py) loads the fine-tuned model and exposes an endpoint for generating responses.

Once you deploy the endpoint beam deploy inference.py:generate, you will get back a URL with the endpoint and can now utilize your fine-tuned LLM.

Fine-Tuning with Hugging Face

The dataset used to fine-tune LLama 3.1B is the alpaca-cleaned dataset which you can view here. This dataset is a cleaned version of the Alpaca dataset, which contains over 52,00 instructions and demonstrations generated using OpenAI’s text-davinci-003 engine. The data can be used to fine-tune models, improving their ability to follow instructions. Some minor changes are needed before training the model on the data. After loading the dataset, format_dataset is called which then calls format_alpaca_prompt. Format_alpaca_prompt alters the data to be formatted like: 

This data transformation helps fine-tune the model because it standardizes the format of instruction-based prompts, ensuring a consistent input-output structure for training. This improves the model's ability to learn task-specific responses.

Optimizing Fine-Tuning for Fast Inference

The goal of Unsloth is to optimize the training and fine-tuning of LLMs. In our case, we want to fine-tune LLama to be more capable of responding to instructions. There are tons of different model parameters that can be changed to improve model performance and choosing the right parameters is a crucial piece of fine-tuning. We want a highly accurate model but don’t want to over or underfit. Let's take a look at some of the model parameters we can modify:

• Learning Rate: How much the model weights adjust per training step
• Epochs: The number of times the model sees the full training data
• Gradient Accumulation: The number of steps over which gradients are accumulated before performing a backward update or optimizer step

In our code we defined the parameters in the TRAIN_CONFIG object:

While most models have default parameters that they recommend following, developers often hyper-tune parameters to optimize performance. You can find additional information about using Unsloth for fine-tuning here.

Conclusion

In this article, we looked at fine-tuning Meta LLaMA 3.1B LLM using Unsloth by adding task-specific data from Hugging Face. This method allows the model to better adapt to the context and nuances of instruction following while maintaining the strength of the pre-trained model. The fine-tuning process is efficient and effective, resulting in an improved model without requiring full retraining.