Fine-tuning optimizations
The pre-training step is by far the biggest in terms of data and compute requirements for the whole of the LLM’s lifecycle. Yet fine-tuning is quite resource-intensive when we compare it to traditional machine learning and deep learning workflows. Fine-tuning is also a very important step in improving the quality of the models; hence, it makes sense to understand how we can optimize this step without impacting the performance. Efficiencies in this step also enable us to iterate faster, thereby improving adaptability in many fast-moving domains. In this section, we will focus on some interesting efficient methods.
Parameter efficient fine-tuning
In the traditional setting, fine-tuning the model refers to updating all parameters of a given model for a specific downstream task. This is not only expensive in terms of time and compute costs but is also becoming increasingly difficult due to the extremely large size of models. In the recent past...
