Hands-on: Instruction tuning
In this section, we will explore the concept of instruction tuning a language model using the Hugging Face library and public dataset.
Problem statement
Translate English to German using a pretrained transformer model in the context of instruction tuning. The task at hand is to extend the capabilities of a GPT-2 model to translate English text to German using instruction tuning. The training objective for instruction tuning remains the same as language modeling (as in the pretraining step) and unlike a typical SFT scenario, where we use sequential modeling for such a task.
The original paper presents the InstructGPT model based on GPT-3 architecture. For the purposes of developing an understanding while keeping compute requirements to a minimum, we illustrate the instruction-tuning setup using GPT-2. If you have access to larger compute/more GPU RAM, you can easily adapt the notebook to larger models, such as Phi-2 or the llama series...
