Fine-tuning on a collection of tasks described by natural language instructions.
Instruction tuning is a specialized form of supervised fine-tuning that has proven to be incredibly effective at improving the usability and zero-shot performance of LLMs. Instead of fine-tuning on a dataset for a single specific task, instruction tuning involves fine-tuning the model on a large and diverse collection of tasks, where each example is formatted as a natural language instruction. The dataset consists of (instruction, input, output) triplets. For example, an instruction could be 'Translate the following sentence to French,' the input could be 'Hello, how are you?,' and the output would be 'Bonjour, comment ca va?'. Another example could be 'Summarize this text,' with an article as input and a summary as output. By training on thousands of different tasks described in this instructional format, the model learns to become a general-purpose 'instruction-following' agent. It learns the general concept of what it means to follow a command, rather than just learning to solve one specific problem. This has several key benefits. First, it significantly improves the model's performance on unseen tasks. After instruction tuning, the model can often perform a new task described in an instruction in a zero-shot fashion, without needing any specific examples for that new task. Second, it makes the models more aligned with user intent and easier to interact with, as they are explicitly trained to respond to commands. Models like Flan-T5 and the instruction-tuned versions of Llama and GPT (like InstructGPT) demonstrate the power of this technique, showing massive performance gains on a wide range of academic benchmarks compared to their non-instruction-tuned base models.