Dolly
LLaMA3 and Mixtral-8x7B are both trained on huge amounts of web data. The next open model we’ll examine, “Dolly,” was created by the company DataBricks to illustrate the power of fine-tuning with smaller datasets. The original version of the Dolly model was created by DataBricks to illustrate how the instruction-following abilities of ChatGPT described in the InstructGPT paper12 can be replicated in smaller models using high-quality datasets.
Instruction-following models are created through additional training on LLMs following the initial training, which focuses on predicting the next token in a prompt given a context window of input text. The textual output generated by this next-token predictor is not well-suited for complex tasks such as brainstorming ideas, summarizing content, or question and answer, nor does it have the toxicity and safety filters needed for commercial use.
Thus, these first-stage models are further refined using Reinforcement...
