Summary
In this chapter, we’ve examined a number of LLMs available in the public domain:
- Llama
- Mixtral
- Dolly
- Falcon
- Grok
Unlike closed-source models, which we might only interact with through an Application Programming Interface (API) or an end user service like ChatGPT, these open-source models expose the architecture and model parameters. This opens the door to flexible fine-tuning, where we can potentially isolate different layers of the network for customization, using techniques such as quantization or distillation to compact models (as we’ll discuss in Chapter 10), or implementing custom transformations on the output. We can also manage version updates more transparently through direct access to the weights, while updates in service-based models may be harder to track.
We’ve seen how we can use these open-source models to perform coding tasks, answer general knowledge questions, and solve reasoning problems...
