From the course: Introduction to Large Language Models

Unlock the full course today

Join today to access over 24,300 courses taught by industry experts.

LLM trends

LLM trends

From the course: Introduction to Large Language Models

Start my 1-month free trial Buy for my team

LLM trends

“

- [Instructor] Over this course, we've looked at scaling laws and lessons from the Chinchilla models. Let's bring this all together as we look at the current trends for large language models. When training of large language models first kicked off, most of the focus was on improving the models, and then the scaling laws were the guiding principle. And the scaling laws suggested that you would get the biggest improvement by increasing the size of the models. So you do this by increasing the model's architecture, so that's the number of layers in a model, the number of attention heads, and so on. But that was only one dimension, because as model providers created large models, the associated training cost became an important consideration. So the second important dimension became training cost, because being able to train a model effectively, given a budget, was important. And the learnings from the Chinchilla paper earlier in the course suggested that large language models were being…

Contents