After a long wait, Meta has finally released Llama 3.1 405B, its largest AI model to date, trained on 405 billion parameters. Not only that, Meta has also released an upgraded model family that includes the Llama 3.1 70B and Llama 3.1 8B models. And all of these models are open source, and the Zuckerberg-led company says that "Meta is committed to open access AI."

All three Llama 3.1 models come with a context length of 128K tokens and support eight different languages. So Meta has incorporated a large context window and multilingual capability. In terms of benchmarks, the largest Llama 3.1 405B model outperforms OpenAI's leading AI models such as GPT-4 and the latest, GPT-4o (Omni).

In the MMLU benchmark, the Llama 3.1 405B scores 88.6 points while the GPT-4o scores 88.7 points, meaning both are almost on par. Apart from this, in almost all other tests including MBPP, GSM8K, ARC Challenge, etc., the 405B model out-competes the GPT-4o. The only major benchmark where the Llama 3.1 405B trails is HumanEval, but by a small margin.

In HumanEval, the GPT-4o scores 90.2 while the 405B model scores 89.0. Finally, regarding multimodality, Meta has unfortunately not released a truly multimodal model yet, even after the Llama 3.1 release. Meta says that Llama 3.1 models support image, video and speech recognition, but they are still under active development and not yet ready for release.