Elon Musk's AI venture, xAI has released an early preview of the Grok 2 model, and it has surprisingly outperformed Claude, Gemini and even ChatGPT as well. The previous Grok-1.5 model was not well received, but the Grok-2 has delivered good performance on the LMSYS leaderboard. xAI has released two new models: the Grok-2 and a smaller Grok-2 mini model.
UNCENSORED GROK 2.0 BROKE just the Internet!
xAI says Grok-2 has improved significantly in key areas, including reasoning, following instructions and providing accurate and factual information. In traditional AI benchmarks, Grok-2 scored a whopping 87.5% in MMLU and 88.4% in HumanEval. This is particularly interesting because the MMLU score has been derived using the 0-shot CoT.
Grok-2 was tested on LMSYS under the name "sus-column-r". With about 12,000 votes, it ranks third, just below ChatGPT-4o-latest, Gemini-1.5-Pro-Experimental, and GPT-40-2024-05-13. However, it outperforms the GPT-4o-mini, Claude 3.5 Sonnet, Gemini 1.5 Pro and Llama 3.1 405B. Woah, another exciting update from the Chatbot Arena❤️🔥Results for @xAI's sus-column-r (Grok 2 early build) are now public**!With over 12,000 community votes, sus-column-r has secured the spot number 3 on the overall leaderboard, even matching GPT-4o! It excels in coding (#2),… https://t.co/gqSWSwYN0z pic.twitter.com/j9UYDBYNt4 — lmsys.org (@lmsysorg) August 14, 2024
Woah, another exciting update from the Chatbot Arena❤️🔥Results for @xAI's sus-column-r (Grok 2 early build) are now public**!With over 12,000 community votes, sus-column-r has secured the spot number 3 on the overall leaderboard, even matching GPT-4o! It excels in coding (#2),… https://t.co/gqSWSwYN0z pic.twitter.com/j9UYDBYNt4 — lmsys.org (@lmsysorg) August 14, 2024
