One of the new flagship AI models published on Saturday, Mavarick, LM Arena is in second placeA test that has human rates that compare outputs of models and choose which one they like. However, it seems that Mavarick’s version of the Meta Arena is different from the version that has been deployed in Arena, which is widely available for developers.
As Several AII Researcher As indicated in X, Meta mentions in his announcement that Mavarick is a “experimental chat version” in LM Arena. On a chart Official Lama websiteMeanwhile, Meta’s LM Akhara test was conducted using “Lama 4 Mavarick conversation” using the Lama 4 Mavarick conversation.
As we have written before, for various reasons, LM Arena may never be the most reliable measure of the AI model performance. However, AI companies usually did not give their models customized or otherwise subtle to score better at LM Arena-or at least not acknowledged to do so.
The problem with a “vanilla” variant of the same model is made in a benchmark, making a model a benchmark and then challenging how well the model will perform in a specific context. It is also confusing. Ideally, benchmarks – as they are as horriblely inadequate – provide a snapshot of single model strengths and weaknesses that covers various tasks.
Indeed, there are researchers of X Stark observation Discrimination Comparatively downloadable Mavarick compared with the model hosted in LM Arena. The LM Arena version uses a lot of emoji and seems to have incredibly long-borne answers.
Ok Lama 4 is a litled cooked lol, this is the city of Yap. pic.twitter.com/y3gvhbvz65
– Nathan Lambert (@Natolambert) April 6, 2025
For any reason, the Lama 4 model in Arena uses a lot more emoji
Together AI, it seems to be better: pic.twitter.com/f74odx4ztt
– Tech Dave Notes (@Techdevonots) April 6, 2025
We arrived at Meta and Chatboat Arena, the LM Arena maintenance company to comment.