Daniel, GREAT job. This is fascinating.
Your leaderboard is based upon ELO, correct?
I can't find anywhere that you describe the formula for calculating the ELO score. It seems like we would need to verify that is indeed an accurate way to judge the Model Leaderboard. Care to share?
I'm a college student, and I enjoy playing word games like Codenames, but it can be hard sometimes to find enough people to play when everyone is available, so I made an app that allows you to fill any seat with an AI teammate and also play against AI. This way it allows for people to get the full strategic experience of codenames just on their own.
I was also able to measure and benchmark a few different LLMs through API calls and give them ELO ratings based on stuff like win rate and clue/guess accuracy. If anyone is interested, they can see more on the methodology of how I benchmark these models over here:wordspies.com/arena. I think it's pretty interesting because it gives you a lot of insight into the NLP capabilities of these various LLMs.
A few interesting stats were that I rated Gemini 4 Argon at 1775, GPT-6.1 Sol at 1744, GPT-6 Astra at 1723, Claude Fable 5.1 at 1623, and Claude Opus 5.5 at 1592. There are quite a few interesting things you'll find from there, like how, although newer and more expensive models tend to perform better, there are some rare exceptions, like Kimi K2.5 slightly edging out Kimi K3.
Check the site out if you're interested in NLP and data science and would like to see what level of frontier LLMs you can successfully win against!
I'm a college student, and I enjoy playing word games like Codenames, but it can be hard sometimes to find enough people to play when everyone is available, so I made an app that allows you to fill any seat with an AI teammate and also play against AI. This way it allows for people to get the full strategic experience of codenames just on their own.
I was also able to measure and benchmark a few different LLMs through API calls and give them ELO ratings based on stuff like win rate and clue/guess accuracy. If anyone is interested, they can see more on the methodology of how I benchmark these models over here:wordspies.com/arena. I think it's pretty interesting because it gives you a lot of insight into the NLP capabilities of these various LLMs.
A few interesting stats were that I rated Gemini 4 Argon at 1775, GPT-6.1 Sol at 1744, GPT-6 Astra at 1723, Claude Fable 5.1 at 1623, and Claude Opus 5.5 at 1592. There are quite a few interesting things you'll find from there, like how, although newer and more expensive models tend to perform better, there are some rare exceptions, like Kimi K2.5 slightly edging out Kimi K3.
Check the site out if you're interested in NLP and data science and would like to see what level of frontier LLMs you can successfully win against!
Daniel