- Vision
- WebDev
- Search
- Coding
- Math
- Creative Writing
- Instruction following
- Long queries
- Multi-turn
This feels like a GPT-4 moment. Google is moving so fast at this point that unless OpenAI is cooking up an absolutely insane upgrade with the o4 model, it's going to be outdated before they can even finish their training run. Google already has a reasoning model that is all around smarter, cheaper, and more multi-modal.
My money is on Google for winning the AI wars. They're winning on every single front. They have the best frontier model, the best cheap models, products that are already insanely popular they can integrate their models into, their own in-house chips and infrastructure, and boatloads of their own (not investors) cash because they're already a hugely profitable company.
[0]: Yeah I know there been some controversies but it's still a solid benchmark
quotingGoogle just can't stop winning. New test run for the Aider Polyglot benchmark is out and Gemini 2.5 Pro with extra thinking turned on is actually so stupidly good now that it trounces o3 by itself AND o3 and GPT 4.1 working together, all while being much less expensive.
nevent1q…e4lg
Not only that, but just yesterday Google brought the 2.5 update for its Gemini models out of preview and dropped the price at the same time, so the models are actually less expensive than what's listed on the benchmark leaderboard. Google's Gemini 2.5 Pro is now better than the top of the line models from Anthropic and OpenAI, Opus and o3, while costing less than Sonnet.
Google is winning and its not even close right now.
https://aider.chat/docs/leaderboards/
nevent1q…kwdp