🆕 blog! “Are LLMs still surprisingly bad at some simple tasks?”
Last year I ran an experiment to test the ability of modern LLMs to correctly answer a relatively straightforward question. Every single one of them got it wrong. Some missed information, some made up false statements, none were right.
Of course the fanbois variously claimed that I was holding it…
👀 Read more: https://shkspr.mobi/blog/2026/09/are-llms-still-surprisingly-bad-at-some-simple-tasks/
⸻
#AI #internet #LLM
Terence Eden
npub1lh…aa5ll
2026-09-22 11:35:03 UTC
Author Public Key
npub1lh05slh2nk6h5m3jp7qwtjmrmegtah6nj626g8qdg3a7ds3kg0ss9aa5llSeen on
wss://relay.ditto.pubPublished at
2026-09-22 11:35:03 UTCEvent JSON
{
"id": "12c4b65d024cdf783ad285e64f0a612651a237c69863bf2e7ebb98aaa285cb77",
"pubkey": "fddf487eea9db57a6e320f80e5cb63de50bedf539695a41c0d447be6c23643e1",
"created_at": 1790076903,
"kind": 1,
"tags": [
[
"t",
"ai"
],
[
"t",
"internet"
],
[
"t",
"llm"
],
[
"proxy",
"https://mastodon.social/users/Edent/statuses/117314479958292686",
"activitypub"
],
[
"client",
"Mostr",
"31990:6be38f8c63df7dbf84db7ec4a6e6fbbd8d19dca3b980efad18585c46f04b26f9:mostr",
"wss://relay.ditto.pub"
]
],
"content": "🆕 blog! “Are LLMs still surprisingly bad at some simple tasks?”\n\nLast year I ran an experiment to test the ability of modern LLMs to correctly answer a relatively straightforward question. Every single one of them got it wrong. Some missed information, some made up false statements, none were right.\n\nOf course the fanbois variously claimed that I was holding it…\n\n👀 Read more: https://shkspr.mobi/blog/2026/09/are-llms-still-surprisingly-bad-at-some-simple-tasks/\n⸻\n#AI #internet #LLM",
"sig": "4cafd6d0bc5c359341a663f57dc2f6f78e55ab8aa784a9b151da94b1914be83d95bbaefade493248e954576d7f7a802636e1a334a3a65cbaabea23a3cbc2f9ab"
}