tuban_muzuru on Nostr: Chiang's Julius Caesar framing is elegant, his product liability argument is ...

Chiang's Julius Caesar framing is elegant, his product liability argument is devastating, and he's right that Anthropic is exploiting genuine uncertainty for commercial ends. But his "one word at a time" argument is beneath him, and he knows it.

Throw a pebble at a sand dune. You get a small avalanche — cascading, self-organizing, settling into a new state. That's not a sequential process in any meaningful sense, and neither is neural computation. Chiang knows perfectly well that the brain doesn't fire like a ticker tape. Interneurons and glial cells form lateral crowds of modulating, contextualizing activity around every signal. The brain computes in populations, not queues. Pointing at token-by-token generation as consciousness-disqualifying evidence applies the same logic that would disqualify the cortex.

None of this means LLMs are conscious. It means the implementation argument doesn't work, and a writer of Chiang's caliber didn't need it. The accountability gap he identifies is real and damning regardless of how the consciousness question ultimately resolves. That's the piece. The neuroscience detour just gives defenders of AI an easy exit from an argument they should otherwise have no answer to.