jonny (nonvenomous) on Nostr: Something that is extremely awesome about LLM code is its propensity to make endless ...

Something that is extremely awesome about LLM code is its propensity to make endless fallback chains to rescue its fragmented and unreliable logic. The LLM in each pass wants the current thing to succeed, and will rig up hacks that give it trivial test passes. This has the wonderful byproduct of making things fail in the wrong direction in surprising cases - failure is not a smooth outcome, you can't just fail something once and conclude the route is blocked off. you have to fuzz and fail everything in all the different ways, because for some reason one of those failures will engage an entirely different mechanism that rolls over weirdly. This is aided by being given the client source with revealing LLM comments, and having access to the server binary on a system rich with tokens and time for brute force disassembly