One of the first prompts I gave ChatGPT back in 2022 came from my main hobby (amateur astronomy). I asked it to tell me something about the extrasolar planets orbiting the star [[fake star ID]].
[[fake star ID]] was something that anyone who knew how to use Wikipedia half-intelligently could verify was fake within a few minutes.
I wasn't even trying to be deceptive; I genuinely wanted to see how ChatGPT would handle a request for information that I knew couldn't be in its training data.
The torrents of bullshit it produced -- paragraph after paragraph of totally confabulated data about these nonexistent planets orbiting a nonexistent star -- told me everything I needed to know about ChatGPT and its buddies, and I've never been tempted to use them for anything serious since.
David Nash
npub1vg…jyu5s
2025-08-07 16:13:18 UTC
in reply to nevent1q…8q7h
Author Public Key
npub1vgrrtppfx6quyrz5jw9k6pannzhz87f8gvsvh76fwt2h3vn44gaqqjyu5sSeen on
wss://relay.momostr.pinkPublished at
2025-08-07 16:13:18 UTCEvent JSON
{
"id": "e81150bd7e5534407504b5da21703d80bc9b2e8394b19ace538d32ea23274497",
"pubkey": "62063584293681c20c54938b6d07b398ae23f9274320cbfb4972d578b275aa3a",
"created_at": 1754583198,
"kind": 1,
"tags": [
[
"e",
"abfbc46524bf7c46b03326ba6980c9fd75ddf06134f6f2bdd6767cd34947e738",
"",
"root",
"ac05d25d11afd46780cbd4352a59ea265cfa20dc52d2c65433034cc3af33a79b"
],
[
"p",
"ac05d25d11afd46780cbd4352a59ea265cfa20dc52d2c65433034cc3af33a79b"
],
[
"proxy",
"https://c.im/@dpnash/114988364481195995",
"web"
],
[
"proxy",
"https://c.im/users/dpnash/statuses/114988364481195995",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://c.im/users/dpnash/statuses/114988364481195995",
"pink.momostr"
],
[
"-"
]
],
"content": "One of the first prompts I gave ChatGPT back in 2022 came from my main hobby (amateur astronomy). I asked it to tell me something about the extrasolar planets orbiting the star [[fake star ID]].\n\n[[fake star ID]] was something that anyone who knew how to use Wikipedia half-intelligently could verify was fake within a few minutes.\n\nI wasn't even trying to be deceptive; I genuinely wanted to see how ChatGPT would handle a request for information that I knew couldn't be in its training data.\n\nThe torrents of bullshit it produced -- paragraph after paragraph of totally confabulated data about these nonexistent planets orbiting a nonexistent star -- told me everything I needed to know about ChatGPT and its buddies, and I've never been tempted to use them for anything serious since.",
"sig": "d7f9ed06b31454c44314eb6cb5803c076716c22a7430fa5c59c6e8b15c6e5c7c9a63ee8f6e56d0e47fd5a67f4d9e86e62cc4e6008b5be406bc3818a3a77753fc"
}