my "two office computer towers" comparison was mainly about physical footprint size.
I don't know the specifics of what hardware giant commercial LLMs like ChatGPT run on, but I would assume that the type of hardware is similar because the math being done is basically the same for any deep learning system (it's weighted sums all the way down).
In the video, John McDonald talks about how these servers are *only* used for the inferencing step. The training is (or at least was at the time of this video) done locally on his work computer.
Ben Lubar (any pronouns)
npub1yq…jmwzc
2026-04-30 12:24:08 UTC
in reply to nevent1q…jd0r
Author Public Key
npub1yq4ljwhqj9unxqpmdfgdq3gpgjj0f4edyd3s0zckpv9quxffhrgs2jmwzcSeen on
wss://relay.momostr.pinkPublished at
2026-04-30 12:24:08 UTCEvent JSON
{
"id": "4853aca57816bd62d97ce2b744241bf7585888705bb0d4c74d277f350982c329",
"pubkey": "202bf93ae0917933003b6a50d0450144a4f4d72d2363078b160b0a0e1929b8d1",
"created_at": 1777551848,
"kind": 1,
"tags": [
[
"p",
"c1812ca9433e5d31c5e35e9546d0f819d016ca6eabe98053daa6d40831179ec2"
],
[
"proxy",
"https://mastodon.lubar.me/@ben/116493637959745431",
"web"
],
[
"e",
"fc38077e902e0a94cd3e5290e9a710f7f2d8fe0ae08f5b91faa1b1582b0774cc",
"",
"root",
"202bf93ae0917933003b6a50d0450144a4f4d72d2363078b160b0a0e1929b8d1"
],
[
"p",
"202bf93ae0917933003b6a50d0450144a4f4d72d2363078b160b0a0e1929b8d1"
],
[
"e",
"30b0319cbecae9ffa4406aa2665bf89d6907eb512aeaca21c23f13ad4903e1b6",
"",
"reply",
"c1812ca9433e5d31c5e35e9546d0f819d016ca6eabe98053daa6d40831179ec2"
],
[
"proxy",
"https://mastodon.lubar.me/users/ben/statuses/116493637959745431",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://mastodon.lubar.me/users/ben/statuses/116493637959745431",
"pink.momostr"
],
[
"-"
]
],
"content": "my \"two office computer towers\" comparison was mainly about physical footprint size.\n\nI don't know the specifics of what hardware giant commercial LLMs like ChatGPT run on, but I would assume that the type of hardware is similar because the math being done is basically the same for any deep learning system (it's weighted sums all the way down).\n\nIn the video, John McDonald talks about how these servers are *only* used for the inferencing step. The training is (or at least was at the time of this video) done locally on his work computer.",
"sig": "6fb7792ff4c205af7c0d72169c9c3efa06b575bad95f47dcf104744ff727c3e98f748436818958f6e883f50b899bee8cf6124721d3eeb19248fec75697bf79c5"
}