Baby food companies in the US are apparently legally obligated to test their foods for heavy metals content and to publish the lab results on their websites. I have a colleague who is looking to collect this data, so I started looking at the Gerber website.
Not only is the data not available in bulk, the website is *actively* hostile towards scraping. Standard cookie checks, etc... are bad enough. However, here, you have to retrieve data for each UPC/Batch pair one page at a time... and the actual lab results (for **one** UPC/Batch pair, and so about 20 integer values stored in a json blob)... come along with nearly **half-a megabyte** of interface goop.
Ugh.
#WebFail #API
Oliver Kennedy
npub1gr…z83e7
2025-10-12 17:45:54 UTC
Author Public Key
npub1gr0qt2duklqtrq0jgm6m8xu3cqy8mxurmdss6jfkewjf6fdk8lnqfz83e7Seen on
wss://relay.ditto.pubPublished at
2025-10-12 17:45:54 UTCEvent JSON
{
"id": "bfb180fccae6ecb74ff4b0a28ec149edbeaab813ce843ac1d85a96f683f756d4",
"pubkey": "40de05a9bcb7c0b181f246f5b39b91c0087d9b83db610d4936cba49d25b63fe6",
"created_at": 1760291154,
"kind": 1,
"tags": [
[
"t",
"webfail"
],
[
"t",
"API"
],
[
"proxy",
"https://discuss.systems/users/okennedy/statuses/115362441127237717",
"activitypub"
],
[
"client",
"Mostr",
"31990:6be38f8c63df7dbf84db7ec4a6e6fbbd8d19dca3b980efad18585c46f04b26f9:mostr",
"wss://relay.mostr.pub"
]
],
"content": "Baby food companies in the US are apparently legally obligated to test their foods for heavy metals content and to publish the lab results on their websites. I have a colleague who is looking to collect this data, so I started looking at the Gerber website.\n\nNot only is the data not available in bulk, the website is *actively* hostile towards scraping. Standard cookie checks, etc... are bad enough. However, here, you have to retrieve data for each UPC/Batch pair one page at a time... and the actual lab results (for **one** UPC/Batch pair, and so about 20 integer values stored in a json blob)... come along with nearly **half-a megabyte** of interface goop.\n\nUgh. \n\n#WebFail #API",
"sig": "c10894c3c7954ec77afddf1a642a7108d34124d55ef2fe8dc73c5fa1a2ba4db3d4ecdf68d9d18badaa14b971a4aa8ad05b97d91cc9a7f7806201f6e8320b8bb1"
}