If you have an Apple Silicon Mac with >24GB of RAM and >5GB of available disk space and uv installed, try running this command to see the new model in action (replace IMG_4414.JPG at the end with a path to your own image)
uv run \
--with mlx-vlm \
--with torch \
python -m mlx_vlm.generate \
--model mlx-community/SmolVLM-Instruct-bf16 \
--max-tokens 500 \
--temp 0.5 \
--prompt "Describe this image in detail" \
--image IMG_4414.JPG
Simon Willison
npub13v…4w5eu
2024-11-28 20:33:15 UTC
in reply to nevent1q…ujyk
Author Public Key
npub13v97j0kknscwnf5pt87nsn7cxzxwfwl3dsu7ss8qsq7ukmqgwg8q84w5euSeen on
wss://relay.mostr.pubPublished at
2024-11-28 20:33:15 UTCEvent JSON
{
"id": "85dfc0d1497c254e31c0b59842eb249476c436b8f0ee4216dba52ba643c266b4",
"pubkey": "8b0be93ed69c30e9a68159fd384fd8308ce4bbf16c39e840e0803dcb6c08720e",
"created_at": 1732825995,
"kind": 1,
"tags": [
[
"e",
"53288f34cf39b69d32e4d9bd3f26d95529f418a01fe40e4aedf4d4768cf5e08f",
"wss://relay.mostr.pub",
"reply"
],
[
"proxy",
"https://fedi.simonwillison.net/users/simon/statuses/113562484438041728",
"activitypub"
]
],
"content": "If you have an Apple Silicon Mac with \u003e24GB of RAM and \u003e5GB of available disk space and uv installed, try running this command to see the new model in action (replace IMG_4414.JPG at the end with a path to your own image)\n\nuv run \\\n --with mlx-vlm \\\n --with torch \\\n python -m mlx_vlm.generate \\\n --model mlx-community/SmolVLM-Instruct-bf16 \\\n --max-tokens 500 \\\n --temp 0.5 \\\n --prompt \"Describe this image in detail\" \\\n --image IMG_4414.JPG",
"sig": "c43227baff8a0c4ba35161e967767ae7e384575f855b3f0514a190b6e6d69aa128ff5b74fffe37ef2b616f66e4573161bcf9ccdd9dd3fff32cbc8b8be265a77e"
}