Join Nostr
2026-06-11 08:44:29 UTC

s-ol on Nostr: hey folks, i could use some DevOps advice on a work-related topic! tl;dr: anyone have ...

hey folks, i could use some DevOps advice on a work-related topic!
tl;dr: anyone have some experiences with Talos or Kairos and/or k3s and k0s to share?

For context, we do a lot of prototyping and deployments on varied platforms (some cloud/web, interactive media on windows on-prem, SBCs/Raspis) without a clear general approach.

I have a project coming up that will involve deploying ~100 SBCs (amd64) that will run a kiosk mode browser and I would like to take the opportunity to implement a deployment architecture here that we can lean on in the future.

Requirements:
- immutable OS partition: soldered eMMC means minimizing writes is imperative
- small (worker) footprint: needs to run decent on 4GB SBCs
- integrates well with other/non-bare-metal deployments (cloud/linux servers, ideally even windows)
- bonus: compatible with a "federated" setup where there's local management nodes (per site and project), but ops data can also be forwarded and aggregated centrally

Anti-Requirements:
- cloud-based / requires Internet: needs to work with interrupted internet connectivity. Prefer sandboxing workers at the network level
- Siloed all-in-one solution with limited interoperability

The compatibility and siloing are my main gripes with Balena: it has features like monitoring and VPN built in which sure that's cool but we already have a (different) VPN, and if I can't integrate my non-balena nodes in the deployment process or your monitoring solution you're not helping.
That's why I've started looking at the Kubernetes-based solutions, it seems that would let me use a Talos or Kairos for SBCs (and maybe bare-metal servers and VMs, too) and mix-and-match that with other Kube infrastructure.

Between Talos and Kairos so far I find Talos the more technically convincing implementation, but Kairos has actually open governance and would make it easier to customize the base image where necessary (e.g. pinctrl driver for one of the SBCs we're using). I also have no idea how I'd go about choosing between k3s or k0s for Kairos and what choosing either would mean for interoperability with eventual other clusters. Help! Does anyone have experience with either they'd be willing to share?