Hey Daniel try podman instead of docker. Out entire engineering team switched. There is a handy alias u can use and podman will run like docker. Just faster and rootless...
Really appreciate the progression from Dockerfiles to K8s manifests to Helm charts here. The way you framed containers vs VMs (kernel reuse vs full OS) clarified somehting I'd been fuzzy on. Particlarly interesting how Helm's dependency system turns infrastructure into composable building blocks. Kinda reminds me of how package managers work in traditional dev, except you're composing entire distributed systems instead of libraries. Makes me wonder if the cognitive overhead pays off untill you're actually hitting multi-node scale tho.
I appreciate the feedback, I'm glad you enjoyed it!
While I was programming it, the Chess app started to feel like it was converging on being worth the abstraction. For the pi estimation one should probably use spark, or just multiprocess within a single machine.
I struggle to think of, off the top of my head, a "job" like thing that would be advantageous to do in k8s rather than spark, though I'm sure there is one. I suppose If I had a k8s cluster lying around and I needed to do work on the data that already existed in it.
Fantastic breakdown of the container orchestration stack. The map-reduce pi estimation example really drives home why you'd want Job controllers over Deployments for finite tasks. One thing worth noting is how Redis Queue here sidesteps alot of complexity you'd normally hit with something like Celery, trading off some durability for simplicity wich is perfect for this usecase. Would love to see a followup on handling stateful workloads.
Hey Daniel try podman instead of docker. Out entire engineering team switched. There is a handy alias u can use and podman will run like docker. Just faster and rootless...
I'll check it out!
Really appreciate the progression from Dockerfiles to K8s manifests to Helm charts here. The way you framed containers vs VMs (kernel reuse vs full OS) clarified somehting I'd been fuzzy on. Particlarly interesting how Helm's dependency system turns infrastructure into composable building blocks. Kinda reminds me of how package managers work in traditional dev, except you're composing entire distributed systems instead of libraries. Makes me wonder if the cognitive overhead pays off untill you're actually hitting multi-node scale tho.
I appreciate the feedback, I'm glad you enjoyed it!
While I was programming it, the Chess app started to feel like it was converging on being worth the abstraction. For the pi estimation one should probably use spark, or just multiprocess within a single machine.
https://iaee.substack.com/p/apache-spark-intuitively-and-exhaustively
I struggle to think of, off the top of my head, a "job" like thing that would be advantageous to do in k8s rather than spark, though I'm sure there is one. I suppose If I had a k8s cluster lying around and I needed to do work on the data that already existed in it.
Fantastic breakdown of the container orchestration stack. The map-reduce pi estimation example really drives home why you'd want Job controllers over Deployments for finite tasks. One thing worth noting is how Redis Queue here sidesteps alot of complexity you'd normally hit with something like Celery, trading off some durability for simplicity wich is perfect for this usecase. Would love to see a followup on handling stateful workloads.
I appreciate the feedback. Yeah, I think covering Celery and stateful application design would be fantastic follow up topics!