I wanted to spend some time writing about my lab infrastructure, as I think it’s a key part of my learning, development, and professional growth. More importantly, I like tinkering with stuff, and this is a fairly cheap and useful way to do exactly that.
At this point, in all its incarnations, my homelab is almost a decade old. Like a phoenix, it has burned to the ground more than once, only to be reborn. This site itself is on its 4th iteration, check the wayback machine.
this site
This site is hosted on a single node k8s (k3s) running on a proxmox VM. Traffic is sent to the VM directly via portforward where traefik picks it up directly. The container is just a caddy server serving hugo generated assets. traefik does TLS as well via a HTTP-01 challenge, which is way more convenient than the DNS challenges I used to do.
This is probably the most simplistic k8s setup I could imagine, and it works really well for me. The rollouts are painless and the gitops flow works really well for me.
future plans
- building out multi-machine k8s
- rancher (or similar) for shared storage
- MetalLB for traffic
- comment system, still use hugo
general lab
VMs and LXCs
I use Proxmox for the main hypervisor. I’ve used other stuff, but I am deeply familiar with the main opensource components Proxmox is built on, so the learning curve is very low. I have used other stuff (xcp-ng, Nutanix CE, MicroCloud) but for my lab use, it’s great. I have had some serious hiccups, but nothing that cannot be solved. I like ZFS and use it extensively on my NAS and as the root storage for Proxmox.
Nodes are old Dell Optiplexes with a mix of hardware, one has an older Nvidia card for hardware decoding/encoding for Jellyfin and dedicated 10G SFP links. Storage is all flash, surplus SSDs from eBay. Drives have a little under 50-60k hours on them.
routing
I used OPNsense for routing, it runs on its own baremetal Optiplex with a dedicated NIC. I use an older Brocade ICX-7250 for a primary switch, it is the most/cheapest 10G switch you can buy as of early 2025 IMO. Communication via serial and flashing was done with this excellent firmware and explanation.
applications
jellyfin
I have jellyfin clients on all devices (phones, roku, etc) and it’s a very nice way to play series or movies we already have. I have some trouble with hardware decoding, in particular silent NVENC failures that show up with certain movies and only in the jellyfin systemd log. From my perspective, NBD, but from anyone using the service, it can be frustrating. The automatic metadata addition and organization is very good.
devbox
I have a generic debian box I use as a jumphost and general workstation. This makes it very easy to work from my laptop, remotely, or via another machine. I highly recommend it.
k8s
I am using K3s, although I have used a bunch of other distros in the past.
homeassistant
HomeAssistant is an amazing piece of software, truly a shining light of what open source can achieve. I have zigbee devices scattered around my home, primarily for temperature, humidity, CO2 and smart plugs. The service has been rock solid, low maintenance, with great simplistic defaults.
frigate
I have frigate deployed to monitor my doorbell and workshop cameras. It works alright, especially on the older hardware I am using. I found the learning curve to be a bit steep, and it’s still not fully configured.