Wanted to downsize from the power hungry and massive heat generator that is my three node VMWare cluster, and I decided on building a K3s kubernetes cluster using OrangePi 5 Ultras. Each node has an 8 core RK3588 ARM64 CPU, 16GB of RAM, a 64GB eMMC module for the OS (Armbian), and a 1TB NVME SSD. The NAS is a FriendlyElec CM3588 Plus (16GB RAM, 64GB eMMC) with 4x 2TB NVME SSDs running Openmediavauilt. Everything is hooked up via 2.5GbE to a Unifi Flex 2.5G switch with a 10GbE uplink to the rest of the network. Everything is powered by a Henxlco 5V 200W power supply.
The rack is entirely 3D printed from various PLAs and PETGs that I had left over, which is why all the colors are different, also my 3D printer is a small A1 mini, so I had to split prints and glue/melt them together which is why a lot of things have seams in them.
As far as the software goes, the nodes are running K3s on Armbian bookworm managed by Rancher. The 1TB drives are hyperconverged using longhorn to provide storage for the workloads. Currently running are authentik, immich, jellyfin, docker-mailserver and roundcube, n.eko, nextcloud, postgresql, technitium, unifi, and vaultwarden. I was able to get jellyfin and immich to use the mali10 GPU and NPU hardware accelleration on the nodes. Still trying to figure out if I can get it to work with n.eko, but even with software encoding it works at 1080p60 no problem.
My goals with this were to retire my VMWare cluster, reduce my idle power consumption and heat production, and to learn Kubernetes. My VMWare cluster is off, my idle power consumption went from 310W to 35W (under full load I’ve never seen it above 60W), but I’m still working on that Kubernetes knowledge.
💬 Discussion r/homelab (231 points, 10 commentaires) 🔗 Source