Llama-Cpp

TL;DR I wanted to know which models I can trust for the defensive side of running a homelab: reviewing code for vulnerabilities, reviewing Kubernetes and Terraform config, triaging logs, and reading threat intel without making things up. So I built a harness with 112 frozen synthetic cases, graded everything deterministically (no LLM judge), and ran every model-written fix inside a Proxmox VM whose network card was deleted at the hypervisor. ...

TL;DR I loaded Z.ai’s GLM-5.1 — a 744B parameter MoE model with 40B active parameters — onto a Mac Studio M3 Ultra with 256GB unified memory using a 2-bit quantized GGUF via llama.cpp. It runs at 5.8 tok/s with a 120-second time to first token. The financial analysis quality is genuinely impressive, but it eats 222GB of the 256GB available, leaving room for literally nothing else. It’s a “clear the schedule” model, not an always-on one. ...