Reading option logits from a small local model Reading option logits from a small local model

Jev in 25 lines: building the decision model locally

TL;DR In the design review I concluded that Jev, TypeSafe AI’s typed decision model, is cheaper and faster than an LLM and not more accurate. That raised an obvious build-vs-buy question: if the value is “cheap probability over a fixed set of options,” can I get that from a small open model running on hardware I already own? Mostly yes, with one catch that matters. The technique is to prompt with lettered options, run one forward pass, and read the logits for just the option letters. A parody post called “Jev in 25 lines of Python” does exactly this, and the technique under the joke is real. The catches are calibration (raw softmax is bad until you fit it) and serving (Ollama could not do it when I wrote the design). ...

October 16, 2026 · 8 min · zolty
Security model evaluation results Security model evaluation results

Grading Seven AI Models on Blue-Team Security Work in a VM With No Network Card

TL;DR I wanted to know which models I can trust for the defensive side of running a homelab: reviewing code for vulnerabilities, reviewing Kubernetes and Terraform config, triaging logs, and reading threat intel without making things up. So I built a harness with 112 frozen synthetic cases, graded everything deterministically (no LLM judge), and ran every model-written fix inside a Proxmox VM whose network card was deleted at the hypervisor. ...

October 12, 2026 · 16 min · zolty
GLM-5.1 benchmark on Mac Studio GLM-5.1 benchmark on Mac Studio

Running GLM-5.1 (744B) Locally on a Mac Studio: Benchmark Results

TL;DR I loaded Z.ai’s GLM-5.1 — a 744B parameter MoE model with 40B active parameters — onto a Mac Studio M3 Ultra with 256GB unified memory using a 2-bit quantized GGUF via llama.cpp. It runs at 5.8 tok/s with a 120-second time to first token. The financial analysis quality is genuinely impressive, but it eats 222GB of the 256GB available, leaving room for literally nothing else. It’s a “clear the schedule” model, not an always-on one. ...

April 13, 2026 · 8 min · zolty

Affiliate Disclosure: Some links on this site are affiliate links (Amazon Associates, DigitalOcean referral). As an Amazon Associate, I earn from qualifying purchases. This does not affect the price you pay or my editorial independence — I only recommend products and services I personally use and trust.