Reading option logits from a small local model Reading option logits from a small local model

Jev in 25 lines: building the decision model locally

TL;DR In the design review I concluded that Jev, TypeSafe AI’s typed decision model, is cheaper and faster than an LLM and not more accurate. That raised an obvious build-vs-buy question: if the value is “cheap probability over a fixed set of options,” can I get that from a small open model running on hardware I already own? Mostly yes, with one catch that matters. The technique is to prompt with lettered options, run one forward pass, and read the logits for just the option letters. A parody post called “Jev in 25 lines of Python” does exactly this, and the technique under the joke is real. The catches are calibration (raw softmax is bad until you fit it) and serving (Ollama could not do it when I wrote the design). ...

October 16, 2026 · 8 min · zolty
A gallery of surplus lot photos feeding a structured identification schema A gallery of surplus lot photos feeding a structured identification schema

Reading a surplus lot from its photos: the identification pipeline

TL;DR Resale is the hobby that funds the hobby, and the expensive mistake in it is not a bad price. It is a confident wrong model number, because everything downstream (comps, margin, bid ceiling) is built on it. Auction search results give you one thumbnail and a title the seller typed in a hurry. The real information is in the gallery: wide shots for counting, close-ups for labels. The pipeline is split on purpose: vision reads the lot, a comps step prices what was read, and plain deterministic arithmetic turns that into a number. The model never does the math. Fetching the full gallery moved make/model accuracy from 0.375 to 0.875 on a 24-lot hand-labelled bench. That is the single biggest lever I have found, bigger than any model swap. Most of what went wrong later was not misreading a badge. It was counting and itemizing: accessories sold as products, “X with Y” counted twice, a screen size parsed as a quantity. A final sanitizer can only tighten a verdict. And if the gallery fetch fails, the lot is deferred, not guessed. I wrote about how lots get found in the first place. This post is about the step after: given a lot that looks interesting, how do I know what is actually in the pile? ...

October 13, 2026 · 11 min · zolty
Security model evaluation results Security model evaluation results

Grading Seven AI Models on Blue-Team Security Work in a VM With No Network Card

TL;DR I wanted to know which models I can trust for the defensive side of running a homelab: reviewing code for vulnerabilities, reviewing Kubernetes and Terraform config, triaging logs, and reading threat intel without making things up. So I built a harness with 112 frozen synthetic cases, graded everything deterministically (no LLM judge), and ran every model-written fix inside a Proxmox VM whose network card was deleted at the hypervisor. ...

October 12, 2026 · 16 min · zolty
A permission classifier sitting between an AI agent and a shell A permission classifier sitting between an AI agent and a shell

Claude Code's auto mode denied me 301 times in seven weeks

TL;DR Claude Code’s auto permission mode puts a classifier between the agent and any tool call that is not already covered by an allow rule. I ran in that mode all summer. In the transcripts I still have (mid-August to early October), 301 tool results mention that classifier. About 288 are outright denials, about ten are “no verdict” errors, and a few just quote the phrase. Of roughly 57,700 tool calls in September, that is about 0.5%. Almost all of it landed in September (285 of the 301), and it peaked on September 6 with 29 in one day. Two things helped: adding environment text that tells the classifier what my infrastructure is, and adding narrow allow rules so routine commands never reach it. Neither one was a clean win, and I will show where the data does and does not support that. Sometime between late September and early October I switched the default mode to bypass. The thing that makes that tolerable is a deterministic hook that runs before permission rules in every mode. The classifier was not wrong often. It was expensive in attention, and it was the wrong tool for unattended work. This is part of a small batch about the classifiers I run around agents. The rewrite of my destructive-command guard came first, and a field guide to all of them will follow. ...

October 10, 2026 · 11 min · zolty
A CI gate matching a test change against an exact fingerprint A CI gate matching a test change against an exact fingerprint

Never delete tests to get green: exact-fingerprint exceptions

TL;DR In July I wrote about a test-deletion sentinel: CI hard-fails if a merge request deletes tests, unless the commit message carries a [test-refactor] tag and an issue number. I was pleased with it. It turns out a tag in a commit message is a bypass that an agent (or I, on a bad day) can write in one line, and it excused every test change in the MR, not just the one I meant. ...

October 7, 2026 · 8 min · zolty
A shell command being parsed into argv before a guard decides whether to block it A shell command being parsed into argv before a guard decides whether to block it

Rewriting my agent's destructive-command guard to parse, not grep

TL;DR I run a PreToolUse hook that blocks a short list of unrecoverable shell commands before my coding agent can run them. Version 1 was a list of regular expressions over the raw command string. Replaying 90 historical blocks and about 82,000 historical commands showed that roughly 40% of v1’s blocks were not commands at all. They were grep patterns, commit messages, and markdown heredocs that happened to contain a scary phrase. v2 tokenizes the command the way a shell would and applies each rule only where a program actually runs. On the same replay it allowed every false text match, still caught every real destructive command v1 caught, caught a few that v1 missed because of flag order, threw zero parser errors, and took about 51 ms per call end to end. It fails toward blocking. If it cannot parse something it falls back to text rules, and if Python is missing the shell wrapper falls back to the old regex list. It is a tripwire for an agent that makes mistakes, not a security boundary against one that is trying to hide. I show a case it does not catch below. I will write up my experience with Claude Code’s auto-mode classifier separately. The short version is that this hook, not the classifier, is what stands between the agent and my cluster’s worst commands. ...

October 3, 2026 · 11 min · zolty
A ladder of model releases with per-model settings attached to each rung A ladder of model releases with per-model settings attached to each rung

Upgrading my Claude Code setup through five model releases

TL;DR Between late June and late September I moved my daily setup through Sonnet 5, Opus 5, Fable 5.1, Opus 5.5 and Sonnet 5.5. Haiku stayed at 4.5 the whole time. I never saw a Haiku 5 in any log. There is no controlled benchmark in anything I kept. What follows is operational evidence: what I changed, what broke, what I verified, and what I would have gotten wrong if I had trusted what a model told me about itself. The changes that mattered were small: a per-model effort block in settings.json and a subagent default model. The expensive lessons were two traps. One is a bug where background-job subagents silently ran Haiku 4.5. The other is that a worker’s report of its own model cannot be believed. If you want the earlier, more general version of this, see re-tuning my setup for a new Opus model. This is what the next five releases taught me. The ladder These are first-seen dates from my own records: commit trailers in my repositories and, where I still have them, session transcripts. They are when I started using each model, not necessarily when it became available. ...

October 1, 2026 · 9 min · zolty
A trip page maintained by agents and a scheduled conditions bot A trip page maintained by agents and a scheduled conditions bot

The Trip Page Nobody Edits: An Agent-Kept Git Site

TL;DR I built a private trip page for a week at a backcountry cabin in a large Ontario park, and nobody ever edited its HTML. Four JSON page specs go through a generator; every change is a typed operation that becomes a self-merging merge request. A GitLab CI schedule runs every 3 hours, pulls public Canadian weather, park and fire data, and commits a “conditions: refresh”. It made about 120 of those commits. The conditions script is stdlib-only, deterministic and offline-safe. If a source is down it keeps the last good copy, marks it stale, and never invents a number. The cabin has no cell service. Power and internet came from a Starlink Mini, a portable power station and a 120 W panel, and the whole workflow assumed a slow, metered link. Two real bugs: a bot that pushed as the wrong identity and got a 403, and a “fire” layer that turned out to be a static 2011 zone map. Why a page at all A week in the backcountry with a friend means a pile of shared decisions: gear, a meal plan with calorie math, a route with a water taxi and a portage, a charging plan for the drive, and some sense of whether we would be crossing a big lake in a 35 km/h wind. The usual answer is a group chat and a spreadsheet. ...

September 28, 2026 · 10 min · zolty
A design review of a small typed decision model A design review of a small typed decision model

Reviewing Jev: a decision model that is cheaper, not smarter

TL;DR Jev is TypeSafe AI’s first “System One” model, public since 2026-09-15. You send it a state object and a few typed questions, and it returns probabilities in 70 to 500 ms. I wrote a design doc for using it in my own tooling, and then rewrote it twice in two days because the first draft was wrong in ways I only noticed by rereading the sources. Nothing is built. There is no client, no table, no spike, no measured number of mine in this post. This is a review of a third-party product before I adopt it. Every accuracy and cost figure below belongs to the vendor or to a third party, and I label it that way. ...

September 25, 2026 · 11 min · zolty
A blurry equipment data plate upscaled into crisp but wrong text A blurry equipment data plate upscaled into crisp but wrong text

My Enhance button made the model plate worse

TL;DR I added an “Enhance” button to my photo viewer, backed by RealESRGAN running in ComfyUI on the Mac Studio. Its stated job was to help me read model plates and serial numbers in blurry surplus photos. I benched it against a plate where I knew the right answer. At small glyph sizes it did the opposite of its job: crisp, confident, wrong text. G1312B-60050 came back as 01D12B-(0)00. It is not a bad model pick. Four different upscalers garbled the same lines the same way. The usual advice (pre-upscale with a classical filter first) changed nothing. The failure is the generative prior, not a starved input. What I do now: the original pixels stay the source of truth, there is a text-safe sharpen view that invents nothing, and the AI view composites real pixels back over anything that looks like fine detail. A wrong model number off an enhanced photo is a wrong comp, which is a wrong bid. This one was worth catching before it spent money. Why I wanted it The most valuable pixels in a surplus lot photo are the smallest ones. The difference between a generic median price and an exact-model price is often a part number on a data plate, shot from a few feet away at a resolution where a 13-point label is about twelve pixels tall. I already pull the full gallery for every lot and archive it, so I had the best pixels the seller provided. They just were not enough for some plates. ...

September 22, 2026 · 7 min · zolty

Affiliate Disclosure: Some links on this site are affiliate links (Amazon Associates, DigitalOcean referral). As an Amazon Associate, I earn from qualifying purchases. This does not affect the price you pay or my editorial independence — I only recommend products and services I personally use and trust.