Hardware

TL;DR I bought nine Dell OptiPlex all-in-ones (a mix of 7760 and 7780) in one government surplus lot for $1,209.37 all-in, about $134 a unit. Tax and shipping were zero; I picked them up myself. They run Debian 13 with a LUKS-encrypted root that unlocks through the TPM, Secure Boot on, and a signed NVIDIA module. Getting all three to coexist was most of the work. Eight of them joined my second, small k3s cluster (the “office cluster” at the shop where I do resale and bench work) as workers. Together they add 96 cores and 170 GiB of allocatable memory to a cluster whose three agent VMs had 12 cores and 23 GiB between them. The join is a dedicated Ansible playbook that reuses the existing k3s_agent role. One canary first, then the other seven in two waves. I didn’t measure power draw on these, so there’s no watts figure in this post. The older OptiPlex desktop cluster number doesn’t transfer. Why all-in-ones Most of what I buy for the hobby that funds the hobby is resale stock. This lot was different: nine machines, all wiped, “last powered on six months ago”, sold as-is, pickup by appointment at a school. The seller doesn’t box or ship. The hammer price was $1,075.00, plus a $134.37 buyer premium. I allocated the cost per unit in the inventory database and marked all nine as kept rather than for sale. ...

TL;DR The dangerous remote failure isn’t the one that errors. It’s the one that reports success — exit code 0, green tick, desktop loads, nothing in the logs. Five real ones from the last two months, each with the wrong hypothesis I actually held. Your diagnostic tool can be the thing lying to you. A packet capture showed traffic arriving on a port that was being dropped, because capture taps before the firewall does. Provisioning needs sentinel files, not vibes. “It looks finished” is not a completion signal; a file the process writes at the end is. Where agents genuinely help: assembling the four facts a diagnosis needs, and recalling the failure signature from three weeks ago. Where they hurt: confidently narrating a plausible cause for a symptom they can’t see. Every one of these produced a page in the knowledge base with the wrong hypothesis recorded, because that’s the field that saves the next hour. Success is not a signal I run machines I can’t put my hands on: a bench of surplus desktops at an office across town, cluster nodes, a couple of workstations belonging to other people. Remote hands come from a KVM device and SSH, and I’ve written about that hardware setup already. ...

TL;DR An unmodelled repair backlog becomes a pile of dusty things you feel bad about. Ask me how I know. Four ticket states: open, blocked, done, abandoned. blocked and abandoned are the two that do the work. abandoned is a feature. A queue you can’t quit is a queue that grows forever, and “I’ll get to it” is not a state. Parts are modelled as purchases and uses separately, because you buy a bag of ten capacitors and use two. Otherwise your cost-per-repair is nonsense. Labour gets logged per worker with an hourly rate, and that number flows all the way back into what I’m willing to bid on a lot. The no-go rule does more for profitability than any repair skill I have: if parts and time exceed roughly 60% of realistic resale, or resale is under about $120, it doesn’t get a ticket. Other people work this queue. Four access roles keep them on inventory and tickets and away from anything financial. Why a pile forms Buying surplus means buying broken things, on purpose, because broken is cheap and some broken things are trivially fixable. A cassette deck with a perished belt is a $4 part and twenty minutes. A workstation that won’t POST is often one reseated stick of RAM. ...

TL;DR The work between “I own this thing” and “money arrived” is mostly paperwork and printing, and it’s where the hours actually go. Listings get built through the marketplace API, not by driving a browser — structured item specifics as JSON beat fighting a typeahead widget every time. Drafts are created automatically. Publishing is always manual. Publish is a money-moving, buyer-facing, irreversible action; it gets a human every time. Barcode labels render from the inventory record itself: ZPL → PDF → print job to a thermal printer, triggered from the item’s page. No separate label app, no retyping serials. Pack slips print on a schedule, and the job verifies the printer’s page counter moved before marking the slip printed. Optimistic “it printed” flags are how orders ship without paperwork. Anything bulk or financial — repricing, printing five-plus labels — sits behind an explicit confirmation. Cheap friction in exactly the right places. The boring half is the expensive half Sourcing gets all the attention. I’ve written about the funnel that finds lots and the data model that tracks the units. Both were interesting problems. ...

TL;DR I keep four small-form-factor PCs on a bench for testing and repurposing — bought used, need fresh OS images, fresh BIOS settings, and no monitor or keyboard. A PiKVM V4 Plus with a multiport switch gives me eyes and hands on all four boxes over the network. Dell’s cctk command-line tool (Command | Configure) lets me bake BIOS settings — boot order, AHCI mode, Wake-on-LAN, power-on-after-failure — into scripted runs instead of clicking through F2 menus. No monitor, no keyboard, no physical access for weeks at a time. Everything repeatable, everything as code. ...

TL;DR My production homelab runs on Lenovo M920q tinies, and I still think those are the sweet spot. But if I were starting over today with a tight budget, I’d buy a stack of government-surplus Dell OptiPlex 7060 and 7070 desktops instead. They go for around $150 each refurbished — 6-core 8th/9th-gen Intel, an SSD, and Windows 11 already on them — and they make excellent Kubernetes nodes with exactly two cheap upgrades: a bit more RAM and a second network card. ...

TL;DR Purpose-built smart-home wall panels are expensive, locked down, and usually underpowered. A used Microsoft Surface Pro 7 — Core i5 or i7, 16 GB RAM, a sharp 12.3" touchscreen — runs about $150 on the surplus market and makes a fantastic wall-mounted dashboard for Home Assistant, Grafana, or whatever you self-host. It’s a full x86 PC behind a great touchscreen, so it runs a real browser with your real dashboards, not a stripped-down panel app. Here’s the build. ...

TL;DR A homelab feels free until you read the meter. After a year of running seven k3s nodes plus a pair of Mac Studios under whatever workload I felt like throwing at them, I sat down with a Kill-a-Watt and worked out what the cluster actually costs to keep on. Idle is genuinely cheap. Sustained LLM inference is not. The honest break-even against cloud inference is workload-shaped, and for my workloads, on-prem wins — but only because I run them often enough to amortize the wattage. The numbers below are mine; substitute your electricity rate to get yours. ...

TL;DR Consumer mesh Wi-Fi is expensive and locked down. Meanwhile, enterprises retire perfectly good 802.11ac access points by the pallet and dump them on the surplus market for a few dollars each. The Extreme Networks WS-AP3825i — a 3x3 MIMO, dual-band, PoE business AP — runs about $6 used, and it’ll happily run OpenWrt: no vendor controller, no license, no cloud account, just a clean Linux router you own. I bought a pallet of them. Here’s the why, the how, and the one mistake that turns one into a paperweight. ...

TL;DR Everyone reaches for a mini PC or a Pi for a homelab node. The thing nobody tells you: a used business laptop is a server with a built-in UPS, screen, and keyboard bolted on for free. A Dell Latitude 7400 — 8th-gen Core i5, 16 GB RAM, NVMe SSD — runs about $150 used, draws ~10 W with the lid shut, and when the power flickers it doesn’t even notice, because it’s running off its own battery. I run a couple as edge nodes. Here’s the case for it and the five-minute headless setup. ...