Data

TL;DR Everything in my resale pipeline runs on CronJobs, which is correct until the moment you want fresh data now and the next run is 40 minutes out. Fix: a refresh button on each dashboard that creates a Kubernetes Job from the existing CronJob’s template, via the API, using a dedicated ServiceAccount with a tightly scoped Role. The web app gets permission to create Jobs in one namespace and nothing else. No cluster-admin, no shelling out to kubectl, no shared token. Six dashboards: opportunities, active bids, sold, listings, pickups, contacts. Every table sorts on every column, which is not a nice-to-have — unsorted tables hide your worst-performing categories. The metric that changed my buying wasn’t margin. It was days from sourced to sold, by category — and it says two categories I liked were quietly eating all my working capital. Cron is right until you’re standing in front of the shelf The pipeline is scheduled work. Crawlers every six hours, evaluation hourly, order sync every six hours, pack slips twice an hour. That’s the right architecture: it’s cheap, it’s resilient, it recovers on the next tick, and nothing depends on me being awake. ...

TL;DR You buy a lot. You sell units. That asymmetry is the whole data-modelling problem, and spreadsheets lose to it immediately. The spine is four tables: purchase_lot → inventory_item → listing → sale, with join tables where the relationships are genuinely many-to-many. Cost gets allocated from the lot down to units, and the allocation locks once the first unit sells. Retroactively re-allocating cost after a sale is how you accidentally rewrite last quarter’s profit. A unit walks a status machine: sourced → processing → drafted → listed → sold → shipped → reconciled, with dead and personal as the two honest escape hatches. Selling quantity-N on one listing is N sales, not one sale with a quantity column. I modelled it the wrong way first and spent August unpicking it. Packing several units into one sellable bundle, and splitting a bundle back into units, both need to conserve cost exactly. Getting that wrong is silent and it shows up as fake margin. The mismatch Here’s the transaction that starts everything: I pay $340 for a pallet described as Assorted Networking and Computer Equipment. One line item, one payment, one date. ...

TL;DR Government and institutional surplus auctions are a firehose: thousands of live lots, most of them badly photographed, vaguely titled, and worthless. Browsing them by hand doesn’t scale and — worse — it’s biased. You click the pretty photos, and the pretty photos are the ones everyone else clicked. The fix is a funnel: ~24 scheduled crawlers feeding a single Postgres table, photos and paperwork in object storage, an hourly evaluation pass, and one ranked board. The judgement layer (reading the goods out of photos, estimating what they’re worth) runs as a small model pipeline with a second pass that checks the first. I’m deliberately light on its internals here. The output isn’t a decision. It’s a shortlist with reasons attached — and a “this is wrong” button that puts the lot back in the queue. The single highest-value change in three months was raising how many photos get looked at. The label is often in photo nine. None of this is my job. It’s the hobby that pays for the hobby, which is the only reason it’s allowed to be this over-engineered. Why I built a data pipeline for a weekend habit Let’s be honest about the proportions here. I built two dozen scheduled crawlers, an object store, an hourly evaluation pass and a ranked dashboard, in order to buy second-hand equipment on the internet slightly better than I did before. ...

TL;DR Active eBay listings are asking prices, not realized prices—they lie. Use SOLD comps (~90-day history) to anchor your math. Card identity is everything: name + set + edition + variant + grade. “Charizard” is not a price; “Base Set Unlimited Charizard, PSA 9” is. Sell-through velocity (sales per week) predicts how long your capital sits locked up and is the variable most buyers ignore. Expected value = (median sold price × (1 − fee rate)) − shipping − grading. Apply a 30% margin of safety before you bid. Confidence gates protect you: skip lots where you can’t find ≥3 grade-matched comps or where the price spread exceeds ~6× the median. Buying Cards Rewards Discipline I’ve overpaid for Pokémon lots I “knew” were steals. The problem was never the cards—it was that I was bidding on nostalgia and gut feeling, not math. ...