Build or buy is not an engineering decision

The engine fact-checks every claim it makes. The least this post could do was return the favor.

If you've ever run engineering for more than one product, you know the question. It comes back every quarter, wearing a different costume: a vector database this time, a billing system the next, a fraud model after that. Do we build this ourselves, or pay someone else for it? This is a tool that does the homework behind that question, cites every source, argues with itself, and then hands the decision back to you. It does not decide. That part is still yours, and it should be.

The question that looks technical but isn't

It's a capital decision wearing an engineering costume. Engineers are handed it because it arrives dressed as an architecture question, and they answer it with the tools they have.

Get it wrong and you don't just lose a sprint. You inherit a vendor you can't leave, a "cheap" subscription with switching costs nobody priced, or a homegrown system that quietly eats two engineers a year for the rest of its life. The bill doesn't arrive on launch day. It arrives in year three, with interest.

Now run a portfolio of companies, and it gets worse. The same question gets answered independently, six times, by six teams who never compared notes. Six vendor evaluations for the same database. Six gut calls. A thing that would've been cheap to build once and share gets bought six times instead. Nobody's keeping the receipts.

A real one, before any of the machinery

A small, bootstrapped fintech needs reliable historical and live data for Indian capital markets. Money's tight. Data quality is non-negotiable, because they're running anomaly detection on it. There's a hard four-week deadline. And Indian market data sits behind real regulatory walls: the exchanges and the regulator strictly control who can redistribute it.

The engine's recommendation was buy-then-extend. License a compliant commercial API for the feed, because four weeks is not enough time to build licensed exchange integrations and scraping would be illegal. Then own a thin layer on top, storing the data in an open format, so costs stay yours and the vendor isn't forever.

Then its own challenger pass argued for plain buy: if the commercial tiers are cheap enough, the founder's time is worth more than a custom caching layer. Neither answer is "correct," and that is the point. The run produced the real tension: quality and the law pull toward a paid source, a bootstrapped budget pulls the other way, with both cases on the table and every claim clickable back to its page.

Why the usual answers don't hold up

The default playbook for this decision is thin, and everyone knows it:

• A gut feeling off a vendor demo. Demos are sales. Of course it looked good.
• A spreadsheet with weighted scores. This one's the sneakiest, because it looks rigorous. You pick fourteen factors, assign weights, multiply, and out pops a number: 4.1. But you invented every weight, and the number inherited all of your guessing while losing the part where you could see it. It's false precision. Confidence cosplay.
• A generic AI research bot. It'll research anything, which is exactly the problem. It doesn't know what a build-vs-buy decision is shaped like.

None of them compound, and none of them know the decision's anatomy. So I built one that does.

What it produces is narrower than "an answer": a reviewable comparison, built around the constraints you actually stated, with the assumptions and the evidence sitting where you can argue with them. The research discipline gets written down once instead of reassembled by whoever happens to own the decision this quarter. Whether the discipline then compounds across a portfolio is a separate question, and one this version does not answer. See the last section.

What it actually does

You describe a capability in plain English. It interviews you rather than handing you a form, pulling out what moves the decision: how core this is, the budget, the timeline, the compliance constraints. It knows nothing about your industry in advance, and that's deliberate. It asks instead of assuming.

Then two research tracks run in parallel. One digs into building it: real engineering effort, the open-source pieces you'd stand on, the maintenance bill that compounds. The other digs into buying it: who sells it, what it costs now and at renewal, how locked in you'd get. Two piles of evidence. Same photo, two different lights.

It reasons over four ways out rather than two: build, buy, buy-then-extend, or adopt-and-self-host, because most real decisions land on one of the middle two. Before it commits, a second pass builds the strongest case it can for a different path. It parks for your approval three times along the way, and a steer at the last gate re-runs synthesis without repeating the research, because every stage is keyed by a hash of its inputs and a cached stage is skipped rather than re-paid for.

Pipeline architecture and the grounded claim trust layer

The one idea worth stealing

That sounds like a small plumbing detail. It's the whole ballgame, but it is worth being exact about what the code enforces, because "enforces carefulness" would be overstating it. Three things are mechanical, and one is not.

• A quote must be findable before any model judges it. Location runs in three tiers: exact substring, then whitespace-tolerant, then a fuzzy longest-match that is accepted only if it covers most of the quote. A quote that cannot be located raises an error before an LLM sees it, so a fabricated quotation fails on arithmetic rather than on a judgment call.
• Sources are frozen when they are read. Every fetched page is content-addressed by a hash of its URL and stored as bytes, with private and internal targets rejected before the fetch. Verification re-reads those bytes, never the author's displayed quote, so the judge and the author see identical input, and a reviewer in six months inspects the evidence as it stood at decision time, not as the vendor's pricing page reads today.
• The judge can still be wrong. It is a separate model call against a three-way rubric, and where it returns nothing the claim defaults to unsupported. What the architecture buys is independence and a paper trail, not infallibility. The thing that writes a claim is never the thing that clears it, for the same reason your accountant doesn't audit their own books.

Same rule, two other places it had to be built: Assay keeps a coding agent away from the tests that grade it, and Ratchet keeps an implementer away from the evaluator that scores it.

That separation earns its keep. A 2026 study of deep-research agents found frontier models keep citation link validity above 94% and topical relevance above 80%, while the citation's actual factual accuracy lands at 39–77%, and degrades the longer the agent digs, with one model falling from 78.6% to 16.7% after 150 tool calls. The citations look fine. That is the problem.

Two more decisions in the same spirit:

• It refuses to score. No weighted 1-to-5 theater. It names the handful of factors that actually decided things and shows its reasoning, so you can argue with the logic instead of squinting at a number.
• It knows the edge of its own competence. An earlier version could recommend acquiring a company. I cut it. Valuing an acquisition needs financial and legal diligence that no amount of web research can responsibly produce. A tool that knows what it can't know is worth more than one that fakes it.

What it won't do

It won't approve a purchase. It won't replace your lawyers, your security review, or your procurement team. It won't pretend a missing price is anything other than missing: a vendor who hides their pricing is a finding, not a hole to fill with a guess. And it won't make the call.

That last one isn't a limitation. It's the posture. Research is the part you can safely automate. Judgment about where your money goes is the part you shouldn't. The machine does the reading; a person makes the decision.

Where it goes from here

Right now it's sharp for a single decision, made well. The next step is memory: every decision a company makes gets stored, so the next service it evaluates starts with that homework already done. Extend it across a portfolio and one company's decision becomes every other company's head start.

The code, the design documents, and every technical decision behind this live in the repo, linked at the top of this page. The repository is licensed GNU AGPL-3.0.