INQUIRING LINE

Companies are rushing to buy AI tools instead of building their own — but is that because bought tools actually work better?

Does buying third-party AI applications actually outperform custom internal builds?

This explores whether companies get better results from buying ready-made AI applications than from building their own, and what the collection can actually say about that.


This explores whether companies get better results from buying ready-made AI applications than from building their own. The short answer is that the collection documents a big shift toward buying, but it does not contain a head-to-head study showing that bought tools perform better. What it does have explains why the shift is happening, and those reasons are more interesting than a simple scoreboard. In a 2025 survey of 100 enterprise CIOs, a16z found that companies have clearly moved from building toward buying over the past year Are enterprises abandoning custom AI apps for third-party solutions?. The reason those leaders gave wasn't that vendor tools are smarter. It was that internal tools stopped paying off once teams counted the work of keeping them current: models change constantly, and every change means re-optimizing.

Other notes in the collection show why that upkeep is so heavy. Much of an AI system's real performance comes from the machinery around the model, not the model itself. One study raised scores on a hard coding benchmark across several models by improving only the execution harness, without touching the model weights. The same playbook carried over to newer models unchanged Can execution harnesses lift model performance without retuning weights?. A related argument holds that AI 'context' (the prompt, conversation history, retrieved documents, hidden state) keeps shifting in a way traditional software never did. Keeping it working is an ongoing engineering discipline, not a one-time build How does AI context differ from conventional software context?. Put those together and you get a plausible reason vendors win: they spread the cost of that ongoing harness and context work across many customers, while an internal team pays it alone.

There's a catch in the other direction, though. Benedict Evans argues that making AI tools easier to build doesn't fix the harder problems. Most workers don't see their own tasks as automatable, and real adoption depends on decisions that cut across departments Does easier tool-building actually solve enterprise adoption problems?. Buying a tool doesn't solve those problems either. It just moves the engineering burden somewhere else. Evaluation is also a problem for anyone comparing options. An analysis of 960 real occupational workflows found that agents that ace benchmarks often fail long, realistic professional tasks Why do agent benchmarks not predict real economic value?. So vendor claims based on benchmark wins are weak evidence that a product will beat a custom build on your actual work.

The developer world shows a similar pattern. Stack Overflow's 2025 survey found AI tool use climbing to 80% while trust in the tools' accuracy fell to 29%. Code that looks right but hides subtle errors creates a checking burden that doesn't go away Why do developers keep using AI tools they don't trust?. That suggests 'outperform' may be the wrong measure. Whether you buy or build, much of the real cost is in checking and maintaining the output. Neither option removes that cost, so the choice comes down to who carries it.

The less obvious takeaway is that 'buy vs. build' is mostly a question about who maintains the scaffolding: the harness, the context pipeline, and the re-tuning after every model update. The model itself matters less. If a company's advantage comes from knowing which of its workflows are worth automating, buying the scaffolding can make sense, as long as it keeps that knowledge in-house. The collection doesn't yet have rigorous outcome comparisons between bought and built tools, and that's a gap worth noticing.


Sources 6 notes

Are enterprises abandoning custom AI apps for third-party solutions?

a16z's 2025 survey of 100 enterprise CIOs found a marked reversal toward purchasing third-party AI apps over the past year. Enterprise leaders cite constant model optimization demands and high maintenance burden as reasons internally built tools no longer deliver business advantage.

Can execution harnesses lift model performance without retuning weights?

StateM improves Terminal-Bench 2.1 accuracy across multiple models by optimizing execution systems around frozen weights. The same runbook transfers to newer models without modification, achieving 95.3% on GPT-5.6 and lifting DeepSeek-V4 Flash by 5.4 points.

How does AI context differ from conventional software context?

AI interactions operate on a substrate of constantly shifting context—prompt, history, retrieved data, hidden state—that users cannot internalize like traditional UIs. This structural mutability demands a new design discipline centered on context engineering rather than interface design.

Does easier tool-building actually solve enterprise adoption problems?

Evans argues that reducing coding friction masks two structural barriers: most workers don't see their own tasks as automatable, and enterprise adoption requires organizational decisions that span departments and timelines—not just technical capability.

Why do agent benchmarks not predict real economic value?

ALE's analysis of 960 real occupational workflows shows agents excel at abstract contests but fail long-horizon professional tasks. The gap is not model capability but benchmark design—the field optimizes what it measures, and it has measured contests rather than work.

Show all 6 sources
Why do developers keep using AI tools they don't trust?

Stack Overflow's 2025 survey shows 80% of developers use AI tools while trust in accuracy fell from 40% to 29%. The primary complaint: AI code that looks correct but contains subtle errors, creating a verification burden that erodes confidence faster than usage grows.

Papers this line draws on 8

The research behind the notes this line reads — ranked by how closely each paper relates.