Solving the AI Cost Crisis
Gartner forecasts 40% of agentic AI projects will be cancelled by 2027. Not because the models aren’t capable, but because the economics don’t hold. Here’s the routing pattern I keep returning to.
Ferraris delivering pizza
Picture a Ferrari delivering a pizza. That’s what happens every time an enterprise routes a basic intent-extraction request to a frontier-class LLM. It works. But the compute bill scales faster than the business value does. Adoption is nearly universal (98% of organizations are experimenting, developing, or already in production with gen AI), and cost-efficiency has become the dominant priority for 83% of leaders building on top of it.
Gartner’s number matches what I keep seeing in conversations: the teams struggling aren’t blocked on intelligence. They’re blocked on a pricing model that punishes you for every token, regardless of whether the task actually needed a reasoning engine.
Sources: Google Cloud, 2025 State of AI Infrastructure research report; Gartner agentic AI forecast (2025).
This is one of the challenges I’ve been researching and experimenting with. My current proofs of concept run on simulated data, but as I evaluate how large organizations would actually scale a similar architecture into production, I’m learning the same thing most of them are: you can’t afford to throw a premium LLM at every user click. The question isn’t “which model?” It’s “which tier of compute is the cheapest one that still answers safely?”
02 · The frameworkThe Complexity Router & the three tiers
The routing mechanism I’ve been prototyping is a Complexity Router: a lightweight classifier plus a handful of heuristics that sits in front of the stack and decides where a request goes, based on cost and risk. The same customer-support system, looked at through the router, stops being a monolith and breaks into three visibly different kinds of traffic:
“Check order status”
Skip AI entirely. Deterministic policy, enforced in code. Free to run, 100% auditable, and impossible to hallucinate, because there’s no generation happening. The majority of support traffic is structurally this shape; teams just don’t see it until they instrument it.
“I want to return order #12345”
Route to a fast, cheap Small Language Model whose only job is to extract intent and the order number, then hand off to a deterministic backend. No frontier reasoning required. The next phase of the HCAI work I’m doing is exploring running Gemma 4 locally for this tier: zero API cost, full data locality, predictable latency.
“My shipment is late, I need this for an event tomorrow, how can we fix this?”
High-stakes ambiguity. Requires specialized agents to plan, fetch data, weigh policy, and draft a response under strict guardrails. Output is a mitigation strategy handed to a human for final judgement (see §3). This tier is expensive, which is exactly why you only want the 10% of traffic that actually needs it landing here.
This is also why 73% of leaders now prioritize edge and distributed deployment: moving Tier 1 and Tier 2 work off the hot path of centralized premium inference isn’t just a cost argument, it’s a latency and data-sovereignty argument too. The router pays for itself three ways.
03 · The human in Tier 3Why Tier 3 still needs a human gate
Tier 3 is the only tier where I think a Human-On-The-Loop (HOTL) checkpoint is worth the friction. Tier 1 is deterministic, so there’s nothing to review. Tier 2 is constrained extraction, so a bad output fails loudly and gets caught by validation. But Tier 3 is where the model is genuinely making judgement calls under ambiguity. That’s where the risk compounds.
The risk isn’t just a wrong answer. It’s data lineage. When an orchestrator pulls from three systems, weighs a policy, and drafts an action, the chain of reasoning has to be auditable. Otherwise you’re one regulatory question away from not being able to explain why the system told a customer what it did. A human reviewer at the Tier 3 gate isn’t just a safety net; they’re the anchor that makes the rest of the tier legally defensible.
The working prototype where I’ve been implementing this pattern is TradePilot. It’s the place where the HOTL gate actually lives in code: drafts are generated automatically, but a human signs off before anything touches a client portfolio.
Related prototype TradePilot →Where the Tier 3 HOTL pattern lives in practice. Multi-agent orchestration for high-reasoning financial workflows, with a human checkpoint before any action leaves the system.
04 · Strategic ROIThe right tool for the job
What the architecture really buys is discipline: the right tool for the job, and nothing more. External research on enterprise AI economics is converging on the same conclusion: the practical application story is finally catching up to the hype story, and the teams doing it well are the ones who stopped treating one model as the answer to every question.
Match workload to compute tier: a mix of spot, reserved, and pay-as-you-go billing, driven by intelligent resource management rather than a blanket premium-model default.
Tier 1 and Tier 2 automate the repetitive, rules-shaped work that was eating human capital. People shift to the 10% of problems that actually need human judgement.
Top priority for 55% of manufacturing firms and 67% of retail firms. The router makes internal automation economically viable at scale, because most internal workflows are Tier 1 or Tier 2 problems, not Tier 3 ones.
What I’m testing next
Being honest about where this work actually is: it’s research and prototyping, not a productized answer. The three threads I’m actively pulling on right now:
- What I’m mapping: which categories of tasks cleanly map to deterministic rules vs. SLM extraction vs. Tier 3 orchestration, and where the boundaries get fuzzy.
- What I’m testing: running Gemma 4 locally as the Tier 2 engine, to see whether zero-API-cost routing is practical for a realistic volume of support traffic.
- What’s still open: the HOTL interface design for Tier 3, end-to-end data lineage tracking across tiers, and making the Complexity Router itself auditable so its decisions can be reviewed.
The goal of enterprise AI isn’t to use the smartest model for every task. It’s to use the most cost-optimized compute layer that solves the business problem safely.
← Back to WritingScoping a complexity router, or trying to rescue an AI footprint that’s already burning?
- → Mapping traffic to tiers on an existing gen AI deployment.
- → Designing the boundary between rules, SLMs, and frontier models.
- → Designing a Tier 3 HOTL checkpoint that teams will actually use.