Coinbase says it cut its AI bill nearly in half while token usage kept climbing.
The company got there through cheaper defaults for routine work, task-aware routing, better caching, leaner context, and clearer visibility into usage. Employees could keep using AI. The architecture absorbed the growth.
That result should get the attention of any company watching its AI budget climb. Most enterprise teams start with the strongest model they can access because the early goal is simple: prove the use case works.
The pilot ships. Another team copies it. Soon the same frontier model is classifying tickets, retrieving policy text, drafting boilerplate, and handling the hardest reasoning in the system.
The first invoice is manageable. The tenth workflow is where the architecture starts to show.
A useful model strategy gives each workload the level of intelligence, context, control, and verification it actually needs. This guide covers the four decisions behind that strategy, where routing and fine-tuning fit, and how to tell when optimization will pay off.
Model portfolios get expensive by accident
Few companies sit down and deliberately design a wasteful model portfolio. Sprawl arrives one successful pilot at a time.
One team chooses a frontier API. Another team signs up for a different provider. A third team deploys an open-weight model because a client has a strict data requirement.
Each choice may make sense on its own. Together, they create duplicated integrations, inconsistent security controls, weak usage data, and no reliable way to compare cost with outcomes.
Model selection also tends to happen too early. A team debates Claude, OpenAI, Gemini, or an open model before it has defined the work. The model becomes the strategy.
In our work at HatchWorks AI, the useful conversation starts one level lower: What job is the system performing, and what has to be true for that job to count as successful?
Once that answer is clear, model choice gets much easier.
Coinbase changed the economics around the model
In June 2026, Coinbase CEO Brian Armstrong described the company's playbook for keeping AI spend under control while token usage grew. Five changes did most of the work:
1. Cheaper defaults. Employees could still choose a frontier model when the work required it, while routine usage started on less expensive models.
2. Task-aware routing. An internal gateway could send work to different models based on difficulty, price, availability, and cache state.
3. Prompt caching. Reused context did not need to be processed at full price on every turn. One Coinbase implementation reportedly moved its cache hit rate from about 5% to 60%.
4. Leaner context. Teams reduced unnecessary files, tools, and stale conversation history before each request.
5. Visible usage. Coinbase made spend visible without turning every token into an approval process. Armstrong said 91% of employees had not been reaching the old usage caps anyway.
The bigger lesson is architectural. Coinbase improved the system surrounding a portfolio of models instead of betting its economics on a single cheaper model.
Its July 2026 earnings update gave the broader outcome: AI usage was growing faster than spend, pull requests processed per engineer had increased 2.2 times year over year, and integration-test coverage had grown 2.5 times in six months.
The portfolio was doing its job.
The four decisions inside an AI model strategy
A model strategy becomes easier to reason about when four decisions stay separate. They influence one another, but they answer different questions.
1. Define success for the workload
Start with the work. A useful workload profile answers:
- What outcome should the system produce?
- How will you test whether the outcome is correct?
- What happens when the system is wrong?
- How much latency can the workflow tolerate?
- How often will it run?
- What data will it touch?
- Where does a person need to review or approve the result?
A high-volume classification job and a one-off strategic analysis have different economics. A customer-support answer and a payment decision have different risk. A model that wins a public benchmark may still fail the business test that matters inside your workflow.
We usually start by building the evaluation set before debating the final architecture. Twenty to fifty representative tasks can expose where a cheaper model clears the bar, where proprietary context is missing, and where the strongest model still struggles.
2. Choose model access and deployment together
The next decision combines the model you use with where it runs.
Proprietary models offer strong capability through a managed service. Open-weight models give teams more control over customization, hosting, and portability. Either category can be part of a secure enterprise architecture.
Deployment requirements need more precision than a simple cloud-versus-on-premises choice. Data residency, inference residency, retention, zero-data-retention controls, private networking, and full infrastructure isolation describe different needs.
Managed providers can support some regional-processing and retention requirements. A strict requirement that data never leave a company's controlled environment may point toward a privately hosted open-weight model.
The right answer comes from the actual policy and workload. It should also account for licensing, vendor support, model provenance, security testing, and the operational burden of hosting.
For a deeper comparison, see Open-Source LLMs vs. Closed LLMs.
3. Match the adaptation to the gap
Many model-strategy diagrams jump from an off-the-shelf model straight to fine-tuning. There are useful steps in between.
Prompting is the fastest way to establish a baseline and shape the output.
Retrieval-augmented generation, or RAG, supplies current or proprietary facts at runtime. It is often the better choice for policies, product data, account history, and knowledge that changes regularly.
Fine-tuning changes how a model behaves. It is useful when a recurring task needs stronger domain performance, consistent formatting, specialized terminology, or a pattern the base model does not follow reliably.
It becomes more attractive when the task is stable, volume is high, labeled examples exist, and the evaluation set can prove the improvement.
Continued pretraining or training a model from scratch demands far more data, infrastructure, and ongoing ownership. Most enterprise applications will not need that level of investment. The cases that do usually have unusual capability, ownership, scale, or sovereignty requirements.
These methods can work together. A fine-tuned open-weight model can retrieve current company knowledge. A proprietary model can sit behind a router. A production workflow can use both.
AI Inference, Training, and Fine-Tuning goes deeper on the technical distinctions.
4. Orchestrate the portfolio
Orchestration decides what happens at runtime. It can route a request, reuse cached context, call a deterministic service, escalate to a stronger model, verify the result, or send a high-risk action to a person for approval.
A common pattern uses a smaller model for repeatable, high-volume work and reserves a frontier model for ambiguous or difficult steps. That pattern is a starting hypothesis. Production evaluations decide whether it holds for a specific workload.
The routing logic can begin with clear rules:
- Send known lookups to an API or database query.
- Send current knowledge questions through retrieval.
- Send stable classification and extraction work to the lowest-cost model that passes the evaluation set.
- Escalate ambiguous, high-risk, or failed work to a stronger model or a person.
- Keep a conversation on the same model while its prompt cache remains useful.
Over time, the router can become more dynamic. It still needs maintenance. Every new model has to be integrated, evaluated, governed, and monitored before the portfolio benefits from it.
Coinbase's CEEcil shows the architecture in practice
Coinbase recently published the design of CEEcil, an internal support system built to retain team context and help engineers across time zones. Its request path is a concrete example of model allocation.
Ask CEEcil who is on call and a direct API lookup returns the answer. Ask about internal knowledge and the system retrieves from a versioned knowledge base. Open-ended investigations reach the full agent runtime.
A small model helps classify intent, then deterministic services handle the tasks that should return the same answer every time.
CEEcil also uses two model tiers. A smaller model handles frequent work such as classification and observation extraction. A stronger model handles multi-step reasoning and nightly consolidation. The service layer controls routing, memory, tool access, kill switches, spend limits, and every Slack reply.
That design is more useful than a generic rule about which model is best. It lets cost and latency follow the difficulty of the request. It also gives the company clear places to test, log, approve, and stop the system.
Routing and fine-tuning solve different problems
Routing and fine-tuning are often presented as competing options. In production, they are frequently used together.
Routing answers a runtime question: Where should this request go?
Fine-tuning answers an adaptation question: How should this model behave on a recurring task?
Consider a claims-processing workflow. A router could send straightforward document classification to a fine-tuned smaller model, retrieve the current policy language needed for a coverage question, call deterministic code for calculations, and escalate an unusual case to a frontier model and a human reviewer.
Fine-tuning becomes worth testing when the same failure pattern appears across a stable, high-volume workload. The economic case should compare the full cost of the current path with the tuned path:
Monthly opportunity = (current cost per successful task − tuned cost per successful task) × monthly task volume
The investment side includes data preparation, training, hosting, evaluations, monitoring, and maintenance. A lower token price alone does not guarantee a better system.
Measure the cost of successful work
Price per million tokens is easy to compare. It is also incomplete.
A cheap model that retries three times, produces more human corrections, or fails a high-value task can cost more than the frontier model it replaced. The useful unit is cost per successful task within the workflow's quality, latency, and risk requirements.
Track at least:
- End-to-end task completion
- Quality or accuracy on the production evaluation set
- Human correction and approval rates
- Retry, fallback, and escalation rates
- Latency and throughput
- Tokens and total model cost per task
- Cache hit rate and context size
- Errors by route, model, and workload type
This is where an evaluation harness earns its keep. It gives the team a stable way to compare models as prices, capabilities, and providers change.
Agent reliability comes from the whole system
Zapier's AutomationBench offers a useful reality check. The benchmark asks agents to complete multi-step work across 47 simulated business tools in sales, marketing, operations, support, finance, and HR. It grades the final state of the systems rather than the agent's written answer.
As of September 24, 2026, the top system on version 1.0.6 completed 42.47% of the held-out tasks under a strict pass-or-fail measure. GPT 6 Astra at maximum effort completed 41.4%. Those figures will move, probably quickly.
The failure pattern matters more: an agent can say the job is finished while the records, messages, or calculations remain wrong.
We like this benchmark because it measures completed work. It also shows why a production agent needs more than a capable model. Narrow scope, trusted context, deterministic steps, permissions, verification, observability, fallbacks, and human approval all affect reliability.
At HatchWorks AI, we describe that surrounding architecture as an Intelligence Layer: the connection between company knowledge, data, systems, models, and agents, with shared controls around them.
Open-weight models add control and responsibility
Open-weight models can improve economics, customization, portability, and deployment control. They also shift more operational responsibility to the company running them.
The procurement review should cover the model's license, training-data and provider disclosures, security history, software supply chain, support model, export controls, evaluation results, and update process.
A model's country of origin may matter to a specific organization or jurisdiction, but it is one part of a broader risk review.
Self-hosting also carries real costs: infrastructure, serving performance, patching, monitoring, and specialized people. Control has a price. The workload still has to justify it.
Portfolio optimization is worth the effort when the usage is real
An organization has a strong case for model-portfolio work when several of these conditions are present:
Optimization can wait when spend is low and flat, the use case changes every week, or the team has no reliable way to test quality. Even then, basic usage logging and a small evaluation set are worth putting in place. They make the later decision much easier.
A practical starting point
At HatchWorks AI, we use a four-stage sequence:
1. Evaluate. Map the workloads, current models, quality, latency, risks, and economics.
2. Specialize. Add retrieval or fine-tuning where domain performance creates a measurable advantage.
3. Orchestrate. Route each task, or the hardest part of it, to the model and control path best suited to handle it.
4. Optimize. Keep measuring quality, latency, token usage, and cost as the systems and models evolve.
The first deliverable should be a workload map. Preferred-vendor lists come later. For each important task, define the success test, data boundary, model path, fallback, approval rule, and cost per successful outcome.
The models will change again next quarter. A clear workload map, evaluation set, and orchestration layer give you a way to change with them without rebuilding every application.
