Every Task Has Its Own Frontier

There’s a Pareto frontier for every task in terms of cost, accuracy, and latency. I think a huge part of the opportunity in AI is helping companies get to that frontier for the tasks they actually care about.

At the same time, every task has a target price and a required level of accuracy, where the economic value created makes the margin profile attractive. A customer support response, a contract review, and parsing a page of a financial report have very different economics. The right operating point depends on the value of getting the task right, the cost of getting it wrong, and how quickly the result needs to come back.

Frontier models in apps like Codex, given a “simple” prompt + connected MCP servers + filesystem access, will continue to provide increasing accuracy for many tasks. They continue to absorb a vast amount of domain knowledge - I expect they will keep improving in reasoning and large-context retrieval, helped by all the work going into training data.

However, there usually exists a massive opportunity to continue improving accuracy, reducing cost, and reducing latency for a specific task.

Getting to the right operating point

The first step is defining the right evaluation rubric for what you want to achieve. There are many ways of defining a goal: through natural language evaluated by an LLM as a judge, a deterministic metric, or a combination of both. The rubric should capture the things that make the output useful: preserving table structure, following a refund policy, or flagging contract terms against your own standards.

You also want to measure the total cost per successful task, including retries and human review. A cheaper model that requires someone to fix every other output can get expensive pretty quickly.

After that, you can optimize a task in various ways:

Preloading the model + agent with relevant context instead of having it retrieve that context through MCPs. If you already know which policies, documents, or account details the task needs, you can eliminate a lot of unnecessary discovery and tool calls. This can massively reduce latency and cost when the context is focused and reusable; dumping everything into the prompt has its own costs.

Encoding human preferences inherent to the task so the agent can better satisfy the rubric at hand. Some examples include a legal team’s preferred fallback clauses, an analyst’s valuation methodology, or the exact schema a downstream system expects. This prevents humans from needing to spend time explaining the same preferences and correcting the same outputs themselves with a general-purpose agent like Codex.

Model routing and harness optimization. This can mean using a mixture of cheaper models for simpler tasks and stronger models for difficult cases. This can also mean compiling an agentic run into a semi-deterministic workflow: once you know the steps that reliably work, you don’t need the agent to rediscover them on every run. You can keep the flexible reasoning for the parts that actually need it.

Post-training, or generally any type of training. This allows you to encode desired behaviors into the model itself, which can lead to higher accuracy on a specific task with a smaller, cheaper model. The opportunity is to reach the accuracy you need at an order of magnitude less inference cost, even if frontier models gradually catch up in capability. Whether that pays off depends on task volume, the quality of your training signal, and the cost of maintaining the model.

These approaches can build on each other. A frontier model can help you discover a workflow, successful runs can become training examples, and your evaluation rubric can tell you which steps a cheaper model can take over.

This is what “owning your own intelligence” means to me - owning the task definition, the context and preferences, and the feedback loop that improves performance over time. You can start with a general-purpose agent and build up these assets as you learn where the system fails. Each human correction becomes a potential evaluation case, workflow improvement, or training example.

Packaging the outcome

I see many companies pursuing this opportunity, either offering optimized outcomes directly or offering the tools to help users get there themselves.

Vertical AI companies like Sierra, Harvey, and Rogo focus on customer service, legal, and finance. The value of the “prepackaged” approach is that users can get domain-specific workflows and context without assembling and operating everything themselves with general-purpose agents.

Infra companies like Applied Compute and Trajectory offer tools for users to train and improve models on their own tasks and production data.

At LlamaIndex, we see ourselves as a specialized model company building for the Pareto frontier of document parsing. The task is specific, but the failure modes are varied: preserving reading order, reconstructing tables, understanding charts, and handling layouts that look nothing like the examples you tested on. The useful output is a representation that downstream agents and workflows can actually trust.

As frontier models improve, we can use them to tackle harder documents and improve the systems around them. The next set of failures becomes the next set of evals and training examples, while the parts we already know how to solve should keep getting cheaper and faster.