Why We Added an IR Compiler to Fetch Hive's AI Orchestration Layer
By
Tom Dallimore
Published
A few weeks ago we asked Fetch Hive's Dashboard Copilot to build what should have been a fairly straightforward company research workflow.
The idea was simple. Take a list of companies, iterate over them, pass each company to an existing Research Agent with access to proper research tools, then take those results and produce a final report.
Something roughly like this:
Fetch Hive already supported every part of this. You could build the workflow manually in the editor and it worked exactly as expected.
The Copilot built something else.
Instead of using the Research Agent, it generated a list of search queries, iterated over those and called Google Search directly. The workflow was technically valid. It ran. There were no spectacular errors or exploding servers.
It was just a worse workflow.
That distinction ended up being much more interesting than the original bug.
At first glance this looks like a model problem. Maybe the prompt needs another instruction. Maybe we need another example showing when an Agent should be used. Maybe a stronger model will make the right choice more consistently.
Once we followed the problem through the stack, though, that explanation fell apart. The model was not simply making a bad decision. Parts of our own authoring system disagreed about what a valid Agent workflow step actually looked like.
The editor expected one contract. The MCP authoring layer still described parts of an older contract. Some validation logic expected something slightly different again. The correct workflow existed, the runtime supported it, and a human could build it, but the path available to the Copilot could not reliably represent it.
That was the point where this stopped looking like a prompting problem and started looking like an architecture problem.
What we were originally trying to build
The larger goal behind this work is something we currently call Ask Fetch Hive.
Fetch Hive already has a lot of different ways of executing AI work. There are normal prompts, deterministic Workflows, autonomous Agents, Hive Agents that can split a task across multiple agents and verify the results, integrations, human approval steps, knowledge bases, schedules and a growing set of tools.
That flexibility is useful when you know the product, but it also creates a fairly obvious problem for somebody arriving for the first time.
If a user says:
Every Monday, research 20 companies in the AI agent space, find how much funding they have raised, verify the information and send me a report in Slack.
they should not need to understand our internal product vocabulary first.
They should not have to decide that this probably needs a scheduled Workflow containing a Hive Agent, some parallel research, verification and a Slack action. Fetch Hive already understands those concepts. The system should be able to work out which combination makes sense.
The basic idea behind Ask Fetch Hive is therefore deliberately simple: tell Fetch Hive what you want done, and let Fetch Hive decide how to get it done.
We did not want to build another execution engine to achieve that. The existing runtimes already solved those problems. Workflows are good at durable, repeatable execution. Agents are good when the outcome is known but the exact path can change. Hive Agents make more sense when a larger task needs to be decomposed, worked on independently and verified.
The conversational layer should sit above those systems and compose them.
Initially, letting the model orchestrate those capabilities directly seemed like the obvious way to do it.
For a while, it was.
Direct orchestration works surprisingly well when the system is small
Imagine the orchestration layer only knows about five operations:
A capable model can work with that fairly easily.
If somebody asks it to find recent announcements from ten companies and summarise them, there are only so many sensible ways to assemble the pieces. Iterate over the companies, search for information, possibly scrape the useful pages and pass the results into a prompt.
You can explain the available tools and their schemas in the model context without things getting ridiculous. The model has enough information to reason about both what should happen and how to build it.
The problems arrive gradually.
You add another search provider. Then another.
You add conditional branches. Parallel iteration. Structured output. Agents. Agent tools. Knowledge bases. Human approvals. File generation. Connected integrations. Multiple accounts for the same integration. Different output shapes for different search tools.
Every new capability brings a few more rules.
One search step returns an array directly. Another returns results under a field. One operation can sit inside an iteration but requires a specific failure policy. A branch needs two correctly labelled paths. An Agent workflow step references an existing Agent rather than owning the Agent's model and tools itself.
None of these rules are particularly unreasonable on their own.
The problem is where you put them.
We had gradually been putting more and more of them into the model's world.
Our prompt was slowly becoming an internal programming manual
This is an easy trap to fall into because every fix looks sensible.
The model references the wrong output field, so you explain the correct output shape.
It gets an iteration wrong, so you add the iteration rules.
It builds an invalid branch, so you explain how branch edges should work.
It selects a model that does not exist anymore, so you tell it to inspect the available models first.
You add another integration and explain how authorization works.
One rule becomes five, then five becomes twenty.
Eventually you realise the model is no longer being asked only to answer:
What is the right execution strategy for this request?
It is also being asked:
Can you remember the implementation contract for most of our orchestration platform and produce the exact final representation our runtime expects?
Those are two very different jobs.
The first involves reasoning.
The second looks suspiciously like compilation.
The Agent workflow bug made that impossible to ignore
The company research workflow was useful because it showed how badly those responsibilities had become mixed together.
An Agent workflow step in Fetch Hive is conceptually very simple. The workflow references an existing Agent and provides the message that should be sent to it.
Something like:
The Agent itself owns things like its instructions, model and tools.
The workflow step does not need to reproduce all of that configuration.
That was already how the product worked, but the authoring path available to the Copilot had drifted. Parts of it still described an older inline Agent concept with fields the current step no longer owned.
Some of those fields were then removed when the workflow was built. Elsewhere, validation logic could reject the actual valid shape because it was still expecting the old one.
No amount of clever reasoning from the model fixes that cleanly.
We could give it another paragraph of instructions explaining the inconsistency and ask it to work around us, but at that point we are making the model compensate for deterministic problems in our application.
That seemed backwards.
More importantly, this was only one step type. Fetch Hive already had dozens of capabilities and we intend to keep adding them.
Fixing every future mismatch by making the orchestration prompt increasingly specific was obviously going to get worse.
We realised the LLM had become the compiler
Once we started describing the problem this way, the solution became a lot easier to reason about.
The interesting thing was that much of the back half already existed.
Fetch Hive already had code for validating workflow specifications. We already had graph validation, model validation, dry-run building and checks around the resources being created.
What we were missing was a proper front half.
There was no clean translation layer between:
This is what the user wants to happen.
and:
This is the exact Fetch Hive representation required to make it happen.
The model was effectively acting as that translation layer.
So rather than continuing to increase the amount of implementation detail we pushed into the model, we decided to separate the two responsibilities.
The architecture we are building now looks roughly like this:
The model still plans the job.
It just stops being responsible for compiling the final implementation.
Introducing the Plan IR
IR stands for intermediate representation. The idea is common in compiler design: instead of translating a high-level language directly into whatever the final machine expects, you introduce an intermediate format that is easier to analyse and transform.
We are applying roughly the same separation to orchestration.
The planner no longer needs to generate the final Fetch Hive workflow schema. Instead it produces a much smaller representation describing the intended execution.
A simplified plan might look something like:
intent: research companies and produce a report
This is intentionally not a Fetch Hive workflow definition.
There are no database IDs in there. There are no low-level graph edges, internal interpolation paths, integration authorization IDs or provider-specific model fields.
The planner is expressing the parts that actually require planning: what needs to happen, which pieces depend on each other and what outcome each node is trying to achieve.
The compiler can derive the rest.
That distinction turned out to be one of the most important design rules we made.
If software can infer it, the model should not invent it
Take an iteration.
The planner should be able to say:
The Fetch Hive runtime may need considerably more information than that. The final graph needs to enter the iteration, execute the correct nodes inside it, continue between items and converge after the loop has completed.
Those edges are not creative decisions.
Given the plan, there is a correct way to construct them.
So the compiler does it.
The same applies when one node consumes the output of another. The plan can simply say:
reads:
The compiler can turn that symbolic relationship into whatever internal reference Fetch Hive requires.
Model selection is another example. A planner might indicate that a particular step needs strong reasoning or a large context window. It does not necessarily need to know the exact provider slug, model identifier and policy rules required at the moment the workflow is created.
That is application state. It changes.
The compiler can resolve it.
Our general rule became: if something is a deterministic consequence of the plan or the current workspace, try very hard not to put it in the Plan IR.
Otherwise the IR eventually becomes another giant application schema and we end up exactly where we started.
Why the IR is flat
Another decision we made was to keep the canonical Plan IR as a flat, execution-ordered list of nodes rather than building a deeply nested tree.
An iteration looks like this:
rather than burying the Agent inside several levels of nested body objects.
At first glance the nested version can look more natural because it visually resembles the workflow.
The flat version is much easier to work with once you start thinking about validation, repairs and different model providers.
Every node has a stable ID. Relationships are explicit references. If the compiler finds a problem, it can say that research references an Agent that does not exist, rather than asking the model to find something buried inside a nested structure.
It also makes repairs smaller.
If one node is wrong, the planner should not need to regenerate half of the graph just to change it.
What the Intent Compiler actually does
The compiler itself is being built as a deterministic Rails service because that is where the registries, workspace state, tenancy and existing validators already live.
Conceptually, the compilation process has several stages.
First, the frontend parses the Plan IR, checks the version and normalises the representation.
Then capability resolvers map symbolic requests onto things that actually exist in the workspace. Agents, models, tool capabilities, integrations, knowledge bases and connected accounts are resolved here instead of being hallucinated by the model.
After that, target-specific lowering converts the plan into the type of Fetch Hive resource being created. A Workflow needs a graph. An Agent needs its tool and model configuration. A Hive Agent has another representation again.
Compiler passes then fill in the deterministic details: identifiers, graph edges, references, prompt scaffolding, model policy and failure behaviour.
Finally, the generated resource passes through the validation stack we already use.
That last part matters because we do not want the compiler to become a second interpretation of what a valid Fetch Hive workflow is.
The runtime remains the authority.
The compiler should produce resources the existing runtime already understands.
This is probably the most technical graphic and would fit directly beside this section.
Integrations are a good example of what belongs outside the LLM
Suppose somebody asks:
Send the final report to Slack #marketing.
The model understands the intent perfectly well.
There is very little value in also asking it to reason about internal Slack authorization IDs.
If the workspace only has one Slack connection, use it.
If there are three Slack workspaces but only one contains a matching #marketing channel, resolve it.
If multiple valid destinations still exist, then there is a genuine decision the system cannot make by itself and it can ask the user.
This is a pattern we want throughout Ask Fetch Hive: never ask the user for information the system can discover itself, and never ask the model to invent information the application already knows.
That sounds obvious written down.
It becomes surprisingly easy to violate once an LLM sits in the middle of everything.
Where MCP fits into this
This work is happening while we are also moving more of Fetch Hive's control surface towards MCP.
Those things are related, but they solve different problems.
MCP gives models a consistent way to discover and invoke capabilities. That is useful, especially as more tools and services become available through a common interface.
What MCP does not mean is that the model should be responsible for manually assembling every implementation detail required to use those capabilities correctly.
Without the compiler, a sufficiently complex authoring process starts looking like:
That can work, but the amount of probabilistic behaviour in the authoring process keeps increasing.
The architecture we are moving towards is closer to:
MCP remains part of the capability and control layer.
The compiler absorbs the translation rules.
This is particularly important as Fetch Hive's tool surface continues to grow.
Going from ten tools to a hundred should not require ten times the prompt
This is probably the biggest architectural reason for doing the work.
If every new capability requires another block of instructions explaining exactly how the model should represent it, the orchestration layer gets harder to maintain every time the product gets better.
That is the wrong scaling curve.
The compiler is registry-driven instead.
A capability can advertise what it does, what inputs it accepts and what output shape it produces. The compiler can then use that information when lowering a plan.
The planner still needs enough information to understand that, for example, a search capability exists and is appropriate for the current task.
It does not necessarily need every internal field required by that search implementation sitting in its context.
There will always be some relationship between the size of the platform and the amount of context required by the planner. We are not pretending that disappears.
The goal is to stop implementation trivia growing at the same rate as capability.
Going from ten tools to a hundred tools should make Fetch Hive much more capable. It should not require the orchestration model to carry an increasingly fragile internal API manual around on every request.
Failure becomes easier to reason about too
The IR also gives us a better unit for handling problems.
Instead of getting to the end of workflow generation and returning something vague like:
Invalid workflow specification
the compiler can attach errors to the part of the plan that caused them.
For example:
or:
or:
That gives the planner something specific to repair.
It also means the repair loop can stay bounded.
If the model made a fixable planning mistake, give it the diagnostic and let it repair the affected part. If the problem requires information only the user can provide, ask the user.
We already use a similar pattern elsewhere in Fetch Hive when validating Hive Agent plans, so there is a precedent for this approach working well.
What we do not want is an orchestration system that takes an invalid 30-step workflow, feeds the whole thing back into the model and says "try again" until something eventually passes.
At that point your validation strategy is basically persistence.
The model still makes the interesting decisions
None of this removes the LLM from orchestration.
It removes the jobs where an LLM adds very little value.
Consider these two requests.
The first:
Research AI coding agents and summarise what changed this month.
That might be a one-off Agent or Hive Agent execution. There is probably no reason to create durable infrastructure if the user only wants the answer once.
Now change it to:
Every Monday, research AI coding agents, compare the market with last week, verify anything important that changed and post the report to Slack.
That requires a different execution strategy.
The task is recurring. Previous state matters. Research and comparison are required. Verification is useful. The output needs to be delivered somewhere.
A sensible execution plan might become:
Working out that architecture involves reasoning.
Remembering how a particular Slack action encodes its connection ID does not.
Deciding whether the research is complex enough to justify Hive Agents involves reasoning.
Generating the exact graph edges required by an iteration step does not.
That is the boundary we are trying to make explicit.
This is the foundation for Ask Fetch Hive
Ultimately this compiler is not being built because we particularly wanted another service called IntentCompiler.
It exists because a conversational interface becomes a very different engineering problem once the conversation starts creating durable infrastructure.
A demo where an LLM calls a couple of tools is relatively easy to make look impressive.
The harder version is when somebody says:
Do this every Monday.
Now whatever the model creates needs to exist next month. It needs to be inspectable. Editable. Valid. Observable. It needs to use the right connected accounts and survive changes in the model that originally planned it.
The top-level conversational system should not wake up every Monday and invent a completely new interpretation of the automation.
It should build the automation once and let the workflow runtime own the repeatable execution.
That is why Ask Fetch Hive sits above our existing primitives rather than replacing them.
The chat interface is simply the easiest way for the user to describe the outcome.
Underneath, we still want boring, inspectable infrastructure.
Boring is quite good when something is supposed to run unattended at 8am every Monday for the next year.
What we learned from it
The original mistake was not letting an LLM orchestrate Fetch Hive.
Models are genuinely useful at understanding ambiguous requests and deciding what kind of work needs to happen.
The mistake was gradually allowing orchestration and compilation to become the same job.
As the platform grew, the model became responsible for more schemas, more output shapes, more graph rules, more IDs and more implementation-specific contracts. Every individual addition was manageable, but collectively they made the authoring layer much more fragile than it needed to be.
The company research workflow made that visible because the model did not spectacularly fail. It built something that looked reasonable and passed validation, but it had quietly chosen a weaker route because the better one was effectively blocked by our own contracts.
That is exactly the kind of problem that becomes painful at scale.
So the direction we are taking is fairly straightforward. Let the model understand what the user is trying to achieve and decide on the execution strategy. Express that decision through a deliberately constrained intermediate representation. Let deterministic application code translate it into real Fetch Hive resources, then let the existing runtime validate and execute them.
It adds another layer to the architecture, and I generally try not to add layers just because they make an architecture diagram look more impressive.
In this case, though, the new layer takes a growing amount of deterministic complexity away from the least deterministic component in the system.
That feels like the right place to put it.
Share this post
