Built Jitterbit's Integration Platform Agent and API Lifecycle Management Agent

Architecture and reasoning are shared here; customer workloads and confidential details are omitted.

At Jitterbit I built two production agents: an Integration Platform Agent that turns a plain-language request into a working integration, and an API Lifecycle Management Agent that designs, secures, publishes and monitors APIs.

Someone in warehouse operations who needs a new storefront order to reach the ERP should not have to learn a canvas of connectors first. The platform was already good; from October 2024, through production rollout, I owned the layer that lets them ask in a sentence. An agent that drafts an integration is a demo. One that can change an order flow or expose an API is an operator, and operators need boundaries.

Integration Platform Agent

One request can cross warehouse operations, CRM, ERP, finance and payroll, and grow from a sentence into a long run of lookups and confirmations. The bill climbed turn over turn, answer quality drifted as sessions lengthened, and some sessions ran out of context. All three had one cause: the conversation was the unit of context, so every turn re-sent the whole session.

I split the work across a LangGraph orchestrator and two agents. A Planner works out what the user is asking for and plans against the recipe corpus; an Executor carries the plan out against the platform. The Planner emits a Pydantic-validated JSON model and the Executor never re-reads the user's wording, so natural language becomes a plan in exactly one place, validated at the boundary. Every agent confirms with the user before it implements anything.

Diagram: a natural-language turn enters a LangGraph orchestrator, which routes to a Planner and an Executor. The Planner reads intent and slots itself and does RAG over the recipe corpus on Pinecone; it passes a Pydantic-validated JSON contract to the Executor, which acts on that rather than re-reading the user's wording. Each agent holds its own context. Every agent confirms with the user before implementing. Underneath sit a rolling summary regenerated on a token threshold and run off the hot path, checkpoints that resume a session across restarts, and a cache for repeated lookups. — The Planner is the only component that reads what the user wrote.

Each agent holds only the slice it acts on: the Executor sees the plan, not the transcript. Once agents stopped sharing a context they no longer had to share a model, so the Planner gets the strongest one and nothing else pays for it. Summarization keeps what carries forward bounded.

Three fixes I turned down

A sliding window is cheapest and wrong here: a user states the requirement early and spends nine turns adjusting details, and a window drops the requirement while keeping the adjustments.

A longer-context model buys headroom at a higher cost per call, in exactly the dimension that was already the complaint. The ceiling moves; it does not go away.

Retrieval over the conversation was the most tempting. When retrieval misses over documents, an answer is worse and the user can push back. When it misses over session state, the system forgets something the user said and proceeds confidently. I would not put session state behind a recall metric.

What it cost, and how I measured it

Summarization added around nine seconds when a session crossed its token threshold and ran inside the response path. The trigger is the amount of context, not a turn schedule, because sessions grow at different rates. I moved it off the hot path by regenerating the summary between turns, and cached repeated lookups.

Langfuse tracing made the latency visible and set the token thresholds from observation. DeepEval checked whether an answer late in a session still reflected what was established early; retrieval was evaluated on context precision, context recall, and answer faithfulness and groundedness. Human review covered what automation would not reliably catch: a summary that reads perfectly and has dropped the one constraint the user cared about.

Context precision improved by over 60% and retrieval latency dropped 30% on the RAG-augmented planning agent. The summarization and memory layer held answer quality as sessions grew while cutting LLM spend by around 40%. These were measured at the time on the platform's own tracing; I no longer have access to those systems, so I quote them as recorded.

API Lifecycle Management Agent

This one had the opposite shape. API lifecycle management is a fixed, countable set of workflows that barely branch, so there was no route for an orchestrator to plan. The agent ran on LangChain, with every action a tool call. Each new action added a tool and each tool added instructions, until the system prompt held all the procedural knowledge and was paid for on every request.

I moved the fixed workflows into skills, bounded procedures that call the tools in a defined sequence. The model selects the skill and the skill carries the workflow, so adding an action stopped taxing every conversation.

Securing and publishing an API means handling credentials. The obvious path is to accept them in the conversation and then defend them: mask them in logs, encrypt them at rest, keep them out of traces, and be right on every future code path. Instead, a browser popup captures credentials from the user and passes them straight to the runtime that needs them. The model never sees one, so there is nothing to scrub and no prompt change can leak one.

Two workflow shapes, two architectures, one rule: give the model only the context it needs to act, and keep secrets outside its reach.