The nine layers an FDE builds, from client brief to production

A Forward Deployed Engineer gets a brief from a business, not a ticket. "Make an agent that does what our people do." Between that sentence and a system that runs in production sit many layers of work. This article takes one real build apart, layer by layer. For each layer it shows who at the client asked for it, what the FDE built, how it was proved, and when you can skip it.

Who it is for: engineers moving into FDE roles, and teams that build agents for a customer's business.

What this covers

1. The journey, and the loop inside every layer
2. Layer 1: the rules live in the backend
3. Layer 2: a harness that works a queue
4. Layer 3: state that survives, and a wait for a person
5. Layer 4: many agents at once, scored against the experts
6. Layer 5: sign-in, and an agent that improves under a rule
7. Layer 6: security before the cloud
8. Layer 7: context, memory and a gateway
9. Layer 8: a managed runtime
10. Layer 9: a copilot for each user
11. What carries to the next client

client brief ─► plan ─► client decides ─► build ─► prove live ─► next brief
                                                                    │
 9  copilot for each user          ◄── the handlers                 │
 8  managed runtime                ◄── platform team                │
 7  context, memory, gateway       ◄── data governance              │
 6  security before the cloud      ◄── information security         │
 5  sign-in + improver agent       ◄── the whole claims team        │
 4  many agents + evaluations      ◄── claims operations            │
 3  durable state + human wait     ◄── IT                           │
 2  harness: queue + recovery      ◄── claims operations + IT       │
 1  backend owns every rule        ◄── head of claims ──────────────┘

1. The journey, and the loop inside every layer

Each layer starts when a new person at the client asks a new question.

The example client is a travel insurer. Three claims handlers work a queue by hand. Each one reads the customer's story, checks the policy and its wording, then approves with a payout, declines, or refers. The head of claims wants an agent that does the same, and explains each decision in two or three sentences that cite the right section.

That first brief gives you layer 1. Then IT asks about crashes. Information security asks who can call what. Data governance asks where the facts come from. The platform team asks how it runs on their runtime. Each question becomes a layer.

Inside every layer, the FDE runs the same loop:

• Brief: someone at the client writes what they need.
• Plan: phases, each with checks the FDE can verify alone. A test, a command's output, an API response, or a page checked while signed in.
• Choices for you: what the documents don't decide goes back to the client, with options, a recommendation and the reason.
• Stop: the client reviews the plan before any code.
• Build: phase by phase. Tests, then checks, then a commit.
• Prove live: runs on the real model, each reporting what it cost.
• Update the documents.

A few rules hold across all nine layers:

• Tests never call a model. A stub model plays scripted replies, so the suite runs with no keys and no network.
• Thresholds are never lowered to get a green run. If a rule looks wrong, the FDE stops and asks.
• The handlers' past decisions are the answer key. The agent can never read them.

What to do: write the "choices for you" list before you write code. The client's answers shape every layer after it.

2. Layer 1: the rules live in the backend

The model may be wrong or misled, so it must never be the thing that enforces a rule.

Who asked: the head of claims. A queue, an agent that decides one claim at a time, a dashboard for the handlers. Any payout over $1,000 needs a handler's sign-off.

What the FDE built:

• The payment limit lives in the payment route, not in the agent's instructions. The agent can only ask. The backend decides.
• The agent gets six tools, the same ones a handler uses: read the claim, the policy, the history, the wording; record a decision; pay. There is no sign-off tool. Only a person signs off.
• Calls go one way. The agent and the dashboard both call the backend. The backend never calls the agent.
• The agent sees a customer's earlier claims, but not their address, phone or date of birth. It doesn't need them to decide.

How it was proved:

• Approve $420 → decided. Approve $1,250 → waits for sign-off. Paying it before sign-off → refused.
• The split-payment trick, $1,000 then $250 on the same claim → both refused.
• A refused payment comes back to the model as text. The run carries on instead of crashing.

Skip it: never. Every later layer assumes the backend holds the rules.

3. Layer 2: a harness that works a queue

The agent should organise its own work, but the backend still owns the rules of the run.

Who asked: claims operations, with IT. Claims will get harder, with more documents and more rules. They want one run to work the whole queue, with a to-do list the team can watch, a cap for cheap trials, and recovery that never decides or pays twice.

What the FDE built:

• A queue agent hands each claim to a fresh claim agent, on a new thread, which reports back in one line. A fresh agent per claim stops each step getting more expensive as the run goes on.
• The cap, the "second failure goes to a person" rule and the "you missed a claim" check are backend refusals, like the payment limit.
• When the agent tries to finish early, the backend refuses and names the claims it missed. A refusal is feedback that steers the agent.
• An error inside one claim becomes a line of text for the queue agent. One bad claim never stops the run.
• The documents have numbered sections, so the agent lists them and reads only what it needs. No search index.

How it was proved: a 6th claim on a cap-5 run is refused. Fail once → decided. Fail twice → left for a handler. Live, with a cap of 5: the to-do list filled and was ticked off, 5 claims ended and 20 stayed waiting.

Skip it: when the agent does one thing per request and never works a queue.

4. Layer 3: state that survives, and a wait for a person

An approval can wait days for a person, and a crash must not lose the work.

Who asked: IT, before the move to the cloud. When the agent approves a payout over the limit, its work ends. A restart loses the claim's progress.

What the FDE built:

• The claim agent becomes a LangGraph graph. The wait for a sign-off is a node of its own, using interrupt(). A checkpoint is saved in Postgres after every step.
• Paying is a graph step, not a model tool. The graph reads the claim's status from the backend, so the limit stays in one place.
• Every backend write is safe to repeat. The same decision again returns the claim. The same payment again counts as done.
• The sign-off is saved first, and the agent is told after. The sign-off stands even if the agent can't be reached.

How it was proved: live, a held claim signed off in the browser was paid within 5 seconds. A run killed partway resumed on the same thread with one decision. A defect showed up live: sign-off to payment took 13 seconds, because "localhost" tried IPv6 first. Switching to 127.0.0.1 brought it to 2 seconds.

Skip it: when nothing waits for a person and every run ends in seconds.

5. Layer 4: many agents at once, scored against the experts

Picking the next claim is a rule, so it needs no model. And nobody can say how good the agent is without a scoreboard.

Who asked: claims operations, with IT. One at a time, 25 claims take over 10 minutes. Nobody knows how often the agent agrees with the handlers, or what a claim costs.

What the FDE built:

• The queue agent is gone. A graph with no model sends one task per claim, with a cap on how many run at once.
• Each claim keeps its own thread, because a claim can wait days for a sign-off.
• Evaluations run on copies of the claims, through the same agent, so live data is never touched.
• The scoreboard: accuracy against the handlers, the cost of mistakes in dollars, the share of claims handled straight through, and cost and time per claim.

How it was proved:

Skip it: when items go one at a time and nobody needs a quality score.

6. Layer 5: sign-in, and an agent that improves under a rule

The agent can propose its own improvements, but code decides what is kept.

Who asked: the whole claims team, now that everyone uses the dashboard. Improving the agent was slow manual work: read the misses, edit the instructions, rerun, decide.

What the FDE built:

• Handlers sign in. The backend checks each token against the identity provider's public keys. Only a handler who can approve over the limit can run an improvement round.
• An improver agent reads the misses, proposes a new version of the instructions, and pauses for a person's approval.
• A rule in code keeps or rolls back each version. Keep it only when it meets the objectives, at least one measure improves by more than its noise, and none gets worse by more than its tolerance. The rule compares scores on held-out claims the improver never sees.
• A judge model grades each explanation. Each round has a spending limit, $0.50 at first.

How it was proved: round 1 kept a change: held-out accuracy 78% → 94%, for $0.199. Round 2 rolled back, because no measure improved by more than its noise. The kept version scored lower on the set the improver learned from (88% → 80%) while it rose on the held-out set. Tuning to the learning set would have kept the wrong change.

Skip it: when nobody signs in and the configuration is changed by hand.

7. Layer 6: security before the cloud

Before the app gets a public address, the agent needs its own identity, and untrusted text needs a check.

Who asked: IT, with information security, with three findings. The agent sends nothing to identify itself. A claim can hide an instruction in its story: the large model declined it, a smaller model approved it in full. And the app only runs on a laptop.

What the FDE built:

• The agent has its own API key, compared in constant time. An unset key refuses everything. The key never stands in for a handler.
• A check runs before the claim agent's first model call. A second model reads everything the customer wrote and says whether it tries to instruct whoever handles it. On a hit, the claim agent never sees the claim. It goes to a person.
• The agent's API document lists only its 8 operations. A gateway turns every operation it sees into a tool, so a narrow document keeps the rest out of reach.
• Every route belongs to exactly one access group, and a test enforces it.

How it was proved: live, at 25 agents, the check passed 25 real claims and flagged only the planted one. Its reason: the story told the assistant to approve $4,800 and said a clause had been waived. The run took 34 s and cost $0.031.

Skip it: never, for a product on the internet.

8. Layer 7: context, memory and a gateway

The agent needs facts from across the business, each with a named source, and a policy that sees every call.

Who asked: IT, with claims operations and data governance. The facts the agent needs are spread across systems. Data governance gathered them into 7 files, each naming its source system, owner and last review.

What the FDE built:

• A context server over MCP serves those files through read-only tools. Every answer names its file, its owner and its review date.
• The tools sit behind AgentCore Gateway. A Cedar policy refuses any payment over $1,000 before the call reaches the app.
• Thin tools: reading the claim takes no id from the model, so the model can act only on its own claim.
• Memory of each customer is saved in code after each decision, never at the model's choice. The customer's story is never saved, because that is where a hidden instruction would live.

Choices the client made: putting both documents in every prompt would add about 300 lines per model call and multiply the cost, so the context server also lists sections. A private database would need a NAT gateway at about $32 a month, so the client chose a public one that requires TLS.

How it was proved: a stand-in gateway in the tests refused a $1,500 approval, held it, and it was paid once after a senior handler signed off. The suite ended at 362 Python, 57 dashboard and 14 end-to-end tests.

Skip it: when the agent needs nothing beyond its own data and nothing must be remembered.

9. Layer 8: a managed runtime

The platform team owns the runtime's rules, so the product changes to fit them.

Who asked: IT, with claims operations and the platform team. Handlers also want to send a claim back to the agent with a note, instead of just leaving it waiting.

What the FDE built:

• The answer lives in the app, not in the request. "Signed off" or "sent back" is read from the claim's record, so a retry sees the same answer.
• Every "only once" is one conditional update of one row, checked by the row count. A check followed by an insert doesn't stop two starts.
• Nothing runs at import. Every runtime session starts from one snapshot, so a key opened early would be shared by all of them.
• Work is registered with the runtime before the agent answers, so the runtime knows it is busy and doesn't end the session. Unregistered work was killed a minute after the last request.

How it was proved: two sign-offs at once: one pays, the other is refused. A sign-off and a send-back at the same moment: exactly one goes through. Live: a full run, a claim signed off and paid, a claim sent back and decided again.

Skip it: when the agent runs on your own servers.

10. Layer 9: a copilot for each user

The last layer puts an agent beside each person, acting as that person and no one else.

Who asked: the handlers. Each task takes a few pages and a few clicks. They want a sidebar where they ask in plain English, "what needs me today?", and it acts for them, with their permissions.

What the FDE designed:

• Each turn starts by asking the app who is signed in. If the app refuses, the turn ends before any model call.
• The user's token travels in the graph's context, never in its saved state, so it is never written to a checkpoint.
• Before any sign-off, a confirm step pauses for the user. The card it shows is filled from the app's records, never from what the model wrote.
• Figures are worked out in code and handed to the model as a finished table.

Status: a reviewed design, not built yet.

Skip it: when users never need to ask or act through chat.

11. What carries to the next client

The client, the documents and the rules change. These patterns don't.

• The backend owns every rule. The agent can only ask. (Layer 1)
• A refusal from the backend is feedback that steers the agent. (Layer 2)
• A wait for a person is its own step, with a checkpoint after every step. (Layer 3)
• Every write is safe to repeat. Every "only once" is a conditional update. (Layers 3, 8)
• Evaluate on copies. Keep the answer key out of the agent's reach. (Layer 4)
• Keep a new version on measured held-out scores, never on the model's opinion. (Layer 5)
• The agent has its own key and its own narrow API. The key never stands in for a person. (Layers 6, 9)
• Screen untrusted text before the model sees it, and never save it to memory. (Layers 6, 7)
• Put tools behind a gateway with a policy. Every answer names its source and owner. (Layer 7)
• A user's agent acts with that user's sign-in. Figures come from code. (Layer 9)

Checklist

• Write the client's open choices down, with a recommendation, before the first line of code.
• Put every business rule in the backend. Give the agent no tool for what only a person may do.
• Return refusals to the model as text, and name what it missed.
• Make every write safe to repeat, and every "only once" a conditional update.
• Build the scoreboard before you scale out. Score against the experts' past decisions, on copies.
• Let an improver propose, a person approve, and code decide on held-out scores.
• Give the agent its own key and its own narrow API, and screen customer text before the model sees it.
• Save memory in code, never at the model's choice, and never the untrusted text.

Limits

• Layers 1 to 8 were built, tested and run live, on a 25-claim example queue. The AWS half of layer 7 (the real gateway, policy and memory) was written but not yet run.
• Layer 9 is a reviewed design, not built.
• The client is a made-up travel insurer. The numbers come from live runs on that example, not from a production claims team.

Sources

• LangGraph, durable execution and human-in-the-loop: https://langchain-ai.github.io/langgraph/
• Amazon Bedrock AgentCore (Runtime, Gateway, Memory): https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/what-is-bedrock-agentcore.html
• Cedar policy language: https://github.com/cedar-policy/cedar
• Model Context Protocol: https://github.com/modelcontextprotocol
• Every number in this article comes from our own live runs of the build described. None comes from these sources.