Deploying AI agents in regulated industries changes five things: the agent gets a scoped identity instead of broad service credentials; consequential actions route through a human approval gate; every action lands in an append-only audit trail; the system runs inside your compliance boundary instead of a vendor's cloud; and the team writes compliance requirements before picking a model. Teams that design for these five from the first commit pass their security review. Teams that bolt them on at month four fail it, at their own expense.
I build AI systems for government and utility clients: a document-audit agent for a federal-sector program, and a requirements tool for a utility that went through NERC and state-commission review before cutover. A reviewer made us prove each of the five.
Why agent pilots fail the review
The industry numbers describe a boundary problem. MIT's Project NANDA found 95 percent of organizations deploying generative AI saw no measurable return, and traced the failures to data readiness and governance gaps. Databricks' Opsin Labs report found 60 percent of enterprise AI agents run over-permissioned. Gartner projects 40 percent of agentic AI projects will be canceled by 2027, citing cost, unclear value, and weak risk controls.
A demo never meets a reviewer. The pilot works in a sandbox with a shared API key and an admin token, and everyone calls it a success. Then the security team asks who the agent authenticates as, what it can write to, and where the record of its actions lives. In a regulated environment those questions have required answers, and retrofitting them costs more than designing for them.
The five changes
1. A scoped identity replaces the shared service account
An agent in production is a principal. It needs its own identity, its own credential lifecycle, and a permission set matched to its task, the same discipline you apply to a new hire. Over-permissioning is the norm because a broad role makes the demo work faster, and the Databricks finding puts a number on how often teams leave it that way.
The pattern that survives review is default-closed. Every capability an agent could reach is off until a named configuration turns it on, and the deploy list is the inventory a reviewer reads. On one of our systems, the fastest audit check is a probe: an endpoint that answers 401 has its gate live, an endpoint that answers 400 is running logic before auth, and the second one is a finding. Cheap tests like that only exist when someone designed the gates as code.
The reviewer will ask: "Show me every action this agent can take, and show me where each one was granted."
2. Consequential actions route through human approval
Human-in-the-loop AI governance means tiered autonomy, not a person watching every token. Reads and drafts run free. Writes that touch money, records, production, or the outside world stop in an approval queue where a person with the right role sees the proposed action, the evidence behind it, and an approve or reject control. In our own operating system, the queue is the product surface where the team meets its agents.
The gate earns its keep on the cases you did not predict. One of our automated reviewers blocked a deletion that every technical signal said was safe, because it found a meeting record from three weeks earlier showing the affected client relationship was active. A person confirmed, we split the deletion into its own decision, and the system deleted nothing a stakeholder still needed. That save cost one approval click. The incident it prevented would have cost a client conversation.
The reviewer will ask: "Which actions can this agent complete with no human in the path, and who decided that list?"
3. An append-only audit trail
Regulators ask narrow questions years after the fact: what did the system know on this date, who approved this action, what did the model return before someone edited it. Logs that rotate away in 30 days cannot answer, and neither can records the application can overwrite. We store agent activity as append-only history: the input, the output, the identity, the approval, and the timestamp, with no update path.
Append-only also settles measurement arguments. On the utility engagement, pre-cutover review included proving what the system did in specific past sessions. Because history was immutable, the answer was a query, not a reconstruction from fallible memory.
The reviewer will ask: "Reconstruct last Tuesday. Show me every action, its approver, and its evidence."
4. The boundary comes first: your tenancy, your controls
Regulated data does not travel to a vendor's stack to meet a model. The system comes to the data: your cloud account, your identity provider, your key management, your audit logging, with model access through services already inside your authorization boundary. On the federal-sector program, the agent runs against a managed knowledge base inside the client's own cloud, under the client's IAM, and the data never crosses the boundary. The utility build enforces tenant isolation in the database itself through row-level security, so a query cannot read another tenant's rows even when application code has a bug.
Architecture inside the boundary is the difference between a security review that takes weeks and one that never converges. It is also the reason our approach is embedded engineering rather than a hosted platform: the deliverable has to live where your auditors can reach it.
5. Requirements written before the model is picked
The regulatory floor is rising on a schedule. The EU AI Act became fully applicable on August 2, 2026, with penalties up to 35 million euros or 7 percent of global turnover, and member states now run mandatory AI regulatory sandboxes. The U.S. Treasury published an AI risk-management framework for financial services in February 2026. NERC and state commissions already treat AI components as part of the reviewed system, which is how our utility work entered a pre-cutover remediation sweep with named P0 and P1 findings and exit criteria.
Write the compliance requirements into the engagement before model selection: data classification, residency, retention, the approval tiers from change two, and the evidence the regulator will want. A team that starts from the framework treats the review as a checklist. A team that starts from the model treats it as a surprise.
What makes an AI agent FedRAMP compliant?
No FedRAMP category exists for AI agents. Compliance is inherited: the agent must run on services inside an existing FedRAMP authorization boundary, and every component it touches, model endpoints, vector stores, queues, logging, must hold the same authorization level the data requires. An agent that calls one convenient external API from inside a FedRAMP High workload has broken the boundary, and the assessor will treat it that way. The practical test for FedRAMP compliant AI agents: list every network egress from the agent's runtime, and justify each one against the authorization package. Agencies answer to fedramp.gov's marketplace on what is authorized. Your architecture answers for everything else.
What compliance requirements apply to AI agents?
The frameworks differ in vocabulary and agree on substance. FedRAMP wants the authorization boundary held. NERC CIP wants access control, change management, and evidence for systems that touch critical infrastructure. HIPAA's Technical Safeguards want access controls, audit controls, and integrity for anything reading patient data. The EU AI Act adds risk classification, documented human oversight, and post-market monitoring. All four reduce, for an agent, to the five changes above: scoped identity, human approval gates, immutable audit, boundary residency, and requirements written first.
The trade
The five changes cost speed at the start. Scoped identities are slower to set up than an admin token. Approval queues add a person to paths that pure automation would clear in seconds. Append-only storage grows.
Take the trade. The alternative is joining the 40 percent whose projects die at the risk-control stage after the demo already succeeded. In regulated industries the review is the path to production, and systems designed for it are the ones still running a year later.
FAQ
Are AI agents allowed in FedRAMP and NERC environments? Yes, as components of the reviewed system. Both frameworks evaluate the system's controls, not the presence of AI. The agent must inherit the boundary, the access model, and the evidence obligations of the environment it runs in.
Does every agent action need human approval? No. Tier by consequence: automate reads and reversible drafts, gate writes that touch records, money, production, or external parties. The reviewer cares that the tiers were decided by a person and that the gates hold under test.
Can we run agents fully inside our own tenancy? Yes, and in regulated environments you should require it. Managed model services inside your cloud boundary, your IAM in front, your logging behind, no data egress to a vendor stack.
