A junior analyst on a reporting team has access to her team’s shared drive. Not finance. Not HR. She opens the internal assistant and asks it to summarize the company payroll spreadsheet by department.

The agent resolves the document. The retrieval service fetches it using the platform’s service credential, because that is the only credential the retrieval service has ever had. The gateway forwards the content to the model provider over a federated token that expires in fifteen minutes. The model returns a tidy table of salary bands by level.

Every token in that chain was valid. Every hop ran over mutual TLS with a certificate our own PKI issued an hour earlier. The audit log records a successful, authenticated, authorized call with a complete request trace.

She now knows what her manager earns.

Nothing was breached. The system did exactly what it was built to do, which is why no detection rule you own will ever fire on it.

Norm Hardy wrote this up in 1988. A compiler ran with privileges its callers lacked, callers handed it an output filename, and one of them handed it the name of the billing file. The deputy was confused about whose authority it was acting under. That was the entire exploit, and we have spent nearly forty years building disciplines around it: capability passing, OAuth 2.0 Token Exchange, on-behalf-of flows, SPIFFE.

Then we put an inference gateway in front of a model provider, gave it a credential with broad reach, pointed an agent framework at it, and called the identity problem solved because we had removed the static key.

Workload identity federation authenticates your platform, not your user

I architect a clearing platform where Kubernetes and Kafka run multi-region across AWS and our own data centres behind roughly $4.5 trillion of daily settlement, a 99.99% availability target, SEC and CFTC oversight, and Regulation SCI. The first question an examiner asks about an AI feature is not which model you chose. It is who authorized this access, and show me the record.

So we did the obvious thing first and federated the gateway’s identity.

The trust relationship, constrained to one namespace and one service account:

resource "google_iam_workload_identity_pool_provider" "inference_gateway" {
  workload_identity_pool_id          = google_iam_workload_identity_pool.platform.workload_identity_pool_id
  workload_identity_pool_provider_id = "cluster-inference-gw"
  oidc {
    issuer_uri        = var.cluster_oidc_issuer
    allowed_audiences = ["sts.googleapis.com"]
  }
  attribute_mapping = {
    "google.subject"      = "assertion.sub"
    "attribute.namespace" = "assertion[\"kubernetes.io\"][\"namespace\"]"
  }
  attribute_condition = "attribute.namespace == \"ai-gateway\""
}

That is good engineering and I would defend it in any review. It is the same pattern we used to delete long-lived static credentials everywhere else: an enterprise PKI issuing short-lived certificates, mutual TLS between workloads, service identity derived from where the workload actually runs rather than from a secret somebody pasted into a config map years ago.

Read the assertion and notice what is not in it

The projected token carries an issuer, an audience, a namespace, a service account, an expiry. It carries no human being.

It cannot answer whether the analyst is allowed to read payroll, because that fact was never encoded in it and never could be. Federation proves the caller is your gateway. It says nothing about the requester standing behind four layers of application code.

If your authorization story ends at federation, you have built a deputy that holds powerful credentials and serves anyone who can reach its port. That is not a hardening gap. It is the 1988 bug with better TLS.

Your gateway holds the credentials, so it owns the authorization decision

The chain that actually holds has five links:

• User authentication at the edge against your identity provider, at the assurance level the data class demands.
• Identity propagation into the agent runtime, so the subject, groups and tenant survive the hop instead of being dropped at the first internal call.
• A scoped task token minted per request, naming the user, one action, one resource class, and an expiry measured in minutes.
• Gateway enforcement, where a policy decision point answers one question: is this user allowed to do this on this data, right now.
• Gateway to provider over federation, then inference.

Link four is the one teams skip, and they skip it for an understandable reason. Links one through three feel like authorization. You authenticated someone, you carried their identity, you minted a narrow token. It looks finished.

It is not finished, because nothing in links one through three ever said no.

A task token trimmed to the claims that matter:

{
  "iss": "https://idp.internal/agent-runtime",
  "aud": "inference-gateway",
  "sub": "u-48213",
  "groups": ["reporting-analysts", "all-staff"],
  "act": { "sub": "spiffe://platform/ns/agents/sa/assistant" },
  "task": {
    "action": "summarize",
    "resource_class": "shared-drive-document",
    "resource_ref": "doc:9f12c0",
    "exp_seconds": 120
  },
  "exp": 1780000120
}

The act claim is the honest part. It records that a workload is acting on behalf of a subject, which means the gateway can no longer pretend the workload is the subject. Two identities, both present, neither substituting for the other.

The token is the cheap part

Minting that token takes an afternoon. Deciding on it is the engineering.

A gateway policy that evaluates both identities:

package ai.gateway.authz
import future.keywords.if
import future.keywords.in
default decision := {"allow": false, "reason": "no matching entitlement"}
decision := {"allow": true, "reason": "entitlement match", "policy": policy_version} if {
	some grant in data.entitlements[input.subject.sub]
	grant.action == input.task.action
	grant.resource_class == input.task.resource_class
	input.task.exp_seconds <= 300
}
decision := {"allow": false, "reason": "task token exceeds max lifetime", "policy": policy_version} if {
	input.task.exp_seconds > 300
}
policy_version := "2026-06-02.1"

Hold on to that data.entitlements lookup. It is the thing I got wrong, and I will come back to it.

Authorization in the caller is a convention, and the second caller always arrives

Our first gateway did rate limiting, token accounting and audit. Authorization lived in the chat application, which checked entitlements before calling out. Defensible design, cleanly reviewed, shipped.

Four months later a second team integrated against the gateway directly. Batch summarization for a reporting workflow, legitimate business need, approved network path, their own service account.

They inherited none of the entitlement checks, because those checks lived in an application they had never opened.

We caught it during a tabletop exercise rather than in production, and I am clear-eyed about how much of that was luck rather than design. The structural lesson is the one worth carrying: if a control lives in one caller, it is not a control. It is a convention, and conventions expire the moment somebody finds a second way in.

A semantic cache keyed on the prompt is an authorization bypass with a hit rate

The second failure still irritates me, because the design review had three experienced people in it and we all nodded.

We put a semantic cache in front of the model to cut cost and latency. The key was a hash of the normalized prompt plus the model identifier. Hit rates in the high teens on repeated analytical questions, which on this workload is real money (illustrative — the production figures are not mine to publish).

Identity was not in the key.

Two users asking a similar question about a document only one of them could read would be served the same cached answer. The retrieval step that would have enforced anything was never reached, because the cache answered first and the cache does not know what a clearance is. We found it by reading the cache implementation during a security review, not by a test, and not before it had run against non-regulated data for weeks.

Any cache in front of inference must be keyed by the authorization context, not the prompt. If that collapses your hit rate, the hit rate was quietly telling you how many of your users should not have been able to ask that question.

Content filters are a detective control wearing a preventive control’s clothes

The seductive fix is an interceptor. Scan the prompt, scan the response, block anything that looks like compensation data.

I have watched three teams reach for this and I will keep arguing against it. It fails on paraphrase. It fails on aggregation, where no individual chunk is sensitive and the summary is. It fails on the examiner’s actual question, because “the filter did not match” is not an authorization decision and cannot be mapped to a control.

Put the decision where it can be recorded with the inputs that produced it. Our gateway emits one record per call: subject, groups as resolved at decision time, action, resource class, policy version, allow or deny. That is the same evidence discipline we already apply to Kubernetes admission control, aimed at a new control point.

Pulling the deny record for a specific call:

aws logs filter-log-events \
  --log-group-name /platform/ai-gateway/decisions \
  --filter-pattern '{ $.subject = "u-48213" && $.allow = false }' \
  --start-time "$(date -d '24 hours ago' +%s)000" \
  | jq -r '.events[].message | fromjson
           | [.ts, .subject, .task.action, .task.resource_class, .reason, .policy] | @tsv'

If you cannot produce that line for an arbitrary request from last Tuesday, you do not have an authorization layer. You have logging.

Mirroring entitlements into policy was my mistake; fetch as the user instead

Back to data.entitlements.

We replicated entitlement facts into the policy layer so the gateway could decide without a network round trip. Fast, deterministic, easy to test. Also stale by Thursday. Somebody changes teams, a document gets reclassified, a group gets nested inside another group, and your policy data is confidently, auditably wrong.

The design I would build from day one: the retrieval path fetches as the user. Exchange the task token for a downstream credential bound to that subject and let the system of record refuse. The gateway still decides whether the action is permitted and still writes the decision record. The data store decides whether this person may read this object, because it is the only component that actually knows.

Two decisions, each made by the thing that holds the truth. It costs latency on every retrieval. It survives reorganizations, which replicated ACLs do not.

The weaker version of this argument is that you should do both. I used to say that. I no longer do, because every team that holds a mirrored copy eventually trusts the copy under deadline pressure, and then the round trip exists only on the paths nobody optimized.

Federation is table stakes; the open question is whether anything in your stack knows who is asking

The confused deputy was never fixed. It was managed, by engineers who learned to distrust privileged intermediaries that follow instructions well.

Agents are the most instruction-following intermediaries we have ever put into production, and we have handed them credentials that read across the enterprise. They will do precisely what they are asked, with whatever authority is available to them, and they will leave a clean audit trail while doing it.

So federate the workloads. Delete the static keys. Then go and trace one real request end to end and find out which component, if any, could have said no to the analyst.

In most architectures I review, the answer is none of them, and the diagram does not show it.