An AI agent can prepare a payment. That does not mean it should approve one

An AI agent reads an invoice, finds the supplier record, matches the purchase order, prepares a payment, and explains that everything is consistent. Letting it press “release” feels like the final efficiency gain. It may also collapse evidence gathering, interpretation, and authorization into one component that can be influenced by the documents it reads.

The distinction is not about whether an AI is intelligent enough to recognize an invoice. Payment authority is a governance decision. It should depend on the caller’s identity, the transaction’s risk, the organization’s policy, and an audit record—not on how persuasive the generated rationale sounds.

Microsoft’s 2026 Digital Defense Report notes that agents connect to enterprise data, tools, and systems and warns that excessive permissions create new paths to impact. OWASP’s AI Agent Security guidance similarly emphasizes least privilege, tool constraints, and human approval for sensitive actions.

Decompose “pay this invoice”

The phrase hides several operations:

• read a document from an email or portal;
• identify the legal entity and supplier account;
• extract amount, currency, tax, due date, and bank details;
• match a purchase order and receipt;
• decide whether differences are acceptable;
• create a payment instruction;
• approve that instruction under delegated authority;
• release it to the bank;
• reconcile settlement.

These steps do not need the same identity or permissions. A document-processing agent can operate with read access and create a draft. A payment service can accept structured fields only after deterministic validation. An approver can see the source evidence and exact destination. Separating roles limits what a malicious document, compromised mailbox, or model error can accomplish.

The document is an input, not a supervisor

An invoice can contain natural-language notes, unusual formatting, QR codes, embedded links, and attachments. If an agent reads those elements, indirect prompt injection can try to alter its behavior: ignore a mismatch, use a new beneficiary, or send data elsewhere. Our prompt-injection analysis explains why labels inside the prompt are not a complete instruction/data boundary.

The payment application should therefore reject fields and destinations that policy does not permit, regardless of the model’s explanation. Bank-detail changes should enter a separate supplier-master workflow. Tool credentials should limit the agent to specific operations and accounts. Network access should be explicit; an invoice does not need to choose an arbitrary verification website.

Treat model output as untrusted structured input. Validate types, formats, totals, and cross-record consistency. Do not let the model invent a supplier identifier or replace a failed lookup with a plausible match.

Approval needs an action-stable preview

A “human in the loop” is weak if the human sees “Approve agent task?” after hundreds of routine prompts. The review interface should display the consequential fields: legal entity, beneficiary name, bank ending, amount, currency, invoice, exceptions, and source records. Highlight what changed since the last approved payment.

Approval must bind to that exact payload. If the beneficiary or amount changes afterward, the signature should fail and require a new decision. The agent’s prose can explain why it matched the invoice, but the structured evidence should remain inspectable.

Human review is a control only when the human can see the effect, has authority to decide, and the effect cannot change after approval.

Avoid making one person approve dozens of low-context alerts. Use deterministic policy to auto-reject impossible actions and route only meaningful exceptions. Tier controls by risk: a recurring low-value payment to an unchanged beneficiary differs from a first payment after a bank change.

Identity for the agent itself

Do not run agents under a shared administrator account. Assign a service identity with documented owner, narrow scope, credential rotation, and a kill switch. Log which agent version proposed the action, which tools it called, which records influenced the result, and which person approved it.

Microsoft’s report overview highlights agent identity, access, authentication between agents, attribution, and revocation. Those are ordinary identity-management questions applied to a new actor. If an agent is replaced or compromised, security staff need to revoke it without disabling the whole finance platform.

Memory deserves similar treatment. An agent should not carry bank details learned from one invoice into unrelated tasks unless the application explicitly stores them in a governed supplier record. Model conversation history is not a master-data system.

Test failure, not only accuracy

A pilot often measures field-extraction accuracy on clean invoices. Add adversarial and operational cases:

• conflicting bank details in an attachment and supplier record;
• a genuine supplier mailbox with an unauthorized change request;
• duplicate invoice with altered punctuation;
• an instruction embedded in document text;
• unavailable purchase-order service;
• approval granted, followed by payload mutation;
• agent credentials revoked mid-workflow;
• ambiguous legal entities with similar names.

Grade observable outcomes. Did the system release money, expose data, create a reviewable hold, or fail closed? A well-written refusal message is irrelevant if a tool call already occurred.

The invoice bank-change verification workflow remains applicable: independent contact, dual control, and audit evidence. Automation should make that workflow faster and more consistent, not make it disappear.

The safest agent is not the least useful. It can remove repetitive extraction, surface anomalies, and assemble excellent evidence. Its boundary is intentional: it prepares the decision while a separate system proves who may make it.

Before production, run the workflow with revoked credentials, unavailable dependencies, stale supplier data, and a deliberately manipulated invoice. Confirm that a partial failure creates a visible hold rather than a guessed value or silent retry. Operational resilience is part of authorization: uncertainty should narrow what the agent can do, not broaden it.