As software agents powered by large AI models start wiring money, executing trades, and signing contracts on behalf of their human owners, a new type of risk is rapidly emerging: when the bot makes a bad call, who pays?
A group of researchers from Microsoft, Google DeepMind, Columbia University, and two AI startups, Virtuals Protocol and t54.ai, say that today’s AI safety toolkit doesn’t answer that question. Guardrails, red-teaming, and better model alignment might reduce the frequency of catastrophic mistakes-but they can’t guarantee outcomes in high‑stakes financial environments.
To fill that gap, they propose something closer to financial infrastructure than model tuning: an “Agentic Risk Standard,” a settlement‑layer framework that works like an insurance and escrow system for AI agents. The goal is to make sure end users are actually compensated when an agent misfires-whether that’s a misexecuted trade, a failure to deliver a paid service, or a direct monetary loss.
Why behavioral safety isn’t enough
Most AI safety research today zeroes in on model behavior: preventing harmful outputs, improving accuracy, and reducing hallucinations. Those are important, the authors note, but they only ever deliver probabilistic reliability. Even a model that’s “99.9% safe” isn’t nearly good enough when it’s driving real‑world financial decisions that can wipe out an account in seconds.
In regulated industries such as banking and insurance, risk is managed not only by trying to prevent failure, but also by building robust structures around what happens when failure inevitably occurs. Capital reserves, underwriting, claims processes, and clear liability rules all exist precisely because no system, human or machine, can be made perfectly reliable.
By contrast, most AI‑agent frameworks today assume that if the model is “safe enough,” the problem is solved. The researchers argue that this misses the core question for finance: outcome guarantees. If an AI-driven trading bot disobeys a user’s intent or mishandles an order, the user currently has little more than platform support channels to rely on-if that.
The core idea: an agentic settlement layer
The proposed Agentic Risk Standard introduces a settlement layer that sits between AI agents and the financial systems they interact with. Instead of treating every interaction as a simple direct transaction, it wraps those transactions in a structure that makes them insurable and auditable.
Key components of the framework include:
– Escrowed fees: Payments associated with an AI agent’s actions-like trading fees, execution commissions, or task payments-are not immediately and irrevocably disbursed. Instead, they are held in escrow until the underlying task is completed and validated.
– Underwriters and risk pools: Specialized underwriters assess the risks of a given type of agent behavior (for example, autonomous trading under certain constraints) and provide coverage. If the agent’s actions cause damage beyond a defined threshold, the underwriter pays out from a dedicated risk pool.
– Clear event definitions: The framework specifies what counts as a compensable event. Misexecution, non‑delivery of a promised service, or financial loss clearly linked to an agent’s deviation from agreed parameters would all be in scope.
– Settlement protocols: When things go wrong, the system doesn’t rely on ad‑hoc negotiation. Instead, predefined procedures determine how to investigate what happened, whether the loss qualifies for compensation, and how much is owed.
In other words, instead of trusting that agents will behave well, the system assumes they will sometimes fail, and builds structured remedies around that fact.
From “black box” AI to verifiable obligations
A central theme of the proposal is moving from “trust the AI” to “verify the outcome.” Traditional AI safety work aims to make the black box less dangerous; the Agentic Risk Standard tries to make the black box less consequential for users by tying it to external audit and enforcement mechanisms.
The authors emphasize that, in many real‑world scenarios, what users actually care about is not how the model arrived at a decision, but whether a promised outcome materialized:
– Did the funds arrive in the correct account?
– Was the trade executed within the specified limits?
– Did the agent respect the spending cap set by the user?
The settlement layer formalizes these expectations as obligations that can be checked after the fact. If an AI personal assistant agrees to pay a monthly bill up to $200 and instead authorizes a $2,000 payment, the event is not just a model error-it’s a breach of a contract-like constraint that the settlement framework can recognize, price, and compensate.
How escrow changes the incentives
Escrow is a key tool in the proposed design because it reshapes incentives for all parties involved.
By holding transaction fees or a portion of the notional value in escrow until successful completion, the framework:
– Aligns service providers with user outcomes: Platforms and agent providers get paid in full only when tasks complete correctly, which discourages shipping fragile agents into high‑risk contexts.
– Creates a buffer for restitution: If an error occurs and the user is entitled to compensation, funds in escrow can be redirected toward covering part or all of the loss, reducing the need for extended recovery processes.
– Enables tiered guarantees: Different levels of escrow and coverage can be offered depending on risk. A low‑stakes task could have minimal escrow and partial coverage, while high‑value trades or large transfers might require higher collateral and stricter underwriting.
This structure borrows elements from existing financial practices-letters of credit, performance bonds, and margin requirements-but applies them specifically to AI‑driven operations.
The role of underwriters in AI‑driven finance
Underwriters in this system would function similarly to their counterparts in traditional insurance and capital markets: they analyze risk profiles, set pricing, and decide what they are willing to cover.
For AI agents, that could mean:
– Evaluating the specific model architecture and training regime.
– Assessing the guardrails and constraints placed around the agent.
– Reviewing logs and historical performance data.
– Imposing limits on trade sizes, leverage, or asset classes the agent can touch.
– Demanding monitoring hooks and transparency for post‑incident analysis.
In return, users or platforms pay a premium-explicitly or implicitly bundled into fees-for coverage against defined categories of loss. This shifts the financial burden from individual users to a risk‑sharing pool, much like traditional insurance spreads risk across many policyholders.
Crucially, this also introduces a financial actor with skin in the game who cares deeply about agent reliability. If underwriters face large potential payouts, they exert pressure for more responsible agent design and deployment.
Why existing AI safety controls fall short in finance
The researchers’ criticism of current AI safety practice is not that it’s useless, but that it stops at the model layer. Techniques like:
– Reinforcement learning from human feedback (RLHF),
– Guardrail frameworks and content filters,
– Adversarial testing and red‑teaming,
– Policy‑based access control,
all reduce the chance of failure but can’t fully eliminate it. Moreover, they generally don’t define who is accountable when failure still occurs.
In financial markets, regulators care less about how elegant the risk controls look on paper and more about two things: who bears the loss and whether the system remains solvent under stress. Without an explicit mechanism for loss allocation and compensation, AI‑agent ecosystems look fragile from a regulatory and consumer‑protection perspective.
Implications for brokers, exchanges, and fintech apps
If such an Agentic Risk Standard were adopted, intermediaries that host or integrate AI agents-brokerages, exchanges, wallets, trading apps, payment platforms-would likely need to adjust their operating models.
Potential changes include:
– Tiered agent permissions: Only agents that meet specific underwriting and settlement-layer requirements could be granted access to high-risk operations such as derivatives, margin trading, or large cross‑border transfers.
– Standardized logging and transparency: Platforms would need to expose machine-readable logs of agent actions and system responses to support post‑incident investigations and settlements.
– User‑configurable risk profiles: End users might choose between low‑fee, low‑coverage modes and higher‑fee, insured modes, similar to how shipping, travel, or device insurance is currently offered.
– Regulatory reporting: Because guarantees and risk pools mirror existing financial instruments, regulators may require disclosures, capital buffers, and reporting akin to those imposed on insurers or clearinghouses.
This could slow down the rollout of fully autonomous financial agents-but it would also make them more palatable to institutional players and regulators.
How this could protect everyday users
For non‑expert users, the benefits of such a system are straightforward: mistakes become less existential.
Imagine a few common scenarios:
– A retail trader lets an AI agent rebalance their crypto portfolio monthly. Due to a bug, the agent sells into an illiquid market and slams the price down, crystallizing a far larger loss than the configured risk limits allowed. Under the proposed framework, that deviation from user‑specified constraints could trigger a settlement and partial or full compensation.
– A small business connects an AI agent to its corporate wallet to manage invoices. The agent misinterprets a supplier’s email and pays a fraudulent account. If the transaction flows through a settlement layer that recognizes misrouting or confirmed fraud as a covered event, the firm may not have to absorb the entire hit.
– A user subscribes to an “autonomous yield optimizer” that promises not to exceed a given risk score. If the agent allocates funds into a disallowed strategy and losses occur, the underwriter behind that product would be on the hook according to the agreed standard.
Instead of “use at your own risk,” AI financial services could move closer to “use with clear, priced, and enforceable guarantees.”
Open questions and potential challenges
The proposal also raises difficult implementation questions:
– Attribution of fault: When a loss happens, how do you attribute it-model error, platform bug, user misconfiguration, market conditions, or adversarial activity? The standard would need robust, tamper‑resistant logging and clear evidentiary rules.
– Moral hazard: If users know they are protected, they may be more willing to allow aggressive autonomy. Underwriters must design coverage terms that limit reckless delegation to agents.
– Pricing the risk of frontier models: AI models are evolving quickly, and their behavior in complex financial environments can be hard to predict. Estimating tail risk and setting premiums will be non‑trivial.
– Interoperability across jurisdictions: Financial regulation is fragmented. Designing a settlement layer that satisfies different legal systems and regulatory expectations is a major challenge.
– Standardization vs. innovation: A rigid standard could stifle innovation in agent architectures and business models. Too much flexibility, on the other hand, makes the “standard” toothless.
These issues suggest that the Agentic Risk Standard is less a finished product and more a blueprint for how AI and finance might be integrated responsibly.
Beyond trading: broader applications for agentic risk
While the research is framed around financial risk, the underlying concept-tying AI agents to an insurance‑like settlement layer-has wider relevance.
Potential domains include:
– Supply chain management: AI agents negotiating and scheduling shipments could be covered against misbookings, missed delivery windows, or unauthorized contract changes.
– Cloud operations and DevOps: Autonomous agents deploying infrastructure or managing workloads could carry guarantees against downtime or data loss beyond agreed SLAs.
– Healthcare billing and claims: Agents handling complex billing rules might be wrapped in similar risk standards to protect providers and patients from costly misclassifications or denials.
– Enterprise procurement: Agents authorized to place orders or sign low‑value contracts could be required to operate under capped, insured exposure.
In each case, the pattern is the same: instead of trusting AI behavior absolutely, the ecosystem formalizes what happens financially when behavior diverges from specification.
A step toward regulated AI autonomy in finance
As financial institutions experiment with AI co‑pilots and, increasingly, fully autonomous agents, the question is no longer whether machines will participate in markets, but under what rules.
The Agentic Risk Standard aims to bring familiar financial concepts-escrow, underwriting, settlement, and guarantees-into the AI era. It does not attempt to replace technical safety work, but to complement it with the legal, economic, and infrastructural tools that real‑world finance already uses to manage uncertainty.
If such frameworks gain traction, the next generation of AI financial agents may be built less like experimental apps and more like regulated financial products, with clear responsibilities, priced risk, and formal recourse when trades go wrong.