surehand
All articlesRisks

Retries, idempotency and side effects: why a retried agent pays twice

Reference5 min readSurehand

Your agent calls the ERP to pay an invoice. The call times out. Did the payment go through? You do not know. The agent does not know either. So it tries again. If the first call did land, you just paid the supplier twice. Idempotency is the property that makes the second call harmless. Most agent setups do not have it on the actions that matter.

In one sentence: a timeout means "unknown", not "failed", and an agent that retries an unknown payment needs the receiving system to recognise the retry.

Where it comes from

Distributed systems engineers have fought this for decades. Microsoft's retry guidance gives the plain case. A service "might receive the request, process the request successfully, but fail to send a response". The retry logic "might re-send the request, assuming that the first request wasn't received"3. Its rule: if the operation is idempotent, "it's inherently safe to retry. Otherwise, retries could cause the operation to be executed more than once, with unintended side effects"3.

Amazon's answer is a key the caller supplies. Its preferred approach is "a unique caller-provided client request identifier" in the API contract1. Requests with the same identifier "can be considered duplicate requests and can be dealt with accordingly"1.

Payment companies built it into their APIs. Stripe "supports idempotency for safely retrying requests without accidentally performing the same operation twice"2. It saves the result of the first request for a given key. "Subsequent requests with the same key return the same result, including 500 errors"2.

How it works

An idempotency key is a label on the business action. The receiving system keeps a short memory of labels it has seen. A repeat label gets the original answer back. No second action.

Take one AP invoice. The numbers are illustrative.

StepWithout a keyWith a key
Agent sends "pay INV 7731, 2,400.00"Payment createdPayment created, key stored
Response lost in a timeoutAgent sees an errorAgent sees an error
Agent retriesSecond payment createdERP finds the key, returns the first payment
Month-endSupplier has 4,800.00. AP chases a refundOne payment. Nothing to chase

Two details decide whether this works. Stripe's docs name both.

The key must match the intent. Stripe compares incoming parameters with the original request "and errors if they're not the same to prevent accidental misuse"2. A retry with a different amount is not a retry. It should be refused, not silently merged.

The memory is not forever. Stripe says keys can be removed "after they're at least 24 hours old"2. A retry after that window is treated as new. An agent that resumes a paused run the next morning can land outside it.

Why agents make it worse

Classic software retries the same call, the same way, a fixed number of times. An agent does not.

The model decides to retry. It sees an error message and tries again. You did not write that loop. It may not stop at three.

It may rephrase. The second attempt has different parameters. A memo line is reworded, the date format changes. A strict key check sees a new request.

It may pick another tool. The payment API timed out, so the agent tries the bulk payment import. OWASP's excessive agency entry is about exactly this: more functionality than the job needs5. Each extra path to the same action is another way to duplicate it.

Your ERP's guard may not cover it. NetSuite lets an administrator warn or block on duplicate document numbers. The middle setting is "Warn (UI only)". Under it, imports that use Advanced Numbering go through with duplicate numbers: NetSuite "imports all transactions"4. Only "Warn and Block" stops imported duplicates4. NetSuite's own help adds a line every agent builder should read: "If you click Submit more than one time, multiple transactions may be created"4. An agent is a user that can click Submit very fast.

What it is good at, and what it is not

Good at. Turning "did it go through?" into a question the system answers. With a key on every write, a retry is always safe. Your agent can be aggressive about recovering from network errors without creating money problems.

Not good at. Catching duplicates the agent creates on purpose. The same invoice arrives twice, as a PDF and as an email body. The agent reads both as new. Each gets its own key. Idempotency protects the call. It does not know two documents are one bill. That is duplicate invoice detection, a matching problem, and the agent needs it as a rule.

Not a fix for missing keys upstream. If your ERP's API does not accept an idempotency key, you cannot add one from outside. You need a layer in front that remembers what it already sent.

What to check

  1. /01

    Which write actions accept an idempotency key? List every tool that creates or changes money, records or messages.

  2. /02

    Who makes the key? It should come from the business action (invoice number plus supplier plus amount), not a random value per call.

  3. /03

    What happens when the agent retries with different parameters? You want an error, not a second action.

  4. /04

    How long does the receiving system remember keys? Can a paused run resume outside that window?

  5. /05

    What is your ERP's duplicate setting, and does it apply to API and import paths?

  6. /06

    Can the agent reach the same action through two tools? If yes, do both share one key?

Where it is going

Agent frameworks are adding durable execution: a run that pauses, survives a restart and resumes where it stopped. That is progress, and it moves the problem. A resumed run replays its last step. Our view: within two years buyers will ask "is every write idempotent?" in the same breath as "is there a human in the loop?" A hold does not help if the approved payment is sent twice.

Gatehouse fit

Gatehouse checks every action against the rules before it runs and writes each attempt to the record. A retried payment shows up as a second attempt your approver and auditor can see, not a line buried in an application log. It does not replace idempotency keys in your ERP's API. The two work together. The composite invoice matching deployment shows where the check sits in an AP run.

At a glance

CategoryRisks
Also calledDuplicate side effects, at-least-once delivery, double submission
Borrowed fromDistributed systems. Payment API design
Key docsAmazon Builders' Library. Stripe idempotent requests. Azure retry pattern. NetSuite duplicate number warnings
Typical ownerIT or the integration owner builds keys. AP owns the duplicate-invoice rule
The one testTime out a payment call in test. Does the retry create a second payment?

Sources

  1. [1]Making retries safe with idempotent APIs, Amazon Builders' Libraryaws.amazon.com In text
  2. [2]Idempotent requests, Stripe API referencedocs.stripe.com In text
  3. [3]Retry pattern, Azure Architecture Center, Microsoft Learnlearn.microsoft.com In text
  4. [4]Duplicate Number Warnings, NetSuite Applications Suite, Oracle Help Centerdocs.oracle.com In text
  5. [5]LLM06:2025 Excessive Agency, OWASP Top 10 for LLM Applicationsgenai.owasp.org In text

Read next

[ your next step ]

Bring us the queue nobody wants.

One process, studied in writing. You keep the document, whatever it says.

support@surehand.io