AI agent action verification matters when your bot says a refund is complete before the payment service has confirmed it. The tool call timed out, but the model turned “I tried” into “I did”. The customer now has a promise your application can’t support.
The first fix is to stop treating tool execution as a simple success-or-failure story. Some outcomes are unknown. We’ll record that uncertainty, then make the final wording depend on the recorded result.
Create an operation before calling the provider
Give the refund request an internal operation ID. Store the order, permitted amount, customer confirmation and provider correlation fields. Validate the amount and the operator’s authority before starting the action.
The model can propose an action, but it doesn’t establish eligibility. The refund service should check what was paid, what has already been refunded and which policy applies.
Use an operation key that stays stable across retries of the same business intent. A newly generated key on every attempt makes provider-side idempotency ineffective for that intent.
Give unknown its own state
Useful states include requested, submitting, pending, succeeded, failed and unknown. The state should describe what the application knows, not what it hopes happened.
A connection timeout after submission belongs in unknown until you have more evidence. Stripe’s error-handling documentation describes why network outcomes and safe retries need special treatment. Provider contracts differ, so use the exact one your integration offers. Stripe error handling.
Don’t convert unknown into failed just to simplify a dashboard. That conversion can trigger another refund while the first is still processing.
Bind the customer reply to the action state
The example below simulates a provider accepting a refund and then losing the response. Reconciliation later finds the successful result. The response builder cannot say “completed” while the operation is unknown.
import assert from 'node:assert/strict';
const provider = new Map();
const operation = { id:'refund-1', state:'requested', amount:1200 };
async function submit(op) {
op.state = 'submitting';
provider.set(op.id, { state:'succeeded', amount:op.amount });
throw new Error('Response lost after provider acceptance');
}
function reply(op) {
if (op.state === 'succeeded') return 'Your refund is complete.';
if (op.state === 'failed') return 'The refund could not be completed.';
return 'I am checking the refund status before confirming the outcome.';
}
try { await submit(operation); } catch { operation.state = 'unknown'; }
assert.equal(operation.state, 'unknown');
assert.equal(reply(operation).includes('is complete.'), false);
const evidence = provider.get(operation.id);
assert.equal(evidence.amount, operation.amount);
operation.state = evidence.state;
assert.equal(reply(operation), 'Your refund is complete.');
assert.equal(provider.size, 1);
console.log('PASS: lost response stays unknown until original operation is reconciled');
The important result is the change in wording as evidence changes. We haven’t proved that a model always follows a prompt. We’ve removed the model’s discretion over the factual completion statement.
You can still use a model to explain the next step politely. Keep the action outcome as a structured fact and validate any generated completion claims before sending.
Reconcile against the original operation
Store the provider reference when you receive it. If the response disappears before that reference arrives, use supported idempotency or lookup facilities tied to the original operation. Avoid searching loosely by amount and time and assuming the first match is yours.
A reconciliation worker should record what it queried, when and what evidence it received. An unresolved result stays unresolved. A repeated lookup that returns no record may still require waiting under the provider’s consistency and processing rules.
Once the provider establishes the outcome, update the action ledger and any customer-facing projection together or through a recoverable event. Don’t leave a successful ledger entry paired with an old “failed” support label indefinitely.
Don’t confuse accepted with completed
Some providers return a refund object while processing continues. A successful HTTP response may establish that a request exists, not that funds have reached the customer. Model the provider’s actual lifecycle.
Your response might say the refund was requested, then later explain that it completed. Keep those messages tied to state transitions so reprocessing a webhook doesn’t repeat the same announcement.
For the business record, store requested amount and confirmed result separately. A partial refund should never be described as a full refund merely because a tool returned a generic success flag.
Make the audit trail useful
Record the initiating conversation, authenticated actor, approval evidence, operation key and provider result. Avoid putting payment secrets or unnecessary personal content into logs.
Keep the customer’s request distinct from the model’s interpretation. If someone disputes the amount, you need to know what was asked, what was authorised and what was submitted.
An operator should be able to inspect unknown operations without manually replaying tool calls. Build that investigation path before enabling automatic retries for actions that change business records.
Test the missing response
Simulate a provider that fails before processing, accepts then disconnects, returns a pending state and completes through a later event. Check both the ledger transition and the exact factual claim sent to the customer.
Also retry the same business operation and verify that it keeps the same operation key. Then try reusing the key with a different amount; your application should reject that mismatch.
Start by making unknown visible in your action model. Once the system can admit what it doesn’t yet know, you can make the agent’s answer both more useful and more trustworthy.
Run the example and adapt it
Save the code as 10-truthful-agent-action-results.mjs and run it with Node.js 24 or later:
node 10-truthful-agent-action-results.mjs
The code above includes assertions you can run locally. It tests the local decision logic with controlled inputs; it doesn’t connect to a live provider or establish production performance. Keep that boundary when adapting it to your application.
