All research Research note 02

Bounded duplicate suppression for agent effects

Fibric ResearchEffects6 min read

Distributed systems have spent decades reducing duplicate work. An agent's effects can land outside systems you control, where a sent message stays sent and an opened door stays open. This reference note describes the controls we use to suppress duplicates, and the boundaries those controls cannot erase.

Reference architectureControl objective, not an end-to-end guarantee

The comfortable version of the problem

Engineers who have built payment systems or message queues know the duplicate-delivery problem well. A producer sends a message. The network may drop the acknowledgement. The producer retries. Without care, the consumer processes the message twice. The standard mitigation is a stable deduplication key plus idempotent processing: the consumer remembers which keys it has seen and refuses to repeat work it can identify as the same intent.

True end-to-end "exactly-once delivery" is impossible in an asynchronous network with failures. Claims about exactly-once processing or effects always depend on a bounded transaction, a durable ledger, stable identity, and cooperation from the downstream system. Once an action crosses into a vendor API or physical device, acknowledgements can be ambiguous and the platform cannot prove the external world changed only once.

You cannot guarantee end-to-end exactly-once delivery or effect across every external system. You can suppress known duplicates within stated boundaries and make uncertainty visible for review.

Why the agent case is harder

The comfortable version assumes the deduplicated effect is something inside your own systems: a row written, a balance updated, a record marked processed. If something goes wrong, you can usually compensate, reverse the row, refund the balance, because you own both ends.

An agent breaks that assumption. Its effects are actions on the real world, and the real world does not have a rollback. Consider the ladder of reversibility an agent's actions actually live on:

The effect
What "undo" means in the real world
Write a status row
Fully reversible. Delete or correct the row.
Place an order on hold
Reversible, but the hold may have already changed someone's plan.
Send a customer a message
Not reversible. They have read it. A correction is a second message, not an undo.
Unlock a door
Not reversible. The moment of access already happened.

Once an effect is irreversible, compensation is not an undo. Prevention, bounded authority, downstream idempotency where available, and an explicit ambiguous-outcome path all matter. No single control turns an external effect into a reversible transaction.

The reference control pattern

The Fibric reference architecture treats duplicate suppression as a control objective, using two familiar primitives in the governed execution path. BearScope applies governed records and bounded controls on its supported live paths; this note is not a claim that every catalog connector or external device has the same semantics.

  • A stable idempotency key where identity is well defined. The key is derived from what the action is: this notification, for this order, about this event. A durable settled result can suppress later attempts with the same key, provided the first outcome is known.
  • Single-flight per entity within the governed runtime. Serializing execution reduces concurrent races against the same entity; it cannot prevent independent writers or commands issued outside that boundary.

Together these controls reduce duplicate risk, but they do not make blind retry safe. A timeout after an external side effect can leave an ambiguous result, so the connector contract must define whether it can query, reconcile, retry, compensate, or stop for review.

The boundary, and the receipt

The reference design separates proposal from governed execution and assigns a stable key on the deterministic side. After a crash, a settled local record can stop a repeat. If the crash occurs after an external effect but before settlement, the result remains ambiguous unless the downstream system supports its own idempotency or a reliable status lookup.

Supported execution paths should record the proposal, key, policy decision, attempt, downstream response, and reconciliation state. A record is evidence of what the system observed; it is not proof of an external effect when the downstream outcome is unknown.

What we are still working on

The hard frontier is keys for effects whose identity is fuzzy. "Notify the customer about a delay" is easy to key. "Adjust the setpoint to keep the room comfortable" is a continuous intent, not a discrete event, and reasonable people can disagree about when two adjustments are "the same" effect. We model these as single-flight intents per entity rather than keyed one-shot effects, but getting the boundary right, between a fresh decision and a duplicate of an old one, is open work, and we would rather be conservative and re-decide than risk a repeat.

Grounding

  1. Duplicate suppression in distributed messaging: at-least-once delivery plus stable identities and idempotent, deduplicated processing within a defined boundary.
  2. The impossibility of true exactly-once delivery in asynchronous networks, and why external effects require explicit ambiguity handling.
  3. Idempotency keys in payment and API design: deriving a stable key from the action's identity so retries are safe.
  4. Single-flight / request coalescing: collapsing concurrent duplicate work against the same entity into one.
Get started → All research

Related: The 657-message problem · Governed autonomy