A correctness-first transaction core

The one invariant that matters

A transaction core has exactly one job it can never get wrong: money in equals money out, always. Everything else — throughput, latency, features — is negotiable. Correctness is not. So I built this POC backwards from a single invariant: for every committed transfer, the sum of all postings is zero, and no client request is ever applied more than once.

The hard part isn’t the happy path. It’s what happens when a client times out and retries, when two transfers touch the same account at the same instant, or when the process is killed after the database commits but before anything downstream runs. Those three failure modes are where real payment systems leak money, so they’re what I designed against first.

Architecture

The core commits one database transaction per request. That transaction writes the ledger postings, records the idempotency key, and appends an outbox row — atomically. Nothing is published to a queue inline. Async workers drain the outbox afterward and drive settlement, notifications, and downstream effects.

The shape is deliberate. Because the outbox row lives in the same transaction as the ledger write, there is no state where a transfer is committed but its downstream work is lost — and none where downstream work fires for a transfer that rolled back. The queue is a delivery mechanism, never a source of truth.

Idempotency without a race

The naïve approach — “check Redis for the key, if absent do the work, then write the key” — has a race: two concurrent retries can both miss the cache and both apply the transfer. I closed that window by storing the idempotency key inside the same transaction as the postings, with a unique constraint:

create table idempotency_keys (
  key           text primary key,
  request_hash  bytea not null,      -- guards against key reuse with a different body
  response      jsonb not null,      -- replayed verbatim on a duplicate
  created_at    timestamptz not null default now()
);

A duplicate request now fails the insert ... on conflict and we return the stored response verbatim. The database — not application logic — is the arbiter of “have I seen this before,” which means the guarantee holds under any amount of concurrency.

Postings are append-only; balances are derived

I never mutate a balance column. A transfer inserts paired postings (a debit and a credit) that sum to zero, and an account’s balance is the sum of its postings. This makes the ledger auditable by construction — you can always reconstruct how an account reached its current state — and it removes the update-in-place contention that plagues balance-column designs.

The tradeoff is read cost: computing a balance means aggregating postings. In the POC I keep a per-account materialized balance updated in the same transaction as an optimization, treated strictly as a cache of the append-only truth, never as the truth itself.

Where it would break, and why it doesn’t

I built a chaos harness that kills the process at randomized points and hammers endpoints with duplicate keys. The three classic leaks it targets:

• Retry storm → closed by the in-transaction idempotency key.
• Concurrent same-account transfers → closed by serializable isolation on the balance-check path; conflicts surface as serialization errors and are retried by the client-facing layer.
• Crash after commit, before enqueue → structurally impossible, because there is no separate enqueue step — the outbox relay is idempotent and simply resumes.

TODO(hung): drop in the chaos-harness results and the invariant-check query output here — the “sum of all postings across the ledger equals zero” assertion is the single most convincing artifact for this write-up.