Preventing WAL Bloat from Inactive Slots

A single inactive replication slot whose consumer has died holds restart_lsn frozen and forces PostgreSQL to retain every WAL segment behind it, so an idle slot — not a busy one — is the most common reason a healthy primary silently fills pg_wal and then refuses writes. The defence is bounded failure: cap how much WAL any one slot may pin, let PostgreSQL sacrifice a hopeless slot instead of the whole primary, and reap orphaned slots before they ever reach the cap.

This page is about the safety controls around a stalled slot specifically — the wal_status state machine from reserved to lost, max_slot_wal_keep_size as the circuit breaker, wal_sender_timeout as the reaper of dead connections, and the procedure for detecting and safely dropping an orphan. It is not about aggregate WAL velocity across all consumers; measuring the publisher’s generation rate and forecasting headroom is covered in monitoring WAL generation and retention. All behavior is validated against PostgreSQL 13 through 17, where wal_status and the invalidation cap were introduced and refined.

wal_status Transition Semantics #

The wal_status column in pg_replication_slots is the single field that tells you whether a lagging slot is merely behind or already doomed. Each value is a precise statement about the WAL the slot still needs versus the WAL PostgreSQL is still keeping, and the transitions are driven by two GUCs plus the checkpoint cycle. The table fixes the exact meaning, targeting PostgreSQL 13–17.

The operational insight is that unreserved is a warning, not a death certificate. A slot in unreserved still holds its WAL until the next checkpoint physically recycles it, so a consumer that reconnects and confirms within that window walks back to reserved. Once wal_status = lost, the segments are gone and there is no catch-up — the downstream must be re-initialised. Setting max_slot_wal_keep_size is therefore choosing, in advance, the point at which PostgreSQL will protect the primary by sacrificing a consumer; leaving it at the default -1 means the primary will instead fill its disk and stop accepting writes. The broader slot lifecycle this fits inside is covered in the parent replication slot types guide.

Diagnostic Patterns #

The pre-outage signature is unmistakable once you know the shape: a slot with active = false, a restart_lsn that has not moved, and retained_wal climbing. This is the query to run on every publisher on a short interval:

-- Inactive slots ranked by the WAL they are pinning. The top row is your risk.
SELECT slot_name,
       active,
       wal_status,                                   -- reserved | extended | unreserved | lost
       pg_size_pretty(
         pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained_wal,
       coalesce(active_pid::text, '—')               AS walsender_pid
FROM pg_replication_slots
WHERE slot_type = 'logical'
ORDER BY pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) DESC;

Escalate by state, not just by size. Any slot showing active = false for more than 300 s on a production pipeline is an incident — the consumer is gone and retention is only growing. A wal_status of extended is a yellow flag; unreserved should page immediately because the next checkpoint may finish the slot; lost means the downstream is already broken and must be reseeded. Find how long a slot has been idle by joining against the walsender view — an orphan has no matching row at all:

-- Slots with no live walsender are orphans: consumer never reconnected.
SELECT s.slot_name, s.active, s.wal_status,
       pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), s.restart_lsn)) AS retained_wal
FROM pg_replication_slots s
LEFT JOIN pg_stat_replication r ON r.pid = s.active_pid
WHERE s.slot_type = 'logical'
  AND s.active = false
  AND r.pid IS NULL;

Confirm the checkpoint horizon so you know how much time an unreserved slot really has — the danger window closes at the next checkpoint, whose cadence is governed by checkpoint_timeout and max_wal_size:

SELECT name, setting, unit FROM pg_settings
WHERE name IN ('checkpoint_timeout','max_wal_size','max_slot_wal_keep_size','wal_sender_timeout');

Safe Deployment Sequence #

Installing the bounded-failure controls is a reload-only change, so it is zero-downtime and reversible. Apply it before you need it — the cap only protects slots that cross it after it is set.

1. Set the invalidation cap. Choose max_slot_wal_keep_size as the most WAL you are willing to let one dead slot pin before PostgreSQL invalidates it. This is the line between extended and unreserved:

   ALTER SYSTEM SET max_slot_wal_keep_size = '40GB';  -- bounded failure, never -1 in prod
   SELECT pg_reload_conf();
   

2. Tighten the dead-connection reaper. A shorter wal_sender_timeout frees a slot from a hung consumer faster so a reaper can act, but must exceed your consumer’s keepalive interval or healthy streams will be killed:

   ALTER SYSTEM SET wal_sender_timeout = '30s';   -- must be > consumer keepalive
   SELECT pg_reload_conf();
   

3. Safely drop a confirmed orphan. Only drop a slot you have proven is abandoned — a live consumer will error with replication slot is active. Verify it is inactive with no walsender, then release it:

   -- Guard: refuses to run if the slot is still active.
   SELECT pg_drop_replication_slot('dead_etl_slot')
   WHERE EXISTS (
     SELECT 1 FROM pg_replication_slots
     WHERE slot_name = 'dead_etl_slot' AND active = false
   );
   

   Dropping releases the pinned WAL immediately and the next checkpoint recycles it. Record in the runbook which downstream owned the slot, because reattaching it later requires a fresh snapshot — the WAL it needed is gone.

4. Verify recovery. Confirm retained_wal for the remaining slots is falling and no slot sits in unreserved:

   SELECT slot_name, wal_status,
          pg_size_pretty(pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)) AS retained_wal
   FROM pg_replication_slots WHERE slot_type = 'logical';
   

5. Revert. Both GUCs are reload-scoped: ALTER SYSTEM RESET max_slot_wal_keep_size; and RESET wal_sender_timeout; followed by pg_reload_conf() restore the prior behavior with no restart. Never revert the cap to -1 on a production primary — that removes the only guardrail between a stalled consumer and a full disk.

Pipeline Integration #

The reliable pattern is an automated reaper that distinguishes a temporarily disconnected consumer from a permanently orphaned slot and only drops the latter, after a grace period, with an alert. A blind cron that drops any inactive slot will eventually kill a consumer that was mid-reconnect.

# Reaps orphaned logical slots after a grace period; never touches active ones.
import time, psycopg2

GRACE_SECONDS      = 900               # 15 min disconnected before a slot is "orphaned"
KEEP_BYTES_ALERT   = 30 * 1024**3      # page when retained WAL nears the cap

def reap_orphans(dsn, seen: dict):
    conn = psycopg2.connect(dsn); conn.autocommit = True
    cur = conn.cursor()
    cur.execute("""
        SELECT slot_name, active, wal_status,
               pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn) AS retained
        FROM pg_replication_slots WHERE slot_type = 'logical'
    """)
    now = time.monotonic()
    for slot, active, status, retained in cur.fetchall():
        if active:
            seen.pop(slot, None)                 # healthy — reset any grace timer
            continue
        first_seen = seen.setdefault(slot, now)  # start the grace clock on first sighting
        if retained > KEEP_BYTES_ALERT or status in ("unreserved", "lost"):
            alert(f"slot {slot} status={status} retained={retained} bytes")
        if status == "lost":
            drop_and_reseed(cur, slot)           # WAL already gone — reseed downstream
        elif now - first_seen > GRACE_SECONDS:
            # Confirmed orphan: safe to drop, releasing pinned WAL.
            cur.execute("SELECT pg_drop_replication_slot(%s)", (slot,))
            seen.pop(slot, None)

Two rules keep the reaper safe. First, key the grace period on continuous inactivity — reset the timer the instant a slot reports active = true again, so a consumer that flaps is never dropped mid-flap. Second, treat wal_status = lost as a signal to trigger the downstream reseed workflow, not merely to drop the slot: the data gap already exists, and dropping without reseeding produces a silent hole in the change stream. Export the per-slot retained and wal_status as labelled gauges so the invalidation risk is visible before the reaper acts, using the collector described in the monitoring and alerting guide.

Failover handling. After a promotion, slots do not follow the old primary before PostgreSQL 17’s failover slots, so the new primary starts with none of the orphans — but it also starts with no retention protection unless max_slot_wal_keep_size is set in its config too. A divergent cap between primary and standby is the classic reason WAL bloat only appears after failover. Ship the identical retention GUCs to every node so a promoted standby enforces the same bounded failure the old primary did.

Authoritative references #

• PostgreSQL: pg_replication_slots (wal_status, restart_lsn) — the state column and LSN fields this page reads.
• PostgreSQL: max_slot_wal_keep_size — the invalidation cap and its interaction with checkpoints.
• PostgreSQL: wal_sender_timeout — the keepalive-driven reaper of dead walsenders.
• PostgreSQL: replication management functions — pg_drop_replication_slot semantics.

Related #

• Monitoring WAL generation and retention — the aggregate-rate view that complements this single-slot failure analysis.
• Replication slot types — the parent guide to the slot lifecycle whose end state this page defends against.
• Monitoring and alerting — alert rules for inactive-slot and wal_status thresholds.