pgEdge ColdFront

ColdFront keeps tables in PostgreSQL and cold data in Apache Iceberg (Parquet on S3-compatible, Azure, or GCS storage), and the cold tier is both readable and writable through the same SQL with no application changes. The application queries every table as an ordinary PostgreSQL relation, and both operating modes present the same standard SQL surface.

ColdFront provides two operating modes:

• Tiered mode keeps recent data in native PostgreSQL partitions and archives older data to Iceberg on a watermark; the application reads a single unified view, and the archiver moves rows from hot to cold on a schedule.
• Decoupled mode stores the table entirely in Iceberg from the first row; PostgreSQL holds a thin wrapper view and a registry row, and the coldfront extension handles every data-modifying statement on that view.

Both modes coexist within one database, and you choose the mode per table at creation time. The SQL surface is identical for both modes: standard SELECT, INSERT, UPDATE, and DELETE against the relation.

Decoupled mode scales out horizontally across many PostgreSQL nodes that share one Lakekeeper catalog and one object store. The bakery protocol in the coldfront extension serializes Iceberg commits on the PostgreSQL side using Spock-replicated Snowflake tickets, so concurrent writers never collide at the catalog. The protocol implements Lamport mutual exclusion with the Ricart-Agrawala deferred-reply optimization, and the formal model verifies its safety with TLA+.

How It Works

ColdFront runs inside PostgreSQL and rewrites each statement to the correct tier, so the application sees one relation:

                       Application
                            │
              SELECT / INSERT / UPDATE / DELETE
              against one relation: "events"
                            │
                 PostgreSQL 16 / 17 / 18
        events VIEW: reads union hot + cold
        coldfront extension: rewrites writes to the right tier
              ┌─────────────┴───────────────┐
              │                             │
          hot tier                      cold tier
      _events: native PG            pg_duckdb: in-process
      range partitions              DuckDB, Iceberg reads + writes
              │                             │
              │                   Lakekeeper (Iceberg REST catalog)
              │                             │
              │                   object store, S3 / Azure / GCS
              │                   (Parquet data + Iceberg metadata)
              │                             ▲
              └──── Archiver (Go, cron) ────┘
                    moves partitions past the hot window: hot → cold

Quickstart

Build the image (see the Installation guide) and bring up the stack:

docker compose up -d --build

Bootstrap Lakekeeper and create a warehouse (see the one-time setup in the Using ColdFront guide), then create a table in psql:

CREATE EXTENSION IF NOT EXISTS pg_duckdb;
CREATE EXTENSION IF NOT EXISTS coldfront;
SELECT coldfront.set_storage_secret('admin', 'adminsecret', 'seaweedfs:8333');

-- Decoupled (iceberg-only) table, stored entirely in Iceberg on S3:
SELECT coldfront.create_iceberg_table('public', 'events',
  '[{"name":"id","type":"bigint"},{"name":"ts","type":"timestamptz"},{"name":"note","type":"text"}]'::jsonb);
INSERT INTO events VALUES (1, now(), 'hello');
SELECT count(*) FROM events;

For compliance environments that cannot store an object-store credential, coldfront.set_storage_secret_vended() runs with no credential in the database: Lakekeeper mints short-lived per-table credentials at access time. See Vended credentials.

Least-privilege application roles

Application roles need no superuser and no server-file access, yet they read and write the cold tier through the same transparent view. Onboarding an application role is a single call:

SELECT coldfront.grant_app_access('alice');

grant_app_access grants only the minimum the cold path needs: membership in duckdb.postgres_role, schema USAGE, SELECT on the registry, DML on every registered view and the hot table and sequences behind it (all derived from the registry, not hardcoded), plus EXECUTE on a fixed allow-list of runtime cold-path functions. The call is idempotent and is not executable by PUBLIC, so an application role can never self-grant. The role is never granted pg_read_server_files or pg_write_server_files, so it has no host-file access. CREATE ROLE and GRANT both replicate over Spock, so you onboard a role once on any node and it propagates across the mesh.

For how the non-superuser path works under the hood - the SECURITY DEFINER attach helpers, the PGC_SUSET / GUC_SUPERUSER_ONLY config hardening, the turnkey duckdb.postgres_role default, and how least privilege holds across a Spock mesh - see Architecture: non-superuser app roles.

Caveats

Iceberg on Azure ADLS Gen2 requires Blob soft-delete, container soft-delete, and change feed (blob events) to be OFF on the storage account. Lakekeeper warehouse creation otherwise fails with HTTP 409 ("This endpoint does not support BlobStorageEvents or SoftDelete"). Disable those features on the storage account before using it as a cold tier.

Next Steps

New here? Run the guided walkthrough to see all three modes in action with copy-pasteable commands.

To go further with ColdFront, consult the following guides:

• The Installation guide covers building ColdFront and bringing up the stack.
• The Using ColdFront guide covers both modes, the standalone partition manager, supported types, and tuning.
• The Object Store Setup guide takes you from an empty bucket to a working cold tier on cloud S3.
• The Architecture overview explains the shared mechanics and links to the per-mode deep dives.