Skip to content

Configuring ColdFront

After installing ColdFront packages or building from source, several components still need configuring before the cold tier comes online:

  • PostgreSQL and ColdFront's configuration is detailed in this guide, regardless of which installation path you followed.
  • installing, configuring, and bootstrapping Lakekeeper, and the one-time per-database setup that ties everything together, documented in the Configuring Lakekeeper guide.
  • your object store, documented in Configuring your Object Store.

Configuring PostgreSQL

A package installation does not configure PostgreSQL. Add the following settings to postgresql.conf, then restart PostgreSQL:

shared_preload_libraries = 'pg_duckdb,coldfront'
duckdb.extension_directory = '/usr/lib/pgedge/coldfront/duckdb-extensions'
duckdb.allow_unsigned_extensions = true
duckdb.autoinstall_known_extensions = false
duckdb.postgres_role = 'coldfront_duckdb'
coldfront.iceberg_async_parquet = on
coldfront.iceberg_bakery_patch = on
coldfront.warehouse = '<warehouse-name>'
coldfront.lakekeeper_endpoint = 'http://<lakekeeper-host>:8181/catalog'
coldfront.local_pg_dsn = 'host=/var/run/postgresql dbname=<db> user=<role>'

Together, duckdb.extension_directory, duckdb.allow_unsigned_extensions, and duckdb.autoinstall_known_extensions point pg_duckdb at the directory where pgedge-coldfront-duckdb-extensions installs ColdFront's patched DuckDB extensions, rather than letting it fetch the unpatched upstream build - the same substitution that reintroduces HTTP 409 failures under concurrent cold writes. The patched extensions are unsigned, so duckdb.allow_unsigned_extensions must stay on; and because a missing extension file would otherwise trigger that same fallback, duckdb.autoinstall_known_extensions stays off.

duckdb.postgres_role names the role whose members may run DuckDB. Without it, only superusers can, and coldfront.grant_app_access() fails. Create the role before you grant access to an application role:

CREATE ROLE coldfront_duckdb NOLOGIN;

The two coldfront.iceberg_* settings make a node upload the Parquet files of an INSERT or COPY outside its write lock (the bakery claim on a Spock mesh, an advisory lock on a single node) and serialize only the catalog commit. The packaged duckdb-iceberg includes the patch that this ordering requires, and the Configuring a Distributed Deployment guide describes the mesh settings.

ColdFront also exposes the following settings, which adjust write behavior and execution; tune them as needed:

Setting Description
coldfront.allow_mixed_writes Controls tiered-mode UPDATE/DELETE whose WHERE cannot be proven to target one tier. on emits a dual-tier CTE; off rejects with an error and a hint. Not relevant in decoupled mode (every write is single-tier by definition). The default is on.
coldfront.cold_write_batch_size Sets how many cold rows a tiered INSERT gathers (see Caveats) before writing them to Iceberg as one INSERT. A larger value writes fewer, larger Parquet files; the remainder always flushes, so a small write stays one file. The default is 10000, with a minimum of 1.
coldfront.vector_probe Sets whether a recognized similarity search reads only the clusters nearest its query vector. off gives an exact scan of the whole corpus. Affects only a table with a trained vector column (see Working with Embeddings). The default is on.
coldfront.vector_nprobe Sets how many clusters such a search reads, overriding the column's own nprobe; at or above the column's nlist the search is exhaustive. The default is 0, which uses the configured value.
duckdb.force_execution Benchmark before enabling: on a mixed workload it helps count(distinct) and similar but regresses index lookups, top-K with PK ordering, and JSON access. The default is off.
duckdb.temporary_directory Sets where DuckDB spills. Each backend gets its own subdirectory there, named after its process id, so concurrent spills cannot collide; one left by a departed backend is reclaimed. See DuckDB: Spill Files Are Not Namespaced per Instance.
duckdb.max_temp_directory_size A cap per connection, not a cluster total. For a total budget, divide it by the concurrent sessions, or give the temp path its own filesystem or quota. The default is 90% of free space per session.

Writing ColdFront's Configuration

Server-level settings live in postgresql.conf, each managed table's configuration is a row in coldfront.partition_config, and the cold-store credential lives in coldfront.storage_secret.

Write the configuration with coldfront.set_storage_secret() and the register command. set_storage_secret() writes the following to coldfront.storage_secret; the Configuring your Object Store guide shares the exact values for each supported backend:

  • <access-key-id> - your object store's access key.
  • <secret-key> - the matching secret.
  • <endpoint> - your object store's endpoint.
SELECT coldfront.set_storage_secret('<access-key-id>', '<secret-key>', '<endpoint>');

register writes the following to coldfront.partition_config:

  • <table> - the table name.
  • <period> - the partition cadence, e.g. monthly.
  • <hot-period> - how long a partition stays hot before the archiver moves it to the cold tier.
./bin/archiver register --table <table> \
    --period <period> --hot-period "<hot-period>"

Or do both in one step by importing a deployment YAML:

archiver import --config deploy.yaml

The pgedge-coldfront package installs an example deployment YAML at /etc/pgedge/coldfront/config.yaml. The file is only an example; edit it and pass it to import - no ColdFront tool reads it unless --config names it. After an import, the server holds the configuration. A later run that is given a YAML checks the file against the server and refuses to run if any value differs. The only value such a run takes from the file is postgres.dsn, which connects when --dsn is unset. The Managing Partitioned Tables (CLI) section of the Using ColdFront guide describes register, import, and export.