mtree table-diff
Compares Merkle trees of a table across nodes to find inconsistencies.
By default, updates trees first using CDC.
If a node's replication slot is already held by another consumer — normally a
running mtree listen, but also a concurrent table-diff/update on the same
node (the slot is shared across all of a node's Merkle trees) — table-diff
skips the CDC drain for that node (printing a warning) and compares against the
already-maintained tree instead of failing. The comparison is best-effort and
may omit the most recent changes on those nodes, so divergence can be
under-reported; the skipped nodes are listed in the diff summary
(cdc_skipped_nodes). Ensure no mtree listen or other mtree operation is
holding the node's slot, then re-run, if you need a guaranteed-current drain.
Usage
./ace mtree table-diff [flags] [cluster] <schema.table>
Arguments
[cluster]— Optional; overridesdefault_cluster.
Flags
| Flag | Alias | Description | Default |
|---|---|---|---|
--dbname |
-d |
Database name | |
--nodes |
-n |
Nodes to include (comma or all) |
all |
--max-cpu-ratio |
-m |
Max CPU ratio | 0.5 |
--output |
-o |
json or html |
json |
--skip-cdc |
-U |
Skip CDC processing (only rehash and compare) | false |
--cdc-timeout |
Seconds to drain CDC before giving up (0 = use cdc_processing_timeout / default) |
0 |
|
--quiet |
-q |
Suppress output | false |
--debug |
-v |
Debug logging | false |
Example
./ace mtree table-diff --dbname=mydatabase my-cluster public.my_table
Notes
- With
--output html, both JSON and HTML reports are generated with matching timestamps. - The number of differing rows collected per node pair is bounded by
mtree.diff.max_diff_rows(the shippedace.yamlsets1000000; if the key is absent or0, the diff is unbounded). When the cap is reached, enumeration for that pair stops, the report'sdiff_row_limit_reachedflag is set, and a warning is logged. This keeps a heavily diverged table from exhausting memory; lower the value inace.yamlon memory-constrained hosts, and re-run after repairing to surface the remaining differences. - When all mismatched blocks between a node pair turn out to hold no row
differences, the tree hashes were stale rather than the data divergent. The
diff reports this, refreshes those blocks from live data (recorded in the
summary's
stale_blocks_refreshed), and the next diff runs clean. A pair with any real differences, an incomplete comparison, or an--untilfilter is not refreshed. If the trees still disagree after a refresh, the nodes' leaf ranges have diverged; runace mtree buildto realign them.