Which is better and why: lookup or full-compaction changelog producer for a 30-minute-latency pipeline?

For a pipeline whose consumers only need updates roughly every 30 minutes, full-compaction is generally the better fit, precisely because its coarser cadence matches (rather than wastes) the freshness the pipeline actually needs.

lookup reconstructs the before-value on every single commit, which is work spent buying freshness the 30-minute consumer isn't asking for — every commit pays a lookup cost even though nothing downstream reads that granularity of change. full-compaction instead ties changelog generation to periodic full compactions, which can be tuned (via full-compaction.delta-commits or a time trigger) to land roughly every 30 minutes, batching the changelog work into fewer, larger operations that align with the compaction the table needs to run anyway.

The exception would be if the pipeline's write throughput is itself very high and full compactions at that cadence become disruptively expensive — in that specific case, it's worth benchmarking both, since a well-tuned lookup setup can sometimes spread cost more evenly than a large periodic full compaction. But as a default recommendation for a 30-minute-latency requirement, full-compaction avoids paying for freshness nobody consumes.