How do you tune Hudi for trillion-record-scale upsert workloads?

At trillion-record scale, the bottlenecks shift from "does it work" to specific, well-understood pressure points, and production deployments (Uber's own engineering being the most public example) converge on a similar tuning playbook.

• Enable and rely on the Metadata Table for all file listing, avoiding direct, throttling-prone listing calls against cloud storage.
• Use the Record-Level Index instead of Bloom/HBase indexes for O(1) upsert lookups, since bloom-filter scanning and external index round-trips both degrade at this scale.
• Decouple table services from ingestion — run compaction and clustering asynchronously on their own resources so a slow table-service run never becomes a write-path bottleneck.
• Tune file sizing aggressively — both too many small files and too few oversized files hurt; target a file size that balances index lookup cost against parallelism.
• Use Non-Blocking Concurrency Control where multiple concurrent writers (streaming ingestion plus async table services) need to touch the same tables without lock contention.
• Monitor Metadata Table health specifically, not just raw job metrics, since at this scale the metadata layer itself becomes a first-class system to operate.