AI / Apache Burr Interview questions
How can you optimize parallel Burr actions using a custom executor?
Burr’s synchronous parallelism runs on a concurrent.futures.Executor, and you can control it at the application level with .with_parallel_executor(...), which becomes the default executor passed down to every parallel action:
app = ( ApplicationBuilder() .with_parallel_executor(MultiThreadedExecutor(max_concurrency=10)) .build() )
Because it’s just a standard executor interface, you can subclass concurrent.futures.Executor to route work onto whatever backend fits your workload — a process pool for CPU-bound reduction, or a custom implementation that submits to something like Ray or Modal for distributed execution across machines.
Async parallelism takes a different path: it doesn’t use an executor at all, relying on asyncio.gather instead, but it does require you to reach for async persisters backed by a connection pool (use_pool=True) rather than a single direct connection, since concurrent tasks can’t safely share one connection. Skipping the pool under concurrency is a common source of async persistence errors.
More Related questions...
