Why doesn't increasing the thread pool size always improve throughput?

Adding more threads only helps while there's a genuine resource, whether CPU cores, I/O bandwidth, or a downstream service, that those extra threads can actually put to use. Once a workload's real bottleneck is saturated, more threads just add overhead instead of more useful work.

For CPU-bound work, once the thread count passes the number of available cores, additional runnable threads can't run any faster in parallel, they just compete for the same cores. The OS then spends more time context-switching between them, and each switch flushes CPU caches and pipeline state, so throughput can actually drop as pool size grows past that point.

For any workload, a larger pool also means more threads can be simultaneously contending for the same shared lock, database connection pool, or downstream API, which increases contention and can push a shared resource like a database into its own overload state, slowing every thread down rather than speeding the overall system up.

More threads also cost memory, roughly a megabyte of stack space per platform thread by default, and a large pool sitting mostly idle waiting on a bottleneck elsewhere still consumes that memory and adds scheduler bookkeeping for no benefit. The practical takeaway is to size a pool based on the actual bottleneck resource, cores for CPU-bound work, the downstream system's real concurrency limit for I/O-bound work, rather than assuming "more threads" is a universal lever for "more throughput."