BigData / Apache StreamPark Interview questions
Why does StreamPark recommend Application mode over Session mode for production jobs?
The recommendation comes down to blast radius. In Session mode, jobs share a JobManager, so a resource-hungry or misbehaving job can degrade or crash the shared JobManager and take every other job in that session down with it — a classic noisy-neighbor problem. Application mode gives each job its own JobManager, so a problem in one job stays contained to that job.
Application mode also avoids the client-side resource consumption pattern that some Session workflows have, since the job's driver logic runs on the cluster rather than tying up a submitting client process.
The trade-off StreamPark users weigh against this is resource cost: many small Application-mode jobs each need their own JobManager overhead, which is heavier in aggregate than one shared Session. Teams that need to run large numbers of small, low-priority jobs sometimes deliberately choose Session mode and accept the shared-fate risk to save on resource overhead, using two JobManager instances for high availability to mitigate the single point of failure.
More Related questions...