BigData / Apache StreamPark Interview questions
How would you design team and queue isolation for a multi-department YARN cluster on StreamPark?
Designing this well means combining StreamPark's Team and Yarn Queue Management features so that organizational boundaries and cluster resource boundaries reinforce each other, rather than leaving one to drift out of sync with the other.
- Mirror departments as teams — create one StreamPark Team per department (or, for larger departments, per sub-team that owns distinct jobs), and have an ADMIN bind each department's members only to their own team.
- Provision matching YARN queues — work with cluster administrators to create dedicated YARN queues (with appropriate capacity guarantees) per department, mirroring the same boundaries as the StreamPark teams.
- Register queues per team via Yarn Queue Management — so a department's job configuration only ever offers that department's own valid queues, closing off accidental cross-department submission entirely.
- Scope alert and variable configuration per team — since credentials and alert routing are also team-scoped resources, this keeps a department's secrets and notification targets out of another department's view.
- Review periodically — queue capacity and team membership both drift as departments grow; treat the mapping between StreamPark teams and YARN queues as something to re-validate, not a one-time setup.
Done this way, a developer in one department simply doesn't have the option to misconfigure a job into another department's resources — the isolation is structural, not just a matter of following a convention.
More Related questions...