BigData / Apache StreamPark Interview questions
1. What is Apache StreamPark?
Apache StreamPark is an all-in-one streaming application development framework and cloud-native real-time computing platform. It standardizes how teams configure, build, test, deploy, and operate stream processing jobs built on Apache Flink and Apache Spark ....
2. What are the two core components of Apache StreamPark?
StreamPark is built around two complementary pieces that split "writing the job" from "running the job." StreamPark Core — a development framework.
3. What is StreamPark Core?
StreamPark Core is the development-framework half of the project. It's positioned squarely as a coding tool rather than a web platform, and it follows a "convention over configuration" philosophy: project layout, configuration files, and startup parameters follow a standard shape...
4. What is StreamPark Console?
StreamPark Console is the web-based operations half of the platform — a low-code UI for managing the full lifecycle of Flink (and Spark) jobs without hand-writing submission scripts. From the console you can compile a project pulled from a git...
5. What stream processing engines does StreamPark support?
StreamPark supports Apache Flink and Apache Spark as its two execution engines, with multi-version support for each so a team isn't locked to a single engine release. Flink is the primary and most mature integration, covering streaming jobs end to...
6. What is the origin of the name StreamPark?
The project didn't start under this name. It began as a personal project called StreamX , and the maintainers renamed it StreamPark in August 2022, ahead of donating it to the Apache Software Foundation....
7. What deployment modes does StreamPark support for Flink jobs?
StreamPark covers essentially every way Flink itself can be deployed, exposing each as a selectable execution mode when a job is configured. Cluster Modes offered YARN Application, Per-Job, Session Kubernetes Application, Session Standalone Remote (submit to an already-running cluster) This...
8. What is Project Management in StreamPark?
Project Management is the Console feature that ties a Flink or Spark codebase, sitting in a git repository, to StreamPark's build and deploy pipeline. You register a project by giving StreamPark its git URL, and pick a branch to track....
9. What alert channels does StreamPark support?
StreamPark's alert configuration currently supports four channels, and any combination of them can be enabled at once: E-mail — standard mail notification. DingTalk — via a DingTalk custom group robot....
10. What is Team Management in StreamPark?
Team Management is how StreamPark organizes applications and projects by department or group rather than exposing everything to every user. An ADMIN creates teams, and a normal USER account only sees the applications and projects of the team it's bound...
11. What is Variable Management in StreamPark?
Variable Management is a team-scoped store for values a job needs at runtime — database URLs, credentials, API keys, or any other configuration that shouldn't be hard-coded into a project or repeated across job definitions. Once a variable is defined...
12. How do you create a Flink SQL job in StreamPark Console?
StreamPark supports two ways to define a Flink job: uploading a pre-built jar, or writing Flink SQL directly in the console. For the SQL path, you don't need a separate project or a Maven build at all....
13. What is the purpose of Yarn Queue Management in StreamPark?
Yarn Queue Management exists to stop a very common, very disruptive mistake: someone typing a YARN queue name (or queue and label) by hand when configuring a job, and getting it slightly wrong. A typo can send a job to...
14. What technologies power StreamPark Console under the hood?
StreamPark Console is a fairly conventional Java web application, assembled from well-known open-source pieces rather than exotic custom infrastructure. Layer Technology Backend framework Spring Boot Data access MyBatis / MyBatis-Plus Frontend Vue with Ant Design of Vue In-browser terminal xterm.
15. What is a savepoint, and how does StreamPark use it?
A savepoint is Flink's mechanism for capturing a job's complete state at a point in time, in a self-contained, portable format that can later be used to restart the same job, or a modified version of it, from exactly where...
16. What database does StreamPark use to store its own metadata?
StreamPark Console needs a relational database to store its own operational data — users, teams, projects, application/job definitions, variables, and alert configurations. It ships ready to use with an embedded H2 database for quick starts and evaluation, so a first...
17. What is the difference between StreamPark Core and StreamPark Console?
They solve different halves of the same problem, and mixing them up is a common beginner confusion. StreamPark Core StreamPark Console A coding-time development framework A runtime, web-based operations platform Used by developers writing Flink/Spark code Used by developers and...
18. What is the difference between YARN Application mode and YARN Session mode in StreamPark?
Both submit Flink jobs to a YARN cluster, but they differ in how many jobs share a JobManager. Application mode — StreamPark starts a dedicated JobManager per job, and the job's main() runs inside that JobManager on the cluster rather...
19. What is the difference between Kubernetes Application mode and Kubernetes Session mode?
The distinction mirrors the YARN case, but on Kubernetes the isolation boundary is the pod rather than the JobManager process alone. Kubernetes Application mode — StreamPark deploys a dedicated Flink cluster (JobManager and TaskManager pods) for a single job....
20. Why does StreamPark recommend Application mode over Session mode for production jobs?
The recommendation comes down to blast radius. In Session mode, jobs share a JobManager, so a resource-hungry or misbehaving job can degrade or crash the shared JobManager and take every other job in that session down with it — a...
21. How does StreamPark's Project Management integrate with CI/CD pipelines?
Project Management is StreamPark's own lightweight take on CI/CD rather than a wrapper around an external pipeline tool like Jenkins. Source — a project is registered with a git URL and a tracked branch....
22. How do you configure a DingTalk alert in StreamPark?
DingTalk alerts route through a DingTalk custom group robot, so the setup happens partly in DingTalk and partly in StreamPark. In the target DingTalk group, add a Custom Robot and generate its webhook token, following DingTalk's own robot documentation....
23. When should you use StreamPark's Remote (Standalone) deployment mode?
Remote mode is the right choice when the Flink cluster a job should run on already exists and is managed outside of StreamPark — StreamPark submits into it rather than provisioning it. This fits organizations that run a Flink Standalone...
24. How does StreamPark support running multiple Flink versions side by side?
Different teams on one StreamPark install often need different Flink versions — one job stuck on an older version for compatibility, another on the latest release. StreamPark handles this by letting administrators register multiple Flink distributions (multiple FLINK_HOME installations) on
25. What is the difference between Upload Jar and Flink SQL job development in StreamPark?
These are the two ways to define what a job actually runs, and they suit different kinds of work. Upload Jar Flink SQL You bring a pre-built jar (own build or via Project Management) You write SQL directly in the...
26. Why isn't the MySQL JDBC driver bundled with StreamPark by default?
It's a licensing conflict, not an oversight. StreamPark is released under the Apache License 2.0, and the widely used MySQL Connector/J driver is distributed under a GPL-based license....
27. How does the Team concept enable multi-tenancy in StreamPark?
StreamPark's Team is the boundary that lets one shared installation serve multiple departments without them stepping on each other. Each team functions like an isolated workspace: applications, projects, and (in features like Yarn Queue Management) resource scoping all attach to...
28. What is the difference between the ADMIN and USER roles in StreamPark?
StreamPark keeps its role model deliberately simple: two account types with very different default reach. ADMIN — the system administrator....
29. How do you troubleshoot a Flink job that fails to start from StreamPark Console?
StreamPark centralizes the diagnostics you'd otherwise gather from several places, so troubleshooting a startup failure usually follows the same short path. Check the start log — in the job's row in the application list, an eye-shaped icon opens the submitted...
30. What role do flame graphs play in StreamPark's job monitoring?
Flame graphs give a visual answer to "which code is actually eating CPU" for a running job's operator, without attaching an external profiler. The capability itself is Flink's own, native since Flink 1.13 ( rest.flamegraph.enabled ): stack traces are sampled...
31. How does Yarn Queue Management prevent queue submission errors?
It replaces free-text queue entry with selection from a validated list, closing off the main way a submission ends up in the wrong place. Without this feature, a job's queue (or queue and label) is typed by hand each time...
32. When would you choose StreamPark over writing raw Flink CLI submission scripts?
Raw flink run scripts work fine for a single developer running a handful of jobs, but they stop scaling once more people, more jobs, or more operational rigor enter the picture. StreamPark earns its keep once you need any of:...
33. How does StreamPark integrate with Apache Paimon for streaming warehouses?
StreamPark positions itself as more than a job runner — its stated goals include building a real-time data warehouse and unified stream/batch data solution, and Apache Paimon is one of the ecosystem tools it's built to integrate with toward that...
34. Why does StreamPark offer both Scala and Java interfaces for development?
Flink and Spark themselves are used heavily from both Java and Scala, and teams tend to standardize on one based on history and taste rather than technical necessity — so StreamPark Core exposes its RuntimeContext and connector APIs in both...
35. What is the difference between StreamPark's HOCON config support and Flink's default flink-conf.yaml?
Both describe configuration, but they differ in expressiveness and where they apply. flink-conf.yaml is Flink's own flat YAML configuration file — key-value settings that apply cluster- or job-wide, written in plain YAML with limited structuring.
36. How does StreamPark's LDAP login support work alongside its built-in user accounts?
StreamPark's default authentication is its own built-in username/password accounts, managed inside its own database. LDAP login support was added as an additional, optional authentication path rather than a replacement for that default....
37. Explain the execution flow of submitting a Flink job through StreamPark Console?
Submitting a job from StreamPark Console walks through several distinct stages, most of which are invisible to the user beyond a status change on screen. sequenceDiagram participant U as User participant C as StreamPark Console participant DB as Metadata DB...
38. Explain the internal working of StreamPark's multi-version Flink support through custom classloading?
Running several Flink versions from one long-lived Console process is trickier than it sounds, because different Flink versions ship different, sometimes incompatible, classes on their client-side classpaths. If Console naively loaded every registered Flink version's classes into the same JVM...
39. How can you optimize resource utilization when running many Flink jobs on a shared YARN cluster through StreamPark?
With many jobs on one cluster, the goal is balancing isolation against overhead, and StreamPark gives several levers to tune that balance rather than one fixed answer. Mix execution modes deliberately — put critical, latency-sensitive jobs in Application mode for...
40. Explain the lifecycle of a StreamPark Application from creation to termination?
A StreamPark "Application" moves through a well-defined set of states, and understanding the flow makes it easier to reason about where a stuck job actually is. stateDiagram-v2 [*] --> Created Created --> Building: Project build triggered Building --> Ready: Build...
41. How do you troubleshoot alert delivery throttling when many jobs fail simultaneously?
This is a real operational failure mode reported by teams running StreamPark at scale: when something like a shared YARN Session cluster goes down, many jobs fail within seconds of each other, and the alert channels themselves — not StreamPark...
42. What is the difference between StreamPark's DataStream extensions and the plain Flink DataStream API?
StreamPark Core doesn't replace the Flink DataStream API; it extends it with methods and context that remove repetitive setup, while leaving the underlying Flink semantics untouched. Plain Flink DataStream code typically starts with manually building a StreamExecutionEnvironment , wiring up...
43. Explain the internal working of StreamPark Core's RuntimeContext abstraction?
RuntimeContext is the piece of StreamPark Core that turns "a Flink program" into "a StreamPark job" by standardizing how the program learns about its own environment and configuration at startup. When a job built with StreamPark Core starts, RuntimeContext is...
44. How does StreamPark recover a Flink job from a savepoint after a Console restart?
StreamPark Console being restarted (a deploy, an upgrade, a host reboot) shouldn't take down the Flink jobs it manages, and the design reflects that: Console is a control-plane process, not the thing keeping jobs alive. Already-running jobs on YARN or...
45. Why hasn't StreamPark standardized a built-in SMS alert channel?
StreamPark's built-in alert channels — Email, DingTalk, WeChat Work, and Lark — all have one thing in common: each has a single, widely used API that the whole community can integrate against once. SMS doesn't share that property....
46. Explain the execution flow of a Kubernetes Application mode submission from StreamPark?
Kubernetes Application mode submission has more moving parts than YARN, because StreamPark is asking Kubernetes to materialize an entire dedicated Flink cluster for one job, from container images rather than a pre-installed Flink binary. sequenceDiagram participant U as User participant...
47. How would you design team and queue isolation for a multi-department YARN cluster on StreamPark?
Designing this well means combining StreamPark's Team and Yarn Queue Management features so that organizational boundaries and cluster resource boundaries reinforce each other, rather than leaving one to drift out of sync with the other. Mirror departments as teams —...
48. Explain the difference between StreamPark's convention-over-configuration approach and manually configuring a Flink project?
The difference isn't in what's ultimately possible — both approaches can express the same Flink job — but in how many decisions a developer has to make explicitly before anything runs. A manually configured Flink project starts from a blank...
49. How does StreamPark's permission model prevent one team from accessing another team's alert configurations?
The intended design is straightforward: alert configurations, like applications and projects, are meant to be scoped to the team that created them, so a user in Team A should never see Team B's DingTalk tokens or webhook secrets. In practice,...
50. Explain the execution flow of a Flink SQL job submitted through StreamPark's SQL editor?
A Flink SQL job takes a different path than an uploaded jar, since there's no pre-built artifact — StreamPark has to turn SQL text into a running Flink job at submission time. flowchart LR A[SQL written in Monaco editor] -->...
