BigData / Apache StreamPark Interview questions
1. What is Apache StreamPark?
Apache StreamPark is an all-in-one streaming application development framework and cloud-native real-time computing platform. It standardizes how teams configure, build, test, deploy, and operate stream processing jobs built on Apache Flink and Apache Spark . The project has two halves. StreamPar...
2. What are the two core components of Apache StreamPark?
StreamPark is built around two complementary pieces that split "writing the job" from "running the job." StreamPark Core — a development framework. It standardizes project configuration, encourages a functional programming style, and gives developers a RuntimeContext plus out-of-the-box con...
3. What is StreamPark Core?
StreamPark Core is the development-framework half of the project. It's positioned squarely as a coding tool rather than a web platform, and it follows a "convention over configuration" philosophy: project layout, configuration files, and startup parameters follow a standard shape so developers sp...
4. What is StreamPark Console?
StreamPark Console is the web-based operations half of the platform — a low-code UI for managing the full lifecycle of Flink (and Spark) jobs without hand-writing submission scripts. From the console you can compile a project pulled from a git repository, publish a build, configure startup ...
5. What stream processing engines does StreamPark support?
StreamPark supports Apache Flink and Apache Spark as its two execution engines, with multi-version support for each so a team isn't locked to a single engine release. Flink is the primary and most mature integration, covering streaming jobs end to end — DataStream, Flink SQL, and all of Str...
6. What is the origin of the name StreamPark?
The project didn't start under this name. It began as a personal project called StreamX , and the maintainers renamed it StreamPark in August 2022, ahead of donating it to the Apache Software Foundation. After incubation as incubator-streampark , it graduated to a full Apache Top-Level Project in...
7. What deployment modes does StreamPark support for Flink jobs?
StreamPark covers essentially every way Flink itself can be deployed, exposing each as a selectable execution mode when a job is configured. Cluster Modes offered YARN Application, Per-Job, Session Kubernetes Application, Session Standalone Remote (submit to an already-running cluster) This range...
8. What is Project Management in StreamPark?
Project Management is the Console feature that ties a Flink or Spark codebase, sitting in a git repository, to StreamPark's build and deploy pipeline. You register a project by giving StreamPark its git URL, and pick a branch to track. From there StreamPark can pull the code, build it with Maven,...
9. What alert channels does StreamPark support?
StreamPark's alert configuration currently supports four channels, and any combination of them can be enabled at once: E-mail — standard mail notification. DingTalk — via a DingTalk custom group robot. WeChat (Enterprise WeChat / WeCom) — via a custom group robot. Lark (Feishu) ...
10. What is Team Management in StreamPark?
Team Management is how StreamPark organizes applications and projects by department or group rather than exposing everything to every user. An ADMIN creates teams, and a normal USER account only sees the applications and projects of the team it's bound to. A team behaves like a workspace: switchi...
11. What is Variable Management in StreamPark?
Variable Management is a team-scoped store for values a job needs at runtime — database URLs, credentials, API keys, or any other configuration that shouldn't be hard-coded into a project or repeated across job definitions. Once a variable is defined for a team, it can be referenced by name...
12. How do you create a Flink SQL job in StreamPark Console?
StreamPark supports two ways to define a Flink job: uploading a pre-built jar, or writing Flink SQL directly in the console. For the SQL path, you don't need a separate project or a Maven build at all. Create a new application and choose Flink SQL as the development mode instead of Upload Jar. Wr...
13. What is the purpose of Yarn Queue Management in StreamPark?
Yarn Queue Management exists to stop a very common, very disruptive mistake: someone typing a YARN queue name (or queue and label) by hand when configuring a job, and getting it slightly wrong. A typo can send a job to a non-existent queue, causing slow, confusing submission failures, or send it ...
14. What technologies power StreamPark Console under the hood?
StreamPark Console is a fairly conventional Java web application, assembled from well-known open-source pieces rather than exotic custom infrastructure. Layer Technology Backend framework Spring Boot Data access MyBatis / MyBatis-Plus Frontend Vue with Ant Design of Vue In-browser terminal xterm....
15. What is a savepoint, and how does StreamPark use it?
A savepoint is Flink's mechanism for capturing a job's complete state at a point in time, in a self-contained, portable format that can later be used to restart the same job, or a modified version of it, from exactly where it left off. StreamPark surfaces savepoint creation as a first-class conso...
16. What database does StreamPark use to store its own metadata?
StreamPark Console needs a relational database to store its own operational data — users, teams, projects, application/job definitions, variables, and alert configurations. It ships ready to use with an embedded H2 database for quick starts and evaluation, so a first install can be running ...
17. What is the difference between StreamPark Core and StreamPark Console?
They solve different halves of the same problem, and mixing them up is a common beginner confusion. StreamPark Core StreamPark Console A coding-time development framework A runtime, web-based operations platform Used by developers writing Flink/Spark code Used by developers and operators running ...
18. What is the difference between YARN Application mode and YARN Session mode in StreamPark?
Both submit Flink jobs to a YARN cluster, but they differ in how many jobs share a JobManager. Application mode — StreamPark starts a dedicated JobManager per job, and the job's main() runs inside that JobManager on the cluster rather than on the client. Each job is fully isolated: one job'...
19. What is the difference between Kubernetes Application mode and Kubernetes Session mode?
The distinction mirrors the YARN case, but on Kubernetes the isolation boundary is the pod rather than the JobManager process alone. Kubernetes Application mode — StreamPark deploys a dedicated Flink cluster (JobManager and TaskManager pods) for a single job. When the job ends, its pods are...
20. Why does StreamPark recommend Application mode over Session mode for production jobs?
The recommendation comes down to blast radius. In Session mode, jobs share a JobManager, so a resource-hungry or misbehaving job can degrade or crash the shared JobManager and take every other job in that session down with it — a classic noisy-neighbor problem. Application mode gives each j...
21. How does StreamPark's Project Management integrate with CI/CD pipelines?
Project Management is StreamPark's own lightweight take on CI/CD rather than a wrapper around an external pipeline tool like Jenkins. Source — a project is registered with a git URL and a tracked branch. Build — StreamPark pulls the code and runs a Maven build, streaming the build log...
22. How do you configure a DingTalk alert in StreamPark?
DingTalk alerts route through a DingTalk custom group robot, so the setup happens partly in DingTalk and partly in StreamPark. In the target DingTalk group, add a Custom Robot and generate its webhook token, following DingTalk's own robot documentation. In StreamPark, go to Setting > Alert Settin...
23. When should you use StreamPark's Remote (Standalone) deployment mode?
Remote mode is the right choice when the Flink cluster a job should run on already exists and is managed outside of StreamPark — StreamPark submits into it rather than provisioning it. This fits organizations that run a Flink Standalone cluster maintained by a separate infrastructure team, ...
24. How does StreamPark support running multiple Flink versions side by side?
Different teams on one StreamPark install often need different Flink versions — one job stuck on an older version for compatibility, another on the latest release. StreamPark handles this by letting administrators register multiple Flink distributions (multiple FLINK_HOME installations) on ...
25. What is the difference between Upload Jar and Flink SQL job development in StreamPark?
These are the two ways to define what a job actually runs, and they suit different kinds of work. Upload Jar Flink SQL You bring a pre-built jar (own build or via Project Management) You write SQL directly in the console's Monaco editor Full DataStream API flexibility, custom logic, ML/complex pr...
26. Why isn't the MySQL JDBC driver bundled with StreamPark by default?
It's a licensing conflict, not an oversight. StreamPark is released under the Apache License 2.0, and the widely used MySQL Connector/J driver is distributed under a GPL-based license. Apache Software Foundation policy doesn't allow shipping GPL-licensed code inside an Apache-licensed release, so...
27. How does the Team concept enable multi-tenancy in StreamPark?
StreamPark's Team is the boundary that lets one shared installation serve multiple departments without them stepping on each other. Each team functions like an isolated workspace: applications, projects, and (in features like Yarn Queue Management) resource scoping all attach to a specific team. ...
28. What is the difference between the ADMIN and USER roles in StreamPark?
StreamPark keeps its role model deliberately simple: two account types with very different default reach. ADMIN — the system administrator. Has every permission across the entire StreamPark management page and across every team: creating teams, binding users to them, managing global setting...
29. How do you troubleshoot a Flink job that fails to start from StreamPark Console?
StreamPark centralizes the diagnostics you'd otherwise gather from several places, so troubleshooting a startup failure usually follows the same short path. Check the start log — in the job's row in the application list, an eye-shaped icon opens the submitted job's start logs, showing exact...
30. What role do flame graphs play in StreamPark's job monitoring?
Flame graphs give a visual answer to "which code is actually eating CPU" for a running job's operator, without attaching an external profiler. The capability itself is Flink's own, native since Flink 1.13 ( rest.flamegraph.enabled ): stack traces are sampled repeatedly for an operator, and render...
31. How does Yarn Queue Management prevent queue submission errors?
It replaces free-text queue entry with selection from a validated list, closing off the main way a submission ends up in the wrong place. Without this feature, a job's queue (or queue and label) is typed by hand each time it's configured for YARN Application or Session mode. A typo can point the ...
32. When would you choose StreamPark over writing raw Flink CLI submission scripts?
Raw flink run scripts work fine for a single developer running a handful of jobs, but they stop scaling once more people, more jobs, or more operational rigor enter the picture. StreamPark earns its keep once you need any of: a shared view of every job's status across a team, click-button savepoi...
33. How does StreamPark integrate with Apache Paimon for streaming warehouses?
StreamPark positions itself as more than a job runner — its stated goals include building a real-time data warehouse and unified stream/batch data solution, and Apache Paimon is one of the ecosystem tools it's built to integrate with toward that end. In practice, this integration works thro...
34. Why does StreamPark offer both Scala and Java interfaces for development?
Flink and Spark themselves are used heavily from both Java and Scala, and teams tend to standardize on one based on history and taste rather than technical necessity — so StreamPark Core exposes its RuntimeContext and connector APIs in both languages rather than forcing a choice. Scala's co...
35. What is the difference between StreamPark's HOCON config support and Flink's default flink-conf.yaml?
Both describe configuration, but they differ in expressiveness and where they apply. flink-conf.yaml is Flink's own flat YAML configuration file — key-value settings that apply cluster- or job-wide, written in plain YAML with limited structuring. StreamPark added HOCON (Human-Optimized Conf...
36. How does StreamPark's LDAP login support work alongside its built-in user accounts?
StreamPark's default authentication is its own built-in username/password accounts, managed inside its own database. LDAP login support was added as an additional, optional authentication path rather than a replacement for that default. When LDAP is configured, StreamPark can authenticate a user ...
37. Explain the execution flow of submitting a Flink job through StreamPark Console?
Submitting a job from StreamPark Console walks through several distinct stages, most of which are invisible to the user beyond a status change on screen. sequenceDiagram participant U as User participant C as StreamPark Console participant DB as Metadata DB participant CL as Target Cluster U->>C:...
38. Explain the internal working of StreamPark's multi-version Flink support through custom classloading?
Running several Flink versions from one long-lived Console process is trickier than it sounds, because different Flink versions ship different, sometimes incompatible, classes on their client-side classpaths. If Console naively loaded every registered Flink version's classes into the same JVM cla...
39. How can you optimize resource utilization when running many Flink jobs on a shared YARN cluster through StreamPark?
With many jobs on one cluster, the goal is balancing isolation against overhead, and StreamPark gives several levers to tune that balance rather than one fixed answer. Mix execution modes deliberately — put critical, latency-sensitive jobs in Application mode for isolation, and group many s...
40. Explain the lifecycle of a StreamPark Application from creation to termination?
A StreamPark "Application" moves through a well-defined set of states, and understanding the flow makes it easier to reason about where a stuck job actually is. stateDiagram-v2 [*] --> Created Created --> Building: Project build triggered Building --> Ready: Build succeeds Building --> BuildFaile...
41. How do you troubleshoot alert delivery throttling when many jobs fail simultaneously?
This is a real operational failure mode reported by teams running StreamPark at scale: when something like a shared YARN Session cluster goes down, many jobs fail within seconds of each other, and the alert channels themselves — not StreamPark — become the bottleneck. Chat-based robot...
42. What is the difference between StreamPark's DataStream extensions and the plain Flink DataStream API?
StreamPark Core doesn't replace the Flink DataStream API; it extends it with methods and context that remove repetitive setup, while leaving the underlying Flink semantics untouched. Plain Flink DataStream code typically starts with manually building a StreamExecutionEnvironment , wiring up param...
43. Explain the internal working of StreamPark Core's RuntimeContext abstraction?
RuntimeContext is the piece of StreamPark Core that turns "a Flink program" into "a StreamPark job" by standardizing how the program learns about its own environment and configuration at startup. When a job built with StreamPark Core starts, RuntimeContext is responsible for parsing the job's con...
44. How does StreamPark recover a Flink job from a savepoint after a Console restart?
StreamPark Console being restarted (a deploy, an upgrade, a host reboot) shouldn't take down the Flink jobs it manages, and the design reflects that: Console is a control-plane process, not the thing keeping jobs alive. Already-running jobs on YARN or Kubernetes keep running independently of Cons...
45. Why hasn't StreamPark standardized a built-in SMS alert channel?
StreamPark's built-in alert channels — Email, DingTalk, WeChat Work, and Lark — all have one thing in common: each has a single, widely used API that the whole community can integrate against once. SMS doesn't share that property. SMS delivery is fragmented by provider and by country:...
46. Explain the execution flow of a Kubernetes Application mode submission from StreamPark?
Kubernetes Application mode submission has more moving parts than YARN, because StreamPark is asking Kubernetes to materialize an entire dedicated Flink cluster for one job, from container images rather than a pre-installed Flink binary. sequenceDiagram participant U as User participant C as Stre...
47. How would you design team and queue isolation for a multi-department YARN cluster on StreamPark?
Designing this well means combining StreamPark's Team and Yarn Queue Management features so that organizational boundaries and cluster resource boundaries reinforce each other, rather than leaving one to drift out of sync with the other. Mirror departments as teams — create one StreamPark T...
48. Explain the difference between StreamPark's convention-over-configuration approach and manually configuring a Flink project?
The difference isn't in what's ultimately possible — both approaches can express the same Flink job — but in how many decisions a developer has to make explicitly before anything runs. A manually configured Flink project starts from a blank slate: project structure, dependency version...
49. How does StreamPark's permission model prevent one team from accessing another team's alert configurations?
The intended design is straightforward: alert configurations, like applications and projects, are meant to be scoped to the team that created them, so a user in Team A should never see Team B's DingTalk tokens or webhook secrets. In practice, getting this right requires every endpoint that touche...
50. Explain the execution flow of a Flink SQL job submitted through StreamPark's SQL editor?
A Flink SQL job takes a different path than an uploaded jar, since there's no pre-built artifact — StreamPark has to turn SQL text into a running Flink job at submission time. flowchart LR A[SQL written in Monaco editor] --> B[Parse & validate SQL] B --> C[Resolve declared connector depende...
