Prev Next

BigData / Apache StreamPark Interview questions

Explain the execution flow of a Kubernetes Application mode submission from StreamPark?

Kubernetes Application mode submission has more moving parts than YARN, because StreamPark is asking Kubernetes to materialize an entire dedicated Flink cluster for one job, from container images rather than a pre-installed Flink binary.

sequenceDiagram participant U as User participant C as StreamPark Console participant D as Docker Registry participant K as Kubernetes API participant P as Job/TM Pods U->>C: Start Application (mode = kubernetes-application) C->>C: Build Flink client command with K8s parameters C->>K: Create Namespace/ServiceAccount context (if needed) C->>D: Reference configured Flink image C->>K: Submit Application via Flink Kubernetes client K->>P: Schedule JobManager pod, then TaskManager pod(s) P-->>K: Report pod status K-->>C: Return ClusterId / deployment status C-->>U: Show WebUI link and Running status

Before submission, the target namespace and the service account used for RBAC need to already be set up on the cluster — StreamPark doesn't create Kubernetes-level RBAC policy from nothing. Docker registry credentials are configured once in Console's Docker Setting, so the Flink image reference in the job's configuration can actually be pulled.

When Start is clicked, Console builds the Kubernetes-specific parameters — image, namespace, resource requests/limits, ClusterId — and submits through Flink's own Kubernetes client integration rather than shelling out to raw kubectl. Kubernetes then schedules a JobManager pod first, which in turn requests TaskManager pods once it's up. StreamPark tracks the resulting ClusterId, and from then on uses the remote REST API plus the Kubernetes API together to report status back into Console, including a link to the job's own Flink WebUI.

What must already exist on the Kubernetes cluster before StreamPark submits a job?
How does StreamPark actually submit to Kubernetes?

More Related questions...

What is Apache StreamPark? What are the two core components of Apache StreamPark? What is StreamPark Core? What is StreamPark Console? What stream processing engines does StreamPark support? What is the origin of the name StreamPark? What deployment modes does StreamPark support for Flink jobs? What is Project Management in StreamPark? What alert channels does StreamPark support? What is Team Management in StreamPark? What is Variable Management in StreamPark? How do you create a Flink SQL job in StreamPark Console? What is the purpose of Yarn Queue Management in StreamPark? What technologies power StreamPark Console under the hood? What is a savepoint, and how does StreamPark use it? What database does StreamPark use to store its own metadata? What is the difference between StreamPark Core and StreamPark Console? What is the difference between YARN Application mode and YARN Session mode in StreamPark? What is the difference between Kubernetes Application mode and Kubernetes Session mode? Why does StreamPark recommend Application mode over Session mode for production jobs? How does StreamPark's Project Management integrate with CI/CD pipelines? How do you configure a DingTalk alert in StreamPark? When should you use StreamPark's Remote (Standalone) deployment mode? How does StreamPark support running multiple Flink versions side by side? What is the difference between Upload Jar and Flink SQL job development in StreamPark? Why isn't the MySQL JDBC driver bundled with StreamPark by default? How does the Team concept enable multi-tenancy in StreamPark? What is the difference between the ADMIN and USER roles in StreamPark? How do you troubleshoot a Flink job that fails to start from StreamPark Console? What role do flame graphs play in StreamPark's job monitoring? How does Yarn Queue Management prevent queue submission errors? When would you choose StreamPark over writing raw Flink CLI submission scripts? How does StreamPark integrate with Apache Paimon for streaming warehouses? Why does StreamPark offer both Scala and Java interfaces for development? What is the difference between StreamPark's HOCON config support and Flink's default flink-conf.yaml? How does StreamPark's LDAP login support work alongside its built-in user accounts? Explain the execution flow of submitting a Flink job through StreamPark Console? Explain the internal working of StreamPark's multi-version Flink support through custom classloading? How can you optimize resource utilization when running many Flink jobs on a shared YARN cluster through StreamPark? Explain the lifecycle of a StreamPark Application from creation to termination? How do you troubleshoot alert delivery throttling when many jobs fail simultaneously? What is the difference between StreamPark's DataStream extensions and the plain Flink DataStream API? Explain the internal working of StreamPark Core's RuntimeContext abstraction? How does StreamPark recover a Flink job from a savepoint after a Console restart? Why hasn't StreamPark standardized a built-in SMS alert channel? Explain the execution flow of a Kubernetes Application mode submission from StreamPark? How would you design team and queue isolation for a multi-department YARN cluster on StreamPark? Explain the difference between StreamPark's convention-over-configuration approach and manually configuring a Flink project? How does StreamPark's permission model prevent one team from accessing another team's alert configurations? Explain the execution flow of a Flink SQL job submitted through StreamPark's SQL editor?
Show more question and Answers...

Web

Comments & Discussions