Java / Azure Kubernetes Service (AKS) Interview questions

Azure Kubernetes Service (AKS) is Microsoft's managed Kubernetes offering. Azure runs the control plane for you (API server, etcd, scheduler and controller manager), and you only manage the worker nodes that run your containers.

Day-to-day chores such as provisioning the cluster, patching, version upgrades and health monitoring of the control plane are handled by the platform. You keep control over node sizes, networking, scaling rules and the workloads you deploy.

AKS plugs into the rest of Azure: Microsoft Entra ID for sign-in, Azure Container Registry for images, Azure Monitor for telemetry, and virtual networks for private connectivity.

An AKS cluster has two halves: a Microsoft-managed control plane and customer-owned nodes.

The nodes are Azure VMs grouped into node pools. AKS places them, along with load balancers and disks it creates, in a separate node resource group whose name starts with MC_.

A node pool is a group of worker nodes that share the same VM size, OS type and settings. On AKS each pool is backed by a Virtual Machine Scale Set, so scaling a pool just changes the scale set instance count.

Pools let you mix hardware in one cluster. You might keep a general-purpose pool for web pods, a GPU pool for model inference and a Windows pool for a legacy .NET service, then steer pods to the right pool with labels and taints.

az aks nodepool add \
  --resource-group rg-demo --cluster-name aks-demo \
  --name gpupool --node-count 2 \
  --node-vm-size Standard_NC6s_v3 \
  --node-taints sku=gpu:NoSchedule

System node pools host AKS's critical pods such as CoreDNS, metrics-server and the konnectivity agent. User node pools are where your own application pods run.

Every cluster needs at least one system pool. Tainting it keeps application pods off those nodes, so a noisy app cannot starve CoreDNS.

AKS charges nothing for the control plane on the Free tier, but there is no financially backed SLA, so it suits dev and test. You still pay for the nodes.

Standard and Premium add a per-cluster hourly fee on top of node costs. Check the Azure pricing page for current rates.

A pod is the smallest deployable unit in Kubernetes. It wraps one or more containers that share a network namespace (one IP, talking over localhost) and can share mounted volumes.

Pods are disposable. If a node dies, the pod is not moved, a controller such as a Deployment creates a replacement. That is why production workloads should not be run as bare pods.

apiVersion: v1
kind: Pod
metadata:
  name: web
spec:
  containers:
  - name: nginx
    image: nginx:1.27
    ports:
    - containerPort: 80

AKS supports several CNI choices that differ mainly in where pod IPs come from.

The Cilium dataplane can be layered on Overlay or Pod Subnet to get eBPF networking and network policy. For new clusters, Overlay is the usual starting point.

You need a resource group first, then a single az aks create call.

1. Sign in and pick the subscription with az login.
2. Create a resource group.
3. Create the cluster with your node count, VM size and network plugin.
4. Pull credentials and verify with kubectl.

az group create -n rg-demo -l eastus

az aks create -g rg-demo -n aks-demo \
  --node-count 3 --node-vm-size Standard_D4s_v5 \
  --network-plugin azure --network-plugin-mode overlay \
  --generate-ssh-keys

az aks get-credentials -g rg-demo -n aks-demo
kubectl get nodes

Provisioning usually takes a few minutes. Add --tier standard for a production SLA.

Run az aks get-credentials. It downloads the cluster's connection details and merges them into ~/.kube/config as a new context.

az aks get-credentials -g rg-demo -n aks-demo
kubectl config current-context
kubectl get nodes

If the cluster uses Microsoft Entra ID, kubectl needs kubelogin to fetch tokens (az aks install-cli installs both). The --admin flag fetches a local admin certificate, but it only works while local accounts are enabled and is best avoided in production.

Azure Container Registry (ACR) is Azure's private, managed registry for container images and Helm charts. AKS pulls application images from it when pods start.

The cleanest integration is to attach the registry to the cluster. This grants the AcrPull role to the cluster's kubelet managed identity, so you do not need image pull secrets.

az aks update -g rg-demo -n aks-demo --attach-acr myregistry

# reference the image in your manifest
# image: myregistry.azurecr.io/shop/api:1.4.2

A Service gives a stable virtual IP and DNS name to a changing set of pods. The type decides who can reach it.

For a private load balancer, add the annotation service.beta.kubernetes.io/azure-load-balancer-internal: "true" to the Service.

An Ingress is a Kubernetes object that routes external HTTP and HTTPS traffic to Services based on host name and URL path, and can terminate TLS. It works at layer 7, so one public IP can front many apps.

The object does nothing on its own. An ingress controller must be running to read the rules and configure a proxy. On AKS that is commonly the application routing add-on, Application Gateway for Containers or a self-managed controller.

The community Ingress-NGINX project has been retired and the Ingress API is frozen, so newer designs lean towards Gateway API.

A managed identity is an Azure-managed credential that AKS uses to call Azure APIs, so you never store or rotate a service principal secret.

A cluster uses two of them. The control plane identity lets AKS create and manage resources such as load balancers, routes and disks. The kubelet identity is used by nodes, mainly to pull images from ACR.

You can let AKS create them (system-assigned for the control plane) or bring your own user-assigned identities, which is handy when you must pre-grant roles before the cluster exists.

A ConfigMap holds non-sensitive settings such as feature flags or a log level. A Secret holds sensitive values such as passwords and tokens. Both can reach a pod as environment variables or as files in a mounted volume.

Secret values are only base64-encoded, which is encoding, not encryption. Anyone with read access to the Secret can decode it. AKS encrypts etcd at rest, and you can add KMS encryption with Key Vault, but tight RBAC matters just as much.

kubectl create configmap app-config --from-literal=LOG_LEVEL=info
kubectl create secret generic db-pass --from-literal=password='S3cret!'

The cluster autoscaler adds nodes to a node pool when pods cannot be scheduled for lack of CPU or memory, and removes nodes that stay underused.

It is enabled per node pool with a minimum and maximum count. By default a node becomes a scale-down candidate after it has been unneeded for about 10 minutes and its utilization is below 50%.

az aks nodepool update -g rg-demo --cluster-name aks-demo -n userpool \
  --enable-cluster-autoscaler --min-count 2 --max-count 8

The Horizontal Pod Autoscaler (HPA) changes the replica count of a Deployment based on observed metrics, usually CPU or memory from metrics-server, which AKS ships by default.

The controller computes desired = ceil(current replicas x current metric / target metric). For CPU-based scaling, pods must declare resource requests, otherwise utilization cannot be calculated.

kubectl autoscale deployment web --cpu-percent=70 --min=3 --max=15

Scale nodes with az aks scale (or az aks nodepool scale for one pool) and scale pods with kubectl scale.

# nodes
az aks nodepool scale -g rg-demo --cluster-name aks-demo -n userpool --node-count 5

# pods
kubectl scale deployment web --replicas=10

If the cluster autoscaler is enabled on that pool, you cannot set a fixed count. Change the min and max values, or disable the autoscaler first.

A PersistentVolume (PV) is storage that outlives any single pod. A pod asks for it through a PersistentVolumeClaim (PVC), and a StorageClass tells Kubernetes how to provision it dynamically.

AKS ships CSI-based classes such as managed-csi (Azure Disk) and azurefile-csi (Azure Files). When a PVC is created, the driver provisions the Azure resource and binds it automatically.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
spec:
  accessModes: [ReadWriteOnce]
  storageClassName: managed-csi
  resources:
    requests:
      storage: 20Gi

Container Insights is the Azure Monitor feature that collects container logs, node and pod performance data and Kubernetes events, and sends them to a Log Analytics workspace.

Enabling the monitoring add-on deploys the ama-logs agent as a DaemonSet. You then query data with KQL or use the built-in workbooks in the portal.

az aks enable-addons -g rg-demo -n aks-demo -a monitoring

For metrics, Azure Monitor managed service for Prometheus and managed Grafana are the usual companions.

AKS Automatic is an opinionated cluster SKU where Azure manages far more than the control plane, including node provisioning, scaling, upgrades and many security defaults.

It comes pre-configured with node auto-provisioning (Karpenter-based), Azure CNI Overlay with Cilium, managed Prometheus and Grafana, Entra ID with Azure RBAC, workload identity and deployment safeguards. You focus on deploying apps.

az aks create -g rg-demo -n aks-auto --sku automatic

The trade-off is less freedom to tweak low-level settings than on AKS Standard.

AKS upgrades the control plane first, then the node pools. You pick a supported target and AKS performs a rolling upgrade.

1. List what is available with az aks get-upgrades.
2. Test the new version in a lower environment and check for removed APIs.
3. Run az aks upgrade (use --control-plane-only to stage the nodes later).
4. Confirm node versions with kubectl get nodes.

az aks get-upgrades -g rg-demo -n aks-demo -o table
az aks upgrade -g rg-demo -n aks-demo --kubernetes-version 1.33.2

You cannot skip minor versions, so 1.31 goes to 1.32 before 1.33. To automate this, set an auto-upgrade channel such as patch, stable or rapid and define a planned maintenance window.

With kubenet nodes get VNet IPs while pods get addresses from a separate private range and are NATed through the node. With Azure CNI pods receive routable VNet IPs directly.

Plan to move kubenet clusters to Azure CNI Overlay, which keeps the low IP footprint without kubenet's limits.

Choose Azure CNI Overlay when VNet address space is scarce or you expect a large cluster. Pods draw IPs from a private overlay CIDR (default 10.244.0.0/16), so only nodes consume VNet addresses.

It supports up to 250 pods per node and clusters of up to 5,000 nodes, and the same pod CIDR can be reused across clusters because it is never advertised on the VNet.

The catch is that pods are not directly routable from outside the cluster. External systems must reach workloads through Services, ingress or load balancers. If something must address pod IPs directly, pick Pod Subnet instead.

az aks create -g rg-demo -n aks-demo \
  --network-plugin azure --network-plugin-mode overlay \
  --pod-cidr 192.168.0.0/16

With AKS-managed Entra integration, users sign in with their Entra identity and the API server validates the token. You nominate Entra groups as cluster admins, and access is then decided by Kubernetes RBAC or Azure RBAC.

sequenceDiagram
  participant U as User
  participant E as Microsoft Entra ID
  participant A as AKS API server
  U->>E: az login / kubelogin
  E-->>U: Access token
  U->>A: kubectl request + token
  A->>E: Validate token
  A-->>U: Allow or deny via RBAC

az aks create -g rg-demo -n aks-demo \
  --enable-aad --aad-admin-group-object-ids <group-id> \
  --disable-local-accounts

Disabling local accounts removes shared admin certificates, so every action is tied to a named identity.

Both answer the question "what can this user do?", but they store the rules in different places.

Azure RBAC for Kubernetes authorization is switched on with --enable-azure-rbac. Do not confuse it with the Azure roles that control the AKS resource itself (for example Contributor), which govern who can scale or delete the cluster, not who can read pods.

Workload Identity lets a pod call Azure services (Key Vault, Storage, SQL) as a managed identity without any secret. It uses OIDC federation between the cluster and Microsoft Entra ID.

flowchart LR
  A["Pod with annotated service account"] --> B["Projected service account token"]
  B --> C["Microsoft Entra ID checks federated credential"]
  C --> D["Entra issues Azure access token"]
  D --> E["Pod calls Key Vault or Storage"]

1. Enable --enable-oidc-issuer --enable-workload-identity on the cluster.
2. Create a user-assigned managed identity and grant it the needed Azure role.
3. Create a federated credential tied to system:serviceaccount:<ns>:<sa>.
4. Annotate the service account with the identity's client ID and label the pod azure.workload.identity/use: "true".

It replaces the retired AAD Pod Identity.

Use the Secrets Store CSI Driver with the Azure Key Vault provider. It mounts secrets, keys or certificates from Key Vault into the pod as files, so values never sit in your manifests.

1. Enable the add-on: az aks enable-addons -a azure-keyvault-secrets-provider.
2. Give the pod an identity (Workload Identity is recommended) with Get permission on the vault.
3. Define a SecretProviderClass naming the vault and objects.
4. Mount it as a CSI volume in the pod spec.

Secrets can optionally be synced to a native Kubernetes Secret if an app insists on environment variables. Enable rotation polling if you want updated vault values to appear in running pods.

The application routing add-on is a managed ingress solution. It deploys NGINX ingress controllers and wires up Azure DNS records and Key Vault-backed TLS certificates for you.

az aks approuting enable -g rg-demo -n aks-demo

# Ingress uses this class
# ingressClassName: webapprouting.kubernetes.azure.com

Upstream Ingress-NGINX reached end of life in March 2026. Microsoft committed to critical security patches for the add-on's NGINX implementation only through November 2026, so existing users should plan a move to the add-on's Gateway API implementation or another Gateway API option.

The Ingress API is frozen and relies on controller-specific annotations for anything beyond basic host and path routing. Gateway API is its successor and models traffic with separate, role-based resources.

On AKS the main options are Application Gateway for Containers, the Istio-based Gateway API mode in the application routing add-on (in preview when announced, so verify its status) and the Istio service mesh add-on. With Ingress-NGINX retired, new designs should start on Gateway API.

Use Azure Disk for single-writer, latency-sensitive data such as a database. Use Azure Files when several pods on different nodes must read and write the same files.

Also keep the per-VM disk attach limit in mind when packing many stateful pods on one node.

KEDA (Kubernetes Event-driven Autoscaling) scales workloads based on external event sources such as Service Bus queue length, Event Hubs lag, Kafka topics or a Prometheus query. AKS offers it as a managed add-on with --enable-keda.

Unlike plain HPA, KEDA can scale a Deployment down to zero when there is no work and back up when events arrive. For one-or-more replicas it feeds an HPA behind the scenes.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: orders-worker
spec:
  scaleTargetRef:
    name: orders-worker
  minReplicaCount: 0
  maxReplicaCount: 30
  triggers:
  - type: azure-servicebus
    metadata:
      queueName: orders
      messageCount: "20"

By default every pod can talk to every other pod. A NetworkPolicy restricts that, but it only works if the cluster was created with a policy engine: azure, calico or cilium set through --network-policy.

A common pattern is a default-deny rule per namespace, then explicit allows. Policies are additive, so any matching allow rule opens the path.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-web-to-api
  namespace: shop
spec:
  podSelector:
    matchLabels: {app: api}
  policyTypes: [Ingress]
  ingress:
  - from:
    - podSelector:
        matchLabels: {app: web}
    ports:
    - port: 8080

For new clusters Cilium (eBPF) is generally the direction Microsoft favors, but check current docs before choosing.

Enable the Azure Policy add-on. It installs Gatekeeper (OPA) admission webhooks and syncs Azure Policy definitions into the cluster as constraints.

az aks enable-addons -g rg-demo -n aks-demo -a azure-policy

You then assign built-in definitions or initiatives at subscription, resource group or cluster scope. Typical rules block privileged containers, require resource limits, allow images only from approved registries or forbid hostPath mounts.

Each policy runs in Audit mode (report only) or Deny mode (reject at admission). Starting in Audit lets you see what would break before enforcing. Compliance shows up in the Azure Policy dashboard.

A private cluster exposes the API server only on a private IP inside your VNet, through Private Link, so there is no public endpoint.

az aks create -g rg-demo -n aks-private \
  --enable-private-cluster \
  --private-dns-zone system \
  --network-plugin azure --network-plugin-mode overlay

Because the endpoint is private, kubectl must run from somewhere with a network path: a VM in the VNet or a peered one, a VPN or ExpressRoute link, or a self-hosted CI agent. For quick one-off commands, az aks command invoke runs kubectl through the Azure control plane.

An alternative is API Server VNet Integration, which places the API server behind an internal load balancer in a delegated subnet and avoids Private Link.

A Kubernetes version upgrade moves the control plane and kubelet to a new Kubernetes release. A node image upgrade keeps the Kubernetes version but replaces the node OS image with a newer one containing OS and runtime security patches.

Many teams automate node images with the NodeImage or SecurityPatch auto-upgrade channel and reserve Kubernetes upgrades for planned windows.

A spot node pool runs on Azure Spot VMs, which are much cheaper but can be evicted with about 30 seconds' notice when Azure needs the capacity. Use them for interruption-tolerant work such as batch jobs, CI runners and stateless workers.

az aks nodepool add -g rg-demo --cluster-name aks-demo -n spotpool \
  --priority Spot --eviction-policy Delete --spot-max-price -1 \
  --enable-cluster-autoscaler --min-count 0 --max-count 10

AKS automatically taints these nodes with kubernetes.azure.com/scalesetpriority=spot:NoSchedule, so pods must carry a matching toleration. Spot pools cannot be the system pool, and critical or stateful services should stay on regular nodes.

A taint on a node repels pods that do not tolerate it. A toleration on a pod allows, but does not force, scheduling onto a matching tainted node.

Effects are NoSchedule (do not place new pods), PreferNoSchedule (try to avoid) and NoExecute (also evict running pods).

# taint at pool creation
az aks nodepool add ... --node-taints sku=gpu:NoSchedule

# pod spec
tolerations:
- key: sku
  operator: Equal
  value: gpu
  effect: NoSchedule

Because a toleration only permits placement, pair it with a nodeSelector or node affinity if the pod must land on that pool.

Neither wins outright, it depends on how much Kubernetes you actually need. Azure Container Apps (ACA) is a serverless platform built on Kubernetes, KEDA and Envoy that hides the Kubernetes API. AKS hands you the full API.

Pick ACA when you want containers running fast without cluster operations. Pick AKS when you need operators, custom networking, specific schedulers or portability of existing Kubernetes tooling.

Availability zones are physically separate datacenters in a region. Spreading nodes across them lets the cluster survive the loss of one zone.

• Create node pools with --zones 1 2 3. This must be set at pool creation.
• On Standard and Premium tiers, the control plane is also zone-redundant in supported regions.
• Spread replicas using topologySpreadConstraints on topology.kubernetes.io/zone.

Watch storage: a standard Azure Disk lives in one zone, so a pod using it cannot restart in another zone. Use ZRS disks or Azure Files ZRS for stateful workloads that must fail over.

The cluster autoscaler grows and shrinks existing node pools. Every pool has a fixed VM size, so you must design the pools in advance. Node auto-provisioning (NAP), built on Karpenter, looks at pending pods and creates nodes of whatever VM size fits best, without predefined pools.

NAP is attractive for mixed or bursty workloads. The classic autoscaler remains simpler and more predictable when you already run well-sized, purpose-built pools.

Traffic flows in both directions, using different paths. Kubelets and kube-proxy initiate outbound connections to the API server. The API server needs to reach kubelets for kubectl logs, exec and port-forward, so AKS uses a reverse tunnel.

The konnectivity-agent pods in kube-system open an outbound, authenticated tunnel to the managed control plane. The API server sends kubelet requests through it, so no inbound firewall holes are needed on your nodes.

flowchart LR
  K["Kubelet on node"] -->|outbound 443| API["API server"]
  AG["konnectivity-agent pod"] -->|outbound tunnel| CP["Control plane konnectivity-server"]
  API --> CP
  CP -->|logs, exec, port-forward| AG
  AG --> K

This is why locked-down clusters must allow outbound access to the API server FQDN and required Microsoft endpoints. With API Server VNet Integration, nodes reach the API server directly over the VNet instead.

AKS upgrades a node pool in a rolling fashion so pods keep running. The speed and capacity are controlled by max surge, which defaults to 10% of the pool for new pools (33% is a common production setting).

flowchart TD
  A["Start pool upgrade"] --> B["Create surge node with new version"]
  B --> C["Cordon an old node"]
  C --> D["Drain: evict pods respecting PDBs"]
  D --> E["Replace or reimage old node"]
  E --> F{More old nodes?}
  F -- Yes --> C
  F -- No --> G["Remove surge nodes, upgrade complete"]

1. Extra surge nodes are created first.
2. A node is cordoned so nothing new is scheduled on it.
3. It is drained; evictions respect PodDisruptionBudgets and termination grace periods.
4. The node is replaced and rejoins, and the loop repeats.

If a PodDisruptionBudget cannot be satisfied within the drain timeout (30 minutes by default), the upgrade fails. Use --max-surge, --drain-timeout and --node-soak-duration to tune it.

When the kubelet stops reporting, the node controller marks the node Unknown after roughly 40 seconds and adds the node.kubernetes.io/unreachable and not-ready taints.

Pods carry a default toleration for these taints of 300 seconds. After five minutes without recovery they are evicted, and their Deployments or ReplicaSets create replacements on healthy nodes. Services stop routing to endpoints whose readiness fails.

Stateful pods are slower to recover because Azure Disks must be detached from the dead node and attached to another one.

AKS node auto-repair also watches for this. If a node stays NotReady for around 10 minutes, it escalates through reboot, reimage and redeploy. Check kubectl describe node for conditions such as memory pressure, disk pressure or a kubelet crash.

Pending means the scheduler has not placed the pod yet. Start with the events, they almost always give the reason.

kubectl describe pod <pod> -n <ns>
kubectl get events -n <ns> --sort-by=.lastTimestamp

Look for the FailedScheduling message and map it to a cause:

If the cluster autoscaler should have reacted, check its status ConfigMap and logs, and confirm the pod fits a node of the pool's VM size at all.

Run kubectl describe pod and read the pull error. It usually points to one of four things: authorization, a wrong image reference, networking, or registry throttling.

1. Image name: confirm the registry, repository and tag exist (az acr repository show-tags).
2. Auth: run az aks check-acr -g rg -n aks --acr myregistry.azurecr.io, and confirm the kubelet identity holds AcrPull.
3. Network: with ACR firewall, private endpoint or UDR egress, check DNS resolution and rules from the node subnet.
4. Throttling: pulls from Docker Hub can hit rate limits, so import the image into ACR.

A 401 or 403 points to permissions, manifest unknown to a missing tag, and a timeout to the network path. Also check for an architecture mismatch, such as an arm64-only image on amd64 nodes.

Nodes are the biggest line item, so most savings come from running fewer, better-packed and cheaper nodes.

Enable the AKS cost analysis add-on to attribute spend to namespaces and workloads, so you know where to look first. Review it monthly, because new deployments quietly change the picture.

Run an independent cluster in each region (typically a paired region), and put a global entry point in front. Keep clusters stateless and rebuildable so any one of them can be lost.

flowchart TD
  U[Users] --> FD["Azure Front Door"]
  FD --> A["AKS East US"]
  FD --> B["AKS West US"]
  A --> DB[(Replicated data tier)]
  B --> DB
  ACR["ACR geo-replicated"] --> A
  ACR --> B
  G["Git repo with GitOps"] --> A
  G --> B

• Traffic: Azure Front Door or Traffic Manager with health probes for failover.
• Images: ACR Premium with geo-replication so each region pulls locally.
• Config parity: GitOps so both clusters apply the same manifests.
• Data: Cosmos DB multi-region writes or SQL failover groups, since Kubernetes does not replicate state.
• Fleet operations: Azure Kubernetes Fleet Manager to coordinate upgrades and propagate workloads.

Decide between active-active (lower failover time, higher cost) and active-passive (cheaper, slower to take over) based on your RTO and RPO.

With GitOps, a Git repository is the source of truth and an in-cluster agent continuously pulls and applies it. AKS offers Flux v2 as a managed extension (microsoft.flux).

flowchart LR
  Dev["Developer pushes to Git"] --> Repo[(Git repo)]
  Repo --> SC["source-controller fetches"]
  SC --> KC["kustomize-controller / helm-controller"]
  KC --> K["Cluster resources applied"]
  K --> D{Drift from Git?}
  D -- Yes --> KC

az k8s-configuration flux create -g rg-demo -c aks-demo -t managedClusters \
  -n platform --namespace flux-system --scope cluster \
  --url https://github.com/contoso/platform --branch main \
  --kustomization name=apps path=./apps prune=true

The source-controller fetches the repo, then the kustomize or helm controller renders and applies it on every interval. Because it reconciles, manual changes made with kubectl are reverted, and prune=true deletes objects removed from Git.

It takes several settings working together, not just a rolling update.

1. Run at least 2-3 replicas spread across nodes or zones.
2. Use a rolling strategy with maxUnavailable: 0 so capacity never drops.
3. Add a readiness probe so traffic reaches only pods that are ready.
4. Add a preStop sleep and a sufficient terminationGracePeriodSeconds so in-flight requests finish and endpoints are removed first.
5. Create a PodDisruptionBudget so node drains and upgrades never take down too many pods.

strategy:
  type: RollingUpdate
  rollingUpdate: {maxSurge: 1, maxUnavailable: 0}
template:
  spec:
    terminationGracePeriodSeconds: 45
    containers:
    - name: api
      readinessProbe:
        httpGet: {path: /ready, port: 8080}
      lifecycle:
        preStop:
          exec: {command: ["sleep", "10"]}

Without the preStop delay, a pod can receive requests after it starts shutting down, because endpoint removal and SIGTERM race each other.

Work from the pod outward: confirm the symptom, then check CoreDNS, then the upstream path.

1. Test from a pod: kubectl run dnstest --rm -it --image=registry.k8s.io/e2e-test-images/agnhost:2.39 -- nslookup kubernetes.default.
2. Check CoreDNS: kubectl get pods -n kube-system -l k8s-app=kube-dns, then read its logs for SERVFAIL or timeouts.
3. If only external names fail, look at the upstream: custom VNet DNS servers must forward to Azure DNS 168.63.129.16.
4. Check NSGs, firewalls or UDRs that block UDP/TCP 53 between nodes and the DNS servers.
5. Review /etc/resolv.conf in the pod; ndots:5 makes short names trigger many extra lookups.

Intermittent failures under load often mean the per-VM limit on Azure DNS (about 500 queries per second) is being hit. Deploying NodeLocal DNSCache and scaling CoreDNS reduces that pressure. Custom rules belong in the coredns-custom ConfigMap, not in edits to the managed CoreDNS config.