Cloud / AWS Systems Manager Interview questions

AWS Systems Manager (SSM) is an AWS service for operating the machines you run - EC2 instances, on-premises servers, edge devices and VMs - from one place, without logging in to each one.

It lets you run commands, patch operating systems, keep configuration consistent, store parameters and secrets, open shell sessions, collect inventory and run automated runbooks. Most of this works through the SSM Agent on each node, which talks to the Systems Manager service over HTTPS.

A short interview answer: it is the operations hub for your fleet, covering node management, change management, application management and operations management.

Systems Manager groups its tools by purpose. The ones you are most likely to be asked about are below.

All of them rely on the node being a managed node, meaning the SSM Agent is running and has permissions.

A managed node is any machine that Systems Manager can manage: an EC2 instance, an on-premises server, an edge device or a VM in another cloud or data center.

To qualify, a node needs three things: the SSM Agent installed and running, an IAM identity that allows it to call Systems Manager, and outbound HTTPS connectivity to the Systems Manager endpoints.

EC2 instances show up with IDs like i-0abc..., while hybrid nodes registered through an activation get IDs prefixed with mi-. If a node does not appear under Fleet Manager, one of those three requirements is usually missing.

SSM Agent is the lightweight software installed on a node that receives requests from Systems Manager, executes them and reports back. Run Command, Session Manager, Patch Manager and State Manager all depend on it.

It is preinstalled on Amazon Linux 2 and 2023, recent Ubuntu AMIs and Windows Server AMIs from AWS. On other systems you install the package yourself.

The agent only makes outbound HTTPS calls (port 443) to the ssm, ssmmessages and ec2messages endpoints, which is why no inbound firewall rule is needed.

sudo systemctl status amazon-ssm-agent
sudo tail -f /var/log/amazon/ssm/amazon-ssm-agent.log

The instance needs an instance profile whose role allows the SSM Agent to talk to Systems Manager. The simplest option is attaching the AWS managed policy AmazonSSMManagedInstanceCore.

That policy covers registration, receiving commands, Session Manager connections and basic messaging. Add more only when needed - for example S3 read access for patch or document downloads, or KMS access for encrypted parameters and session logs.

Note that the role is for the instance. The person or pipeline calling Systems Manager (for example running SendCommand) needs separate IAM permissions on their own identity.

Parameter Store is a Systems Manager capability for storing configuration data and secrets as named parameters - database URLs, feature flags, AMI IDs, license keys, passwords.

Values can be plain text or encrypted with KMS. You control access with IAM, and every change is versioned. Applications read parameters through the API or CLI instead of hard-coding values or baking them into images.

aws ssm put-parameter --name /myapp/prod/db/url --value "db.example.internal" --type String
aws ssm get-parameter --name /myapp/prod/db/url

The standard tier is free, which makes it a popular first choice for non-rotating configuration.

Parameter Store supports three types.

1. String - a plain text value such as a URL or a flag.
2. StringList - comma-separated values stored as one parameter, for example us-east-1,us-west-2.
3. SecureString - a value encrypted with a KMS key, used for passwords, tokens and keys.

For SecureString you can use the default aws/ssm key or a customer managed key. Reading the plain value requires the --with-decryption flag plus KMS decrypt permission.

Session Manager gives you an interactive shell (bash, PowerShell) on a managed node from the AWS console or CLI, with no SSH keys, no bastion host and no inbound ports.

Access is controlled by IAM, every session is tied to a user identity, and you can record sessions to S3 or CloudWatch Logs for audit. It also supports port forwarding to reach private databases or web apps.

aws ssm start-session --target i-0123456789abcdef0

The CLI needs the Session Manager plugin installed locally.

Run Command executes commands or scripts on one or many managed nodes remotely, safely and with an audit trail. You pick a document, choose targets, and Systems Manager handles delivery and status.

Typical documents are AWS-RunShellScript for Linux and AWS-RunPowerShellScript for Windows. Targets can be instance IDs, tags or resource groups, and you can limit blast radius with max concurrency and max errors.

aws ssm send-command --document-name AWS-RunShellScript \
  --targets Key=tag:Env,Values=staging \
  --parameters commands="uptime"

Output goes to the console, or to S3 or CloudWatch Logs if you need the complete text.

An SSM document is a JSON or YAML file that defines the actions Systems Manager performs on your nodes or resources. It is the "what to do" part of Run Command, State Manager, Automation and Session Manager.

Common document types are Command (Run Command and State Manager), Automation (runbooks), Session, Package (Distributor) and ChangeCalendar. AWS provides many documents prefixed AWS-, and you can author your own and share them with other accounts.

schemaVersion: '2.2'
description: Print hostname
mainSteps:
  - action: aws:runShellScript
    name: hostname
    inputs:
      runCommand:
        - hostname

State Manager keeps your nodes and AWS resources in a defined state through associations. An association links a document, a set of targets and a schedule.

For example, an association can run AWS-UpdateSSMAgent weekly on every node tagged Env=prod, or ensure an antivirus package stays installed. If the node drifts, the next scheduled run puts it back.

Results show up as association compliance, so you can see which nodes are compliant and which failed.

Patch Manager automates scanning for and installing operating system and application patches on managed nodes, using rules you define in a patch baseline.

It supports Windows, Linux distributions and macOS. You can scan only to report compliance, or install patches during a maintenance window, and view per-node results in the Patch Manager dashboard.

The work is executed by the AWS-RunPatchBaseline document, usually triggered by a maintenance window, State Manager association or patch policy.

A patch baseline is the rule set that decides which patches are approved for installation. Rules filter by classification (Security, Bugfix), severity and product, and can add an auto-approval delay in days.

You can also hard-list approved or rejected patches. AWS supplies predefined baselines per operating system, such as AWS-DefaultPatchBaseline, which cannot be edited. Create a custom baseline when you need stricter or slower approval rules.

Each node uses one baseline per operating system - the default unless its patch group is registered to a custom one.

Automation runs multi-step workflows called runbooks to perform common maintenance and deployment tasks on EC2 instances and other AWS resources - without scripting your own orchestration.

Each runbook is an Automation document made of steps like aws:executeAwsApi, aws:runCommand, aws:createImage, aws:approve and aws:branch. AWS provides runbooks for tasks such as restarting instances, creating AMIs, or remediating AWS Config findings.

Automation can be started from the console, CLI, EventBridge, a maintenance window or a State Manager association, and it runs with an AutomationAssumeRole.

Maintenance Windows let you schedule disruptive actions, such as patching or restarts, during times that are safe for your workloads.

A window has a schedule (cron or rate), a duration, a cutoff, registered targets and registered tasks. Task types include Run Command, Automation, Lambda and Step Functions.

Tasks run in priority order, and no new tasks start once the cutoff time is reached, so you avoid work spilling past the allowed period.

Inventory collects metadata from managed nodes and shows what is installed and how each node is configured. It gathers installed applications, AWS components, network configuration, Windows updates, services, and instance details, plus any custom inventory you define.

Collection is configured through a State Manager association using AWS-GatherSoftwareInventory. The data can be queried in the console or exported through a resource data sync to S3 for analysis with Athena.

Fleet Manager is the console experience for managing your managed nodes remotely. From one screen you can see node health, browse file systems, view logs and performance counters, manage users and groups, edit the Windows registry and start remote desktop sessions.

It is handy for ad-hoc troubleshooting where you want a quick look at a node without opening a separate SSH or RDP tool. It uses the same SSM Agent and IAM permissions as the rest of Systems Manager.

A hybrid activation lets you register non-EC2 machines, such as on-premises servers or VMs in another cloud, as managed nodes.

You create an activation with an IAM service role and get an activation code and ID. On each server you install SSM Agent and register it with those values. The activation expires after 24 hours by default and can be set up to 30 days, and you can cap how many machines register with it.

sudo amazon-ssm-agent -register -code &quot;<code>&quot; -id &quot;<id>&quot; -region &quot;us-east-1&quot;

Registered machines appear with mi- IDs.

Quick Setup is a guided way to configure common Systems Manager capabilities across accounts and Regions quickly, using best-practice defaults.

You pick a configuration type, such as host management, patch policy, or Explorer, then choose targets by tag or organizational unit. Quick Setup deploys the underlying associations and IAM roles using CloudFormation StackSets behind the scenes.

It is ideal when you want a working baseline for a new account or an entire AWS Organization without hand-building every association.

OpsCenter is a central place to view, investigate and resolve operational issues, which it tracks as OpsItems.

An OpsItem collects context such as the affected resource, related CloudWatch alarms, runbooks and similar past items. OpsItems can be created manually or automatically through EventBridge rules or alarms, and you can run Automation runbooks right from the item.

That means the on-call engineer sees the problem and the fix in one screen instead of jumping between consoles.

Both tiers store the same types of parameters, but advanced parameters raise limits and add features.

Parameter policies add expiration and notification rules, which only advanced parameters support. You can upgrade a standard parameter to advanced, but you cannot downgrade it back.

Session Manager reverses the connection direction. The SSM Agent on the node opens an outbound HTTPS connection to the Systems Manager service, and the shell traffic travels over that connection.

When you start a session, the service authenticates you with IAM, then relays the session data between your client and the agent through the ssmmessages endpoint. Nothing in your security group needs to allow port 22 or 3389.

sequenceDiagram
  participant U as User
  participant S as Systems Manager
  participant A as SSM Agent
  U->>S: StartSession (IAM authenticated)
  A->>S: Outbound HTTPS channel already open
  S-->>A: Session request
  U->>S: Shell input
  S->>A: Relay input
  A-->>U: Shell output via service

This also means private instances without public IPs are reachable if they can reach the endpoints or VPC endpoints.

The main advantages are security, auditability and less maintenance.

1. No inbound ports - no port 22 exposed, so nothing to scan or brute force.
2. No key management - no SSH keys to rotate or leak; access follows IAM.
3. Full audit trail - CloudTrail records who started a session, and session logs can go to S3 or CloudWatch Logs.
4. No bastion to patch - one less instance to run, harden and pay for.

You can restrict access with IAM conditions on tags, for example allowing engineers to connect only to Env=dev. If you still need SSH tooling, SSH can be tunneled through Session Manager.

There are two layers. CloudTrail always records the API calls such as StartSession and TerminateSession, showing who connected and when.

For the actual commands and output, enable session logging in the Session Manager preferences and choose a destination: an S3 bucket, a CloudWatch Logs log group, or both. You can encrypt the logs with KMS, and enable session data encryption for the stream itself.

The node's instance profile needs permission to write to the chosen bucket or log group, otherwise logging silently fails and sessions may be blocked if you require it.

Run Command is pull-based. You call SendCommand, Systems Manager records the request and queues it for the targets. Each SSM Agent polls the service, picks up its command, downloads and runs the document, and reports status back.

sequenceDiagram
  participant U as User
  participant S as Systems Manager
  participant A as SSM Agent
  U->>S: SendCommand (document, targets)
  A->>S: Poll for commands
  S-->>A: Command payload
  A->>A: Run document plugins
  A->>S: Status and output
  S-->>U: Invocation results

Each node gets its own command invocation with a status such as Pending, In Progress, Success, Failed or Timed Out. Large output is truncated in the console, so send it to S3 or CloudWatch Logs when you need all of it.

Targets can be given as explicit instance IDs, as tags (for example Key=tag:Env,Values=prod), or as an AWS Resource Group. Tag targeting is preferred because new instances are picked up automatically.

For rollout safety, set max concurrency (how many nodes at once, number or percentage) and max errors (how many failures before Systems Manager stops sending to more nodes).

aws ssm send-command --document-name AWS-RunShellScript \
  --targets Key=tag:Env,Values=prod \
  --max-concurrency 10% --max-errors 1 \
  --parameters commands="systemctl restart app"

With 10% concurrency and 1 error allowed, a bad script stops after hitting very few nodes.

Run Command is a one-time, on-demand action. State Manager is ongoing enforcement on a schedule.

Use Run Command to restart a service right now. Use State Manager to make sure the CloudWatch agent is always installed and running.

Both store sensitive values, but Secrets Manager is built for secrets that must be rotated, while Parameter Store is a broader, cheaper configuration store.

Many teams use both: Parameter Store for general config, Secrets Manager for credentials. Parameter Store can even reference a secret through the /aws/reference/secretsmanager/ path.

Create the parameter with --type SecureString, optionally naming your own KMS key with --key-id. The value is encrypted at rest and never shown in plain text unless the caller asks for decryption.

aws ssm put-parameter --name /myapp/prod/db/password \
  --value 'S3cr3t!' --type SecureString --key-id alias/myapp-key
aws ssm get-parameter --name /myapp/prod/db/password --with-decryption

The reader needs two permissions: ssm:GetParameter on the parameter ARN and kms:Decrypt on the key. With a customer managed key, the key policy must also allow that principal. Keep the value out of shell history by reading it from a file or stdin.

Use slash-separated names like /app/env/component/setting, for example /shop/prod/db/host. Hierarchies let you fetch and secure parameters as groups.

The GetParametersByPath API returns all parameters under a path, with --recursive to include deeper levels. IAM policies can also target a path with a wildcard, such as arn:aws:ssm:*:*:parameter/shop/prod/*, so the prod app role cannot read dev values.

aws ssm get-parameters-by-path --path /shop/prod --recursive --with-decryption

A hierarchy can be up to 15 levels deep.

Every time you overwrite a parameter, Parameter Store creates a new version and keeps the history (up to 100 versions). You can read an older one by appending the version to the name, such as /app/color:2.

Labels are friendly pointers attached to a version, like prod or stable. Reading /app/color:prod returns whichever version carries that label, so you can promote or roll back by moving the label instead of changing application code.

aws ssm label-parameter-version --name /app/color --parameter-version 3 --labels stable

Parameter policies are available only on advanced parameters. They attach time-based rules to a parameter.

1. Expiration - deletes the parameter at a set date and time.
2. ExpirationNotification - sends an EventBridge event a number of days before it expires.
3. NoChangeNotification - alerts you if the parameter has not been updated within a period.

This is useful for temporary credentials or tokens that must be rotated. Notifications arrive through Amazon EventBridge, which you can route to SNS, Lambda or OpsCenter.

Patch Manager looks at the node's operating system, finds its patch baseline (through patch group or the default) and evaluates available patches against that baseline.

A patch is installed if it matches an approval rule - by product, classification such as Security or CriticalUpdates, and severity - and the auto-approval delay has passed. Patches in the approved list are always included, and patches in the rejected list are excluded or only allowed as dependencies.

On Linux, the source is the node's configured package repositories. On Windows, it is Windows Update or WSUS.

The Operation parameter of AWS-RunPatchBaseline decides what the run does.

A common pattern is a frequent Scan association for visibility, plus a scheduled Install task in a maintenance window. The RebootOption value (RebootIfNeeded or NoReboot) controls restarts after install.

A patch group associates a set of nodes with a specific patch baseline. You do it by tagging nodes with the key Patch Group (or PatchGroup) and a value like web-prod, then registering that value with a baseline.

aws ssm register-patch-baseline-for-patch-group \
  --baseline-id pb-0123456789abcdef0 --patch-group web-prod

Nodes without the tag use the default baseline for their operating system. One node can belong to only one patch group, and a patch group can be registered to only one baseline per operating system. Patch policies in Quick Setup are the newer, organization-wide way to do the same job.

A typical production flow ties tags, baselines and a window together.

1. Tag nodes with a patch group, such as web-prod.
2. Create a custom baseline (for example Security patches, 7-day delay) and register it to the group.
3. Create a maintenance window for Sunday night and target nodes by the patch group tag.
4. Register a Run Command task with AWS-RunPatchBaseline, Operation Install.
5. Review compliance in the Patch Manager dashboard afterwards.

flowchart LR
  A["Tag nodes: Patch Group"] --> B["Custom patch baseline"]
  B --> C["Maintenance window targets tag"]
  C --> D["AWS-RunPatchBaseline Install"]
  D --> E["Compliance report"]

Use rate controls on the task so only a portion of the fleet reboots at once.

The duration is how long the window stays open, from 1 to 24 hours. The cutoff is how many hours before the end Systems Manager stops starting new tasks, from 0 to 23.

Example: a 4-hour window with a 1-hour cutoff accepts new tasks for the first 3 hours. Tasks that were already running when the cutoff arrives are allowed to continue until the window closes.

Set the cutoff longer than your slowest task so nothing begins that cannot finish inside the allowed period.

Choose a maintenance window when the work is disruptive, needs ordering, and must stay inside an approved time slot - patching with reboots, database upgrades, restarts of production services.

Choose a State Manager association when the action is safe to repeat and should keep running, such as updating the SSM Agent, ensuring software is installed, or gathering inventory.

Windows also give you task priority, multiple task types (including Lambda and Step Functions), and the cutoff safety net, which associations do not offer. Associations are simpler and also apply to newly launched matching nodes automatically.

When you start an Automation, Systems Manager creates an execution, assumes the AutomationAssumeRole and processes steps in order from the runbook's mainSteps.

Each step calls its action - an AWS API, a Run Command, a script, an approval - waits for the result, optionally validates it, then moves to the next step. A step can branch with aws:branch, pause for approval, or fail, in which case onFailure decides whether to abort, continue, or jump to another step.

flowchart TD
  A["Start execution"] --> B["Assume AutomationAssumeRole"]
  B --> C["Run step 1"]
  C --> D{Succeeded?}
  D -- Yes --> E["Next step"]
  D -- No --> F["onFailure: abort, continue or step"]
  E --> G["Execution complete"]

Step-level status and outputs are visible in the execution detail page.

Most runbooks combine a small set of actions.

You chain these with outputs from earlier steps, for example {{ createStep.ImageId }}, to build workflows like stop - snapshot - patch - start.

Add an aws:approve step before the risky action. The execution pauses at that step until the required approvers approve or deny, or until a timeout expires.

- name: approveRestart
  action: aws:approve
  timeoutSeconds: 3600
  inputs:
    NotificationArn: arn:aws:sns:us-east-1:111122223333:ops-approvals
    Message: Approve production restart?
    MinRequiredApprovals: 1
    Approvers:
      - arn:aws:iam::111122223333:role/OpsLead

An SNS notification tells approvers a decision is waiting. If anyone denies, the execution stops as failed. For formal change control across the organization, Change Manager adds templates and approval workflows on top of Automation.

The step's failure handling properties decide the outcome.

1. maxAttempts retries the step before it is marked failed.
2. timeoutSeconds fails a step that runs too long.
3. onFailure can be Abort (default), Continue, or step:stepName to jump to a cleanup or rollback step.
4. isCritical: false lets the overall execution succeed even when that step fails.

A good runbook uses onFailure to jump to a rollback step, for example restoring from the AMI it created, instead of leaving resources half-changed.

Automation supports multi-account and multi-Region executions through TargetLocations. You list account IDs (or organizational units) and Regions, and Systems Manager runs the runbook in each location.

In every target account, the execution assumes the role named by ExecutionRoleName, which must trust the administrator account's Automation role. Rate controls like MaxConcurrency and MaxErrors keep the rollout gradual.

aws ssm start-automation-execution --document-name MyRunbook \
  --target-locations "Accounts=111122223333,444455556666,Regions=us-east-1,eu-west-1,ExecutionRoleName=AutomationExecRole"

This is how organizations run remediation or AMI cleanup fleet-wide from one place.

Work through the three requirements in order.

1. Agent - confirm SSM Agent is installed and running, and check /var/log/amazon/ssm/amazon-ssm-agent.log for errors.
2. IAM - verify an instance profile with AmazonSSMManagedInstanceCore (or Default Host Management Configuration) is attached.
3. Network - make sure outbound 443 reaches the ssm, ssmmessages and ec2messages endpoints, through internet, NAT or VPC endpoints.
4. Time and region - check the system clock is correct and the console Region matches the instance.

The Systems Manager Automation runbook AWSSupport-TroubleshootManagedInstance automates several of these checks.

Create interface VPC endpoints so the SSM Agent can reach the service privately.

Enable private DNS on the endpoints and allow inbound 443 from the instances in the endpoint security group. Without S3 access some features, such as patching Amazon Linux repositories, will fail even if the agent registers.

Default Host Management Configuration (DHMC) lets Systems Manager manage your EC2 instances without attaching an instance profile to each one. You turn it on per account and Region, and Systems Manager uses the role AWSSystemsManagerDefaultEC2InstanceManagementRole to give instances baseline permissions.

It reduces setup friction, since you no longer need to remember to add AmazonSSMManagedInstanceCore to every role. If an instance does have its own profile, the profile's permissions take precedence.

It requires IMDSv2 and a recent SSM Agent, and it only covers EC2 instances, not hybrid nodes.

Register them as hybrid managed nodes.

1. Create an IAM service role that Systems Manager can assume for the nodes.
2. Create a hybrid activation and note the activation code and ID.
3. Install SSM Agent on each server and register it with the code, ID and Region.
4. Confirm the nodes appear in Fleet Manager with mi- IDs.

flowchart LR
  A["IAM service role"] --> B["Create hybrid activation"]
  B --> C["Install SSM Agent on server"]
  C --> D["Register with code and ID"]
  D --> E["Managed node mi-xxxx"]

Standard tier covers up to 1,000 hybrid nodes per account and Region. Beyond that, or to use Session Manager on non-EC2 nodes, you need the advanced-instances tier.

Inventory data lives per node in Systems Manager, which makes fleet-wide questions awkward. A resource data sync continuously copies the inventory from all your nodes, accounts and Regions into one S3 bucket.

From there you can query it with Athena and build dashboards in QuickSight. Questions like "which servers still run OpenSSL 1.0?" or "which instances lack the monitoring agent?" become simple SQL queries.

For organizations, the sync can aggregate data from every account in AWS Organizations. The bucket needs a policy allowing Systems Manager to write to it, and you can encrypt the data with KMS.

Scope what people and roles can do, and which resources they can touch.

1. Grant ssm:StartSession only on specific instances or by tag, using the condition ssm:resourceTag/Env.
2. Limit ssm:SendCommand to chosen documents, such as read-only diagnostics.
3. Restrict parameter access by path, for example parameter/shop/prod/*.
4. Control SecureString decryption with a KMS key policy.
5. Use Session Manager preferences to enforce logging, encryption and idle timeouts.

Pair this with CloudTrail so every API call is auditable. Avoid giving the instance profile broad S3 or admin rights just because SSM runs on it.

Each service has a native way to read parameters, so values stay out of templates and code.

1. CloudFormation - use a dynamic reference such as {{resolve:ssm:/shop/prod/db/host}}, or a parameter of type AWS::SSM::Parameter::Value<String>.
2. ECS - map a parameter to a container environment variable in the task definition using secrets with the parameter ARN.
3. Lambda - call GetParameter in code, cache the result outside the handler, or use the Parameters and Secrets Lambda extension.

Parameters:
  AmiId:
    Type: AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>
    Default: /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64

The task or function role needs ssm:GetParameter and, for SecureString, kms:Decrypt.