Testing / Mutation Testing Interview questions
Last updated
1. What is mutation testing?
Mutation testing is a technique that measures how good your test suite is by deliberately injecting small bugs into the code and checking whether the tests notice.
A tool makes a tiny change, for example turning a + b into a - b, and then re-runs the tests. If at least one test fails, the bug was caught. If every test still passes, the tests have a blind spot.
The result is a mutation score, which tells you the percentage of injected bugs your tests detected. Unlike line coverage, it checks whether tests actually assert behavior, not just execute code.
Take quiz
The execution speed of production code
The number of lines in the codebase
The ability of the test suite to detect injected bugs
The readability of test names
The production code is now optimized
The tests are very strong
The build tool is misconfigured
The tests have a gap in what they verify
2. What is a mutant in mutation testing?
A mutant is a copy of the program that contains exactly one small, automatically introduced change, such as flipping > to >= or removing a method call.
Each mutant represents a bug a developer could plausibly write. The tool runs the tests against every mutant separately, so one run can involve hundreds or thousands of mutants.
A mutant ends up either killed (a test failed) or survived (all tests passed).
Typical mutants include a flipped condition, a changed constant, a swapped operator, or a deleted statement. A tool might create dozens of mutants from a ten-line method, each one tested on its own.
Take quiz
Every operator in the method
One whole deleted class
A full rewrite of the module
One small change
Killed or survived
Compiled or linked
Merged or rebased
Cached or flushed
3. What is a mutation operator?
A mutation operator (also called a mutator) is a rule that describes how to change code to create a mutant. It defines both what to look for and what to replace it with.
For example, a relational operator replacement rule finds < and swaps it for <=, while a return-value rule replaces a returned object with null.
Different tools ship different operator sets, and most let you enable or disable them individually to control noise and run time.
Take quiz
A rule for turning a piece of code into a mutant
A way to schedule tests in parallel
A format for the HTML report
A method of measuring line coverage
A loop is unrolled
A returned object is replaced with null
A test is marked as skipped
A package is renamed
4. What does it mean to kill a mutant?
A mutant is killed when the test suite fails (or errors out) while running against it. The failure proves that some test is sensitive to the behavior that was changed.
Killing a mutant is the goal. The more mutants killed, the more confidence you have that the tests would catch a real regression in the same spot.
Tools usually count timeouts caused by the mutant, such as an infinite loop, as kills too.
A kill can also come from an exception or timeout thrown by the mutated code, not only from a failed assertion. What matters is that the suite reacted to the change, so a test that always passes regardless of the code kills nothing.
Take quiz
When the code compiles cleanly
When at least one test fails against it
When the mutant is deleted from disk
When all tests pass
It proves the code has no bugs
It speeds up the build
It shows the tests are sensitive to that behavior
It reduces the line count
5. What is a surviving mutant?
A surviving mutant is one where every test still passes even though the code was changed. It points to behavior that no test verifies properly.
There are three common reasons: a test is missing, a test runs the code but has a weak or absent assertion, or the mutant is equivalent and behaves the same as the original.
Survivors are the useful output of mutation testing. Each one is a to-do item: add a test, strengthen an assertion, or mark it as an acceptable equivalent.
Take quiz
Ignore it because tests passed
Delete the original code
Investigate whether a test or assertion is missing
Rerun the build until it is killed
The mutant is equivalent to the original
An assertion is too weak
No test covers that line
The mutation tool compiled the mutant too slowly
6. What is the mutation score?
The mutation score is the percentage of mutants your tests killed. The formula is:
score = killed mutants / (total mutants - equivalent mutants) x 100
In practice tools cannot detect equivalents automatically, so most report killed divided by total generated. A score of 80% means 8 of every 10 injected bugs were caught.
Treat it as a trend and a conversation starter, not a target to hit at any cost. Chasing 100% usually means writing tests for equivalent or trivial mutants.
Take quiz
16%
62.5%
40%
80%
Equivalent and trivial mutants make it costly or impossible
Tools cannot report scores above 90%
Higher scores slow down production code
Scores above 50% are invalid
7. What is the purpose of mutation testing?
The purpose of mutation testing is to test the tests. It tells you whether your suite would actually fail if someone broke the logic.
It exposes weak assertions, untested branches, and boundary conditions that coverage tools report as fully covered. It also helps teams decide where to invest more testing effort.
It does not find bugs in production code directly. It finds places where a bug could hide unnoticed.
Teams use it to rank where to add tests next: modules with many survivors are the weakest safety net. It is also a useful check on a refactoring plan, since a high score means the tests will likely flag accidental behavior changes.
Take quiz
Weaknesses in the test suite
Syntax errors in production code
Memory leaks at runtime
Unused dependencies
Slow database queries
Lines executed by tests that never assert on the result
Missing documentation
Unused import statements
8. What is a first-order mutant?
A first-order mutant contains a single change to the original program. It is the default in nearly every mutation tool.
Keeping to one change per mutant makes results easy to read: if a test fails, you know exactly which edit caused it. It also keeps the number of mutants manageable.
The idea relies on the coupling effect, which says that tests catching simple faults tend to catch more complex combined faults as well.
Example: from return a + b; a first-order mutant gives return a - b;. A second change in the same method would make it a second-order mutant, which most tools never create.
Take quiz
Exactly two
Exactly one
One per line of the file
As many as there are tests
They always survive
They need no test execution
Results are easy to trace and the mutant count stays manageable
They only work for Java
9. What is a higher-order mutant?
A higher-order mutant combines two or more changes in the same program, for example flipping a condition and also removing a statement.
The number of possible combinations grows explosively, so these are used mostly in research. Some studies use them to build harder-to-kill mutants or to find combinations that simulate more realistic faults.
Most day-to-day tools skip them because first-order mutants already give good signal at far lower cost.
Example: a mutant that both changes > to >= and removes a following break. Researchers study such pairs because two faults can mask each other, which makes the combination harder to kill than either alone.
Take quiz
It is generated at runtime only
It only mutates test code
It applies two or more changes together
It can never be killed
They are illegal in most languages
They do not compile
They are identical to equivalent mutants
The number of combinations grows very quickly
10. What is an equivalent mutant?
An equivalent mutant changes the code but not the observable behavior, so no test can ever kill it.
Example: in for (int i = 0; i < 10; i++), changing the condition to i != 10 yields identical behavior. In another case, replacing x * 1 with x / 1 also changes nothing.
They are the main source of false alarms. Detecting them automatically is undecidable in general, because it reduces to proving two programs equivalent, so teams review survivors manually.
Take quiz
It never gets compiled
It is stored outside the repository
It only runs in production
Its observable behavior is identical to the original
Proving program equivalence is undecidable in general
Equivalent mutants are always timeouts
Tools do not read source code
Every language forbids it
11. What are the types of mutation operators?
Operators are usually grouped by what they change:
- Arithmetic:
+becomes-,*becomes/. - Relational / conditional boundary:
<becomes<=,==becomes!=. - Logical:
&&becomes||, a condition is negated. - Return value: a returned value is replaced with
null,0, orfalse. - Statement / method call removal: a void call or whole statement is deleted.
- Unary and increment:
i++becomesi--, a sign is flipped.
Exact names differ by tool, but the categories are widely shared.
Take quiz
Logical operator replacement
Arithmetic operator replacement
Return value mutation
Increment mutation
Renames the method
Deletes the call so its side effect never happens
Converts it to a static method
Adds a second call
Relational mutators
Return value mutators
Unary and increment mutators
Statement removal mutators
12. List popular mutation testing tools?
Most mainstream languages have a mature tool:
| Language | Tool |
| Java / JVM | PIT (Pitest), Major |
| JavaScript / TypeScript | StrykerJS |
| C# / .NET | Stryker.NET |
| Scala | Stryker4s |
| Python | mutmut, Cosmic Ray |
| PHP | Infection |
| Ruby | Mutant |
| Rust | cargo-mutants |
| Go | go-mutesting |
| C / C++ | Mull |
For a first project, pick the tool that integrates with your existing build, such as the PIT Maven plugin or Stryker with npm.
Check how actively a tool is maintained and whether it supports your test runner before picking it. For JVM projects PIT has plugins for JUnit 5, TestNG, and Gradle, and for JavaScript StrykerJS supports Jest, Mocha, Karma, and Vitest.
Take quiz
Mutmut
PIT (Pitest)
Infection
Mull
Stryker.NET
cargo-mutants
Infection
go-mutesting
C and C++
Rust
Ruby
JavaScript and TypeScript
13. What is PIT (Pitest)?
PIT is the most widely used mutation testing system for Java and other JVM languages. It is fast because it mutates bytecode in memory rather than recompiling source for every mutant.
It runs only the tests that cover a mutated line, uses multiple threads, and produces an HTML report showing which mutants were killed or survived, line by line.
It integrates with Maven, Gradle, and Ant. JUnit 5 projects need the separate pitest-junit5-plugin.
A minimal run reports line coverage, mutation coverage, and test strength. Typical statuses in the report are KILLED, SURVIVED, NO_COVERAGE, TIMED_OUT, and NON_VIABLE, and the HTML view highlights each line that has survivors.
Take quiz
Raw source files on disk
Database schemas
Bytecode, in memory
Network packets
pitest-spring-boot
junit-vintage-only
mockito-pit
pitest-junit5-plugin
14. What is Stryker Mutator?
Stryker is a family of mutation testing tools: StrykerJS for JavaScript and TypeScript, Stryker.NET for C#, and Stryker4s for Scala.
It runs your existing test runner (Jest, Mocha, Karma, xUnit, NUnit and others) against each mutant and reports the results in the terminal and as an HTML report. Configuration lives in a file such as stryker.config.json.
It supports score thresholds, per-test coverage analysis, and incremental runs, which makes it practical for CI.
A typical command is npx stryker run after npx stryker init creates the config. The HTML report highlights killed, survived, and no-coverage mutants directly on your source files.
Take quiz
Only Java
Only Python
Only Go and Rust
JavaScript/TypeScript, C#, and Scala
In a config file like stryker.config.json
In the production source code
In the Git commit message
In the browser cache
15. What is the competent programmer hypothesis?
The competent programmer hypothesis assumes developers write programs that are almost correct, so real faults are usually small slips such as an off-by-one or a wrong operator.
This is the reason mutation testing uses small syntactic changes. If real bugs are small deviations from correct code, then small mutants are a realistic stand-in for them.
It is an assumption, not a proven law, and it does not cover big design errors or missing features.
For example, a developer might write i <= items.length instead of i < items.length. Mutation operators are modeled on slips like this, which is why a boundary mutator exists at all.
Take quiz
Real faults are mostly small deviations from correct code
Programmers never write bugs
Most bugs come from hardware
Large rewrites are safer than small fixes
Off-by-one errors
Missing features and large design errors
Wrong comparison operators
Swapped arithmetic signs
16. What is the coupling effect in mutation testing?
The coupling effect says that a test set able to detect simple faults will also detect most complex faults, because complex faults are made of simple ones coupled together.
That is why tools bother with only first-order mutants. If your tests kill all the single-change mutants, they are likely to catch combined changes too.
Empirical studies have generally supported this, though not perfectly.
In practice it lets a tool generate a few hundred single-change mutants instead of billions of combinations. If a suite kills a + b to a - b and a separate flipped condition, it will probably also detect both faults occurring together.
Take quiz
Skipping unit tests
Using mostly first-order mutants
Running mutation tests only in production
Deleting equivalent mutants by hand
Always unrelated to simple faults
Impossible to simulate
Combinations of simple faults
Found only by fuzzing
17. What is weak mutation testing?
In weak mutation, a mutant counts as killed as soon as the internal program state differs from the original right after the mutated statement runs. The change does not have to reach the final output.
This is cheaper, because you do not need to run a full test to the end, but it can overstate the score. A state difference might get overwritten or masked later, so no real failure would ever be visible.
It contrasts with strong mutation, where the output or test result must differ. Most practical tools use strong mutation.
Take quiz
Only when the process crashes
Only when the output file is deleted
When the internal state differs right after the mutated statement
When the compiler warns
It cannot run on small programs
It needs more mutants
It requires manual review of every mutant
A state difference may be masked later and never affect output
18. What is strong mutation testing?
In strong mutation, a mutant is killed only when the program's observable result (output, return value, or a test outcome) differs from the original.
It matches what you actually care about: would a user or a test see the difference? That makes it more reliable than weak mutation, but costlier, because each mutant must be run through the tests to completion.
PIT, Stryker, mutmut and Infection all use strong mutation.
Strong mutation is the safer measure when you need to know that a bug would really be noticed. The practical cost is that every mutant needs real test execution, which is why scoping and incremental runs matter so much.
Take quiz
Only a local variable in memory
The file timestamp
The commit hash
The observable result or test outcome
It is costlier but closer to real failure detection
It never needs tests to run
It is used only in academic papers
It cannot be applied to unit tests
19. What is a stillborn (non-viable) mutant?
A stillborn mutant is a mutant that fails to compile, or is rejected at load time, so it can never run. A mutation that produces a type error is a typical example.
Such mutants add no information, so they should be excluded from the score. Tools handle this differently: StrykerJS and Stryker.NET report a CompileError status, and PIT labels them NON_VIABLE.
Good operators are designed to avoid generating these in the first place.
For instance, replacing a String return with 0 in a typed language can break compilation. Always check that the stillborn count is excluded from the denominator, otherwise the score looks worse than it is.
Take quiz
It fails to compile or load and never runs
It always survives
It is counted as killed
It runs twice
SURVIVED
NON_VIABLE
KILLED
TIMED_OUT
20. How do you run PIT with Maven?
Add the pitest-maven plugin to your pom.xml and run the mutationCoverage goal.
<plugin> <groupId>org.pitest</groupId> <artifactId>pitest-maven</artifactId> <version>LATEST_VERSION</version> <configuration> <targetClasses><param>com.acme.shop.*</param></targetClasses> <targetTests><param>com.acme.shop.*</param></targetTests> </configuration> </plugin>
mvn org.pitest:pitest-maven:mutationCoverage
The HTML report is written under target/pit-reports. Replace LATEST_VERSION with the current release, and add the JUnit 5 plugin as a plugin dependency if you use JUnit 5.
If the run finds nothing to mutate, check that targetClasses matches your package names and that tests have been compiled first. For Gradle, the equivalent is the pitest plugin and the pitest task.
Take quiz
mutationPackage
mutationCoverage
pitDeploy
surefire:mutate
src/main/resources
~/.m2/repository
target/pit-reports
target/classes
21. Why is mutation testing slower than a normal test run?
Each mutant needs its own test execution. If a project produces 2,000 mutants and the relevant tests take 10 seconds, that is potentially hours of work compared with one 10-second run.
The cost comes from three factors: the number of mutants, the tests executed per mutant, and the setup per run, such as JVM startup or module loading.
Tools reduce this by running only tests that cover the mutated line, stopping at the first failing test, running in parallel, and skipping unchanged code in incremental mode.
As a rough example, 5,000 mutants at 2 seconds each is almost three hours on a single thread. With 8 threads and per-test coverage it can drop to minutes, which is why tuning matters more than hardware.
Take quiz
The tools disable caching of all kinds
Reports are written in PDF
Tests must be re-run for each mutant
Mutants are sent to a remote server
Running tests in random order
Adding more assertions
Using bigger mutants
Running only tests that cover the mutated line
22. Why does 100% line coverage not guarantee strong tests?
Coverage measures which lines ran, not whether any test checked the outcome. A test can execute every line and assert nothing.
@Test void checkoutRuns() { orderService.checkout(cart); // covers lines, asserts nothing }
Every mutant inside checkout would survive this test, yet coverage reports 100%. Even with assertions, boundary values and negative paths can be missed.
Mutation testing closes that gap by asking whether a changed line makes any test fail.
The reverse is a useful reading too: a suite with moderate coverage but strong assertions can score better on mutation than one with 100% coverage and no checks. That is why teams look at both numbers together.
Take quiz
Coverage tools remove the mutants
The cart is always empty
checkout is a private method
The test runs the code but asserts no outcome
Which lines were executed during tests
Whether assertions are correct
How many bugs were fixed
How fast tests run
23. What is the difference between code coverage and mutation score?
| Code coverage | Mutation score |
| Measures which code was executed by tests. | Measures how many injected bugs tests detected. |
| Cheap and quick to compute. | Expensive, needs many test runs. |
| Can be high with no assertions. | Drops when assertions are weak. |
| Tells you what is definitely untested. | Tells you how well the tested code is verified. |
They complement each other. Low coverage means large untested areas. High coverage with a low mutation score means tests that run code but verify little.
A practical rule: use coverage to find untested code and mutation score to judge weakly tested code. Neither replaces review of whether tests match the requirements.
Take quiz
Code coverage
Mutation score
Both always drop to zero
Neither one
Tests are too fast
Tests execute code but assert too little
Mutants are not compiled
Coverage is measured incorrectly by definition
24. How does mutation testing work step by step?
A mutation run follows the same loop for every mutant:
- Run the test suite on the original code. It must be green, otherwise results are meaningless.
- Find mutation points and generate mutants using the enabled operators.
- For each mutant, run the relevant tests.
- Mark it killed if a test fails or times out, or survived if all pass.
- Calculate the score and produce a report listing survivors.
- Strengthen the tests and repeat.
flowchart TD
A["Run tests on original code"] --> B{All green?}
B -- No --> X["Fix tests first"]
B -- Yes --> C["Generate mutants"]
C --> D["Run tests per mutant"]
D --> E{Any test fails?}
E -- Yes --> F[Killed]
E -- No --> G[Survived]
F --> H["Compute score and report"]
G --> H
Take quiz
Mutants only compile after a green build
Otherwise failing tests would falsely count as kills
The report requires a passing build number
Tools cannot read failing tests
One test is skipped
The report shows a warning
Every test passes against the mutated code
The compiler prints a note
25. How does PIT generate mutants at the bytecode level?
PIT loads your compiled .class files and uses the ASM library to walk through the bytecode instructions. When it finds an instruction that a mutator targets, such as IF_ICMPLT for a < comparison, it creates a modified version of the class in memory.
The modified class is loaded by a custom class loader in a separate JVM, so the original files on disk are never touched and no recompilation is needed. That is the main reason PIT is much faster than source-level tools.
Because it works on bytecode, a mutation maps to a line number through debug info. Compiler-generated code can occasionally create mutants that look odd in source form.
Take quiz
Lombok
Hibernate
ASM
Jackson
It skips running tests
It generates fewer operators by default only
It uses remote build servers
It avoids recompiling source for each mutant
In memory inside a separate JVM
Overwritten in the src folder
In the Git history
In the user's home folder
26. How can you handle equivalent mutants?
There is no perfect automatic method, so teams combine a few practical approaches:
- Review manually. Look at each survivor and decide whether any input could expose a difference.
- Exclude noise. Disable mutators or code regions that tend to produce equivalents, such as logging or toString methods.
- Use disable comments. Stryker supports
// Stryker disable next-line all, PIT supports excluding methods or classes. - Refactor. Often an equivalent mutant signals redundant code that can be simplified.
Accept that a few remain. Track the trend of survivors instead of demanding zero.
Common examples are mutations in loop conditions that only change the number of redundant iterations, or in logging and metrics calls. Document why a mutant is ignored so a reviewer can challenge it later.
Take quiz
A broken build server
A missing license file
A slow database
Redundant or dead code that could be simplified
A disable comment such as // Stryker disable next-line all
Deleting the stryker config file
Renaming the test
Adding more mutators
27. How can you optimize mutation testing run time?
The aim is to run fewer mutants and fewer tests per mutant.
- Scope it: limit target classes or files to the module you care about, or to files changed in a pull request.
- Go incremental: reuse previous results for unchanged code (PIT
withHistory, Stryker--incremental). - Parallelize: raise the thread or concurrency count.
- Use per-test coverage: run only tests that touch a mutant (Stryker
coverageAnalysis: "perTest"). - Prune operators: disable low-value mutators.
- Speed up the tests: fast unit tests help far more than any flag.
For nightly builds, run the full suite. For pull requests, run the diff only.
Take quiz
Re-testing mutants in code that has not changed
Compiling the source
Running the original tests
Writing the report
thresholds.break
coverageAnalysis set to perTest
reporters.html
mutate.exclude
28. What is the difference between mutation testing and fault injection?
| Mutation testing | Fault injection |
| Changes the code under test. | Changes the environment or runtime behavior. |
| Goal: evaluate the quality of the test suite. | Goal: evaluate the resilience of the system. |
Examples: flip > to >=, remove a call. |
Examples: drop network packets, throw an exception, fill the disk. |
| Runs during development and CI. | Often runs in staging or chaos experiments. |
They answer different questions: "Would my tests catch this bug?" versus "How does my system behave when something fails?"
Tools like Chaos Monkey or Toxiproxy belong to the fault injection family. Mutation tools like PIT and Stryker belong to the other, and a mature pipeline may use both at different stages.
Take quiz
Fault injection
Mutation testing
Load testing
Smoke testing
Mutation testing
Static analysis
Fault injection
Code coverage
29. When should you run mutation testing in CI?
Running the full mutation suite on every commit is usually too slow. A layered approach works better.
- Pull requests: run only on changed files, with a modest threshold that fails the build if the score drops.
- Nightly or weekly: run the full suite and publish the report.
- Before releases: review survivors in critical modules.
flowchart LR
A["Pull request"] --> B["Unit tests"]
B --> C["Mutation on changed files"]
C --> D{Score above threshold?}
D -- Yes --> E[Merge]
D -- No --> F["Add or strengthen tests"]
G["Nightly job"] --> H["Full mutation run and report"]
Make sure unit tests pass first, since mutation on a red build wastes time.
Take quiz
The entire codebase every time
Only the README
Only the files changed in the PR
Only third-party libraries
Inside every developer's IDE save action
Only after production incidents
Never in automation
A nightly or weekly scheduled job
30. How does incremental analysis work in mutation testing?
Incremental analysis stores the result of each mutant along with a fingerprint of the code and tests it depended on. On the next run, if neither the mutated class nor the covering tests changed, the old result is reused.
In PIT, you enable it with the withHistory option, which writes a history file. StrykerJS offers --incremental and stores results in a JSON file such as reports/stryker-incremental.json.
Only mutants affected by changed code or changed tests are re-run. This can cut run time dramatically on pull requests, but you must persist the history file between CI runs, for example via a cache.
Take quiz
The mutant has a short name
The mutant was created on a Monday
The build is running on Linux
Neither the mutated code nor the covering tests changed
Persist the history or incremental file between runs
Delete the file before each run
Commit the HTML report
Disable all tests
31. How do you troubleshoot surviving mutants?
Open the report, pick a survivor, and work through these checks in order:
- Is it covered? If no test runs that line, add one.
- Is the assertion too weak? A test that runs the code but only checks
assertNotNulllets many return-value mutants live. - Is a boundary missing? Add tests at the edge values (for example exactly 10 for
> 10). - Is it equivalent? Try to find any input that behaves differently. If none exists, suppress it.
- Is the code dead or redundant? Remove it.
Start with survivors in business-critical logic, and ignore noise in logging or generated code.
Take quiz
Whether any test actually covers the mutated line
Whether the file name is long
Whether the mutant uses a lambda
Whether the IDE theme is dark
Only compile errors
Return-value and logic mutants
None, it is a perfect test
Only mutants in comments
32. Why do boundary mutants often survive?
A boundary mutant swaps > for >=, or < for <=. It only changes behavior when the input is exactly on the boundary, which tests often skip.
int discount(int qty) { return qty > 10 ? 20 : 0; } // Tests use 15 and 5 only. // Mutant: qty >= 10 behaves the same for both, so it survives.
To kill it, add a test for qty == 10 that expects 0. Boundary value analysis, testing just below, on, and just above each limit, is the standard fix.
The same applies to off-by-one issues in loops and index checks. Parameterized tests are an easy way to cover several boundary values with little extra code.
Take quiz
qty equal to 15
qty equal to 10
qty equal to 5
qty equal to 100
The inputs are too small to compile
Mutants only apply to strings
Both inputs behave identically before and after the mutation
The method is static
33. Why should you target mutation testing at changed code only?
Running every mutant across the whole repo is costly and the results are hard to act on. Developers rarely fix survivors in code they did not touch.
Diff-based mutation puts the feedback exactly where it is useful: in the pull request, for lines the author just wrote and still remembers. It also keeps run times within minutes.
In PIT this can be done with the SCM plugin goal scmMutationCoverage. Stryker offers file-level selection through --mutate globs or incremental mode. Large organizations also surface survivors directly as review comments.
A reasonable setup fails the pull request only when the score on changed lines drops below an agreed value, while the full-project score is tracked on a dashboard instead of gating merges.
Take quiz
It generates more mutants
It replaces unit tests
Fast feedback on code the author just changed
It removes the need for CI
mutationCoverage only
scmCommit
deployChanged
scmMutationCoverage
34. How does Stryker's per-test coverage analysis speed up runs?
With coverageAnalysis: "perTest", StrykerJS first does a dry run that records which tests execute which mutated locations. For each mutant it then runs only the tests that touched that location, instead of the whole suite.
{ "testRunner": "jest", "coverageAnalysis": "perTest" }
Mutants that no test reaches are reported as NoCoverage immediately without running anything. The cost is a slightly longer dry run, and it needs a test runner that supports per-test reporting, such as Jest, Mocha, Karma, or Vitest. The alternative "all" setting runs every test that covered the file at all, which is less precise.
Example: if a mutant sits in a function covered by only 2 of 400 tests, Stryker runs just those 2 tests. The speed-up grows as the suite grows, since the number of tests per mutant stays small.
Take quiz
Runs the full suite anyway
Deletes the source file
Counts it as killed
Marks it NoCoverage without running tests
A longer dry run in exchange for faster mutant runs
No dry run but slower mutants
It disables the HTML report
It works only for CSS
35. What is the difference between PIT's DEFAULTS and STRONGER mutator groups?
DEFAULTS is the conservative group PIT uses unless you say otherwise. It includes conditional boundary, increments, invert negatives, math, negate conditionals, void method call removal, and several return-value mutators.
STRONGER builds on DEFAULTS and adds extra mutators, notably REMOVE_CONDITIONALS and the experimental switch mutator. It generates more mutants and catches more weak spots, at a higher runtime cost and with a slightly higher chance of equivalents.
There is also an ALL group for everything available. Pick the group in the plugin settings:
<mutators> <mutator>STRONGER</mutator> </mutators>
Start with DEFAULTS, then move to STRONGER once the score stabilizes.
Mutators in DEFAULTS include CONDITIONALS_BOUNDARY, MATH, and VOID_METHOD_CALLS. You can also list mutators individually instead of choosing a group, which is helpful when one operator creates too much noise.
Take quiz
DEFAULTS
ALL
STRONGER
EXPERIMENTAL_ONLY
Fewer mutants and faster runs
More mutants and run time for deeper checks
No change in mutant count
The report format changes to PDF
36. How do you set a mutation threshold in PIT?
Use mutationThreshold to make the build fail if the mutation score drops below a number. You can also set coverageThreshold for line coverage.
<configuration> <mutationThreshold>75</mutationThreshold> <coverageThreshold>80</coverageThreshold> </configuration>
If the score is 72, the Maven build fails with a message saying the threshold was not met. Set it to your current baseline so it blocks regressions, then raise it gradually.
The check runs at the end of the analysis, so you still get the HTML report even when the build fails. In a multi-module build, set the threshold per module because scores often vary widely between modules.
Run mvn verify with the plugin bound to the verify phase, if you want the gate to run as part of the normal build.
Take quiz
The tests are deleted
The build fails
The report is skipped silently
PIT lowers the threshold
100
0 forever
The current baseline score
A random value
37. How do you configure score thresholds in Stryker?
StrykerJS has three thresholds in the thresholds block:
{ "thresholds": { "high": 80, "low": 60, "break": 50 } }
- high: scores at or above this are shown green.
- low: scores below this are shown red, and between low and high is yellow/warning.
- break: if the score falls below it, Stryker exits with a non-zero code, which fails the CI job.
The defaults are high 80, low 60, and break unset. Only break affects the exit code, the other two just color the report.
Pick break a few points below the current score, to absorb normal variation, and raise it over time. A break that is set too high will block merges for noise rather than quality.
Take quiz
high
low
break
report
The number of threads
The test timeout
The file glob for mutation
The color shown for good scores in the report
38. How does PIT handle mutants that cause infinite loops?
A mutation like removing i++ in a while loop can make the code spin forever. PIT guards against this with a timeout calculated from how long the tests took on unmutated code.
The formula is roughly normal time x timeoutFactor + timeoutConstant. The defaults are a factor of 1.25 and a constant of 4000 ms. A mutant that exceeds it is marked TIMED_OUT and counted as killed, since a real regression like that would be noticed too.
On slow or shared CI machines, raise timeoutConstant to avoid false timeouts.
Typical causes are mutations on loop counters, such as flipping i++ or inverting a loop condition. A high count of timeouts is normal and not a problem, but a sudden jump can signal an overloaded build agent.
Take quiz
SURVIVED
NON_VIABLE
NO_COVERAGE
TIMED_OUT, counted as killed
timeoutConstant
mutationThreshold
targetClasses
outputFormats
A fixed one hour limit
The test time measured on the unmutated code
The size of the repository
The developer's laptop speed
39. How do mocks affect mutation testing results?
Heavy mocking can hide real logic from the tests. If a collaborator is mocked, mutants inside the real collaborator are never exercised by that test and may show up as NoCoverage or survive.
Tests that only verify interactions, for example that repo.save() was called once, tend to let arithmetic and condition mutants survive, because they never check the values produced.
On the other side, interaction checks are what kill void method call removal mutants. A good balance is to assert on results and state wherever possible, and use verify for side effects that matter.
Example: a service test with a mocked repository checks only that save was called. A mutant that changes the price calculation before saving survives, so add an ArgumentCaptor or equivalent to assert the saved value.
Take quiz
Void method call removal
Arithmetic replacement
Boundary change
Increment mutation
Mocks delete the mutants
Real logic behind the mock is never exercised or asserted
Mocking disables the JVM
Mutants cannot be created in tests
40. Why do flaky tests break mutation testing?
Mutation testing treats any test failure as a kill. A flaky test that fails randomly will "kill" mutants for no reason, so the score is inflated, and you lose trust in the numbers.
Tools try to protect against this by running the suite once on clean code first and aborting when it fails. That does not catch tests that pass most of the time and fail occasionally, such as timing-dependent or order-dependent ones.
Fix or quarantine flaky tests before adopting mutation testing. If a score swings between runs on the same code, that is a hint to look for flakiness.
Take quiz
They lower the score every time
Random failures falsely kill mutants
They make all mutants survive
They delete reports
The report is too colorful
The build is faster than usual
The score changes between runs on identical code
Coverage reaches 100%
41. How do you introduce mutation testing in a legacy project?
Do not start with the whole codebase. A gradual rollout avoids a wall of red survivors.
- Pick one important module with reasonably fast unit tests.
- Run once and record the baseline score.
- Set the break threshold at that baseline so it can never get worse.
- Run diff-only mutation on pull requests so new code is held to a higher standard.
- Fix survivors in the highest-risk logic first, then raise the threshold in small steps.
- Expand to other modules over time.
This approach, often called a ratchet, gives steady improvement without blocking existing work.
Agree on the process with the team up front, including who triages survivors and how often. Without clear ownership, the report is read once and forgotten.
Take quiz
To delete old tests
To publish it as a release note
To set a threshold that prevents regressions
To reset coverage to zero
Rewriting all tests at once
Disabling the unit tests
Raising the threshold to 100 immediately
Running mutation only on changed code in pull requests
42. What is the difference between mutation testing and property-based testing?
| Mutation testing | Property-based testing |
| Evaluates the tests. | Generates inputs to test the code. |
| Modifies the code to see if tests fail. | Runs the code with many random inputs against stated properties. |
| Output: mutation score and survivors. | Output: pass, or a minimal failing example. |
| Examples: PIT, Stryker. | Examples: jqwik, Hypothesis, fast-check. |
They work well together. Property-based tests often kill more mutants than a handful of example-based tests, and mutation testing can show whether your properties are strong enough.
Example: a jqwik property such as "reversing a list twice returns the original" kills many mutants in reverse logic that a single example test would miss. Mutation testing then confirms that the property is actually strong.
Take quiz
Mutation testing
Code coverage
Static analysis
Property-based testing
Use mutation testing to check how strong your properties are
Use property tests to compile mutants
Replace production code with properties
Run them only in production
43. What is mutant subsumption?
Mutant A subsumes mutant B if every test that kills A also kills B. In that case B is redundant, since killing A already tells you everything B would.
The mutants that are not subsumed by any other are called dominator mutants. A test suite that kills all dominator mutants kills all mutants.
This matters for two reasons. First, the mutation score can be inflated by lots of easy, redundant mutants. Second, tools can save time by generating or running fewer, more informative mutants. Finding true subsumption is expensive, so in practice it is approximated using heuristics or test results.
Take quiz
Any test killing A also kills B
B always survives
A and B are equivalent mutants
Tests must kill B first
They cause compile errors
Easy duplicate mutants inflate the kill rate
They disable the harder mutants
They always time out
44. How does predictive mutation testing work?
Predictive mutation testing (PMT) uses machine learning to guess whether a mutant will be killed without actually running the tests. It trains a model on features of earlier runs, such as the type of mutation, how many tests cover that code, and code complexity.
After training, the model predicts a result for each new mutant in a fraction of the time. Research reports good accuracy, which makes it attractive for big projects where full runs are too slow.
The result is still an estimate. It works best as a fast screening step to spot likely survivors, followed by real execution for confirmation.
Take quiz
Writing source code
Actually running tests for every mutant
Compiling the project
Reading the documentation
The developer's name
The Git branch color
The number of tests covering the mutated code
The IDE version
45. How do you interpret a mutation testing report?
Start with the summary: total mutants, killed, survived, no coverage, and the score. PIT also shows test strength, which is killed divided by covered mutants only, so it separates "not tested" from "tested but weak".
Then drill into the per-file view. Green lines were killed, red lines have survivors, and clicking a line lists the exact mutations that survived.
- NoCoverage: write a test.
- Survived: strengthen an assertion or add a boundary case.
- Timeout: usually fine, counted as killed.
Prioritize by business risk, not by count. A survivor in payment logic is worth more than ten in a logging helper.
Take quiz
Total lines of test code
The speed of the slowest test
Kills as a share of covered mutants only
Memory usage during the run
Ignore it, it is harmless
Increase the timeout
Switch to weak mutation
Write a test that exercises that code
46. How do you exclude code from mutation in PIT and Stryker?
Excluding noise keeps the score meaningful. Typical candidates are logging, generated code, and toString or equals boilerplate.
PIT (Maven configuration):
<excludedClasses><param>com.acme.generated.*</param></excludedClasses> <excludedMethods><param>toString</param></excludedMethods> <avoidCallsTo><avoidCallsTo>org.slf4j</avoidCallsTo></avoidCallsTo>
StrykerJS: use mutate globs with ! to exclude files, mutator.excludedMutations to disable operators such as StringLiteral, or inline comments like // Stryker disable next-line all.
Always leave a short reason next to each exclusion, so it can be reviewed later.
Use these options sparingly. Excluding too much makes the score look better without improving safety, so review the exclusion list during code review, just like production code.
Take quiz
excludeLogs
skipSlf4j
mutateLess
avoidCallsTo
A comment such as // Stryker disable next-line all
Renaming the file to .bak
Setting thresholds.high to 0
Deleting package.json
47. What is the difference between mutation score and mutation score indicator (MSI)?
They describe the same idea but come from different tools. The mutation score is the general term: the percentage of mutants detected by tests.
The MSI is the term used by Infection, the PHP tool. It reports two numbers:
- MSI: detected mutants as a share of all generated mutants, including those with no coverage.
- Covered Code MSI: detected mutants as a share of only mutants in code that tests execute.
The covered-code figure is comparable to PIT's test strength. A big gap between the two numbers means large parts of the code have no tests at all.
Example: 100 mutants, 70 detected, 20 not covered, 10 covered but surviving. MSI is 70%, while Covered Code MSI is 70 out of 80, roughly 87.5%. The gap points to untested code, not weak assertions.
Take quiz
Infection
PIT only
Stryker.NET only
go-mutesting
The tests are all flaky
Many mutants sit in code that no test executes
The report is broken
Mutants are equivalent
48. What are the limitations of mutation testing?
It is powerful but not free of drawbacks:
- Cost: many test runs, so large projects need scoping and incremental analysis.
- Equivalent mutants: they add noise and need manual review.
- Limited fault model: small syntactic changes do not represent missing requirements, concurrency bugs, performance issues, or integration faults.
- Dependence on test quality: it needs a fast, deterministic suite.
- Score misuse: forcing a high number can lead to low-value tests written only to kill mutants.
Use it as one signal alongside coverage, code review, and exploratory testing.
Another practical limit is language support: some ecosystems have immature tools or limited mutators, so check this before you plan a rollout across all teams.
Take quiz
Wrong comparison operators
Missing requirements
Swapped arithmetic signs
Removed method calls
The score becomes negative
Tests stop compiling
Developers may write low-value tests just to kill mutants
Mutants stop being generated
It only works with UI tests
It cannot parse slow tests
It needs random failures
It re-runs tests for many mutants and treats any failure as a kill
49. How can you use mutation testing to review AI-generated tests?
AI assistants can produce tests quickly, and those tests often reach high line coverage while asserting little, for example only checking that a method returns something. Coverage alone cannot spot this.
Run mutation testing on the generated tests. A low score or many survivors in the report show where the assertions are shallow. You can then feed those survivors back, asking the assistant to write a test that fails for that exact mutation, and re-run to confirm it is killed.
Always review the new tests yourself. A test that kills a mutant only by hard-coding the expected value from the current implementation can still be brittle or wrong about the requirement.
Take quiz
Too few comments
Slow compile times
High coverage with shallow assertions
Missing license headers
Delete the mutation report
Assume it is correct
Disable all other tests
Re-run mutation and review the test yourself
50. Which is better for mutation testing: unit tests or integration tests?
Unit tests are usually the better fit, for three reasons.
- Speed: thousands of mutants multiply the cost of every slow test.
- Precision: a failing unit test points straight at the mutated logic, and per-test coverage runs only a handful of tests per mutant.
- Stability: fewer external dependencies mean fewer false kills from network or database problems.
Integration tests execute large amounts of code, so many mutants look "covered" even when assertions are loose. They also tend to be slower and flakier. Use them selectively, for example on a small critical module, and rely on unit tests for the bulk of the score.