A green test suite can execute every relevant line and still let a behavioral change slip through. Coverage tells you whether the code was executed; mutation testing asks whether the tests would notice a deliberate change to that code.
The difference shows up quickly around boundary conditions. A test calls the heavy-shipping rule but only checks whether the returned value is greater than or equal to the base fee. The line was covered. The rule, not so much.
We will build a small Java SE module, run PIT twice, and inspect the actual receipts. In the first cycle, two of the six mutants survive even with 100% line coverage in the mutated classes. After strengthening the boundary checks, all six are killed. This measures how well this suite detects these mutants; it does not prove that the entire application is correct.
Versions used
The example was executed on Linux amd64 with:
- Temurin JDK 25.0.4;
- Maven 3.9.16;
- PIT and
pitest-maven1.30.0; pitest-junit5-plugin1.2.3;- JUnit 6.1.3;
- Maven Compiler Plugin 3.16.0;
- Maven Surefire Plugin 3.6.0.
JDK 25 is an LTS release according to the official Java roadmap. Maven 3.9.16 was the stable version recommended on the official download page, while Maven 4 remained in preview. PIT 1.30.0 includes mutator adjustments for Java 25, as described in the release notes.
The integration plugin documentation still uses the name “JUnit 5,” but the combination with JUnit 6.1.3 was successfully executed: PIT added junit-platform-launcher 6.1.3, discovered both Jupiter tests, and completed the analysis. There is therefore no downgrade to JUnit 5 in this example. The current JUnit guide requires Java 17 or later, which JDK 25 already satisfies.
Create the Maven module
Start with the minimum structure:
mkdir -p pit-shipping/src/main/java/academy/devdojo/shipping
mkdir -p pit-shipping/src/test/java/academy/devdojo/shipping
cd pit-shipping
java -version
mvn --versionUse this pom.xml. The versions are pinned, mutation is scoped to the example package, and the HTML and XML reports go to a stable path without a timestamped directory.
<project xmlns="http://maven.apache.org/POM/4.0.0">
<modelVersion>4.0.0</modelVersion>
<groupId>academy.devdojo</groupId>
<artifactId>pit-shipping</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<maven.compiler.release>25</maven.compiler.release>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.junit</groupId>
<artifactId>junit-bom</artifactId>
<version>6.1.3</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.16.0</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.6.0</version>
</plugin>
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<version>1.30.0</version>
<dependencies>
<dependency>
<groupId>org.pitest</groupId>
<artifactId>pitest-junit5-plugin</artifactId>
<version>1.2.3</version>
</dependency>
</dependencies>
<configuration>
<targetClasses>
<param>academy.devdojo.shipping.*</param>
</targetClasses>
<targetTests>
<param>academy.devdojo.shipping.*</param>
</targetTests>
<outputFormats>
<param>HTML</param>
<param>XML</param>
</outputFormats>
<timestampedReports>false</timestampedReports>
<failWhenNoMutations>true</failWhenNoMutations>
</configuration>
</plugin>
</plugins>
</build>
<profiles>
<profile>
<id>pitest-ci</id>
<build>
<plugins>
<plugin>
<groupId>org.pitest</groupId>
<artifactId>pitest-maven</artifactId>
<configuration>
<mutationThreshold>80</mutationThreshold>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>The JUnit integration plugin belongs in the PIT plugin dependencies, not in the application's test dependencies. The pitest-junit5-plugin documentation also requires PIT 1.19.4 or later for version 1.2.3; we are using 1.30.0.
A small rule and an overly green test
Create src/main/java/academy/devdojo/shipping/ShippingFee.java:
package academy.devdojo.shipping;
public final class ShippingFee {
public int cents(int weightGrams) {
if (weightGrams < 0) throw new IllegalArgumentException("weightGrams");
int base = 500;
if (weightGrams >= 1000) base += 250;
return base;
}
}Now create src/test/java/academy/devdojo/shipping/ShippingFeeTest.java with the first version:
package academy.devdojo.shipping;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
class ShippingFeeTest {
private final ShippingFee fee = new ShippingFee();
@Test
void lightPackage() {
assertEquals(500, fee.cents(100));
}
@Test
void heavyPackage() {
assertTrue(fee.cents(1500) >= 500);
}
}Run the tests and the official mutationCoverage goal:
mvn -B -ntp test org.pitest:pitest-maven:1.30.0:mutationCoverageBoth tests pass. PIT also completes successfully because the regular execution does not yet have a configured threshold. The actual result was:
Line Coverage (for mutated classes only): 5/5 (100%)
Generated 6 mutations Killed 4 (67%)
Mutations with no coverage 0. Test strength 67%Line coverage is 100%, but two mutants survived. The XML report recorded both as ConditionalsBoundaryMutator:
ShippingFee.java:4 SURVIVED changed conditional boundary
ShippingFee.java:6 SURVIVED changed conditional boundaryOn line 4, the mutator shifted the boundary of the negative-weight validation. Because the lightest test uses 100, it cannot distinguish < 0 from <= 0. On line 6, the same mutator family changed the 1,000-gram boundary. The test uses 1500 and accepts any return value >= 500; it passes without checking either the boundary or the expected 750 cents.
The test took the right branch and returned without complaint. A sightseeing tour through the if, basically.
Before changing the test, preserve the weak result. The second execution reuses target/pit-reports; copy the directory now so you do not lose the comparison.
mkdir -p evidence
cp src/test/java/academy/devdojo/shipping/ShippingFeeTest.java \
evidence/ShippingFeeTest-weak.java
cp -R target/pit-reports evidence/pit-weakThe HTML at target/pit-reports/index.html is useful for human investigation. The XML at target/pit-reports/mutations.xml works better as a CI artifact and for automation. The PIT Maven documentation describes these formats and the scoping options.
Strengthen the boundaries, not the test count
Replace the test with the version below:
package academy.devdojo.shipping;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;
class ShippingFeeTest {
private final ShippingFee fee = new ShippingFee();
@Test
void lightPackage() {
assertEquals(500, fee.cents(100));
}
@Test
void boundaries() {
assertEquals(500, fee.cents(0));
assertEquals(500, fee.cents(999));
assertEquals(750, fee.cents(1000));
assertEquals(750, fee.cents(1001));
assertThrows(IllegalArgumentException.class, () -> fee.cents(-1));
}
}The 0 case matters. Testing only -1 proves that an exception is thrown on the invalid side, but it still allows a mutation that changes < 0 to <= 0 to survive. For the weight rule, 999, 1000, and 1001 pin down both sides and the exact point where the behavior changes.
Run it again and preserve the second report:
mvn -B -ntp test org.pitest:pitest-maven:1.30.0:mutationCoverage
cp src/test/java/academy/devdojo/shipping/ShippingFeeTest.java \
evidence/ShippingFeeTest-strong.java
cp -R target/pit-reports evidence/pit-strongThe actual execution still ran two Jupiter tests and changed the result to six mutants killed out of six generated. Both boundary mutants were now killed by the boundaries() method. There were no mutants without coverage, timeouts, or execution errors.
The improvement did not come from chasing more lines. It came from asserting observable behavior at the points where an operator could change without the suite noticing.
A gradual, scoped CI gate
Do not put PIT into every mvn test run for a monolith, and do not start by requiring a 100% mutation score. The PIT FAQ notes that equivalent mutants exist and discourages the ALL set. The score is not proof of correctness either: it is the proportion of generated mutants that the suite killed.
The pitest-ci profile in the POM requires 80% only when activated. The package remains limited by targetClasses and targetTests, and the mutators remain in the default set. Run it like this:
mvn -B -ntp -Ppitest-ci \
test org.pitest:pitest-maven:1.30.0:mutationCoverageThis gate was verified against both project variants. With the weak test, Maven exited with code 1 and reported Mutation score of 67 is below threshold of 80. With the strong test, it exited with code 0 and BUILD SUCCESS.
The value 80 is not a universal recommendation. In this small module, it deliberately separates the variant that lets two boundary mutants survive. In a real project, first run PIT without a threshold, review surviving and equivalent mutants, limit the initial gate to a module or package the team can maintain, and adopt a floor close to the accepted baseline. Raise it later when the fixes remain reviewable—not when someone discovers that YAML also accepts ambitious numbers.
Publish at least these files as CI artifacts:
target/pit-reports/index.html
target/pit-reports/mutations.xmlKeep the command, the Java and Maven versions, and the analyzed commit as well. The XML distinguishes KILLED, SURVIVED, NO_COVERAGE, and TIMED_OUT; these are different diagnoses. SURVIVED means the mutated code was covered and the tests stayed green. NO_COVERAGE points to mutated code that no test covered. A timeout needs its own investigation and should not automatically be treated as an infinite loop.
The official PIT mutator list helps turn each survivor into a question about the rule. In the example, the question was simple: “Does the test pin down both sides and the exact boundary?”. That reading is more useful than looking only at the final percentage.
Next step
Choose a small package with deterministic rules, run PIT without a threshold, and preserve the first HTML/XML report. Fix one survivor that represents relevant behavior, run it again, and only then propose a gate that fits the reviewed baseline.
Use the report to guide an assertion, clarify a rule, or adjust the gate's scope. The percentage alone does not replace that review.