How to reject -DskipTests in a coding agent's diff

The coding agent finishes the task, runs the build, gets BUILD SUCCESS and reports "tests passing." Then you open the transcript and see that the command it actually ran was mvn test -DskipTests. Or the agent left the command alone and added <maven.test.skip>true</maven.test.skip> to pom.xml. Maven exited 0 and no test ran. JUnit never even loaded, so nobody was around to say everything was fine.

The fix is a small gate that uses only the Python standard library. It rejects skipTests and maven.test.skip in three places: the command the agent ran, the captured log, and the diff of pom.xml and .mvn/maven.config. Every Maven and gate command below was run on 2026-10-21 (JDK 25.0.4, Maven 3.9.12).

Where this fits with what we've already published

We've already covered the Surefire receipt, which requires a recent TEST-*.xml with zero failures. There's also the gate against loosened assertions and @Disabled, which reads the src/test diff, and the \*IT that Failsafe never ran during mvn test. Today's problem comes earlier than any of those. Nobody weakened a test, and no test was left in a later lifecycle phase. A flag switched off the whole suite, and that flag appears in either the command or the diff. Those are the places the gate checks.

Environment

• Temurin JDK 25.0.4, Maven 3.9.12, maven.compiler.release=25, no --enable-preview
• maven-compiler-plugin 3.16.0. Central's latest is 4.0.0-beta-5, which was left out because it's still a beta.
• maven-surefire-plugin 3.6.0
• junit-bom / junit-jupiter 6.1.3
• Packaging jar, no Spring, no Failsafe, no Docker
• Python 3, standard library only, for the gate

skipTests and maven.test.skip don't do the same thing

Surefire's Skipping Tests page shows both options. You can set skipTests in the POM with <skipTests>true</skipTests> or pass it on the command line with mvn install -DskipTests. The page introduces maven.test.skip with "if you absolutely must," and it goes further: it also skips compiling the tests. According to the page, Surefire, Failsafe and the Compiler Plugin all honor this property. Its example is mvn install -Dmaven.test.skip=true.

The reference page for the surefire:test goal lists the two parameters separately:

The compilation side is documented in compiler:testCompile. Its skip parameter, with user property maven.test.skip, tells the plugin not to compile test sources.

In practice, -DskipTests leaves AdderTest.class in target/test-classes, but nothing runs it. -Dmaven.test.skip=true doesn't even produce the .class file. Both exit with code 0.

Repro: one test, five exit 0s

The module is as small as it gets. POM:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>devdojo</groupId>
  <artifactId>skip-tests-gate</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <packaging>jar</packaging>
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.release>25</maven.compiler.release>
  </properties>
  <dependencyManagement>
    <dependencies>
      <dependency>
        <groupId>org.junit</groupId>
        <artifactId>junit-bom</artifactId>
        <version>6.1.3</version>
        <type>pom</type>
        <scope>import</scope>
      </dependency>
    </dependencies>
  </dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.junit.jupiter</groupId>
      <artifactId>junit-jupiter</artifactId>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.16.0</version>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.6.0</version>
      </plugin>
    </plugins>
  </build>
</project>

Production code, in src/main/java/devdojo/Adder.java:

package devdojo;

public final class Adder {
  private Adder() {}

  public static int add(int a, int b) {
    return a + b;
  }
}

Test, in src/test/java/devdojo/AdderTest.java:

package devdojo;

import static org.junit.jupiter.api.Assertions.assertEquals;

import org.junit.jupiter.api.Test;

class AdderTest {

  @Test
  void onePlusOneEqualsTwo() {
    assertEquals(2, Adder.add(1, 1), "skipTestsGateUniqueString");
  }
}

The commands ran in this order:

Run with -B and without -q, Maven reports what it's doing. With clean test -DskipTests:

[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ skip-tests-gate ---
[INFO] Compiling 1 source file with javac [debug release 25] to target/test-classes
[INFO]
[INFO] --- surefire:3.6.0:test (default-test) @ skip-tests-gate ---
[INFO] Tests are skipped.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

With clean test -Dmaven.test.skip=true, the skipping starts earlier:

[INFO] --- resources:3.3.1:testResources (default-testResources) @ skip-tests-gate ---
[INFO] Not copying test resources
[INFO]
[INFO] --- compiler:3.16.0:testCompile (default-testCompile) @ skip-tests-gate ---
[INFO] Not compiling test sources
[INFO]
[INFO] --- surefire:3.6.0:test (default-test) @ skip-tests-gate ---
[INFO] Tests are skipped.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS

The table shows two traps.

-q hides the only clue. Tests are skipped. is an [INFO] line, and quiet mode suppresses it. An agent that runs mvn -q test -DskipTests returns an empty log and exit 0, which looks exactly like a quiet green build. Quiet mode does exactly what it says.

The old XML is still there. Without clean, -DskipTests doesn't delete target/surefire-reports. The TEST-devdojo.AdderTest.xml from the 05:08:32 run stayed on disk with the same mtime and failures=0. If you read the directory without checking the file's age, you'll accept a report from an earlier run as proof that the agent's task passed. That age check is what the Surefire receipt does. Here we reject the flag before opening any XML.

The gate: command, log, and diff

The gate searches for text tokens. It doesn't interpret the Maven model, read Surefire XML, or look for @Disabled. It exits 1 if it finds any of the tokens and 0 if it finds none.

#!/usr/bin/env python3
"""Fail closed when a coding-agent command, log, or diff introduces skipTests / maven.test.skip.

Does not parse Surefire XML as the primary receipt.
Does not inspect Failsafe / *IT / skipITs.
Does not look for @Disabled or loosened assertions.
"""
from __future__ import annotations

import argparse
import re
import sys
from pathlib import Path

# Tokens that zero the unit-test suite without failing the Maven process.
# Keep skipITs out: that is Failsafe-only.
PATTERNS: list[tuple[str, re.Pattern[str]]] = [
    ("-DskipTests", re.compile(r"-DskipTests\b")),
    ("skipTests=true", re.compile(r"\bskipTests\s*=\s*true\b", re.IGNORECASE)),
    ("-Dmaven.test.skip", re.compile(r"-Dmaven\.test\.skip\b")),
    ("maven.test.skip=true", re.compile(r"\bmaven\.test\.skip\s*=\s*true\b", re.IGNORECASE)),
    ("<skipTests>true</skipTests>", re.compile(r"<skipTests>\s*true\s*</skipTests>", re.IGNORECASE)),
    (
        "<maven.test.skip>true</maven.test.skip>",
        re.compile(r"<maven\.test\.skip>\s*true\s*</maven\.test\.skip>", re.IGNORECASE),
    ),
]


def findings(text: str) -> list[str]:
    hits: list[str] = []
    for label, pattern in PATTERNS:
        if pattern.search(text):
            hits.append(label)
    return hits


def read_text(path: Path) -> str:
    return path.read_text(encoding="utf-8", errors="replace")


def main() -> int:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--command", default="", help="Agent command string")
    parser.add_argument("--log", type=Path, help="Captured Maven / agent log")
    parser.add_argument("--diff", type=Path, help="Unified diff (pom.xml / .mvn/maven.config)")
    parser.add_argument("--file", action="append", default=[], type=Path, help="File contents to scan")
    args = parser.parse_args()

    problems: list[str] = []
    if args.command:
        for hit in findings(args.command):
            problems.append(f"command: {hit}")
    if args.log is not None:
        for hit in findings(read_text(args.log)):
            problems.append(f"log {args.log}: {hit}")
    if args.diff is not None:
        for hit in findings(read_text(args.diff)):
            problems.append(f"diff {args.diff}: {hit}")
    for path in args.file:
        for hit in findings(read_text(path)):
            problems.append(f"file {path}: {hit}")

    if problems:
        for item in problems:
            print("FAIL:", item)
        return 1
    print("PASS")
    return 0


if __name__ == "__main__":
    sys.exit(main())

Basic usage:

python3 gate.py --command "mvn -q test"                 # exit 0
python3 gate.py --command "mvn -q test -DskipTests"     # exit 1
python3 gate.py --diff pom-skipTests.diff               # exit 1

Results of every gate run:

Where the flag gets in

The agent doesn't have to type -DskipTests in the terminal. The gate covers these paths:

• Command line. -DskipTests, -DskipTests=true, -Dmaven.test.skip=true.
• pom.xml. This can be <skipTests>true</skipTests> in the Surefire configuration, which is the form shown on the official page. It can also be <skipTests> or <maven.test.skip> under <properties>, including inside a profile.
• .mvn/maven.config. In Maven 3.9 this file takes one argument per line. A -DskipTests added there applies to every mvn command run in the project, including yours.
• MAVEN_ARGS. Since Maven 3.9.0, the mvn script reads this variable, so -DskipTests can arrive through it. MAVEN_OPTS is different: it holds JVM options. Because the gate only reads text, the simplest approach is to append the environment variables to --command and let the regex find the flag.

In the harness, the setup looks roughly like this (this shell snippet itself was not run):

git diff "$BASE" -- pom.xml .mvn/maven.config > agent.diff
python3 gate.py \
  --command "$AGENT_MVN_CMD ${MAVEN_ARGS:-} ${MAVEN_OPTS:-}" \
  --log agent.log \
  --diff agent.diff \
  || { echo "task rejected: test suite disabled by flag" >&2; exit 1; }

With no second commit, git diff "$BASE" -- ... compares the working tree against the base. That means it also catches changes the agent left uncommitted, which are usually the ones nobody reviewed.

One catch with the log: with -B, Maven's output doesn't repeat the flag. It only prints Tests are skipped.. The --log check catches the token only when the agent's log records the command it ran, as the agent log in the table does. If your harness keeps only Maven's stdout, --command is the check that catches the flag. Adding a pattern for Tests are skipped. would be a reasonable extension. However, it wasn't part of this gate or the runs above, and with -q that line doesn't appear anyway.

Limits of the gate

• Failsafe is left out on purpose. -DskipITs passes the gate because it only affects Failsafe. The Failsafe skipping page also notes that since 3.6.0, -DskipTests no longer skips ITs. That case is covered in the article on \*IT and Failsafe.
• Missing or stale XML is covered by the Surefire receipt. @Disabled and loosened asserts are covered by the src/test diff gate. Use all three gates together, because none of them replaces the others.
• -DskipTests=false also fails. The regex -DskipTests\b ignores the value. That's intentional, because an agent harness should fail closed. Some projects follow the documented pattern: skipping is on by default in the POM, and -DskipTests=false turns tests back on. If yours does, add exactly that string to an explicit allowlist in the harness. Loosening the regex is the wrong fix.
• The gate matches text; it doesn't understand the POM. A POM comment containing <skipTests>true</skipTests> trips the gate. So does a removed line in the diff, because the gate scans the whole diff. Both are false positives, which is the safe direction for this gate to err. It can also miss things: a bare <skip>true</skip> in the Surefire configuration matches none of the patterns. If that shows up in your repository, add a pattern for it.

Next step

Put gate.py next to the agent harness and run it before the task is marked as tested. Pass it the command (including MAVEN_ARGS), the log, and the git diff of pom.xml and .mvn/maven.config against the base commit. If the gate exits 1, the task goes back to the agent with the gate's stdout and doesn't become a PR. DevDojo recommends rejecting these flags every time and allowing them only through an allowlist a human has approved. If that allowlist starts growing with every task, the problem to fix is the project's default-skip setup, not the gate. If the gate starts failing on comments or removed diff lines, filter the diff down to its + lines before you loosen any pattern. If the command contains skipTests or maven.test.skip, BUILD SUCCESS tells you nothing about the tests, because they never ran.