Property-Based Testing with jqwik: The Edge Case Your Fixed Example Misses

Every example-based test has the same structural blind spot: it only covers what the author remembered to write. assertEquals(5, sum(2, 3)) passes, the PR gets approved, and nobody tried what happens when one of the two numbers sits right next to the type's limit. In production that "almost never" shows up eventually — a value too large, an empty string, a Unicode character nobody planned for — and it breaks an invariant that no test ever checked, because no test ever generated that value.

Property-based testing (PBT) attacks this blind spot from a different angle: instead of writing given/when/then for one fixed value, you declare a property — something that should hold for any valid input — and the tool generates hundreds of inputs trying to break it. The example here is jqwik 1.10.1, a test engine for the JUnit Platform, running on plain Java SE (no Spring, no Testcontainers) on Java 25. The goal is to reproduce, with real numbers from an actual run, an overflow bug that a fixed example never catches and that a property finds in seconds — then reduces to its minimal case.

Prerequisites

• JDK Temurin 25.0.4+7 (maven.compiler.release=25).
• Maven 3.9.12, maven-compiler-plugin 3.14.1, maven-surefire-plugin 3.6.0.
• net.jqwik:jqwik:1.10.1, which pulls in jqwik-engine 1.10.1 and pins JUnit Platform 1.14.4 as its minimum dependency.

One version detail worth stating once: current JUnit is the 6.x line (Platform and Jupiter have shared version numbers since 6.0.0), but jqwik 1.10.1 still runs on Platform 1.x — the release notes call this probably the last jqwik release on Platform 1.x, with a future migration to Platform 6 still without a published artifact. There is no combination of net.jqwik:jqwik:1.10.1 plus junit-bom:6.x that works; the supported matrix is jqwik 1.10.1 with Platform 1.14.4, full stop.

An example is a property with a single try

In jqwik's model, @Example and @Property aren't as different as they look. A method annotated with @Example is, internally, treated as a property with the number of tries hardcoded to 1. A @Property method runs, by default, 1000 tries with randomly generated values, plus a set of automatically mixed-in edge cases (EdgeCasesMode.MIXIN by default).

That explains why an example test can stay green forever while hiding a bug: it tests exactly the pair of values the author chose, nothing else. Take this domain class:

package academy.devdojo.jqwik;

public record Money(long cents) {
  public static Money ofCents(long cents) {
    if (cents < 0) {
      throw new IllegalArgumentException("cents");
    }
    return new Money(cents);
  }

  /** Broken: narrows to int, so sums that fit in long overflow int. */
  public Money plusBroken(Money other) {
    return ofCents(Math.addExact((int) this.cents, (int) other.cents));
  }

  public Money plus(Money other) {
    return ofCents(Math.addExact(this.cents, other.cents));
  }
}

And this example test:

@Example
boolean twoPlusThreeIsFive() {
  return Money.ofCents(2).plusBroken(Money.ofCents(3)).cents() == 5;
}

twoPlusThreeIsFive passes, and keeps passing as long as nobody adds two values that together overflow an int. It's the same mechanism behind a classic gotcha from jqwik's own user guide: Math.abs(Integer.MIN_VALUE) isn't positive — in the JDK, Math.abs of Integer.MIN_VALUE returns Integer.MIN_VALUE itself, still negative, because there's no positive int representation of that value. An example test with Math.abs(-5) will never find that contract violation; only a generator that eventually produces Integer.MIN_VALUE finds it.

Custom generators: @Provide and Arbitrary

A property receives its parameters via @ForAll, and jqwik needs to know how to generate values of that type. Primitives come with ready-made support (Arbitraries.integers(), Arbitraries.strings()); for a domain type like Money, you write a @Provide method that returns an Arbitrary<Money>:

@Provide
Arbitrary<Money> money() {
  return Arbitraries.longs().between(0, Integer.MAX_VALUE + 10_000L).map(Money::ofCents);
}

The chosen range deliberately crosses Integer.MAX_VALUE (2147483647): part of the generated values fit in an int, part don't. That's exactly the kind of generator — one that crosses the smaller type's boundary instead of staying comfortably inside it — that makes a property worth writing. For composite types with more than one field, Combinators.combine(...).as(...) builds the object from several smaller Arbitrary instances; it tends to shrink better than chained flatMap, which only pays off when one value genuinely depends on another generated earlier.

With the generator above, the property that exposes the plusBroken bug looks like this:

@Property
boolean brokenPlusDoesNotOverflow(@ForAll("money") Money a, @ForAll("money") Money b) {
  a.plusBroken(b);
  return true;
}

The property doesn't claim anything sophisticated — just that adding two valid Money values shouldn't throw. That's enough.

What the real run showed

Running mvn test against the class with plusBroken, plusIsCommutative, plusIsAssociative, twoPlusThreeIsFive, and a golden property (goldenLedgerRows, covered further down) — five tests total — the result was:

Tests run: 5, Failures: 0, Errors: 1, Skipped: 0

twoPlusThreeIsFive stayed green, as expected. brokenPlusDoesNotOverflow failed after 63 tries (seed = -8242242476536634201), with:

java.lang.IllegalArgumentException: cents
  at academy.devdojo.jqwik.Money.ofCents(Money.java:6)
  at academy.devdojo.jqwik.Money.plusBroken(Money.java:13)

Notice the exception type: it's not an ArithmeticException from Math.addExact — it's an IllegalArgumentException from Money.ofCents, thrown two layers after the actual overflow. The mechanism, confirmed by running the calculation in isolation on this host:

1. a.cents() was generated as 2147483648 (long) — one more than Integer.MAX_VALUE.
2. (int) 2147483648L overflows the cast and becomes -2147483648 (Integer.MIN_VALUE), because the sign bit simply reappears.
3. Math.addExact(-2147483648, 187030144) (both already int) doesn't overflow — the result, -1960453504, fits in int — so addExact returns normally with a negative number.
4. Money.ofCents(-1960453504) rejects the negative value and throws IllegalArgumentException.

In other words: addExact is doing exactly what it promises (catching int overflow), except the damage had already happened one step earlier, in the silent long-to-int cast. addExact protects the sum; it doesn't protect the cast that precedes it.

The failure report: original sample and shrunk sample

When a property is falsified, jqwik tries to "shrink" the counterexample down to a simpler one before reporting. In the words of the official guide: if a property could be falsified with a generated set of values, jqwik will try to shrink this sample in order to find a "smaller" sample that also falsifies the property. jqwik uses integrated shrinking (not type-based shrinking), which tends to produce counterexamples that are more relevant to the domain.

In the real run, the original generated pair was:

Original Sample
---------------
  a: Money[cents=2147493646]
  b: Money[cents=187030144]

And after shrinking:

Shrunk Sample (5 steps)
-----------------------
  a: Money[cents=2147483648]
  b: Money[cents=187030144]

Five steps took a from 2147493646 down to exactly 2147483648 — one more than Integer.MAX_VALUE, the smallest value that still overflows the cast. b didn't change, because in jqwik's parameter-by-parameter shrinking, each value is reduced in isolation, and 187030144 was already necessary for the scenario to keep failing.

A warning showed up in the log of that same run:

WARNING: Shrinking timeout reached after 10 seconds.
You can switch on full shrinking with '@Property(shrinking = ShrinkingMode.FULL)'

The default mode is ShrinkingMode.BOUNDED, with a 10-second cap (jqwik.shrinking.bounded.seconds) — and in this run the cap was actually hit, not a hypothetical scenario. That means the reported "Shrunk Sample" is the best the shrinker managed within the time budget, not necessarily the smallest possible counterexample; the guide itself is explicit that the minimal sample isn't even unique — it depends on the seed, and different seeds can shrink to equally small but different shapes. Switching to ShrinkingMode.FULL removes the time cap at the cost of running until the reductions are exhausted; fine for a one-off investigation, not as a CI default.

Fixing it, and reproducing with the same seed

The fix swaps the cast for long arithmetic end to end — Money's plus method already did this; plusBroken was the one broken by design:

public Money plus(Money other) {
  return ofCents(Math.addExact(this.cents, other.cents));
}

Switching the property to use plus instead of plusBroken and running again:

MoneyPropertiesFixed:plusIsCommutative — tries = 1000, seed = -925629694754968486
MoneyPropertiesFixed:plusDoesNotOverflow — tries = 1000, seed = 6257245524895003827
MoneyPropertiesFixed:goldenLedgerRows — tries = 2, generation = DATA_DRIVEN

mvn test exited with code 0. Reducing the bug to a minimal case is useful, but the fact that actually closes the loop is the seed: with seed = -8242242476536634201, anyone on the team reproduces the exact same sequence of tries by running again with @Property(seed = "-8242242476536634201"), without depending on luck or a CI screenshot.

Maven, the JUnit Platform, and the include detail

jqwik isn't a Jupiter extension — it's a separate test engine running on the same JUnit Platform, discovered via ServiceLoader. That has two practical consequences for pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0">
  <modelVersion>4.0.0</modelVersion>
  <groupId>academy.devdojo</groupId>
  <artifactId>jqwik-money</artifactId>
  <version>1.0-SNAPSHOT</version>
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.release>25</maven.compiler.release>
    <jqwik.version>1.10.1</jqwik.version>
  </properties>
  <dependencies>
    <dependency>
      <groupId>net.jqwik</groupId>
      <artifactId>jqwik</artifactId>
      <version>${jqwik.version}</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.14.1</version>
        <configuration>
          <compilerArgs>
            <arg>-parameters</arg>
          </compilerArgs>
        </configuration>
      </plugin>
      <plugin>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.6.0</version>
        <configuration>
          <includes>
            <include>**/*Properties.java</include>
            <include>**/*Tests.java</include>
            <include>**/*Examples.java</include>
          </includes>
        </configuration>
      </plugin>
    </plugins>
  </build>
</project>

First: Surefire's default include pattern is **/*Test.java (and variants with Test in the name). A class named MoneyProperties.java doesn't match that pattern — without the explicit <includes> above, Maven runs zero tests and reports success, which is a good deal worse than a failure. Second: -parameters on the compiler isn't strictly required for jqwik to work, but without it failure reports show arg0, arg1 instead of a, b — worth the compile-time cost.

There's no combination where adding org.junit:junit-bom at version 6.x helps here: it just widens the dependency surface without fixing anything, since jqwik-engine 1.10.1 already pins junit-platform-commons and junit-platform-engine at 1.14.4 through its POM. If you ever need to mix jqwik with Jupiter tests in the same module, that's where junit-jupiter:5.14.4 comes in — not the 6.x line, which switches Platform major versions.

When a property doesn't replace an example

Not every assertion is "this holds for any input." Specific business rules — a regulatory table value, a particular regression case, the classic FizzBuzz where 3 must become exactly "Fizz" — aren't general invariants, they're known, expected outcomes. For those cases jqwik offers @Example (one fixed value) and data-driven properties via @FromData:

@Data
Iterable<Tuple.Tuple2<Long, Long>> ledgerGoldens() {
  return Table.of(Tuple.of(0L, 0L), Tuple.of(1L, 99L));
}

@Property
@FromData("ledgerGoldens")
boolean goldenLedgerRows(@ForAll long a, @ForAll long b) {
  return Money.ofCents(a).plus(Money.ofCents(b)).cents() == a + b;
}

In the run, goldenLedgerRows executed with tries = 2 and generation = DATA_DRIVEN — both pairs came straight from the table, with no random generation and no shrinking (the guide is blunt about it: data-driven properties don't go through shrinking, because there's no "smaller" version of a manually recorded row). That's a feature, not a limitation to work around: a ledger row with zero cents on both sides is a known case you want pinned, not sampled.

Diagnosis, fix, and how to observe it later

Worth pointing out the difference with mutation testing: mutation testing asks whether the tests you already have notice a deliberate change in behavior; property-based testing asks which input the author never got around to writing. They're complementary checks, not the same question wearing a different name.

Recommendation

jqwik is worth adopting when the logic under test is a mathematical or structural invariant — addition commutes, parse is the inverse of format, serialization is idempotent — because a generic property there is worth hundreds of examples nobody would have the patience to write by hand. It's not worth forcing a property onto a business rule that is, by nature, a closed list of cases with a specific expected result: @Example and @FromData remain the right tool there, and trying to generalize that into a property just produces artificial generators that simulate a lookup table.

A sign it's time to back off: if you're writing filter or Assume.that to discard most of the generated values, the generator is wrong, not the technique — the jqwik guide already warns that high discard rates end in "exhausted after tries," and the fix is modeling an Arbitrary that only produces valid values, not filtering out invalid ones after generation. Beyond that, the jqwik 1.10.1 + Platform 1.14.4 + Java 25 combination is solid for plain Java SE today; the next version decision only comes once jqwik publishes something on Platform 6, and that migration isn't worth teaching before it exists.