Graceful Drain in Spring Boot 4.1: HTTP and Workers

A pod receives SIGTERM, the application shuts down, and the deployment finishes. Even so, an in-flight request returns an error and the final outbox batch stops halfway through. The process exited; the work was less cooperative.

In Spring Boot 4.1.1, graceful HTTP drain is already the default behavior. The part that often goes missing is coordinating three different clocks: the web server, application workers, and the deadline Kubernetes gives the process. The outcome we want to verify is straightforward: accepted requests finish, new connections stop arriving, no new batch starts, and only then does the process release resources managed by the context.

The examples use Spring Boot 4.1.1 with Java 25. According to the official Spring Boot system requirements, this version requires Java 17 and supports versions through Java 26, inclusive. Java 25 fits that range and is the baseline here; Boot 3.x and Java 21 are not the starting point.

What actually happens after SIGTERM

We do not need to register a signal handler. When the process receives SIGTERM, the shutdown hook installed by SpringApplication closes the ApplicationContext. Closing the context publishes ContextClosedEvent near the beginning and starts stopping components that implement SmartLifecycle.

The server participates in that sequence through WebServerGracefulShutdownLifecycle. Its phase is WebServerApplicationContext.GRACEFUL_SHUTDOWN_PHASE, equivalent to SmartLifecycle.DEFAULT_PHASE - 1024. Because higher phases stop first, a worker in the default phase begins shutting down before the web server enters its drain phase.

During graceful shutdown, Tomcat, Jetty, and Reactor Netty stop accepting new requests at the network layer and allow in-flight requests to finish within the phase timeout. That is the behavior documented in Graceful Shutdown. In 4.1.1, server.shutdown already defaults to graceful; immediate disables it. Repeating the property in YAML merely records the decision.

The default value of spring.lifecycle.timeout-per-shutdown-phase is 30 seconds.

server:
  shutdown: graceful # padrão no Boot 4.1.1; explícito para registrar a decisão

spring:
  lifecycle:
    timeout-per-shutdown-phase: 20s

management:
  endpoint:
    health:
      probes:
        enabled: true
  endpoints:
    web:
      exposure:
        include: health

Availability state and HTTP observation during drain are not the same thing. The documentation for probes and application lifecycle describes the application as REFUSING_TRAFFIC during graceful shutdown, but it also says HTTP probes stop accepting traffic. Calling /actuator/health/readiness after the signal and requiring a 503 is therefore not reliable proof. The connector may refuse the connection before it can return any status at all.

The useful proof is behavioral:

• a request accepted before the signal finishes;
• a new connection is not accepted during drain;
• SmartLifecycle components receive the stop request and finish within the deadline.

Graceful HTTP shutdown does not drain the outbox

server.shutdown=graceful controls the web server. It knows nothing about the unit of work inside an @Scheduled method, a queue listener, or an outbox relay. If a worker can fetch another batch while the context is closing, the application may begin a transaction that no longer has enough time to finish.

The fix is to give the worker its own protocol: stop admitting batches, wait for the active batch with an explicit limit, and invoke the SmartLifecycle callback when that batch finishes or the limit expires. The DataSource remains Spring's responsibility and is destroyed after lifecycle components have stopped; the worker should not close it itself.

The example below is a minimal executable application. OutboxBatchProcessor waits only to make the batch visible in a local test. In a real application, that method calls the transactional service that selects and commits an outbox batch.

package devdojo.drain;

import java.time.Duration;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.SmartLifecycle;
import org.springframework.scheduling.annotation.EnableScheduling;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@SpringBootApplication
@EnableScheduling
public class DrainApplication {

  public static void main(String[] args) {
    SpringApplication.run(DrainApplication.class, args);
  }
}

@RestController
class SlowController {

  @GetMapping("/slow")
  String slow() throws InterruptedException {
    Thread.sleep(Duration.ofSeconds(8));
    return "concluida";
  }
}

@Component
class OutboxBatchProcessor {

  void processNextBatch() throws InterruptedException {
    // Substitua pelo serviço transacional e idempotente do outbox.
    Thread.sleep(Duration.ofSeconds(5));
  }
}

@Component
class OutboxWorker implements SmartLifecycle {

  private static final Logger log = LoggerFactory.getLogger(OutboxWorker.class);
  private static final Duration DRAIN_TIMEOUT = Duration.ofSeconds(15);

  private final OutboxBatchProcessor processor;
  private final ReentrantLock lock = new ReentrantLock();
  private final Condition batchFinished = lock.newCondition();

  private volatile boolean running;
  private boolean accepting;
  private boolean activeBatch;

  OutboxWorker(OutboxBatchProcessor processor) {
    this.processor = processor;
  }

  @Override
  public void start() {
    lock.lock();
    try {
      accepting = true;
      running = true;
    } finally {
      lock.unlock();
    }
  }

  @Scheduled(fixedDelay = 500)
  void poll() {
    lock.lock();
    try {
      if (!accepting || activeBatch) {
        return;
      }
      activeBatch = true;
    } finally {
      lock.unlock();
    }

    try {
      processor.processNextBatch();
    } catch (InterruptedException interrupted) {
      Thread.currentThread().interrupt();
    } finally {
      lock.lock();
      try {
        activeBatch = false;
        batchFinished.signalAll();
      } finally {
        lock.unlock();
      }
    }
  }

  @Override
  public void stop(Runnable callback) {
    lock.lock();
    try {
      accepting = false;
    } finally {
      lock.unlock();
    }

    Thread.ofVirtual().name("outbox-drain").start(() -> {
      boolean completed = false;
      lock.lock();
      try {
        long remaining = DRAIN_TIMEOUT.toNanos();
        while (activeBatch && remaining > 0) {
          remaining = batchFinished.awaitNanos(remaining);
        }
        completed = !activeBatch;
        running = false;
      } catch (InterruptedException interrupted) {
        Thread.currentThread().interrupt();
        running = false;
      } finally {
        lock.unlock();
        log.info("Outbox drain finalizado: completed={}", completed);
        callback.run();
      }
    });
  }

  @Override
  public void stop() {
    stop(() -> { });
  }

  @Override
  public boolean isRunning() {
    return running;
  }

  @Override
  public boolean isAutoStartup() {
    return true;
  }

  @Override
  public int getPhase() {
    return SmartLifecycle.DEFAULT_PHASE;
  }
}

The lock closes the small race between “may I start?” and “has shutdown started?” Once stop changes accepting to false, no later execution can mark another batch as active. A batch that has already started may finish; the virtual thread only waits and invokes the callback without blocking the processing of other phases.

The default phase stops this worker before WebServerGracefulShutdownLifecycle. That is appropriate for an independent poller such as an outbox relay. If an in-flight HTTP request submits work directly to this same in-memory worker, the design must change: persist the work before responding or choose another phase deliberately. Changing phase numbers until the test turns green is configuration by astrology.

The callback must run both when the batch finishes and when DRAIN_TIMEOUT expires; without it, Spring never receives confirmation that the component stopped. The batch should also be idempotent and have its own transactional timeout, because coordinated shutdown does not rescue an operation with no deadline.

Kubernetes: remove the route before sending the signal

In Kubernetes, preStop runs before SIGTERM. That interval allows EndpointSlice and load-balancer changes to propagate while the process can still serve traffic. After the hook finishes, the kubelet sends the signal and Spring begins shutting down.

terminationGracePeriodSeconds must cover the preStop duration, the sum of the timeouts that shutdown phases may consume, and an operational margin. Because the worker in the default phase and the server in GRACEFUL_SHUTDOWN_PHASE may use separate limits, calculate the value for the entire sequence. When the grace period ends, Kubernetes terminates the process with SIGKILL.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: drain-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: drain-demo
  template:
    metadata:
      labels:
        app: drain-demo
    spec:
      terminationGracePeriodSeconds: 70
      containers:
        - name: app
          image: registry.example.com/drain-demo:1.0.0
          ports:
            - name: http
              containerPort: 8080
          lifecycle:
            preStop:
              sleep:
                seconds: 10
          readinessProbe:
            httpGet:
              path: /actuator/health/readiness
              port: http
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /actuator/health/liveness
              port: http
            periodSeconds: 10

The sleep handler requires Kubernetes 1.32 or later. On older versions, the equivalent is a preStop.exec that runs sleep, provided the image contains the executable. This YAML is a configuration template, not a report of a cluster test. The order and time budget are documented in the Pod lifecycle and the guide to Pod and endpoint termination.

There is no guarantee that every proxy stops forwarding the instant an endpoint becomes terminating. That is precisely the window preStop absorbs. Measure the right duration in your environment rather than copying the example's ten seconds and hoping the network respects round numbers.

How to prove it locally with kill -TERM

With spring-boot-starter-web and spring-boot-starter-actuator in the project, build the jar and run the application with Java 25:

./mvnw clean package
java -jar target/drain-demo-0.0.1-SNAPSHOT.jar > /tmp/drain-demo.log 2>&1 &
pid=$!

until curl -fsS http://localhost:8080/actuator/health/readiness; do
  sleep 1
done

curl -fsS http://localhost:8080/slow > /tmp/slow-response.txt &
slow_curl=$!
sleep 1

kill -TERM "$pid"
sleep 1

if curl --connect-timeout 2 -fsS http://localhost:8080/actuator/health/readiness; then
  printf 'ERRO: uma nova requisição foi aceita durante o drain\n'
else
  printf 'OK: a nova conexão não foi aceita\n'
fi

wait "$slow_curl"
printf 'Resposta em voo: '
cat /tmp/slow-response.txt

wait "$pid"
printf 'Exit status: %s\n' "$?"
grep 'Outbox drain finalizado' /tmp/drain-demo.log

The expected signals are the concluida response from the call started before SIGTERM, failure of the new connection, and the worker log containing completed=true. The process's final status may reflect the signal and the launcher being used; it does not replace those three checks.

For a negative control, repeat the test while starting the jar with immediate shutdown:

SERVER_SHUTDOWN=immediate \
  java -jar target/drain-demo-0.0.1-SNAPSHOT.jar \
  > /tmp/drain-immediate.log 2>&1 &
pid=$!

until curl -fsS http://localhost:8080/actuator/health/readiness; do
  sleep 1
done

curl -fsS http://localhost:8080/slow > /tmp/immediate-response.txt &
slow_curl=$!
sleep 1
kill -TERM "$pid"
wait "$slow_curl"
printf 'curl exit status com shutdown imediato: %s\n' "$?"

In this control, the slow request does not receive the graceful-shutdown window, so curl should fail. This compares shutdown semantics, not performance; throughput and percentiles have no role here.

Stopping the process with an IDE button is not a guaranteed substitute for SIGTERM, either. Some launchers terminate through a different path and skip the sequence you meant to test.

Remediation and production observation

The symptoms overlap, so one table is more useful than three nearly identical checklists:

Do not rely only on a “shutdown complete” message. It confirms the ending, but says nothing about whether the batch committed, whether SIGKILL occurred, or whether a new connection entered during the window.

When to adopt this design—and when to step back

DevDojo would adopt this design for Spring Boot services with rolling deployments when accepted requests must not be cut off or workers acknowledge work in a database, SQS, or an outbox. Each unit of work must be idempotent, have a known timeout, and produce an observable completion signal.

The team would step back from an in-memory worker if the worst-case batch could not fit comfortably inside the operational termination window, if the processor could not be interrupted safely, or if HTTP traffic created unpersisted work during drain. In those cases, reduce batch size and make batches idempotent, persist delivery before responding, or move consumption to a component with an explicit lease and redelivery protocol. Increasing the grace period without fixing those conditions merely schedules the same failure for later.

As a next step, run the local recipe once with graceful and once with immediate. Then replace the demonstration processor with one idempotent batch from your outbox and verify the same three signals: the in-flight request finishes, the new connection is refused, and the final batch is committed.