SQS redrive policy: the poison message your in-app retry won't let go

It usually goes like this. A Java consumer receives a message from the pedidos-source queue. The payload is broken, so the handler throws. The catch block waits a few seconds and tries again, three times in all. While that's going on, the message's visibility timeout runs out and SQS hands it to another instance, which goes through the same routine. Someone set up the DLQ carefully, and it's still empty. Meanwhile the poison message keeps showing up on the source queue, round after round.

SQS already knows how to take a message that always fails out of circulation. The catch is that it counts receives, not exceptions. If you retry inside the application, or never set a maxReceiveCount, that count never gets to the limit.

Here's what this article covers: the RedrivePolicy contract as the Amazon SQS Developer Guide describes it, why retrying inside the process doesn't move the counter, a consumer that makes one attempt per receive, and a Python script that checks the configuration in local JSON. You won't need an AWS account or an SDK.

The contract: RedrivePolicy is a string inside the attributes map

You link the source queue to the DLQ through the RedrivePolicy attribute on the source queue, which you send with SetQueueAttributes. The API reference describes that value as a string containing a JSON object with two keys:

• deadLetterTargetArn: the ARN of the DLQ that SQS moves the message to once maxReceiveCount is exceeded;
• maxReceiveCount: how many times the source queue delivers the message before it goes to the DLQ. Inside the policy, the documented default is 10.

The rest of the article uses the local fixture below. The account 123456789012, the region sa-east-1, and the queue names are placeholders. This isn't output from a real GetQueueAttributes call:

{
  "QueueUrl": "https://sqs.sa-east-1.amazonaws.com/123456789012/pedidos-source",
  "Attributes": {
    "VisibilityTimeout": "90",
    "MessageRetentionPeriod": "345600",
    "ReceiveMessageWaitTimeSeconds": "20",
    "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:sa-east-1:123456789012:pedidos-dlq\",\"maxReceiveCount\":\"5\"}"
  }
}

Once you deserialize it, the RedrivePolicy looks like this:

{
  "deadLetterTargetArn": "arn:aws:sqs:sa-east-1:123456789012:pedidos-dlq",
  "maxReceiveCount": "5"
}

The escaped quotes are deliberate. In the attributes map, RedrivePolicy is a string, and so is every other value, numbers included. The API's own example puts maxReceiveCount in quotes too. If you build the map by hand and put a nested object where the string should be, you're sending something else.

The DLQ fixture has no RedrivePolicy, and it keeps messages longer:

{
  "QueueUrl": "https://sqs.sa-east-1.amazonaws.com/123456789012/pedidos-dlq",
  "Attributes": {
    "VisibilityTimeout": "30",
    "MessageRetentionPeriod": "1209600"
  }
}

The dead-letter queue guide and the configuration guide add a few more rules:

• SQS doesn't create the DLQ for you. You create the queue first and then point the source's policy at it;
• the DLQ has to live in the same account and region as the source, and it has to be the same type: standard with standard, FIFO with FIFO. With FIFO, a DLQ breaks strict ordering, and the guide says so outright;
• in the console, "Maximum receives" accepts values from 1 to 1,000;
• on a standard queue, the original enqueue timestamp stays the same when a message moves to the DLQ. That's why the guide recommends a longer retention on the DLQ than on the source. In the fixture it's 14 days versus 4;
• according to SetQueueAttributes, an attribute change can take up to 60 seconds to propagate. Testing the policy two seconds after you apply it doesn't prove anything.

What maxReceiveCount counts (and what it doesn't)

The guide defines maxReceiveCount as the number of times a consumer can receive the message from the source queue before it gets moved. Exceptions don't count. Neither do iterations of a for loop. Receives are the only thing it counts.

The visibility timeout guide describes one receive cycle like this: the message is delivered and becomes invisible in the queue. If nobody calls DeleteMessage before the timeout ends (30 seconds by default), the message becomes visible again and can be received again. That next receive is what counts toward the limit.

So what happens to the poison message depends on what the handler does:

Wrapping the handler in a retry annotation or a resilience library doesn't help either. To SQS it's the same as the for loop. The SDK's own retries cover the calls to the queue API, not your processing.

There's also a documented behavior on standard queues: with a redrive policy and a maxReceiveCount above 3, a message received 3 or more times without being deleted goes to the back of the queue. It doesn't disappear. It just loses its place.

The stubborn consumer

To keep the example independent of any SDK or framework version, the code uses a minimal port. The identifiers are in Portuguese, same as in the Portuguese edition of this article: apagar means delete and mudarVisibilidade means change visibility. In a real implementation, apagar calls DeleteMessage and mudarVisibilidade calls ChangeMessageVisibility:

package academy.devdojo.pedidos;

record Mensagem(String messageId, String receiptHandle, String body) {}

interface FilaSqs {
    void apagar(String receiptHandle);                          // DeleteMessage
    void mudarVisibilidade(String receiptHandle, int segundos); // ChangeMessageVisibility
}

interface ProcessadorPedido {
    void processar(String body);
}

class PayloadInvalido extends RuntimeException {
    PayloadInvalido(String motivo) {
        super(motivo);
    }
}

Here's the anti-pattern, with both classic mistakes in one place:

package academy.devdojo.pedidos;

class ConsumidorTeimoso {

    private final FilaSqs fila;
    private final ProcessadorPedido processador;

    ConsumidorTeimoso(FilaSqs fila, ProcessadorPedido processador) {
        this.fila = fila;
        this.processador = processador;
    }

    void onMessage(Mensagem msg) throws InterruptedException {
        for (int tentativa = 1; tentativa <= 3; tentativa++) {
            try {
                processador.processar(msg.body());
                fila.apagar(msg.receiptHandle());
                return;
            } catch (Exception e) {
                Thread.sleep(30_000);
            }
        }
        fila.apagar(msg.receiptHandle()); // "so the queue doesn't get stuck"
    }
}

The fixture has a 90-second VisibilityTimeout. Three attempts plus three 30-second pauses take longer than that, so the message reappears while the loop is still running and another instance starts the same loop. Then the apagar on the last line ends things for good: the queue doesn't get stuck, the DLQ stays empty, and the broken order disappears without a trace. At least the comment is honest about what it's for.

One attempt per receive

The practical rule splits failures into two kinds:

• permanent (invalid payload, broken contract, a business error that won't change): don't wait and don't retry. Give the message back without deleting it, so the next receive counts against maxReceiveCount and the message ends up in the DLQ;
• transient (a dependency timeout, throttling): don't sleep here either. Don't delete, and let the visibility timeout decide when the retry happens. The visibility timeout guide recommends this pairing: the timeout handles the next attempt, and the DLQ handles failures that persist.

package academy.devdojo.pedidos;

import java.lang.System.Logger;
import java.lang.System.Logger.Level;

class ConsumidorPedidos {

    private static final Logger LOG = System.getLogger(ConsumidorPedidos.class.getName());

    private final FilaSqs fila;
    private final ProcessadorPedido processador;

    ConsumidorPedidos(FilaSqs fila, ProcessadorPedido processador) {
        this.fila = fila;
        this.processador = processador;
    }

    void onMessage(Mensagem msg) {
        try {
            processador.processar(msg.body());
        } catch (PayloadInvalido e) {
            LOG.log(Level.WARNING, "invalid payload messageId=" + msg.messageId(), e);
            // permanent: visible again right away; the next receive uses up one more of the limit
            fila.mudarVisibilidade(msg.receiptHandle(), 0);
            return;
        } catch (RuntimeException e) {
            LOG.log(Level.WARNING, "transient failure messageId=" + msg.messageId(), e);
            // transient: no delete, no sleep; the visibility timeout decides when to try again
            return;
        }
        fila.apagar(msg.receiptHandle());
    }
}

DeleteMessage only runs on the success path. Per the guide, calling ChangeMessageVisibility with 0 ends the timeout and makes the message available right away. For a permanent failure, all that does is get it to the DLQ sooner. If your listener deletes the message when the method returns normally and leaves it alone when the method throws, rethrowing the exception does the same job as the return statements above.

Redelivery also makes delivery at-least-once on the consumer side. The DLQ takes care of the message that never gets processed, but it doesn't stop a message that was processed twice from charging the customer twice. That's a job for idempotency, as covered in the article on idempotency and outbox.

Visibility timeout versus handler duration

Even with correct code, two situations can keep the problem alive.

The first is a missing maxReceiveCount. With no RedrivePolicy on the source there's no deadLetterTargetArn, and no number of receives will move the message. It stays until retention runs out. The default of 10 only applies inside a policy that exists.

The second is a timeout that's too short for the processing time. If a healthy handler takes longer than the VisibilityTimeout, the message becomes visible again before the delete, another instance receives it, and the counter goes up with no failure at all. With a low maxReceiveCount, a perfectly good message can land in the DLQ just for being slow. The guide recommends starting from your typical maximum processing time. For long-running work, extend visibility with ChangeMessageVisibility while the handler is still busy. An extension doesn't create a new receive. The ceiling is 12 hours from the first receive, and extending doesn't reset that clock.

To tune maxReceiveCount and the timeout together, some rough arithmetic helps: with 5 receives and 90 seconds, the poison message spends at least about 5 × 90 s circulating before it moves queues, plus the wait between receives. That's arithmetic on the configuration, not a measured SLA. I'm also not stating the exact receive where the message moves (N or N+1) as fact. The API says "after exceeding", and an example in the guide says a maxReceiveCount of 1 moves the message on the first failure. You'd confirm that in an integration test.

Checking the configuration without an AWS account

The script below reads both fixtures using nothing but json. It checks that RedrivePolicy is a string, that the policy has exactly two keys, that the DLQ is in the same account and region, that maxReceiveCount falls in the console's range, and that the DLQ keeps messages longer than the source:

import json
import sys

with open(sys.argv[1]) as f:
    origem = json.load(f)
with open(sys.argv[2]) as f:
    dlq = json.load(f)

attrs = origem["Attributes"]
bruto = attrs["RedrivePolicy"]
assert isinstance(bruto, str), "RedrivePolicy must be a JSON string, not an object"

politica = json.loads(bruto)
assert set(politica) == {"deadLetterTargetArn", "maxReceiveCount"}, sorted(politica)

arn = politica["deadLetterTargetArn"]
_, _, servico, regiao, conta, nome = arn.split(":")
assert servico == "sqs", arn
# the DLQ must be in the same account and region as the source queue
assert origem["QueueUrl"].startswith(f"https://sqs.{regiao}.amazonaws.com/{conta}/"), arn
assert dlq["QueueUrl"].endswith(f"/{conta}/{nome}"), dlq["QueueUrl"]

# the API example sends the number in quotes
assert isinstance(politica["maxReceiveCount"], (str, int)), politica["maxReceiveCount"]
max_receives = int(politica["maxReceiveCount"])
assert 1 <= max_receives <= 1000, max_receives

assert "RedrivePolicy" not in dlq["Attributes"]
assert int(dlq["Attributes"]["MessageRetentionPeriod"]) > int(attrs["MessageRetentionPeriod"])

print(f"DLQ={nome} maxReceiveCount={max_receives} VisibilityTimeout={attrs['VisibilityTimeout']}s")

python3 check_redrive.py sqs-source-queue-attributes.json sqs-dlq-attributes.json

Expected output with the fixtures above:

DLQ=pedidos-dlq maxReceiveCount=5 VisibilityTimeout=90s

This is where the check stops, so I'll say it once: the JSON proves names, types, and nesting. It doesn't prove that the handler skips the delete on failure. It doesn't show which receive moves the message, and it doesn't measure how long the message takes to reach the DLQ. All of that needs an integration test against a real queue.

What to watch in production

The documentation gives you two signals, one on each side.

On the message side, request the ApproximateReceiveCount system attribute through MessageSystemAttributeNames on ReceiveMessage (the older AttributeNames parameter is marked deprecated). Log that value with the messageId on every failure. If it goes up from one receive to the next, the queue is doing its job. If the log shows three attempts per receive, the inner loop is still there. If it goes up with no failure logged, the handler is running past the visibility timeout.

On the DLQ side, what you care about is the DLQ no longer being empty. ApproximateNumberOfMessages from GetQueueAttributes gives you an approximate count of available messages. For alerting, the DLQ guide points to CloudWatch alarms on the queue's visible-messages metric, ApproximateNumberOfMessagesVisible. An alarm on the DLQ with a threshold above zero completes the picture: the poison message leaves the source queue, and someone hears about it.

For tomorrow morning

1. Export your source queue's attributes to a file and run the script. If there's no RedrivePolicy, you've found your problem.
2. Make sure maxReceiveCount is low enough that the message leaves quickly and high enough to absorb transient failures, and that the DLQ's retention is longer than the source's.
3. Look for sleep calls and retry loops around the handler, and for DeleteMessage inside a catch.
4. Compare the handler's maximum run time with the VisibilityTimeout. If they're close, extend visibility during processing instead of hoping it finishes in time.