Symfony Messenger is great. It has so many features and is super flexible. I want to highlight one of these features in this small post: Failures and retries.
Over in the Symfony docs failures and retires are briefly mentioned and it explains what values you can configure. However, it does not give a good recommendation for what values to use. The simple answer to “What values should I use?” is “It depends”.
Depending on what your application requirements are these values will obviously be different.
The parameters we can work with are:
- max_retries: How many times we will try to process a message.
- delay: A time unit in milliseconds.
- multiplier: Used with the
delayto calculate the time to wait between retries.
- max_delay: The longest time between retries.
The complex thing here is how the time between retries is calculated. The default
calculation could be found in
delay * (multiplier ^ retry_count)
So with a delay of 10ms, multiplier of 2:
- Dispatch message: 0ms
- Retry 1: 10 * (2 ^ 1) = 20ms
- Retry 2: 10 * (2 ^ 2) = 40ms
- Retry 3: 10 * (2 ^ 3) = 80ms
- Retry 4: 10 * (2 ^ 4) = 160ms
max_retries limit has been reached, the message will be moved to the
failure_transport. The failure transport is special and how one retry messages
in that transport is described in the Symfony documentation.
My retry strategy
If a message fail, I assume it is because of something temporary. Say that an HTTP request failed because the remote server was unavailable. So I want to retry that message fairly soon. That is why I start with a low delay.
If that message has failed a few times, I want to wait to back off quickly. That is
why I use a multiplier of 4. But I also don’t want it to wait too long. So I set
max_delay to 30.000ms.
- Dispatch message: 0ms
- Retry 1: 1.000 * (4 ^ 1) = 4.000ms
- Retry 2: 1.000 * (4 ^ 2) = 16.000ms
- Retry 3: 1.000 * (4 ^ 3) = 30.000ms
- Retry 4: 1.000 * (4 ^ 4) = 30.000ms
After 4 retries the message is moved to the failure queue.
The failure queue is also configured with retries, but it reties messages more slowly.
Here I’ve configured the
delay to 300.000 and a
multiplier of 3. The
is configured to 86.400.000 (1 day).
- Retry 1: 15 min
- Retry 2: 45 min
- Retry 3: 2 hours 15 min
- Retry 4: 6 hours 45 min
- Retry 5: 20 hours 15 min
- Retry 6: 24 hours
- Retry 7: 24 hours
- Retry 8: 24 hours
- Retry 9: 24 hours
After 20 retries I’ve decided to remove the message completely. I assume (maybe incorrectly) that if an error has not been fixed after more than 2 weeks, then there is no point of continue to retry.
The Symfony configuration looks like this: