Laravel Job timeout vs retry_after: The Ordering Rule Nothing Enforces

Two numbers, in two different configuration files, decide whether a queued job runs once or twice. One is retry_after, in config/queue.php. The other is your worker's timeout, in config/horizon.php or on the job class itself. Laravel ships them 30 seconds apart and mentions the relationship twice in prose. No code checks it at boot, at dispatch, or when a supervisor starts a worker.

Get the order wrong and a job that is still running is handed to a second worker. What you see next depends on a third setting, tries. Either two workers execute the same job concurrently and both report success, or a job you never saw run twice lands in your failed jobs list with MaxAttemptsExceededException, for work that completed. The rest of this article covers the mechanism behind both symptoms, the exact defaults, and the ordering rule you have to enforce yourself.

Key takeaways

• retry_after is a lease, not a delay. Popping a job reserves it for retry_after seconds. When the lease expires the job goes back on the ready queue whether or not it is still running.
• The defaults are 60 and 90, if you kept the shipped config. Worker timeout defaults to 60s, the shipped retry_after to 90s. Omit retry_after from a connection and the framework falls back to 60, making the two equal.
• Equal values are not safe. The lease starts at pop() and the timeout alarm is armed a few steps later, so with matching values the reservation always expires first.
• The symptom depends on tries. With tries => 1 (the value in Horizon's published config) you get a failure recorded against a job that succeeded. With tries => 2 or more you get concurrent double execution, and nothing fails.
• The rule: job timeout < supervisor timeout < retry_after, with margin at each step. Nothing in Laravel tells you when it stops holding.

The two numbers and their defaults

Both settings are measured in seconds and are easy to confuse. retry_after belongs to the queue connection and sets how long a reservation is honoured. timeout belongs to the worker and sets how long a single job may occupy a process. They have different owners, run on different clocks, and nothing enforces a relationship between them.

A stock Laravel skeleton pairs a 60-second timeout with a 90-second retry_after, which is a correct configuration with 30 seconds of slack. But the 90 is a value in your configuration file, not a framework default. The framework's own fallback, used whenever a connection array has no retry_after key, is 60:

// Illuminate\Queue\Connectors\RedisConnector
return new RedisQueue(
    $this->redis, $config['queue'],
    Arr::get($config, 'connection', $this->connection),
    Arr::get($config, 'retry_after', 60),   // not 90
    // ...
);

A connection written by hand (a second Redis connection for a dedicated queue, a connection built in a test, anything copied from a blog post rather than from config/queue.php) gets 60 with no warning, equal to the default worker timeout. That configuration is broken from the first job that runs long, and it looks reasonable in a pull request.

What retry_after does

The name suggests a delay before a retry. It is a lease: popping the job sets a deadline on its reservation, and once that deadline passes any worker can take the job, including while the first worker is still running it.

On Redis, popping a job is a single Lua script. It lpops the payload off the ready list, increments the payload's attempts counter, and adds the result to a sorted set, scored with the moment the reservation expires:

-- Illuminate\Queue\LuaScripts::pop()
local job = redis.call('lpop', KEYS[1])
local reserved = false

if(job ~= false) then
    -- Increment the attempt count and place job on the reserved queue...
    reserved = cjson.decode(job)
    reserved['attempts'] = reserved['attempts'] + 1
    reserved = cjson.encode(reserved)
    redis.call('zadd', KEYS[2], ARGV[1], reserved)
    redis.call('lpop', KEYS[3])
end

ARGV[1] is now + retry_after. No timer watches that score. Instead, every pop() on that queue begins by sweeping it:

// Illuminate\Queue\RedisQueue
public function pop($queue = null, $index = 0)
{
    $this->migrate($prefixed = $this->getQueueRedisKey($queue));
    // ...
}

migrate() moves two things back onto the ready list: delayed jobs whose time has come, and reserved jobs whose lease has expired. It cannot tell the difference between a job whose worker was killed by a deploy and a job whose worker is still running it. Both look like an expired score.

The database driver reaches the same outcome through SQL rather than a sorted set. isReservedButExpired() widens the "next available job" query to include any row whose reserved_at is older than now - retry_after, so it also has no way to know whether the original worker is alive.

Two drivers behave differently. Beanstalkd has no retry_after default of its own and falls back to Pheanstalk's TTR. SQS has no retry_after at all: the connector never reads the key, because the lease lives on Amazon's side as the queue's Default Visibility Timeout. Its default is 30 seconds, half of Laravel's default worker timeout. An SQS queue created with the console defaults and consumed by a stock worker is misconfigured from the start, and no edit to config/queue.php will fix it.

What timeout does

The worker's timeout is independent of the queue. It is a POSIX alarm, set per job, inside the worker process:

// Illuminate\Queue\Worker::registerTimeoutHandler()
pcntl_signal(SIGALRM, function () use ($job, $options) {
    // ...record the failure, dispatch JobTimedOut...
    $this->kill(static::$timedOutExitCode ?? static::EXIT_ERROR, $options, WorkerStopReason::TimedOut);
}, true);

pcntl_alarm(
    max($this->timeoutForJob($job, $options), 0)
);

Three consequences follow from this code, and each can break the ordering rule without anyone touching config/queue.php:

• The job's own timeout takes precedence. timeoutForJob() returns the job's $timeout property or #[Timeout] attribute if it has one, and only falls back to the worker's option. A single #[Timeout(300)] on a slow report job overrides a correctly configured 60-second supervisor and outlasts a 90-second lease.
• --timeout=0 disables it. pcntl_alarm(0) cancels the alarm rather than firing immediately, so a zero timeout means no timeout. Nothing bounds the job's runtime, while the lease keeps expiring on schedule.
• Without pcntl there is no timeout. The handler sits behind a supportsAsyncSignals() check. On a build without the extension no alarm is set and nothing reports that, so long jobs are duplicated with no local symptom to reproduce.

Why the two collide, and why equal values fail

Take a 90-second job with retry_after and timeout both set to 90, a configuration that looks tidy in a config file:

t=0.00   worker A pops the job; reservation scored to expire at t=90.00
t=0.02   JobReserved fires, the worker enters process()
t=0.03   pcntl_alarm(90) armed, will fire at t=90.03
t=90.00  worker B pops the same queue; migrate() sees an expired score
         and moves the still-running job back onto the ready list
t=90.00  worker B pops it, attempts becomes 2
t=90.03  worker A's alarm fires, 30ms after worker B took the job

The lease starts at the pop. The alarm is armed several steps later: after the reservation is written, after the JobReserved event and its listeners run, and after the worker enters process(). With identical values the reservation expires first every time, by the length of that setup. Even when the alarm does fire first, the handler still has to record the failure and exit before the process lets go of anything.

This is also why the bug is hard to reproduce. Migration is lazy: it only happens when somebody pops that queue. A supervisor with a single worker, occupied by the long job, never sweeps its own expired reservation, so the misconfiguration stays invisible until a second worker exists. Autoscale from one process to two, or move from a laptop to production, and the same code starts running jobs twice.

The two symptoms, and which one you get

Once the job has been migrated back and popped by a second worker, tries decides what happens next. The Lua script incremented attempts on that second pop, and Worker::process() checks the attempt count before it fires the job:

// Illuminate\Queue\Worker::process()
$this->raiseBeforeJobEvent($connectionName, $job);

$this->markJobAsFailedIfAlreadyExceedsMaxAttempts(
    $connectionName, $job, (int) $options->maxTries
);

// ...

$job->fire();

With tries => 1: a failure on work that succeeded

One attempt is what most Horizon installs run with: the config/horizon.php that Horizon publishes sets 'tries' => 1 on its supervisor, and a job class with no $tries of its own inherits it. (Drop the key entirely and horizon:supervisor falls back to --tries=0, which means unlimited.) The migrated copy arrives at worker B carrying attempts = 2, exceeds the limit before fire() is reached, and is failed immediately with MaxAttemptsExceededException. Its body never runs.

There is no duplicate execution, but you now have a failed job in the dashboard for work that is still running and completes successfully a few seconds later. Worker A finishes, tries to delete its reservation, and removes nothing, because the entry was migrated away while it worked. The same job id ends up with one failure and one success, and the stack trace points at a timeout that never happened.

If you have chased "MaxAttemptsExceededException on a job that clearly worked", this is one of two ways to get there. The other is lock-based middleware spending attempts on releases, which has its own set of failure modes.

With tries => 2 or more: concurrent execution with no error

Most applications raise tries eventually, because retries are the point of a queue and because middleware like RateLimited and WithoutOverlapping consume attempts. With two or more attempts allowed, the check passes, and worker B calls fire() while worker A is inside the same job's handle().

Both copies run to completion, so the card is charged twice or the email goes out twice. Neither copy throws or is marked failed, and the dashboard shows one completed job. Laravel, Horizon and your error tracker record nothing unusual. The evidence is in your data, as duplicate rows, doubled counters or a customer with two receipts.

Four ways a correct config becomes an incorrect one

timeout usually ends up above retry_after as a side effect of a change that looked local:

• A job gained a timeout of its own. A nightly export starts taking four minutes, someone adds #[Timeout(300)] to the job class, and the supervisor's 60 no longer applies to that job while the connection's 90-second lease is unchanged. The job class is the highest-precedence setting and the one furthest from the config file where the constraint lives.
• A supervisor timeout was raised to fix timeouts. Jobs are being killed at 60 seconds, so the supervisor timeout goes to 120. The kills stop and the lease problem above starts, because retry_after is in a different file and was not part of the change.
• A new connection was added without the key. It gets the 60-second framework fallback described above. The values are equal from the first deploy, and the diff contains no line that would catch it.
• The queue moved to SQS. The lease is now a Default Visibility Timeout of 30 seconds set on the AWS side, and config/queue.php has no say in it.

The opposite mistake is setting 'retry_after' => null on a Redis connection. That skips reserved-job migration altogether, so a job whose worker dies is never recovered. You avoid duplicates and lose jobs instead, with no error to show for it.

The ordering rule, with numbers

The full chain has three links, because Horizon's supervisor timeout is a separate value from the job's, and with balance => 'auto' Horizon force-kills workers it considers hung during scale-down:

job timeout < supervisor timeout < retry_after

Each step needs more than a second of headroom. A worked example for a job whose realistic worst case is three minutes:

Over-shooting retry_after has a small, bounded cost: a job whose worker did die waits longer before it is recovered. Under-shooting it causes duplicate execution, so when in doubt, make the lease longer.

A supervisor serves one connection, so you can check the constraint in a test:

// tests/Feature/QueueTimeoutOrderingTest.php
public function test_supervisor_timeouts_stay_below_their_connection_lease(): void
{
    foreach (config('horizon.defaults') as $name => $supervisor) {
        $connection = $supervisor['connection'];

        // The 60 mirrors the framework's own fallback when the key is absent.
        $retryAfter = config("queue.connections.{$connection}.retry_after", 60);

        $this->assertLessThan(
            $retryAfter,
            $supervisor['timeout'] ?? 60,
            "Supervisor [{$name}] can hand a running job to a second worker.",
        );
    }
}

The test covers supervisors. It cannot see a $timeout property or a #[Timeout] attribute on an individual job class, which is the most common way the chain breaks. Those you can only catch at runtime, when the worker resolves the effective timeout for the job in front of it.

Skyline runs the supervisor half of that check for you: php artisan horizon warns at startup about any supervisor whose timeout is not below its connection's retry_after.

Catching it in the logs

This misconfiguration can survive in production for months because the queue's own instrumentation reports it as ordinary activity. An expired reservation looks like a recovered job, and a second worker picking the job up looks like any other pickup.

Skyline's job lifecycle logging writes one line per transition, tagged with the job id, including the transitions Laravel does not log: reserved, migrated, released and timed out. When a job leaves the reserved set because its lease ran out, the line is a warning that names the cause, and the pattern to look for is a single job id reserved twice with that warning between the two reservations:

[11:04:12] queue.DEBUG: [job:91827364] reserved from [exports] and started processing.
[11:05:42] queue.WARNING: [job:91827364] was still reserved on [exports] when its reservation expired (retry_after=90s) and has been put back on the queue. Either the worker running attempt 1 died, or the job runs longer than retry_after and this copy will run while the first is still going; keep every job's timeout below retry_after (reason=reservation_expired).
[11:05:45] queue.DEBUG: [job:91827364] reserved from [exports] and started processing.
[11:06:20] queue.DEBUG: [job:91827364] completed.

The job was picked up at 11:04:12. Ninety seconds later, still running because no completion line had appeared, it was put back on the ready queue and picked up again three seconds after that. A grep for the job id is enough to tell a retry from a double reservation, which the dashboard alone cannot do. The reserved lines are debug, so enable that level on the channel for the investigation and turn it off afterwards; the expired-reservation line is a warning and can stay on permanently.

Further reading

If you are working through queue reliability more broadly, 12 best practices for Laravel background jobs covers the idempotency habits that make a double execution survivable rather than expensive, and rate-limited APIs and Laravel queues covers the middleware that pushes jobs toward their timeout in the first place. For the logging shown above, see job lifecycle logging. Skyline vs Horizon covers what else changes when you swap the package.

Common questions

What is retry_after in Laravel queues?

It is a lease on a reserved job, not a delay before a retry. Popping a job writes it to the connection's reserved set with an expiry of now plus retry_after seconds. Every later pop on that queue first sweeps that set and moves anything expired back onto the ready queue, whether or not the worker that reserved it is still running it. Nothing checks the original worker is alive, so an expired lease on a live job looks the same as one on a worker killed by a deploy.

What are the default values of timeout and retry_after?

The worker timeout defaults to 60 seconds. That is the default of queue:work --timeout, queue:listen --timeout and the Horizon supervisor timeout option. retry_after is 90 seconds in the config/queue.php that ships with the Laravel skeleton, but the framework's own fallback, used whenever a connection array omits the key, is 60. SQS has no retry_after at all; its lease is the queue's Default Visibility Timeout on the AWS side, which defaults to 30 seconds.

Why is my Laravel job running twice?

Almost always because its effective timeout is not smaller than the connection's retry_after, so the reservation expires while the job is still running and a second worker picks the same payload up. The effective timeout may not be the one in your supervisor config: a $timeout property or #[Timeout] attribute on the job class takes precedence, and --timeout=0 disables the alarm entirely rather than firing it immediately.

Is it safe to set timeout equal to retry_after?

No. The reservation clock starts at pop(), while the timeout alarm is armed several steps later: after the reservation is written, after JobReserved and its listeners run, and after the worker enters process(). With identical values the reservation expires first every time, and the alarm handler still has to record the failure and exit after that.

Why did my job fail with MaxAttemptsExceededException when it actually succeeded?

Because the reserved copy was migrated back and popped by a second worker, and the pop incremented the attempt count. With tries set to 1, as in the config/horizon.php that Horizon publishes, the second copy exceeds its limit before fire() is reached and is failed immediately, while the first copy is still running and goes on to complete. With tries above 1 the attempt check passes and both copies execute, with no failure recorded to tell you.