Laravel Queues Under the Hood

Queues are the secret engine behind many apps. They can turn slow, fragile tasks into fast, reliable workflows. Whether you’re sending thousands of emails, processing videos, or orchestrating complex pipelines.

This article goes behind the curtains on how Laravel’s queue system works. From the moment you dispatch a job, to how it’s serialized, stored, popped, executed and retried. We’ll trace the full lifecycle of a message, looking into workers, visibility timeouts, backoff, chains and batches.

Get a coffee and prepare to really understand how queues work in Laravel (and in general).

Queues keep your app fast and reliable. By moving slow or risky work off the request cycle, Laravel lets you return responses quickly, absorb traffic spikes, run tasks in parallel and even retry them in a safe way if something goes wrong. They decouple responsibilities (web -> workers), isolate failures, and give you control over timing, and priority.

• Offload I/O-heavy tasks: emails, webhooks, PDFs, image/video processing, AI/API calls.

• Improve reliability: retries with backoff and timeouts.

• Scale predictably: add workers without changing business code.

Before diving into how Queues work in Laravel, let's learn some concepts that we will be using in this article that can help make things simpler.

• Job: A small class (ShouldQueue) whose handle() does the work.

• Dispatch: Sends a job to a connection with metadata (queue, delay, attempts).

• Connection: The backend you use (Redis, Database, SQS), chosen by name.

• Queue (name): A lane within a connection for priorities (high, default, low).

• Driver/Connector: The implementation that pushes/pops jobs on the connection.

• Payload & serialization: JSON envelope where models serialize to IDs and closures via SerializableClosure.

• Worker: php artisan queue:work, a long-running process that pops, runs middleware, and acknowledges jobs.

• Visibility timeout: The “lock” window. It must exceed job runtime to avoid duplicates.

• Acknowledgement: Delete on success. Release to retry later. Bury/fail after limits.

• Retries & backoff: Control how often and how long to wait between attempts.

• Failures: Failed jobs land in failed_jobs.

• Middleware: Cross-cutting rules like throttling and rate limiting.

• Chains & batches: Sequential flows and grouped jobs with progress/failure rules.

• Idempotency: Safe to run twice. Use unique keys, upserts, and guards.

If we look it in a really simplified way, Laravel's Queue System is a simple pipeline: create a job, serialize it, store it, pop it, run it, and acknowledge it. All of it, having safety mechanisms for retries and failures.

Dispatch

You call Job::dispatch(...). The Bus serializes the job (class + data + metadata) and hands it to the Queue Manager.

Connector

The Queue Manager picks a connector (Redis, Database, SQS) and pushes a JSON payload onto the chosen queue (name/lane).

Worker

The php artisan queue:work runs as a long‑lived process. It reserves a job, locks it (visibility timeout), and resolves dependencies from the container.

Middlewares

Throttling, retries, rate limits, etc., wrap the Jobs handle() method.

Acknowledge or retry

On success, deletes the job. Exceptions trigger release (retry with backoff) or mark as failed after maxTries.

Example

class ProcessReport implements ShouldQueue
{
    use Queueable, SerializesModels;

    public int $timeout = 30;

    public function __construct(public Report $report) {}

    public function backoff(): array
    {
        return [5, 30, 90];
    }

    public function handle(): void
    {
        // Process report logic here
    }
}

// Dispatch after DB commit, on a high-priority queue
ProcessReport::dispatch($report)
    ->afterCommit()
    ->onQueue('high');

# Running the worker
php artisan queue:work --queue=high,default --timeout=35 --max-jobs=1000 --max-time=3600

Before diving a little deeper on how Laravel handles the dispatch logic. Let's check a high-overview on how the job goes from your code to the queue connection.

• You dispatch the job: Job::dispatch(...) and Laravel's Bus collects the options (queue, connection, delay, etc).
• The job is serialized (we're going to check how this works in the next section).
• The Queue Manager picks a connection (Redis, Database, SQS) and queue name, uses a connector to push the payload, respecting delay/visibility settings.

Now that we have an idea on how it works, let's dive a little more on what happens when you call Job::dispatch(...).

The Dispatchable trait's dispatch() method returns a PendingDispatch wrapper around your job and chained calls like ->onConnection(), ->delay(), are proxied to the job.

The Queueable trait is the one that provides these fluent methods above and where the properties like $connection, $delay, and others are set.

When the PendingDispatch dispatches, either by explicitly calling the dispatch() method or implicitly on destruction, the Bus checks:

• ShouldQueue: decides between dispatching to queue vs run inline. It also checks if it should dispatch only after the response is sent.

• Connection/queue: $job->connection and $job->queue (or config defaults).

• Timing: $job->delay, $job->timeout, backoff(), retryUntil()

• Safety: ShouldBeUnique/uniqueId(), ShouldBeEncrypted, afterCommit

The Queue Manager then picks the connector (Redis/Database/SQS), applies delay/visibility settings, and pushes a JSON payload that includes the job class, data, and all the collected metadata.

When you dispatch a job in Laravel, the framework wraps it in a compact JSON “envelope” that workers can pop, decode, and run safely and repeatedly.

Below we're going to see some of the most important values that are stored in the JSON "envelope" that Laravel creates when dispatching a job.

• job: Always Illuminate\Queue\CallQueuedHandler@call (the generic invoker).

• uuid and displayName: For tracking and logs.

• data.commandName: Your job class name.

• data.command: A serialized clone of your job object.

• Timing properties: timeout, timeoutAt, retryUntil, backoff, maxTries, maxExceptions

• tags: For metrics.

• encrypted: true (only if the job implements ShouldBeEncrypted).

Below we're going to see how the serialization works for different types of data when dispatching a job.

• Job object: The Bus clones your job and serializes it using PHP’s serialize(). Drivers store that serialized blob inside JSON (and may base64 it).

• Eloquent models: If your job uses the SerializesModels trait, each model becomes a lightweight ModelIdentifier (class, id, connection, relations). On the worker, Laravel re-hydrates models by re-querying the database—keeping payloads tiny and consistent.

• Closures: Closure-based jobs are serialized using the laravel/serializable-closure package. They work, but prefer concrete job classes for safer rolling deploys.

• Encryption: Jobs that implement the ShouldBeEncrypted interface have their serialized payload encrypted before being written to the connection.

{
  "uuid": "b4c1e7c6-...",
  "displayName": "App\\Jobs\\ProcessReport",
  "job": "Illuminate\\Queue\\CallQueuedHandler@call",
  "maxTries": 3,
  "timeout": 30,
  "backoff": [5, 30, 90],
  "tags": ["important", "client:42"],
  "data": {
    "commandName": "App\\Jobs\\ProcessReport",
    "command": "SERIALIZED JOB HERE"
  }
}

Redis is the most popular Laravel queue connection, it's simple, fast and even better when you pair it with Laravel Horizon. We're going to check how it works under the hood, but for that we need first to take a look at some concepts from Redis.

• zset (sorted set): An ordered collection where each member has a numeric score. The items are retrieved by score order (e.g., timestamps for scheduling).

• RPUSH: Appends a value to the tail of a list.

• ZADD: Adds or updates a member in a sorted set with a score.

• BLPOP: Blocking pop from the head of a list. It waits until an item is available (or timeout).

Under the hood, it uses one list, plus two sorted sets.

• Main queue (list): key queues:default - RPUSH payloads here

• Delayed jobs (zset): key queues:default:delayed - ZADD with score = now + delay

• Reserved jobs (zset): key queues:default:reserved - ZADD with score = now + visibilityTimeout

The popping of jobs from the queue is done atomically, and it has three phases.

It moves the items whose score <= now from the delayed set to the main list, so they become ready.

It moves the expired reserved items (visibility timeout passed) from the reserved set to the main list, so crashed/slow workers get retried.

Both these migration actions uses Lua scripts for atomicity.

It runs a BLPOP in the main list and immediately runs a ZADD for the popped payload into the reserved set with score = now + retry_after (visibility timeout).

The visibility timeout can be configured in the retry_after setting of the config/queue.php file in the redis section.

The BLPOP time can be configured in the same file and section, in the block_for setting.

After the BLPOP, the worker executes the handle() method of the job with all the configured middleware, timeouts, etc.

We have three different paths when it comes to acknowledging or releasing the jobs.

• Success: Remove the payload from the reserved zset.

• Retry: Increment attempts and move it to delayed sorted set with a new score.

• Fail: Remove the payload from the reserved sorted set and record in failed_jobs.

Now is time to learn a little more on how things work under the hood for the workers, when you run php artisan queue:work.

It is a long‑running daemon that boots your app once, then loops: pop → reserve → run → ack/retry → repeat. It’s fast, observable, and designed for safe restarts. Let's understand how it works under the hood.

When you run php artisan queue:work, it boots the container and your app, and then it enters a loop.

The worker asks the driver for a job. The driver atomically reserve the job with a visibility window, the retry_after settting.

The job is reconstructed from its JSON payload, dependencies are resolved, and the middlewares, if any, are executed wrapping the handle() method.

While running the job, a per-job timeout is enforced, and if the job overruns it, the job is killed. The job will be available again after the visibility timeout is reached.

In a success run, the job is removed from the reserved set and the JobProcessed event is fired.

If an Exception happens, it increment attempts, compute backoff and release to the delayed set.

If maxTries/maxExceptions/retryUntil is hit, the job is marked as failed, added to the failed_jobs and the JobFailed event is fired.

For each loop, the worker also do some housekeeping and checks for things like checking if the app is in maintenance mode, checking if the max jobs setting was not reached as well as handling graceful quit flags, for example when using the queue:restart.

Below is a tiny and simple mental model using pseudocode for a better understanding the mechanics of the worker.

while (! shouldQuit()) {
    if (! $job = $queue->pop($queues)) {
        sleepOrBlock(); // e.g., Redis BLPOP
        continue;
    }

    startTimeoutTimer($job->timeout); // pcntl on Unix
    try {
        runMiddlewareThenHandle($job);
        ack($job);
        dispatch(JobProcessed::class);
    } catch (\Throwable $e) {
        handleFailureOrRelease($job, $e); // backoff / fail
    } finally {
        stopTimer();
    }

    enforceLimits(); // max jobs, max time, quit flags
}

Laravel also offers some features to better orchestrate the jobs and the queues, let's see how some of these work.

This is a middleware provided by Laravel that prevents two workers from handling the same "thing" at the same time.

public function middleware(): array
{
    return [
        new WithoutOverlapping($this->userId),
    ];
}

Under the hood, when you add this middleware, it acquires an atomic cache lock cache()->lock($key, $ttl). If the lock is held, the job isn’t processed now, it’s released back to the queue with a small delay so another attempt can try later. On success/failure, the lock is released. If the worker crashes, the lock auto-expires after its TTL, preventing deadlocks.

Laravel also provides a way for duplicate jobs not even entering the queue. On dispatch, Laravel tries to acquire a cache lock keyed by job class + uniqueId(). If it can’t, the new dispatch is skipped. The lock expires after the $uniqueFor seconds is achieved (if set) or when the job is released.

Laravel provides two different interfaces for this.

• ShouldBeUnique: Lock lasts until the job finishes (or the TTL ends), preventing multiple queued or running duplicates.

• ShouldBeUniqueUntilProcessing: Lock is released as soon as a worker starts processing, allowing a new copy to be queued while the current one runs (but not queued in parallel beforehand).

As we saw above, the WithoutOverlapping middleware and the ShouldBeUnique and ShouldBeUniqueUntilProcessing interfaces rely on cache locks with a TTL set. The locks must expire to avoid permanent blocks in case of a worker dies. In Redis this uses the SET NX EX under the hood.

• SET: Writes a value to a key.

• NX: Only set if the key does not already exist.

• EX: Set a time-to-live so the key auto-expires after N seconds.

With this, Laravel releases the lock on completion, and if the process crashes, Redis expires it automatically.

Laravel has a consistent and safe way to handle retries and failures of jobs in the queue, so you don't have to worry when something goes wrong. Here's how it works under the hood.

When a worker pops a job from the queue, it increments the attempts on the payload. If a timeout or an Exception happens, instead of acknowledging and deleting the job, it will reappear again in the queue after the visibility window and it will increment the attempts once again when it's popped.

Before the job is again released to be retried, it checks the value from the backoff() method or an explicit call to the releaseAfter() method, and then it sends the job to the delayed sorted set, incremeting now() with one of these set values. If none of these are found, the job is released to be attempted again immediately.

To check if the job should be retried, the worker checks the $tries, $maxExceptions properties and the retryUntil() method. If the number of tries and exceptions exceed the configured ones or now() > retryUntil(), the job is then marked as failed.

When marked as failed, the worker throws the MaxAttemptsExceededException exception, deletes the job from the queue, records it in the failed_jobs table, fires the JobFailed event and calls the job failed() method.

Below we can see the same job we had before, but with a better configuration for the retries and failure.

class ProcessReport implements ShouldQueue
{
    use Queueable, SerializesModels;

    public int $tries = 5;
    public int $timeout = 30;
    public int $maxExceptions = 3;

    public function __construct(public Report $report) {}

    public function backoff(): array
    {
        return [5, 30, 90];
    }

    public function retryUntil(): DateTime
    {
        return now()->addMinutes(30);
    }

    public function handle(): void
    {
        // Process report logic here
    }

    public function failed(?Throwable $exception): void
    {
        // Send notification of failure
    }
}

When handling jobs in the queues, timing is very important, and Laravel makes it easy to manage this. Three "clocks" are the most important: when a job becomes available (delay), how long it stays "locked" for a worker (visibility) and how long your code is allowed to run (timeout). Let's check how these work using the Redis connection.

Delay

Redis uses the delayed sorted set for this with a score now() + delay. The delay can be set when dispatching the job:

ProcessReport::dispatch($report)
    ->delay(now()->addMinutes(10));

Visibility

When the worker pops the message from the queue, the payload is moved to the reserved sorted set with a score now() + retry_after. This can be configured in the config/queue.php file under the redis section, and if the job is not acknowledged in time, the job is then migrated back to the main queue for a possible retry.

Timeout

The worker will enforce a maximum time to live for the job. This can be set either in the $timeout property of the job itself for job-specific timeouts, or in the --timeout option when running the php artisan queue:work command, that will apply to all jobs. After this timeout is exceeded, the job is killed and after the set visibility period, it is migrated back to the main queue for a possible retry.

Laravel's Bus gives us more powers than just running plain jobs. It offers a way to orchestrate sequential chains or parallel batches of jobs, opening a lot of new possibilities when managing jobs in the queue.

The concept of a Chain of jobs is quite simple, it runs jobs in order, and if any of these fails, the remaining jobs in the chain are skipped.

The payload of the jobs in the chain contain a serialized chained array with all the jobs on it. After a job is acknowledged, the worker then dispatches the next serialized job.

If a job fails, the remaining jobs in the chain are skipped and if you have defined a catch() callback, it will be run.

Bus::chain([
    new ProcessPayment($order),
    new GenerateInvoice($order),
    new EmailInvoice($order),
])->onConnection('redis')
    ->onQueue('high')
    ->catch(fn (Throwable $e) => report($e))
    ->dispatch();

While the concept of a Chain is to run the jobs in sequence, a Batch is when you want to run multiple jobs in parallel. You can track the progress of a batch, handle completion and failure hooks, allow cancellation of it and even add more jobs to it.

Under the hood, all this tracking happens in the job_batches table, where it tracks how many jobs the batch has, how many are pending or failed, which ones failed and if the batch is cancelled.

Each job in the batch will receive a batchId, and when one of these jobs are complete or fail, the job_batches table is updated with the information. Using this tracking, the batch offers us different hooks that we can run additional logic based in the state of the batch.

class ImportVideoChunk implements ShouldQueue
{
    use Batchable, Queueable, SerializesModels;

    public function __construct(
        public Video $video,
        public ?int start,
        public ?int end,
    ) {}

    public function handle(): void
    {
        if ($this->batch()->cancelled()) {
            return;
        }

        // Process video import logic here
    }
}

$batch = Bus::batch([
    new ImportVideoChunk(video: $video, end: 60),
    new ImportVideoChunk(video: $video, start: 61, end: 120),
    new ImportVideoChunk(video: $video, start: 121, end: 180),
    new ImportVideoChunk(video: $video, start: 181, end: 240),
    new ImportVideoChunk(video: $video, start: 241),
])->before(function (Batch $batch) {
    // Runs when the batch is created, but no jobs were run
})->progress(function (Batch $batch) {
    // Runs when a job from the batch has completed without errors
})->then(function (Batch $batch) {
    // Runs when all jobs are completed without errors
})->catch(function (Batch $batch, Throwable $e) {
    // Runs when a job from the batch fails
})->finally(function (Batch $batch) {
    // Runs when the batch executed all the jobs
})->dispatch();

As you can see above, the hooks provide you access to a Batch $batch object, that you can use to inspect and manage the current batch. The same object can be accessed inside the jobs with the $this->batch() method provided by the Batchable trait.

Just to sum everything, Queues are how Laravel turns slow, fragile work into fast, reliable experiences. You dispatch a small job, Laravel serializes it, a connector (like Redis) stores it, workers pop and run it through middleware, and the system retries or fails gracefully with clear events and metrics.

All of this achieved with an elegant and simple API that every developer can use for improving applications.

With this article you learned about the mechanics under the hood that empower the Laravel Queue System, how queues work in general, as well as having a quick look on how the Redis connector works with the Laravel Queue System.

I hope that you liked this article and if you do, don’t forget to share this article with your friends!!! See ya!