Laravel's queue system works out of the box. php artisan queue:work runs a worker, jobs get processed, life is good — until traffic grows and the queue gets backed up, or workers start consuming gigabytes of memory, or a slow job pins everything while urgent work waits.
This post is the configuration knobs that actually matter when you take queue workers seriously, what each one does, and the failure modes that come from getting them wrong.
How a Worker Actually Runs
A queue worker is a long-lived PHP process. It pulls jobs from the queue, executes them in sequence, and loops. Between jobs, it stays alive — Laravel deliberately does not reboot the process per job to avoid PHP's bootstrap cost.
This has consequences. The worker's memory, opcaches, and singletons live across jobs. A memory leak in one job affects the next. A bug that mutates a global registry persists.
Horizon adds management on top of this: process supervision, queue-balancing, metrics. The fundamental model is the same.
Process Count
The first knob: how many worker processes per server.
Too few, and the queue backs up. Too many, and you starve the database or hit other concurrency limits. There is no magic formula; the answer depends on what your jobs do.
For CPU-bound jobs (PDF generation, image processing): processes ≈ CPU cores. Going higher does not help — you are limited by CPU.
For I/O-bound jobs (HTTP calls, database queries): processes ≈ 4–8× CPU cores. Most of the time is spent waiting; you can run more in parallel without saturating CPU.
For mixed workloads: start at 2× CPU cores and tune from there.
Horizon configuration:
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default'],
'balance' => 'auto',
'minProcesses' => 2,
'maxProcesses' => 20,
'balanceMaxShift' => 1,
'balanceCooldown' => 3,
'memory' => 128,
'tries' => 3,
],
],
The balance: auto setting lets Horizon scale processes within min and max based on queue load.
Memory Limit
The memory option restarts the worker if its memory exceeds the limit (in MB). The limit catches leaks before they take down the server.
A reasonable default is 128 MB for general workloads. For jobs that intentionally use more — image processing, large CSV exports — increase as needed.
Setting it too low causes constant worker restarts. Setting it too high lets leaks compound. Watch the worker's actual memory under production load and pick a value 50% above the steady-state peak.
Job Timeouts
Each job has a maximum runtime. If it exceeds, the worker kills it and (if --tries > 1) requeues for retry.
final class GenerateInvoicePdf implements ShouldQueue
{
public int $timeout = 60;
public function handle(): void { /* ... */ }
}
Match the timeout to the job's actual work. A PDF that takes 5 seconds should have a timeout of 30, not 600. A timeout that is too generous means a stuck job ties up a worker for that long.
Worker-level timeout via --timeout=N is a hard ceiling above per-job timeouts. Set it to the longest expected job plus headroom.
Retries and Backoff
Failed jobs retry. The retry policy matters:
final class ChargePayment implements ShouldQueue
{
public int $tries = 5;
public array $backoff = [5, 30, 60, 300, 600]; // seconds between attempts
public function handle(): void { /* ... */ }
}
Exponential or jittered backoff prevents the failed-job cluster from retrying in lockstep. Same logic as HTTP retry backoff — give the failing dependency time to recover.
After the final retry, the job lands in failed_jobs. Have a real plan for what happens there: dashboard review, alerts, automatic notification to the affected user.
Concurrency Limits
When a job class hits an external rate limit or a single-tenant resource, you do not want all workers running it at once. Laravel's WithoutOverlapping and Redis-based concurrency limits help.
public function middleware(): array
{
return [
(new WithoutOverlapping($this->customer_id))->expireAfter(180),
];
}
This prevents two jobs with the same customer ID from running concurrently. Useful for jobs that mutate per-tenant state.
Global concurrency limits via Bus::chain or Queue::pushOn with rate-limit middleware control total throughput against external APIs.
Queues per Workload
The most underused pattern: split work onto separate queues with separate workers.
ChargePayment::dispatch($order)->onQueue('payments');
ExportReport::dispatch($report)->onQueue('exports');
SendWelcomeEmail::dispatch($user)->onQueue('emails');
'supervisor-payments' => [
'queue' => ['payments'],
'minProcesses' => 4,
'maxProcesses' => 10,
],
'supervisor-exports' => [
'queue' => ['exports'],
'minProcesses' => 1,
'maxProcesses' => 3,
],
'supervisor-emails' => [
'queue' => ['emails'],
'minProcesses' => 2,
'maxProcesses' => 20,
],
Why split? Three reasons:
- Different SLAs. Payments need to clear in seconds; reports can wait. Separating queues prevents a slow report from delaying a payment.
- Resource isolation. A bug in one queue does not consume workers needed by others.
- Different scaling. Email volume spikes during marketing campaigns; payment volume is steadier. Each queue scales independently.
Long-Running Jobs
Jobs that take minutes to complete need different treatment.
- Don't run them in the same worker pool as fast jobs. They will pin workers and back up the queue.
- Use a dedicated queue and a small worker count for long jobs.
- Consider whether the job can be broken into smaller chunks. A "generate report for all customers" job is much worse than ten "generate report for tenant X" jobs.
Long jobs also struggle with the worker process lifecycle. Code changes are not picked up mid-job. Deploys can interrupt long jobs unless you handle graceful shutdown.
Graceful Shutdown
When a deploy restarts workers, in-flight jobs need to finish or be requeued. php artisan queue:restart signals workers to exit after finishing the current job. The new worker generation picks up the next job.
The worker respects SIGTERM. Your supervisor (Supervisor, systemd, Horizon) should send SIGTERM and wait at least worker.timeout seconds before sending SIGKILL.
Misconfigured shutdown causes jobs to be killed mid-execution. If the job is not idempotent, the side effects are partial. Double-check this.
Idempotency
A job can run more than once. A worker can be killed mid-job and the job requeued. A retry on transient failure can repeat partial effects.
final class ChargePayment implements ShouldQueue
{
public function handle(): void
{
if ($this->order->fresh()->status === 'paid') {
return; // idempotency check
}
$charge = $this->paymentApi->charge([
'amount' => $this->order->total,
'idempotency_key' => "order:{$this->order->id}",
]);
$this->order->markPaid($charge->id);
}
}
Idempotency is not optional. Every queued job should be safe to run more than once.
Observability
The minimum: Horizon's dashboard or equivalent showing queue length, throughput, failure rate, average runtime.
Beyond that:
- Failed job alerts in Slack or PagerDuty
- Per-job-class metrics in Prometheus or your APM
- Long-running job alerts (jobs older than N minutes still processing)
- Queue length alerts (when waiting jobs cross a threshold)
Queues are silent until they break. Build the observability before you need it.
Common Mistakes
- One queue for everything. Slow jobs starve fast jobs. Split.
- Default timeouts. Laravel's default is 60 seconds. Real workloads need explicit per-job timeouts.
- No backoff. A failing job retries immediately, fails immediately, exhausts retries in seconds.
- No idempotency. Jobs that re-run partially are bugs waiting to be filed.
- Sync queue in production. The
syncdriver runs jobs inline. Easy to forget when copying staging config.
Tuning the queue is one of the highest-leverage operational investments in a Laravel application. Half a day of getting these settings right prevents weeks of incident response later.
Looking at a queue that has started misbehaving as traffic has grown? We help teams tune workers and split queues without a full rewrite. scopeforged.com