Using Laravel Horizon outside of Production and Local

James Bannister • February 11, 2019 • 2 minute read

laravel

Laravel is pretty awesome as far as frameworks go. But what really makes it amazing is the suite of open-source software sitting behind it; one of which is Horizon.

Horizon is a dashboard and queue-worker configuration tool built to provide powerful insight and control over Redis queues.

Getting started is pretty easy, the documentation covers this in detail and there's a tonne of tutorials out there if you need further help getting started.

It doesn't take long and within 10 or 15 minutes, you have this powerful dashboard up and running and already processing some test jobs in your local environment.

You're happy with the installation and setup, and so you push the work you've done up to version control and then deploy it to your staging, pre-production, beta, whatever-you-call-it-before-production environment. You make a few actions to fire off a few queued jobs and then you check your Horizon dashboard and see this:

Horizon with no queue workers

Horizon is running, there are jobs queued, but there are not queue workers running.

You check your local machine again, as you swear it was working and you see:

Horizon with queue workers

Huh?! It's working in local, but not on your staging site...

Turns out this is covered in a closed issue reported in the Horizon repo.

The issue lies at the bottom of your config/horizon.php file, under the 'Queue Worker Configuration' comment.

Out-of-the-box, Horizon ships with the below queue worker configuration:

1// config/horizon.php
2 
3'environments' => [
4 'production' => [
5 'supervisor-1' => [
6 'connection' => 'redis',
7 'queue' => ['default'],
8 'balance' => 'simple',
9 'processes' => 10,
10 'tries' => 3,
11 ],
12 ],
13 'local' => [
14 'supervisor-1' => [
15 'connection' => 'redis',
16 'queue' => ['default'],
17 'balance' => 'simple',
18 'processes' => 3,
19 'tries' => 3,
20 ],
21 ],
22],

That is to say, Horizon ships with an environment setting for production and local, but nothing else. If the environment that you've deployed to is not listed here then Horizon is not able to boot up any queue workers or supervisors as it doesn't know what to load.

Knowing this, the solution becomes fairly simple; add an environment configuration for that which you have deployed to. In my case, it is staging:

1// config/horizon.php
2 
3'environments' => [
4 'production' => [
5 'supervisor-1' => [
6 'connection' => 'redis',
7 'queue' => [
8 env('QUEUE_DEFAULT', 'default'),
9 ],
10 'balance' => 'simple',
11 'processes' => 10,
12 'tries' => 3,
13 ],
14 ],
15 'staging' => [
16 'supervisor-1' => [
17 'connection' => 'redis',
18 'queue' => [
19 env('QUEUE_DEFAULT', 'default'),
20 ],
21 'balance' => 'simple',
22 'processes' => 5,
23 'tries' => 3,
24 ],
25 ],
26 'local' => [
27 'supervisor-1' => [
28 'connection' => 'redis',
29 'queue' => [
30 env('QUEUE_DEFAULT', 'default'),
31 ],
32 'balance' => 'simple',
33 'processes' => 3,
34 'tries' => 3,
35 ],
36 ],
37],

You can see that I've added a configuration for staging, but those of you with keen eyes may also notice another change; I've updated the queue array in the configuration.

Credit for this goes to sebastiansulinski on the same Horizon issue. This prevents multiple instances of Horizon (occurs if you are running more than one Laravel application per server with Horizon running) competing for jobs from other Laravel applications.

By following this approach, and updating our config/queue.php file appropriately, we ensure that we are pushing our queued jobs to a queue specific to that Laravel instance we are running and that our Horizon instance is only completing jobs on that queue:

1// config/queue.php
2 
3'connections' => [
4 
5 'redis' => [
6 'driver' => 'redis',
7 'connection' => 'default',
8 'queue' => env('QUEUE_DEFAULT'),
9 'retry_after' => 90,
10 ],
11 
12],
13 
14'queues' => [
15 'default' => env('QUEUE_DEFAULT'),
16]

With both of these approaches implemented, you'll have Horizon running across all of your application environments and projects, as well as the ability for them all to run on the same server, with multiple instances of Horizon running.


This has been my second post on my new site. If you have any feedback, thoughts, or different approaches to this then I'd love to hear from you; I'm @jryd_13 on Twitter.

Thanks!