Although RQ features the
use_connection() command for convenience, it
is deprecated, since it pollutes the global namespace. Instead, prefer explicit
connection management using the
with Connection(...): context manager, or
pass in Redis connection references to queues directly.
The use of
use_connection is deprecated.
Please don't use `use_connection` in your scripts.
Instead, use explicit connection management.
In development mode, to connect to a default, local Redis server:
In production, to connect to a specific Redis server:
Be aware of the fact that
use_connection pollutes the global namespace. It
also implies that you can only ever use a single connection.
However, the single connection pattern facilitates only those cases where you
connect to a single Redis instance, and where you affect global context (by
replacing the existing connection with the
use_connection() call). You can
only use this pattern when you are in full control of your web stack.
In any other situation, or when you want to use multiple connections, you
Connection contexts or pass connections around explicitly.
Each RQ object instance (queues, workers, jobs) has a
argument that can be passed to the constructor. Using this, you don’t need to
use_connection(). Instead, you can create your queues like this:
Every job that is enqueued on a queue will know what connection it belongs to. The same goes for the workers.
This approach is very precise, but rather verbose, and therefore, tedious.
There is a better approach if you want to use multiple connections, though. Each RQ object instance, upon creation, will use the topmost Redis connection on the RQ connection stack, which is a mechanism to temporarily replace the default connection to be used.
An example will help to understand it:
You can think of this as if, within the
Connection context, every newly
created RQ object instance will have the
connection argument set implicitly.
Enqueueing a job with
q2 will enqueue it in the second (remote) Redis
backend, even when outside of the connection context.
If your code does not allow you to use a
with statement, for example, if you
want to use this to set up a unit test, you can use the
pop_connection() methods instead of using the context manager.
To use redis sentinel, you must specify a dictionary in the configuration file. Using this setting in conjunction with the systemd or docker containers with the automatic restart option allows workers and RQ to have a fault-tolerant connection to the redis.