Manage message queues
You can manage message queues from the command line using cron jobs or an external process manager to ensure that consumers are retrieving messages.
Process management
Cron jobs are the default mechanism to restart consumers. Processes started by cron
consume the specified number of messages and then terminate. Re-running cron
restarts the consumer.
The following example shows the Magento crontab
configuration for running consumers:
/app/code/Magento/MessageQueue/etc/crontab.xml
1
2
3
4
5
...
<job name="consumers_runner" instance="Magento\MessageQueue\Model\Cron\ConsumersRunner" method="run">
<schedule>* * * * *</schedule>
</job>
...
How often you check message queues depends on your business logic and available system resources. In general, you’ll probably want to check for newly created customers and send welcome emails more frequently than a more resource intensive process (e.g., updating your catalog). You should define cron
schedules according to your business needs.
It can be configured in Admin Panel Stores > Settings > Configuration > Advanced > System > Cron configuration options for group: consumers
See Configure and run cron for more information about using cron
with Magento.
You can also use a process manager such as Supervisor to monitor the status of processes. The manager can use the command line to restart the processes as needed.
Configuration
Behavior by default
- Cron job
consumers_runner
is enabled - Cron job
consumers_runner
runs all defined consumers - Each consumer processes 10000 messages and then terminates
If your Adobe Commerce store is hosted on the Cloud platform, use the CRON_CONSUMERS_RUNNER
to configure the consumers_runner
cron job.
Specific configuration
Edit the /app/etc/env.php
file to configure the cron job consumers_runner
.
1
2
3
4
5
6
7
8
9
10
...
'cron_consumers_runner' => [
'cron_run' => false,
'max_messages' => 20000,
'consumers' => [
'consumer1',
'consumer2',
]
],
...
cron_run
- A boolean value that enables or disables theconsumers_runner
cron job (default =true
).max_messages
- The maximum number of messages each consumer must process before terminating (default =10000
). Although we do not recommend it, you can use 0 to prevent the consumer from terminating. Seeconsumers_wait_for_messages
to configure how consumers process messages from the message queue.-
consumers
- An array of strings specifying which consumer(s) to run. An empty array runs all consumers.If your Adobe Commerce store is hosted on the Cloud platform, use the
CONSUMERS_WAIT_FOR_MAX_MESSAGES
to configure how consumers process messages from the message queue.
View a list of available message queue consumers
To view a list of all consumers:
1
bin/magento queue:consumers:list
Start message queue consumers
To start message queue consumers:
1
bin/magento queue:consumers:start [--max-messages=<value>] [--batch-size=<value>] [--single-thread] [--area-code=<value>] [--multi-process=<value>] <consumer_name>
After consuming all available messages, the command terminates. You can run the command again manually or with a cron job. You can also run multiple instances of the magento queue:consumers:start
command to process large message queues. For example, you can append &
to the command to run it in the background, return to a prompt, and continue running commands:
1
bin/magento queue:consumers:start <consumer_name> &
See queue:consumers:start in the Magento command-line reference for details about the command options, parameters, and values.
The --multi-process
option is present in the queue:consumers:start
command, but to run consumers with parallel processes, configure the multiple_processes
option in /app/etc/env.php
. Otherwise, if queue:consumers:start
is called with the --multi-process
option, it will only work on a single thread.