Configure Cavalcade for Scaling WordPress wp-cron to 100+ Jobs

Cron is a scheduled task system that runs on Linux. WordPress has its own version of cron that it calls wp-cron. This is where all of the scheduled tasks from WordPress core, plugins and themes store their future jobs to run (WooCommerce uses something called action_scheduler for many of its background processes). Sometimes you do need hundreds of cronjobs for one reason or another on your WordPress site. If that is the case you should consider using a cron manager built for scale called Cavalcade.

This tutorial will walk you through counting your cronjobs in WordPress with WP-CLI and setting up CavalcadeCavalcade to take over the cron responsibilities.

Checking WordPress Cron Jobs with WP-CLI

You can list all of your cron scheduled tasks in WordPress with this WP-CLI command.

If you are not comfortable in the terminal of your web server or host then you can use the WP Crontrol plugin

wp cron event list --allow-root

The output will show all of the scheduled tasks for your WordPress site

+------------------------------------------------+---------------------+----------------------+---------------+
| hook                                           | next_run_gmt        | next_run_relative    | recurrence    |
+------------------------------------------------+---------------------+----------------------+---------------+
| action_scheduler_run_queue                     | 2020-04-16 18:34:51 | 18 seconds           | 1 minute      |
| wc_admin_process_orders_milestone              | 2020-04-16 18:40:51 | 6 minutes 18 seconds | 1 hour        |
| wp_privacy_delete_old_export_files             | 2020-04-16 18:40:51 | 6 minutes 18 seconds | 1 hour        |
| wc_admin_unsnooze_admin_notes                  | 2020-04-16 18:41:19 | 6 minutes 46 seconds | 1 hour        |
| imagify_sync_files                             | 2020-04-17 01:00:00 | 6 hours 25 minutes   | 1 day         |
| wp_version_check                               | 2020-04-17 01:40:51 | 7 hours 6 minutes    | 12 hours      |
| wp_update_plugins                              | 2020-04-17 01:40:51 | 7 hours 6 minutes    | 12 hours      |
| wp_update_themes                               | 2020-04-17 01:40:51 | 7 hours 6 minutes    | 12 hours      |
| imagify_update_library_size_calculations_event | 2020-04-17 03:00:00 | 8 hours 25 minutes   | 1 week        |
| wpseo-premium-prominent-words-recalculate      | 2020-04-17 13:40:51 | 19 hours 6 minutes   | 1 day         |
| wpseo-reindex-links                            | 2020-04-17 13:48:39 | 19 hours 14 minutes  | 1 day         |
| wp_scheduled_delete                            | 2020-04-17 13:48:39 | 19 hours 14 minutes  | 1 day         |
| delete_expired_transients                      | 2020-04-17 13:48:39 | 19 hours 14 minutes  | 1 day         |
| ivole_send_reminder                            | 2020-04-21 13:41:20 | 4 days 19 hours      | Non-repeating |
| ivole_send_reminder                            | 2020-04-21 13:46:23 | 4 days 19 hours      | Non-repeating |
| ivole_send_reminder                            | 2020-04-21 13:46:24 | 4 days 19 hours      | Non-repeating |
...
+------------------------------------------------+---------------------+----------------------+---------------+

That is a very long list! I cut it off to spare you from unnecessarily scrolling through about 200 ivole_send_reminder tasks.

We can make it only display the unique cron tasks by only showing the hooks.

wp cron event list --field=hook --allow-root | sort -u

Count your jobs in wp-cron with this WP-CLI command

wp cron event list --allow-root | wc -l

Wow that is over 200!

210

The cron tasks are stored in the wp_options table in the option named cron.

The cron option is autoloaded and the larger that this gets it can actually slow down your site! See how to clean autoloaded data here.

What we can do to allow for a lot of cronjobs is to used a dedicated cron database table managed by Cavalcade created by the incredibly talented HumanMade who have a few very cool open source projects on Github.

Configure Cavalcade to Scale wp-cron

Here is a list of the commands to run to set up HumanMade’s Cavalcade on Linux.

Set up Cavalcade Plugin

We need to make an mu-plugin for the Cavalcade plugin to handle WordPress cron from now on.

You will need to have git installed on your system in order to clone the repository.

mkdir -p wp-content/mu-plugins
cd wp-content/mu-plugins
echo "<?php require_once __DIR__ . '/cavalcade/plugin.php';" > cavalcade.php
wp config set DISABLE_WP_CRON true
git clone https://github.com/humanmade/Cavalcade cavalcade
cd cavalcade

Done with the Cavalcade plugin!

Set up Cavalcade Runner

First we need to clone the Cavalcade Runner from git

git clone https://github.com/humanmade/Cavalcade-Runner runner
cd runner/bin

Now we can execute the Cavalcade Runner and make sure to pass the path of your WordPress or WooCommerce installation as the first argument.

Here the path is /www/wpbulletsdev_970/public.
If you want to change the log path then change the path and file /www/wpbulletdev_970/logs/cavalcade.log.

./cavalcade /www/wpbulletsdev_970/public & > /www/wpbulletdev_970/logs/cavalcade.log

See output – means we have daemonized the Cavalcade runner so it will keep running even if we close the SSH session

[1] 9076

Script to make sure cavalcade is running, we can use grep to scan for the Cavlcade runner script running

 ps -aux | grep -v grep | grep cavalcade

output

wpbullet+   9076  0.0  0.3 411916 25176 pts/0    S    12:23   0:00 php ./cavalcade /www/degrosdev_970/public

Create a new file to manage the Cavalcade runner in this path wp-content/mu-plugins/cavalcade/manage.sh

This script works on this host that specializes in scaling and high performance websites.

Make sure to change the path in red below.

You will also want to change the export PATH line to reflect the output of echo $PATH from your terminal on the web server.

#!/usr/bin/env bash
# Purpose - Keep Cavalcade running
# Author Mike from WP Bullet

export PATH="/www/wpbulletdev_970/bin:/www/wpbulletdev_970/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games"

# homepath - include trailing slash
HOMEPATH="/www/wpbulletdev_970/public/"

# enter the Cavalcade plugin directory
cd ${HOMEPATH}/wp-content/mu-plugins/cavalcade

# check if the Cavalcade runner is in the list of processes
TESTCAVALCADE=$( ps -aux | grep -v grep | grep "mu-plugins/cavalcade/runner/bin/cavalcade" )

# check if Cavalcade is running and restart if it isn't
if [[ -z "${TESTCAVALCADE}" ]]; then
    echo "Cavalcade not running"
    /usr/bin/php ${HOMEPATH}wp-content/mu-plugins/cavalcade/runner/bin/cavalcade ${HOMEPATH}  > /www/wpbulletdev_970/logs/cavalcade.log
    echo "Cavalcade running"
fi

You will need to make a cronjob to execute the Cavalcade manager script so put this in cron and make sure to adjust the path in red below

*/2 * * * * /bin/bash /www/wpbulletdev_970/public/wp-content/mu-plugins/cavalcade/manager.sh > /tmp/cavalcadestart 2>&1

Sources

Cavalcade Runner
Cavalcade Github
Check if process is running