Using Free Tideways XHProf + XHGUI PHP 7.x for WordPress Code Profiling

WordPress performance has become incredibly important these days. Google has devoted an engineering team for WordPress performance development! Knowing which code is slowing your site down can help boost visitor retention and improve conversion rates for WooCommerce. XHProf is an amazing tool that can help you identify bottlenecks in your WordPress code. This took me several days of frustration to get Tideways XHProf and XHGUI working with PHP 7.x, shout out to Derrick Hammer (github) for assisting with getting this working smoothly!

WordPress and WooCommerce are quite the maze of actions, filters, PHP functions and objects. XHProf with XHGui helps you navigate these to help identify bottlenecks and suboptimal coding logic.

My hope is that this tutorial will get you up and running with profiling your WordPress PHP code quickly and without too much headache on Debian and Ubuntu systems!

If you’d like to get personalized, tailored support for getting Tideways XHProf set up please get in touch with me on Codeable or via my contact form.

If you are looking for a VPS to install XHProf on with your WordPress site, try Vultr or Digital Ocean.

These services all make it relatively easy to install WordPress on a VPS so you have an installation in which to install XHProf and XHGui.

• Runcloud
• GridPane
• EasyEngine
• Webinoly
• ServerPilot

You will need SSH access to complete this tutorial with root or sudo privileges!

Install XHProf PHP Extension

Install git and graphviz (for the XHGui graphs or they will not display)

sudo apt-get update
sudo apt-get install git graphviz -y

I have divided the rest of this section into the different PHP versions.

PHP 7.0

Install the PHP development package for 7.0

sudo apt-get install php7.0-dev -y

Build XHProf

cd /tmp
git clone https://github.com/tideways/php-xhprof-extension
cd php-xhprof-extension
phpize
./configure
make
sudo make install

Enable the PHP XHProf extension for PHP 7.0.

For nginx with php7.0-fpm, create the tideways-xhprof.ini file

sudo nano /etc/php/7.0/mods-available/tideways-xhprof.ini

Add this extension line

[tideways-xhprof-7.0]
extension=tideways_xhprof.so

Ctrl+X, Y and Enter to Save and exit.

Symbolically link the tideways-xhprof.ini – this way it can be easily disabled

sudo ln -s /etc/php/7.0/mods-available/tideways-xhprof.ini /etc/php/7.0/fpm/conf.d/tideways-xhprof.ini
sudo ln -s /etc/php/7.0/mods-available/tideways-xhprof.ini /etc/php/7.0/cli/conf.d/tideways-xhprof.ini

Then restart php7.0-fpm

sudo service php7.0-fpm restart

For Apache2 create the symlink to the apache2/conf.d folder

sudo ln -s /etc/php/7.0/mods-available/tideways-xhprof.ini /etc/php/7.0/apache2/conf.d/30-tidways-xhprof.ini

Then reload Apache

sudo service apache2 reload

Now you can jump down to Installing XHGui

PHP 7.1

Install the PHP development package for 7.1

sudo apt-get install php7.1-dev -y

Build XHProf

cd /tmp
git clone https://github.com/tideways/php-xhprof-extension
cd php-xhprof-extension
phpize
./configure
make
sudo make install

Enable the PHP XHProf extension for PHP 7.1

For nginx with php7.1-fpm, create the tideways-xhprof.ini file

sudo nano /etc/php/7.1/mods-available/tideways-xhprof.ini

Add this extension line

[tideways-xhprof-7.1]
extension=tideways_xhprof.so

Ctrl+X, Y and Enter to Save and exit.

Symbolically link the tideways-xhprof.ini – this way it can be easily disabled

sudo ln -s /etc/php/7.1/mods-available/tideways-xhprof.ini /etc/php/7.1/fpm/conf.d/tideways-xhprof.ini
sudo ln -s /etc/php/7.1/mods-available/tideways-xhprof.ini /etc/php/7.1/cli/conf.d/tideways-xhprof.ini

Then restart php7.1-fpm

sudo service php7.1-fpm restart

For Apache2 create the symlink to the apache2/conf.d folder

sudo ln -s /etc/php/7.1/mods-available/tideways-xhprof.ini /etc/php/7.1/apache2/conf.d/30-tidways-xhprof.ini

Then reload Apache

sudo service apache2 reload

Now you can jump down to Installing XHGui

PHP 7.2

Install the PHP development package for 7.2

sudo apt-get install php7.2-dev -y

Build XHPRof

cd /tmp
git clone https://github.com/tideways/php-xhprof-extension
cd php-xhprof-extension
phpize
./configure
make
sudo make install

For nginx with php7.2-fpm, create the tideways-xhprof.ini file

sudo nano /etc/php/7.2/mods-available/tideways-xhprof.ini

Add this extension line

[tideways-xhprof-7.2]
extension=tideways_xhprof.so

Ctrl+X, Y and Enter to Save and exit.

Symbolically link the tideways-xhprof.ini – this way it can be easily disabled

sudo ln -s /etc/php/7.2/mods-available/tideways-xhprof.ini /etc/php/7.2/fpm/conf.d/tideways-xhprof.ini
sudo ln -s /etc/php/7.2/mods-available/tideways-xhprof.ini /etc/php/7.2/cli/conf.d/tideways-xhprof.ini

Then restart php7.2-fpm

sudo service php7.2-fpm restart

For Apache2 create the symlink to the apache2/conf.d folder

sudo ln -s /etc/php/7.2/mods-available/tideways-xhprof.ini /etc/php/7.2/apache2/conf.d/30-tidways-xhprof.ini

Then reload Apache

sudo service apache2 reload

XHGUI for XHProf with PHP 7.x

You need to make sure some MySQL database server is installed and the PHP extension for interacting with it. Adjust the php-7.0-mysqli below for your version of PHP.

sudo apt-get install mariadb-server mariadb-client php7.0-mysqli -y

Create the MySQL user, password and database for XHGui’s database

mysql -u root

These credentials will be used for the MySQL database XHGui needs in the xhprof/xhprof_lib/config.php file.

CREATE USER xhprofuser@localhost IDENTIFIED BY 'passw0rd';
CREATE DATABASE xhprof;
GRANT ALL PRIVILEGES ON xhprof.* TO xhprofuser@localhost IDENTIFIED BY 'passw0rd';
FLUSH PRIVILEGES;
quit;

Make sure these PHP extensions are installed already

sudo apt-get install php7.0-mysqli php7.0-json -y

Enter your WordPress folder and use git to clone XHProf (replace the path /var/ww/sitename with your actual site path)

cd /var/www/sitename
git clone https://github.com/preinheimer/xhprof.git

Enter the xhprof_lib folder to modify the configuration files

cd xhprof/xhprof_lib
nano config.php

To make this easier, you can copy the entire XHGui config.php and paste it :).

xhprofuser is the MySQL user created earlier
passw0rd is the MySQL password
xhprof is the MySQL database

Change http://guides.wp-bullet.com to your site’s URL (the siteurl row in the option_name column in the wp_options table which should match your nginx or Apache server_name variable).

<?php
$_xhprof = array();

// Change these:
$_xhprof['dbtype'] = 'mysql'; // Only relevant for PDO
$_xhprof['dbhost'] = 'localhost';
$_xhprof['dbuser'] = 'xhprofuser';
$_xhprof['dbpass'] = 'passw0rd';
$_xhprof['dbname'] = 'xhprof';
$_xhprof['dbadapter'] = 'Mysqli';
$_xhprof['servername'] = 'wpb';
$_xhprof['server name'] = 'wpb';
$_xhprof['namespace'] = 'wpb';
$_xhprof['url'] = 'http://guides.wp-bullet.com/xhprof/xhprof_html';
$_xhprof['getparam'] = "_profile";

/*
 * MySQL/MySQLi/PDO ONLY
 * Switch to JSON for better performance and support for larger profiler data sets.
 * WARNING: Will break with existing profile data, you will need to TRUNCATE the profile data table.
 */
$_xhprof['serializer'] = 'php';

//Uncomment one of these, platform dependent. You may need to tune for your specific environment, but they're worth a try

//These are good for Windows
/*
$_xhprof['dot_binary']  = 'C:\\Programme\\Graphviz\\bin\\dot.exe';
$_xhprof['dot_tempdir'] = 'C:\\WINDOWS\\Temp';
$_xhprof['dot_errfile'] = 'C:\\WINDOWS\\Temp\\xh_dot.err';
*/

//These are good for linux and its derivatives.

$_xhprof['dot_binary']  = '/usr/bin/dot';
$_xhprof['dot_tempdir'] = '/tmp';
$_xhprof['dot_errfile'] = '/tmp/xh_dot.err';

$ignoreURLs = array();
// Do not track URIs containing xhprof
$ignoreURLs[] = "/xhprof/";

$ignoreDomains = array();

$exceptionURLs = array();

$ignoreDomains = array();

$exceptionURLs = array();

$exceptionPostURLs = array();
$exceptionPostURLs[] = "login";

$_xhprof['display'] = false;
$_xhprof['doprofile'] = false;

//Control IPs allow you to specify which IPs will be permitted to control when profiling is on or off within your application, and view the results via the UI.

$controlIPs = false; //Disables access controls completely.

//$controlIPs = array();
//$controlIPs[] = "127.0.0.1";   // localhost, you'll want to add your own ip here
//$controlIPs[] = "::1";         // localhost IP v6

//$otherURLS = array();

// ignore builtin functions and call_user_func* during profiling
//$ignoredFunctions = array('call_user_func', 'call_user_func_array', 'socket_select');

//Default weight - can be overidden by an Apache environment variable 'xhprof_weight' for domain-specific values
$weight = 100;

if($domain_weight = getenv('xhprof_weight')) {
	$weight = $domain_weight;
}

unset($domain_weight);

  /**
  * The goal of this function is to accept the URL for a resource, and return a "simplified" version
  * thereof. Similar URLs should become identical. Consider:
  * http://example.org/stories.php?id=2323
  * http://example.org/stories.php?id=2324
  * Under most setups these two URLs, while unique, will have an identical execution path, thus it's
  * worthwhile to consider them as identical. The script will store both the original URL and the
  * Simplified URL for display and comparison purposes. A good simplified URL would be:
  * http://example.org/stories.php?id=
  * 
  * @param string $url The URL to be simplified
  * @return string The simplified URL 
  */
  function _urlSimilartor($url)
  {
      //This is an example 
      $url = preg_replace("!\d{4}!", "", $url);
      
      // For domain-specific configuration, you can use Apache setEnv xhprof_urlSimilartor_include [some_php_file]
      if($similartorinclude = getenv('xhprof_urlSimilartor_include')) {
      	require_once($similartorinclude);
      }
      
      $url = preg_replace("![?&]_profile=\d!", "", $url);
      return $url;
  }
  
  function _aggregateCalls($calls, $rules = null)
  {
    $rules = array(
        'Loading' => 'load::',
        'mysql' => 'mysql_'
        );

    // For domain-specific configuration, you can use Apache setEnv xhprof_aggregateCalls_include [some_php_file]
  	if(isset($run_details['aggregateCalls_include']) && strlen($run_details['aggregateCalls_include']) > 1)
		{
    	require_once($run_details['aggregateCalls_include']);
		}        
        
    $addIns = array();
    foreach($calls as $index => $call)
    {
        foreach($rules as $rule => $search)
        {
            if (strpos($call['fn'], $search) !== false)
            {
                if (isset($addIns[$search]))
                {
                    unset($call['fn']);
                    foreach($call as $k => $v)
                    {
                        $addIns[$search][$k] += $v;
                    }
                }else
                {
                    $call['fn'] = $rule;
                    $addIns[$search] = $call;
                }
                unset($calls[$index]);  //Remove it from the listing
                break;  //We don't need to run any more rules on this
            }else
            {
                //echo "nomatch for $search in {$call['fn']}<br />\n";
            }
        }
    }
    return array_merge($addIns, $calls);
  }

Now we need to modify the XHProf header file or it will not work, change sitepath to your WordPress installation so you can open the header.php file.

nano /var/www/sitepath/xhprof/external/header.php

Change these instances from tideways to tideways_xhprof

function getExtensionName()
{
    if (extension_loaded('tideways'))
    {
        return 'tideways';
    }elseif(extension_loaded('xhprof')) {
        return 'xhprof';
    }
    return false;
}

so it should look like this with all instances changed to tideways_xhprof

function getExtensionName()
{
    if (extension_loaded('tideways_xhprof'))
    {
        return 'tideways_xhprof';
    }elseif(extension_loaded('xhprof')) {
        return 'xhprof';
    }
    return false;
}

Ctrl+X, Y and Enter to Save and Exit.

Add the MySQL Schema for the XHGUI details table or there will be no XHProf runs stored!

Start by creating an empty file

nano /tmp/xhgui-details.sql

Paste this MySQL schema for XHGui

CREATE TABLE `details` 
             ( 
                          `id`    CHAR(17) NOT NULL, 
                          `url`   VARCHAR(255) DEFAULT NULL, 
                          `c_url` VARCHAR(255) DEFAULT NULL, 
                          `timestamp` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP on 
             UPDATE CURRENT_TIMESTAMP, 
                    `server name` VARCHAR(64) DEFAULT NULL, 
                    `perfdata` MEDIUMBLOB, 
                    `type` TINYINT(4) DEFAULT NULL, 
                    `cookie` BLOB, 
                    `post` BLOB, 
                    `get` BLOB, 
                    `pmu`                    INT(11) UNSIGNED DEFAULT NULL, 
                    `wt`                     INT(11) UNSIGNED DEFAULT NULL, 
                    `cpu`                    INT(11) UNSIGNED DEFAULT NULL, 
                    `server_id`              CHAR(3) NOT NULL DEFAULT 't11', 
                    `aggregatecalls_include` VARCHAR(255) DEFAULT NULL, 
                    PRIMARY KEY (`id`), 
                    KEY `url` (`url`), 
                    KEY `c_url` (`c_url`), 
                    KEY `cpu` (`cpu`), 
                    KEY `wt` (`wt`), 
                    KEY `pmu` (`pmu`), 
                    KEY `timestamp` (`timestamp`) 
             ) 
             engine=myisam DEFAULT charset=utf8;

Ctrl+X, Y and Enter to Save and Exit.

You can import the XHGui MySQL scheme with this command, please change xhprof if you chose a different database name

mysql -u root xhprof < /tmp/xhgui-details.sql

Almost there!

Adding XHProf header.php to WordPress

XHProf’s PHP code needs to be added to WordPress so it can start monitoring, enter your WordPress directory

cd /var/www/guides.wp-bullet.com

Create a drop-in PHP .user.ini file

nano .user.ini

Paste this text, change your site’s path (here guides.wp-bullet.com)

auto_prepend_file="/var/www/guides.wp-bullet.com/xhprof/external/header.php";

Ctrl+X, Y and Enter to Save and Exit

You will see a debug footer at the bottom of pages eventually after you enable profiling by appending the ?_profile=1 query string later in this guide.

nginx vhost

If you do not want to use .user.ini to prepend the header, we can prepend the XHProf code directly using nginx with the code below

server {
    listen   80;
    server_name default;

    # root directive should be global
    root   /var/www/xhgui/;
    index  index.php;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ \.php$ {
        try_files $uri =404;
        include fastcgi_params;
        fastcgi_pass unix:/var/run/php/php7.0-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        # add xhprof header.php
        fastcgi_param PHP_VALUE "auto_prepend_file=/var/www/guides.wp-bullet.com/xhprof/external/header.php";
        # old version before header.php included footer.php
        # fastcgi_param PHP_VALUE "auto_prepend_file=/var/www/xhprof/external/header.php \n auto_append_file=/var/www/xhprof/external/footer.php";
    }
}

Thanks to Justin Carmony for the above method.

Using XHGUI and XHProf

This section has two components.

First you will learn how to enable XHProf’s monitoring.

Afterwards you will see how to use it in the browser.

Making XHGUI and XHProf Active

XHGui requires a query string to enable it, simply tack on ?_profile=1 to your URL like e.g. https://guides.wp-bullet.com/?_profile=1 which sets the cookie

Then to disable XHProf monitoring you append the /?_profile=0 query string to your site’s URL e.g. https://guides.wp-bullet.com/?_profile=0

Using XHGUI and XHProf in the Browser

Client new Product Bundles plugin was the problem but it is essential to the business. New Relic was running but because the process never executed there was no useful data, in comes XHProf and XHGui!

Reducing these values during debugging can save you significant amounts of waiting time when troubleshooting!

Protip: Turn max_execution_time = 30 and max_input_time=30 to make sure any infinite loops don’t get to run for too long causing gateway timeouts. If you are experiencing 504 Gateway timeout errors on your slow WordPress or WooCommerce site, then I highly recommend changing these when running XHProf.

;;;;;;;;;;;;;;;;;;;
; Resource Limits ;
;;;;;;;;;;;;;;;;;;;

; Maximum execution time of each script, in seconds
; http://php.net/max-execution-time
; Note: This directive is hardcoded to 0 for the CLI SAPI
max_execution_time = 30

; Maximum amount of time each script may spend parsing request data. It's a good
; idea to limit this time on productions servers in order to eliminate unexpectedly
; long running scripts.
; Note: This directive is hardcoded to -1 for the CLI SAPI
; Default Value: -1 (Unlimited)
; Development Value: 60 (60 seconds)
; Production Value: 60 (60 seconds)
; http://php.net/max-input-time
max_input_time = 30

Make sure to restart php-fpm or Apache after making the php.ini modifications.

Demonstration

I had a client with a products table that wasn’t loading, disabling all plugins didn’t help which meant the issue likely was in the child theme’s code. There were over 5000 lines in their functions.php so I wanted to avoid reading it all if possible.

I found this WC_Product_Bundle call taking way too long, over 20 seconds! I tracke dthis down to the Product Bundles plugin from WooCommerce.

Before 20.513482 seconds!

After correcting the function in the child theme, the same function now only takes 187 milliseconds

Get in touch with me via the contact form if you’d like help getting xhprof configured to diagnose your slow WordPress or WooCoommerce site.

Troubleshooting Tideways xhprof  and xhgui

If you see this error ‘Error producing callgraph, check /tmp/xh_dot.err ‘then install graphviz

sudo apt-get install git graphviz -y

If you see this error: ‘You do not have permission to view this page.’ then it’s the $controlIPs you need to change, after you change this you have to do add the ?_profile=1 query string again. It could also be basic permission errors on your web server so make sure to correct those as well.

If you do not see new runs it can also be because ?_profile=1 hasn’t been used yet or there is some page caching enabled, disable page caching and/or log in to bypass the cache and force PHP to process rather than hitting the page cache.

Make sure to add the table schema or you won’t see any runs at all, you also will not see any runs if you did not put ?_profile=1 this problem can also be traced in the logs

PHP message: PHP Warning:  mysqli_fetch_assoc() expects parameter 1 to be mysqli_result, boolean given in /var/www/mcat.wpbullet.me/xhprof/xhprof_lib/utils                    /Db/Mysqli.php on line 57
PHP message: PHP Warning:  mysqli_fetch_assoc() expects parameter 1 to be mysqli_result, boolean given in /var/www/mcat.wpbullet.me/xhprof/xhprof_lib/utils                    /Db/Mysqli.php on line 57" while reading response header from upstream, client: 104.221.54.228, server: mcat.wpbullet.me, request: "GET /xhprof/xhprof_html                    /? HTTP/1.1", upstream: "fastcgi://unix:/run/php/php7.2-fpm.sock:", host: "mcat.wpbullet.me", referrer: "https://mcat.wpbullet.me/xhprof/xhprof_html/?last=                    25"
2018/03/17 17:40:14 [error] 17773#17773: *137 FastCGI sent in stderr: "PHP message: PHP Warning:  mysqli_fetch_assoc() expects parameter 1 to be mysqli_res                    ult, boolean given in /var/www/mcat.wpbullet.me/xhprof/xhprof_lib/utils/Db/Mysqli.php on line 57
PHP message: PHP Warning:  mysqli_fetch_assoc() expects parameter 1 to be mysqli_result, boolean given in /var/www/mcat.wpbullet.me/xhprof/xhprof_lib/utils                    /Db/Mysqli.php on line 57
PHP message: PHP Warning:  mysqli_fetch_assoc() expects parameter 1 to be mysqli_result, boolean given in /var/www/mcat.wpbullet.me/xhprof/xhprof_lib/utils                    /Db/Mysqli.php on line 57" while reading response header from upstream, client: 104.221.54.228, server: mcat.wpbullet.me, request: "GET /xhprof/xhprof_html                    /? HTTP/1.1", upstream: "fastcgi://unix:/run/php/php7.0-fpm.sock:", host: "mcat.wpbullet.me", referrer: "https://mcat.wpbullet.me/xhprof/xhprof_html/?last=                    25"

If you see ‘given XHProf isn’t found’ then see here  for the CLI problem and if this php -i command doesn’t show xhprof then create the tideways-xhprof.ini file in the php-cli folder.

It could also be related to a miscopy of the SQL command for creating the schema for XHGui.

 php -i | grep xhprof
/etc/php/7.2/cli/conf.d/tideways-xhprof.ini
tideways_xhprof
The 'tideways_xhprof' extension provides a subset of the functionality of our commercial Tideways offering in a modern, optimized fork of the XHProf extension from Facebook as open-source. (c) Tideways GmbH 2014-2017, (c) Facebook 2009

If CPU and RAM info are not showing up in XHGUI then check this and make sure in header.php you modified both of these to be tideways_xhprof

function getExtensionName()
{
    if (extension_loaded('tideways_xhprof'))
    {
        return 'tideways_xhprof';
    }elseif(extension_loaded('xhprof')) {
        return 'xhprof';
    }
    return false;
}

Sources

XHProf Github
Tideways for PHP 7
Set up XHProf and XHGui on Ubuntu 14.04 – DigitalOcean
XHProf and XHGui Sitepoint
XHGui MongoDB version
Preinheimer XHProf Fork
Phacility XHProf Fork