I love caching with Varnish. Its vcl (Varnish Control Language) allows me to be precise about what and how I want to cache specific URLs in WordPress and WooCommerce. One area which can be a pain for performance enthusiasts is showing up-to-date recent posts in a sidebar widget or show off your popular posts in the footer. The traditional way to work around this is by using AJAX so javascript makes a call for PHP to pull fresh data out of the MySQL database. This is very convenient but also potentially resource heavy and slow because of the entire process flow. Let’s cache those instead with Varnish.
The Varnish 4 version of this tutorial can be found here.
Speed Benefits of Caching AJAX Get Requests with Varnish
When I AJAXify the Recently and WordPress Popular Posts widget it takes between 200-300ms for the site to display the data. That is not the plugins’ fault, it is just the nature of AJAX (technical explanation here). Luckily we can cache this AJAX request and get it down to 50ms with Varnish 3 – a Varnish 4 equivalent will be published as well. I will assume you already have Varnish 3 installed and configured on your VPS or dedicated server.
Here are some pingdom results before and after caching the AJAX requests.
Uncached AJAX Response Time
Using the WordPress Popular Posts AJAXified widget without Varnish took 141 ms.
Varnish cached AJAX Response Time
Using the WordPress Popular Posts AJAXified with Varnish took 68 ms.
That is over a 50% increase in performance by caching the AJAX request with Varnish.
How to Cache AJAX GET Requests with Varnish 3
First let’s understand how AJAX requests work.
How AJAX GET Requests Work in WordPress
The diagram Regular AJAX GET Request with WordPress shows how many steps are involved in a typical AJAX request in WordPress.
The diagram Varnish cached AJAX GET Request with WordPress shows how much faster it is to serve AJAX GET requests with Varnish
Cache AJAX GET Requests using Varnish 3
Open up your Varnish vcl file
sudo nano /etc/varnish/default.vclAdd the red sections which are carefully placed in the vcl to not conflict with the vcl order. First we need to catch the AJAX requests for non-logged in users before wp-admin requests are passed and then set the time to live in sub vcl_fetch for our captured and cached AJAX GET requests. The snippets are explained in the next section.
...
sub vcl_recv {
...
# PIPE ALL NON-STANDARD REQUESTS
# ##########################################################
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
return (pipe);
}
# ONLY CACHE GET AND HEAD REQUESTS
# ##########################################################
if (req.request != "GET" && req.request != "HEAD") {
return (pass);
}
##Possibility to cache admin-ajax GET requests
if ((req.url ~ "admin-ajax.php") && !req.http.cookie ~ "wordpress_logged_in" ) {
return (lookup);
}
# OPTIONAL: DO NOT CACHE LOGGED IN USERS (THIS OCCURS IN FETCH TOO, EITHER
# COMMENT OR UNCOMMENT BOTH
# ##########################################################
if ( req.http.cookie ~ "wordpress_logged_in" ) {
return( pass );
}
# IF THE REQUEST IS NOT FOR A PREVIEW, WP-ADMIN OR WP-LOGIN
# THEN UNSET THE COOKIES
# ##########################################################
if (!(req.url ~ "wp-(login|admin)")
&& !(req.url ~ "&preview=true" )
){
unset req.http.cookie;
}
# IF BASIC AUTH IS ON THEN DO NOT CACHE
# ##########################################################
if (req.http.Authorization || req.http.Cookie) {
return (pass);
}
# IF YOU GET HERE THEN THIS REQUEST SHOULD BE CACHED
# ##########################################################
return (lookup);
}
# HIT FUNCTION
# ##########################################################
sub vcl_hit {
# IF THIS IS A PURGE REQUEST THEN DO THE PURGE
# ##########################################################
if (req.request == "PURGE") {
purge;
error 200 "Purged.";
}
return (deliver);
}
# MISS FUNCTION
# ##########################################################
sub vcl_miss {
if (req.request == "PURGE") {
purge;
error 200 "Purged.";
}
return (fetch);
}
# FETCH FUNCTION
# ##########################################################
sub vcl_fetch {
# I SET THE VARY TO ACCEPT-ENCODING, THIS OVERRIDES W3TC
# TENDANCY TO SET VARY USER-AGENT. YOU MAY OR MAY NOT WANT
# TO DO THIS
# ##########################################################
set beresp.http.Vary = "Accept-Encoding";
# IF NOT WP-ADMIN THEN UNSET COOKIES AND SET THE AMOUNT OF
# TIME THIS PAGE WILL STAY CACHED (TTL)
# ##########################################################
#set the length of time to cache ajax GET requests
if ((req.url ~ "admin-ajax.php") && !req.http.cookie ~ "wordpress_logged_in" ) {
unset beresp.http.set-cookie;
set beresp.ttl = 1d;
set beresp.grace = 1d;
}
##set how long to cache for anything that is not admin or ajax
if (!(req.url ~ "wp-(login|admin)|admin-ajax.php") && !req.http.cookie ~ "wordpress_logged_in") {
unset beresp.http.set-cookie;
set beresp.ttl = 52w;
set beresp.grace = 1w;
}
if (beresp.ttl <= 0s ||
beresp.http.Set-Cookie ||
beresp.http.Vary == "*") {
set beresp.ttl = 120 s;
return (hit_for_pass);
}
return (deliver);
}
.....Save the configuration with Ctrl+X, Y and Enter, don’t restart Varnish yet.
Caching AJAX Requests with Varnish Explained
In this Varnish snippet from sub vcl_recv section we are catching AJAX requests for non-logged in users and telling Varnish to look them up in its cache. It is important this snippet comes before you pass wp-admin in Varnish!
##Possibility to cache admin-ajax GET requests
if ((req.url ~ "admin-ajax.php") && !req.http.cookie ~ "wordpress_logged_in" ) {
return (lookup);
}If you want to be even more specific by finding the exact AJAX URL in Chrome’s Developer tools or Firefox Firebug. For example for the WordPress Popular Posts widget the URL is https://guides.wp-bullet.com/wp-admin/admin-ajax.php?action=wpp_get_popular&id=2. You can then target it like this in sub vcl_recv for my example Popular Posts URL by doing a regular expression match, here the ~ means ‘contains’.
##Possibility to cache admin-ajax GET requests
if ((req.url ~ "wpp_get_popular") && !req.http.cookie ~ "wordpress_logged_in" ) {
return (lookup);
}In this Varnish snippet from sub vcl_fetch we are specifying how long Varnish should store the cache for in days, you can use h for hours or m for minutes instead if you prefer. This is setting the TTL (time to live) for the cached URL. We also set the beresp.grace period for the AJAX GET requests so if Varnish can’t contact the backend to get fresh data after the beresp.ttl expires, it will keep delivering the old data for another day or until the backend becomes available.
if ((req.url ~ "admin-ajax.php") && !req.http.cookie ~ "wordpress_logged_in" ) {
unset beresp.http.set-cookie;
set beresp.ttl = 1d;
set beresp.grace = 1d;
}Similarly to the other snippet you can target the specific URL instead, here wpp_get_popular
if ((req.url ~ "wpp_get_popular") && !req.http.cookie ~ "wordpress_logged_in" ) {
unset beresp.http.set-cookie;
set beresp.ttl = 1d;
set beresp.grace = 1d;
}Test your Varnish configuration is working
varnishd -C -f /etc/varnish/default.vclReload the Varnish configuration
sudo sevice varnish reloadVerify Varnish is Caching the AJAX GET Requests
If you have the X-Cache header set you can use curl to grab the AJAXified URL. You can get the full URL of the AJAX request by using something like Firebug in Firefox or the Developer tools in Chrome.
sudo apt-get install curl -yNow you can curl the URL of the AJAX request, the -I switch outputs the response headers. You should do this at least twice to ensure there is a HIT.
curl -I "https://guides.wp-bullet.com/wp-admin/admin-ajax.php?action=wpp_get_popular&id=2"The output should look something like this, notice the X-Cache header is reporting a HIT.
HTTP/1.1 200 OK
Date: Mon, 23 May 2016 10:49:42 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Set-Cookie: __cfduid=d5d000d0a8fe7915564b4de8bb5874cff1464000582; expires=Tue, 23-May-17 10:49:42 GMT; path=/; domain=.wp-bullet.com; HttpOnly
X-Robots-Tag: noindex
X-Content-Type-Options: nosniff
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
Pragma: no-cache
X-Frame-Options: SAMEORIGIN
Vary: Accept-Encoding
X-Varnish: 1694929555 1694929352
Age: 214
Via: 1.1 varnish
X-Cache: HIT
Server: cloudflare-nginx
CF-RAY: 2a77f158e1ff3470-LHRYou can use Varnishlog to see what is going on as well and how the GET requests are being cached
varnishlogYou can see the requests are now cached and showing a HIT status
29 SessionOpen c 188.114.103.205 14908 :80
29 ReqStart c 188.114.103.205 14908 1694930274
29 RxRequest c GET
29 RxURL c /wp-admin/admin-ajax.php?action=wpp_get_popular&id=2
29 RxProtocol c HTTP/1.1
29 RxHeader c Host: guides.wp-bullet.com
29 RxHeader c Connection: Keep-Alive
29 RxHeader c Accept-Encoding: gzip
29 RxHeader c CF-IPCountry: IT
29 RxHeader c X-Forwarded-For: 87.8.64.231
29 RxHeader c CF-RAY: 2a7801fe52104322-MXP
29 RxHeader c X-Forwarded-Proto: https
29 RxHeader c CF-Visitor: {"scheme":"http"}
29 RxHeader c accept: */*
29 RxHeader c origin: https://guides.wp-bullet.com
29 RxHeader c user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/601.5.17 (KHTML, like Gecko) Version/9.1 Safari/601.5.17
29 RxHeader c accept-language: it-it
29 RxHeader c referer: https://guides.wp-bullet.com/
29 RxHeader c CF-Connecting-IP: 87.8.64.231
29 VCL_call c recv
29 VCL_acl c NO_MATCH purge
29 VCL_return c lookup
29 VCL_call c hash
29 Hash c /wp-admin/admin-ajax.php?action=wpp_get_popular&id=2
29 Hash c guides.wp-bullet.com
29 VCL_return c hash
29 Hit c 1694716489
29 VCL_call c hit deliver
29 VCL_call c deliver deliver
29 TxProtocol c HTTP/1.1
29 TxStatus c 200
29 TxResponse c OK
29 TxHeader c Content-Type: text/html; charset=UTF-8
29 TxHeader c Access-Control-Allow-Credentials: true
29 TxHeader c X-Robots-Tag: noindex
29 TxHeader c X-Content-Type-Options: nosniff
29 TxHeader c Expires: Wed, 11 Jan 1984 05:00:00 GMT
29 TxHeader c Cache-Control: no-cache, must-revalidate, max-age=0
29 TxHeader c Pragma: no-cache
29 TxHeader c X-Frame-Options: SAMEORIGIN
29 TxHeader c Content-Encoding: gzip
29 TxHeader c Vary: Accept-Encoding
29 TxHeader c Content-Length: 663
29 TxHeader c Accept-Ranges: bytes
29 TxHeader c Date: Mon, 23 May 2016 11:01:04 GMT
29 TxHeader c X-Varnish: 1694930274 1694716489
29 TxHeader c Age: 52866
29 TxHeader c Via: 1.1 varnish
29 TxHeader c Connection: keep-alive
29 TxHeader c X-Cache: HIT
29 Length c 663
29 ReqEnd c 1694930274 1464001264.969143867 1464001264.969459534 0.000077724 0.000202417 0.000113263Now you have successfully cached WordPress admin-ajax GET requests with Varnish 3 and your AJAXified content will load faster while PHP and MySQL get a much deserved rest.