NginX Proxy Prevent Dog Pile

From Tomp Online Wiki

Introduction

When using the NginX reverse proxy cache, if the web site is very busy, when the cached items expire, multiple requests go to the backend servers to refresh the cache.

This sudden spike of traffic to the backends can cause the servers to crash or respond slowly.

This is referred to as a "dog pile" and also affects other cache systems, such as memcached.

NginX has built in protection for this, but it must be enabled.

proxy_cache_use_stale

When configuring reverse proxy in NginX, add the following statement to your virtual host:

proxy_cache_use_stale   error timeout updating;

This instructs NginX to use expired items out of the cache, whilst a single thread goes to get the updated objects.

E.g

upstream backend  {
	server ip1:80   max_fails=0;
        server ip2:80	max_fails=0;
        server ip3:80   max_fails=0;
        server ip4:80	max_fails=0;
}

server {
        listen          80 default;
        server_name     default;

        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP	$remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_connect_timeout   5;
        proxy_send_timeout      10;
        proxy_read_timeout      15;
        proxy_cache_use_stale   error timeout updating;

        location / {
                proxy_pass      http://backend;
                proxy_cache     cache1;
        }
}