How to Pass Clients/Visitors IP Through Varnish to Nginx

Varnish is a great open source software known as a HTTP reverse proxy is typically run in front of web servers such as Apache or Nginx. In this case we will discuss Varnish and Nginx . The aim of Varnish is to stores the Varnish cache and remember what web server response to the user at the time of the first content access. Then return the cached copy for subsequent requests from end users without asking Nginx web server again. Therefore, Nginx access logs will display the local IP proxy (usually 127.0.0.1 if installed on the same server) instead of user’s IP as per below Nginx access logs.

127.0.0.1 - - [16/Feb/2015:01:03:09 +0800] "GET /red-hat-details-next-linux-and-storage-platforms-for-cloud-big-data-era/?share=google-plus-1 HTTP/1.1" 302 5 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
127.0.0.1 - - [16/Feb/2015:01:03:15 +0800] "GET /how-to-install-and-configure-epel-repository-on-centos-5-8/ HTTP/1.1" 200 15212 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"
127.0.0.1 - - [16/Feb/2015:01:03:22 +0800] "POST /ngx_pagespeed_beacon?url=http%3A%2F%2Fwww.ehowstuff.com%2Fhow-to-install-telnet-client-on-centos-6-3%2F HTTP/1.1" 404 564 "https://hostingcult.com/howto/how-to-install-telnet-client-on-centos-6-3/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.111 Safari/537.36"
127.0.0.1 - - [16/Feb/2015:01:03:23 +0800] "GET /how-to-setup-squid-proxy-server-on-linux-centos-6-3/ HTTP/1.1" 200 16246 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"

This will be a problem for a software like awstats or log analysis software due to incomplete information of the visitors. This article will teach you how to relay your blog visitor IP address through Varnish, to Nginx logs. The steps has been tested on CentOS 6.6 and CentOS 7. But before we start please make sure that http_realip_module has been enabled. This module allows to change the client’s IP address to value from request header (e. g. X-Real-IP or X-Forwarded-For). This module isn’t built by default, enable it with the configure option

--with-http_realip_module

Steps 1

Include “X-Forwareded-For” in the sub vcl_recv default.vcl :

[root@centos66 ~]# vi /etc/varnish/default.vcl

sub vcl_recv {
        # IP forwarding
        if (req.restarts == 0) {
                if (req.http.x-forwarded-for) {
                set req.http.X-Forwarded-For =
                        req.http.X-Forwarded-For + ", " + client.ip;
                } else {
                set req.http.X-Forwarded-For = client.ip;
                }
        }
..
..

Steps 2

Add the following in nginx.conf :

[root@centos66 ~]# vi /etc/nginx/nginx.conf

http {
..
..
    set_real_ip_from   127.0.0.1;
    real_ip_header      X-Forwarded-For;

..
..
}

Steps 3
Restart Nginx web server and Varnish :

[root@centos66 ~]# service nginx restart
[root@centos66 ~]# service varnish restart

Steps 4

Check and monitor the nginx access log again. It should returned the actual visitor IP as below :

157.55.39.102 - - [16/Feb/2015:01:06:04 +0800] "GET /how-to-download-centos-6-2-iso/ HTTP/1.1" 200 14622 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
68.180.228.247 - - [16/Feb/2015:01:06:16 +0800] "GET /tag/centos-6-2/page/4/ HTTP/1.1" 200 14474 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)"
220.181.108.178 - - [16/Feb/2015:01:06:25 +0800] "GET /howto-guides/howto-centos/ HTTP/1.1" 200 13863 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
66.249.79.116 - - [16/Feb/2015:01:06:33 +0800] "GET /how-to-enable-root-login-on-ubuntu-14-04/ HTTP/1.1" 200 15547 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"