This came from a posting by Jacob Reif Jacob (dot) Rief (at) Tiscover (dot) com 25 Apr 2003.

It is common to run multiple websites (Jacob has 100s) on the same IP, using name based http to differentiate the websites. Sometimes webdesigners use some kind of include-function to include content from one website into another, by means of server-side-includes. (see http://www.php.net/manual/en/function.require.php) using http-subrequests. The include requires a client process running on the webserver, to make a request to a different website on the same IP. If the website is running on an LVS, then the realservers need to be able to make a request to the VIP. For LVS-DR and LVS-Tun this is no problem: the realserver has the VIP (and the services presented on that IP), so requests by http clients running on the realserver to the VIP, will be answered locally.

For LVS-NAT, the services are all running on the RIP (remember, there is no IP with the VIP on realservers for LVS-NAT). Here's what happens when the client on the realserver requests a page at VIP:80

realserver_1 makes a request to VIP:80, which goes to the director. The director demasquerades (rewrites) dst_addr from VIP to RIP_2. realserver_2 then services the request and fires off a reply packet with src_addr=RIP_2, dst_addr=RIP_1. This goes to realserver_1 directly (rather than being masqueraded through the director), but realserver_1 refuses the packet because it expected a reply from VIP and not from RIP_2.

           +-------------+
           |     VIP     |
           |  director   |
           +-------------+
            ^           |
            |           |req
            |req        v
  +-------------+     +-------------+
  |   RIP_1     |<--- |   RIP_2     |
  |  Realserver | ans |  Realserver |
  |  = client   | wer |  = server   |
  +-------------+     +-------------+

Here are the current attempts at solutions to the problem, or you can go straight to Jacob's solution

Jacob's solution: The solution proposed here does not put that extra load onto the director. However each realserver always contacts itself (which isn't a problem). Put the following entry into each realserver. Now the realservers can access the httpd on RIP as if it were on VIP.

realserver#  iptables -t nat -A OUTPUT -p tcp -d $VIP --dport 80 -j DNAT --to ${RIP}:80

Carlos Lozano clozano (at) andago (dot) com 02 Jul 2004

We have a machine that must be both a client and director. The two problems to solve are

I have written a ip_vs_core.c.diff (http://www.austintek.com/LVS/LVS-HOWTO/HOWTO/files/ip_vs_core.c.diff) patch for 2.4.26 using IPVS-NAT. It works correctly in my testcase. The schema is:

External client ---> IPVS:443 --> Local:443 ---> IPVS:80 ---> RealServer

The problem happens when Local:443 goes to localIPVS:80, because the packet is discarded by the next lines in ip_vs_core.c:

if (skb->pkt_type != PACKET_HOST) || skb->dev == &loopback_dev) { 
        IP_VS_DBG(12, "packet type=%d proto=%d daddr=%d.%d.%d.%d ignored\n",
                       skb->pkt_type,
                       iph->protocol,
                       NIPQUAD(iph->daddr));
        return NF_ACCEPT;
}  

Ratz

Why do you need this? Seems like a replication of mod_proxy/mod_rewrite. Your patch obviously makes it work but I wonder if such a functionality is really needed.

We are using it like an ssl accelerator. The first ipvs (443) sends the request to localhost:443 or to a different director, and the second ipvs(80), distributes the traffic in the realservers.

Ext. client --> IPVS:443 --> Local:443 --> IPVS:80 --> RealServer1
                         |-> Director2:443         |-> RealServer2

In the first case, it is a scheme "external machine client+director", but in the second case it is a "client+director in the same machine". This part of the patch only solves the output packet, the return is handled by the second part of the patch. (what is really a bad hack)

For a mini-HOWTO on using this patch see https_on_localnode. Matt Venn has tested it, it works using the local IP of the director, but not 127.0.0.1.

Graeme Fowler is looking for a solution for realservers that can't use iptables

Graeme Fowler graeme (at) graemef (dot) net 11 Feb 2005

After a long day spent tracing packets through the LVS and netfilter trail whilst trying to do cleverness with policy routing using the iproute2 package, I can condense quite a lot of reading (and trial, mainly followed by error!) down as follows:

Conclusions: mixing policy routing and LVS sounds like a great idea, and probably is if you're using LVS-DR or LVS-TUN. Just with LVS-NAT, it's a no-go (for me at the moment, anyway).