[Openvpn-devel] multihome broken in the presence of asymmetric routing

Message ID e70826e2-7ad7-e64b-76b5-14588ef98b04@del.bg
State New
Headers show
  • [Openvpn-devel] multihome broken in the presence of asymmetric routing
Related show

Commit Message

Teodor Milkov March 12, 2018, 10:37 a.m.

I have the following multihomed setup:

            BGP1                        BGP2
             ^                           ^
             |                           |
+-----------+-----------+    +----------+-----------+
| IP1.1   IP1.2   IP1.3 |    | IP2.1  IP2.2   IP3.1 |
|                       |    |                      |
|                       |    |                      |
|          RTR1         |    |         RTR2         |
|                       |    |                      |
|                       |    |                      |
|          vIP          |    |         vIP          |
+-----------+-----------+    +----------+-----------+
             |                           |
             |                           |
             |                           |
             |            VRRP           |

I.e. two routers with BGP to multiple ISPs. VRRP running at inner side 
keeping one virtual IP (vIP) up at the master router.

Someday I hope I'll be able to use the vIP only for openvpn server bind 
IP ("local" config option). Untill then, for legacy reasons, I have to 
use the "multihome" config option, so that clients could connect to each 
of the 7 IPs (IP1.1, IP1.2, IP1.3, IP2.1, IP2.2, IP2.3, vIP).

Unfortunately, Linux would not respond to /some/ clients over UDP. It 
seems like this is due to the way "multihome" forces the output 
interface using IP_PKTINFO when we have asymetric path to/from vpn clients.

To provide single-socket UDP multihoming, openvpn uses the IP_PKTINFO 
data from recvmsg() and passes it as-is to sendmsg(). Apparently 
ipi_ifindex behaves in an interesting way under Linux. man 7 ip states:

     If  IP_PKTINFO  is  passed  to  sendmsg(2)  and ipi_spec_dst  is  not
     zero, then it is used as the local source address for the routing table
     lookup and for setting up IP source route options.  When ipi_ifindex is
     not zero, the primary local address of the interface  specified  by
     the  index overwrites ipi_spec_dst for the routing table lookup.

In my tests it's like /ipi_ifindex/ will override any routing table 
decision. But then if there is no routing table entry for the 
destination via
that interface, the destination will be assumed to be on-link and will 
not be routed via a gateway. No error is returned to
userspace, but if the destination does not respond to an ARP request on 
that link, the packet will be silently dropped. That's what I see with 
tcpdump on the openvpn server: arp requests for the dst address of the 
openvpn client. For example if a.b.c.100 is an openvpn client and 
x.y.z.1 is the virtual IP on the openvpn server to which the client 
tries to connect:

11:52:40.695830 ARP, Request who-has a.b.c.100 tell x.y.z.1, length 28

I got it working by patching socket.c in the following manner:

I guess this should be confirmed by someone else too, and I haven't 
looked into IPv6 side at all, but the above patch works for me ™.

Best regards,
Teodor Milkov
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot


--- openvpn-2.4.0/src/openvpn/socket.c  2016-12-26 13:51:00.000000000 +0200
+++ openvpn-2.4.0/src/openvpn/socket.c  2018-03-09 15:37:10.015832657 +0200
@@ -3379,7 +3379,7 @@ 
                  struct in_pktinfo *pkti;
                  pkti = (struct in_pktinfo *) CMSG_DATA(cmsg);
-                pkti->ipi_ifindex = to->pi.in4.ipi_ifindex;
+                pkti->ipi_ifindex = 0;
                  pkti->ipi_spec_dst = to->pi.in4.ipi_spec_dst;
                  pkti->ipi_addr.s_addr = 0;