| Message ID | e70826e2-7ad7-e64b-76b5-14588ef98b04@del.bg |
|---|---|
| State | Rejected |
| Headers | show |
| Series | [Openvpn-devel] multihome broken in the presence of asymmetric routing | expand |
Am 12.03.18 um 11:37 schrieb Teodor Milkov: > Hello, > > I have the following multihomed setup: > > > BGP1 BGP2 > ^ ^ > | | > +-----------+-----------+ +----------+-----------+ > | IP1.1 IP1.2 IP1.3 | | IP2.1 IP2.2 IP3.1 | > | | | | > | | | | > | RTR1 | | RTR2 | > | | | | > | | | | > | vIP | | vIP | > +-----------+-----------+ +----------+-----------+ > | | > | | > | | > | VRRP | > +---------------------------+ > > > I.e. two routers with BGP to multiple ISPs. VRRP running at inner side > keeping one virtual IP (vIP) up at the master router. > > Someday I hope I'll be able to use the vIP only for openvpn server bind > IP ("local" config option). Untill then, for legacy reasons, I have to > use the "multihome" config option, so that clients could connect to each > of the 7 IPs (IP1.1, IP1.2, IP1.3, IP2.1, IP2.2, IP2.3, vIP). > > Unfortunately, Linux would not respond to /some/ clients over UDP. It > seems like this is due to the way "multihome" forces the output > interface using IP_PKTINFO when we have asymetric path to/from vpn clients. > > To provide single-socket UDP multihoming, openvpn uses the IP_PKTINFO > data from recvmsg() and passes it as-is to sendmsg(). Apparently > ipi_ifindex behaves in an interesting way under Linux. man 7 ip states: > > If IP_PKTINFO is passed to sendmsg(2) and ipi_spec_dst is not > zero, then it is used as the local source address for the routing table > lookup and for setting up IP source route options. When ipi_ifindex is > not zero, the primary local address of the interface specified by > the index overwrites ipi_spec_dst for the routing table lookup. > > > In my tests it's like /ipi_ifindex/ will override any routing table > decision. But then if there is no routing table entry for the > destination via > that interface, the destination will be assumed to be on-link and will > not be routed via a gateway. No error is returned to > userspace, but if the destination does not respond to an ARP request on > that link, the packet will be silently dropped. That's what I see with > tcpdump on the openvpn server: arp requests for the dst address of the > openvpn client. For example if a.b.c.100 is an openvpn client and > x.y.z.1 is the virtual IP on the openvpn server to which the client > tries to connect: > > 11:52:40.695830 ARP, Request who-has a.b.c.100 tell x.y.z.1, length 28 > > I got it working by patching socket.c in the following manner: Sorry for replying so late. I am just going through all open patches. but with this i am not sure that it does not break the other multihomed use cases where this behaviour is actually intended and you also only tested it for your specific use case, so I am a bit cautious to just include it. Arne
--- openvpn-2.4.0/src/openvpn/socket.c 2016-12-26 13:51:00.000000000 +0200 +++ openvpn-2.4.0/src/openvpn/socket.c 2018-03-09 15:37:10.015832657 +0200 @@ -3379,7 +3379,7 @@ { struct in_pktinfo *pkti; pkti = (struct in_pktinfo *) CMSG_DATA(cmsg); - pkti->ipi_ifindex = to->pi.in4.ipi_ifindex; + pkti->ipi_ifindex = 0; pkti->ipi_spec_dst = to->pi.in4.ipi_spec_dst; pkti->ipi_addr.s_addr = 0; }
Hello, I have the following multihomed setup: BGP1 BGP2 ^ ^ | | +-----------+-----------+ +----------+-----------+ | IP1.1 IP1.2 IP1.3 | | IP2.1 IP2.2 IP3.1 | | | | | | | | | | RTR1 | | RTR2 | | | | | | | | | | vIP | | vIP | +-----------+-----------+ +----------+-----------+ | | | | | | | VRRP | +---------------------------+ I.e. two routers with BGP to multiple ISPs. VRRP running at inner side keeping one virtual IP (vIP) up at the master router. Someday I hope I'll be able to use the vIP only for openvpn server bind IP ("local" config option). Untill then, for legacy reasons, I have to use the "multihome" config option, so that clients could connect to each of the 7 IPs (IP1.1, IP1.2, IP1.3, IP2.1, IP2.2, IP2.3, vIP). Unfortunately, Linux would not respond to /some/ clients over UDP. It seems like this is due to the way "multihome" forces the output interface using IP_PKTINFO when we have asymetric path to/from vpn clients. To provide single-socket UDP multihoming, openvpn uses the IP_PKTINFO data from recvmsg() and passes it as-is to sendmsg(). Apparently ipi_ifindex behaves in an interesting way under Linux. man 7 ip states: If IP_PKTINFO is passed to sendmsg(2) and ipi_spec_dst is not zero, then it is used as the local source address for the routing table lookup and for setting up IP source route options. When ipi_ifindex is not zero, the primary local address of the interface specified by the index overwrites ipi_spec_dst for the routing table lookup. In my tests it's like /ipi_ifindex/ will override any routing table decision. But then if there is no routing table entry for the destination via that interface, the destination will be assumed to be on-link and will not be routed via a gateway. No error is returned to userspace, but if the destination does not respond to an ARP request on that link, the packet will be silently dropped. That's what I see with tcpdump on the openvpn server: arp requests for the dst address of the openvpn client. For example if a.b.c.100 is an openvpn client and x.y.z.1 is the virtual IP on the openvpn server to which the client tries to connect: 11:52:40.695830 ARP, Request who-has a.b.c.100 tell x.y.z.1, length 28 I got it working by patching socket.c in the following manner: I guess this should be confirmed by someone else too, and I haven't looked into IPv6 side at all, but the above patch works for me ™. Best regards, Teodor Milkov ------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot