[Openvpn-devel,v2,13/25] dco: implement dco support for p2p/client code path

Message ID 20220804071401.12410-1-a@unstable.cc
State Accepted
Headers show
Series None | expand

Commit Message

Antonio Quartulli Aug. 3, 2022, 9:14 p.m. UTC
With this change we introduce ovpn-dco support only along the p2p/client
code path. Server codebase is still unchanged.

Signed-off-by: Antonio Quartulli <a@unstable.cc>
---

Changes from v1:
* rebased
---
 src/openvpn/dco.c     | 91 +++++++++++++++++++++++++++++++++++++++++++
 src/openvpn/dco.h     | 48 +++++++++++++++++++++++
 src/openvpn/event.h   |  3 ++
 src/openvpn/forward.c | 63 ++++++++++++++++++++++++++++--
 src/openvpn/init.c    | 33 ++++++++++++++++
 src/openvpn/init.h    |  2 +-
 src/openvpn/socket.h  |  1 +
 7 files changed, 237 insertions(+), 4 deletions(-)

Comments

Gert Doering Aug. 4, 2022, 3:30 a.m. UTC | #1
Acked-by: Gert Doering <gert@greenie.muc.de>

After all the preliminary infrastructure building, *this* is the beginning
of the real thing :-)

I have tested

 - full set of server side tests, without --enable-dco
   (this system does not have kernel DCO, so it does not matter)
   --> all works

 - full set of client side tests, with --enable-dco, but no kernel DCO
   ("existing setup")
   --> all works

 - full set of client side tests, with --enable-dco AND kernel DCO
   (wohoo!)
   --> some test instances disable DCO (like, SOCKS or HTTP proxy, or
       TAP mode), and the fallback works ("pings succeed")

	1b:openvpn.log  Note: --http-proxy disables data channel offload.
	1c:openvpn.log  Note: --http-proxy disables data channel offload.
	1d:openvpn.log  Note: --socks-proxy disables data channel offload.
	1e:openvpn.log  Note: --socks-proxy disables data channel offload.
	1z:openvpn.log  Note: Using compression disables data channel offload.
	2a:openvpn.log  Note: cipher 'BF-CBC' in --data-ciphers is not supported by ovpn-dco, disabling data channel offload.
	2d:openvpn.log  Note: --socks-proxy disables data channel offload.
	2e:openvpn.log  Note: --socks-proxy disables data channel offload.
	2z:openvpn.log  Note: Using compression disables data channel offload.
	3z:openvpn.log  Note: Using compression disables data channel offload.
	4:openvpn.log  Note: dev-type not tun, disabling data channel offload.
	4a:openvpn.log  Note: dev-type not tun, disabling data channel offload.
	4b:openvpn.log  Note: dev-type not tun, disabling data channel offload.
	6:openvpn.log  Note: --fragment disables data channel offloa .
	8:openvpn.log  Note: Using compression disables data channel offload.
	9:openvpn.log  Note: dev-type not tun, disabling data channel offload.
	23:openvpn.log  Note: --data-cipher-fallback with cipher 'BF-CBC' disables data channel offload.
	23a:openvpn.log  Note: Using compression disables data channel offload.
	23s:openvpn.log  Note: --data-cipher-fallback with cipher 'BF-CBC' disables data channel offload.
	24:openvpn.log  Note: Using compression disables data channel offload.
	24a:openvpn.log  Note: Using compression disables data channel offload.

   --> other instances claim to are using DCO ("ip -d link show"
       shows "ovpn-dco") *and* packets are moved, so I guess it's
       using DCO...

   these tests include "normal --client clients", "p2p --secret",
   "p2p --tls-secret", and "p2p --tls-secret with P2P NCP", using
   varying ciphers (-> BF-CBC/none forcing non-DCO, etc.)

	Test sets succeeded: 1 1a 1b 1c 1d 1e 1z 2 2a 2d 2e 2z 3 3z 4 4a 4b 5 6 8 9 23 23a 23s 24 24a.
	Test sets failed: 2b 2c 2f.

   The 3 failures (2b, 2c, 2f) are all "IPv6 UDP fragments" (ping -s 3000,
   encapsulated in IPv6 UDP), which needs closer investigation.  This works
   on a "--disable-dco" build, but the whole topic of UDP fragmentation
   is "outside OpenVPN", so this is not something a patch to OpenVPN
   can affect.  tcpdump on an intermediate host can see outgoing fragments
   in the DCO case, but no replies - different from the non-DCO case, so
   this is going to be an interesting root cause hunt...


 - I have not done performance tests, because the current test
   environment is not really suited for it yet (server instances
   are all non-DCO)


Plus, stared at the code and discussed with Antonio on IRC :-) - 
(especially the process_outgoing_link() change confused me a bit - the
obvious answer to this is "these are control channel packets, which are
still created by userland, but the design requires to avoid accessing
the socket directly, so send to DCO module, and that one forwards").

I removed one spurious blank line from dco_p2p_add_new_peer().

Your patch has been applied to the master branch.

commit b6f7b285767e66f5cbd3854cf0ff918e87b31202
Author: Antonio Quartulli
Date:   Thu Aug 4 09:14:01 2022 +0200

     dco: implement dco support for p2p/client code path

     Signed-off-by: Antonio Quartulli <a@unstable.cc>
     Acked-by: Gert Doering <gert@greenie.muc.de>
     Message-Id: <20220804071401.12410-1-a@unstable.cc>
     URL: https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg24798.html
     Signed-off-by: Gert Doering <gert@greenie.muc.de>


--
kind regards,

Gert Doering
Antonio Quartulli Aug. 4, 2022, 3:51 a.m. UTC | #2
On 04/08/2022 15:30, Gert Doering wrote:
> Acked-by: Gert Doering <gert@greenie.muc.de>
[]>
> Your patch has been applied to the master branch.

Wohooo! Great stuff!

Thanks to everybody who contributed to this first milestone!
However, the party is not over yet :-)

Cheers,

> 
> commit b6f7b285767e66f5cbd3854cf0ff918e87b31202
> Author: Antonio Quartulli
> Date:   Thu Aug 4 09:14:01 2022 +0200
> 
>       dco: implement dco support for p2p/client code path
> 
>       Signed-off-by: Antonio Quartulli <a@unstable.cc>
>       Acked-by: Gert Doering <gert@greenie.muc.de>
>       Message-Id: <20220804071401.12410-1-a@unstable.cc>
>       URL: https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg24798.html
>       Signed-off-by: Gert Doering <gert@greenie.muc.de>
> 
> 
> --
> kind regards,
> 
> Gert Doering
>
Gert Doering Aug. 4, 2022, 5:17 a.m. UTC | #3
Hi,

On Thu, Aug 04, 2022 at 03:30:25PM +0200, Gert Doering wrote:
> 	Test sets failed: 2b 2c 2f.

So, this is an interesting one.

To trigger this, you need to connect over UDP + IPv6 transport, and
then you need to inject a packet into "openvpn with DCO" that is
fragmented into two parts, which *both* create an UDP packet after
encapsulation which is bigger than 1500 bytes.

$ ping -c1 -s 2900 10.194.2.1

does this.

Tcpdump on the "tun0" interface shows

15:49:16.433666 IP (tos 0x0, ttl 64, id 18639, offset 0, flags [+], proto ICMP (1), length 1500)
    ubuntu2004 > 10.194.2.1: ICMP echo request, id 170, seq 1, length 1480
15:49:16.433703 IP (tos 0x0, ttl 64, id 18639, offset 1480, flags [none], proto ICMP (1), length 1448)
    ubuntu2004 > 10.194.2.1: icmp

so this is one "full sized 1500 byte frame", containing the first fragment,
and then a 1448 byte frame containing the rest.


Connecting with --disable-dco yields this on the outside:

17:09:14.763519 IP6 (flowlabel 0x0082a, hlim 62, next-header Fragment (44) payload length: 1456) ubuntu2004 > phillip: frag (0x277a4245:0|1448) 34646 > 51194: UDP, bad length 1524 > 1440
17:09:14.763548 IP6 (flowlabel 0x0082a, hlim 62, next-header Fragment (44) payload length: 92) ubuntu2004 > phillip: frag (0x277a4245:1448|84)
17:09:14.763576 IP6 (flowlabel 0x0082a, hlim 62, next-header Fragment (44) payload length: 1456) ubuntu2004 > phillip: frag (0x1dec474f:0|1448) 34646 > 51194: UDP, bad length 1472 > 1440
17:09:14.763593 IP6 (flowlabel 0x0082a, hlim 62, next-header Fragment (44) payload length: 40) ubuntu2004 > phillip: frag (0x1dec474f:1448|32)

this is "4 packets are sent" (and, subsequently, 4 packets come back).

Both "(close to) full inside packets" are too big for an external 1500 byte
packet, and get fragmented.   Is this ugly?  Yes.  Is it according to IP
specs?  Yes.  Do I expect this to work?  Yes!  (t_client excercises this
case with the 3000-byte-pings).


Now, *with* DCO, we get this:

17:10:43.771215 IP6 (hlim 62, next-header Fragment (44) payload length: 1456) ubuntu2004 > phillip: frag (0xe79d84cc:0|1448) 58755 > 51194: UDP, bad length 1524 > 1440
17:10:43.771246 IP6 (hlim 62, next-header Fragment (44) payload length: 92) ubuntu2004 > phillip: frag (0xe79d84cc:1448|84)

(only *two* packets, then only keepalives)

And at the same time, the client (!) complains

2022-08-04 17:10:38 net_iface_new: add tun0 type ovpn-dco
2022-08-04 17:10:38 DCO device tun0 opened
(so, we're using DCO)
2022-08-04 17:10:43 read UDPv6 [CMSG=50|EMSGSIZE Path-MTU=1500]: Message too long (fd=3,code=90)



Now... reducing the ping size to 2500...

gert@ubuntu2004:~$ ping -c1 -s 2500 10.194.2.1
2508 bytes from 10.194.2.1: icmp_seq=1 ttl=64 time=126 ms

... actually works.

What is sent out by DCO in this case is 3 packets, 2 fragments, and
one regular UDP packet

17:12:21.512762 IP6 (hlim 62, next-header Fragment (44) payload length: 1456) ubuntu2004 > phillip: frag (0x12728a31:0|1448) 58755 > 51194: UDP, bad length 1524 > 1440
17:12:21.512806 IP6 (hlim 62, next-header Fragment (44) payload length: 92) ubuntu2004 > phillip: frag (0x12728a31:1448|84)
17:12:21.512864 IP6 (hlim 63, next-header UDP (17) payload length: 1080) ubuntu2004.58755 > phillip.51194: [udp sum ok] UDP, length 1072

... because the second half of the fragmented ICMP packet now fits into
one UDP/1500 byte even after openvpn encapsulation.


So my current guess is that the linux kernel sets *some* flag on
"non-primary ICMP fragments" (inside) that get carried over to the final 
encapsulated packet, preventing further fragmentation (outside), but
eliciting an "PMTU fail" error instead - which the newly renovated
extended socket error handler picks up.  In userland :-)


To add a data point:

gert@ubuntu2004:~$ ping -c1 -s 3500 10.194.2.1

creates *3* packets in the DCO case

17:14:08.780733 IP6 (hlim 62, next-header Fragment (44) payload length: 1456) ubuntu2004 > phillip: frag (0x16acdc72:0|1448) 58755 > 51194: UDP, bad length 1524 > 1440
17:14:08.780757 IP6 (hlim 62, next-header Fragment (44) payload length: 92) ubuntu2004 > phillip: frag (0x16acdc72:1448|84)
17:14:08.780842 IP6 (hlim 63, next-header UDP (17) payload length: 600) ubuntu2004.58755 > phillip.51194: [udp sum ok] UDP, length 592

... and one error message.

In the non-DCO case, this creates *5* packets - 2 1500-inside packet
that get outside fragmented to 2+2 and one smaller "leftover" packet.

So the 2nd fragment is, again, refusing to be fragmented again...


(ping -s 5000 yields 7 packet without DCO, and only 3 with DCO, so a
pattern starts to form ;-) )

gert

Patch

diff --git a/src/openvpn/dco.c b/src/openvpn/dco.c
index 8c22b7ea..b5cc8a70 100644
--- a/src/openvpn/dco.c
+++ b/src/openvpn/dco.c
@@ -36,6 +36,7 @@ 
 #include "crypto.h"
 #include "dco.h"
 #include "errlevel.h"
+#include "multi.h"
 #include "networking.h"
 #include "openvpn.h"
 #include "options.h"
@@ -382,4 +383,94 @@  dco_check_pull_options(int msglevel, const struct options *o)
     return true;
 }
 
+int
+dco_p2p_add_new_peer(struct context *c)
+{
+    if (!dco_enabled(&c->options))
+    {
+        return 0;
+    }
+
+
+    struct tls_multi *multi = c->c2.tls_multi;
+    struct link_socket *ls = c->c2.link_socket;
+
+    struct in6_addr remote_ip6 = { 0 };
+    struct in_addr remote_ip4 = { 0 };
+
+    struct in6_addr *remote_addr6 = NULL;
+    struct in_addr *remote_addr4 = NULL;
+
+    const char *gw = NULL;
+
+    ASSERT(ls->info.connection_established);
+
+    /* In client mode if a P2P style topology is used we assume the
+     * remote-gateway is the IP of the peer */
+    if (c->options.topology == TOP_NET30 || c->options.topology == TOP_P2P)
+    {
+        gw = c->options.ifconfig_remote_netmask;
+    }
+    if (c->options.route_default_gateway)
+    {
+        gw = c->options.route_default_gateway;
+    }
+
+    /* These inet_pton conversion are fatal since options.c already implements
+     * checks to have only valid addresses when setting the options */
+    if (c->options.ifconfig_ipv6_remote)
+    {
+        if (inet_pton(AF_INET6, c->options.ifconfig_ipv6_remote, &remote_ip6) != 1)
+        {
+            msg(M_FATAL,
+                "DCO peer init: problem converting IPv6 ifconfig remote address %s to binary",
+                c->options.ifconfig_ipv6_remote);
+        }
+        remote_addr6 = &remote_ip6;
+    }
+
+    if (gw)
+    {
+        if (inet_pton(AF_INET, gw, &remote_ip4) != 1)
+        {
+            msg(M_FATAL, "DCO peer init: problem converting IPv4 ifconfig gateway address %s to binary", gw);
+        }
+        remote_addr4 = &remote_ip4;
+    }
+    else if (c->options.ifconfig_local)
+    {
+        msg(M_INFO, "DCO peer init: Need a peer VPN addresss to setup IPv4 (set --route-gateway)");
+    }
+
+    struct sockaddr *remoteaddr = &ls->info.lsa->actual.dest.addr.sa;
+
+    int ret = dco_new_peer(&c->c1.tuntap->dco, multi->peer_id,
+                           c->c2.link_socket->sd, NULL, remoteaddr,
+                           remote_addr4, remote_addr6);
+    if (ret < 0)
+    {
+        return ret;
+    }
+
+    c->c2.tls_multi->dco_peer_added = true;
+    c->c2.link_socket->info.dco_installed = true;
+
+    return 0;
+}
+
+void
+dco_remove_peer(struct context *c)
+{
+    if (!dco_enabled(&c->options))
+    {
+        return;
+    }
+
+    if (c->c1.tuntap && c->c2.tls_multi && c->c2.tls_multi->dco_peer_added)
+    {
+        dco_del_peer(&c->c1.tuntap->dco, c->c2.tls_multi->peer_id);
+        c->c2.tls_multi->dco_peer_added = false;
+    }
+}
+
 #endif /* defined(ENABLE_DCO) */
diff --git a/src/openvpn/dco.h b/src/openvpn/dco.h
index fbb35906..602dafb7 100644
--- a/src/openvpn/dco.h
+++ b/src/openvpn/dco.h
@@ -151,6 +151,36 @@  int init_key_dco_bi(struct tls_multi *multi, struct key_state *ks,
  */
 void dco_update_keys(dco_context_t *dco, struct tls_multi *multi);
 
+/**
+ * Install a new peer in DCO - to be called by a CLIENT (or P2P) instance
+ *
+ * @param c         the main instance context
+ * @return          0 on success or a negative error code otherwise
+ */
+int dco_p2p_add_new_peer(struct context *c);
+
+/**
+ * Modify DCO peer options. Special values are 0 (disable)
+ * and -1 (do not touch).
+ *
+ * @param dco                DCO device context
+ * @param peer_id            the ID of the peer to be modified
+ * @param keepalive_interval keepalive interval in seconds
+ * @param keepalive_timeout  keepalive timeout in seconds
+ * @param mss                TCP MSS value
+ *
+ * @return                   0 on success or a negative error code otherwise
+ */
+int dco_set_peer(dco_context_t *dco, unsigned int peerid,
+                 int keepalive_interval, int keepalive_timeout, int mss);
+
+/**
+ * Remove a peer from DCO
+ *
+ * @param c         the main instance context of the peer to remove
+ */
+void dco_remove_peer(struct context *c);
+
 #else /* if defined(ENABLE_DCO) */
 
 typedef void *dco_context_t;
@@ -223,5 +253,23 @@  dco_update_keys(dco_context_t *dco, struct tls_multi *multi)
     ASSERT(false);
 }
 
+static inline bool
+dco_p2p_add_new_peer(struct context *c)
+{
+    return true;
+}
+
+static inline int
+dco_set_peer(dco_context_t *dco, unsigned int peerid,
+             int keepalive_interval, int keepalive_timeout, int mss)
+{
+    return 0;
+}
+
+static inline void
+dco_remove_peer(struct context *c)
+{
+}
+
 #endif /* defined(ENABLE_DCO) */
 #endif /* ifndef DCO_H */
diff --git a/src/openvpn/event.h b/src/openvpn/event.h
index a472afbe..f2438f97 100644
--- a/src/openvpn/event.h
+++ b/src/openvpn/event.h
@@ -72,6 +72,9 @@ 
 #define MANAGEMENT_WRITE    (1 << (MANAGEMENT_SHIFT + WRITE_SHIFT))
 #define FILE_SHIFT          8
 #define FILE_CLOSED         (1 << (FILE_SHIFT + READ_SHIFT))
+#define DCO_SHIFT           10
+#define DCO_READ            (1 << (DCO_SHIFT + READ_SHIFT))
+#define DCO_WRITE           (1 << (DCO_SHIFT + WRITE_SHIFT))
 
 /*
  * Initialization flags passed to event_set_init
diff --git a/src/openvpn/forward.c b/src/openvpn/forward.c
index 38d2683c..55c939c4 100644
--- a/src/openvpn/forward.c
+++ b/src/openvpn/forward.c
@@ -1110,6 +1110,39 @@  process_incoming_link(struct context *c)
     perf_pop();
 }
 
+static void
+process_incoming_dco(struct context *c)
+{
+#if defined(ENABLE_DCO) && defined(TARGET_LINUX)
+    struct link_socket_info *lsi = get_link_socket_info(c);
+    dco_context_t *dco = &c->c1.tuntap->dco;
+
+    dco_do_read(dco);
+
+    if (dco->dco_message_type == OVPN_CMD_DEL_PEER)
+    {
+        trigger_ping_timeout_signal(c);
+        return;
+    }
+
+    if (dco->dco_message_type != OVPN_CMD_PACKET)
+    {
+        msg(D_DCO_DEBUG, "%s: received message of type %u - ignoring", __func__,
+            dco->dco_message_type);
+        return;
+    }
+
+    struct buffer orig_buff = c->c2.buf;
+    c->c2.buf = dco->dco_packet_in;
+    c->c2.from = lsi->lsa->actual;
+
+    process_incoming_link(c);
+
+    c->c2.buf = orig_buff;
+    buf_init(&dco->dco_packet_in, 0);
+#endif /* if defined(ENABLE_DCO) && defined(TARGET_LINUX) */
+}
+
 /*
  * Output: c->c2.buf
  */
@@ -1633,9 +1666,17 @@  process_outgoing_link(struct context *c)
                 socks_preprocess_outgoing_link(c, &to_addr, &size_delta);
 
                 /* Send packet */
-                size = link_socket_write(c->c2.link_socket,
-                                         &c->c2.to_link,
-                                         to_addr);
+                if (c->c2.link_socket->info.dco_installed)
+                {
+                    size = dco_do_write(&c->c1.tuntap->dco,
+                                        c->c2.tls_multi->peer_id,
+                                        &c->c2.to_link);
+                }
+                else
+                {
+                    size = link_socket_write(c->c2.link_socket, &c->c2.to_link,
+                                             to_addr);
+                }
 
                 /* Undo effect of prepend */
                 link_socket_write_post_size_adjust(&size, size_delta, &c->c2.to_link);
@@ -1905,6 +1946,9 @@  io_wait_dowork(struct context *c, const unsigned int flags)
 #ifdef ENABLE_ASYNC_PUSH
     static int file_shift = FILE_SHIFT;
 #endif
+#ifdef TARGET_LINUX
+    static int dco_shift = DCO_SHIFT;    /* Event from DCO linux kernel module */
+#endif
 
     /*
      * Decide what kind of events we want to wait for.
@@ -2012,6 +2056,12 @@  io_wait_dowork(struct context *c, const unsigned int flags)
      */
     socket_set(c->c2.link_socket, c->c2.event_set, socket, (void *)&socket_shift, NULL);
     tun_set(c->c1.tuntap, c->c2.event_set, tuntap, (void *)&tun_shift, NULL);
+#if defined(TARGET_LINUX)
+    if (socket & EVENT_READ && c->c2.did_open_tun)
+    {
+        dco_event_set(&c->c1.tuntap->dco, c->c2.event_set, (void *)&dco_shift);
+    }
+#endif
 
 #ifdef ENABLE_MANAGEMENT
     if (management)
@@ -2134,4 +2184,11 @@  process_io(struct context *c)
             process_incoming_tun(c);
         }
     }
+    else if (status & DCO_READ)
+    {
+        if (!IS_SIG(c))
+        {
+            process_incoming_dco(c);
+        }
+    }
 }
diff --git a/src/openvpn/init.c b/src/openvpn/init.c
index 4423e162..340c75d9 100644
--- a/src/openvpn/init.c
+++ b/src/openvpn/init.c
@@ -2106,6 +2106,22 @@  do_deferred_options_part2(struct context *c)
         return false;
     }
 
+    if (dco_enabled(&c->options)
+        && (c->options.ping_send_timeout || c->c2.frame.mss_fix))
+    {
+        int ret = dco_set_peer(&c->c1.tuntap->dco,
+                               c->c2.tls_multi->peer_id,
+                               c->options.ping_send_timeout,
+                               c->options.ping_rec_timeout,
+                               c->c2.frame.mss_fix);
+        if (ret < 0)
+        {
+            msg(D_DCO, "Cannot set parameters for DCO peer (id=%u): %s",
+                c->c2.tls_multi->peer_id, strerror(-ret));
+            return false;
+        }
+    }
+
     return true;
 }
 
@@ -2150,6 +2166,19 @@  do_up(struct context *c, bool pulled_options, unsigned int option_types_found)
             }
         }
 
+        if (c->mode == MODE_POINT_TO_POINT)
+        {
+            /* ovpn-dco requires adding the peer now, before any option can be set,
+             * but *after* having parsed the pushed peer-id in do_deferred_options()
+             */
+            int ret = dco_p2p_add_new_peer(c);
+            if (ret < 0)
+            {
+                msg(D_DCO, "Cannot add peer to DCO: %s (%d)", strerror(-ret), ret);
+                return false;
+            }
+        }
+
         /* do_deferred_options_part2() and do_deferred_p2p_ncp() *must* be
          * invoked after open_tun().
          * This is required by DCO because we must have created the interface
@@ -4363,6 +4392,10 @@  close_instance(struct context *c)
         /* free buffers */
         do_close_free_buf(c);
 
+        /* close peer for DCO if enabled, needs peer-id so must be done before
+         * closing TLS contexts */
+        dco_remove_peer(c);
+
         /* close TLS */
         do_close_tls(c);
 
diff --git a/src/openvpn/init.h b/src/openvpn/init.h
index 5f412a33..f53b65ee 100644
--- a/src/openvpn/init.h
+++ b/src/openvpn/init.h
@@ -30,7 +30,7 @@ 
  * Baseline maximum number of events
  * to wait for.
  */
-#define BASE_N_EVENTS 4
+#define BASE_N_EVENTS 5
 
 void context_clear(struct context *c);
 
diff --git a/src/openvpn/socket.h b/src/openvpn/socket.h
index a75adb00..0d521d22 100644
--- a/src/openvpn/socket.h
+++ b/src/openvpn/socket.h
@@ -120,6 +120,7 @@  struct link_socket_info
     sa_family_t af;                     /* Address family like AF_INET, AF_INET6 or AF_UNSPEC*/
     bool bind_ipv6_only;
     int mtu_changed;            /* Set to true when mtu value is changed */
+    bool dco_installed;
 };
 
 /*