[Openvpn-devel] Bulk mode - Feature request - Patch diff [updated]

Thanks to Gert's help on this, I was able to finally configure and compile
and run and test the bulk mode changes against the latest git source code
to ensure everything still works correctly.

I also fixed up some other issues like properly freeing the extra buffer
allocations and removing the unneeded batched data prefixes and converting
a remaining while loop to a max limited for loop and properly resetting the
outgoing tun buffer pointer at the end of the write method when finished.

Thanks,
Jon C

Example updated pull request:
https://github.com/OpenVPN/openvpn/pull/814/files

git formatted diff patch:

From 985e88a9af26a39554f113f37ee18032a2f41c3e Mon Sep 17 00:00:00 2001
From: Jon Chiappetta <root@fossjon.com>
Date: Wed, 6 Aug 2025 16:33:18 -0400
Subject: [PATCH] bulk mode

---
 src/openvpn/forward.c | 217 ++++++++++++++++++++++++++++++++++++++++--
 src/openvpn/forward.h |   4 +
 src/openvpn/init.c    |  56 +++++++++++
 src/openvpn/mtu.c     |  10 +-
 src/openvpn/mtu.h     |  13 +++
 src/openvpn/multi.c   |   7 +-
 src/openvpn/openvpn.h |  10 ++
 src/openvpn/options.c |   8 ++
 src/openvpn/options.h |   3 +
 9 files changed, 316 insertions(+), 12 deletions(-)

--
2.39.5 (Apple Git-154)

Am 07.08.25 um 20:29 schrieb Jon Chiappetta via Openvpn-devel:
> Thanks to Gert's help on this, I was able to finally configure and 
> compile and run and test the bulk mode changes against the latest git 
> source code to ensure everything still works correctly.
> 
> I also fixed up some other issues like properly freeing the extra buffer 
> allocations and removing the unneeded batched data prefixes and 
> converting a remaining while loop to a max limited for loop and properly 
> resetting the outgoing tun buffer pointer at the end of the write method 
> when finished.


It would still good to explain what you are trying to achieve here and 
what the idea behind the patch is to be able to review and understand 
your patch.

The patch itself basically has no comments at all, so it is very hard to 
decipher for me from the patch what it is trying to to do. Eg there is a 
variable flag_ciph that fiddles with encryption of packets.

You are talking and describing this bulk mode as if it was obvious but 
it is not. The description on your blog says:

 > [...] read 8192 bytes off of the client’s TCP sockets directly and
 > proxy them in one write call over TCP directly to the VPN server
 > without needing a tunnel interface with a small sized MTU which
 > bottlenecks reads+writes to <1500 bytes per function call.

It also not helping as you talking about TCP write/reads, where I can 
see some improvement by cutting down the number of reads/writes. But the 
second part then talks about not using a tunnel with a small sized MTU.
But if you use a larger sized TUN interface with a larger MTU, then you 
already have larger reads/writes to the TCP socket.

Also your speedtest showing 562 is meaningless without having any 
comparison without your patch.

Arne

Hi Arne,

You are correct, I didn't do a very good job of explaining the code in my
blog post, I usually keep those short with more screen captures because I
figure that not many people would actually take the time to read through it
there. Also, I didn't really add many comments either but I did try to copy
the present style of the code to try and make it match and be more
consistent throughout, even if not perfect yet. I'm still testing out the
change myself in my own home setup here to see if I run into any bad edge
cases along the way.

I can always try to explain the different code parts as I am indeed
modifying the core parts of the read and write operations for tun and tcp
so it's a big change to make to the code base. Basically this change is
important to me in particular because of my setup and requirements in
specific. I have WiFi LAN clients which all assume a 1500 byte MTU on their
side and I have a router WAN client which enforces a 1500 byte MTU on the
internet's side. In the middle of my core network is a VPN box and almost
every VPN software will operate in UDP mode with a sub-1500 MTU in the
middle of this network pipeline. This is not a good design to have in
general as I don't want to waste cycles fragmenting and/or compressing the
data into smaller sized UDP packets. With the code change I am presenting,
I am able to specify a true 1500 byte VPN MTU interface with the exact
matching 1500 byte read calls to the TUN interface itself (the code base
had to be modified to allow for this because it was adding onto the
payload_size used in the read call which I didn't want as I am operating on
exact multiples of 1500 bytes in specific).

With this change, my network pipeline is a true 1500 byte MTU which matches
all the way from the client side to the vpn link to the internet side and
to the server side (end to end to end to end). In addition, I also added
the ability to batch together multiple 1500 byte read calls (specifically 6
x 1500 bytes into 9000 bytes) into one single encrypt sign call and one
single TCP write call. This allows the encryption method to operate only
once on a much larger payload size as well as allow the linux kernel to
efficiently transfer the data with order and delivery guaranteed as fast as
possible. The code base had to be modified to allow for all of this as well
as it was preventing me from performing this much larger sized ssl sign and
encrypt + tcp read and write (the code base assumes you are operating on
only 1 tun read call worth of data at a time everywhere).

This is exactly why I prefer using TCP to tunnel encrypted network data as
my solution provided can properly set a full sized 1500 byte MTU as well as
perform an exact matching read call of 1500 bytes to get the full amount of
data from the interface and then bulk it together to efficiently encrypt it
and then use the magic of TCP to transfer that data all at once as quickly
as possible without any need for fragmentation or compression. I don't
think any other VPN product on the market offers that kind of functionality
as far as I am aware as most other VPN products use a smaller sized MTU as
well as the packet size limitations of UDP. I believe that this could be a
distinguishing feature for OpenVPN as well as automatically solve some of
the issues that folks run into when inserting a VPN appliance into the
middle of their network setups.

I've been running this change on my own setup to at least make sure it
works and it seems to be running pretty nicely so far. I haven't
experienced any fragmentation or performance issues as any sized data that
comes off the clients LAN side is fully taken care of now through the VPN
side and onto the WAN and server side.

If this is something you are not interested in I can understand that, I can
stop posting here and the most I can do is at least submit a pull request
in case anyone in the future is indeed interested in such work. It'd be
nice to contribute to a good quality open source project that I have used
for many many years and something which may help solve other community
member's issues with regards to the small sized MTU + UDP problem which
does exist in practice and really hampers connections along the way in a
network design.

I also don't mind explaining my code parts if you actually want, I just
need to take time to write them out and describe what they are doing and
why. As you can see, I am trying to achieve a very specific and exact
design goal that the code base wasn't originally allowing for, so I had to
make some modifications to be able to accomplish it.

Thanks,
Jon C




On Fri, Aug 8, 2025 at 5:53 AM Arne Schwabe <arne@rfc2549.org> wrote:

> Am 07.08.25 um 20:29 schrieb Jon Chiappetta via Openvpn-devel:
> > Thanks to Gert's help on this, I was able to finally configure and
> > compile and run and test the bulk mode changes against the latest git
> > source code to ensure everything still works correctly.
> >
> > I also fixed up some other issues like properly freeing the extra buffer
> > allocations and removing the unneeded batched data prefixes and
> > converting a remaining while loop to a max limited for loop and properly
> > resetting the outgoing tun buffer pointer at the end of the write method
> > when finished.
>
>
> It would still good to explain what you are trying to achieve here and
> what the idea behind the patch is to be able to review and understand
> your patch.
>
> The patch itself basically has no comments at all, so it is very hard to
> decipher for me from the patch what it is trying to to do. Eg there is a
> variable flag_ciph that fiddles with encryption of packets.
>
> You are talking and describing this bulk mode as if it was obvious but
> it is not. The description on your blog says:
>
>  > [...] read 8192 bytes off of the client’s TCP sockets directly and
>  > proxy them in one write call over TCP directly to the VPN server
>  > without needing a tunnel interface with a small sized MTU which
>  > bottlenecks reads+writes to <1500 bytes per function call.
>
> It also not helping as you talking about TCP write/reads, where I can
> see some improvement by cutting down the number of reads/writes. But the
> second part then talks about not using a tunnel with a small sized MTU.
> But if you use a larger sized TUN interface with a larger MTU, then you
> already have larger reads/writes to the TCP socket.
>
> Also your speedtest showing 562 is meaningless without having any
> comparison without your patch.
>
> Arne
>

[replying to my own update thread]

I pushed an update to the PR with the following small changes:

- rebased to the latest master commit
- increased the data buffer size to be slightly bigger than the read buffer
size
- copied two more buf resets so that the additional bulk forward functions
match the present logic checks

Example updated pull request:
https://github.com/OpenVPN/openvpn/pull/814/files

$ cat 0001-bulk-mode.patch
From 2ce0b023d105e7ecc289a414cd26f7ebc8bcbcaf Mon Sep 17 00:00:00 2001
From: Jon Chiappetta <root@fossjon.com>
Date: Wed, 6 Aug 2025 16:33:18 -0400
Subject: [PATCH] bulk mode

---
 src/openvpn/forward.c | 228 ++++++++++++++++++++++++++++++++++++++++--
 src/openvpn/forward.h |   4 +
 src/openvpn/init.c    |  57 +++++++++++
 src/openvpn/mtu.c     |  10 +-
 src/openvpn/mtu.h     |  13 +++
 src/openvpn/multi.c   |   7 +-
 src/openvpn/openvpn.h |  10 ++
 src/openvpn/options.c |   8 ++
 src/openvpn/options.h |   3 +
 9 files changed, 328 insertions(+), 12 deletions(-)

diff --git a/src/openvpn/forward.c b/src/openvpn/forward.c
index 75ca9d5c..0af983e9 100644
--- a/src/openvpn/forward.c
+++ b/src/openvpn/forward.c
@@ -46,6 +46,9 @@

 #include "mstats.h"

+#include <sys/select.h>
+#include <sys/time.h>
+
 counter_type link_read_bytes_global;  /* GLOBAL */
 counter_type link_write_bytes_global; /* GLOBAL */

@@ -78,6 +81,32 @@ show_wait_status(struct context *c)

 #endif /* ifdef ENABLE_DEBUG */

+bool check_bulk_mode(struct context *c)
+{
+    if ((c->c2.frame.bulk_size > 0) && (c->c1.tuntap != NULL) &&
(c->c2.buffers != NULL))
+    {
+        return true;
+    }
+    return false;
+}
+
+void xfer_io(struct context *c, struct context *b)
+{
+    int plen = 0;
+    if (check_bulk_mode(b))
+    {
+        int leng = (b->c2.buffers->bufs_indx + 1);
+        for (int x = 0; x < leng; ++x)
+        {
+            plen = BLEN(&b->c2.bufs[x]);
+            if (plen < 1) { c->c2.bufs[x].len = 0; }
+            else { c->c2.bufs[x] = b->c2.bufs[x]; }
+        }
+        c->c2.buffers->bufs_indx = b->c2.buffers->bufs_indx;
+        b->c2.buffers->bufs_indx = -1;
+    }
+}
+
 static void
 check_tls_errors_co(struct context *c)
 {
@@ -605,6 +634,21 @@ buffer_turnover(const uint8_t *orig_buf, struct buffer
*dest_stub, struct buffer
     }
 }

+uint8_t *buff_prepsize(uint8_t *buff, int *size)
+{
+    buff[0] = ((*size >> 8) & 0xff);
+    buff[1] = ((*size >> 0) & 0xff);
+    buff += 2;
+    return buff;
+}
+
+uint8_t *buff_postsize(uint8_t *buff, int *size)
+{
+    *size = ((buff[0] << 8) + (buff[1] << 0));
+    buff += 2;
+    return buff;
+}
+
 /*
  * Compress, fragment, encrypt and HMAC-sign an outgoing packet.
  * Input: c->c2.buf
@@ -1031,6 +1075,7 @@ process_incoming_link_part1(struct context *c, struct
link_socket_info *lsi, boo
         fprintf(stderr, "R");
     }
 #endif
+
     msg(D_LINK_RW, "%s READ [%d] from %s: %s", proto2ascii(lsi->proto,
lsi->af, true),
         BLEN(&c->c2.buf), print_link_socket_actual(&c->c2.from, &gc),
PROTO_DUMP(&c->c2.buf, &gc));

@@ -1211,6 +1256,27 @@ process_incoming_link_part2(struct context *c,
struct link_socket_info *lsi,
     }
 }

+void process_incoming_link_part3(struct context *c)
+{
+    int leng = BLEN(&c->c2.buf);
+    if (leng > 0)
+    {
+        if (check_bulk_mode(c))
+        {
+            c->c2.buffers->send_tun_max.offset = TUN_BAT_OFF;
+            c->c2.buffers->send_tun_max.len = leng;
+            bcopy(BPTR(&c->c2.buf), BPTR(&c->c2.buffers->send_tun_max),
leng);
+            //dmsg(M_INFO, "FWD BAT LINK 0 [%d] [%d] [%d] [%d] [%d]",
BLEN(&c->c2.buf), BLEN(&c->c2.to_tun), BLEN(&c->c2.buffers->read_link_buf),
BLEN(&c->c2.buffers->read_link_buf), BLEN(&c->c2.buffers->send_tun_max));
+            c->c2.to_tun.offset += 2;
+            c->c2.buf.offset += 2;
+        }
+    }
+    else
+    {
+        buf_reset(&c->c2.to_tun);
+    }
+}
+
 static void
 process_incoming_link(struct context *c, struct link_socket *sock)
 {
@@ -1221,6 +1287,7 @@ process_incoming_link(struct context *c, struct
link_socket *sock)

     process_incoming_link_part1(c, lsi, false);
     process_incoming_link_part2(c, lsi, orig_buf);
+    process_incoming_link_part3(c);

     perf_pop();
 }
@@ -1321,7 +1388,7 @@ process_incoming_dco(struct context *c)
  */

 void
-read_incoming_tun(struct context *c)
+read_incoming_tun_part2(struct context *c)
 {
     /*
      * Setup for read() call on TUN/TAP device.
@@ -1382,6 +1449,55 @@ read_incoming_tun(struct context *c)
     perf_pop();
 }

+void read_incoming_tun_part3(struct context *c)
+{
+    fd_set rfds;
+    struct timeval timo;
+    if (check_bulk_mode(c))
+    {
+        int plen = 0;
+        int fdno = c->c1.tuntap->fd;
+        for (int x = 0; x < TUN_BAT_MAX; ++x)
+        {
+            int leng = plen;
+            int indx = (c->c2.buffers->bufs_indx + 1);
+            if (indx >= TUN_BAT_MIN) { break; }
+            if (leng < 1)
+            {
+                FD_ZERO(&rfds);
+                FD_SET(fdno, &rfds);
+                timo.tv_sec = 0;
+                timo.tv_usec = 0;
+                select(fdno+1, &rfds, NULL, NULL, &timo);
+                if (FD_ISSET(fdno, &rfds))
+                {
+                    read_incoming_tun_part2(c);
+                    plen = BLEN(&c->c2.buf);
+                } else { break; }
+            }
+            //dmsg(M_INFO, "FWD BAT READ 0 [%d] [%d] [%d] [%d] [%d]",
c->c2.buffers->bufs_indx + 1, fdno, BLEN(&c->c2.buf),
BLEN(&c->c2.buffers->read_tun_buf), BLEN(&c->c2.buffers->read_tun_max));
+            leng = plen;
+            if (leng > 0)
+            {
+                c->c2.buffers->read_tun_bufs[indx].offset = TUN_BAT_OFF;
+                c->c2.buffers->read_tun_bufs[indx].len = leng;
+                bcopy(BPTR(&c->c2.buf),
BPTR(&c->c2.buffers->read_tun_bufs[indx]), leng);
+                c->c2.bufs[indx] = c->c2.buffers->read_tun_bufs[indx];
+                c->c2.buffers->bufs_indx = indx;
+            } else { break; }
+            plen = 0;
+        }
+    }
+}
+
+void read_incoming_tun(struct context *c)
+{
+    if (c->c2.frame.bulk_size <= 0) {
+        read_incoming_tun_part2(c);
+    }
+    read_incoming_tun_part3(c);
+}
+
 /**
  * Drops UDP packets which OS decided to route via tun.
  *
@@ -1469,7 +1585,7 @@ drop_if_recursive_routing(struct context *c, struct
buffer *buf)
  */

 void
-process_incoming_tun(struct context *c, struct link_socket *out_sock)
+process_incoming_tun_part2(struct context *c, struct link_socket *out_sock)
 {
     struct gc_arena gc = gc_new();

@@ -1488,7 +1604,7 @@ process_incoming_tun(struct context *c, struct
link_socket *out_sock)
 #endif

     /* Show packet content */
-    dmsg(D_TUN_RW, "TUN READ [%d]", BLEN(&c->c2.buf));
+    dmsg(D_TUN_RW, "TUN READ [%d] [%d]", BLEN(&c->c2.buf),
c->c2.frame.buf.payload_size);

     if (c->c2.buf.len > 0)
     {
@@ -1512,7 +1628,9 @@ process_incoming_tun(struct context *c, struct
link_socket *out_sock)
     }
     if (c->c2.buf.len > 0)
     {
+        if ((c->c2.buffers == NULL) || (c->c2.buffers->flag_ciph != -2)) {
         encrypt_sign(c, true);
+        }
     }
     else
     {
@@ -1522,6 +1640,67 @@ process_incoming_tun(struct context *c, struct
link_socket *out_sock)
     gc_free(&gc);
 }

+void process_incoming_tun_part3(struct context *c, struct link_socket
*out_sock)
+{
+    if (c->c2.buf.len > 0)
+    {
+        if (check_bulk_mode(c))
+        {
+            c->c2.buffers->flag_ciph = -2;
+            c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
+            c->c2.buffers->read_tun_max.len = 0;
+            uint8_t *temp = BPTR(&c->c2.buffers->read_tun_max);
+            int plen = 0, fdno = c->c1.tuntap->fd;
+            int maxl = 0, leng = (c->c2.buffers->bufs_indx + 1);
+            if ((fdno > 0) && (leng > 0))
+            {
+                for (int x = 0; x < leng; ++x)
+                {
+                    c->c2.buf = c->c2.bufs[x];
+                    //dmsg(M_INFO, "FWD BAT INPT 0 [%d] [%d] [%d] [%d]
[%d]", x, fdno, BLEN(&c->c2.buf), BLEN(&c->c2.buffers->read_tun_buf),
BLEN(&c->c2.bufs[x]));
+                    process_incoming_tun_part2(c, out_sock);
+                    if (BLEN(&c->c2.buf) < 1)
+                    {
+                        c->c2.bufs[x].len = 0;
+                    }
+                }
+                for (int x = 0; x < leng; ++x)
+                {
+                    plen = c->c2.bufs[x].len;
+                    if (plen > 0)
+                    {
+                        temp = buff_prepsize(temp, &plen);
+                        bcopy(BPTR(&c->c2.bufs[x]), temp, plen);
+                        temp += plen; maxl += (plen + 2);
+                    }
+                }
+                if (maxl > 0)
+                {
+                    c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
+                    c->c2.buffers->read_tun_max.len = maxl;
+                    c->c2.buf = c->c2.buffers->read_tun_max;
+                    //dmsg(M_INFO, "FWD BAT INPT 1 [%d] [%d] [%d] [%d]
[%d]", maxl, fdno, BLEN(&c->c2.buf), BLEN(&c->c2.buffers->read_tun_buf),
BLEN(&c->c2.buffers->read_tun_max));
+                    encrypt_sign(c, true);
+                }
+            }
+            c->c2.buffers->bufs_indx = -1;
+            c->c2.buffers->flag_ciph = -1;
+        }
+    }
+    else
+    {
+        buf_reset(&c->c2.to_link);
+    }
+}
+
+void process_incoming_tun(struct context *c, struct link_socket *out_sock)
+{
+    if (c->c2.frame.bulk_size <= 0) {
+        process_incoming_tun_part2(c, out_sock);
+    }
+    process_incoming_tun_part3(c, out_sock);
+}
+
 /**
  * Forges a IPv6 ICMP packet with a no route to host error code from the
  * IPv6 packet in buf and sends it directly back to the client via the tun
@@ -1748,7 +1927,7 @@ process_outgoing_link(struct context *c, struct
link_socket *sock)

     perf_push(PERF_PROC_OUT_LINK);

-    if (c->c2.to_link.len > 0 && c->c2.to_link.len <=
c->c2.frame.buf.payload_size)
+    if (c->c2.to_link.len > 0 && (c->c2.to_link.len <=
c->c2.frame.buf.payload_size || c->c2.frame.bulk_size > 0))
     {
         /*
          * Setup for call to send/sendto which will send
@@ -1793,6 +1972,7 @@ process_outgoing_link(struct context *c, struct
link_socket *sock)
                 fprintf(stderr, "W");
             }
 #endif
+
             msg(D_LINK_RW, "%s WRITE [%d] to %s: %s",
                 proto2ascii(sock->info.proto, sock->info.af, true),
BLEN(&c->c2.to_link),
                 print_link_socket_actual(c->c2.to_link_addr, &gc),
PROTO_DUMP(&c->c2.to_link, &gc));
@@ -1892,7 +2072,7 @@ process_outgoing_link(struct context *c, struct
link_socket *sock)
  */

 void
-process_outgoing_tun(struct context *c, struct link_socket *in_sock)
+process_outgoing_tun_part2(struct context *c, struct link_socket *in_sock)
 {
     /*
      * Set up for write() call to TUN/TAP
@@ -1912,7 +2092,7 @@ process_outgoing_tun(struct context *c, struct
link_socket *in_sock)
     process_ip_header(c, PIP_MSSFIX | PIPV4_EXTRACT_DHCP_ROUTER |
PIPV4_CLIENT_NAT | PIP_OUTGOING,
                       &c->c2.to_tun, in_sock);

-    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size)
+    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size ||
c->c2.frame.bulk_size > 0)
     {
         /*
          * Write to TUN/TAP device.
@@ -1925,7 +2105,8 @@ process_outgoing_tun(struct context *c, struct
link_socket *in_sock)
             fprintf(stderr, "w");
         }
 #endif
-        dmsg(D_TUN_RW, "TUN WRITE [%d]", BLEN(&c->c2.to_tun));
+
+        dmsg(D_TUN_RW, "TUN WRITE [%d] [%d]", BLEN(&c->c2.to_tun),
c->c2.frame.buf.payload_size);

 #ifdef PACKET_TRUNCATION_CHECK
         ipv4_packet_size_verify(BPTR(&c->c2.to_tun), BLEN(&c->c2.to_tun),
TUNNEL_TYPE(c->c1.tuntap),
@@ -1981,6 +2162,39 @@ process_outgoing_tun(struct context *c, struct
link_socket *in_sock)
     perf_pop();
 }

+void process_outgoing_tun_part3(struct context *c, struct link_socket
*in_sock)
+{
+    if (check_bulk_mode(c))
+    {
+        int maxl = 0, plen = 0;
+        int leng = BLEN(&c->c2.buffers->send_tun_max);
+        uint8_t *temp = BPTR(&c->c2.buffers->send_tun_max);
+        for (int x = 0; x < TUN_BAT_MAX; ++x)
+        {
+            temp = buff_postsize(temp, &plen);
+            if ((leng > 0) && (plen > 0) && ((maxl + plen) < leng))
+            {
+                c->c2.to_tun = c->c2.buffers->to_tun_max;
+                c->c2.to_tun.offset = TUN_BAT_OFF;
+                c->c2.to_tun.len = plen;
+                bcopy(temp, BPTR(&c->c2.to_tun), plen);
+                temp += plen; maxl += (plen + 2);
+                //dmsg(M_INFO, "FWD BAT OUTP 1 [%d] [%d] [%d] [%d]", x,
BLEN(&c->c2.buf), BLEN(&c->c2.to_tun), BLEN(&c->c2.buffers->read_link_buf));
+                process_outgoing_tun_part2(c, in_sock);
+            } else { break; }
+        }
+    }
+    buf_reset(&c->c2.to_tun);
+}
+
+void process_outgoing_tun(struct context *c, struct link_socket *in_sock)
+{
+    if (c->c2.frame.bulk_size <= 0) {
+        process_outgoing_tun_part2(c, in_sock);
+    }
+    process_outgoing_tun_part3(c, in_sock);
+}
+
 void
 pre_select(struct context *c)
 {
diff --git a/src/openvpn/forward.h b/src/openvpn/forward.h
index d5641491..9fda1583 100644
--- a/src/openvpn/forward.h
+++ b/src/openvpn/forward.h
@@ -79,6 +79,8 @@ void pre_select(struct context *c);

 void process_io(struct context *c, struct link_socket *sock);

+void xfer_io(struct context *c, struct context *b);
+

 /**********************************************************************/
 /**
@@ -196,6 +198,8 @@ bool process_incoming_link_part1(struct context *c,
struct link_socket_info *lsi
 void process_incoming_link_part2(struct context *c, struct
link_socket_info *lsi,
                                  const uint8_t *orig_buf);

+void process_incoming_link_part3(struct context *c);
+
 /**
  * Transfers \c float_sa data extracted from an incoming DCO
  * PEER_FLOAT_NTF to \c out_osaddr for later processing.
diff --git a/src/openvpn/init.c b/src/openvpn/init.c
index 40ae2c8c..bbdbad46 100644
--- a/src/openvpn/init.c
+++ b/src/openvpn/init.c
@@ -2971,6 +2971,10 @@ frame_finalize_options(struct context *c, const
struct options *o)
     tailroom += COMP_EXTRA_BUFFER(payload_size);
 #endif

+    if (frame->bulk_size > 0) {
+        payload_size = frame->tun_mtu;
+    }
+
     frame->buf.payload_size = payload_size;
     frame->buf.headroom = headroom;
     frame->buf.tailroom = tailroom;
@@ -3473,6 +3477,9 @@ do_init_frame_tls(struct context *c)
     if (c->c2.tls_multi)
     {
         tls_multi_init_finalize(c->c2.tls_multi, c->options.ce.tls_mtu);
+        if (c->c2.frame.bulk_size > 0) {
+            c->c2.tls_multi->opt.frame.buf.payload_size =
c->c2.frame.tun_mtu;
+        }
         ASSERT(c->c2.tls_multi->opt.frame.buf.payload_size <=
c->c2.frame.buf.payload_size);
         frame_print(&c->c2.tls_multi->opt.frame, D_MTU_INFO, "Control
Channel MTU parms");

@@ -3536,6 +3543,14 @@ do_init_frame(struct context *c)
         c->c2.frame.extra_tun += c->options.ce.tun_mtu_extra;
     }

+    /*
+     * Adjust bulk size based on the --bulk-mode parameter.
+     */
+    if (c->options.ce.bulk_mode)
+    {
+        c->c2.frame.bulk_size = c->options.ce.tun_mtu;
+    }
+
     /*
      * Fill in the blanks in the frame parameters structure,
      * make sure values are rational, etc.
@@ -3676,9 +3691,41 @@ init_context_buffers(const struct frame *frame)

     size_t buf_size = BUF_SIZE(frame);

+    if (frame->bulk_size > 0) {
+        size_t off_size = (frame->buf.headroom + TUN_BAT_OFF +
frame->buf.tailroom);
+        buf_size = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, off_size);
+    }
+
+    dmsg(M_INFO, "MEM NEW [%ld] [%d+%d+%d]", buf_size,
frame->buf.headroom, frame->buf.payload_size, frame->buf.tailroom);
+
     b->read_link_buf = alloc_buf(buf_size);
     b->read_tun_buf = alloc_buf(buf_size);

+    if (frame->bulk_size > 0) {
+        for (int x = 0; x < TUN_BAT_MAX; ++x)
+        {
+            size_t part_size = BUF_SIZE(frame);
+            b->read_tun_bufs[x] = alloc_buf(part_size);
+            b->read_tun_bufs[x].offset = TUN_BAT_OFF;
+            b->read_tun_bufs[x].len = 0;
+        }
+
+        b->read_tun_max = alloc_buf(buf_size);
+        b->read_tun_max.offset = TUN_BAT_OFF;
+        b->read_tun_max.len = 0;
+
+        b->send_tun_max = alloc_buf(buf_size);
+        b->send_tun_max.offset = TUN_BAT_OFF;
+        b->send_tun_max.len = 0;
+
+        b->to_tun_max = alloc_buf(buf_size);
+        b->to_tun_max.offset = TUN_BAT_OFF;
+        b->to_tun_max.len = 0;
+    }
+
+    b->bufs_indx = -1;
+    b->flag_ciph = -1;
+
     b->aux_buf = alloc_buf(buf_size);

     b->encrypt_buf = alloc_buf(buf_size);
@@ -3701,6 +3748,16 @@ free_context_buffers(struct context_buffers *b)
         free_buf(&b->read_tun_buf);
         free_buf(&b->aux_buf);

+        if (b->to_tun_max.data) {
+            free_buf(&b->to_tun_max);
+            free_buf(&b->send_tun_max);
+            free_buf(&b->read_tun_max);
+            for (int x = 0; x < TUN_BAT_MAX; ++x)
+            {
+                free_buf(&b->read_tun_bufs[x]);
+            }
+        }
+
 #ifdef USE_COMP
         free_buf(&b->compress_buf);
         free_buf(&b->decompress_buf);
diff --git a/src/openvpn/mtu.c b/src/openvpn/mtu.c
index a419e32d..7e35c837 100644
--- a/src/openvpn/mtu.c
+++ b/src/openvpn/mtu.c
@@ -41,9 +41,15 @@ void
 alloc_buf_sock_tun(struct buffer *buf, const struct frame *frame)
 {
     /* allocate buffer for overlapped I/O */
-    *buf = alloc_buf(BUF_SIZE(frame));
+    size_t alen = BUF_SIZE(frame);
+    size_t blen = frame->buf.payload_size;
+    if (frame->bulk_size > 0) {
+        alen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_OFF);
+        blen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_NOP);
+    }
+    *buf = alloc_buf(alen);
     ASSERT(buf_init(buf, frame->buf.headroom));
-    buf->len = frame->buf.payload_size;
+    buf->len = blen;
     ASSERT(buf_safe(buf, 0));
 }

diff --git a/src/openvpn/mtu.h b/src/openvpn/mtu.h
index 925ef0bf..eb799fb3 100644
--- a/src/openvpn/mtu.h
+++ b/src/openvpn/mtu.h
@@ -58,6 +58,14 @@
  */
 #define TUN_MTU_MIN 100

+/*
+ * Bulk mode static define values.
+ */
+#define TUN_BAT_MIN        6
+#define TUN_BAT_MAX        9
+#define TUN_BAT_OFF        256
+#define TUN_BAT_NOP        0
+
 /*
  * Default MTU of network over which tunnel data will pass by TCP/UDP.
  */
@@ -152,6 +160,10 @@ struct frame
                             *   which defaults to 0 for tun and 32
                             *   (\c TAP_MTU_EXTRA_DEFAULT) for tap.
                             *   */
+
+    int bulk_size;              /**< Signal to the init frame function
+                                 *   to allow for bulk mode TCP transfers.
+                                 *   */
 };

 /* Forward declarations, to prevent includes */
@@ -171,6 +183,7 @@ struct options;
  * larger than the headroom.
  */
 #define BUF_SIZE(f) ((f)->buf.headroom + (f)->buf.payload_size +
(f)->buf.tailroom)
+#define BAT_SIZE(a, b, c) ((a * b) + c)

 /*
  * Function prototypes.
diff --git a/src/openvpn/multi.c b/src/openvpn/multi.c
index e1ce32ab..9e089703 100644
--- a/src/openvpn/multi.c
+++ b/src/openvpn/multi.c
@@ -3414,6 +3414,7 @@ multi_process_incoming_link(struct multi_context *m,
struct multi_instance *inst
                 }

                 process_incoming_link_part2(c, lsi, orig_buf);
+                process_incoming_link_part3(c);
             }
             perf_pop();

@@ -3558,9 +3559,7 @@ multi_process_incoming_tun(struct multi_context *m,
const unsigned int mpp_flags
         const int dev_type = TUNNEL_TYPE(m->top.c1.tuntap);
         int16_t vid = 0;

-#ifdef MULTI_DEBUG_EVENT_LOOP
-        printf("TUN -> TCP/UDP [%d]\n", BLEN(&m->top.c2.buf));
-#endif
+        msg(D_MULTI_DEBUG, "TUN -> TCP/UDP [%d]", BLEN(&m->top.c2.buf));

         if (m->pending)
         {
@@ -3610,6 +3609,8 @@ multi_process_incoming_tun(struct multi_context *m,
const unsigned int mpp_flags
                         {
                             /* transfer packet pointer from top-level
context buffer to instance */
                             c->c2.buf = m->top.c2.buf;
+                            /* todo determine if to call this
(multi_process_incoming_tun) for each bulk item read? */
+                            xfer_io(c, &m->top);
                         }
                         else
                         {
diff --git a/src/openvpn/openvpn.h b/src/openvpn/openvpn.h
index cd99cd40..21fa8967 100644
--- a/src/openvpn/openvpn.h
+++ b/src/openvpn/openvpn.h
@@ -112,6 +112,14 @@ struct context_buffers
      */
     struct buffer read_link_buf;
     struct buffer read_tun_buf;
+
+    struct buffer read_tun_bufs[TUN_BAT_MAX];
+    struct buffer read_tun_max;
+    struct buffer send_tun_max;
+    struct buffer to_tun_max;
+
+    int bufs_indx;
+    int flag_ciph;
 };

 /*
@@ -376,6 +384,8 @@ struct context_2
     struct buffer to_tun;
     struct buffer to_link;

+    struct buffer bufs[TUN_BAT_MAX];
+
     /* should we print R|W|r|w to console on packet transfers? */
     bool log_rw;

diff --git a/src/openvpn/options.c b/src/openvpn/options.c
index c54032d8..041d17d0 100644
--- a/src/openvpn/options.c
+++ b/src/openvpn/options.c
@@ -304,6 +304,7 @@ static const char usage_message[] =
     "                  'maybe' -- Use per-route hints\n"
     "                  'yes'   -- Always DF (Don't Fragment)\n"
     "--mtu-test      : Empirically measure and report MTU.\n"
+    "--bulk-mode     : Use bulk TUN/TCP reads/writes.\n"
 #ifdef ENABLE_FRAGMENT
     "--fragment max  : Enable internal datagram fragmentation so that no
UDP\n"
     "                  datagrams are sent which are larger than max
bytes.\n"
@@ -3005,6 +3006,9 @@ options_postprocess_mutate_ce(struct options *o,
struct connection_entry *ce)
             ce->tun_mtu_extra_defined = true;
             ce->tun_mtu_extra = TAP_MTU_EXTRA_DEFAULT;
         }
+        if (ce->proto != PROTO_TCP && ce->proto != PROTO_TCP_SERVER &&
ce->proto != PROTO_TCP_CLIENT) {
+            ce->bulk_mode = false;
+        }
     }

     /*
@@ -9926,6 +9930,10 @@ add_option(struct options *options, char *p[], bool
is_inline, const char *file,
             goto err;
         }
     }
+    else if (streq(p[0], "bulk-mode"))
+    {
+        options->ce.bulk_mode = true;
+    }
     else
     {
         int i;
diff --git a/src/openvpn/options.h b/src/openvpn/options.h
index 38e67c8d..d1b0586d 100644
--- a/src/openvpn/options.h
+++ b/src/openvpn/options.h
@@ -174,6 +174,9 @@ struct connection_entry

     /* Allow only client that support resending the wrapped client key */
     bool tls_crypt_v2_force_cookie;
+
+    /* Bulk mode allows for multiple tun reads + larger tcp writes */
+    bool bulk_mode;
 };

 struct remote_entry
--
2.39.5 (Apple Git-154)




On Thu, Aug 7, 2025 at 2:29 PM Jon Chiappetta <root@fossjon.com> wrote:

> Thanks to Gert's help on this, I was able to finally configure and compile
> and run and test the bulk mode changes against the latest git source code
> to ensure everything still works correctly.
>
> I also fixed up some other issues like properly freeing the extra buffer
> allocations and removing the unneeded batched data prefixes and converting
> a remaining while loop to a max limited for loop and properly resetting the
> outgoing tun buffer pointer at the end of the write method when finished.
>
> Thanks,
> Jon C
>
> Example updated pull request:
> https://github.com/OpenVPN/openvpn/pull/814/files
>
> git formatted diff patch:
>
> From 985e88a9af26a39554f113f37ee18032a2f41c3e Mon Sep 17 00:00:00 2001
> From: Jon Chiappetta <root@fossjon.com>
> Date: Wed, 6 Aug 2025 16:33:18 -0400
> Subject: [PATCH] bulk mode
>
> ---
>  src/openvpn/forward.c | 217 ++++++++++++++++++++++++++++++++++++++++--
>  src/openvpn/forward.h |   4 +
>  src/openvpn/init.c    |  56 +++++++++++
>  src/openvpn/mtu.c     |  10 +-
>  src/openvpn/mtu.h     |  13 +++
>  src/openvpn/multi.c   |   7 +-
>  src/openvpn/openvpn.h |  10 ++
>  src/openvpn/options.c |   8 ++
>  src/openvpn/options.h |   3 +
>  9 files changed, 316 insertions(+), 12 deletions(-)
>
> diff --git a/src/openvpn/forward.c b/src/openvpn/forward.c
> index 75ca9d5c..d9a98607 100644
> --- a/src/openvpn/forward.c
> +++ b/src/openvpn/forward.c
> @@ -46,6 +46,9 @@
>
>  #include "mstats.h"
>
> +#include <sys/select.h>
> +#include <sys/time.h>
> +
>  counter_type link_read_bytes_global;  /* GLOBAL */
>  counter_type link_write_bytes_global; /* GLOBAL */
>
> @@ -78,6 +81,32 @@ show_wait_status(struct context *c)
>
>  #endif /* ifdef ENABLE_DEBUG */
>
> +bool check_bulk_mode(struct context *c)
> +{
> +    if ((c->c2.frame.bulk_size > 0) && (c->c1.tuntap != NULL) &&
> (c->c2.buffers != NULL))
> +    {
> +        return true;
> +    }
> +    return false;
> +}
> +
> +void xfer_io(struct context *c, struct context *b)
> +{
> +    int plen = 0;
> +    if (check_bulk_mode(b))
> +    {
> +        int leng = (b->c2.buffers->bufs_indx + 1);
> +        for (int x = 0; x < leng; ++x)
> +        {
> +            plen = BLEN(&b->c2.bufs[x]);
> +            if (plen < 1) { c->c2.bufs[x].len = 0; }
> +            else { c->c2.bufs[x] = b->c2.bufs[x]; }
> +        }
> +        c->c2.buffers->bufs_indx = b->c2.buffers->bufs_indx;
> +        b->c2.buffers->bufs_indx = -1;
> +    }
> +}
> +
>  static void
>  check_tls_errors_co(struct context *c)
>  {
> @@ -605,6 +634,21 @@ buffer_turnover(const uint8_t *orig_buf, struct
> buffer *dest_stub, struct buffer
>      }
>  }
>
> +uint8_t *buff_prepsize(uint8_t *buff, int *size)
> +{
> +    buff[0] = ((*size >> 8) & 0xff);
> +    buff[1] = ((*size >> 0) & 0xff);
> +    buff += 2;
> +    return buff;
> +}
> +
> +uint8_t *buff_postsize(uint8_t *buff, int *size)
> +{
> +    *size = ((buff[0] << 8) + (buff[1] << 0));
> +    buff += 2;
> +    return buff;
> +}
> +
>  /*
>   * Compress, fragment, encrypt and HMAC-sign an outgoing packet.
>   * Input: c->c2.buf
> @@ -1031,6 +1075,7 @@ process_incoming_link_part1(struct context *c,
> struct link_socket_info *lsi, boo
>          fprintf(stderr, "R");
>      }
>  #endif
> +
>      msg(D_LINK_RW, "%s READ [%d] from %s: %s", proto2ascii(lsi->proto,
> lsi->af, true),
>          BLEN(&c->c2.buf), print_link_socket_actual(&c->c2.from, &gc),
> PROTO_DUMP(&c->c2.buf, &gc));
>
> @@ -1211,6 +1256,23 @@ process_incoming_link_part2(struct context *c,
> struct link_socket_info *lsi,
>      }
>  }
>
> +void process_incoming_link_part3(struct context *c)
> +{
> +    int leng = BLEN(&c->c2.to_tun);
> +    if (leng > 0)
> +    {
> +        if (check_bulk_mode(c))
> +        {
> +            c->c2.buffers->send_tun_max.offset = TUN_BAT_OFF;
> +            c->c2.buffers->send_tun_max.len = leng;
> +            bcopy(BPTR(&c->c2.to_tun),
> BPTR(&c->c2.buffers->send_tun_max), leng);
> +            //dmsg(M_INFO, "FWD BAT LINK 0 [%d] [%d] [%d] [%d] [%d]",
> BLEN(&c->c2.buf), BLEN(&c->c2.to_tun), BLEN(&c->c2.buffers->read_link_buf),
> BLEN(&c->c2.buffers->read_link_buf), BLEN(&c->c2.buffers->send_tun_max));
> +            c->c2.to_tun.offset += 2;
> +            c->c2.buf.offset += 2;
> +        }
> +    }
> +}
> +
>  static void
>  process_incoming_link(struct context *c, struct link_socket *sock)
>  {
> @@ -1221,6 +1283,7 @@ process_incoming_link(struct context *c, struct
> link_socket *sock)
>
>      process_incoming_link_part1(c, lsi, false);
>      process_incoming_link_part2(c, lsi, orig_buf);
> +    process_incoming_link_part3(c);
>
>      perf_pop();
>  }
> @@ -1321,7 +1384,7 @@ process_incoming_dco(struct context *c)
>   */
>
>  void
> -read_incoming_tun(struct context *c)
> +read_incoming_tun_part2(struct context *c)
>  {
>      /*
>       * Setup for read() call on TUN/TAP device.
> @@ -1382,6 +1445,55 @@ read_incoming_tun(struct context *c)
>      perf_pop();
>  }
>
> +void read_incoming_tun_part3(struct context *c)
> +{
> +    fd_set rfds;
> +    struct timeval timo;
> +    if (check_bulk_mode(c))
> +    {
> +        int plen = 0;
> +        int fdno = c->c1.tuntap->fd;
> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
> +        {
> +            int leng = plen;
> +            int indx = (c->c2.buffers->bufs_indx + 1);
> +            if (indx >= TUN_BAT_MIN) { break; }
> +            if (leng < 1)
> +            {
> +                FD_ZERO(&rfds);
> +                FD_SET(fdno, &rfds);
> +                timo.tv_sec = 0;
> +                timo.tv_usec = 0;
> +                select(fdno+1, &rfds, NULL, NULL, &timo);
> +                if (FD_ISSET(fdno, &rfds))
> +                {
> +                    read_incoming_tun_part2(c);
> +                    plen = BLEN(&c->c2.buf);
> +                } else { break; }
> +            }
> +            //dmsg(M_INFO, "FWD BAT READ 0 [%d] [%d] [%d] [%d] [%d]",
> c->c2.buffers->bufs_indx + 1, fdno, BLEN(&c->c2.buf),
> BLEN(&c->c2.buffers->read_tun_buf), BLEN(&c->c2.buffers->read_tun_max));
> +            leng = plen;
> +            if (leng > 0)
> +            {
> +                c->c2.buffers->read_tun_bufs[indx].offset = TUN_BAT_OFF;
> +                c->c2.buffers->read_tun_bufs[indx].len = leng;
> +                bcopy(BPTR(&c->c2.buf),
> BPTR(&c->c2.buffers->read_tun_bufs[indx]), leng);
> +                c->c2.bufs[indx] = c->c2.buffers->read_tun_bufs[indx];
> +                c->c2.buffers->bufs_indx = indx;
> +            } else { break; }
> +            plen = 0;
> +        }
> +    }
> +}
> +
> +void read_incoming_tun(struct context *c)
> +{
> +    if (c->c2.frame.bulk_size <= 0) {
> +        read_incoming_tun_part2(c);
> +    }
> +    read_incoming_tun_part3(c);
> +}
> +
>  /**
>   * Drops UDP packets which OS decided to route via tun.
>   *
> @@ -1469,7 +1581,7 @@ drop_if_recursive_routing(struct context *c, struct
> buffer *buf)
>   */
>
>  void
> -process_incoming_tun(struct context *c, struct link_socket *out_sock)
> +process_incoming_tun_part2(struct context *c, struct link_socket
> *out_sock)
>  {
>      struct gc_arena gc = gc_new();
>
> @@ -1488,7 +1600,7 @@ process_incoming_tun(struct context *c, struct
> link_socket *out_sock)
>  #endif
>
>      /* Show packet content */
> -    dmsg(D_TUN_RW, "TUN READ [%d]", BLEN(&c->c2.buf));
> +    dmsg(D_TUN_RW, "TUN READ [%d] [%d]", BLEN(&c->c2.buf),
> c->c2.frame.buf.payload_size);
>
>      if (c->c2.buf.len > 0)
>      {
> @@ -1512,7 +1624,9 @@ process_incoming_tun(struct context *c, struct
> link_socket *out_sock)
>      }
>      if (c->c2.buf.len > 0)
>      {
> +        if ((c->c2.buffers == NULL) || (c->c2.buffers->flag_ciph != -2)) {
>          encrypt_sign(c, true);
> +        }
>      }
>      else
>      {
> @@ -1522,6 +1636,60 @@ process_incoming_tun(struct context *c, struct
> link_socket *out_sock)
>      gc_free(&gc);
>  }
>
> +void process_incoming_tun_part3(struct context *c, struct link_socket
> *out_sock)
> +{
> +    if (check_bulk_mode(c))
> +    {
> +        c->c2.buffers->flag_ciph = -2;
> +        c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
> +        c->c2.buffers->read_tun_max.len = 0;
> +        uint8_t *temp = BPTR(&c->c2.buffers->read_tun_max);
> +        int plen = 0, fdno = c->c1.tuntap->fd;
> +        int maxl = 0, leng = (c->c2.buffers->bufs_indx + 1);
> +        if ((fdno > 0) && (leng > 0))
> +        {
> +            for (int x = 0; x < leng; ++x)
> +            {
> +                c->c2.buf = c->c2.bufs[x];
> +                //dmsg(M_INFO, "FWD BAT INPT 0 [%d] [%d] [%d] [%d] [%d]",
> x, fdno, BLEN(&c->c2.buf), BLEN(&c->c2.buffers->read_tun_buf),
> BLEN(&c->c2.bufs[x]));
> +                process_incoming_tun_part2(c, out_sock);
> +                if (BLEN(&c->c2.buf) < 1)
> +                {
> +                    c->c2.bufs[x].len = 0;
> +                }
> +            }
> +            for (int x = 0; x < leng; ++x)
> +            {
> +                plen = c->c2.bufs[x].len;
> +                if (plen > 0)
> +                {
> +                    temp = buff_prepsize(temp, &plen);
> +                    bcopy(BPTR(&c->c2.bufs[x]), temp, plen);
> +                    temp += plen; maxl += (plen + 2);
> +                }
> +            }
> +            if (maxl > 0)
> +            {
> +                c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
> +                c->c2.buffers->read_tun_max.len = maxl;
> +                c->c2.buf = c->c2.buffers->read_tun_max;
> +                //dmsg(M_INFO, "FWD BAT INPT 1 [%d] [%d] [%d] [%d] [%d]",
> maxl, fdno, BLEN(&c->c2.buf), BLEN(&c->c2.buffers->read_tun_buf),
> BLEN(&c->c2.buffers->read_tun_max));
> +                encrypt_sign(c, true);
> +            }
> +        }
> +        c->c2.buffers->bufs_indx = -1;
> +        c->c2.buffers->flag_ciph = -1;
> +    }
> +}
> +
> +void process_incoming_tun(struct context *c, struct link_socket *out_sock)
> +{
> +    if (c->c2.frame.bulk_size <= 0) {
> +        process_incoming_tun_part2(c, out_sock);
> +    }
> +    process_incoming_tun_part3(c, out_sock);
> +}
> +
>  /**
>   * Forges a IPv6 ICMP packet with a no route to host error code from the
>   * IPv6 packet in buf and sends it directly back to the client via the tun
> @@ -1748,7 +1916,7 @@ process_outgoing_link(struct context *c, struct
> link_socket *sock)
>
>      perf_push(PERF_PROC_OUT_LINK);
>
> -    if (c->c2.to_link.len > 0 && c->c2.to_link.len <=
> c->c2.frame.buf.payload_size)
> +    if (c->c2.to_link.len > 0 && (c->c2.to_link.len <=
> c->c2.frame.buf.payload_size || c->c2.frame.bulk_size > 0))
>      {
>          /*
>           * Setup for call to send/sendto which will send
> @@ -1793,6 +1961,7 @@ process_outgoing_link(struct context *c, struct
> link_socket *sock)
>                  fprintf(stderr, "W");
>              }
>  #endif
> +
>              msg(D_LINK_RW, "%s WRITE [%d] to %s: %s",
>                  proto2ascii(sock->info.proto, sock->info.af, true),
> BLEN(&c->c2.to_link),
>                  print_link_socket_actual(c->c2.to_link_addr, &gc),
> PROTO_DUMP(&c->c2.to_link, &gc));
> @@ -1892,7 +2061,7 @@ process_outgoing_link(struct context *c, struct
> link_socket *sock)
>   */
>
>  void
> -process_outgoing_tun(struct context *c, struct link_socket *in_sock)
> +process_outgoing_tun_part2(struct context *c, struct link_socket *in_sock)
>  {
>      /*
>       * Set up for write() call to TUN/TAP
> @@ -1912,7 +2081,7 @@ process_outgoing_tun(struct context *c, struct
> link_socket *in_sock)
>      process_ip_header(c, PIP_MSSFIX | PIPV4_EXTRACT_DHCP_ROUTER |
> PIPV4_CLIENT_NAT | PIP_OUTGOING,
>                        &c->c2.to_tun, in_sock);
>
> -    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size)
> +    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size ||
> c->c2.frame.bulk_size > 0)
>      {
>          /*
>           * Write to TUN/TAP device.
> @@ -1925,7 +2094,8 @@ process_outgoing_tun(struct context *c, struct
> link_socket *in_sock)
>              fprintf(stderr, "w");
>          }
>  #endif
> -        dmsg(D_TUN_RW, "TUN WRITE [%d]", BLEN(&c->c2.to_tun));
> +
> +        dmsg(D_TUN_RW, "TUN WRITE [%d] [%d]", BLEN(&c->c2.to_tun),
> c->c2.frame.buf.payload_size);
>
>  #ifdef PACKET_TRUNCATION_CHECK
>          ipv4_packet_size_verify(BPTR(&c->c2.to_tun), BLEN(&c->c2.to_tun),
> TUNNEL_TYPE(c->c1.tuntap),
> @@ -1981,6 +2151,39 @@ process_outgoing_tun(struct context *c, struct
> link_socket *in_sock)
>      perf_pop();
>  }
>
> +void process_outgoing_tun_part3(struct context *c, struct link_socket
> *in_sock)
> +{
> +    if (check_bulk_mode(c))
> +    {
> +        int maxl = 0, plen = 0;
> +        int leng = BLEN(&c->c2.buffers->send_tun_max);
> +        uint8_t *temp = BPTR(&c->c2.buffers->send_tun_max);
> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
> +        {
> +            temp = buff_postsize(temp, &plen);
> +            if ((leng > 0) && (plen > 0) && ((maxl + plen) < leng))
> +            {
> +                c->c2.to_tun = c->c2.buffers->to_tun_max;
> +                c->c2.to_tun.offset = TUN_BAT_OFF;
> +                c->c2.to_tun.len = plen;
> +                bcopy(temp, BPTR(&c->c2.to_tun), plen);
> +                temp += plen; maxl += (plen + 2);
> +                //dmsg(M_INFO, "FWD BAT OUTP 1 [%d] [%d] [%d] [%d]", x,
> BLEN(&c->c2.buf), BLEN(&c->c2.to_tun), BLEN(&c->c2.buffers->read_link_buf));
> +                process_outgoing_tun_part2(c, in_sock);
> +            } else { break; }
> +        }
> +        buf_reset(&c->c2.to_tun);
> +    }
> +}
> +
> +void process_outgoing_tun(struct context *c, struct link_socket *in_sock)
> +{
> +    if (c->c2.frame.bulk_size <= 0) {
> +        process_outgoing_tun_part2(c, in_sock);
> +    }
> +    process_outgoing_tun_part3(c, in_sock);
> +}
> +
>  void
>  pre_select(struct context *c)
>  {
> diff --git a/src/openvpn/forward.h b/src/openvpn/forward.h
> index d5641491..9fda1583 100644
> --- a/src/openvpn/forward.h
> +++ b/src/openvpn/forward.h
> @@ -79,6 +79,8 @@ void pre_select(struct context *c);
>
>  void process_io(struct context *c, struct link_socket *sock);
>
> +void xfer_io(struct context *c, struct context *b);
> +
>
>  /**********************************************************************/
>  /**
> @@ -196,6 +198,8 @@ bool process_incoming_link_part1(struct context *c,
> struct link_socket_info *lsi
>  void process_incoming_link_part2(struct context *c, struct
> link_socket_info *lsi,
>                                   const uint8_t *orig_buf);
>
> +void process_incoming_link_part3(struct context *c);
> +
>  /**
>   * Transfers \c float_sa data extracted from an incoming DCO
>   * PEER_FLOAT_NTF to \c out_osaddr for later processing.
> diff --git a/src/openvpn/init.c b/src/openvpn/init.c
> index 40ae2c8c..0849dfce 100644
> --- a/src/openvpn/init.c
> +++ b/src/openvpn/init.c
> @@ -2971,6 +2971,10 @@ frame_finalize_options(struct context *c, const
> struct options *o)
>      tailroom += COMP_EXTRA_BUFFER(payload_size);
>  #endif
>
> +    if (frame->bulk_size > 0) {
> +        payload_size = frame->tun_mtu;
> +    }
> +
>      frame->buf.payload_size = payload_size;
>      frame->buf.headroom = headroom;
>      frame->buf.tailroom = tailroom;
> @@ -3473,6 +3477,9 @@ do_init_frame_tls(struct context *c)
>      if (c->c2.tls_multi)
>      {
>          tls_multi_init_finalize(c->c2.tls_multi, c->options.ce.tls_mtu);
> +        if (c->c2.frame.bulk_size > 0) {
> +            c->c2.tls_multi->opt.frame.buf.payload_size =
> c->c2.frame.tun_mtu;
> +        }
>          ASSERT(c->c2.tls_multi->opt.frame.buf.payload_size <=
> c->c2.frame.buf.payload_size);
>          frame_print(&c->c2.tls_multi->opt.frame, D_MTU_INFO, "Control
> Channel MTU parms");
>
> @@ -3536,6 +3543,14 @@ do_init_frame(struct context *c)
>          c->c2.frame.extra_tun += c->options.ce.tun_mtu_extra;
>      }
>
> +    /*
> +     * Adjust bulk size based on the --bulk-mode parameter.
> +     */
> +    if (c->options.ce.bulk_mode)
> +    {
> +        c->c2.frame.bulk_size = c->options.ce.tun_mtu;
> +    }
> +
>      /*
>       * Fill in the blanks in the frame parameters structure,
>       * make sure values are rational, etc.
> @@ -3676,9 +3691,40 @@ init_context_buffers(const struct frame *frame)
>
>      size_t buf_size = BUF_SIZE(frame);
>
> +    if (frame->bulk_size > 0) {
> +        buf_size = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu,
> frame->buf.headroom + frame->buf.tailroom);
> +    }
> +
> +    dmsg(M_INFO, "MEM NEW [%ld] [%d+%d+%d]", buf_size,
> frame->buf.headroom, frame->buf.payload_size, frame->buf.tailroom);
> +
>      b->read_link_buf = alloc_buf(buf_size);
>      b->read_tun_buf = alloc_buf(buf_size);
>
> +    if (frame->bulk_size > 0) {
> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
> +        {
> +            size_t part_size = BUF_SIZE(frame);
> +            b->read_tun_bufs[x] = alloc_buf(part_size);
> +            b->read_tun_bufs[x].offset = TUN_BAT_OFF;
> +            b->read_tun_bufs[x].len = 0;
> +        }
> +
> +        b->read_tun_max = alloc_buf(buf_size);
> +        b->read_tun_max.offset = TUN_BAT_OFF;
> +        b->read_tun_max.len = 0;
> +
> +        b->send_tun_max = alloc_buf(buf_size);
> +        b->send_tun_max.offset = TUN_BAT_OFF;
> +        b->send_tun_max.len = 0;
> +
> +        b->to_tun_max = alloc_buf(buf_size);
> +        b->to_tun_max.offset = TUN_BAT_OFF;
> +        b->to_tun_max.len = 0;
> +    }
> +
> +    b->bufs_indx = -1;
> +    b->flag_ciph = -1;
> +
>      b->aux_buf = alloc_buf(buf_size);
>
>      b->encrypt_buf = alloc_buf(buf_size);
> @@ -3701,6 +3747,16 @@ free_context_buffers(struct context_buffers *b)
>          free_buf(&b->read_tun_buf);
>          free_buf(&b->aux_buf);
>
> +        if (b->to_tun_max.data) {
> +            free_buf(&b->to_tun_max);
> +            free_buf(&b->send_tun_max);
> +            free_buf(&b->read_tun_max);
> +            for (int x = 0; x < TUN_BAT_MAX; ++x)
> +            {
> +                free_buf(&b->read_tun_bufs[x]);
> +            }
> +        }
> +
>  #ifdef USE_COMP
>          free_buf(&b->compress_buf);
>          free_buf(&b->decompress_buf);
> diff --git a/src/openvpn/mtu.c b/src/openvpn/mtu.c
> index a419e32d..7e35c837 100644
> --- a/src/openvpn/mtu.c
> +++ b/src/openvpn/mtu.c
> @@ -41,9 +41,15 @@ void
>  alloc_buf_sock_tun(struct buffer *buf, const struct frame *frame)
>  {
>      /* allocate buffer for overlapped I/O */
> -    *buf = alloc_buf(BUF_SIZE(frame));
> +    size_t alen = BUF_SIZE(frame);
> +    size_t blen = frame->buf.payload_size;
> +    if (frame->bulk_size > 0) {
> +        alen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_OFF);
> +        blen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_NOP);
> +    }
> +    *buf = alloc_buf(alen);
>      ASSERT(buf_init(buf, frame->buf.headroom));
> -    buf->len = frame->buf.payload_size;
> +    buf->len = blen;
>      ASSERT(buf_safe(buf, 0));
>  }
>
> diff --git a/src/openvpn/mtu.h b/src/openvpn/mtu.h
> index 925ef0bf..eb799fb3 100644
> --- a/src/openvpn/mtu.h
> +++ b/src/openvpn/mtu.h
> @@ -58,6 +58,14 @@
>   */
>  #define TUN_MTU_MIN 100
>
> +/*
> + * Bulk mode static define values.
> + */
> +#define TUN_BAT_MIN        6
> +#define TUN_BAT_MAX        9
> +#define TUN_BAT_OFF        256
> +#define TUN_BAT_NOP        0
> +
>  /*
>   * Default MTU of network over which tunnel data will pass by TCP/UDP.
>   */
> @@ -152,6 +160,10 @@ struct frame
>                              *   which defaults to 0 for tun and 32
>                              *   (\c TAP_MTU_EXTRA_DEFAULT) for tap.
>                              *   */
> +
> +    int bulk_size;              /**< Signal to the init frame function
> +                                 *   to allow for bulk mode TCP transfers.
> +                                 *   */
>  };
>
>  /* Forward declarations, to prevent includes */
> @@ -171,6 +183,7 @@ struct options;
>   * larger than the headroom.
>   */
>  #define BUF_SIZE(f) ((f)->buf.headroom + (f)->buf.payload_size +
> (f)->buf.tailroom)
> +#define BAT_SIZE(a, b, c) ((a * b) + c)
>
>  /*
>   * Function prototypes.
> diff --git a/src/openvpn/multi.c b/src/openvpn/multi.c
> index e1ce32ab..9e089703 100644
> --- a/src/openvpn/multi.c
> +++ b/src/openvpn/multi.c
> @@ -3414,6 +3414,7 @@ multi_process_incoming_link(struct multi_context *m,
> struct multi_instance *inst
>                  }
>
>                  process_incoming_link_part2(c, lsi, orig_buf);
> +                process_incoming_link_part3(c);
>              }
>              perf_pop();
>
> @@ -3558,9 +3559,7 @@ multi_process_incoming_tun(struct multi_context *m,
> const unsigned int mpp_flags
>          const int dev_type = TUNNEL_TYPE(m->top.c1.tuntap);
>          int16_t vid = 0;
>
> -#ifdef MULTI_DEBUG_EVENT_LOOP
> -        printf("TUN -> TCP/UDP [%d]\n", BLEN(&m->top.c2.buf));
> -#endif
> +        msg(D_MULTI_DEBUG, "TUN -> TCP/UDP [%d]", BLEN(&m->top.c2.buf));
>
>          if (m->pending)
>          {
> @@ -3610,6 +3609,8 @@ multi_process_incoming_tun(struct multi_context *m,
> const unsigned int mpp_flags
>                          {
>                              /* transfer packet pointer from top-level
> context buffer to instance */
>                              c->c2.buf = m->top.c2.buf;
> +                            /* todo determine if to call this
> (multi_process_incoming_tun) for each bulk item read? */
> +                            xfer_io(c, &m->top);
>                          }
>                          else
>                          {
> diff --git a/src/openvpn/openvpn.h b/src/openvpn/openvpn.h
> index cd99cd40..21fa8967 100644
> --- a/src/openvpn/openvpn.h
> +++ b/src/openvpn/openvpn.h
> @@ -112,6 +112,14 @@ struct context_buffers
>       */
>      struct buffer read_link_buf;
>      struct buffer read_tun_buf;
> +
> +    struct buffer read_tun_bufs[TUN_BAT_MAX];
> +    struct buffer read_tun_max;
> +    struct buffer send_tun_max;
> +    struct buffer to_tun_max;
> +
> +    int bufs_indx;
> +    int flag_ciph;
>  };
>
>  /*
> @@ -376,6 +384,8 @@ struct context_2
>      struct buffer to_tun;
>      struct buffer to_link;
>
> +    struct buffer bufs[TUN_BAT_MAX];
> +
>      /* should we print R|W|r|w to console on packet transfers? */
>      bool log_rw;
>
> diff --git a/src/openvpn/options.c b/src/openvpn/options.c
> index c54032d8..041d17d0 100644
> --- a/src/openvpn/options.c
> +++ b/src/openvpn/options.c
> @@ -304,6 +304,7 @@ static const char usage_message[] =
>      "                  'maybe' -- Use per-route hints\n"
>      "                  'yes'   -- Always DF (Don't Fragment)\n"
>      "--mtu-test      : Empirically measure and report MTU.\n"
> +    "--bulk-mode     : Use bulk TUN/TCP reads/writes.\n"
>  #ifdef ENABLE_FRAGMENT
>      "--fragment max  : Enable internal datagram fragmentation so that no
> UDP\n"
>      "                  datagrams are sent which are larger than max
> bytes.\n"
> @@ -3005,6 +3006,9 @@ options_postprocess_mutate_ce(struct options *o,
> struct connection_entry *ce)
>              ce->tun_mtu_extra_defined = true;
>              ce->tun_mtu_extra = TAP_MTU_EXTRA_DEFAULT;
>          }
> +        if (ce->proto != PROTO_TCP && ce->proto != PROTO_TCP_SERVER &&
> ce->proto != PROTO_TCP_CLIENT) {
> +            ce->bulk_mode = false;
> +        }
>      }
>
>      /*
> @@ -9926,6 +9930,10 @@ add_option(struct options *options, char *p[], bool
> is_inline, const char *file,
>              goto err;
>          }
>      }
> +    else if (streq(p[0], "bulk-mode"))
> +    {
> +        options->ce.bulk_mode = true;
> +    }
>      else
>      {
>          int i;
> diff --git a/src/openvpn/options.h b/src/openvpn/options.h
> index 38e67c8d..d1b0586d 100644
> --- a/src/openvpn/options.h
> +++ b/src/openvpn/options.h
> @@ -174,6 +174,9 @@ struct connection_entry
>
>      /* Allow only client that support resending the wrapped client key */
>      bool tls_crypt_v2_force_cookie;
> +
> +    /* Bulk mode allows for multiple tun reads + larger tcp writes */
> +    bool bulk_mode;
>  };
>
>  struct remote_entry
> --
> 2.39.5 (Apple Git-154)
>
>

[one last self reply]

I did some work today to fix a potential issue with my read tun method
where it may not read from the file descriptor upon reconnection. I believe
it should be fixed now just in case anyone is still interested in a change
like this.

Example POC Pull Request Link:
https://github.com/OpenVPN/openvpn/pull/814/files

$ cat 0001-bulk-mode.patch
From 28d43ff49984f2d3c787fa791d0a7a548ac80fad Mon Sep 17 00:00:00 2001
From: Jon Chiappetta <root@fossjon.com>
Date: Wed, 6 Aug 2025 16:33:18 -0400
Subject: [PATCH] bulk mode

---
 src/openvpn/forward.c | 225 ++++++++++++++++++++++++++++++++++++++++--
 src/openvpn/forward.h |   4 +
 src/openvpn/init.c    |  57 +++++++++++
 src/openvpn/mtu.c     |  10 +-
 src/openvpn/mtu.h     |  13 +++
 src/openvpn/multi.c   |   7 +-
 src/openvpn/openvpn.h |  10 ++
 src/openvpn/options.c |   8 ++
 src/openvpn/options.h |   3 +
 9 files changed, 325 insertions(+), 12 deletions(-)

diff --git a/src/openvpn/forward.c b/src/openvpn/forward.c
index 75ca9d5c..b9d96482 100644
--- a/src/openvpn/forward.c
+++ b/src/openvpn/forward.c
@@ -46,6 +46,9 @@

 #include "mstats.h"

+#include <sys/select.h>
+#include <sys/time.h>
+
 counter_type link_read_bytes_global;  /* GLOBAL */
 counter_type link_write_bytes_global; /* GLOBAL */

@@ -78,6 +81,34 @@ show_wait_status(struct context *c)

 #endif /* ifdef ENABLE_DEBUG */

+bool check_bulk_mode(struct context *c)
+{
+    if ((c->c2.frame.bulk_size > 0) && (c->c1.tuntap != NULL) &&
(c->c2.buffers != NULL))
+    {
+        return true;
+    }
+    return false;
+}
+
+void xfer_io(struct context *c, struct context *b)
+{
+    //dmsg(M_INFO, "BULK MODE xfer_io c [%d] [%p] [%p] [%d] [%d] [%d]",
c->c2.frame.bulk_size, c->c1.tuntap, c->c2.buffers, BLEN(&c->c2.buf),
BLEN(&c->c2.to_tun), BLEN(&c->c2.to_link));
+    //dmsg(M_INFO, "BULK MODE xfer_io b [%d] [%p] [%p] [%d] [%d] [%d]",
b->c2.frame.bulk_size, b->c1.tuntap, b->c2.buffers, BLEN(&b->c2.buf),
BLEN(&b->c2.to_tun), BLEN(&b->c2.to_link));
+    int plen = 0;
+    if (check_bulk_mode(b))
+    {
+        int leng = (b->c2.buffers->bufs_indx + 1);
+        for (int x = 0; x < leng; ++x)
+        {
+            plen = BLEN(&b->c2.bufs[x]);
+            if (plen < 1) { c->c2.bufs[x].len = 0; }
+            else { c->c2.bufs[x] = b->c2.bufs[x]; }
+        }
+        c->c2.buffers->bufs_indx = b->c2.buffers->bufs_indx;
+        b->c2.buffers->bufs_indx = -1;
+    }
+}
+
 static void
 check_tls_errors_co(struct context *c)
 {
@@ -605,6 +636,21 @@ buffer_turnover(const uint8_t *orig_buf, struct buffer
*dest_stub, struct buffer
     }
 }

+uint8_t *buff_prepsize(uint8_t *buff, int *size)
+{
+    buff[0] = ((*size >> 8) & 0xff);
+    buff[1] = ((*size >> 0) & 0xff);
+    buff += 2;
+    return buff;
+}
+
+uint8_t *buff_postsize(uint8_t *buff, int *size)
+{
+    *size = ((buff[0] << 8) + (buff[1] << 0));
+    buff += 2;
+    return buff;
+}
+
 /*
  * Compress, fragment, encrypt and HMAC-sign an outgoing packet.
  * Input: c->c2.buf
@@ -1031,6 +1077,7 @@ process_incoming_link_part1(struct context *c, struct
link_socket_info *lsi, boo
         fprintf(stderr, "R");
     }
 #endif
+
     msg(D_LINK_RW, "%s READ [%d] from %s: %s", proto2ascii(lsi->proto,
lsi->af, true),
         BLEN(&c->c2.buf), print_link_socket_actual(&c->c2.from, &gc),
PROTO_DUMP(&c->c2.buf, &gc));

@@ -1211,6 +1258,26 @@ process_incoming_link_part2(struct context *c,
struct link_socket_info *lsi,
     }
 }

+void process_incoming_link_part3(struct context *c)
+{
+    int leng = BLEN(&c->c2.buf);
+    if (leng > 0)
+    {
+        if (check_bulk_mode(c))
+        {
+            c->c2.buffers->send_tun_max.offset = TUN_BAT_OFF;
+            c->c2.buffers->send_tun_max.len = leng;
+            bcopy(BPTR(&c->c2.buf), BPTR(&c->c2.buffers->send_tun_max),
leng);
+            c->c2.to_tun.offset += 2;
+            c->c2.buf.offset += 2;
+        }
+    }
+    else
+    {
+        buf_reset(&c->c2.to_tun);
+    }
+}
+
 static void
 process_incoming_link(struct context *c, struct link_socket *sock)
 {
@@ -1221,6 +1288,7 @@ process_incoming_link(struct context *c, struct
link_socket *sock)

     process_incoming_link_part1(c, lsi, false);
     process_incoming_link_part2(c, lsi, orig_buf);
+    process_incoming_link_part3(c);

     perf_pop();
 }
@@ -1321,7 +1389,7 @@ process_incoming_dco(struct context *c)
  */

 void
-read_incoming_tun(struct context *c)
+read_incoming_tun_part2(struct context *c)
 {
     /*
      * Setup for read() call on TUN/TAP device.
@@ -1382,6 +1450,54 @@ read_incoming_tun(struct context *c)
     perf_pop();
 }

+void read_incoming_tun_part3(struct context *c)
+{
+    fd_set rfds;
+    struct timeval timo;
+    if (check_bulk_mode(c))
+    {
+        int plen = 0, pidx = -1;
+        int fdno = c->c1.tuntap->fd;
+        for (int x = 0; x < TUN_BAT_MAX; ++x)
+        {
+            int leng = plen, indx = (pidx + 1);
+            if (indx >= TUN_BAT_MIN) { break; }
+            if (leng < 1)
+            {
+                FD_ZERO(&rfds);
+                FD_SET(fdno, &rfds);
+                timo.tv_sec = 0;
+                timo.tv_usec = 0;
+                select(fdno+1, &rfds, NULL, NULL, &timo);
+                if (FD_ISSET(fdno, &rfds))
+                {
+                    read_incoming_tun_part2(c);
+                    plen = BLEN(&c->c2.buf);
+                } else { break; }
+            }
+            leng = plen;
+            if (leng > 0)
+            {
+                c->c2.buffers->read_tun_bufs[indx].offset = TUN_BAT_OFF;
+                c->c2.buffers->read_tun_bufs[indx].len = leng;
+                bcopy(BPTR(&c->c2.buf),
BPTR(&c->c2.buffers->read_tun_bufs[indx]), leng);
+                c->c2.bufs[indx] = c->c2.buffers->read_tun_bufs[indx];
+                pidx = indx;
+            } else { break; }
+            plen = 0;
+        }
+        c->c2.buffers->bufs_indx = pidx;
+    }
+}
+
+void read_incoming_tun(struct context *c)
+{
+    if (c->c2.frame.bulk_size <= 0) {
+        read_incoming_tun_part2(c);
+    }
+    read_incoming_tun_part3(c);
+}
+
 /**
  * Drops UDP packets which OS decided to route via tun.
  *
@@ -1469,7 +1585,7 @@ drop_if_recursive_routing(struct context *c, struct
buffer *buf)
  */

 void
-process_incoming_tun(struct context *c, struct link_socket *out_sock)
+process_incoming_tun_part2(struct context *c, struct link_socket *out_sock)
 {
     struct gc_arena gc = gc_new();

@@ -1488,7 +1604,7 @@ process_incoming_tun(struct context *c, struct
link_socket *out_sock)
 #endif

     /* Show packet content */
-    dmsg(D_TUN_RW, "TUN READ [%d]", BLEN(&c->c2.buf));
+    dmsg(D_TUN_RW, "TUN READ [%d] [%d]", BLEN(&c->c2.buf),
c->c2.frame.buf.payload_size);

     if (c->c2.buf.len > 0)
     {
@@ -1512,7 +1628,9 @@ process_incoming_tun(struct context *c, struct
link_socket *out_sock)
     }
     if (c->c2.buf.len > 0)
     {
+        if ((c->c2.buffers == NULL) || (c->c2.buffers->flag_ciph != -2)) {
         encrypt_sign(c, true);
+        }
     }
     else
     {
@@ -1522,6 +1640,65 @@ process_incoming_tun(struct context *c, struct
link_socket *out_sock)
     gc_free(&gc);
 }

+void process_incoming_tun_part3(struct context *c, struct link_socket
*out_sock)
+{
+    if (c->c2.buf.len > 0)
+    {
+        if (check_bulk_mode(c))
+        {
+            c->c2.buffers->flag_ciph = -2;
+            c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
+            c->c2.buffers->read_tun_max.len = 0;
+            uint8_t *temp = BPTR(&c->c2.buffers->read_tun_max);
+            int plen = 0, fdno = c->c1.tuntap->fd;
+            int maxl = 0, leng = (c->c2.buffers->bufs_indx + 1);
+            if ((fdno > 0) && (leng > 0))
+            {
+                for (int x = 0; x < leng; ++x)
+                {
+                    c->c2.buf = c->c2.bufs[x];
+                    process_incoming_tun_part2(c, out_sock);
+                    if (BLEN(&c->c2.buf) < 1)
+                    {
+                        c->c2.bufs[x].len = 0;
+                    }
+                }
+                for (int x = 0; x < leng; ++x)
+                {
+                    plen = c->c2.bufs[x].len;
+                    if (plen > 0)
+                    {
+                        temp = buff_prepsize(temp, &plen);
+                        bcopy(BPTR(&c->c2.bufs[x]), temp, plen);
+                        temp += plen; maxl += (plen + 2);
+                    }
+                }
+                if (maxl > 0)
+                {
+                    c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
+                    c->c2.buffers->read_tun_max.len = maxl;
+                    c->c2.buf = c->c2.buffers->read_tun_max;
+                    encrypt_sign(c, true);
+                }
+            }
+            c->c2.buffers->bufs_indx = -1;
+            c->c2.buffers->flag_ciph = -1;
+        }
+    }
+    else
+    {
+        buf_reset(&c->c2.to_link);
+    }
+}
+
+void process_incoming_tun(struct context *c, struct link_socket *out_sock)
+{
+    if (c->c2.frame.bulk_size <= 0) {
+        process_incoming_tun_part2(c, out_sock);
+    }
+    process_incoming_tun_part3(c, out_sock);
+}
+
 /**
  * Forges a IPv6 ICMP packet with a no route to host error code from the
  * IPv6 packet in buf and sends it directly back to the client via the tun
@@ -1748,7 +1925,7 @@ process_outgoing_link(struct context *c, struct
link_socket *sock)

     perf_push(PERF_PROC_OUT_LINK);

-    if (c->c2.to_link.len > 0 && c->c2.to_link.len <=
c->c2.frame.buf.payload_size)
+    if (c->c2.to_link.len > 0 && (c->c2.to_link.len <=
c->c2.frame.buf.payload_size || c->c2.frame.bulk_size > 0))
     {
         /*
          * Setup for call to send/sendto which will send
@@ -1793,6 +1970,7 @@ process_outgoing_link(struct context *c, struct
link_socket *sock)
                 fprintf(stderr, "W");
             }
 #endif
+
             msg(D_LINK_RW, "%s WRITE [%d] to %s: %s",
                 proto2ascii(sock->info.proto, sock->info.af, true),
BLEN(&c->c2.to_link),
                 print_link_socket_actual(c->c2.to_link_addr, &gc),
PROTO_DUMP(&c->c2.to_link, &gc));
@@ -1892,7 +2070,7 @@ process_outgoing_link(struct context *c, struct
link_socket *sock)
  */

 void
-process_outgoing_tun(struct context *c, struct link_socket *in_sock)
+process_outgoing_tun_part2(struct context *c, struct link_socket *in_sock)
 {
     /*
      * Set up for write() call to TUN/TAP
@@ -1912,7 +2090,7 @@ process_outgoing_tun(struct context *c, struct
link_socket *in_sock)
     process_ip_header(c, PIP_MSSFIX | PIPV4_EXTRACT_DHCP_ROUTER |
PIPV4_CLIENT_NAT | PIP_OUTGOING,
                       &c->c2.to_tun, in_sock);

-    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size)
+    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size ||
c->c2.frame.bulk_size > 0)
     {
         /*
          * Write to TUN/TAP device.
@@ -1925,7 +2103,8 @@ process_outgoing_tun(struct context *c, struct
link_socket *in_sock)
             fprintf(stderr, "w");
         }
 #endif
-        dmsg(D_TUN_RW, "TUN WRITE [%d]", BLEN(&c->c2.to_tun));
+
+        dmsg(D_TUN_RW, "TUN WRITE [%d] [%d]", BLEN(&c->c2.to_tun),
c->c2.frame.buf.payload_size);

 #ifdef PACKET_TRUNCATION_CHECK
         ipv4_packet_size_verify(BPTR(&c->c2.to_tun), BLEN(&c->c2.to_tun),
TUNNEL_TYPE(c->c1.tuntap),
@@ -1981,6 +2160,38 @@ process_outgoing_tun(struct context *c, struct
link_socket *in_sock)
     perf_pop();
 }

+void process_outgoing_tun_part3(struct context *c, struct link_socket
*in_sock)
+{
+    if (check_bulk_mode(c))
+    {
+        int maxl = 0, plen = 0;
+        int leng = BLEN(&c->c2.buffers->send_tun_max);
+        uint8_t *temp = BPTR(&c->c2.buffers->send_tun_max);
+        for (int x = 0; x < TUN_BAT_MAX; ++x)
+        {
+            temp = buff_postsize(temp, &plen);
+            if ((leng > 0) && (plen > 0) && ((maxl + plen) < leng))
+            {
+                c->c2.to_tun = c->c2.buffers->to_tun_max;
+                c->c2.to_tun.offset = TUN_BAT_OFF;
+                c->c2.to_tun.len = plen;
+                bcopy(temp, BPTR(&c->c2.to_tun), plen);
+                temp += plen; maxl += (plen + 2);
+                process_outgoing_tun_part2(c, in_sock);
+            } else { break; }
+        }
+    }
+    buf_reset(&c->c2.to_tun);
+}
+
+void process_outgoing_tun(struct context *c, struct link_socket *in_sock)
+{
+    if (c->c2.frame.bulk_size <= 0) {
+        process_outgoing_tun_part2(c, in_sock);
+    }
+    process_outgoing_tun_part3(c, in_sock);
+}
+
 void
 pre_select(struct context *c)
 {
diff --git a/src/openvpn/forward.h b/src/openvpn/forward.h
index d5641491..9fda1583 100644
--- a/src/openvpn/forward.h
+++ b/src/openvpn/forward.h
@@ -79,6 +79,8 @@ void pre_select(struct context *c);

 void process_io(struct context *c, struct link_socket *sock);

+void xfer_io(struct context *c, struct context *b);
+

 /**********************************************************************/
 /**
@@ -196,6 +198,8 @@ bool process_incoming_link_part1(struct context *c,
struct link_socket_info *lsi
 void process_incoming_link_part2(struct context *c, struct
link_socket_info *lsi,
                                  const uint8_t *orig_buf);

+void process_incoming_link_part3(struct context *c);
+
 /**
  * Transfers \c float_sa data extracted from an incoming DCO
  * PEER_FLOAT_NTF to \c out_osaddr for later processing.
diff --git a/src/openvpn/init.c b/src/openvpn/init.c
index 40ae2c8c..bbdbad46 100644
--- a/src/openvpn/init.c
+++ b/src/openvpn/init.c
@@ -2971,6 +2971,10 @@ frame_finalize_options(struct context *c, const
struct options *o)
     tailroom += COMP_EXTRA_BUFFER(payload_size);
 #endif

+    if (frame->bulk_size > 0) {
+        payload_size = frame->tun_mtu;
+    }
+
     frame->buf.payload_size = payload_size;
     frame->buf.headroom = headroom;
     frame->buf.tailroom = tailroom;
@@ -3473,6 +3477,9 @@ do_init_frame_tls(struct context *c)
     if (c->c2.tls_multi)
     {
         tls_multi_init_finalize(c->c2.tls_multi, c->options.ce.tls_mtu);
+        if (c->c2.frame.bulk_size > 0) {
+            c->c2.tls_multi->opt.frame.buf.payload_size =
c->c2.frame.tun_mtu;
+        }
         ASSERT(c->c2.tls_multi->opt.frame.buf.payload_size <=
c->c2.frame.buf.payload_size);
         frame_print(&c->c2.tls_multi->opt.frame, D_MTU_INFO, "Control
Channel MTU parms");

@@ -3536,6 +3543,14 @@ do_init_frame(struct context *c)
         c->c2.frame.extra_tun += c->options.ce.tun_mtu_extra;
     }

+    /*
+     * Adjust bulk size based on the --bulk-mode parameter.
+     */
+    if (c->options.ce.bulk_mode)
+    {
+        c->c2.frame.bulk_size = c->options.ce.tun_mtu;
+    }
+
     /*
      * Fill in the blanks in the frame parameters structure,
      * make sure values are rational, etc.
@@ -3676,9 +3691,41 @@ init_context_buffers(const struct frame *frame)

     size_t buf_size = BUF_SIZE(frame);

+    if (frame->bulk_size > 0) {
+        size_t off_size = (frame->buf.headroom + TUN_BAT_OFF +
frame->buf.tailroom);
+        buf_size = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, off_size);
+    }
+
+    dmsg(M_INFO, "MEM NEW [%ld] [%d+%d+%d]", buf_size,
frame->buf.headroom, frame->buf.payload_size, frame->buf.tailroom);
+
     b->read_link_buf = alloc_buf(buf_size);
     b->read_tun_buf = alloc_buf(buf_size);

+    if (frame->bulk_size > 0) {
+        for (int x = 0; x < TUN_BAT_MAX; ++x)
+        {
+            size_t part_size = BUF_SIZE(frame);
+            b->read_tun_bufs[x] = alloc_buf(part_size);
+            b->read_tun_bufs[x].offset = TUN_BAT_OFF;
+            b->read_tun_bufs[x].len = 0;
+        }
+
+        b->read_tun_max = alloc_buf(buf_size);
+        b->read_tun_max.offset = TUN_BAT_OFF;
+        b->read_tun_max.len = 0;
+
+        b->send_tun_max = alloc_buf(buf_size);
+        b->send_tun_max.offset = TUN_BAT_OFF;
+        b->send_tun_max.len = 0;
+
+        b->to_tun_max = alloc_buf(buf_size);
+        b->to_tun_max.offset = TUN_BAT_OFF;
+        b->to_tun_max.len = 0;
+    }
+
+    b->bufs_indx = -1;
+    b->flag_ciph = -1;
+
     b->aux_buf = alloc_buf(buf_size);

     b->encrypt_buf = alloc_buf(buf_size);
@@ -3701,6 +3748,16 @@ free_context_buffers(struct context_buffers *b)
         free_buf(&b->read_tun_buf);
         free_buf(&b->aux_buf);

+        if (b->to_tun_max.data) {
+            free_buf(&b->to_tun_max);
+            free_buf(&b->send_tun_max);
+            free_buf(&b->read_tun_max);
+            for (int x = 0; x < TUN_BAT_MAX; ++x)
+            {
+                free_buf(&b->read_tun_bufs[x]);
+            }
+        }
+
 #ifdef USE_COMP
         free_buf(&b->compress_buf);
         free_buf(&b->decompress_buf);
diff --git a/src/openvpn/mtu.c b/src/openvpn/mtu.c
index a419e32d..7e35c837 100644
--- a/src/openvpn/mtu.c
+++ b/src/openvpn/mtu.c
@@ -41,9 +41,15 @@ void
 alloc_buf_sock_tun(struct buffer *buf, const struct frame *frame)
 {
     /* allocate buffer for overlapped I/O */
-    *buf = alloc_buf(BUF_SIZE(frame));
+    size_t alen = BUF_SIZE(frame);
+    size_t blen = frame->buf.payload_size;
+    if (frame->bulk_size > 0) {
+        alen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_OFF);
+        blen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_NOP);
+    }
+    *buf = alloc_buf(alen);
     ASSERT(buf_init(buf, frame->buf.headroom));
-    buf->len = frame->buf.payload_size;
+    buf->len = blen;
     ASSERT(buf_safe(buf, 0));
 }

diff --git a/src/openvpn/mtu.h b/src/openvpn/mtu.h
index 925ef0bf..eb799fb3 100644
--- a/src/openvpn/mtu.h
+++ b/src/openvpn/mtu.h
@@ -58,6 +58,14 @@
  */
 #define TUN_MTU_MIN 100

+/*
+ * Bulk mode static define values.
+ */
+#define TUN_BAT_MIN        6
+#define TUN_BAT_MAX        9
+#define TUN_BAT_OFF        256
+#define TUN_BAT_NOP        0
+
 /*
  * Default MTU of network over which tunnel data will pass by TCP/UDP.
  */
@@ -152,6 +160,10 @@ struct frame
                             *   which defaults to 0 for tun and 32
                             *   (\c TAP_MTU_EXTRA_DEFAULT) for tap.
                             *   */
+
+    int bulk_size;              /**< Signal to the init frame function
+                                 *   to allow for bulk mode TCP transfers.
+                                 *   */
 };

 /* Forward declarations, to prevent includes */
@@ -171,6 +183,7 @@ struct options;
  * larger than the headroom.
  */
 #define BUF_SIZE(f) ((f)->buf.headroom + (f)->buf.payload_size +
(f)->buf.tailroom)
+#define BAT_SIZE(a, b, c) ((a * b) + c)

 /*
  * Function prototypes.
diff --git a/src/openvpn/multi.c b/src/openvpn/multi.c
index e1ce32ab..9e089703 100644
--- a/src/openvpn/multi.c
+++ b/src/openvpn/multi.c
@@ -3414,6 +3414,7 @@ multi_process_incoming_link(struct multi_context *m,
struct multi_instance *inst
                 }

                 process_incoming_link_part2(c, lsi, orig_buf);
+                process_incoming_link_part3(c);
             }
             perf_pop();

@@ -3558,9 +3559,7 @@ multi_process_incoming_tun(struct multi_context *m,
const unsigned int mpp_flags
         const int dev_type = TUNNEL_TYPE(m->top.c1.tuntap);
         int16_t vid = 0;

-#ifdef MULTI_DEBUG_EVENT_LOOP
-        printf("TUN -> TCP/UDP [%d]\n", BLEN(&m->top.c2.buf));
-#endif
+        msg(D_MULTI_DEBUG, "TUN -> TCP/UDP [%d]", BLEN(&m->top.c2.buf));

         if (m->pending)
         {
@@ -3610,6 +3609,8 @@ multi_process_incoming_tun(struct multi_context *m,
const unsigned int mpp_flags
                         {
                             /* transfer packet pointer from top-level
context buffer to instance */
                             c->c2.buf = m->top.c2.buf;
+                            /* todo determine if to call this
(multi_process_incoming_tun) for each bulk item read? */
+                            xfer_io(c, &m->top);
                         }
                         else
                         {
diff --git a/src/openvpn/openvpn.h b/src/openvpn/openvpn.h
index cd99cd40..21fa8967 100644
--- a/src/openvpn/openvpn.h
+++ b/src/openvpn/openvpn.h
@@ -112,6 +112,14 @@ struct context_buffers
      */
     struct buffer read_link_buf;
     struct buffer read_tun_buf;
+
+    struct buffer read_tun_bufs[TUN_BAT_MAX];
+    struct buffer read_tun_max;
+    struct buffer send_tun_max;
+    struct buffer to_tun_max;
+
+    int bufs_indx;
+    int flag_ciph;
 };

 /*
@@ -376,6 +384,8 @@ struct context_2
     struct buffer to_tun;
     struct buffer to_link;

+    struct buffer bufs[TUN_BAT_MAX];
+
     /* should we print R|W|r|w to console on packet transfers? */
     bool log_rw;

diff --git a/src/openvpn/options.c b/src/openvpn/options.c
index c54032d8..041d17d0 100644
--- a/src/openvpn/options.c
+++ b/src/openvpn/options.c
@@ -304,6 +304,7 @@ static const char usage_message[] =
     "                  'maybe' -- Use per-route hints\n"
     "                  'yes'   -- Always DF (Don't Fragment)\n"
     "--mtu-test      : Empirically measure and report MTU.\n"
+    "--bulk-mode     : Use bulk TUN/TCP reads/writes.\n"
 #ifdef ENABLE_FRAGMENT
     "--fragment max  : Enable internal datagram fragmentation so that no
UDP\n"
     "                  datagrams are sent which are larger than max
bytes.\n"
@@ -3005,6 +3006,9 @@ options_postprocess_mutate_ce(struct options *o,
struct connection_entry *ce)
             ce->tun_mtu_extra_defined = true;
             ce->tun_mtu_extra = TAP_MTU_EXTRA_DEFAULT;
         }
+        if (ce->proto != PROTO_TCP && ce->proto != PROTO_TCP_SERVER &&
ce->proto != PROTO_TCP_CLIENT) {
+            ce->bulk_mode = false;
+        }
     }

     /*
@@ -9926,6 +9930,10 @@ add_option(struct options *options, char *p[], bool
is_inline, const char *file,
             goto err;
         }
     }
+    else if (streq(p[0], "bulk-mode"))
+    {
+        options->ce.bulk_mode = true;
+    }
     else
     {
         int i;
diff --git a/src/openvpn/options.h b/src/openvpn/options.h
index 38e67c8d..d1b0586d 100644
--- a/src/openvpn/options.h
+++ b/src/openvpn/options.h
@@ -174,6 +174,9 @@ struct connection_entry

     /* Allow only client that support resending the wrapped client key */
     bool tls_crypt_v2_force_cookie;
+
+    /* Bulk mode allows for multiple tun reads + larger tcp writes */
+    bool bulk_mode;
 };

 struct remote_entry
--
2.39.5 (Apple Git-154)







On Fri, Aug 8, 2025 at 10:23 AM Jon Chiappetta <root@fossjon.com> wrote:

> [replying to my own update thread]
>
> I pushed an update to the PR with the following small changes:
>
> - rebased to the latest master commit
> - increased the data buffer size to be slightly bigger than the read
> buffer size
> - copied two more buf resets so that the additional bulk forward functions
> match the present logic checks
>
> Example updated pull request:
> https://github.com/OpenVPN/openvpn/pull/814/files
>
> $ cat 0001-bulk-mode.patch
> From 2ce0b023d105e7ecc289a414cd26f7ebc8bcbcaf Mon Sep 17 00:00:00 2001
> From: Jon Chiappetta <root@fossjon.com>
> Date: Wed, 6 Aug 2025 16:33:18 -0400
> Subject: [PATCH] bulk mode
>
> ---
>  src/openvpn/forward.c | 228 ++++++++++++++++++++++++++++++++++++++++--
>  src/openvpn/forward.h |   4 +
>  src/openvpn/init.c    |  57 +++++++++++
>  src/openvpn/mtu.c     |  10 +-
>  src/openvpn/mtu.h     |  13 +++
>  src/openvpn/multi.c   |   7 +-
>  src/openvpn/openvpn.h |  10 ++
>  src/openvpn/options.c |   8 ++
>  src/openvpn/options.h |   3 +
>  9 files changed, 328 insertions(+), 12 deletions(-)
>
> diff --git a/src/openvpn/forward.c b/src/openvpn/forward.c
> index 75ca9d5c..0af983e9 100644
> --- a/src/openvpn/forward.c
> +++ b/src/openvpn/forward.c
> @@ -46,6 +46,9 @@
>
>  #include "mstats.h"
>
> +#include <sys/select.h>
> +#include <sys/time.h>
> +
>  counter_type link_read_bytes_global;  /* GLOBAL */
>  counter_type link_write_bytes_global; /* GLOBAL */
>
> @@ -78,6 +81,32 @@ show_wait_status(struct context *c)
>
>  #endif /* ifdef ENABLE_DEBUG */
>
> +bool check_bulk_mode(struct context *c)
> +{
> +    if ((c->c2.frame.bulk_size > 0) && (c->c1.tuntap != NULL) &&
> (c->c2.buffers != NULL))
> +    {
> +        return true;
> +    }
> +    return false;
> +}
> +
> +void xfer_io(struct context *c, struct context *b)
> +{
> +    int plen = 0;
> +    if (check_bulk_mode(b))
> +    {
> +        int leng = (b->c2.buffers->bufs_indx + 1);
> +        for (int x = 0; x < leng; ++x)
> +        {
> +            plen = BLEN(&b->c2.bufs[x]);
> +            if (plen < 1) { c->c2.bufs[x].len = 0; }
> +            else { c->c2.bufs[x] = b->c2.bufs[x]; }
> +        }
> +        c->c2.buffers->bufs_indx = b->c2.buffers->bufs_indx;
> +        b->c2.buffers->bufs_indx = -1;
> +    }
> +}
> +
>  static void
>  check_tls_errors_co(struct context *c)
>  {
> @@ -605,6 +634,21 @@ buffer_turnover(const uint8_t *orig_buf, struct
> buffer *dest_stub, struct buffer
>      }
>  }
>
> +uint8_t *buff_prepsize(uint8_t *buff, int *size)
> +{
> +    buff[0] = ((*size >> 8) & 0xff);
> +    buff[1] = ((*size >> 0) & 0xff);
> +    buff += 2;
> +    return buff;
> +}
> +
> +uint8_t *buff_postsize(uint8_t *buff, int *size)
> +{
> +    *size = ((buff[0] << 8) + (buff[1] << 0));
> +    buff += 2;
> +    return buff;
> +}
> +
>  /*
>   * Compress, fragment, encrypt and HMAC-sign an outgoing packet.
>   * Input: c->c2.buf
> @@ -1031,6 +1075,7 @@ process_incoming_link_part1(struct context *c,
> struct link_socket_info *lsi, boo
>          fprintf(stderr, "R");
>      }
>  #endif
> +
>      msg(D_LINK_RW, "%s READ [%d] from %s: %s", proto2ascii(lsi->proto,
> lsi->af, true),
>          BLEN(&c->c2.buf), print_link_socket_actual(&c->c2.from, &gc),
> PROTO_DUMP(&c->c2.buf, &gc));
>
> @@ -1211,6 +1256,27 @@ process_incoming_link_part2(struct context *c,
> struct link_socket_info *lsi,
>      }
>  }
>
> +void process_incoming_link_part3(struct context *c)
> +{
> +    int leng = BLEN(&c->c2.buf);
> +    if (leng > 0)
> +    {
> +        if (check_bulk_mode(c))
> +        {
> +            c->c2.buffers->send_tun_max.offset = TUN_BAT_OFF;
> +            c->c2.buffers->send_tun_max.len = leng;
> +            bcopy(BPTR(&c->c2.buf), BPTR(&c->c2.buffers->send_tun_max),
> leng);
> +            //dmsg(M_INFO, "FWD BAT LINK 0 [%d] [%d] [%d] [%d] [%d]",
> BLEN(&c->c2.buf), BLEN(&c->c2.to_tun), BLEN(&c->c2.buffers->read_link_buf),
> BLEN(&c->c2.buffers->read_link_buf), BLEN(&c->c2.buffers->send_tun_max));
> +            c->c2.to_tun.offset += 2;
> +            c->c2.buf.offset += 2;
> +        }
> +    }
> +    else
> +    {
> +        buf_reset(&c->c2.to_tun);
> +    }
> +}
> +
>  static void
>  process_incoming_link(struct context *c, struct link_socket *sock)
>  {
> @@ -1221,6 +1287,7 @@ process_incoming_link(struct context *c, struct
> link_socket *sock)
>
>      process_incoming_link_part1(c, lsi, false);
>      process_incoming_link_part2(c, lsi, orig_buf);
> +    process_incoming_link_part3(c);
>
>      perf_pop();
>  }
> @@ -1321,7 +1388,7 @@ process_incoming_dco(struct context *c)
>   */
>
>  void
> -read_incoming_tun(struct context *c)
> +read_incoming_tun_part2(struct context *c)
>  {
>      /*
>       * Setup for read() call on TUN/TAP device.
> @@ -1382,6 +1449,55 @@ read_incoming_tun(struct context *c)
>      perf_pop();
>  }
>
> +void read_incoming_tun_part3(struct context *c)
> +{
> +    fd_set rfds;
> +    struct timeval timo;
> +    if (check_bulk_mode(c))
> +    {
> +        int plen = 0;
> +        int fdno = c->c1.tuntap->fd;
> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
> +        {
> +            int leng = plen;
> +            int indx = (c->c2.buffers->bufs_indx + 1);
> +            if (indx >= TUN_BAT_MIN) { break; }
> +            if (leng < 1)
> +            {
> +                FD_ZERO(&rfds);
> +                FD_SET(fdno, &rfds);
> +                timo.tv_sec = 0;
> +                timo.tv_usec = 0;
> +                select(fdno+1, &rfds, NULL, NULL, &timo);
> +                if (FD_ISSET(fdno, &rfds))
> +                {
> +                    read_incoming_tun_part2(c);
> +                    plen = BLEN(&c->c2.buf);
> +                } else { break; }
> +            }
> +            //dmsg(M_INFO, "FWD BAT READ 0 [%d] [%d] [%d] [%d] [%d]",
> c->c2.buffers->bufs_indx + 1, fdno, BLEN(&c->c2.buf),
> BLEN(&c->c2.buffers->read_tun_buf), BLEN(&c->c2.buffers->read_tun_max));
> +            leng = plen;
> +            if (leng > 0)
> +            {
> +                c->c2.buffers->read_tun_bufs[indx].offset = TUN_BAT_OFF;
> +                c->c2.buffers->read_tun_bufs[indx].len = leng;
> +                bcopy(BPTR(&c->c2.buf),
> BPTR(&c->c2.buffers->read_tun_bufs[indx]), leng);
> +                c->c2.bufs[indx] = c->c2.buffers->read_tun_bufs[indx];
> +                c->c2.buffers->bufs_indx = indx;
> +            } else { break; }
> +            plen = 0;
> +        }
> +    }
> +}
> +
> +void read_incoming_tun(struct context *c)
> +{
> +    if (c->c2.frame.bulk_size <= 0) {
> +        read_incoming_tun_part2(c);
> +    }
> +    read_incoming_tun_part3(c);
> +}
> +
>  /**
>   * Drops UDP packets which OS decided to route via tun.
>   *
> @@ -1469,7 +1585,7 @@ drop_if_recursive_routing(struct context *c, struct
> buffer *buf)
>   */
>
>  void
> -process_incoming_tun(struct context *c, struct link_socket *out_sock)
> +process_incoming_tun_part2(struct context *c, struct link_socket
> *out_sock)
>  {
>      struct gc_arena gc = gc_new();
>
> @@ -1488,7 +1604,7 @@ process_incoming_tun(struct context *c, struct
> link_socket *out_sock)
>  #endif
>
>      /* Show packet content */
> -    dmsg(D_TUN_RW, "TUN READ [%d]", BLEN(&c->c2.buf));
> +    dmsg(D_TUN_RW, "TUN READ [%d] [%d]", BLEN(&c->c2.buf),
> c->c2.frame.buf.payload_size);
>
>      if (c->c2.buf.len > 0)
>      {
> @@ -1512,7 +1628,9 @@ process_incoming_tun(struct context *c, struct
> link_socket *out_sock)
>      }
>      if (c->c2.buf.len > 0)
>      {
> +        if ((c->c2.buffers == NULL) || (c->c2.buffers->flag_ciph != -2)) {
>          encrypt_sign(c, true);
> +        }
>      }
>      else
>      {
> @@ -1522,6 +1640,67 @@ process_incoming_tun(struct context *c, struct
> link_socket *out_sock)
>      gc_free(&gc);
>  }
>
> +void process_incoming_tun_part3(struct context *c, struct link_socket
> *out_sock)
> +{
> +    if (c->c2.buf.len > 0)
> +    {
> +        if (check_bulk_mode(c))
> +        {
> +            c->c2.buffers->flag_ciph = -2;
> +            c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
> +            c->c2.buffers->read_tun_max.len = 0;
> +            uint8_t *temp = BPTR(&c->c2.buffers->read_tun_max);
> +            int plen = 0, fdno = c->c1.tuntap->fd;
> +            int maxl = 0, leng = (c->c2.buffers->bufs_indx + 1);
> +            if ((fdno > 0) && (leng > 0))
> +            {
> +                for (int x = 0; x < leng; ++x)
> +                {
> +                    c->c2.buf = c->c2.bufs[x];
> +                    //dmsg(M_INFO, "FWD BAT INPT 0 [%d] [%d] [%d] [%d]
> [%d]", x, fdno, BLEN(&c->c2.buf), BLEN(&c->c2.buffers->read_tun_buf),
> BLEN(&c->c2.bufs[x]));
> +                    process_incoming_tun_part2(c, out_sock);
> +                    if (BLEN(&c->c2.buf) < 1)
> +                    {
> +                        c->c2.bufs[x].len = 0;
> +                    }
> +                }
> +                for (int x = 0; x < leng; ++x)
> +                {
> +                    plen = c->c2.bufs[x].len;
> +                    if (plen > 0)
> +                    {
> +                        temp = buff_prepsize(temp, &plen);
> +                        bcopy(BPTR(&c->c2.bufs[x]), temp, plen);
> +                        temp += plen; maxl += (plen + 2);
> +                    }
> +                }
> +                if (maxl > 0)
> +                {
> +                    c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
> +                    c->c2.buffers->read_tun_max.len = maxl;
> +                    c->c2.buf = c->c2.buffers->read_tun_max;
> +                    //dmsg(M_INFO, "FWD BAT INPT 1 [%d] [%d] [%d] [%d]
> [%d]", maxl, fdno, BLEN(&c->c2.buf), BLEN(&c->c2.buffers->read_tun_buf),
> BLEN(&c->c2.buffers->read_tun_max));
> +                    encrypt_sign(c, true);
> +                }
> +            }
> +            c->c2.buffers->bufs_indx = -1;
> +            c->c2.buffers->flag_ciph = -1;
> +        }
> +    }
> +    else
> +    {
> +        buf_reset(&c->c2.to_link);
> +    }
> +}
> +
> +void process_incoming_tun(struct context *c, struct link_socket *out_sock)
> +{
> +    if (c->c2.frame.bulk_size <= 0) {
> +        process_incoming_tun_part2(c, out_sock);
> +    }
> +    process_incoming_tun_part3(c, out_sock);
> +}
> +
>  /**
>   * Forges a IPv6 ICMP packet with a no route to host error code from the
>   * IPv6 packet in buf and sends it directly back to the client via the tun
> @@ -1748,7 +1927,7 @@ process_outgoing_link(struct context *c, struct
> link_socket *sock)
>
>      perf_push(PERF_PROC_OUT_LINK);
>
> -    if (c->c2.to_link.len > 0 && c->c2.to_link.len <=
> c->c2.frame.buf.payload_size)
> +    if (c->c2.to_link.len > 0 && (c->c2.to_link.len <=
> c->c2.frame.buf.payload_size || c->c2.frame.bulk_size > 0))
>      {
>          /*
>           * Setup for call to send/sendto which will send
> @@ -1793,6 +1972,7 @@ process_outgoing_link(struct context *c, struct
> link_socket *sock)
>                  fprintf(stderr, "W");
>              }
>  #endif
> +
>              msg(D_LINK_RW, "%s WRITE [%d] to %s: %s",
>                  proto2ascii(sock->info.proto, sock->info.af, true),
> BLEN(&c->c2.to_link),
>                  print_link_socket_actual(c->c2.to_link_addr, &gc),
> PROTO_DUMP(&c->c2.to_link, &gc));
> @@ -1892,7 +2072,7 @@ process_outgoing_link(struct context *c, struct
> link_socket *sock)
>   */
>
>  void
> -process_outgoing_tun(struct context *c, struct link_socket *in_sock)
> +process_outgoing_tun_part2(struct context *c, struct link_socket *in_sock)
>  {
>      /*
>       * Set up for write() call to TUN/TAP
> @@ -1912,7 +2092,7 @@ process_outgoing_tun(struct context *c, struct
> link_socket *in_sock)
>      process_ip_header(c, PIP_MSSFIX | PIPV4_EXTRACT_DHCP_ROUTER |
> PIPV4_CLIENT_NAT | PIP_OUTGOING,
>                        &c->c2.to_tun, in_sock);
>
> -    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size)
> +    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size ||
> c->c2.frame.bulk_size > 0)
>      {
>          /*
>           * Write to TUN/TAP device.
> @@ -1925,7 +2105,8 @@ process_outgoing_tun(struct context *c, struct
> link_socket *in_sock)
>              fprintf(stderr, "w");
>          }
>  #endif
> -        dmsg(D_TUN_RW, "TUN WRITE [%d]", BLEN(&c->c2.to_tun));
> +
> +        dmsg(D_TUN_RW, "TUN WRITE [%d] [%d]", BLEN(&c->c2.to_tun),
> c->c2.frame.buf.payload_size);
>
>  #ifdef PACKET_TRUNCATION_CHECK
>          ipv4_packet_size_verify(BPTR(&c->c2.to_tun), BLEN(&c->c2.to_tun),
> TUNNEL_TYPE(c->c1.tuntap),
> @@ -1981,6 +2162,39 @@ process_outgoing_tun(struct context *c, struct
> link_socket *in_sock)
>      perf_pop();
>  }
>
> +void process_outgoing_tun_part3(struct context *c, struct link_socket
> *in_sock)
> +{
> +    if (check_bulk_mode(c))
> +    {
> +        int maxl = 0, plen = 0;
> +        int leng = BLEN(&c->c2.buffers->send_tun_max);
> +        uint8_t *temp = BPTR(&c->c2.buffers->send_tun_max);
> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
> +        {
> +            temp = buff_postsize(temp, &plen);
> +            if ((leng > 0) && (plen > 0) && ((maxl + plen) < leng))
> +            {
> +                c->c2.to_tun = c->c2.buffers->to_tun_max;
> +                c->c2.to_tun.offset = TUN_BAT_OFF;
> +                c->c2.to_tun.len = plen;
> +                bcopy(temp, BPTR(&c->c2.to_tun), plen);
> +                temp += plen; maxl += (plen + 2);
> +                //dmsg(M_INFO, "FWD BAT OUTP 1 [%d] [%d] [%d] [%d]", x,
> BLEN(&c->c2.buf), BLEN(&c->c2.to_tun), BLEN(&c->c2.buffers->read_link_buf));
> +                process_outgoing_tun_part2(c, in_sock);
> +            } else { break; }
> +        }
> +    }
> +    buf_reset(&c->c2.to_tun);
> +}
> +
> +void process_outgoing_tun(struct context *c, struct link_socket *in_sock)
> +{
> +    if (c->c2.frame.bulk_size <= 0) {
> +        process_outgoing_tun_part2(c, in_sock);
> +    }
> +    process_outgoing_tun_part3(c, in_sock);
> +}
> +
>  void
>  pre_select(struct context *c)
>  {
> diff --git a/src/openvpn/forward.h b/src/openvpn/forward.h
> index d5641491..9fda1583 100644
> --- a/src/openvpn/forward.h
> +++ b/src/openvpn/forward.h
> @@ -79,6 +79,8 @@ void pre_select(struct context *c);
>
>  void process_io(struct context *c, struct link_socket *sock);
>
> +void xfer_io(struct context *c, struct context *b);
> +
>
>  /**********************************************************************/
>  /**
> @@ -196,6 +198,8 @@ bool process_incoming_link_part1(struct context *c,
> struct link_socket_info *lsi
>  void process_incoming_link_part2(struct context *c, struct
> link_socket_info *lsi,
>                                   const uint8_t *orig_buf);
>
> +void process_incoming_link_part3(struct context *c);
> +
>  /**
>   * Transfers \c float_sa data extracted from an incoming DCO
>   * PEER_FLOAT_NTF to \c out_osaddr for later processing.
> diff --git a/src/openvpn/init.c b/src/openvpn/init.c
> index 40ae2c8c..bbdbad46 100644
> --- a/src/openvpn/init.c
> +++ b/src/openvpn/init.c
> @@ -2971,6 +2971,10 @@ frame_finalize_options(struct context *c, const
> struct options *o)
>      tailroom += COMP_EXTRA_BUFFER(payload_size);
>  #endif
>
> +    if (frame->bulk_size > 0) {
> +        payload_size = frame->tun_mtu;
> +    }
> +
>      frame->buf.payload_size = payload_size;
>      frame->buf.headroom = headroom;
>      frame->buf.tailroom = tailroom;
> @@ -3473,6 +3477,9 @@ do_init_frame_tls(struct context *c)
>      if (c->c2.tls_multi)
>      {
>          tls_multi_init_finalize(c->c2.tls_multi, c->options.ce.tls_mtu);
> +        if (c->c2.frame.bulk_size > 0) {
> +            c->c2.tls_multi->opt.frame.buf.payload_size =
> c->c2.frame.tun_mtu;
> +        }
>          ASSERT(c->c2.tls_multi->opt.frame.buf.payload_size <=
> c->c2.frame.buf.payload_size);
>          frame_print(&c->c2.tls_multi->opt.frame, D_MTU_INFO, "Control
> Channel MTU parms");
>
> @@ -3536,6 +3543,14 @@ do_init_frame(struct context *c)
>          c->c2.frame.extra_tun += c->options.ce.tun_mtu_extra;
>      }
>
> +    /*
> +     * Adjust bulk size based on the --bulk-mode parameter.
> +     */
> +    if (c->options.ce.bulk_mode)
> +    {
> +        c->c2.frame.bulk_size = c->options.ce.tun_mtu;
> +    }
> +
>      /*
>       * Fill in the blanks in the frame parameters structure,
>       * make sure values are rational, etc.
> @@ -3676,9 +3691,41 @@ init_context_buffers(const struct frame *frame)
>
>      size_t buf_size = BUF_SIZE(frame);
>
> +    if (frame->bulk_size > 0) {
> +        size_t off_size = (frame->buf.headroom + TUN_BAT_OFF +
> frame->buf.tailroom);
> +        buf_size = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, off_size);
> +    }
> +
> +    dmsg(M_INFO, "MEM NEW [%ld] [%d+%d+%d]", buf_size,
> frame->buf.headroom, frame->buf.payload_size, frame->buf.tailroom);
> +
>      b->read_link_buf = alloc_buf(buf_size);
>      b->read_tun_buf = alloc_buf(buf_size);
>
> +    if (frame->bulk_size > 0) {
> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
> +        {
> +            size_t part_size = BUF_SIZE(frame);
> +            b->read_tun_bufs[x] = alloc_buf(part_size);
> +            b->read_tun_bufs[x].offset = TUN_BAT_OFF;
> +            b->read_tun_bufs[x].len = 0;
> +        }
> +
> +        b->read_tun_max = alloc_buf(buf_size);
> +        b->read_tun_max.offset = TUN_BAT_OFF;
> +        b->read_tun_max.len = 0;
> +
> +        b->send_tun_max = alloc_buf(buf_size);
> +        b->send_tun_max.offset = TUN_BAT_OFF;
> +        b->send_tun_max.len = 0;
> +
> +        b->to_tun_max = alloc_buf(buf_size);
> +        b->to_tun_max.offset = TUN_BAT_OFF;
> +        b->to_tun_max.len = 0;
> +    }
> +
> +    b->bufs_indx = -1;
> +    b->flag_ciph = -1;
> +
>      b->aux_buf = alloc_buf(buf_size);
>
>      b->encrypt_buf = alloc_buf(buf_size);
> @@ -3701,6 +3748,16 @@ free_context_buffers(struct context_buffers *b)
>          free_buf(&b->read_tun_buf);
>          free_buf(&b->aux_buf);
>
> +        if (b->to_tun_max.data) {
> +            free_buf(&b->to_tun_max);
> +            free_buf(&b->send_tun_max);
> +            free_buf(&b->read_tun_max);
> +            for (int x = 0; x < TUN_BAT_MAX; ++x)
> +            {
> +                free_buf(&b->read_tun_bufs[x]);
> +            }
> +        }
> +
>  #ifdef USE_COMP
>          free_buf(&b->compress_buf);
>          free_buf(&b->decompress_buf);
> diff --git a/src/openvpn/mtu.c b/src/openvpn/mtu.c
> index a419e32d..7e35c837 100644
> --- a/src/openvpn/mtu.c
> +++ b/src/openvpn/mtu.c
> @@ -41,9 +41,15 @@ void
>  alloc_buf_sock_tun(struct buffer *buf, const struct frame *frame)
>  {
>      /* allocate buffer for overlapped I/O */
> -    *buf = alloc_buf(BUF_SIZE(frame));
> +    size_t alen = BUF_SIZE(frame);
> +    size_t blen = frame->buf.payload_size;
> +    if (frame->bulk_size > 0) {
> +        alen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_OFF);
> +        blen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_NOP);
> +    }
> +    *buf = alloc_buf(alen);
>      ASSERT(buf_init(buf, frame->buf.headroom));
> -    buf->len = frame->buf.payload_size;
> +    buf->len = blen;
>      ASSERT(buf_safe(buf, 0));
>  }
>
> diff --git a/src/openvpn/mtu.h b/src/openvpn/mtu.h
> index 925ef0bf..eb799fb3 100644
> --- a/src/openvpn/mtu.h
> +++ b/src/openvpn/mtu.h
> @@ -58,6 +58,14 @@
>   */
>  #define TUN_MTU_MIN 100
>
> +/*
> + * Bulk mode static define values.
> + */
> +#define TUN_BAT_MIN        6
> +#define TUN_BAT_MAX        9
> +#define TUN_BAT_OFF        256
> +#define TUN_BAT_NOP        0
> +
>  /*
>   * Default MTU of network over which tunnel data will pass by TCP/UDP.
>   */
> @@ -152,6 +160,10 @@ struct frame
>                              *   which defaults to 0 for tun and 32
>                              *   (\c TAP_MTU_EXTRA_DEFAULT) for tap.
>                              *   */
> +
> +    int bulk_size;              /**< Signal to the init frame function
> +                                 *   to allow for bulk mode TCP transfers.
> +                                 *   */
>  };
>
>  /* Forward declarations, to prevent includes */
> @@ -171,6 +183,7 @@ struct options;
>   * larger than the headroom.
>   */
>  #define BUF_SIZE(f) ((f)->buf.headroom + (f)->buf.payload_size +
> (f)->buf.tailroom)
> +#define BAT_SIZE(a, b, c) ((a * b) + c)
>
>  /*
>   * Function prototypes.
> diff --git a/src/openvpn/multi.c b/src/openvpn/multi.c
> index e1ce32ab..9e089703 100644
> --- a/src/openvpn/multi.c
> +++ b/src/openvpn/multi.c
> @@ -3414,6 +3414,7 @@ multi_process_incoming_link(struct multi_context *m,
> struct multi_instance *inst
>                  }
>
>                  process_incoming_link_part2(c, lsi, orig_buf);
> +                process_incoming_link_part3(c);
>              }
>              perf_pop();
>
> @@ -3558,9 +3559,7 @@ multi_process_incoming_tun(struct multi_context *m,
> const unsigned int mpp_flags
>          const int dev_type = TUNNEL_TYPE(m->top.c1.tuntap);
>          int16_t vid = 0;
>
> -#ifdef MULTI_DEBUG_EVENT_LOOP
> -        printf("TUN -> TCP/UDP [%d]\n", BLEN(&m->top.c2.buf));
> -#endif
> +        msg(D_MULTI_DEBUG, "TUN -> TCP/UDP [%d]", BLEN(&m->top.c2.buf));
>
>          if (m->pending)
>          {
> @@ -3610,6 +3609,8 @@ multi_process_incoming_tun(struct multi_context *m,
> const unsigned int mpp_flags
>                          {
>                              /* transfer packet pointer from top-level
> context buffer to instance */
>                              c->c2.buf = m->top.c2.buf;
> +                            /* todo determine if to call this
> (multi_process_incoming_tun) for each bulk item read? */
> +                            xfer_io(c, &m->top);
>                          }
>                          else
>                          {
> diff --git a/src/openvpn/openvpn.h b/src/openvpn/openvpn.h
> index cd99cd40..21fa8967 100644
> --- a/src/openvpn/openvpn.h
> +++ b/src/openvpn/openvpn.h
> @@ -112,6 +112,14 @@ struct context_buffers
>       */
>      struct buffer read_link_buf;
>      struct buffer read_tun_buf;
> +
> +    struct buffer read_tun_bufs[TUN_BAT_MAX];
> +    struct buffer read_tun_max;
> +    struct buffer send_tun_max;
> +    struct buffer to_tun_max;
> +
> +    int bufs_indx;
> +    int flag_ciph;
>  };
>
>  /*
> @@ -376,6 +384,8 @@ struct context_2
>      struct buffer to_tun;
>      struct buffer to_link;
>
> +    struct buffer bufs[TUN_BAT_MAX];
> +
>      /* should we print R|W|r|w to console on packet transfers? */
>      bool log_rw;
>
> diff --git a/src/openvpn/options.c b/src/openvpn/options.c
> index c54032d8..041d17d0 100644
> --- a/src/openvpn/options.c
> +++ b/src/openvpn/options.c
> @@ -304,6 +304,7 @@ static const char usage_message[] =
>      "                  'maybe' -- Use per-route hints\n"
>      "                  'yes'   -- Always DF (Don't Fragment)\n"
>      "--mtu-test      : Empirically measure and report MTU.\n"
> +    "--bulk-mode     : Use bulk TUN/TCP reads/writes.\n"
>  #ifdef ENABLE_FRAGMENT
>      "--fragment max  : Enable internal datagram fragmentation so that no
> UDP\n"
>      "                  datagrams are sent which are larger than max
> bytes.\n"
> @@ -3005,6 +3006,9 @@ options_postprocess_mutate_ce(struct options *o,
> struct connection_entry *ce)
>              ce->tun_mtu_extra_defined = true;
>              ce->tun_mtu_extra = TAP_MTU_EXTRA_DEFAULT;
>          }
> +        if (ce->proto != PROTO_TCP && ce->proto != PROTO_TCP_SERVER &&
> ce->proto != PROTO_TCP_CLIENT) {
> +            ce->bulk_mode = false;
> +        }
>      }
>
>      /*
> @@ -9926,6 +9930,10 @@ add_option(struct options *options, char *p[], bool
> is_inline, const char *file,
>              goto err;
>          }
>      }
> +    else if (streq(p[0], "bulk-mode"))
> +    {
> +        options->ce.bulk_mode = true;
> +    }
>      else
>      {
>          int i;
> diff --git a/src/openvpn/options.h b/src/openvpn/options.h
> index 38e67c8d..d1b0586d 100644
> --- a/src/openvpn/options.h
> +++ b/src/openvpn/options.h
> @@ -174,6 +174,9 @@ struct connection_entry
>
>      /* Allow only client that support resending the wrapped client key */
>      bool tls_crypt_v2_force_cookie;
> +
> +    /* Bulk mode allows for multiple tun reads + larger tcp writes */
> +    bool bulk_mode;
>  };
>
>  struct remote_entry
> --
> 2.39.5 (Apple Git-154)
>
>
>
>
> On Thu, Aug 7, 2025 at 2:29 PM Jon Chiappetta <root@fossjon.com> wrote:
>
>> Thanks to Gert's help on this, I was able to finally configure and
>> compile and run and test the bulk mode changes against the latest git
>> source code to ensure everything still works correctly.
>>
>> I also fixed up some other issues like properly freeing the extra buffer
>> allocations and removing the unneeded batched data prefixes and converting
>> a remaining while loop to a max limited for loop and properly resetting the
>> outgoing tun buffer pointer at the end of the write method when finished.
>>
>> Thanks,
>> Jon C
>>
>> Example updated pull request:
>> https://github.com/OpenVPN/openvpn/pull/814/files
>>
>> git formatted diff patch:
>>
>> From 985e88a9af26a39554f113f37ee18032a2f41c3e Mon Sep 17 00:00:00 2001
>> From: Jon Chiappetta <root@fossjon.com>
>> Date: Wed, 6 Aug 2025 16:33:18 -0400
>> Subject: [PATCH] bulk mode
>>
>> ---
>>  src/openvpn/forward.c | 217 ++++++++++++++++++++++++++++++++++++++++--
>>  src/openvpn/forward.h |   4 +
>>  src/openvpn/init.c    |  56 +++++++++++
>>  src/openvpn/mtu.c     |  10 +-
>>  src/openvpn/mtu.h     |  13 +++
>>  src/openvpn/multi.c   |   7 +-
>>  src/openvpn/openvpn.h |  10 ++
>>  src/openvpn/options.c |   8 ++
>>  src/openvpn/options.h |   3 +
>>  9 files changed, 316 insertions(+), 12 deletions(-)
>>
>> diff --git a/src/openvpn/forward.c b/src/openvpn/forward.c
>> index 75ca9d5c..d9a98607 100644
>> --- a/src/openvpn/forward.c
>> +++ b/src/openvpn/forward.c
>> @@ -46,6 +46,9 @@
>>
>>  #include "mstats.h"
>>
>> +#include <sys/select.h>
>> +#include <sys/time.h>
>> +
>>  counter_type link_read_bytes_global;  /* GLOBAL */
>>  counter_type link_write_bytes_global; /* GLOBAL */
>>
>> @@ -78,6 +81,32 @@ show_wait_status(struct context *c)
>>
>>  #endif /* ifdef ENABLE_DEBUG */
>>
>> +bool check_bulk_mode(struct context *c)
>> +{
>> +    if ((c->c2.frame.bulk_size > 0) && (c->c1.tuntap != NULL) &&
>> (c->c2.buffers != NULL))
>> +    {
>> +        return true;
>> +    }
>> +    return false;
>> +}
>> +
>> +void xfer_io(struct context *c, struct context *b)
>> +{
>> +    int plen = 0;
>> +    if (check_bulk_mode(b))
>> +    {
>> +        int leng = (b->c2.buffers->bufs_indx + 1);
>> +        for (int x = 0; x < leng; ++x)
>> +        {
>> +            plen = BLEN(&b->c2.bufs[x]);
>> +            if (plen < 1) { c->c2.bufs[x].len = 0; }
>> +            else { c->c2.bufs[x] = b->c2.bufs[x]; }
>> +        }
>> +        c->c2.buffers->bufs_indx = b->c2.buffers->bufs_indx;
>> +        b->c2.buffers->bufs_indx = -1;
>> +    }
>> +}
>> +
>>  static void
>>  check_tls_errors_co(struct context *c)
>>  {
>> @@ -605,6 +634,21 @@ buffer_turnover(const uint8_t *orig_buf, struct
>> buffer *dest_stub, struct buffer
>>      }
>>  }
>>
>> +uint8_t *buff_prepsize(uint8_t *buff, int *size)
>> +{
>> +    buff[0] = ((*size >> 8) & 0xff);
>> +    buff[1] = ((*size >> 0) & 0xff);
>> +    buff += 2;
>> +    return buff;
>> +}
>> +
>> +uint8_t *buff_postsize(uint8_t *buff, int *size)
>> +{
>> +    *size = ((buff[0] << 8) + (buff[1] << 0));
>> +    buff += 2;
>> +    return buff;
>> +}
>> +
>>  /*
>>   * Compress, fragment, encrypt and HMAC-sign an outgoing packet.
>>   * Input: c->c2.buf
>> @@ -1031,6 +1075,7 @@ process_incoming_link_part1(struct context *c,
>> struct link_socket_info *lsi, boo
>>          fprintf(stderr, "R");
>>      }
>>  #endif
>> +
>>      msg(D_LINK_RW, "%s READ [%d] from %s: %s", proto2ascii(lsi->proto,
>> lsi->af, true),
>>          BLEN(&c->c2.buf), print_link_socket_actual(&c->c2.from, &gc),
>> PROTO_DUMP(&c->c2.buf, &gc));
>>
>> @@ -1211,6 +1256,23 @@ process_incoming_link_part2(struct context *c,
>> struct link_socket_info *lsi,
>>      }
>>  }
>>
>> +void process_incoming_link_part3(struct context *c)
>> +{
>> +    int leng = BLEN(&c->c2.to_tun);
>> +    if (leng > 0)
>> +    {
>> +        if (check_bulk_mode(c))
>> +        {
>> +            c->c2.buffers->send_tun_max.offset = TUN_BAT_OFF;
>> +            c->c2.buffers->send_tun_max.len = leng;
>> +            bcopy(BPTR(&c->c2.to_tun),
>> BPTR(&c->c2.buffers->send_tun_max), leng);
>> +            //dmsg(M_INFO, "FWD BAT LINK 0 [%d] [%d] [%d] [%d] [%d]",
>> BLEN(&c->c2.buf), BLEN(&c->c2.to_tun), BLEN(&c->c2.buffers->read_link_buf),
>> BLEN(&c->c2.buffers->read_link_buf), BLEN(&c->c2.buffers->send_tun_max));
>> +            c->c2.to_tun.offset += 2;
>> +            c->c2.buf.offset += 2;
>> +        }
>> +    }
>> +}
>> +
>>  static void
>>  process_incoming_link(struct context *c, struct link_socket *sock)
>>  {
>> @@ -1221,6 +1283,7 @@ process_incoming_link(struct context *c, struct
>> link_socket *sock)
>>
>>      process_incoming_link_part1(c, lsi, false);
>>      process_incoming_link_part2(c, lsi, orig_buf);
>> +    process_incoming_link_part3(c);
>>
>>      perf_pop();
>>  }
>> @@ -1321,7 +1384,7 @@ process_incoming_dco(struct context *c)
>>   */
>>
>>  void
>> -read_incoming_tun(struct context *c)
>> +read_incoming_tun_part2(struct context *c)
>>  {
>>      /*
>>       * Setup for read() call on TUN/TAP device.
>> @@ -1382,6 +1445,55 @@ read_incoming_tun(struct context *c)
>>      perf_pop();
>>  }
>>
>> +void read_incoming_tun_part3(struct context *c)
>> +{
>> +    fd_set rfds;
>> +    struct timeval timo;
>> +    if (check_bulk_mode(c))
>> +    {
>> +        int plen = 0;
>> +        int fdno = c->c1.tuntap->fd;
>> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
>> +        {
>> +            int leng = plen;
>> +            int indx = (c->c2.buffers->bufs_indx + 1);
>> +            if (indx >= TUN_BAT_MIN) { break; }
>> +            if (leng < 1)
>> +            {
>> +                FD_ZERO(&rfds);
>> +                FD_SET(fdno, &rfds);
>> +                timo.tv_sec = 0;
>> +                timo.tv_usec = 0;
>> +                select(fdno+1, &rfds, NULL, NULL, &timo);
>> +                if (FD_ISSET(fdno, &rfds))
>> +                {
>> +                    read_incoming_tun_part2(c);
>> +                    plen = BLEN(&c->c2.buf);
>> +                } else { break; }
>> +            }
>> +            //dmsg(M_INFO, "FWD BAT READ 0 [%d] [%d] [%d] [%d] [%d]",
>> c->c2.buffers->bufs_indx + 1, fdno, BLEN(&c->c2.buf),
>> BLEN(&c->c2.buffers->read_tun_buf), BLEN(&c->c2.buffers->read_tun_max));
>> +            leng = plen;
>> +            if (leng > 0)
>> +            {
>> +                c->c2.buffers->read_tun_bufs[indx].offset = TUN_BAT_OFF;
>> +                c->c2.buffers->read_tun_bufs[indx].len = leng;
>> +                bcopy(BPTR(&c->c2.buf),
>> BPTR(&c->c2.buffers->read_tun_bufs[indx]), leng);
>> +                c->c2.bufs[indx] = c->c2.buffers->read_tun_bufs[indx];
>> +                c->c2.buffers->bufs_indx = indx;
>> +            } else { break; }
>> +            plen = 0;
>> +        }
>> +    }
>> +}
>> +
>> +void read_incoming_tun(struct context *c)
>> +{
>> +    if (c->c2.frame.bulk_size <= 0) {
>> +        read_incoming_tun_part2(c);
>> +    }
>> +    read_incoming_tun_part3(c);
>> +}
>> +
>>  /**
>>   * Drops UDP packets which OS decided to route via tun.
>>   *
>> @@ -1469,7 +1581,7 @@ drop_if_recursive_routing(struct context *c, struct
>> buffer *buf)
>>   */
>>
>>  void
>> -process_incoming_tun(struct context *c, struct link_socket *out_sock)
>> +process_incoming_tun_part2(struct context *c, struct link_socket
>> *out_sock)
>>  {
>>      struct gc_arena gc = gc_new();
>>
>> @@ -1488,7 +1600,7 @@ process_incoming_tun(struct context *c, struct
>> link_socket *out_sock)
>>  #endif
>>
>>      /* Show packet content */
>> -    dmsg(D_TUN_RW, "TUN READ [%d]", BLEN(&c->c2.buf));
>> +    dmsg(D_TUN_RW, "TUN READ [%d] [%d]", BLEN(&c->c2.buf),
>> c->c2.frame.buf.payload_size);
>>
>>      if (c->c2.buf.len > 0)
>>      {
>> @@ -1512,7 +1624,9 @@ process_incoming_tun(struct context *c, struct
>> link_socket *out_sock)
>>      }
>>      if (c->c2.buf.len > 0)
>>      {
>> +        if ((c->c2.buffers == NULL) || (c->c2.buffers->flag_ciph != -2))
>> {
>>          encrypt_sign(c, true);
>> +        }
>>      }
>>      else
>>      {
>> @@ -1522,6 +1636,60 @@ process_incoming_tun(struct context *c, struct
>> link_socket *out_sock)
>>      gc_free(&gc);
>>  }
>>
>> +void process_incoming_tun_part3(struct context *c, struct link_socket
>> *out_sock)
>> +{
>> +    if (check_bulk_mode(c))
>> +    {
>> +        c->c2.buffers->flag_ciph = -2;
>> +        c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
>> +        c->c2.buffers->read_tun_max.len = 0;
>> +        uint8_t *temp = BPTR(&c->c2.buffers->read_tun_max);
>> +        int plen = 0, fdno = c->c1.tuntap->fd;
>> +        int maxl = 0, leng = (c->c2.buffers->bufs_indx + 1);
>> +        if ((fdno > 0) && (leng > 0))
>> +        {
>> +            for (int x = 0; x < leng; ++x)
>> +            {
>> +                c->c2.buf = c->c2.bufs[x];
>> +                //dmsg(M_INFO, "FWD BAT INPT 0 [%d] [%d] [%d] [%d]
>> [%d]", x, fdno, BLEN(&c->c2.buf), BLEN(&c->c2.buffers->read_tun_buf),
>> BLEN(&c->c2.bufs[x]));
>> +                process_incoming_tun_part2(c, out_sock);
>> +                if (BLEN(&c->c2.buf) < 1)
>> +                {
>> +                    c->c2.bufs[x].len = 0;
>> +                }
>> +            }
>> +            for (int x = 0; x < leng; ++x)
>> +            {
>> +                plen = c->c2.bufs[x].len;
>> +                if (plen > 0)
>> +                {
>> +                    temp = buff_prepsize(temp, &plen);
>> +                    bcopy(BPTR(&c->c2.bufs[x]), temp, plen);
>> +                    temp += plen; maxl += (plen + 2);
>> +                }
>> +            }
>> +            if (maxl > 0)
>> +            {
>> +                c->c2.buffers->read_tun_max.offset = TUN_BAT_OFF;
>> +                c->c2.buffers->read_tun_max.len = maxl;
>> +                c->c2.buf = c->c2.buffers->read_tun_max;
>> +                //dmsg(M_INFO, "FWD BAT INPT 1 [%d] [%d] [%d] [%d]
>> [%d]", maxl, fdno, BLEN(&c->c2.buf), BLEN(&c->c2.buffers->read_tun_buf),
>> BLEN(&c->c2.buffers->read_tun_max));
>> +                encrypt_sign(c, true);
>> +            }
>> +        }
>> +        c->c2.buffers->bufs_indx = -1;
>> +        c->c2.buffers->flag_ciph = -1;
>> +    }
>> +}
>> +
>> +void process_incoming_tun(struct context *c, struct link_socket
>> *out_sock)
>> +{
>> +    if (c->c2.frame.bulk_size <= 0) {
>> +        process_incoming_tun_part2(c, out_sock);
>> +    }
>> +    process_incoming_tun_part3(c, out_sock);
>> +}
>> +
>>  /**
>>   * Forges a IPv6 ICMP packet with a no route to host error code from the
>>   * IPv6 packet in buf and sends it directly back to the client via the
>> tun
>> @@ -1748,7 +1916,7 @@ process_outgoing_link(struct context *c, struct
>> link_socket *sock)
>>
>>      perf_push(PERF_PROC_OUT_LINK);
>>
>> -    if (c->c2.to_link.len > 0 && c->c2.to_link.len <=
>> c->c2.frame.buf.payload_size)
>> +    if (c->c2.to_link.len > 0 && (c->c2.to_link.len <=
>> c->c2.frame.buf.payload_size || c->c2.frame.bulk_size > 0))
>>      {
>>          /*
>>           * Setup for call to send/sendto which will send
>> @@ -1793,6 +1961,7 @@ process_outgoing_link(struct context *c, struct
>> link_socket *sock)
>>                  fprintf(stderr, "W");
>>              }
>>  #endif
>> +
>>              msg(D_LINK_RW, "%s WRITE [%d] to %s: %s",
>>                  proto2ascii(sock->info.proto, sock->info.af, true),
>> BLEN(&c->c2.to_link),
>>                  print_link_socket_actual(c->c2.to_link_addr, &gc),
>> PROTO_DUMP(&c->c2.to_link, &gc));
>> @@ -1892,7 +2061,7 @@ process_outgoing_link(struct context *c, struct
>> link_socket *sock)
>>   */
>>
>>  void
>> -process_outgoing_tun(struct context *c, struct link_socket *in_sock)
>> +process_outgoing_tun_part2(struct context *c, struct link_socket
>> *in_sock)
>>  {
>>      /*
>>       * Set up for write() call to TUN/TAP
>> @@ -1912,7 +2081,7 @@ process_outgoing_tun(struct context *c, struct
>> link_socket *in_sock)
>>      process_ip_header(c, PIP_MSSFIX | PIPV4_EXTRACT_DHCP_ROUTER |
>> PIPV4_CLIENT_NAT | PIP_OUTGOING,
>>                        &c->c2.to_tun, in_sock);
>>
>> -    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size)
>> +    if (c->c2.to_tun.len <= c->c2.frame.buf.payload_size ||
>> c->c2.frame.bulk_size > 0)
>>      {
>>          /*
>>           * Write to TUN/TAP device.
>> @@ -1925,7 +2094,8 @@ process_outgoing_tun(struct context *c, struct
>> link_socket *in_sock)
>>              fprintf(stderr, "w");
>>          }
>>  #endif
>> -        dmsg(D_TUN_RW, "TUN WRITE [%d]", BLEN(&c->c2.to_tun));
>> +
>> +        dmsg(D_TUN_RW, "TUN WRITE [%d] [%d]", BLEN(&c->c2.to_tun),
>> c->c2.frame.buf.payload_size);
>>
>>  #ifdef PACKET_TRUNCATION_CHECK
>>          ipv4_packet_size_verify(BPTR(&c->c2.to_tun),
>> BLEN(&c->c2.to_tun), TUNNEL_TYPE(c->c1.tuntap),
>> @@ -1981,6 +2151,39 @@ process_outgoing_tun(struct context *c, struct
>> link_socket *in_sock)
>>      perf_pop();
>>  }
>>
>> +void process_outgoing_tun_part3(struct context *c, struct link_socket
>> *in_sock)
>> +{
>> +    if (check_bulk_mode(c))
>> +    {
>> +        int maxl = 0, plen = 0;
>> +        int leng = BLEN(&c->c2.buffers->send_tun_max);
>> +        uint8_t *temp = BPTR(&c->c2.buffers->send_tun_max);
>> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
>> +        {
>> +            temp = buff_postsize(temp, &plen);
>> +            if ((leng > 0) && (plen > 0) && ((maxl + plen) < leng))
>> +            {
>> +                c->c2.to_tun = c->c2.buffers->to_tun_max;
>> +                c->c2.to_tun.offset = TUN_BAT_OFF;
>> +                c->c2.to_tun.len = plen;
>> +                bcopy(temp, BPTR(&c->c2.to_tun), plen);
>> +                temp += plen; maxl += (plen + 2);
>> +                //dmsg(M_INFO, "FWD BAT OUTP 1 [%d] [%d] [%d] [%d]", x,
>> BLEN(&c->c2.buf), BLEN(&c->c2.to_tun), BLEN(&c->c2.buffers->read_link_buf));
>> +                process_outgoing_tun_part2(c, in_sock);
>> +            } else { break; }
>> +        }
>> +        buf_reset(&c->c2.to_tun);
>> +    }
>> +}
>> +
>> +void process_outgoing_tun(struct context *c, struct link_socket *in_sock)
>> +{
>> +    if (c->c2.frame.bulk_size <= 0) {
>> +        process_outgoing_tun_part2(c, in_sock);
>> +    }
>> +    process_outgoing_tun_part3(c, in_sock);
>> +}
>> +
>>  void
>>  pre_select(struct context *c)
>>  {
>> diff --git a/src/openvpn/forward.h b/src/openvpn/forward.h
>> index d5641491..9fda1583 100644
>> --- a/src/openvpn/forward.h
>> +++ b/src/openvpn/forward.h
>> @@ -79,6 +79,8 @@ void pre_select(struct context *c);
>>
>>  void process_io(struct context *c, struct link_socket *sock);
>>
>> +void xfer_io(struct context *c, struct context *b);
>> +
>>
>>  /**********************************************************************/
>>  /**
>> @@ -196,6 +198,8 @@ bool process_incoming_link_part1(struct context *c,
>> struct link_socket_info *lsi
>>  void process_incoming_link_part2(struct context *c, struct
>> link_socket_info *lsi,
>>                                   const uint8_t *orig_buf);
>>
>> +void process_incoming_link_part3(struct context *c);
>> +
>>  /**
>>   * Transfers \c float_sa data extracted from an incoming DCO
>>   * PEER_FLOAT_NTF to \c out_osaddr for later processing.
>> diff --git a/src/openvpn/init.c b/src/openvpn/init.c
>> index 40ae2c8c..0849dfce 100644
>> --- a/src/openvpn/init.c
>> +++ b/src/openvpn/init.c
>> @@ -2971,6 +2971,10 @@ frame_finalize_options(struct context *c, const
>> struct options *o)
>>      tailroom += COMP_EXTRA_BUFFER(payload_size);
>>  #endif
>>
>> +    if (frame->bulk_size > 0) {
>> +        payload_size = frame->tun_mtu;
>> +    }
>> +
>>      frame->buf.payload_size = payload_size;
>>      frame->buf.headroom = headroom;
>>      frame->buf.tailroom = tailroom;
>> @@ -3473,6 +3477,9 @@ do_init_frame_tls(struct context *c)
>>      if (c->c2.tls_multi)
>>      {
>>          tls_multi_init_finalize(c->c2.tls_multi, c->options.ce.tls_mtu);
>> +        if (c->c2.frame.bulk_size > 0) {
>> +            c->c2.tls_multi->opt.frame.buf.payload_size =
>> c->c2.frame.tun_mtu;
>> +        }
>>          ASSERT(c->c2.tls_multi->opt.frame.buf.payload_size <=
>> c->c2.frame.buf.payload_size);
>>          frame_print(&c->c2.tls_multi->opt.frame, D_MTU_INFO, "Control
>> Channel MTU parms");
>>
>> @@ -3536,6 +3543,14 @@ do_init_frame(struct context *c)
>>          c->c2.frame.extra_tun += c->options.ce.tun_mtu_extra;
>>      }
>>
>> +    /*
>> +     * Adjust bulk size based on the --bulk-mode parameter.
>> +     */
>> +    if (c->options.ce.bulk_mode)
>> +    {
>> +        c->c2.frame.bulk_size = c->options.ce.tun_mtu;
>> +    }
>> +
>>      /*
>>       * Fill in the blanks in the frame parameters structure,
>>       * make sure values are rational, etc.
>> @@ -3676,9 +3691,40 @@ init_context_buffers(const struct frame *frame)
>>
>>      size_t buf_size = BUF_SIZE(frame);
>>
>> +    if (frame->bulk_size > 0) {
>> +        buf_size = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu,
>> frame->buf.headroom + frame->buf.tailroom);
>> +    }
>> +
>> +    dmsg(M_INFO, "MEM NEW [%ld] [%d+%d+%d]", buf_size,
>> frame->buf.headroom, frame->buf.payload_size, frame->buf.tailroom);
>> +
>>      b->read_link_buf = alloc_buf(buf_size);
>>      b->read_tun_buf = alloc_buf(buf_size);
>>
>> +    if (frame->bulk_size > 0) {
>> +        for (int x = 0; x < TUN_BAT_MAX; ++x)
>> +        {
>> +            size_t part_size = BUF_SIZE(frame);
>> +            b->read_tun_bufs[x] = alloc_buf(part_size);
>> +            b->read_tun_bufs[x].offset = TUN_BAT_OFF;
>> +            b->read_tun_bufs[x].len = 0;
>> +        }
>> +
>> +        b->read_tun_max = alloc_buf(buf_size);
>> +        b->read_tun_max.offset = TUN_BAT_OFF;
>> +        b->read_tun_max.len = 0;
>> +
>> +        b->send_tun_max = alloc_buf(buf_size);
>> +        b->send_tun_max.offset = TUN_BAT_OFF;
>> +        b->send_tun_max.len = 0;
>> +
>> +        b->to_tun_max = alloc_buf(buf_size);
>> +        b->to_tun_max.offset = TUN_BAT_OFF;
>> +        b->to_tun_max.len = 0;
>> +    }
>> +
>> +    b->bufs_indx = -1;
>> +    b->flag_ciph = -1;
>> +
>>      b->aux_buf = alloc_buf(buf_size);
>>
>>      b->encrypt_buf = alloc_buf(buf_size);
>> @@ -3701,6 +3747,16 @@ free_context_buffers(struct context_buffers *b)
>>          free_buf(&b->read_tun_buf);
>>          free_buf(&b->aux_buf);
>>
>> +        if (b->to_tun_max.data) {
>> +            free_buf(&b->to_tun_max);
>> +            free_buf(&b->send_tun_max);
>> +            free_buf(&b->read_tun_max);
>> +            for (int x = 0; x < TUN_BAT_MAX; ++x)
>> +            {
>> +                free_buf(&b->read_tun_bufs[x]);
>> +            }
>> +        }
>> +
>>  #ifdef USE_COMP
>>          free_buf(&b->compress_buf);
>>          free_buf(&b->decompress_buf);
>> diff --git a/src/openvpn/mtu.c b/src/openvpn/mtu.c
>> index a419e32d..7e35c837 100644
>> --- a/src/openvpn/mtu.c
>> +++ b/src/openvpn/mtu.c
>> @@ -41,9 +41,15 @@ void
>>  alloc_buf_sock_tun(struct buffer *buf, const struct frame *frame)
>>  {
>>      /* allocate buffer for overlapped I/O */
>> -    *buf = alloc_buf(BUF_SIZE(frame));
>> +    size_t alen = BUF_SIZE(frame);
>> +    size_t blen = frame->buf.payload_size;
>> +    if (frame->bulk_size > 0) {
>> +        alen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_OFF);
>> +        blen = BAT_SIZE(TUN_BAT_MAX, frame->tun_mtu, TUN_BAT_NOP);
>> +    }
>> +    *buf = alloc_buf(alen);
>>      ASSERT(buf_init(buf, frame->buf.headroom));
>> -    buf->len = frame->buf.payload_size;
>> +    buf->len = blen;
>>      ASSERT(buf_safe(buf, 0));
>>  }
>>
>> diff --git a/src/openvpn/mtu.h b/src/openvpn/mtu.h
>> index 925ef0bf..eb799fb3 100644
>> --- a/src/openvpn/mtu.h
>> +++ b/src/openvpn/mtu.h
>> @@ -58,6 +58,14 @@
>>   */
>>  #define TUN_MTU_MIN 100
>>
>> +/*
>> + * Bulk mode static define values.
>> + */
>> +#define TUN_BAT_MIN        6
>> +#define TUN_BAT_MAX        9
>> +#define TUN_BAT_OFF        256
>> +#define TUN_BAT_NOP        0
>> +
>>  /*
>>   * Default MTU of network over which tunnel data will pass by TCP/UDP.
>>   */
>> @@ -152,6 +160,10 @@ struct frame
>>                              *   which defaults to 0 for tun and 32
>>                              *   (\c TAP_MTU_EXTRA_DEFAULT) for tap.
>>                              *   */
>> +
>> +    int bulk_size;              /**< Signal to the init frame function
>> +                                 *   to allow for bulk mode TCP
>> transfers.
>> +                                 *   */
>>  };
>>
>>  /* Forward declarations, to prevent includes */
>> @@ -171,6 +183,7 @@ struct options;
>>   * larger than the headroom.
>>   */
>>  #define BUF_SIZE(f) ((f)->buf.headroom + (f)->buf.payload_size +
>> (f)->buf.tailroom)
>> +#define BAT_SIZE(a, b, c) ((a * b) + c)
>>
>>  /*
>>   * Function prototypes.
>> diff --git a/src/openvpn/multi.c b/src/openvpn/multi.c
>> index e1ce32ab..9e089703 100644
>> --- a/src/openvpn/multi.c
>> +++ b/src/openvpn/multi.c
>> @@ -3414,6 +3414,7 @@ multi_process_incoming_link(struct multi_context
>> *m, struct multi_instance *inst
>>                  }
>>
>>                  process_incoming_link_part2(c, lsi, orig_buf);
>> +                process_incoming_link_part3(c);
>>              }
>>              perf_pop();
>>
>> @@ -3558,9 +3559,7 @@ multi_process_incoming_tun(struct multi_context *m,
>> const unsigned int mpp_flags
>>          const int dev_type = TUNNEL_TYPE(m->top.c1.tuntap);
>>          int16_t vid = 0;
>>
>> -#ifdef MULTI_DEBUG_EVENT_LOOP
>> -        printf("TUN -> TCP/UDP [%d]\n", BLEN(&m->top.c2.buf));
>> -#endif
>> +        msg(D_MULTI_DEBUG, "TUN -> TCP/UDP [%d]", BLEN(&m->top.c2.buf));
>>
>>          if (m->pending)
>>          {
>> @@ -3610,6 +3609,8 @@ multi_process_incoming_tun(struct multi_context *m,
>> const unsigned int mpp_flags
>>                          {
>>                              /* transfer packet pointer from top-level
>> context buffer to instance */
>>                              c->c2.buf = m->top.c2.buf;
>> +                            /* todo determine if to call this
>> (multi_process_incoming_tun) for each bulk item read? */
>> +                            xfer_io(c, &m->top);
>>                          }
>>                          else
>>                          {
>> diff --git a/src/openvpn/openvpn.h b/src/openvpn/openvpn.h
>> index cd99cd40..21fa8967 100644
>> --- a/src/openvpn/openvpn.h
>> +++ b/src/openvpn/openvpn.h
>> @@ -112,6 +112,14 @@ struct context_buffers
>>       */
>>      struct buffer read_link_buf;
>>      struct buffer read_tun_buf;
>> +
>> +    struct buffer read_tun_bufs[TUN_BAT_MAX];
>> +    struct buffer read_tun_max;
>> +    struct buffer send_tun_max;
>> +    struct buffer to_tun_max;
>> +
>> +    int bufs_indx;
>> +    int flag_ciph;
>>  };
>>
>>  /*
>> @@ -376,6 +384,8 @@ struct context_2
>>      struct buffer to_tun;
>>      struct buffer to_link;
>>
>> +    struct buffer bufs[TUN_BAT_MAX];
>> +
>>      /* should we print R|W|r|w to console on packet transfers? */
>>      bool log_rw;
>>
>> diff --git a/src/openvpn/options.c b/src/openvpn/options.c
>> index c54032d8..041d17d0 100644
>> --- a/src/openvpn/options.c
>> +++ b/src/openvpn/options.c
>> @@ -304,6 +304,7 @@ static const char usage_message[] =
>>      "                  'maybe' -- Use per-route hints\n"
>>      "                  'yes'   -- Always DF (Don't Fragment)\n"
>>      "--mtu-test      : Empirically measure and report MTU.\n"
>> +    "--bulk-mode     : Use bulk TUN/TCP reads/writes.\n"
>>  #ifdef ENABLE_FRAGMENT
>>      "--fragment max  : Enable internal datagram fragmentation so that no
>> UDP\n"
>>      "                  datagrams are sent which are larger than max
>> bytes.\n"
>> @@ -3005,6 +3006,9 @@ options_postprocess_mutate_ce(struct options *o,
>> struct connection_entry *ce)
>>              ce->tun_mtu_extra_defined = true;
>>              ce->tun_mtu_extra = TAP_MTU_EXTRA_DEFAULT;
>>          }
>> +        if (ce->proto != PROTO_TCP && ce->proto != PROTO_TCP_SERVER &&
>> ce->proto != PROTO_TCP_CLIENT) {
>> +            ce->bulk_mode = false;
>> +        }
>>      }
>>
>>      /*
>> @@ -9926,6 +9930,10 @@ add_option(struct options *options, char *p[],
>> bool is_inline, const char *file,
>>              goto err;
>>          }
>>      }
>> +    else if (streq(p[0], "bulk-mode"))
>> +    {
>> +        options->ce.bulk_mode = true;
>> +    }
>>      else
>>      {
>>          int i;
>> diff --git a/src/openvpn/options.h b/src/openvpn/options.h
>> index 38e67c8d..d1b0586d 100644
>> --- a/src/openvpn/options.h
>> +++ b/src/openvpn/options.h
>> @@ -174,6 +174,9 @@ struct connection_entry
>>
>>      /* Allow only client that support resending the wrapped client key */
>>      bool tls_crypt_v2_force_cookie;
>> +
>> +    /* Bulk mode allows for multiple tun reads + larger tcp writes */
>> +    bool bulk_mode;
>>  };
>>
>>  struct remote_entry
>> --
>> 2.39.5 (Apple Git-154)
>>
>>

Am 08.08.2025 um 16:04 schrieb Jon Chiappetta:
> Hi Arne,
>
> You are correct, I didn't do a very good job of explaining the code in 
> my blog post, I usually keep those short with more screen captures 
> because I figure that not many people would actually take the time to 
> read through it there. Also, I didn't really add many comments either 
> but I did try to copy the present style of the code to try and make it 
> match and be more consistent throughout, even if not perfect yet. I'm 
> still testing out the change myself in my own home setup here to see 
> if I run into any bad edge cases along the way.
>
> I can always try to explain the different code parts as I am indeed 
> modifying the core parts of the read and write operations for tun and 
> tcp so it's a big change to make to the code base. Basically this 
> change is important to me in particular because of my setup and 
> requirements in specific. I have WiFi LAN clients which all assume a 
> 1500 byte MTU on their side and I have a router WAN client which 
> enforces a 1500 byte MTU on the internet's side. In the middle of my 
> core network is a VPN box and almost every VPN software will operate 
> in UDP mode with a sub-1500 MTU in the middle of this network 
> pipeline. This is not a good design to have in general as I don't want 
> to waste cycles fragmenting and/or compressing the data into smaller 
> sized UDP packets. With the code change I am presenting, I am able to 
> specify a true 1500 byte VPN MTU interface with the exact matching 
> 1500 byte read calls to the TUN interface itself (the code base had to 
> be modified to allow for this because it was adding onto the 
> payload_size used in the read call which I didn't want as I am 
> operating on exact multiples of 1500 bytes in specific).

OpenVPN already supports arbitrary MTU sizes just fine. The default is 
even MTU 1500. mssfix is also enabled by default to avoid fragmentation, 
which especially for UDP is bad. So I am not sure what you are modifying 
here because what you are describing is already well supported by OpenVPN.


>
> With this change, my network pipeline is a true 1500 byte MTU which 
> matches all the way from the client side to the vpn link to the 
> internet side and to the server side (end to end to end to end). In 
> addition, I also added the ability to batch together multiple 1500 
> byte read calls (specifically 6 x 1500 bytes into 9000 bytes) into one 
> single encrypt sign call and one single TCP write call.

So this needs a different wire format and some extra headers/framing as 
you need to split this jumbo packet again into 1500 bytes packets on the 
receiver.

In order to justify introducing a new framing/packet format, a clear 
benefit should be shown. It would be for example to show what the 
performance benefit here actually is, e.g. by doing a simple test.


do a simple client server OpenVPN connection and run iperf between 
client and server

- without VPN

-  unmodified OpenVPN with --mtu 1500

- unmodified OpenVPN with --mtu 9000

- your approach

That would give an indication what kind of gains we are talking here about.

> This allows the encryption method to operate only once on a much 
> larger payload size as well as allow the linux kernel to efficiently 
> transfer the data with order and delivery guaranteed as fast as 
> possible. The code base had to be modified to allow for all of this as 
> well as it was preventing me from performing this much larger sized 
> ssl sign and encrypt + tcp read and write (the code base assumes you 
> are operating on only 1 tun read call worth of data at a time 
> everywhere).
>
> This is exactly why I prefer using TCP to tunnel encrypted network 
> data as my solution provided can properly set a full sized 1500 byte 
> MTU as well as perform an exact matching read call of 1500 bytes to 
> get the full amount of data from the interface and then bulk it 
> together to efficiently encrypt it and then use the magic of TCP to 
> transfer that data all at once as quickly as possible without any need 
> for fragmentation or compression. I don't think any other VPN product 
> on the market offers that kind of functionality as far as I am aware 
> as most other VPN products use a smaller sized MTU as well as the 
> packet size limitations of UDP. I believe that this could be a 
> distinguishing feature for OpenVPN as well as automatically solve some 
> of the issues that folks run into when inserting a VPN appliance into 
> the middle of their network setups.

In TCP mode can you get away with a lot of thing you are doing. In UDP 
mode, having tunnel inside MTU of 1500 or pushing large fragmented 
IP/UDP packets will be very detrimental to VPN performance.


>
> I've been running this change on my own setup to at least make sure it 
> works and it seems to be running pretty nicely so far. I haven't 
> experienced any fragmentation or performance issues as any sized data 
> that comes off the clients LAN side is fully taken care of now through 
> the VPN side and onto the WAN and server side.

This whole MTU fragment and tun must have 1500 MTU to not see these 
problems is not a thing that I observe and also not a logical conclusion 
for me.

>
> If this is something you are not interested in I can understand that, 
> I can stop posting here and the most I can do is at least submit a 
> pull request in case anyone in the future is indeed interested in such 
> work. It'd be nice to contribute to a good quality open source project 
> that I have used for many many years and something which may help 
> solve other community member's issues with regards to the small sized 
> MTU + UDP problem which does exist in practice and really hampers 
> connections along the way in a network design.

It would be good to understand what you are actually referring to here. 
Small sized MTU + UDP problem is quite vague as there are multiple 
things that can cause probelems with an UDP VPN.

>
> I also don't mind explaining my code parts if you actually want, I 
> just need to take time to write them out and describe what they are 
> doing and why. As you can see, I am trying to achieve a very specific 
> and exact design goal that the code base wasn't originally allowing 
> for, so I had to make some modifications to be able to accomplish it.
>