From patchwork Thu Oct 1 08:41:20 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5426 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6ac1:b0:8b3:6e77:b38b with SMTP id v1csp1809819maw; Thu, 1 Oct 2026 01:43:32 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwgd94izFrFX17r4cR0aXWbKLe02vTdTB55du3rrfid2dEYSEhmRVpwPmSs3qu6zKeDjtyQcEWu3sA=@openvpn.net X-Received: by 2002:a05:6820:1905:b0:6d7:d91d:d77d with SMTP id 006d021491bc7-6dcf5a252a6mr4485790eaf.58.1790844118954; Thu, 01 Oct 2026 01:41:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790844118; cv=none; d=google.com; s=arc-20260327; b=EDjcW+vRnLwMJhYTUSMQLavMXqCkgj8SYPbmr3gc3kCA/2JXVpei4TiYmDT8HkoV1B vvGAdNiFNCaBGGDqbhchn1aXWW2yj1QP54gMFuaqMBJ5qvudF22WgOaHh5fsGdkXpHyz kETvPOyvjk1RXo/femsIucUy2z6/4IBUBY3h8wzhiXMpqHx82egI+wzpzoT1R9hWPLap uoh+36dhpOBkdrb3ViZhm0KSpdTQ8xbQeMb+awIZuP0m6G5RYsvfZoUU/kvwoKg7Gaip cSDLnGnbVbchpvWeZG13WDzQec3y1dS9huiTJOtxpx9T/4yv8bmuHbx9N9k8gdLlBmft bGEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=9Y9soyHYxCfkrXbCiOt3Ha1FjQcnK8weo+LpLMGJsOw=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=d2oi+S2v7Jbu4jorwIjnO7taBKsmx+DfN7XpkhyxBaNEtb4UVxMrkDwVHJ+qSRZP4n Dx3UDalC6xpT48JIkRvgPwlGzTdS87mJ6jiTgQMClJGGH4idYSXDdyPodk5jCZ41Nrtw CBZPLAG7Gc2v6Tg2KMQ24qEwEcrKVNee7jx32S3jZJLpz2RvkWm412GiI/kC7XG64LUk gL+ZKTGdydZ3jVhCFLEn1rhfDKMU1AftRIXSZHE+p0zddw84pyOfQuLR4pMTcGQfv6ev mBPV/7MQbVlHvrXAe0TrGKsC/7VQ+zrbfOQZrXaMXnszhwpHGEptMkFDcAaxgbljjaHd bRcA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=VSGYk2JG; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Cbel1hlV; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=nB0ZHz5G; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=WJoHpIvg; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 006d021491bc7-6dd9a3ba1bdsi4120784eaf.85.2026.10.01.01.41.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Oct 2026 01:41:58 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=VSGYk2JG; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Cbel1hlV; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=nB0ZHz5G; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=WJoHpIvg; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=9Y9soyHYxCfkrXbCiOt3Ha1FjQcnK8weo+LpLMGJsOw=; b=VSGYk2JGYCUkR//4lj3BfbLmlD iYXDWepmB1X3AC29vfhQrOtSHF0JfhJIp4X/+SVX3VRWVycmilDGyUQ0mbpBIzty5QLFsnpCDGXxp xk0Izonxo0EeD0drP8nptnkaANs+/K6phPp45u7aVdl7LmwzvkIKZacm530f/Z9Lsi/k=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xCCMO-0003jB-Sn; Thu, 01 Oct 2026 08:41:49 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xCCMK-0003ia-JC for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=M+3M5N0W2yUD4QwFybZmQKY1rBi566Rz2HRECDCDDEc=; b=Cbel1hlV4VZl1I7d0Vm+0X+jKy GqYfBuGZXxol/bLogrQ12VtVm1dRaIuihrDOFboIhZa80pj8x3WmGKAi2nAuLPxb6/3Dm4Hu2/eh7 RkK3W9atPnn/v4HblKRBLGylKyeyvpEloD13GYlLNGQHXe0AYb3LeVM8oINESYlM7Y1g=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=M+3M5N0W2yUD4QwFybZmQKY1rBi566Rz2HRECDCDDEc=; b=nB0ZHz5GQo34ql5zo3HriTZPqm QMWpMK7zclGDgTnEPIVyXkTbjKFeQoq2LrVei8MIuXbr4Q6yT3EnHpCY4ZAZde7FZ9u7vyaux/7Qh j9eHWJ3FtSUoGnpdhXueY38/WF/fjT2whWDOdQg6vfVI+PcilmZkGJmCLdEMnpx6qX+o=; Received: from mout-b-206.mailbox.org ([195.10.208.51]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xCCMJ-0002wj-Pn for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:45 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-206.mailbox.org (Postfix) with ESMTPS id 4hwQPW5JlfzSW for ; Thu, 01 Oct 2026 10:41:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1790844095; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=M+3M5N0W2yUD4QwFybZmQKY1rBi566Rz2HRECDCDDEc=; b=WJoHpIvgp4uXnK/ARMTCAaab7+XEM3lhlEx8pQKCyrTzwDxaBxRoBA3/h7sea1Th6n3OIR k6Uyo1I077dK72qTL9byGW2YWhAqY9JLN2175NBXETVnO0D2CpFGBFJqRaltvwFSeoehdQ skFHbrN63ZROtUO4fpL0G5lCeLKOIfIVo6u/QLtd6a1UdlCE3/KOcshIwAfaz6zyfza2D7 EfC0zQFbjuUslfsIavIqvxQGQIyPN2AcYW36apuWKiffSSZjnAJhjNimX8McGhpWX9lQeG fiQduL301ObeAJho8p/qQ23rDuqiXJPll7cl5EGW8yk0zALbYaRFHQhUvP5Cfg== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 1 Oct 2026 10:41:20 +0200 Message-ID: <1bbfde37488ade61928554db3a119525adf7b608.1790843153.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xCCMJ-0002wj-Pn Subject: [Openvpn-devel] [PATCH ovpn net-next 1/8] ovpn: advertise checksum offload for GSO packets X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1877836162555377164 X-GMAIL-MSGID: 1877836162555377164 ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before they reach ovpn because TCP GSO packets normally carry CHECKSUM_PARTIAL state. Advertise NETIF_F_HW_CSUM so the stack can pass such packets to ovpn. Complete partial checksums after any GSO segmentation and before submitting packets for encryption, since the inner packet checksum can no longer be fixed after the packet has been encrypted. Also pass the ovpn feature set to skb_gso_segment with GSO capabilities masked out: this forces software segmentation, but still lets the segmenter preserve supported non-GSO properties such as non-linear skb data instead of needlessly linearizing. Signed-off-by: Ralf Lici --- No changes since RFC v4 https://lore.kernel.org/openvpn-devel/1bbfde37488ade61928554db3a119525adf7b608.1789558856.git.ralf@mandelbit.com/ drivers/net/ovpn/io.c | 20 ++++++++++++++++++-- drivers/net/ovpn/main.c | 2 +- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 9526f8096da6..112067ded401 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -358,6 +358,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; struct sk_buff_head skb_list; + netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; __be16 proto; @@ -392,8 +393,13 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_dst_drop(skb); if (skb_is_gso(skb)) { - segments = skb_gso_segment(skb, 0); - if (IS_ERR(segments)) { + /* force software segmentation, but keep ovpn's non-GSO feature + * bits so the generated segments can preserve non-linear skb + * data where possible + */ + features = netif_skb_features(skb); + segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", netdev_name(dev), ret); @@ -418,6 +424,16 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) continue; } + /* NETIF_F_HW_CSUM requires completing partial checksums */ + if (unlikely(curr->ip_summed == CHECKSUM_PARTIAL && + skb_checksum_help(curr) < 0)) { + net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", + netdev_name(dev)); + ovpn_dev_dstats_tx_dropped(ovpn->dev); + kfree_skb(curr); + continue; + } + /* only count what we actually send */ tx_bytes += curr->len; __skb_queue_tail(&skb_list, curr); diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 0708249e9607..28e1eb06e127 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -157,7 +157,7 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { - netdev_features_t feat = NETIF_F_SG | NETIF_F_GSO | + netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Thu Oct 1 08:41:21 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5427 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6c89:b0:8b5:356c:6cee with SMTP id p9csp1750193may; Thu, 1 Oct 2026 01:43:49 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwD7EEZLZoffAAAY3L7Jzp8ok1eThldwuDRpenKil2MiVFY/36fO+E5lQFXr5c34fI0P33tKrT5AXg=@openvpn.net X-Received: by 2002:a05:6808:eca:b0:4e6:fdeb:934e with SMTP id 5614622812f47-4f1b0d0b5d0mr4087766b6e.6.1790844118525; Thu, 01 Oct 2026 01:41:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790844118; cv=none; d=google.com; s=arc-20260327; b=fZCjKByERqHZMeYPoqOx1U1UicdsXg9FE1TzBpvnW+jDJiszAh1oCpyhkSpGH1zlRy NHZTIc6DKCfkt4OmqEsV1nS2JyxSxtHP7Eu+13Q2DsPU9WPfkSxoGz1om0zlBUTtiAtO 5UQjym7PoKqGLu1ARQqzzdIP0dYZZqHmbEfdJ+hIDZtmZcSV0I+Uvx0/Ujg6f6jWvTMs Puht+3fltAjjK6PNV6f6Nj6Gtmk15a0xOw+K2UdSl44QGnjOxZgeWthIVLw5/Fw14Cs+ aYp6svjVQiQ/ERDqZFmX3yjZuKJ3ocwU/X3yDDePEMpYdDPcHKyB6W9ePeNVCSof9+u+ McHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=jl2AJNI8c817z+i/47LK4XFdnDZYvWbNabUwnW2POV0=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=PvmpIMKI2u+wFONUt3VoY8OXoiNWUVHFbQCyRedGwkUHzHl/AAIIE2RXb8y/cTjFdl M7UMEhILrDs3+wthigR80ijSoj8Glcz2X2idDta65ajTj3gR8EfrQA7dnrcWsBlOIi9+ Rx4UTOqVulQDkcMLMwzH8J2aUDKc2zkw8DUOVKV6NtFAmVTJjB3UCfZGXdBSmCJlRbyV bUm51Mn53xacng6PN2Bg0QUw1H4WxpWnCKiAeHKR7lkgB/eHG+QKcHo7yrijRF/pO9Y6 /0M8RGI/A3IzkNRisl+VFhWMHncS1NXTcROFZljk5JBKqMOZ8t5NH8YdhcWbNb25YJdA KiWQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=bj++wUSK; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=WJ++Rg7e; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b="Lm/XqmGL"; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=tUFTo+4y; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5614622812f47-4f352d24c8bsi3040262b6e.93.2026.10.01.01.41.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Oct 2026 01:41:58 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=bj++wUSK; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=WJ++Rg7e; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b="Lm/XqmGL"; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=tUFTo+4y; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jl2AJNI8c817z+i/47LK4XFdnDZYvWbNabUwnW2POV0=; b=bj++wUSKAbEuMcdFLpxA3djeU0 05LIukcBziUmt8EviAgXNEFAN+WImray/wiq2StXj3VD4foXp1YM2gGkO1r2fTKLUEXpkvVbeoH3H MzaEo2pkXrDli4ycxT80XS30fv1BSmAnJBqmySI5RZYNuomhEbxoejkn4ZWbuM+Az7co=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xCCMR-0000BC-Dd; Thu, 01 Oct 2026 08:41:52 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xCCMK-0000Am-OL for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=L/QYUSx4EvRc/2E6p7CIOHwdh1Za13HtWolk4+IhE/s=; b=WJ++Rg7eTMSQ7HkOP/NiEXKtrK nUH7oTEF7WA8cv8VbfJBCp8P2RiADZb5+oRLgtIjfVMCTVhOlUHcbAaWUzNioP98gSy3hfuRqzc8Z nK0pColKcz8cRxGsGFM6zcGby6kV4u5MEQWAX/c1MNZNbNNqjnC4N1j0TN253nNyQ71o=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=L/QYUSx4EvRc/2E6p7CIOHwdh1Za13HtWolk4+IhE/s=; b=Lm/XqmGLJM0bYUpy+bhB0MvopW TU2BogYo4GaZ7VgLnfSr10m1yq9ZrX39aPWld8NFveDQQv+IN5T6jBk+nV0LsqIPjKF0HF5j6sKEb x9FEHxihkGzKD6+JIgVsVgpQ5YMPHm8v2XscbJMeutkbDm4RdQEPWv2B93Mo17Xor8cE=; Received: from mout-b-206.mailbox.org ([195.10.208.51]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xCCMK-0003Tw-Nd for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:45 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-206.mailbox.org (Postfix) with ESMTPS id 4hwQPX2Lk4zV7 for ; Thu, 01 Oct 2026 10:41:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1790844096; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=L/QYUSx4EvRc/2E6p7CIOHwdh1Za13HtWolk4+IhE/s=; b=tUFTo+4yjhy9tE8fSWASGc65znh0cUFS6CY8m4r0oMmZFOXHXRvaAa8ZO6aSCkzzibyrD7 YxNVqbR6iv73EuiJBhghL+pmVnl7RlBf1c+L0naGPPojD5rx7hKh07fI8BuX935/uzlVqs S0PRCNS2NSCe/OWQMWQFMvmR3dvU8zjQB54O6DFPfTXM5TCssveXNKMHV2TjXmss25HKDP wzf7p/mx7bHBDD2l7Yy4tTIr4Z21r7gbFlEoNDsxFmTSxybFwZJ5YHAcpigtqo0RiDnmQf MVRDHinjjW0oVjip99dPQkkMtDGE4YGRTYq2f+HIsj8d/v+Bfmvo6r4jempe5g== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 1 Oct 2026 10:41:21 +0200 Message-ID: <5baa1d94e29e7b109ffd412b4f1d5d113a319389.1790843153.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hwQPX2Lk4zV7 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_ [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xCCMK-0003Tw-Nd Subject: [Openvpn-devel] [PATCH ovpn net-next 2/8] ovpn: accept frag-list GSO input X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1877836162292767836 X-GMAIL-MSGID: 1877836162292767836 Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_FRAGLIST, so generic transmit validation segments these aggregates before calling ovpn_net_xmit. That segmentation is functionally correct, but causes ovpn_net_xmit to be invoked separately for every resulting packet. Advertise frag-list storage so ovpn receives the aggregate intact and performs protocol validation and destination-to-peer lookup once before segmenting it. Frag-list GSO segmentation recovers the complete child skbs, which can then be encrypted in place and transmitted independently. Rebuilding those children into a replacement UDP GSO aggregate was found to add cost rather than improve throughput. The feature also admits non-GSO frag lists, which describe one packet split across several skbs. Let skb_cow_data preserve small lists directly. If a list exceeds the AEAD scatterlist limit, linearize it and continue rather than rejecting an otherwise valid packet. Signed-off-by: Ralf Lici --- No changes since RFC v4 https://lore.kernel.org/openvpn-devel/5baa1d94e29e7b109ffd412b4f1d5d113a319389.1789558856.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 8 ++++++-- drivers/net/ovpn/main.c | 3 ++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 74eaf6fac2f5..2af493fd5735 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -168,8 +168,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, if (unlikely(nfrags < 0)) return nfrags; - if (unlikely(nfrags + 2 > (MAX_SKB_FRAGS + 2))) - return -ENOSPC; + if (unlikely(nfrags > MAX_SKB_FRAGS)) { + ret = skb_linearize(skb); + if (unlikely(ret)) + return ret; + nfrags = 1; + } /* allocate temporary memory for iv, sg and req */ tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 28e1eb06e127..ac4e0d85e215 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -158,7 +158,8 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | - NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; + NETIF_F_GSO_SOFTWARE | NETIF_F_FRAGLIST | + NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Thu Oct 1 08:41:22 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5423 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6c89:b0:8b5:356c:6cee with SMTP id p9csp1748854may; Thu, 1 Oct 2026 01:41:59 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBw02KyFp10jbnzyR3RtVgiLTuAEGH0YIv4MM2LtiJMCNze2WrY3GdrzIsQaASiKWnSRvmlFkQn9G9c=@openvpn.net X-Received: by 2002:a05:6820:2086:b0:6c9:80d8:a1ef with SMTP id 006d021491bc7-6dcf5b2dcb6mr3708763eaf.44.1790844118636; Thu, 01 Oct 2026 01:41:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790844118; cv=none; d=google.com; s=arc-20260327; b=UZjoF+ZuK4oDROVks9uibeaLvVyolE5oTptF6Ms18vWfniLSDJDs3yz/TEkG52mXfC flq2UB7wQqG0mo/VVPUFeuQj7JqN9foQoetNz5RZ0OaZx04FhFA9+yyd9tHROcJr8ZTr fd3QBzIWP4RJX4xrI5e35U7Sk96msX5iWXoEwh9EGSbWOtTXNqs0QNSmG+qViQNPUeeN G9YbU2R+kZ590ke4it4pjtAccWtS/WqVga1E4AmDyGVlAPPR8Tix8dYU8OA8tzsmfRGc C8M+hX+Zf4NcNnxhV4Bah04jU3BeeQ9rv52ytvg4bRKHZDmd3lLnQ6UOhcubaq39+HzF qiWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=E1Mg+AX4Zjt50QYJG/HkhmXGlh81e907VEAv+FSlpH0=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=UjJti54DTNh9vRi4iw1ayVrqaRo5lfHK8VpY0uY4M3BMZTVrx2ycVNIract2MSJNgV sGdPY24YzkGqyYzzvh8nQYsYqpK+FxS2gM87YBGB9bF5Dl/9gTWHQcP0FWRuvhXcpAwp zEm1J1MGd5z2T+GK5+blTBVcJscvmhIC9thhV+HbCK0bYj4xTpcGvXdigugNIXal0REw eru5Eg8olZo7DGEahGb33sFNb+Q2qAZSApgu0NhDhCSKKfmoGdI5zytqOupCrPydmW3y chBG6I1tkaPwlMV1Ld+Z1Z9A/Zy3XG7+gLyvRw3y7xuZKokG7yzTqYZgCALhrI7qVXRy 2BGQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=N17Hrh5A; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="bxHPcrP/"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=ArBHJ1aN; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=A15+FAsr; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-49e0666c80esi579026fac.299.2026.10.01.01.41.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Oct 2026 01:41:58 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=N17Hrh5A; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="bxHPcrP/"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=ArBHJ1aN; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=A15+FAsr; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=E1Mg+AX4Zjt50QYJG/HkhmXGlh81e907VEAv+FSlpH0=; b=N17Hrh5AW7GW5EFzVKaAl5smye 5moFt7qyVJmmmrOF+TaFc4zmYFjAr2qKmU1Ts+YZk69HbFIYabaTzaOTZw+ovekiG+jrGfGAyjCl7 T+sj5qmgYu25ooGOVz/Q/YgyHjXvAAhrEDaqhNgo9xMbv93DEn9wq+t4z1wt1MFir2c8=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xCCMQ-0006p7-HY; Thu, 01 Oct 2026 08:41:50 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xCCMM-0006or-AN for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:46 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=4OvTzfBSkbfGjHCnVYNb14LgxFreQIl7wtoD0QD1jkQ=; b=bxHPcrP/rj+QObnfXiQEjQtYIc 6otcC87CQOFZGxV3zJ8ZDYuzhL6g7CFQ78SPID/mV6VS936KCtyrtxXyeE6nUtvdDVPGMMBJM/+xE +aEHgjFEGfqWu26ODED8dNFDCSLRmLgjhUFhzdtcFhNEZy4/tZajKvNbJ6lVySNh8DfE=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=4OvTzfBSkbfGjHCnVYNb14LgxFreQIl7wtoD0QD1jkQ=; b=ArBHJ1aN+6fBctB0zi/1i8J2dC 9jh4k6F9NJoYpn/O7mr5e/2LiICLh7sqRWZ6ZJnRIVtiM0R495aWIUwtzH5V1OeqGm/IPLgMrpdk5 iYa3qdY7yEzMsMMLEUH9SUz9UEJC1mbhcQZOWoHAYme7mpey/uBijDX44eXVTcctCbF0=; Received: from mout-b-299.mailbox.org ([195.10.208.53]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xCCML-0003U3-G4 for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:46 +0000 Received: from mout-b-202.mailbox.org (mout-b-202.mailbox.org [195.10.208.62]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-299.mailbox.org (Postfix) with ESMTPS id 4hwQPb13DGz7vkr for ; Thu, 01 Oct 2026 10:41:39 +0200 (CEST) Authentication-Results: outgoing_mbo_mout; dkim=pass header.d=mandelbit.com header.s=MBO0001 header.b=A15+FAsr; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 195.10.208.62 as permitted sender) smtp.mailfrom=ralf@mandelbit.com Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-202.mailbox.org (Postfix) with ESMTPS id 4hwQPX6mZszKm5w for ; Thu, 01 Oct 2026 10:41:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1790844096; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4OvTzfBSkbfGjHCnVYNb14LgxFreQIl7wtoD0QD1jkQ=; b=A15+FAsrCMAEAMoK1Y94JLjimJ66dbKrqRCAKUt12soi/1nWU+iHyl3cErjbVDd/XlwDWz BvjgLGubVeutcSpW/nLKguWAn5F2uAHe3Zl2uAaJE52jOe16z170gR/bqaQuQ4wAX7MsKz hNu+zwpcD9OoobgCIW8rA1c6bqlWmgnuLqCilqwrsNyJTNBBlGCRTIt39Z8PahWAh2zpyW IqZkXvz91fk7DqDj0RaINsHqQqJAiHvDaNazw7S87tz8nSglgyYs5wOhBMz16PJDmR323C Z7r1HWVvvlHZqHLcwwSZ6iLkrCDuOTQ1olBxsKydCo5Uh2JofeaYg+j1SRkHRQ== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 1 Oct 2026 10:41:22 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hwQPb13DGz7vkr X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xCCML-0003U3-G4 Subject: [Openvpn-devel] [PATCH ovpn net-next 3/8] ovpn: refactor AEAD encryption helpers X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1877836162619701184 X-GMAIL-MSGID: 1877836162619701184 Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without changing packet handling. Signed-off-by: Ralf Lici --- No changes since RFC v4 https://lore.kernel.org/openvpn-devel/f075b9782b14d64756c84e132c138482f08287b9.1789558856.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 97 ++++++++++++++++++++-------------- drivers/net/ovpn/io.h | 3 +- drivers/net/ovpn/proto.h | 4 ++ 3 files changed, 63 insertions(+), 41 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 2af493fd5735..30299581422d 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -24,9 +24,6 @@ #include "proto.h" #include "skb.h" -#define OVPN_AUTH_TAG_SIZE 16 -#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE) - #define ALG_NAME_AES "gcm(aes)" #define ALG_NAME_CHACHAPOLY "rfc7539(chacha20,poly1305)" @@ -42,7 +39,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * an AEAD request structure with extra space for SG * and IV. * @tfm: the AEAD cipher handle - * @nfrags: the number of fragments in the skb + * @nents: the number of scatterlist entries * * This function calculates the size of a contiguous memory block that includes * the initialization vector (IV), the AEAD request, and an array of scatterlist @@ -54,7 +51,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * Return: the size of the temporary memory that needs to be allocated */ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, - const unsigned int nfrags) + const unsigned int nents) { unsigned int len = OVPN_NONCE_SIZE; @@ -70,8 +67,8 @@ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, /* round up to the next multiple of the scatterlist alignment */ len = ALIGN(len, __alignof__(struct scatterlist)); - /* add enough space for nfrags + 2 scatterlist entries */ - len += array_size(sizeof(struct scatterlist), nfrags + 2); + /* add enough space for the scatterlist entries */ + len += array_size(sizeof(struct scatterlist), nents); return len; } @@ -135,6 +132,53 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, __alignof__(struct scatterlist)); } +static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, + struct sk_buff *skb, + unsigned int nents, u8 **iv) +{ + struct aead_request *req; + void *tmp; + + /* allocate IV, request and scatterlist entries in one block */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + if (unlikely(!tmp)) + return ERR_PTR(-ENOMEM); + + ovpn_skb_cb(skb)->crypto_tmp = tmp; + *iv = ovpn_aead_crypto_tmp_iv(aead, tmp); + req = ovpn_aead_crypto_tmp_req(aead, *iv); + + return req; +} + +static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + u8 *iv, u8 *data) +{ + u32 pktid, op; + int ret; + + /* obtain packet ID, which is used both as a first + * 4 bytes of nonce and last 4 bytes of associated data. + */ + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + if (unlikely(ret < 0)) + return ret; + + /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes + * nonce + */ + ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); + + /* add the packet opcode and wire nonce as associated data */ + op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); + BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); + *(__force __be32 *)data = htonl(op); + memcpy(data + OVPN_OPCODE_SIZE, iv, OVPN_NONCE_WIRE_SIZE); + + return 0; +} + int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { @@ -143,8 +187,6 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *trailer; struct scatterlist *sg; int nfrags, ret; - u32 pktid, op; - void *tmp; u8 *iv; ovpn_skb_cb(skb)->peer = peer; @@ -175,16 +217,9 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), - GFP_ATOMIC); - if (unlikely(!tmp)) - return -ENOMEM; - - ovpn_skb_cb(skb)->crypto_tmp = tmp; - - iv = ovpn_aead_crypto_tmp_iv(ks->encrypt, tmp); - req = ovpn_aead_crypto_tmp_req(ks->encrypt, iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + if (IS_ERR(req)) + return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); /* sg table: @@ -206,28 +241,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, __skb_push(skb, tag_size); sg_set_buf(sg + ret + 1, skb->data, tag_size); - /* obtain packet ID, which is used both as a first - * 4 bytes of nonce and last 4 bytes of associated data. - */ - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + /* make space for the additional data and push it to the front */ + __skb_push(skb, OVPN_AAD_SIZE); + ret = ovpn_aead_encrypt_header(peer, ks, iv, skb->data); if (unlikely(ret < 0)) return ret; - /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes - * nonce - */ - ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); - - /* make space for packet id and push it to the front */ - __skb_push(skb, OVPN_NONCE_WIRE_SIZE); - memcpy(skb->data, iv, OVPN_NONCE_WIRE_SIZE); - - /* add packet op as head of additional data */ - op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); - __skb_push(skb, OVPN_OPCODE_SIZE); - BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); - *((__force __be32 *)skb->data) = htonl(op); - /* AEAD Additional data */ sg_set_buf(sg, skb->data, OVPN_AAD_SIZE); @@ -281,7 +300,7 @@ int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return -ENOSPC; /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags), + tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags + 2), GFP_ATOMIC); if (unlikely(!tmp)) return -ENOMEM; diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index db9e10f9077c..1a94f0fda1d1 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -11,8 +11,7 @@ #define _NET_OVPN_OVPN_H_ /* DATA_V2 header size with AEAD encryption */ -#define OVPN_HEAD_ROOM (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE + \ - 16 /* AEAD TAG length */ + \ +#define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ max(sizeof(struct ipv6hdr), sizeof(struct iphdr))) diff --git a/drivers/net/ovpn/proto.h b/drivers/net/ovpn/proto.h index b7d285b4d9c1..f3b305cbefe5 100644 --- a/drivers/net/ovpn/proto.h +++ b/drivers/net/ovpn/proto.h @@ -39,6 +39,10 @@ #define OVPN_NONCE_WIRE_SIZE (OVPN_NONCE_SIZE - OVPN_NONCE_TAIL_SIZE) #define OVPN_OPCODE_SIZE 4 /* DATA_V2 opcode size */ +#define OVPN_AUTH_TAG_SIZE 16 +#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + \ + OVPN_NONCE_WIRE_SIZE) +#define OVPN_DATA_V2_OVERHEAD (OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE) #define OVPN_OPCODE_KEYID_MASK 0x07000000 #define OVPN_OPCODE_PKTTYPE_MASK 0xF8000000 #define OVPN_OPCODE_PEERID_MASK 0x00FFFFFF From patchwork Thu Oct 1 08:41:23 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5424 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6c89:b0:8b5:356c:6cee with SMTP id p9csp1748850may; Thu, 1 Oct 2026 01:41:58 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBzMrkfwtqlZgMVfeI3m052RoUVyZIizGVGhF4BoZplm3CfEuMRP3/RtwrvrpAatc2CXb+LvcYZqFOY=@openvpn.net X-Received: by 2002:a05:6870:330b:b0:47d:2b69:8d11 with SMTP id 586e51a60fabf-49ddd5559ccmr4394538fac.31.1790844118432; Thu, 01 Oct 2026 01:41:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790844118; cv=none; d=google.com; s=arc-20260327; b=jmujmeoErNntrBABVmlyiMf28qSZhoEwwx+5fdXpoWmkeaGK0xbwvuk3HwdMbcNOdd MqYGtsb0HmAvYSo3NIPlhnmIFPkevgutmNTW/CusotjPw5PTNVNQrCO86eat9b0xwvB8 Mm0cacHF19+RSM126Pox4lmawKiUh1ocE2JEkfnI4yVIUkIVMrj0MPSBEaNgk/SxASM9 C/Zlk5y+KDLTVKZPbSM6bcn2PiJxb8kykzv7pI+RH5BhZDPc+djr2BzGP3uQIzxVZXGX VC1l7qCoMjF6km73sjXzIXQ+OLMcVzr9stDZLZK868bd3/XwkwYf5wbXqVdwMdXfhjxO IIGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=WAPm3K+rOrkepkXFxIW7pFn58V53LtI3w2X5u4AeD6o=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=W13SjxntmZz+oQSxfX00UgZ6H84s3Jgq+Y5TvKd6LLdJImB8Doy/mvK7AE8lF2skBe zt7T+cctdRvFIE9777NrBJFeXtb7AWilDZlGIfnBKofXBU/7ojgmeJI2AjjF8Gioxheu riyB1Tyz5+8NY6J96X3/HXUub/W0SOnMU5G5EXqDGm83yWTQcAg/u6CDY4I/YZvlzLtk zJ8c+RQpDIXd2woAl/plnLIOCGULD4NKF92hbZf5xZI++/yo8zqAv2KqYEmrAy7CdS/b GqWFX2I4y1cFcK1rzRGvfLpB2zj02UM9o/LtT7gJX4urMM+5NcKs9rSOwxLuXqFYrTu4 Omrg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=EmXQbEQo; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="N1Bv/K0B"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=C75hW5PN; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=rpG6grAX; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-49ded3ae893si4288597fac.251.2026.10.01.01.41.58 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Oct 2026 01:41:58 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=EmXQbEQo; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="N1Bv/K0B"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=C75hW5PN; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=rpG6grAX; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=WAPm3K+rOrkepkXFxIW7pFn58V53LtI3w2X5u4AeD6o=; b=EmXQbEQo0fCC37AouVWWjLz8FG Usk+ArInvimO1etR6SLcunWJneUmYHgaRVfI6PizATh17aCMq4PWT3oTT+ODMCesAnPB9/wHpSTX6 IAuaBWlfKFbgJhla8NPmBc+q9/vWIoBhKGW86HlFxZBwaWv2zJnWecji4JhYiREW8OuY=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xCCMR-0006pE-1E; Thu, 01 Oct 2026 08:41:51 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xCCMO-0006oz-T5 for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=KDGzNWpT6bT+0gN97EM6zYt1L3fuQ96vE1reFM8XNRU=; b=N1Bv/K0BlCfMXltZVbtDy2noVC VZP2bByZT9a8bmKIekhxLlhxwf7thVxKyksV6dvanplrpKw31msnOoo9GOWVIX61/c8a1uoLOdoTT IcchCWsHqlJnlGVYfsihIoZs+URXOW17pajCsT+Jxq3Zt0nNAMruBrLZf6UIM2kz/hqc=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=KDGzNWpT6bT+0gN97EM6zYt1L3fuQ96vE1reFM8XNRU=; b=C75hW5PNY9JqXbM60G/tQD4Bdw Kbsf2W63ToNItsnwkPjQULPjqLFcJnwcJDBXTWSN4jD4ngnE52BA+2rfQoWLrXPW6A+kLO3/dcZOt vYNmcSbcg3d/il4hLgY6zfdL6XcXzOmqTO1y3YRY8Twq7jzESUgLoVledGe0o1yiYDd4=; Received: from mout-b-206.mailbox.org ([195.10.208.51]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xCCML-0003U5-Qy for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:48 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-206.mailbox.org (Postfix) with ESMTPS id 4hwQPY4QRQzB3 for ; Thu, 01 Oct 2026 10:41:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1790844097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KDGzNWpT6bT+0gN97EM6zYt1L3fuQ96vE1reFM8XNRU=; b=rpG6grAXB4kTmxa4DbmCQGYgVUca0YFnbBz8w3stkr+ln64uyVUb1LHfXEQ+vXzjqs9eMy eKM2zx5PXMcLl1rDRPTKuHeCXftRbMJhX27kUzWRasJY6xK0RUxz07LjwVmz0ba4zNtg56 DH/eG1QA8yMgDbFJilz/dBCOmNjQ34Rz42sERO7E3aZ4Vo13afl0C2xWf4d86gQGcv8MAC 21LYITVq5wfbIHKDgxfIrpyMjDUHlVGs1vbWOncuVNK1siRz7oq2OV6Ht9ZWMsyVz7ScvX rVIkZ8vbI5GPjpGWgt7LylD4b/UwRVmb2fkouxwk9+kXPh8L+bX7y3sh6YjoIQ== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 1 Oct 2026 10:41:23 +0200 Message-ID: <47eee3045f402b2baf7db759bc9cfd35dee8f73d.1790843153.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hwQPY4QRQzB3 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb while completing its checksums. Allow the segmentation code to retain safe page-backed payload fragments, then emit eligible [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1xCCML-0003U5-Qy Subject: [Openvpn-devel] [PATCH ovpn net-next 4/8] ovpn: convert GSO input into UDP GSO output X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1877836162184269456 X-GMAIL-MSGID: 1877836162184269456 Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb while completing its checksums. Allow the segmentation code to retain safe page-backed payload fragments, then emit eligible fixed-size inputs as UDP GSO skb(s). Allocate one final page-backed aggregate per batch before submitting encryption. Map each segment's linear data and page fragments directly into the AEAD source scatterlist, and have each request write out of place into its final record slot. This avoids copying page-backed payload into a linear segment. The skb core can still linearize externally mutable fragments when needed to complete their checksums safely. Keep the existing in-place path for ordinary packets, where allocating and retiring a separate output skb for every record would provide no aggregate construction benefit. Also retain that path for frag-list GSO input: it already stores complete segments as child skbs, and measurements show that copying those children into another aggregate is counterproductive. Transmit the aggregate as SKB_GSO_UDP_L4 only after every record succeeds, and discard it if any request fails. Split aggregates at the legacy GSO size limit and fall back to individual records when batching is unavailable or the segment geometry is unsuitable. Preserve the input priority, flow hash and sender CPU on the replacement aggregate. If the input has real write ownership, charge the aggregate to the same socket as well. This retains socket lifetime and write-memory accounting and lets lower-device queue selection use the socket's cached TX queue instead of choosing a new queue after crypto completion. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without UDP GSO 12.053 Gbit/s 10.992 Gbit/s With UDP GSO 19.262 Gbit/s 19.718 Gbit/s This is a 59.8% improvement forward and a 79.4% improvement reverse. With lower-device UDP segmentation disabled, the GSO path raised median throughput from 12.386 to 15.489 Gbit/s forward and from 10.206 to 14.816 Gbit/s reverse, improvements of 25.1% and 45.2%, respectively. Signed-off-by: Ralf Lici --- Changes since RFC v4 https://lore.kernel.org/openvpn-devel/9359f737627fc84be08ee7d301415c023ebadeb2.1789558856.git.ralf@mandelbit.com/ - Preserve safe page-backed payload fragments during software segmentation instead of forcing every segment to become linear. - Map each segment's linear data and fragments directly into the AEAD source scatterlist. - Let the encryption caller supply the packet ID, preparing the path for batch range reservation in the following patch. - Refresh the hardware and software segmentation measurements. drivers/net/ovpn/crypto_aead.c | 109 +++++++++++--- drivers/net/ovpn/crypto_aead.h | 4 + drivers/net/ovpn/io.c | 258 ++++++++++++++++++++++++++++++--- drivers/net/ovpn/skb.h | 27 +++- drivers/net/ovpn/stats.h | 16 +- drivers/net/ovpn/tcp.c | 4 +- drivers/net/ovpn/udp.c | 27 +++- 7 files changed, 391 insertions(+), 54 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 30299581422d..76028f048606 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -134,13 +134,17 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, struct sk_buff *skb, - unsigned int nents, u8 **iv) + unsigned int nents, + unsigned int extra, u8 **iv) { struct aead_request *req; void *tmp; - /* allocate IV, request and scatterlist entries in one block */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + /* allocate IV, request, scatterlist entries and caller scratch space + * in one block + */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents) + extra, + GFP_ATOMIC); if (unlikely(!tmp)) return ERR_PTR(-ENOMEM); @@ -151,19 +155,11 @@ static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, return req; } -static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, - struct ovpn_crypto_key_slot *ks, - u8 *iv, u8 *data) +static void ovpn_aead_encrypt_header(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + u32 pktid, u8 *iv, u8 *data) { - u32 pktid, op; - int ret; - - /* obtain packet ID, which is used both as a first - * 4 bytes of nonce and last 4 bytes of associated data. - */ - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); - if (unlikely(ret < 0)) - return ret; + u32 op; /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes * nonce @@ -175,8 +171,6 @@ static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); *(__force __be32 *)data = htonl(op); memcpy(data + OVPN_OPCODE_SIZE, iv, OVPN_NONCE_WIRE_SIZE); - - return 0; } int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, @@ -187,6 +181,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *trailer; struct scatterlist *sg; int nfrags, ret; + u32 pktid; u8 *iv; ovpn_skb_cb(skb)->peer = peer; @@ -217,7 +212,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, 0, &iv); if (IS_ERR(req)) return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); @@ -243,9 +238,11 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, /* make space for the additional data and push it to the front */ __skb_push(skb, OVPN_AAD_SIZE); - ret = ovpn_aead_encrypt_header(peer, ks, iv, skb->data); + /* obtain the packet ID used by both the nonce and associated data */ + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); if (unlikely(ret < 0)) return ret; + ovpn_aead_encrypt_header(peer, ks, pktid, iv, skb->data); /* AEAD Additional data */ sg_set_buf(sg, skb->data, OVPN_AAD_SIZE); @@ -261,6 +258,80 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return crypto_aead_encrypt(req); } +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, + struct sk_buff *gso_skb, unsigned int offset) +{ + const unsigned int dst_nents = skb_shinfo(gso_skb)->nr_frags + 4; + const unsigned int src_nents = skb_shinfo(skb)->nr_frags + 2; + struct scatterlist *src, *dst; + struct aead_request *req; + unsigned int payload_off; + int dst_idx, mapped, ret; + u32 pktid; + u8 *aad, *iv; + + /* each input records the shared peer and key for the common completion + * path but their references remain owned by the output aggregate + */ + ovpn_skb_cb(skb)->peer = peer; + ovpn_skb_cb(skb)->ks = ks; + + req = ovpn_aead_request_alloc(ks->encrypt, skb, src_nents + dst_nents, + OVPN_AAD_SIZE, &iv); + if (IS_ERR(req)) + return PTR_ERR(req); + src = ovpn_aead_crypto_req_sg(ks->encrypt, req); + dst = src + src_nents; + aad = (u8 *)(dst + dst_nents); + + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + if (unlikely(ret < 0)) + return ret; + ovpn_aead_encrypt_header(peer, ks, pktid, iv, aad); + + ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); + if (unlikely(ret < 0)) + return ret; + + /* encrypt out of place from the original segmented skb directly into + * its final range in the UDP GSO skb + */ + sg_init_table(src, src_nents); + sg_set_buf(src, aad, OVPN_AAD_SIZE); + mapped = skb_to_sgvec_nomark(skb, src + 1, 0, skb->len); + if (unlikely(mapped < 0)) + return mapped; + sg_mark_end(&src[mapped]); + + sg_init_table(dst, dst_nents); + dst_idx = skb_to_sgvec_nomark(gso_skb, dst, offset, OVPN_AAD_SIZE); + if (unlikely(dst_idx < 0)) + return dst_idx; + + payload_off = offset + OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE; + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, payload_off, + skb->len); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, + offset + OVPN_AAD_SIZE, + OVPN_AUTH_TAG_SIZE); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + sg_mark_end(&dst[dst_idx - 1]); + + aead_request_set_tfm(req, ks->encrypt); + aead_request_set_callback(req, 0, ovpn_encrypt_post, skb); + aead_request_set_crypt(req, src, dst, skb->len, iv); + aead_request_set_ad(req, OVPN_AAD_SIZE); + + return crypto_aead_encrypt(req); +} + int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { diff --git a/drivers/net/ovpn/crypto_aead.h b/drivers/net/ovpn/crypto_aead.h index fae3b585a43b..8b444744944e 100644 --- a/drivers/net/ovpn/crypto_aead.h +++ b/drivers/net/ovpn/crypto_aead.h @@ -17,6 +17,10 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + struct sk_buff *skb, struct sk_buff *gso_skb, + unsigned int offset); int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 112067ded401..23f52b7cc085 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include "ovpnpriv.h" #include "peer.h" @@ -32,6 +34,12 @@ const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE] = { 0x07, 0xed, 0x2d, 0x0a, 0x98, 0x1f, 0xc7, 0x48 }; +/* Leave room for the largest outer network header. The strict inequality in + * is_skb_forwardable also requires staying one byte below gso_max_size. + */ +#define OVPN_UDP_GSO_MAX_PAYLOAD (GSO_LEGACY_MAX_SIZE - \ + sizeof(struct ipv6hdr) - \ + sizeof(struct udphdr) - 1) /** * ovpn_is_keepalive - check if skb contains a keepalive message * @skb: packet to check @@ -237,11 +245,13 @@ void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb) void ovpn_encrypt_post(void *data, int ret) { + unsigned int orig_len, packets = 1; struct ovpn_crypto_key_slot *ks; struct sk_buff *skb = data; struct ovpn_socket *sock; + struct ovpn_cb *batch_cb; struct ovpn_peer *peer; - unsigned int orig_len; + struct sk_buff *batch; /* encryption is happening asynchronously. This function will be * called later by the crypto callback with a proper return value @@ -249,15 +259,19 @@ void ovpn_encrypt_post(void *data, int ret) if (unlikely(ret == -EINPROGRESS)) return; - ks = ovpn_skb_cb(skb)->ks; + /* ordinary encryption leaves batch zeroed; a GSO input uses it to find + * the aggregate whose lifetime is shared by all segment requests + */ + batch = ovpn_skb_cb(skb)->batch; peer = ovpn_skb_cb(skb)->peer; + ks = ovpn_skb_cb(skb)->ks; /* crypto is done, cleanup skb CB and its members */ kfree(ovpn_skb_cb(skb)->crypto_tmp); if (unlikely(ret == -ERANGE)) { /* we ran out of IVs and we must kill the key as it can't be - * use anymore + * used anymore */ netdev_warn(peer->ovpn->dev, "killing key %u for peer %u\n", ks->key_id, @@ -265,8 +279,30 @@ void ovpn_encrypt_post(void *data, int ret) if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) /* let userspace know so that a new key must be negotiated */ ovpn_nl_key_swap_notify(peer, ks->key_id); + } - goto err; + if (batch) { + batch_cb = ovpn_skb_cb(batch); + /* every segment publishes its result before releasing its + * pending count and only the final completion continues with + * the aggregate + */ + if (unlikely(ret < 0)) + atomic_set(&batch_cb->batch_state.failed, 1); + + kfree_skb(skb); + if (!atomic_dec_and_test(&batch_cb->batch_state.pending)) + return; + + skb = batch; + packets = skb_shinfo(batch)->gso_segs; + if (unlikely(atomic_read(&batch_cb->batch_state.failed))) + goto err; + + /* reaching the final callback with no sticky failure means + * every segment completed successfully + */ + ret = 0; } if (unlikely(ret < 0)) @@ -292,7 +328,7 @@ void ovpn_encrypt_post(void *data, int ret) goto err_unlock; } - ovpn_peer_stats_increment_tx(&peer->link_stats, orig_len); + ovpn_peer_stats_add_tx(&peer->link_stats, orig_len, packets); /* keep track of last sent packet for keepalive */ WRITE_ONCE(peer->last_sent, ktime_get_boottime_seconds()); /* skb passed down the stack - don't free it */ @@ -301,7 +337,7 @@ void ovpn_encrypt_post(void *data, int ret) rcu_read_unlock(); err: if (unlikely(skb)) - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, packets); kfree_skb(skb); if (likely(ks)) ovpn_crypto_key_slot_put(ks); @@ -309,29 +345,124 @@ void ovpn_encrypt_post(void *data, int ret) ovpn_peer_put(peer); } -static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +/* Hold the peer and its primary key for one encryption submission. + * The returned key and the peer each carry one reference which completion must + * release. + */ +static struct ovpn_crypto_key_slot * +ovpn_encrypt_refs_get(struct ovpn_peer *peer) { struct ovpn_crypto_key_slot *ks; /* get primary key to be used for encrypting data */ ks = ovpn_crypto_key_slot_primary(&peer->crypto); if (unlikely(!ks)) - return false; + return NULL; - /* take a reference to the peer because the crypto code may run async. - * ovpn_encrypt_post() will release it upon completion + /* the caller already owns a peer reference, so failure indicates a + * broken reference lifetime elsewhere */ if (unlikely(!ovpn_peer_hold(peer))) { DEBUG_NET_WARN_ON_ONCE(1); ovpn_crypto_key_slot_put(ks); - return false; + return NULL; } + return ks; +} + +static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct ovpn_crypto_key_slot *ks; + + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_encrypt_post(skb, ovpn_aead_encrypt(peer, ks, skb)); return true; } +static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, + struct ovpn_peer *peer, + struct sk_buff *batch, + unsigned int segments) +{ + unsigned int offset = 0, i, len; + struct ovpn_crypto_key_slot *ks; + struct sk_buff *skb; + + /* acquire all shared state before removing the first input skb so that + * failure can leave the queue intact for the ordinary transmit path + */ + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + + /* the aggregate owns these references until every sync or async crypto + * completion has finished + */ + memset(ovpn_skb_cb(batch), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(batch)->peer = peer; + ovpn_skb_cb(batch)->ks = ks; + atomic_set(&ovpn_skb_cb(batch)->batch_state.pending, segments); + atomic_set(&ovpn_skb_cb(batch)->batch_state.failed, 0); + + for (i = 0; i < segments; i++) { + skb = __skb_dequeue(skbs); + len = skb->len + OVPN_DATA_V2_OVERHEAD; + + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(skb)->batch = batch; + ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, + batch, offset)); + offset += len; + } + + return true; +} + +static struct sk_buff *ovpn_udp_gso_alloc(const struct sk_buff *first_segment, + unsigned int batch_len, + unsigned int segments) +{ + struct sk_buff *gso_skb; + int ret; + + gso_skb = alloc_skb_with_frags(OVPN_HEAD_ROOM, batch_len, + SKB_FRAG_PAGE_ORDER, &ret, GFP_ATOMIC); + if (unlikely(!gso_skb)) + return NULL; + + skb_reserve(gso_skb, OVPN_HEAD_ROOM); + gso_skb->len = batch_len; + gso_skb->data_len = batch_len; + gso_skb->priority = first_segment->priority; + + /* Segments retain the originating socket so we keep its send-buffer + * accounting active until the UDP GSO is transmitted. This also + * preserves its cached TX queue. + */ + if (first_segment->sk && is_skb_wmem(first_segment)) + skb_set_owner_w(gso_skb, first_segment->sk); + skb_copy_hash(gso_skb, first_segment); +#ifdef CONFIG_XPS + /* keep the aggregate on the TX queue selected for the original flow + * otherwise async crypto completion on another CPU could move the flow + * to a different queue and cause delay or reordering + */ + gso_skb->sender_cpu = first_segment->sender_cpu; +#endif + + skb_shinfo(gso_skb)->gso_type = SKB_GSO_UDP_L4; + skb_shinfo(gso_skb)->gso_size = first_segment->len + + OVPN_DATA_V2_OVERHEAD; + skb_shinfo(gso_skb)->gso_segs = segments; + + return gso_skb; +} + /* send skb to connected peer, if any */ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, struct ovpn_peer *peer) @@ -343,7 +474,7 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, */ skb_list_walk_safe(skb, curr, next) { if (unlikely(!ovpn_encrypt_one(peer, curr))) { - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); } } @@ -351,19 +482,96 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, ovpn_peer_put(peer); } +/* encrypt fixed-size input segments into one or more UDP GSO aggregates */ +static void ovpn_send_gso(struct sk_buff_head *skbs, struct ovpn_peer *peer) +{ + unsigned int max_segs = 1, seg_len, batch_len, segs; + struct sk_buff *batch; + + seg_len = skb_peek(skbs)->len + OVPN_DATA_V2_OVERHEAD; + max_segs = min_t(unsigned int, UDP_MAX_SEGMENTS, + OVPN_UDP_GSO_MAX_PAYLOAD / seg_len); + + /* if even two encrypted skbs cannot fit, leave the whole queue for the + * ordinary transmit path + */ + if (max_segs < 2) + return; + + /* An input GSO skb might be prduce more than max_segs segments so we + * consume as many as we can for each iteration. A final single skb, or + * the whole remainder after an allocation failure, stays queued and + * fallback to the ordinary transmit path. + */ + while (skb_queue_len(skbs) > 1) { + segs = min_t(unsigned int, skb_queue_len(skbs), max_segs); + + /* all but the final input skb have the same length, so start + * with the full-size calculation and adjust only the final + * group below + */ + batch_len = segs * (skb_peek(skbs)->len + + OVPN_DATA_V2_OVERHEAD); + if (segs == skb_queue_len(skbs)) + batch_len -= skb_peek(skbs)->len - + skb_peek_tail(skbs)->len; + + batch = ovpn_udp_gso_alloc(skb_peek(skbs), batch_len, segs); + if (unlikely(!batch)) + return; + + if (unlikely(!ovpn_encrypt_gso_queue(skbs, peer, + batch, segs))) { + kfree_skb(batch); + return; + } + } +} + +static bool ovpn_peer_supports_udp_gso(struct ovpn_peer *peer) +{ + struct ovpn_socket *sock; + bool udp_gso; + + rcu_read_lock(); + sock = rcu_dereference(peer->sock); + /* UDP GSO requires checksums. These socket settings can change after + * we decide to batch, but an already-built batch remains checksummed. + * Linux's ordinary UDP GSO path makes the same choice. + */ + udp_gso = sock && sock->sk->sk_protocol == IPPROTO_UDP && + !sock->sk->sk_no_check_tx && !udp_get_no_check6_tx(sock->sk); + rcu_read_unlock(); + + return udp_gso; +} + /* Send user data to the network */ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) { struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; + const bool gso_in = skb_is_gso(skb); struct sk_buff_head skb_list; netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; + bool gso_out; __be16 proto; int ret; + /* A frag-list GSO skb already stores complete segments as child skbs. + * Keep those children on the ordinary in-place encryption path instead + * of copying them into a replacement UDP GSO skb. + * + * GSO_BY_FRAGS input must remain on that path too because its variable + * segment sizes cannot be represented by one UDP GSO output size. + */ + gso_out = gso_in && + !(skb_shinfo(skb)->gso_type & SKB_GSO_FRAGLIST) && + skb_shinfo(skb)->gso_size != GSO_BY_FRAGS; + /* reset netfilter state */ nf_reset_ct(skb); @@ -392,13 +600,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) /* dst was needed for peer selection - it can now be dropped */ skb_dst_drop(skb); - if (skb_is_gso(skb)) { - /* force software segmentation, but keep ovpn's non-GSO feature - * bits so the generated segments can preserve non-linear skb - * data where possible + if (gso_in) { + /* force software segmentation and calculate each checksum + * while allowing safe page-backed payload fragments to remain + * non-linear */ features = netif_skb_features(skb); - segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + features &= ~(NETIF_F_GSO_MASK | NETIF_F_CSUM_MASK); + segments = skb_gso_segment(skb, features); if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", @@ -420,7 +629,8 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) if (unlikely(!curr)) { net_err_ratelimited("%s: skb_share_check failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); + gso_out = false; continue; } @@ -429,8 +639,9 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_checksum_help(curr) < 0)) { net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); + gso_out = false; continue; } @@ -446,9 +657,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) ovpn_peer_put(peer); return NETDEV_TX_OK; } - skb_list.prev->next = NULL; ovpn_peer_stats_increment_tx(&peer->vpn_stats, tx_bytes); + + if (gso_out && skb_queue_len(&skb_list) > 1 && + ovpn_peer_supports_udp_gso(peer)) + ovpn_send_gso(&skb_list, peer); + + skb_list.prev->next = NULL; ovpn_send(ovpn, skb_list.next, peer); return NETDEV_TX_OK; @@ -456,7 +672,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) drop: ovpn_peer_put(peer); drop_no_peer: - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); skb_tx_error(skb); kfree_skb_list(skb); return NETDEV_TX_OK; diff --git a/drivers/net/ovpn/skb.h b/drivers/net/ovpn/skb.h index 4fb7ea025426..cca29479c038 100644 --- a/drivers/net/ovpn/skb.h +++ b/drivers/net/ovpn/skb.h @@ -10,6 +10,7 @@ #ifndef _NET_OVPN_SKB_H_ #define _NET_OVPN_SKB_H_ +#include #include #include #include @@ -20,19 +21,35 @@ /** * struct ovpn_cb - ovpn skb control block - * @peer: the peer this skb was received from/sent to - * @ks: the crypto key slot used to encrypt/decrypt this skb * @crypto_tmp: pointer to temporary memory used for crypto operations * containing the IV, the scatter gather list and the aead request + * @peer: peer used by this crypto operation or owned by this aggregate + * @ks: crypto key slot used by this operation or owned by this aggregate * @payload_offset: offset in the skb where the payload starts * @nosignal: whether this skb should be sent with the MSG_NOSIGNAL flag (TCP) + * @batch: UDP GSO aggregate receiving this input skb's encrypted payload + * @batch_state: completion state owned by a UDP GSO aggregate */ struct ovpn_cb { + void *crypto_tmp; struct ovpn_peer *peer; struct ovpn_crypto_key_slot *ks; - void *crypto_tmp; - unsigned int payload_offset; - bool nosignal; + + /* Ordinary encryption leaves this union zeroed. Decryption and TCP use + * their ordinary fields, a UDP GSO input stores its output aggregate, + * and that aggregate uses the same space to coordinate its completions. + */ + union { + struct { + unsigned int payload_offset; + bool nosignal; + }; + struct sk_buff *batch; + struct { + atomic_t pending; + atomic_t failed; + } batch_state; + }; }; static inline struct ovpn_cb *ovpn_skb_cb(struct sk_buff *skb) diff --git a/drivers/net/ovpn/stats.h b/drivers/net/ovpn/stats.h index 3a45b97c0056..b3fe006c01f6 100644 --- a/drivers/net/ovpn/stats.h +++ b/drivers/net/ovpn/stats.h @@ -40,16 +40,26 @@ static inline void ovpn_peer_stats_increment_rx(struct ovpn_peer_stats *stats, ovpn_peer_stats_increment(&stats->rx, n); } +static inline void ovpn_peer_stats_add_tx(struct ovpn_peer_stats *stats, + const unsigned int bytes, + unsigned int packets) +{ + atomic64_add(bytes, &stats->tx.bytes); + atomic64_add(packets, &stats->tx.packets); +} + static inline void ovpn_peer_stats_increment_tx(struct ovpn_peer_stats *stats, const unsigned int n) { - ovpn_peer_stats_increment(&stats->tx, n); + ovpn_peer_stats_add_tx(stats, n, 1); } -static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev) +static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev, + unsigned int packets) { local_bh_disable(); - dev_dstats_tx_dropped(dev); + while (packets--) + dev_dstats_tx_dropped(dev); local_bh_enable(); } diff --git a/drivers/net/ovpn/tcp.c b/drivers/net/ovpn/tcp.c index 8fe8a8e750a4..5cba35e4a8ee 100644 --- a/drivers/net/ovpn/tcp.c +++ b/drivers/net/ovpn/tcp.c @@ -332,7 +332,7 @@ static void ovpn_tcp_send_sock_skb(struct ovpn_peer *peer, struct sock *sk, ovpn_tcp_send_sock(peer, sk); if (peer->tcp.out_msg.skb) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); return; } @@ -354,7 +354,7 @@ void ovpn_tcp_send_skb(struct ovpn_peer *peer, struct sock *sk, if (sock_owned_by_user(sk)) { if (skb_queue_len(&peer->tcp.out_queue) >= READ_ONCE(net_hotdata.max_backlog)) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); goto unlock; } diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 7f69e8890b5b..4802d982de08 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -121,6 +121,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); + skb_mark_not_on_list(skb); ovpn_recv(peer, skb); return 0; @@ -196,9 +197,13 @@ static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind, dst_cache_set_ip4(cache, &rt->dst, fl.saddr); transmit: + /* an already-built UDP GSO needs a checksum seed even if the socket's + * no-check option changed while encryption was in flight + */ udp_tunnel_xmit_skb(rt, sk, skb, fl.saddr, fl.daddr, 0, ip4_dst_hoplimit(&rt->dst), 0, fl.fl4_sport, - fl.fl4_dport, false, sk->sk_no_check_tx, 0); + fl.fl4_dport, false, + !skb_is_gso(skb) && sk->sk_no_check_tx, 0); ret = 0; err: local_bh_enable(); @@ -271,9 +276,13 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind, * udp_tunnel_xmit_skb() */ skb->ignore_df = 1; + /* keep checksum offload enabled for an in-flight UDP GSO batch even if + * the socket's no-check option has changed since batch creation + */ udp_tunnel6_xmit_skb(dst, sk, skb, skb->dev, &fl.saddr, &fl.daddr, 0, ip6_dst_hoplimit(dst), 0, fl.fl6_sport, - fl.fl6_dport, udp_get_no_check6_tx(sk), 0); + fl.fl6_dport, + !skb_is_gso(skb) && udp_get_no_check6_tx(sk), 0); ret = 0; err: local_bh_enable(); @@ -344,8 +353,18 @@ void ovpn_udp_send_skb(struct ovpn_peer *peer, struct sock *sk, skb->dev = peer->ovpn->dev; skb->mark = READ_ONCE(sk->sk_mark); - /* no checksum performed at this layer */ - skb->ip_summed = CHECKSUM_NONE; + if (skb_is_gso(skb)) { + /* udp_tunnel_xmit_skb installs the outer UDP header after this + * function returns: point CHECKSUM_PARTIAL at that future + * header so both hw and sw UDP GSO can complete the checksum. + */ + skb->ip_summed = CHECKSUM_PARTIAL; + skb->csum_start = skb_headroom(skb) - sizeof(struct udphdr); + skb->csum_offset = offsetof(struct udphdr, check); + } else { + /* no checksum performed at this layer */ + skb->ip_summed = CHECKSUM_NONE; + } /* crypto layer -> transport (UDP) */ ret = ovpn_udp_output(peer, &peer->dst_cache, sk, skb); From patchwork Thu Oct 1 08:41:24 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5421 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6c89:b0:8b5:356c:6cee with SMTP id p9csp1748831may; Thu, 1 Oct 2026 01:41:57 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvByyS4mBQwsvyLBylOjDP84Ke2ZCCUL/b4y09eEIk8QU0vvXs0x5HDdpRqtqySDRbWBc89ezmf5HMXc=@openvpn.net X-Received: by 2002:a05:6871:c317:b0:483:ac2e:71bf with SMTP id 586e51a60fabf-49df05a44d7mr1971294fac.10.1790844116871; Thu, 01 Oct 2026 01:41:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790844116; cv=none; d=google.com; s=arc-20260327; b=IGDUHilTuJ61emTXB6oGp9OssZZYkll57BAtCYW7drCCwPM/JERrje2bfgyQf3zTUV bJhV5I5bZlAjAIemZKEz7PPUggjDofg4DcMXctZrv/weZir/n8EIx1Uf71Glks7LpFDp xtAyY1r2CJ/87QJNzWstUPr0Mf2N3J5F0og7PYGqWBziPwLVVW+nOtKnMxYJ7cYBbpWx h2j1n+j4IF/vx0/OwptJWCSV7MN30u/R4rVyJDaXxmsciSsB7RUt1yhHfvN4q7oT/vLI 23xSuE5N983Yhn9H8vlHPY7c1jexOlC2d8CL4cpnVDuh9D+eIH2KHDB+tSPYiZwCEa0h b7Mw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=dlPGhCp1g0uwc8/jgEJM42NdHvAWLalnqfXnJ25ED+Y=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=MGKcJrr9fQf/AVZyPV/XctFvIXrt9ibQ+AqUSP2eIrID36FVdKe1V0PpWwoYqUZtM7 MWEYyr+Jcrg0LRnoaKHrb4XxttBj0ADkq4Tat1rFIsVBYrCOWMd89Ji4KLQDHlGuPHfB q5IfNkx6RabtMeeBpnSnyafVpPia7/kAYXfvCziS1agYH+AZ2OspapTjK1/uX9Qm4KA4 ZuctGyJJzxekshHUTic7Ktc2bW8mx5kQWLHPU2Q/7N4VjYe9caoFmz7UStD7F6H3LNl3 needHFX+/ntDDKaAT4eT5NlGj/8TuR9JO2BNFGGmePedcX71pfUG5BZdPg2HTwbC1mDv hLow==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=bDAYRECU; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=gx3D2hHb; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=RTypQ07Q; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b="T/0feUAu"; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-49ded30ceb4si4963412fac.177.2026.10.01.01.41.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Oct 2026 01:41:56 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=bDAYRECU; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=gx3D2hHb; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=RTypQ07Q; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b="T/0feUAu"; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dlPGhCp1g0uwc8/jgEJM42NdHvAWLalnqfXnJ25ED+Y=; b=bDAYRECUfHQPEkKphMb1MCbU0L Dlglq+uwSgFj/N0+QGwYuyC78tbYVbyZyZi6qWaPt07nVsrmkuV2AyyFDL+fTG/aMCTyekyJf/ceP z0gNmRPZjlZr3La/VYZiYgdqHq9YUyk4qWKGJI5iiPcjKWVkz1gQIPm9H1LaOudi3F9o=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xCCMV-0003WE-Fx; Thu, 01 Oct 2026 08:41:53 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xCCMQ-0003Rp-Ap for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=MHZmHUQmEkJ8mMBDsBHiYI0615zGAs2L29/fJGEEtpo=; b=gx3D2hHbZ1VgVcIZFa/fMRtE1l R0hpKPtyTQ2RRKiifx7kcCtjSqotB17bs9p2nQLyjsHAZ2tlzAfbIbjl/zChYtyH3YkkX2OZDZXZV wO0ZupPSx8Jc+L1+pjxAP4DzKdX99bCUIlGX14xxdAmDKB1Gg8go8hI4n+0lgXrX3lH8=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=MHZmHUQmEkJ8mMBDsBHiYI0615zGAs2L29/fJGEEtpo=; b=RTypQ07Qg+BG5kVY0JbJjihenx tI7wwHDccEpOsUj7xS535ewcLqqw25DG+R6GENVh5CQd5NcgbCVu5LTRO4zlAF2DWLyOTB7QLw53c XvT0TU2pWmuo/HOktp0QIa3Z1TBK87ipyslwJ2LRVyV/BY/v+ecHsjfbMNmWy2wM3wsI=; Received: from mout-b-105.mailbox.org ([195.10.208.50]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xCCMM-0002wy-65 for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:47 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-105.mailbox.org (Postfix) with ESMTPS id 4hwQPZ2qljz9tXm for ; Thu, 01 Oct 2026 10:41:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1790844098; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MHZmHUQmEkJ8mMBDsBHiYI0615zGAs2L29/fJGEEtpo=; b=T/0feUAuncjZ95i8vqzc6eYSNBZ6vpobOJeKqdiNbr800NNLUKLklmPBsJmau6W7SGOqiI SD/FMltzjuj94AaSd6WRlEHGF7jDeB9gDb1iVEUa0mEa/mFpI/BD4NXeECsgx/EonoKYxz BSeERPS9irHqG3MH5BW+lhl7Gro9lAnvlE73O9mokj0q0AuEBjhDlHkeFtZh3E4NOcqVIK rsqC7fEIM7qCchx1O8Ps5mpiD2/192BMGL0ufIpIwubbRLgXCmeGN7yFHnkPgUX8yWBBcN eAbNfLvlVYwnzRZCIVsAuA6SJznQHrLVB+v2nY7JtNgkDAop3ThJvCNaDA05tA== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 1 Oct 2026 10:41:24 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: The UDP GSO path currently allocates one packet ID for every input segment. This performs one atomic update per segment even though the whole batch is prepared by a single caller. Reserve the complete consecutive range before dequeuing any input skb, then assign its IDs to segments in queue order. This reduces packet-ID allocation to one atomic operation per batch and leaves th [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xCCMM-0002wy-65 Subject: [Openvpn-devel] [PATCH ovpn net-next 5/8] ovpn: reserve packet-ID ranges for UDP GSO batches X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1877836160606648134 X-GMAIL-MSGID: 1877836160606648134 The UDP GSO path currently allocates one packet ID for every input segment. This performs one atomic update per segment even though the whole batch is prepared by a single caller. Reserve the complete consecutive range before dequeuing any input skb, then assign its IDs to segments in queue order. This reduces packet-ID allocation to one atomic operation per batch and leaves the input queue intact when no complete range remains, allowing the ordinary transmit path to retry with a newly installed key. If the requested range would cross the end of the packet-ID space, publish exhaustion immediately. Packet IDs abandoned after a later encryption failure remain unused, so an AEAD IV is never reused. Signed-off-by: Ralf Lici --- New patch since RFC v4. drivers/net/ovpn/crypto_aead.c | 7 ++--- drivers/net/ovpn/crypto_aead.h | 2 +- drivers/net/ovpn/io.c | 32 ++++++++++++++++++----- drivers/net/ovpn/pktid.h | 48 ++++++++++++++++++++++++++++++++++ 4 files changed, 76 insertions(+), 13 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 76028f048606..1c5387f83eca 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -260,7 +260,8 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, - struct sk_buff *gso_skb, unsigned int offset) + struct sk_buff *gso_skb, unsigned int offset, + u32 pktid) { const unsigned int dst_nents = skb_shinfo(gso_skb)->nr_frags + 4; const unsigned int src_nents = skb_shinfo(skb)->nr_frags + 2; @@ -268,7 +269,6 @@ int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, struct aead_request *req; unsigned int payload_off; int dst_idx, mapped, ret; - u32 pktid; u8 *aad, *iv; /* each input records the shared peer and key for the common completion @@ -285,9 +285,6 @@ int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, dst = src + src_nents; aad = (u8 *)(dst + dst_nents); - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); - if (unlikely(ret < 0)) - return ret; ovpn_aead_encrypt_header(peer, ks, pktid, iv, aad); ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); diff --git a/drivers/net/ovpn/crypto_aead.h b/drivers/net/ovpn/crypto_aead.h index 8b444744944e..92e8ed76fba6 100644 --- a/drivers/net/ovpn/crypto_aead.h +++ b/drivers/net/ovpn/crypto_aead.h @@ -20,7 +20,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, struct sk_buff *gso_skb, - unsigned int offset); + unsigned int offset, u32 pktid); int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 23f52b7cc085..d76d48d945c5 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -243,6 +243,16 @@ void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb) ovpn_decrypt_post(skb, ovpn_aead_decrypt(peer, ks, skb)); } +static void ovpn_encrypt_key_exhausted(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks) +{ + netdev_warn(peer->ovpn->dev, "killing key %u for peer %u\n", + ks->key_id, peer->id); + if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) + /* let userspace know so that a new key must be negotiated */ + ovpn_nl_key_swap_notify(peer, ks->key_id); +} + void ovpn_encrypt_post(void *data, int ret) { unsigned int orig_len, packets = 1; @@ -273,12 +283,7 @@ void ovpn_encrypt_post(void *data, int ret) /* we ran out of IVs and we must kill the key as it can't be * used anymore */ - netdev_warn(peer->ovpn->dev, - "killing key %u for peer %u\n", ks->key_id, - peer->id); - if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) - /* let userspace know so that a new key must be negotiated */ - ovpn_nl_key_swap_notify(peer, ks->key_id); + ovpn_encrypt_key_exhausted(peer, ks); } if (batch) { @@ -392,6 +397,7 @@ static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, unsigned int offset = 0, i, len; struct ovpn_crypto_key_slot *ks; struct sk_buff *skb; + u32 first_pktid; /* acquire all shared state before removing the first input skb so that * failure can leave the queue intact for the ordinary transmit path @@ -400,6 +406,17 @@ static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, if (unlikely(!ks)) return false; + /* reserve the complete range before dequeuing any input skb so that a + * failure leaves the ordinary transmit path available as a fallback + */ + if (unlikely(ovpn_pktid_xmit_range(&ks->pid_xmit, segments, + &first_pktid))) { + ovpn_encrypt_key_exhausted(peer, ks); + ovpn_crypto_key_slot_put(ks); + ovpn_peer_put(peer); + return false; + } + /* the aggregate owns these references until every sync or async crypto * completion has finished */ @@ -416,7 +433,8 @@ static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_skb_cb(skb)->batch = batch; ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, - batch, offset)); + batch, offset, + first_pktid + i)); offset += len; } diff --git a/drivers/net/ovpn/pktid.h b/drivers/net/ovpn/pktid.h index 21845f353bc8..c922e11da6a0 100644 --- a/drivers/net/ovpn/pktid.h +++ b/drivers/net/ovpn/pktid.h @@ -68,6 +68,54 @@ static inline int ovpn_pktid_xmit_next(struct ovpn_pktid_xmit *pid, u32 *pktid) return 0; } +/** + * ovpn_pktid_xmit_range - reserve consecutive packet IDs + * @pid: transmit packet-ID state + * @count: non-zero number of IDs to reserve + * @pktid: first reserved packet ID + * + * The reservation is all-or-nothing. If the requested range would cross the + * end of the 32-bit packet-ID space, exhaust the state without returning the + * unused tail. Packet IDs can be skipped but never reused as an AEAD IV. + * + * Return: 0 on success or -ERANGE when no complete range remains + */ +static inline int ovpn_pktid_xmit_range(struct ovpn_pktid_xmit *pid, + unsigned int count, u32 *pktid) +{ + int old = atomic_read(&pid->seq_num); + u32 first, next; + bool fits; + + for (;;) { + first = (u32)old; + next = first + count; + + /* space is already exhausted */ + if (unlikely(!first)) + return -ERANGE; + + fits = count - 1 <= U32_MAX - first; + /* if it doesn't fit we still have to publish exhaustion (0) + * so concurrent allocators stop using this key + */ + if (unlikely(!fits)) + next = 0; + + /* if another CPU changed the state, we retry using the new + * seq_num value + */ + if (!atomic_try_cmpxchg(&pid->seq_num, &old, (int)next)) + continue; + + if (unlikely(!fits)) + return -ERANGE; + + *pktid = first; + return 0; + } +} + /* Write 12-byte AEAD IV to dest */ static inline void ovpn_pktid_aead_write(const u32 pktid, const u8 nt[], From patchwork Thu Oct 1 08:41:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5425 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6c89:b0:8b5:356c:6cee with SMTP id p9csp1749837may; Thu, 1 Oct 2026 01:43:19 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBygH6f2KHFaWgkL+qg+YfC7TqCa+n1/beoq6Vi4GnMg+Chc3hnoH7puw0/pY2MHGapspUwA68jnZpQ=@openvpn.net X-Received: by 2002:a05:6830:82ab:b0:81f:22fd:6ed5 with SMTP id 46e09a7af769-82049eff90bmr4584713a34.21.1790844119703; Thu, 01 Oct 2026 01:41:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790844119; cv=none; d=google.com; s=arc-20260327; b=VUqDgkoguHLO9s5yKOsiGM7eRCx9IkBKaZYno7wX1y1cOLa+0aSbDTKHBqbhWkfFhF NW+WjCg2T76SgqEiI+5ofrO8jMwc6krJldTfzz8FY4WzLYUNBgE4dgsU97JNt9EhdjwZ tWhj2FdLPqt1WvSZWtIGMW1DKxK8wysCuVBZp6asfKFxNMLSGtub3SNyYrApXNZBKquh /nR3TDgo/1NGDmw/arY91JX5xv5EgpyOOaqank19IMPPeVP9aiwSjj+F6AeFEg6hIaW5 FkuxCkX9JnwGijx6A7l6vRrnpxTBiDK7tFjbz2vFtO9hPOsf93Xf4KfBer3kKHje32JC ljcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=4QgqkDZBvZZSISlG7aAgZD0nINpJx3jkIcaVzlANk3w=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=UA1VJ8x2wmy7hzTMdjupvX3H3xz/5Ht4wOpH+gf4PzwBicUFISr2lFgpb2Bgb+M4hm 5mFGzmRd1OdmvK8B55csCu2nKrXRVxHZb9bgBnnzkbEw5DowJwbTs74U4vdn8/nKe4LV ol8Qq/i2oRdxrBb1FiXP193bRtoSqfBoergPjza+zV3nogC2qUw55Tgy8eI0sdHG79Yz n74M9gWQ+mJSCmxcThlhxpa3MkGT/p7/DNtN+w8gDizHWc1He5vrKY2SsL//b5XHaQN5 +z+5M/JprocmTllfI6eHlcCJJnQQdP4itbkGGDDsPGZMW1m/xg9lW9FMZeIoWBkegcto rM+g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ej9SvIep; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Z3JvaYJD; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Wvj42WXE; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=xnz+7hHN; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-8212ac38f79si3671635a34.78.2026.10.01.01.41.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Oct 2026 01:41:59 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ej9SvIep; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Z3JvaYJD; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Wvj42WXE; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=xnz+7hHN; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=4QgqkDZBvZZSISlG7aAgZD0nINpJx3jkIcaVzlANk3w=; b=ej9SvIepLxwa7ZUnH8FWxHaL5f R8m/zneAalL1BSStS2GY6jLCWSmHYifzz/9FCW2bJdYQZDUUhwOPFEo5rOOKLJVpWySTcXiopasj9 Zt0/zc2VxD9I1IQEXObUg2BAe9xzON5Q+62okT1jG/tJtJwqdGGESRKwupYM12JEneGI=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xCCMZ-0003YG-8E; Thu, 01 Oct 2026 08:41:56 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xCCMS-0003Um-Ox for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ZOsy4WeQY2zuAKUTFvvOLHZJMBr9SESMUtL4y+ypHkE=; b=Z3JvaYJDIOfjdhZrBvw3BuUcx3 nnKKTucIHCGafBA6wzAIJrNbmrMoO/TGtkXBqMpgf4jVVOQ6NPr0UzReTcRvwnm7kf4z9XKm6u9/F 2hYB/GqmRfE1IUzRz4FbzkuH31C0ebSlIYPZ/R/unpCPTEW/UJ9rg3eGowivJ+mmWv7I=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ZOsy4WeQY2zuAKUTFvvOLHZJMBr9SESMUtL4y+ypHkE=; b=Wvj42WXEivk7GL8wZIjEXFw1uw BsABS8Ge/5wl5IExG2wSaaA4dKrerSwbE3e9xufxSdU8bBoV9N84Aa19y5N1Z37DdlUR2IEqfA6en yELsJF2cV2vVxAf9bNc0ME72RZ0XGmM7iEe9C2qwpBrHpq1foFkAKwnutwuj2ZQPno9s=; Received: from mout-b-203.mailbox.org ([195.10.208.52]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xCCMM-0002x8-Su for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:50 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-203.mailbox.org (Postfix) with ESMTPS id 4hwQPb0C5tzLm8j for ; Thu, 01 Oct 2026 10:41:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1790844099; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZOsy4WeQY2zuAKUTFvvOLHZJMBr9SESMUtL4y+ypHkE=; b=xnz+7hHNrWJHWpvxmooYXp0BRc8/sRfTdOJtigxhttyR29enJNlxRpDEzbg/ULjENq/s/2 39JYuA+erRrz/TvOutX5pGnMNQV8QQN4m8uzBHCWGYvA8GDS2Ci7pibqs3PiabynxYWXj3 jW9dCzbbUl8X/3ws666DSCW26j/LIyJO3H/gD2n08ReI9GK2dm9v5st6rZv9Z6becxwPHg jcdRvVrhyrBfSCdHU6xg29TNBVlpcduenC5v3BizIhqvsZta1v64ZkdLo5I145NU38btH/ CqMRgo3grCmG6F25VEL494t3w+PfhrtHNsOcsvKWIb3ES+iSl8we4MkuH31gEA== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 1 Oct 2026 10:41:25 +0200 Message-ID: <0431607b16294b9521e908f4d29a524698e39bb7.1790843153.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hwQPb0C5tzLm8j X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path ca [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URI: mandelbit.com] -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1xCCMM-0002x8-Su Subject: [Openvpn-devel] [PATCH ovpn net-next 6/8] ovpn: coalesce UDP data records with GRO X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1877836163804243057 X-GMAIL-MSGID: 1877836163804243057 Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path can detach and authenticate records independently. Match the complete opcode/key/peer header and require compatible outer- network and checksum state. Flush on short records, differing segment geometry, existing GSO input, or the 64-record limit. Export skb_gro_receive_list, which is already shared by the core UDP and TCP frag-list GRO paths, so modular ovpn can use the same primitive instead of maintaining a local copy. Packet taps may clone a completed aggregate before UDP receives it. Making the aggregate head private retains shared references to its child skbs, so also clone and unshare each shared child before detaching the list and handing its records to crypto. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without receive GRO 19.279 Gbit/s 19.819 Gbit/s With frag-list GRO 22.773 Gbit/s 22.796 Gbit/s The preceding UDP GSO transmit path and hardware UDP segmentation were enabled in both cases. Frag-list GRO improved throughput by 18.1% forward and 15.0% reverse. Signed-off-by: Ralf Lici --- Changes since RFC v4 https://lore.kernel.org/openvpn-devel/e8f042bbdaeca7d9993cae49b08c4aedd8b2cbb9.1789558856.git.ralf@mandelbit.com/ - Refresh the performance measurements. drivers/net/ovpn/io.c | 7 +- drivers/net/ovpn/udp.c | 214 ++++++++++++++++++++++++++++++++++++++++- net/core/gro.c | 1 + net/ipv4/udp_offload.c | 3 +- 4 files changed, 218 insertions(+), 7 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index d76d48d945c5..c6f24b5d85dd 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -70,11 +70,10 @@ static void ovpn_netdev_write(struct ovpn_peer *peer, struct sk_buff *skb) unsigned int pkt_len; int ret; - /* - * GSO state from the transport layer is not valid for the tunnel/data - * path. Reset all GSO fields to prevent any further GSO processing - * from entering an inconsistent state. + /* the transport encapsulation and its GSO metadata do not describe the + * decrypted inner packet */ + skb->encapsulation = 0; skb_gso_reset(skb); /* we can't guarantee the packet wasn't corrupted before entering the diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 4802d982de08..20143eee351f 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -11,8 +11,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -27,6 +29,209 @@ #include "socket.h" #include "udp.h" +/* like UDP and TCP frag-list GRO */ +#define OVPN_UDP_GRO_CNT_MAX 64 + +static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) +{ + const unsigned int offset = skb_gro_offset(skb); + + /* GRO replaces its frag0 pointer after holding an skb, so keep the + * openvpn header linear for later candidate comparisons + */ + if (!pskb_may_pull(skb, offset + OVPN_OPCODE_SIZE)) + return false; + + *header = get_unaligned_be32(skb->data + offset); + return true; +} + +static struct sk_buff *ovpn_udp_gro_receive_fraglist(struct sock *sk, + struct list_head *head, + struct sk_buff *skb) +{ + const unsigned int gso_size = skb_gro_len(skb); + struct sk_buff *p, *pp = NULL; + u32 header, header2; + int ret = 0, nhoff; + bool flush; + + if (!ovpn_udp_gro_header(skb, &header) || + FIELD_GET(OVPN_OPCODE_PKTTYPE_MASK, header) != OVPN_DATA_V2) { + NAPI_GRO_CB(skb)->flush = 1; + return NULL; + } + + /* do not nest an existing GSO packet in the record list */ + if (skb_is_gso(skb)) { + NAPI_GRO_CB(skb)->flush = 1; + return NULL; + } + + list_for_each_entry(p, head, list) { + if (!NAPI_GRO_CB(p)->same_flow) + continue; + + /* match opcode, key ID and peer ID */ + if (!ovpn_udp_gro_header(p, &header2) || header != header2) { + NAPI_GRO_CB(p)->same_flow = 0; + continue; + } + + /* GRO has already matched the outer addresses and UDP ports; + * check the remaining outer IP fields + */ + nhoff = skb_transport_offset(p) - + NAPI_GRO_CB(p)->network_offset; + flush = __gro_receive_network_flush(udp_hdr(skb), udp_hdr(p), p, + nhoff, false); + + /* The first record determines the nominal GSO size. A shorter + * final record may follow it, but a larger record cannot. + * Checksum metadata must also be uniform because the aggregate + * exposes only one checksum state. + */ + if (gso_size > skb_shinfo(p)->gso_size || flush || + skb->ip_summed != p->ip_summed || + skb->csum_level != p->csum_level) { + pp = p; + } else { + /* skb_gro_receive_list pulls the headers already + * processed by GRO before linking this skb to the + * record list so we have to manually preserve the + * outer network header location for later handling + */ + nhoff = NAPI_GRO_CB(skb)->network_offset; + skb_set_network_header(skb, nhoff); + ret = skb_gro_receive_list(p, skb); + } + + /* complete the aggregate if the append failed, or after + * appending a shorter final record, or after reaching the + * record-count limit + */ + if (ret || gso_size != skb_shinfo(p)->gso_size || + NAPI_GRO_CB(p)->count >= OVPN_UDP_GRO_CNT_MAX) + pp = p; + + return pp; + } + + return NULL; +} + +static int ovpn_udp_gro_complete(struct sock *sk, struct sk_buff *skb, + int nhoff) +{ + /* udp_gro_complete has already marked this as a UDP tunnel GSO packet. + * Keep that type so UDP passes the aggregate directly to the encap cb, + * where the original record skbs are detached. + */ + skb_shinfo(skb)->gso_segs = NAPI_GRO_CB(skb)->count; + + /* Each outer UDP checksum was either validated (or accepted in case of + * checksumless UDP) before its record was merged in + * skb_gro_checksum_validate_zero_check. + * The checksum in the aggregate cannot describe the concatenation of + * independent UDP payloads, so we preserve the validation result. + */ + skb->ip_summed = CHECKSUM_UNNECESSARY; + skb->csum_level = 0; + skb->csum_valid = 0; + + return 0; +} + +/* skb_gro_receive_list keeps the first openvpn record in 'skb' and links the + * remaining records through fraglist. Here we segment by detaching that list + * before delivering the records individually, and remove the child skbs from + * the head skb's length and memory accounting so the head describes only the + * first record again. + */ +static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) +{ + unsigned int data_len = 0, truesize = 0; + struct sk_buff *curr, *list; + int ret; + + /* IP reassembly may also use fraglist, but it is not a record batch */ + if (!skb_is_gso(skb) || + !(skb_shinfo(skb)->gso_type & + (SKB_GSO_UDP_TUNNEL | SKB_GSO_UDP_TUNNEL_CSUM))) + return NULL; + + if (unlikely(!skb_shinfo(skb)->frag_list)) + return NULL; + + /* packet taps may have cloned the aggregate before it reached UDP, so + * make shared info private before removing its fraglist + */ + ret = skb_unclone(skb, GFP_ATOMIC); + if (unlikely(ret)) + return ERR_PTR(ret); + + list = skb_shinfo(skb)->frag_list; + + for (curr = list; curr; curr = curr->next) { + data_len += curr->len; + truesize += curr->truesize; + } + + skb_shinfo(skb)->frag_list = NULL; + skb->len -= data_len; + skb->data_len -= data_len; + skb->truesize -= truesize; + + return list; +} + +static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct sk_buff *list, *next; + + list = ovpn_udp_gro_detach(skb); + if (IS_ERR(list)) { + ovpn_dev_dstats_rx_dropped(peer->ovpn->dev); + kfree_skb(skb); + ovpn_peer_put(peer); + return; + } + skb->next = list; + + skb_list_walk_safe(skb, skb, next) + { + /* skb_unclone can leave fraglist children shared with a packet + * tap, so make each skb header private before changing its + * list pointer or control block. skb_cow_data makes the packet + * data writable later in the decrypt path. + */ + skb = skb_share_check(skb, GFP_ATOMIC); + if (unlikely(!skb)) { + ovpn_dev_dstats_rx_dropped(peer->ovpn->dev); + + /* carry the current record's peer reference to the + * next record or release it at the end of the list + */ + if (!next) + ovpn_peer_put(peer); + continue; + } + + skb_mark_not_on_list(skb); + + /* keep the current reference alive for the next record before + * handing this one to crypto + */ + if (next && unlikely(!ovpn_peer_hold(peer))) { + DEBUG_NET_WARN_ON_ONCE(1); + kfree_skb_list(next); + next = NULL; + } + + ovpn_recv(peer, skb); + } +} + /* Retrieve the corresponding ovpn object from a UDP socket * rcu_read_lock must be held on entry */ @@ -121,8 +326,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); - skb_mark_not_on_list(skb); - ovpn_recv(peer, skb); + ovpn_udp_recv(peer, skb); return 0; drop: @@ -408,6 +612,8 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock, .encap_type = UDP_ENCAP_OVPNINUDP, .encap_rcv = ovpn_udp_encap_recv, .encap_destroy = ovpn_udp_encap_destroy, + .gro_receive = ovpn_udp_gro_receive_fraglist, + .gro_complete = ovpn_udp_gro_complete, }; struct ovpn_socket *old_data; int ret; @@ -454,6 +660,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) { struct sock *sk = ovpn_sock->sk; + udp_tunnel_cleanup_gro(sk); + /* Re-enable multicast loopback */ inet_set_bit(MC_LOOP, sk); /* Disable CHECKSUM_UNNECESSARY to CHECKSUM_COMPLETE conversion */ @@ -462,6 +670,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) WRITE_ONCE(udp_sk(sk)->encap_type, 0); WRITE_ONCE(udp_sk(sk)->encap_rcv, NULL); WRITE_ONCE(udp_sk(sk)->encap_destroy, NULL); + WRITE_ONCE(udp_sk(sk)->gro_receive, NULL); + WRITE_ONCE(udp_sk(sk)->gro_complete, NULL); rcu_assign_sk_user_data(sk, NULL); } diff --git a/net/core/gro.c b/net/core/gro.c index 29b4d02bf519..b6acedc919f8 100644 --- a/net/core/gro.c +++ b/net/core/gro.c @@ -262,6 +262,7 @@ int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb) return 0; } +EXPORT_SYMBOL(skb_gro_receive_list); static void gro_complete(struct gro_node *gro, struct sk_buff *skb) { diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index cf07c3c6611a..187f108f3ee8 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -40,7 +40,8 @@ struct udp_tunnel_type_entry { #define UDP_MAX_TUNNEL_TYPES (IS_ENABLED(CONFIG_GENEVE) + \ IS_ENABLED(CONFIG_VXLAN) * 2 + \ IS_ENABLED(CONFIG_NET_FOU) * 2 + \ - IS_ENABLED(CONFIG_XFRM) * 2) + IS_ENABLED(CONFIG_XFRM) * 2 + \ + IS_ENABLED(CONFIG_OVPN)) DEFINE_STATIC_CALL(udp_tunnel_gro_rcv, dummy_gro_rcv); static DEFINE_STATIC_KEY_FALSE(udp_tunnel_static_call); From patchwork Thu Oct 1 08:41:26 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5422 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6ac1:b0:8b3:6e77:b38b with SMTP id v1csp1809341maw; Thu, 1 Oct 2026 01:41:57 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBxBV911hiDKnE2zyIfXj0RVv3S6ZfBnrQX2aX+rTlReGv2sNDSZl4cF7gh4b9iqWIxcJfmVui0P4XA=@openvpn.net X-Received: by 2002:a05:6808:c004:20b0:4f3:890b:5fde with SMTP id 5614622812f47-4f3891aa860mr1306264b6e.3.1790844116884; Thu, 01 Oct 2026 01:41:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790844116; cv=none; d=google.com; s=arc-20260327; b=LZdUvs9UFk0tvS3xXECKAQzBL93Fvj4k/wGYuyPgOh5NwCpodzsXkEwu8YtaQ2a7Iy 5GfrRB7FlFt3aUcleLd7JQpRIsXcCqNa4QIlNqyftjOd8mmGVDgPzV8juSu3I5E3XVKM v4EwPEQpOl+mAasuGgP8WTpsLqcudyvyuUYFiR0QuKrcYiB2ckd+9u/Ui3iB27wd7vmP WgIEHqDKd4p5dMiLvigmGig6vgoyaNHshrVfsXLL9plV6BOTWIDCbhjt3KOx966ysgTI +9oVnVgCXlEx+AHdtLB0ukO1TW91ZY7O81luddcfIwmYHgg9hQEtf74sDUhX9slLfR9y 1CWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=1dDFO9yAYz4OD1/fnqJw357GrEY8nUpQ3q0ZaJcpUo4=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=HY3cRCQKqfyd78X/9GQxvPGlKWv5U/TzWmj04XYMk7IJHSxzbc4QmjOmg+gvW5lDWi q+XNxKY4jG4JKm5mthU/Y79oQphFkQrW3n2QWODB41t1XOHtQjOxrT2zL8iL3K3Z3TWt OQ84slFwtNFMNpjySLKL3WX1SoBLpsKbHnUrsovMeSUsKa+bL2S7fHtZeo6zKGkQ82D6 VYuRNKs+W/zWRWP9uJv2E0AM0dfEXKv1cCYd3yBCnVpOPiq/nlIEjisgIaC8pn21NTGp p2BvxEQ4rDphgiYn4X/u44V8QNLfi8yuIEGVG6cPdGInYTa2bw4FfkjfsTL8dCI0IDkE pFEQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="l4U+/xtC"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="WaC6/ocE"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=VpnaGY1A; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Fe08u8fg; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-49ded30cfc1si4881781fac.176.2026.10.01.01.41.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Oct 2026 01:41:56 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="l4U+/xtC"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="WaC6/ocE"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=VpnaGY1A; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Fe08u8fg; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=1dDFO9yAYz4OD1/fnqJw357GrEY8nUpQ3q0ZaJcpUo4=; b=l4U+/xtCG/TKsXDV7GNHSfsksO W2mzmYYA+4WauGfSPGpkp3DOvKwE9mhz5r2Rcc0lOCIMI1ip4m7HRMgMpyovi9x3kNvUwVxWDD7YT Br9GpY8pWfcDoe4AOifk3RzA+VufQAyvQ+7WD5iFPV772pE9q4p/2EqlT0v10ohPEdIc=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xCCMR-0000BJ-P1; Thu, 01 Oct 2026 08:41:52 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xCCMN-0000B2-WD for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=IlRIG2jW8pR2p+RTOGVuqQqiTAzzcNd0rWOVqhuCtt4=; b=WaC6/ocEHL/zslal1CKlbkw8q8 zFzUQ7msQDBOYn6/3xYqcLbo9/cUEzwPgkCAmk9ooyeSVO09EDCTRpNTEz7nR1w+REXjp+5EOId1X 5F20z6Cu04krREpRW5LdPV1Ym7XHeuinVXuPheftTKop2pCHMoilVFcmTnxX+sTAbhs8=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=IlRIG2jW8pR2p+RTOGVuqQqiTAzzcNd0rWOVqhuCtt4=; b=VpnaGY1A+arK7U4l8q0FXYx6BE G/GARn3yUcuGjslmZCCkoqrMQxjaQVRnsSq9WOO3TbQnLhQQP2497KccrW/x0dfei5E/aS8qdGPZg bgusJE964Damzf36YYE88XL7q9LbbVfOC1iz6cfPPyjv05w1/oW/5J3sHkvucMxH5AII=; Received: from mout-b-201.mailbox.org ([195.10.208.61]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xCCMN-0002xH-RQ for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:48 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-201.mailbox.org (Postfix) with ESMTPS id 4hwQPb3nYtzLll8 for ; Thu, 01 Oct 2026 10:41:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1790844099; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IlRIG2jW8pR2p+RTOGVuqQqiTAzzcNd0rWOVqhuCtt4=; b=Fe08u8fgB7lknk5z9KVAaA0tBN+z8ZXllG9/4tMj2WlZUyCHI1KoxU9mJKgy5ZdVaOCn9A EjzomCkjWfZEnODl1vVKHTwiwP/UTf6BpVC+zbpJQ4lgb/g6oaLxQGPsFKYFa2t2M+jAj5 sb0e20n2ymihL4yFNvCokRVeGpnrZ/uvesdZ9LOVIFqmQHTB7x4OmRajkobASsIIaX+tAH q3S+9k8W1b96wXUIJPvoZ1NKYQ2j2OzAfjG3U8ILRyQi1mer4Ai1NTG+8OYe6xNJmUfI5s WPBtbrX1Mcg9XOaZ4Z3dM1bPN+KeZzrpL4yY9SCo7OKraUOVtcUvrDLuWRng1w== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 1 Oct 2026 10:41:26 +0200 Message-ID: <0651258e11d4a0c290152a1f286b9d8a45c5acd4.1790843153.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (a [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xCCMN-0002xH-RQ Subject: [Openvpn-devel] [PATCH ovpn net-next 7/8] ovpn: prefetch encrypted records before GRO batch decryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1877836160580051763 X-GMAIL-MSGID: 1877836160580051763 Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (and maintain the same distance throughout the batch), overlapping their memory access latency with the current AEAD operation. An ordinary single-record UDP receive has no later record and therefore skips the prefetch path. Only prefetch the linear part of each skb. Walking non-linear fragments here would duplicate the scatterlist walk performed by crypto and could cost more than the cache hint saves. A same-binary comparison using three 30-second samples per direction found distances one and two effectively tied forward, while distance two was 3.3% faster reverse and less variable in both directions. Profiling also measured slightly fewer decrypt cycles at distance two than at one, while wider distances provided no repeatable benefit. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without RX prefetch 22.773 Gbit/s 22.796 Gbit/s With RX prefetch 24.028 Gbit/s 23.405 Gbit/s RX prefetch improved throughput by 5.5% forward and 2.7% reverse. Signed-off-by: Ralf Lici --- Changes since RFC v4 https://lore.kernel.org/openvpn-devel/b4d7ce8d66db966e5e43669f8180f6c59d4f6df0.1789558856.git.ralf@mandelbit.com/ - Cover every linear cache line when skb data is not cache-line aligned. - Rename the prefetch helper to make its linear-only scope explicit. - Document the comparison that selected a prefetch distance of two. - Refresh the performance measurements. drivers/net/ovpn/io.h | 15 +++++++++++++++ drivers/net/ovpn/udp.c | 21 ++++++++++++++++++++- 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index 1a94f0fda1d1..10c568c02234 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -10,6 +10,9 @@ #ifndef _NET_OVPN_OVPN_H_ #define _NET_OVPN_OVPN_H_ +#include +#include + /* DATA_V2 header size with AEAD encryption */ #define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ @@ -21,6 +24,18 @@ #define OVPN_KEEPALIVE_SIZE 16 extern const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE]; +static inline void ovpn_skb_prefetchw_linear(const struct sk_buff *skb) +{ + unsigned long addr = (unsigned long)skb->data; + unsigned long end = addr + skb_headlen(skb); + + /* crypto overwrites data in place, so request write ownership of each + * linear cache line before the AEAD implementation reaches it + */ + for (; addr < end; addr = ALIGN(addr + 1, L1_CACHE_BYTES)) + prefetchw((void *)addr); +} + netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev); void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb); diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 20143eee351f..807914fd2c8d 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -32,6 +32,9 @@ /* like UDP and TCP frag-list GRO */ #define OVPN_UDP_GRO_CNT_MAX 64 +/* leave enough work between a cache hint and the record which consumes it */ +#define OVPN_UDP_GRO_PREFETCH_DISTANCE 2 + static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) { const unsigned int offset = skb_gro_offset(skb); @@ -187,7 +190,8 @@ static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) { - struct sk_buff *list, *next; + struct sk_buff *list, *next, *prefetch; + unsigned int i; list = ovpn_udp_gro_detach(skb); if (IS_ERR(list)) { @@ -198,8 +202,23 @@ static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) } skb->next = list; + /* a fraglist GRO aggregate makes later ciphertext visible before the + * current record is decrypted, so we prime the first two records, then + * keep the cache hints the same distance ahead while draining the list + */ + prefetch = skb->next ? skb : NULL; + for (i = 0; i < OVPN_UDP_GRO_PREFETCH_DISTANCE && prefetch; i++) { + ovpn_skb_prefetchw_linear(prefetch); + prefetch = prefetch->next; + } + skb_list_walk_safe(skb, skb, next) { + if (prefetch) { + ovpn_skb_prefetchw_linear(prefetch); + prefetch = prefetch->next; + } + /* skb_unclone can leave fraglist children shared with a packet * tap, so make each skb header private before changing its * list pointer or control block. skb_cow_data makes the packet From patchwork Thu Oct 1 08:41:27 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5420 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6c89:b0:8b5:356c:6cee with SMTP id p9csp1748837may; Thu, 1 Oct 2026 01:41:57 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBxECoZMD/q7VLHJpQMJOl+FEqNnrQa4U4eQAsovS9RIRzoprgwFu8rk495NWnBQa0RGiPYjMTya6TY=@openvpn.net X-Received: by 2002:a05:6870:8130:b0:475:e066:771d with SMTP id 586e51a60fabf-49ddd58fe1dmr4263019fac.36.1790844117144; Thu, 01 Oct 2026 01:41:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1790844117; cv=none; d=google.com; s=arc-20260327; b=J0qa/jCbnvECcezq7eZRM9SGbjgJ0ja5GPXxf1QRQVot7FK52Kr0KmlwJX/GpP0X5P uhYQqzuuC3Q66SqWLzTB/l3cKfs0detLroq1OOqQs4Y3zV1sSfxbozFz96D7BUZrrnGj 6koBvkJ5Y4EAk7Tk+shV5539cQdPMgvEGBprTEcgQytM+p7SrgmDOBvIC0IEc6tGT36T IqdoG/BhajxbY7ykCT0r2nLNYQfoc3AZIVRqpaEivgj3whf2aCym91k2h2wQRlr/NyME 8ApxfxecPrr7ecnrmqDVd5JHH3822bhHX2FMfqPvI2Vel/C2fg0BquwNbiyvgslq0xA1 Ba5A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=1GPnV02CWGYQGlmQASt+4EDsmB6w/MQ8eoSYuBAlgOA=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=RVYAGQtGAQhjhwbOazJ4Sxvzf5m5MQpOIQOq2LD/kamWnMIZFoVOql/IKEvGCrI3Rs Y6VfkqBlWofinMp7k6ft5agwWNAGywBECHHlKRMY3QpCRumNow+o3Wl3uGgoiGM5P1B2 78hvCwgHkqByGeTaMnphMMMbWqCKQJg1jxW7BQZJ+0NxnYGDddo5a9O3ldCJ0sU2ysz1 Ykcuu1TSdmkIfFXiJ0TRjEPX9uTNozb5PnuewsOp7mcCJsGl0h9Bvzq6upGlnUo13R3P w0icTy35N/Gqw70IdOKtlEw/lWbG0AOMPUoaSmn4Tdt3dHVvoi1BFHrVpYVCIl6miRUv S6CA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=jENC0pbu; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Xj4ov6v9; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Wv0K3F+G; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=D1hSP8A1; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-49ded37805asi4252742fac.235.2026.10.01.01.41.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Oct 2026 01:41:57 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=jENC0pbu; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Xj4ov6v9; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Wv0K3F+G; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=D1hSP8A1; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=1GPnV02CWGYQGlmQASt+4EDsmB6w/MQ8eoSYuBAlgOA=; b=jENC0pbuhw5IxVF9oEursXYgWy o7SJgzNdTkmkDa8oNj8Y8lhxGkv/F4jCJvp7AFQS0poS9GtoIHV5VRnbVFyc19l6XcveStofJ1tl7 04wDa7otfpMbtzHSC+P2/mDfo6GxEH3scrg2dCobFsny1vzSCMBVl+byAYGFv5gtcDW4=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xCCMS-0003jr-FL; Thu, 01 Oct 2026 08:41:53 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xCCMP-0003jS-Me for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:50 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=O37uHT1kjSCYNotZ0DEWCd5x7o5TRKtgQUTkcEF0DnI=; b=Xj4ov6v9M3qR9/xQYb8eeMeQK4 OKdjmZ6v2Onb8PgDRXaaR3rhwfI0Qol/sLXBHL4ELXS8SMRpCHATYIYY2V7vg3IriVeukLNaH7o0T 8u4mTvVHyI4B3t1RwMtwRtJzNkXPWa6/OvLzuLhIgAXlc2g+kwtNUoelEB0w2n2NhjxU=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=O37uHT1kjSCYNotZ0DEWCd5x7o5TRKtgQUTkcEF0DnI=; b=Wv0K3F+G90vjow6lbtqKiRPapM mGGcak6zjIyEss3WHiSqW9GAp4DuDZiGq0IOgPFFJ1nPEjKs0rOVhmSveYfnGWd/2yeXerE7kiN8p a+XU5pZWKMZVODdw+cQQLqQxO368PHUIUGhZkbFF6IvAXDLiY874rATvJiVd4uvHK83U=; Received: from mout-b-106.mailbox.org ([195.10.208.46]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xCCMO-0003UM-8a for openvpn-devel@lists.sourceforge.net; Thu, 01 Oct 2026 08:41:50 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-106.mailbox.org (Postfix) with ESMTPS id 4hwQPc3kQXzNm05 for ; Thu, 01 Oct 2026 10:41:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1790844100; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=O37uHT1kjSCYNotZ0DEWCd5x7o5TRKtgQUTkcEF0DnI=; b=D1hSP8A1HbFOejTdxQIeRzZW4TlFRRsAoZGJ1IjC6VmbWohL4mL2wHR3GYE5+CsNqWr8b9 G0w9Z9uxt/GLuKdnSWuUeIUsC0I8PjiuvgrXvkTP071Hr4AxwhHgdNTU4IHZKsguqtwa+d fRLxOGk4UVNfJQwB3spZ1r2Dn337PEZXMorsv1bAvS/+qggp8lDOWSc8+UPYF9dhU2Mo/Y tcy0kLekr+3FcNA+cfZ0E9VjMpu7cr4MsGrhk+zfZZY+YJTVCIqKO61TqWpFSAQkleeXpG /o4bYvbQzunO3SYBIQNJD1+tgX43+/kcQ3+EX8+nMwCSAep86ISannGG5htLbA== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 1 Oct 2026 10:41:27 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hwQPc3kQXzNm05 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Software segmentation exposes the complete skb list before encryption begins. Use encryption of the current segment to overlap fetching data needed by the next one. For segments encrypted in place, request write ownership of every cache line in the next segment's linear area. When converting ordinary GSO input into an out-of-place UDP GSO aggregate, the linear he [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URI: mandelbit.com] -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1xCCMO-0003UM-8a Subject: [Openvpn-devel] [PATCH ovpn net-next 8/8] ovpn: prefetch GSO segments before encryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1877836161252502603 X-GMAIL-MSGID: 1877836161252502603 Software segmentation exposes the complete skb list before encryption begins. Use encryption of the current segment to overlap fetching data needed by the next one. For segments encrypted in place, request write ownership of every cache line in the next segment's linear area. When converting ordinary GSO input into an out-of-place UDP GSO aggregate, the linear head has just been touched by segmentation while most payload remains in shared page fragments. Read-prefetch the first direct fragment without repeating the complete scatter-gather walk performed by crypto. Fully linear segments and fragments without a directly accessible kernel virtual address skip the fragment hint. A single skb naturally skips both forms of lookahead. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without TX prefetch 24.028 Gbit/s 23.405 Gbit/s With TX prefetch 24.166 Gbit/s 23.558 Gbit/s This is a modest 0.6% increase in both directions. Signed-off-by: Ralf Lici --- Changes since RFC v4 https://lore.kernel.org/openvpn-devel/d085dda71a91670680cbaee1029a37d952338c35.1789558856.git.ralf@mandelbit.com/ - Retain write-prefetch for segments encrypted in place and add read-prefetch of the first payload fragment for segments encrypted into a UDP GSO aggregate. - Account for fragments without a directly accessible kernel virtual address. - Refresh the commit message and add performance measurements. drivers/net/ovpn/io.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index c6f24b5d85dd..6e28800286e1 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -388,6 +388,28 @@ static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) return true; } +static void ovpn_skb_prefetch_frag(const struct sk_buff *skb) +{ + unsigned long addr, end; + const skb_frag_t *frag; + + if (!skb_shinfo(skb)->nr_frags) + return; + + frag = &skb_shinfo(skb)->frags[0]; + /* prefetch requires a directly accessible kernel virtual address */ + addr = (unsigned long)skb_frag_address_safe(frag); + if (!addr) + return; + + end = addr + skb_frag_size(frag); + /* the linear head was just touched by segmentation, so prefetch the + * first shared payload frag without repeating crypto's full sg walk + */ + for (; addr < end; addr = ALIGN(addr + 1, L1_CACHE_BYTES)) + prefetch((void *)addr); +} + static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, struct ovpn_peer *peer, struct sk_buff *batch, @@ -429,6 +451,12 @@ static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, skb = __skb_dequeue(skbs); len = skb->len + OVPN_DATA_V2_OVERHEAD; + /* overlap fetching the next segment's shared payload with + * encryption of the current segment + */ + if (i + 1 < segments) + ovpn_skb_prefetch_frag(skb_peek(skbs)); + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_skb_cb(skb)->batch = batch; ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, @@ -490,6 +518,12 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, * independently */ skb_list_walk_safe(skb, curr, next) { + /* encrypting this segment can hide the cost of fetching the + * next segment's data into the cache + */ + if (next) + ovpn_skb_prefetchw_linear(next); + if (unlikely(!ovpn_encrypt_one(peer, curr))) { ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr);