From patchwork Wed Sep 16 08:35:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5357 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5930056mag; Wed, 16 Sep 2026 01:35:48 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwJSiPHKizcWNleHr4kuoIUh7m6kge45iTXQ8KhgAa1qfD2NC1rDHVOvMR6OGz77/wICN3X5E6H6m8=@openvpn.net X-Received: by 2002:a05:6820:2012:b0:6c6:9743:e118 with SMTP id 006d021491bc7-6c7d1bc8514mr3307770eaf.8.1789547748567; Wed, 16 Sep 2026 01:35:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547748; cv=none; d=google.com; s=arc-20260327; b=Iws29LlQDDezhauqgnvibvIbodYkolUcv6V26QG/gEXMs70FeJi7dd0crqi4djnxiF 2RO+PH3pn8KE4qicT0DEeMag8CCSfdWabMvJkLgJhk8vci27868vvKD02IbqJdJYgcgF Q7lUauPQw7JUCK9acJuWQ+rkibzjdek78ELJi9fbMcT4T6EJgRQmRXyeNItwp4D+FV/1 llFgAtuL3F02fGHlC6nSO4fY9PnZ8V6jZEJnhM4RWCWV+0G/tnWzH3OVN9KSgyChvhgb CBmLdyTNT3DrkAMw7MwH5Kal97Y3CHlY67Ee2M44h8HsTtXlSIG3QGUkgtpoZb+lg4Vp v3mQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=jlX0s5H+x2Q/mV73j+18lEUJG3VMB9oItXO9tda0NIQ=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=QPsTYiVwGDI9dBcM6x2J9JHtVo8o7slS2mOMDxvkWbYjE4lOmCnhI7q+ynVRnnHVEO l8tZw0L57PeCY/4i9ZiIZ/BraifgSrIO8Wxyc8QLXRPHVpUGhA5x0VwHFCgHrJdmTjQL YB29E1+xsKNyZlqR8D7XZhE60cer67V4jZ++uQdDRGIPca8uGZYeJ1BxTtWXpifGdjPX lqmQRr2YJ1r6Qog0+gpEJMJo518EVoCebfHp6IO+Nygk6jMH/6NsUT4I5O7vXaTxRSg6 zUs6+D0VS2oGzX7eE/ZpF6lTIGjhrxq2bQttL/jOi1eZfUHypXn0aMmmV+0SOK78m9RZ 1erQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=EQzOLpny; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=UyOulbXL; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=JiMzGQjI; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=c4bLSl4B; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-4842aac032fsi2492658fac.312.2026.09.16.01.35.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:35:48 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=EQzOLpny; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=UyOulbXL; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=JiMzGQjI; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=c4bLSl4B; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jlX0s5H+x2Q/mV73j+18lEUJG3VMB9oItXO9tda0NIQ=; b=EQzOLpny1+L7FEJi9kQLtQADQR 14mxrqSTpGSJGmiXRF4JSDLtliNS5X+RKbZEsL0IwC/+1fnHwxWw7ejOZ7Km4swXhZO/w1hifJl/X qRda5Ifq1z20tet0/FO4St1McSXJldE66Dzq0EOc0UUjDTeCEJcCE3oGlp6NGgOdOkV0=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7G-0004XZ-6G; Wed, 16 Sep 2026 08:35:42 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7C-0004XQ-HM for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=6+ZJJ8XiuDG++HUlfyuIY73IeUadtL79nf1exCRGXkQ=; b=UyOulbXLktp3BrzHvLpys/CU4V VIr32fIJwbpfKSPrA660eVatqmFktv0K4ptIQEYTeji+RHkS/BGfDn9j/JC08qq1ZuryOdW/cIVOM aCJus0UaQ3SgWsRuYre/5u7N0eFXnZA4ALwLx4I4Lto8RFV2kEOwNgV8t2BkN+Vs6NnU=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=6+ZJJ8XiuDG++HUlfyuIY73IeUadtL79nf1exCRGXkQ=; b=JiMzGQjIS7IK5c8FQgo7BSbdj3 SlcmPQm02DJY8XBKwPigV8k7xNjM22qnzU2QXrtceNKNdFsJl8k1k9utRi/nNo6NVVCq2WiEC3qxX gRyx8mjBb3s6Ojx4nN7WO2m0hD2xF+8xJXKlJ6+T65qcZUlpKhwJVD8Sm1VJ+7DfFz+M=; Received: from mout-b-107.mailbox.org ([195.10.208.47]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7B-0000lq-RY for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:39 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [10.196.197.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-107.mailbox.org (Postfix) with ESMTPS id 4hlBzP3w9Zz3yFR for ; Wed, 16 Sep 2026 10:35:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547729; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6+ZJJ8XiuDG++HUlfyuIY73IeUadtL79nf1exCRGXkQ=; b=c4bLSl4BJ0RaBraLRFqThY4IpyU90P3Stp1QOBxmOnAZCpJ0X28qc31Ghmz7kRAYSN60X6 8vtDZAz+tqD2cU4SvbhWT1Gpj9wiuwPRm4Hs2IzNpPUx+4+7BzZKoh7jcel3gfCSzZHNA7 banJH9rkP28Gq8+sYWNUHA9sHuugZwk3PidK54TwLm0i7HYQMrsf8d2y7EMWtSwUO5GYGn wS3X5Ckm7b/TRah5tVCo1RfM+XMwjePUKTRf0L74yStujhRDOyT2TGgeGcYF84qzI7/Oq9 w1S5psct4+bwDsC+W8w0JBAFHhmMzCIenXoZW9V0JxPG91AhhzmqnDMrtg6luw== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:10 +0200 Message-ID: <1bbfde37488ade61928554db3a119525adf7b608.1789546917.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1x6l7B-0000lq-RY Subject: [Openvpn-devel] [RFC ovpn net-next v3 1/9] ovpn: advertise checksum offload for GSO packets X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476820070469529 X-GMAIL-MSGID: 1876476820070469529 ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before they reach ovpn because TCP GSO packets normally carry CHECKSUM_PARTIAL state. Advertise NETIF_F_HW_CSUM so the stack can pass such packets to ovpn. Complete partial checksums after any GSO segmentation and before submitting packets for encryption, since the inner packet checksum can no longer be fixed after the packet has been encrypted. Also pass the ovpn feature set to skb_gso_segment with GSO capabilities masked out: this forces software segmentation, but still lets the segmenter preserve supported non-GSO properties such as non-linear skb data instead of needlessly linearizing. Signed-off-by: Ralf Lici --- No changes since v2 https://lore.kernel.org/openvpn-devel/1bbfde37488ade61928554db3a119525adf7b608.1789540779.git.ralf@mandelbit.com/ No changes since v1 https://lore.kernel.org/openvpn-devel/1bbfde37488ade61928554db3a119525adf7b608.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/io.c | 20 ++++++++++++++++++-- drivers/net/ovpn/main.c | 2 +- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 9526f8096da6..112067ded401 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -358,6 +358,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; struct sk_buff_head skb_list; + netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; __be16 proto; @@ -392,8 +393,13 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_dst_drop(skb); if (skb_is_gso(skb)) { - segments = skb_gso_segment(skb, 0); - if (IS_ERR(segments)) { + /* force software segmentation, but keep ovpn's non-GSO feature + * bits so the generated segments can preserve non-linear skb + * data where possible + */ + features = netif_skb_features(skb); + segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", netdev_name(dev), ret); @@ -418,6 +424,16 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) continue; } + /* NETIF_F_HW_CSUM requires completing partial checksums */ + if (unlikely(curr->ip_summed == CHECKSUM_PARTIAL && + skb_checksum_help(curr) < 0)) { + net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", + netdev_name(dev)); + ovpn_dev_dstats_tx_dropped(ovpn->dev); + kfree_skb(curr); + continue; + } + /* only count what we actually send */ tx_bytes += curr->len; __skb_queue_tail(&skb_list, curr); diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 0708249e9607..28e1eb06e127 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -157,7 +157,7 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { - netdev_features_t feat = NETIF_F_SG | NETIF_F_GSO | + netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Wed Sep 16 08:35:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5363 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5930389mag; Wed, 16 Sep 2026 01:36:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwCq3CWVwMVmokarGbT4TThhD6XL8WoHt8BBTkWguW/XdskNIjZMi6Z3kgCOTy/TD81kNyi++BXVRQ=@openvpn.net X-Received: by 2002:a05:6871:2b2a:b0:475:a153:2dcb with SMTP id 586e51a60fabf-48478060dc9mr1191773fac.33.1789547772625; Wed, 16 Sep 2026 01:36:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547772; cv=none; d=google.com; s=arc-20260327; b=ZkjX9j7bc5oyONGSDECV2B5Mf5pA5Aar2vG4NE7m+iBCGt9d45m9Onm0zdQUJwTFQ+ TMJgyHErjcCjJhdY9TAI8/6gHgo411HjQMfzp0Rd2lf5mwyvVY7Oe1LCT9x6t4gHgqjR nTkPn/yXSqRTqbbFCvmENJwVnUtK2EG9aARCo+gOcvNk2DoFbrRLpQKkrlhuvjhSNfUj ikx+FObDSp3L5N+W3O7CY/HtPXSB9hgdIy44TVc5Bu2tj37baq9snXm/9EQEdUp34Rai tLfwu+uKDghAwjCtvByipRTjRmVlfzqHIcxH3q0cQvAhwUz5m6LkAGpdqH2pjYCUaDki qScw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=iNaR5O1n2IhtDuUcLh3rSvj3kLbfBdlV/vXw1DShTYc=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=Vn9rIIQTAQCXH2pwITwmKYL7PTk+wQZ72e+ANas7J1ESuAweqRb5C52Oas9fLJ2LIH qjcFt6O9IuHSmfDsSuV/3liNz4TJH+UyTz/FvizeHCzTHLoWkhtlqMvpLIJkFxvTbvSf BcpKorKZJbqjYwCVGFHtgiiNJst5U/GThOa5pe3C2GRfF8WeY523FEbhWoY4YtdJRH6/ W4N+y4wNMb3W+/7ckxPLTvwa6KN98ySLkRHZ3pcaudQ/sFpjX534fTDMNkP3ocBuJh24 7cqGlsV1+qfXkbuI5Qfh8K66bQm7JFCjyHmwcgYO0IV3QEjFVVO1nrrN90vMwmS64p2O Kjkg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="inve/kHG"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=RuqRmo98; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=I5PLNdZ0; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Cz1tPqhK; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-48429cd0981si2313846fac.70.2026.09.16.01.36.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:36:12 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="inve/kHG"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=RuqRmo98; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=I5PLNdZ0; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Cz1tPqhK; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iNaR5O1n2IhtDuUcLh3rSvj3kLbfBdlV/vXw1DShTYc=; b=inve/kHGtK/crmymfkSNTGc453 vLJdm2TMpvJOVTU2f3S026/A83O7MGI0hr0+UaZByitQqN9OBW+9fxWf5SPswoIM1waHp7SckQlQr xOG5GWX/BSCqGAvCwhSWITeyL+Ut8HoMu/kShH9ExSKlJ0pXfyVz17X+k5gXl71NfWk4=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7k-0004jN-GQ; Wed, 16 Sep 2026 08:36:10 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7H-0004a8-Gw for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=/EAaRb2p1+Wq/jWP26wAYJvUx90Q98xdAczu55sYetk=; b=RuqRmo983AsSFVLxqLpMfnjogm H7KqWcTcfayVgTgPXOL46Guakd81X2OdHPo7lm97tsct5X9BFZa2mgdUJVdK0tUHZEOJacvC2eOXk wG5yL7EzWJQeVLe2ffiBGGLq0/CyLX4S3yAu3jlbs3nqrvvcbY230GYgkW4MB8v34QA8=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=/EAaRb2p1+Wq/jWP26wAYJvUx90Q98xdAczu55sYetk=; b=I5PLNdZ0pYxwumK5FhXbZPJCGj orQKjQjhIKj800CuOTwzZwtv41VBA/STu8/ErA4402jkmEE3HELIekqcvLGelPDPqaBP8G06YGd9E LOZHddaPWbEbl3nbd1wJmNCu1a2V7zFSEeU/S2m1OAmT7y0Hjmcqnchg34hBr0PNAEPM=; Received: from mout-b-106.mailbox.org ([195.10.208.46]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7C-00053E-2D for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:40 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:b231:465::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-106.mailbox.org (Postfix) with ESMTPS id 4hlBzQ1djhzNlt8 for ; Wed, 16 Sep 2026 10:35:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547730; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/EAaRb2p1+Wq/jWP26wAYJvUx90Q98xdAczu55sYetk=; b=Cz1tPqhKwNis+faAg0+qsHVcqN4bo345xW98NUTzlABJ789PoVY0iX9RUyxDIvcItpWxCo QFRckzLBAXblMt/+ztLd/1EDNY6c3/mTNWTFp5LzVaDHn7gBCxJ7xpnRKRi9rKpmnmL9e3 UchlkmWQrMszmQElPE6GQ9B1K1ZZZPJCoEnqre8gj2qyOFQnQf5bTWePqcMPg4DAdh0mXq U+HPoZyeiy5zOcYy3zuP/8xUE1Rlh9OLvFI6XECGw4iA9TFHXOOA4vGxM8wRPrFjbb2SbY A9MCLcMFiauNYAQR5Sk6g2Y0N2Q08JaT6d0R7Sc+yVlsA4wgKIVnHVDhXB/KCQ== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::2 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:11 +0200 Message-ID: <5baa1d94e29e7b109ffd412b4f1d5d113a319389.1789546917.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hlBzQ1djhzNlt8 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_ [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1x6l7C-00053E-2D Subject: [Openvpn-devel] [RFC ovpn net-next v3 2/9] ovpn: accept frag-list GSO input X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476845528160971 X-GMAIL-MSGID: 1876476845528160971 Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_FRAGLIST, so generic transmit validation segments these aggregates before calling ovpn_net_xmit. That segmentation is functionally correct, but causes ovpn_net_xmit to be invoked separately for every resulting packet. Advertise frag-list storage so ovpn receives the aggregate intact and performs protocol validation and destination-to-peer lookup once before segmenting it. Frag-list GSO segmentation recovers the complete child skbs, which can then be encrypted in place and transmitted independently. Rebuilding those children into a replacement UDP GSO aggregate was found to add cost rather than improve throughput. The feature also admits non-GSO frag lists, which describe one packet split across several skbs. Let skb_cow_data preserve small lists directly. If a list exceeds the AEAD scatterlist limit, linearize it and continue rather than rejecting an otherwise valid packet. Signed-off-by: Ralf Lici --- No changes since v2 https://lore.kernel.org/openvpn-devel/5baa1d94e29e7b109ffd412b4f1d5d113a319389.1789540779.git.ralf@mandelbit.com/ No changes since v1 https://lore.kernel.org/openvpn-devel/5baa1d94e29e7b109ffd412b4f1d5d113a319389.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 8 ++++++-- drivers/net/ovpn/main.c | 3 ++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 74eaf6fac2f5..2af493fd5735 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -168,8 +168,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, if (unlikely(nfrags < 0)) return nfrags; - if (unlikely(nfrags + 2 > (MAX_SKB_FRAGS + 2))) - return -ENOSPC; + if (unlikely(nfrags > MAX_SKB_FRAGS)) { + ret = skb_linearize(skb); + if (unlikely(ret)) + return ret; + nfrags = 1; + } /* allocate temporary memory for iv, sg and req */ tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 28e1eb06e127..ac4e0d85e215 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -158,7 +158,8 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | - NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; + NETIF_F_GSO_SOFTWARE | NETIF_F_FRAGLIST | + NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Wed Sep 16 08:35:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5359 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5930057mag; Wed, 16 Sep 2026 01:35:48 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBzEQMkkDS+XUdYcEBFy4xde5Y7N36JVmjNv8y/YrBHyHFpuYelbFuSnXw0VQb7+9aLUr+l8iAwFYPw=@openvpn.net X-Received: by 2002:a05:6830:410f:b0:7fa:ab72:9e01 with SMTP id 46e09a7af769-80b2f396f56mr1772221a34.25.1789547748567; Wed, 16 Sep 2026 01:35:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547748; cv=none; d=google.com; s=arc-20260327; b=oGqZk0UPPXMs2QGq2ynnl6rmzw5KRR1sHiZ0APB/qnvMrfKJlqd++kkfzMjyiVCTCb xF+zdt2dnEag4enJnrIdmHBN5Ri8yAtz7cj2R80cFMw1PKol+mKcuWhUD7DVKwndO1Ll YsTUnPtHyFt7nZH+BqU6aiAZBKlhp1PD8I3mmYYmGCHQvMu9jZTwXbVa6Npefyx9lizL dzcDdHtmw4d9FF5cKusUKFOhuVXLEPOnRN3RV+YoIt1RhsYTIS3Cl5u/zzLq1ISi+X+a vliFPr5sHbK9wREKFFVA1iNOkZqgEYG09Fwq2HuUUqmrHaIPba8OGsAqpJ9P8PSQVdlE PzVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=C7FnHyMcVRTkqbafZ6Cm7W4fYzzotAKHIQjXaNs10Uc=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=m+ljNp0wHrN5jm8EyMgtPzDEZFcQPOVbCNIJH48WSO5fUIpRFn/g+BUrL57CtwqOs3 0S4avlgLqTjUYN2Rlt6tsIDXDmYT3tZeLVb1Yrh7JbY+C+Jg/5l0FYPkUV8UwADjUhGb 2uvPHLdAs2GJinlG7ZMs7sZcUDBBCYvh1VcotxjCmIlYhfEB98Al5nqVlpvUb4chxT+H ta+kmaJ502GnnXLM5JmwwhqX+mhKFZu8E5LJVEk8ax8mre373Dqews+NP0WEGzTNbbBh c+uwSHZyCvno/YtGWTsnRUBXWRmUVrRWDm4/vr/GWi54lNGKk1RTkQI/sOfSLetam3Xt 9Wzw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=GuDd0hrw; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=WkN1IAjf; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=D3vWtmBp; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=bJLeO+RH; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-4842a13a46asi2667993fac.178.2026.09.16.01.35.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:35:48 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=GuDd0hrw; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=WkN1IAjf; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=D3vWtmBp; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=bJLeO+RH; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=C7FnHyMcVRTkqbafZ6Cm7W4fYzzotAKHIQjXaNs10Uc=; b=GuDd0hrwR2jsv3PXYaL80gXoPu EpYoSw3Uj6bFm9gnI58DPFsDJ7ugdRas9svVeTW+URInnQYmXNwnMWgJykX3SmJ9yyNSAjx+BmlWz m6FSzHUPb8VstEwmmdBwPNH5v//PNJeXTUw2g2nRpl85DVsoB8ZmDVZsQT8CkD1BiEtI=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7G-0003kl-FO; Wed, 16 Sep 2026 08:35:43 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7D-0003kc-OH for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:41 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=V6hHsVAdXzUo+UFexihULf5k7hdJH9qMlLw5V7oTT8U=; b=WkN1IAjf95JQiWrWFqMDh3g4Rh KYDcl59IXnsdOxJErvyVX6i4kKrmMt9xx9lEKbMKWJzqWQpJYhU8/9/S5nERo2jLo+FO2zR9AqpOn ZOD3/LASE5lqZbMWcGC9SK4eW5FT7p7IWnUHRlDxMy6ichSyfab7E2GFcsukMOZBK00k=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=V6hHsVAdXzUo+UFexihULf5k7hdJH9qMlLw5V7oTT8U=; b=D3vWtmBpzNBykmQGTWQeq0Eao0 6InGzwBSQYDesGyvfJq3AF8OCUkStnXJ28AofFsd2l3+HhtFef/Z0S7I/VVAHopXztLy9R9iceotA knnN4biC4mk9qmyBlqYoLb4hWezN+LCEb9aE8RtnjtZ9+m7qX+qxopAgMdtmXIOr0XKI=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7C-0000lu-Nn for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:40 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:b231:465::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4hlBzQ605Gz5vNT for ; Wed, 16 Sep 2026 10:35:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547730; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=V6hHsVAdXzUo+UFexihULf5k7hdJH9qMlLw5V7oTT8U=; b=bJLeO+RHHrFmfIp/IgA4wgDr1VV3kYMWpY2XsYyUFBgTSS261PxGCk93k4H3p1BIhR5gQP reWlviS+ElEd7XruFkduFpwHGbPoUzs9TMs4d2myvBfWRnjwzhjzGfJ0mic7UWRRXVj5aq nzjyzB3iXpk+9QZrAq7UEDIhgNjQo6FcWvKa779cKivQDa3980a0QkCqpT7Xzd6p+YNJQY q1wOwVYnbXoCLMcn/msyvRNn7cdpBtZfN2n3X2bl5sTb057+qBwbiQk9S9GbpLbAufZV5C Xp09ayvdmiNJnfj1oO7S4guZoyjJ/AOw8CzQDfMOKE/3c1q5TLu2gvvpXrIERw== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::2 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:12 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hlBzQ605Gz5vNT X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6l7C-0000lu-Nn Subject: [Openvpn-devel] [RFC ovpn net-next v3 3/9] ovpn: refactor AEAD encryption helpers X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476820170498746 X-GMAIL-MSGID: 1876476820170498746 Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without changing packet handling. Signed-off-by: Ralf Lici --- No changes since v2 https://lore.kernel.org/openvpn-devel/f075b9782b14d64756c84e132c138482f08287b9.1789540779.git.ralf@mandelbit.com/ No changes since v1 https://lore.kernel.org/openvpn-devel/f075b9782b14d64756c84e132c138482f08287b9.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 97 ++++++++++++++++++++-------------- drivers/net/ovpn/io.h | 3 +- drivers/net/ovpn/proto.h | 4 ++ 3 files changed, 63 insertions(+), 41 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 2af493fd5735..30299581422d 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -24,9 +24,6 @@ #include "proto.h" #include "skb.h" -#define OVPN_AUTH_TAG_SIZE 16 -#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE) - #define ALG_NAME_AES "gcm(aes)" #define ALG_NAME_CHACHAPOLY "rfc7539(chacha20,poly1305)" @@ -42,7 +39,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * an AEAD request structure with extra space for SG * and IV. * @tfm: the AEAD cipher handle - * @nfrags: the number of fragments in the skb + * @nents: the number of scatterlist entries * * This function calculates the size of a contiguous memory block that includes * the initialization vector (IV), the AEAD request, and an array of scatterlist @@ -54,7 +51,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * Return: the size of the temporary memory that needs to be allocated */ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, - const unsigned int nfrags) + const unsigned int nents) { unsigned int len = OVPN_NONCE_SIZE; @@ -70,8 +67,8 @@ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, /* round up to the next multiple of the scatterlist alignment */ len = ALIGN(len, __alignof__(struct scatterlist)); - /* add enough space for nfrags + 2 scatterlist entries */ - len += array_size(sizeof(struct scatterlist), nfrags + 2); + /* add enough space for the scatterlist entries */ + len += array_size(sizeof(struct scatterlist), nents); return len; } @@ -135,6 +132,53 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, __alignof__(struct scatterlist)); } +static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, + struct sk_buff *skb, + unsigned int nents, u8 **iv) +{ + struct aead_request *req; + void *tmp; + + /* allocate IV, request and scatterlist entries in one block */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + if (unlikely(!tmp)) + return ERR_PTR(-ENOMEM); + + ovpn_skb_cb(skb)->crypto_tmp = tmp; + *iv = ovpn_aead_crypto_tmp_iv(aead, tmp); + req = ovpn_aead_crypto_tmp_req(aead, *iv); + + return req; +} + +static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + u8 *iv, u8 *data) +{ + u32 pktid, op; + int ret; + + /* obtain packet ID, which is used both as a first + * 4 bytes of nonce and last 4 bytes of associated data. + */ + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + if (unlikely(ret < 0)) + return ret; + + /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes + * nonce + */ + ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); + + /* add the packet opcode and wire nonce as associated data */ + op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); + BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); + *(__force __be32 *)data = htonl(op); + memcpy(data + OVPN_OPCODE_SIZE, iv, OVPN_NONCE_WIRE_SIZE); + + return 0; +} + int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { @@ -143,8 +187,6 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *trailer; struct scatterlist *sg; int nfrags, ret; - u32 pktid, op; - void *tmp; u8 *iv; ovpn_skb_cb(skb)->peer = peer; @@ -175,16 +217,9 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), - GFP_ATOMIC); - if (unlikely(!tmp)) - return -ENOMEM; - - ovpn_skb_cb(skb)->crypto_tmp = tmp; - - iv = ovpn_aead_crypto_tmp_iv(ks->encrypt, tmp); - req = ovpn_aead_crypto_tmp_req(ks->encrypt, iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + if (IS_ERR(req)) + return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); /* sg table: @@ -206,28 +241,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, __skb_push(skb, tag_size); sg_set_buf(sg + ret + 1, skb->data, tag_size); - /* obtain packet ID, which is used both as a first - * 4 bytes of nonce and last 4 bytes of associated data. - */ - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + /* make space for the additional data and push it to the front */ + __skb_push(skb, OVPN_AAD_SIZE); + ret = ovpn_aead_encrypt_header(peer, ks, iv, skb->data); if (unlikely(ret < 0)) return ret; - /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes - * nonce - */ - ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); - - /* make space for packet id and push it to the front */ - __skb_push(skb, OVPN_NONCE_WIRE_SIZE); - memcpy(skb->data, iv, OVPN_NONCE_WIRE_SIZE); - - /* add packet op as head of additional data */ - op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); - __skb_push(skb, OVPN_OPCODE_SIZE); - BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); - *((__force __be32 *)skb->data) = htonl(op); - /* AEAD Additional data */ sg_set_buf(sg, skb->data, OVPN_AAD_SIZE); @@ -281,7 +300,7 @@ int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return -ENOSPC; /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags), + tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags + 2), GFP_ATOMIC); if (unlikely(!tmp)) return -ENOMEM; diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index db9e10f9077c..1a94f0fda1d1 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -11,8 +11,7 @@ #define _NET_OVPN_OVPN_H_ /* DATA_V2 header size with AEAD encryption */ -#define OVPN_HEAD_ROOM (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE + \ - 16 /* AEAD TAG length */ + \ +#define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ max(sizeof(struct ipv6hdr), sizeof(struct iphdr))) diff --git a/drivers/net/ovpn/proto.h b/drivers/net/ovpn/proto.h index b7d285b4d9c1..f3b305cbefe5 100644 --- a/drivers/net/ovpn/proto.h +++ b/drivers/net/ovpn/proto.h @@ -39,6 +39,10 @@ #define OVPN_NONCE_WIRE_SIZE (OVPN_NONCE_SIZE - OVPN_NONCE_TAIL_SIZE) #define OVPN_OPCODE_SIZE 4 /* DATA_V2 opcode size */ +#define OVPN_AUTH_TAG_SIZE 16 +#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + \ + OVPN_NONCE_WIRE_SIZE) +#define OVPN_DATA_V2_OVERHEAD (OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE) #define OVPN_OPCODE_KEYID_MASK 0x07000000 #define OVPN_OPCODE_PKTTYPE_MASK 0xF8000000 #define OVPN_OPCODE_PEERID_MASK 0x00FFFFFF From patchwork Wed Sep 16 08:35:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5362 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5930151mag; Wed, 16 Sep 2026 01:35:55 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwrsfAcuzvws6fxIv3ymLIw+iwqNCJedT/KjC5YB/9737xX5v8XdRri5Y/rguw05E9niBprK5NATr4=@openvpn.net X-Received: by 2002:a05:6820:4de7:b0:6b6:f16d:d668 with SMTP id 006d021491bc7-6c7d15ce81fmr3611999eaf.4.1789547754757; Wed, 16 Sep 2026 01:35:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547754; cv=none; d=google.com; s=arc-20260327; b=h5AHE8xdMPzhnG8++5ER14uOXC61GNSuDN3htEMqVzJ6RMIpi/buZBoDyGPbVIKYo2 phb1hjVtkTIG8ch0IbNxDExYogDrHAHX+lEAKkih4seZRuJvCfbiyuFfldCR9iKyYBwK H9yQLZcNucZLJ4WZ4QOEUX8tfEmLMlqr3gvEkg+TiHj5bB+7uTilTNxrPI0waeLZHTSR m67mOvC7rby/AOGC7Uu5ur7TirUHATcIh2klVHvlLaL5RBLQ/bUBE2xGO3y1a6PpSttc hItFBqEniQnScHYyMC9+iuBetQI/Vk2mzJJPWyT98EDcyz5S0eolj01tHglcIc8B0ZKV 7ziA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=lp9SNiZ7cbEGSCWRav95+Lx73PZUQbomg5bZlLkzDAY=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=Tgzeu6ulTVWifJYUCzampQq2PleO4z0Cc+AdTjS42c68mLMO6CIrlqLU1omfm3nDR3 z4WwYuMZEUwTOQOkv3q7hyRI+Krr5A05ZCy2LyLhLY3qL5qP2bnZKXwrMJXe7jyv/Im9 jQ07PgX7I4LFG3BFDyWrWHxPjGXuUqSW9wEJLs8f8xiAMvZZQg30IBWJSMz/AZtmGBad Utg+k6oVnwEAv4oURBL2MURHZZL+G/8TzoSqeK/soRC3h/Gku3bKQG4inL4Bn0uU0mLd EY+Ovfiah/zwQsTlSUnAw4K7BG6EH54z53i6K7EzKWxCRKC3QgEf8bewdwY4Hw8rtsQ5 zWiQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=Kun1Tz9s; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=JBTXLZxa; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=H1fyIVO1; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=DRZcM+BV; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 006d021491bc7-6c7a583537bsi2636560eaf.16.2026.09.16.01.35.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:35:54 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=Kun1Tz9s; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=JBTXLZxa; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=H1fyIVO1; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=DRZcM+BV; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=lp9SNiZ7cbEGSCWRav95+Lx73PZUQbomg5bZlLkzDAY=; b=Kun1Tz9sI7RuI6+FjGj2Eq3CIX +Jnb40vzx2MCVSO9Zuujw1nKvOAhWz93ff3PJNr9tFfgneHUCEItV17M2uxTHevMMlrKsNHWmqxJJ FCouZP8XkrK3rjFiynDxIG1eSJNRYGXRb143ZFfN7++362T94pJ77ZqakdQLmysXFiHc=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7I-0003l1-1q; Wed, 16 Sep 2026 08:35:44 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7G-0003kq-Hf for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=oI4YR3T/rL/kixMsHp+0AyIXl4KQs2fmCymuNW5zeFo=; b=JBTXLZxaThyw7wre1v3P1yXnkI V8zx7gy380PS//3GS9wVTXJ6nqKIO87uO3NeBvrZ/rwpdOfgi+LgQikafz6UuaWFLDTI5C3na7nPQ nUPPlR/9UzDA7qwBzdG1Okp7m728/naBNsVMhhMqGUCS82w2k/2JtYDiYKweiYjBhsJo=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=oI4YR3T/rL/kixMsHp+0AyIXl4KQs2fmCymuNW5zeFo=; b=H1fyIVO1RaKb+naOxyobjfEYRl vr6Dx6uWI/rfDcmHyZ+FzwM2ssVW2CmQG0vO59r64o0sEvxi9+NewEv2J0Yna9MowZpogf3TNCED3 tcOwqUtn1j9Q7tRmxcVdOYHTuAm65bkOmp0eXgwv4nTaYMjsttIF3t54BAxkGmgvbnws=; Received: from mout-b-206.mailbox.org ([195.10.208.51]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7D-00053F-RD for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:43 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [10.196.197.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-206.mailbox.org (Postfix) with ESMTPS id 4hlBzR46FvzPV for ; Wed, 16 Sep 2026 10:35:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547731; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oI4YR3T/rL/kixMsHp+0AyIXl4KQs2fmCymuNW5zeFo=; b=DRZcM+BV+G+kIXxIXNpViyvYEeChkfpBz3l0o+fVfnS96BxN+1+n6w3UbZKUHiq1X0/lXn dIk2/h6thkZ0cJDCup/feUd2k8VxwT5m24AvXqfz6sSvt+KEvNudTWdqbVkydgqWERn6k9 Uk4M0VJPsgqp2ntKpCTfwJeDW5e2t/7WQQptKlgsAqOO9sLPjSVKJDM2AjlFOAoGSqT5bW WfOr2qZmOf2jlbaiFjsSO29AnETEvWpk6gxg7fScEz3FLqqpcTg9ZdDOpmc1g6U66IRuX3 vWwYkAy2jkQDMNL3sTr9OdScCRZleGgY+JvMWjUy9f5sjrBF9chKjwhwyTAgqA== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:13 +0200 Message-ID: <9359f737627fc84be08ee7d301415c023ebadeb2.1789546917.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb into linear packets while completing their checksums, then emit eligible fixed-size inputs as UDP GSO skb(s). Allocate one final page-backed aggregate per batch before submitting encryption and have each AEAD request write out of place directly into its record slot. Attempting in-place encryption would be com [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6l7D-00053F-RD Subject: [Openvpn-devel] [RFC ovpn net-next v3 4/9] ovpn: convert GSO input into UDP GSO output X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476826261903654 X-GMAIL-MSGID: 1876476826261903654 Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb into linear packets while completing their checksums, then emit eligible fixed-size inputs as UDP GSO skb(s). Allocate one final page-backed aggregate per batch before submitting encryption and have each AEAD request write out of place directly into its record slot. Attempting in-place encryption would be complex (the OpenVPN wire layout adds a header and authentication tag to every segment) and not necessarily more performant: encrypting into individual skbs would still require assembling or copying those records into the UDP GSO skb. The destination allocation and lifetime are instead amortized over the whole batch. Keep the existing in-place path for ordinary packets, where allocating and retiring a separate output skb for every record would provide no aggregate construction benefit. Also retain that path for frag-list GSO input: it already stores complete segments as child skbs, and measurements show that copying those children into another aggregate is counterproductive. Transmit the aggregate as SKB_GSO_UDP_L4 only after every record succeeds, and discard it if any request fails. Split aggregates at the legacy GSO size limit and fall back to individual records when batching is unavailable or the segment geometry is unsuitable. Preserve the input priority, flow hash and sender CPU on the replacement aggregate. If the input has real write ownership, charge the aggregate to the same socket as well. This retains socket lifetime and write-memory accounting and lets lower-device queue selection use the socket's cached TX queue instead of choosing a new queue after crypto completion. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following throughput: Forward Reverse Before this change 11.522 Gbit/s 9.902 Gbit/s Software UDP segmentation 12.879 Gbit/s 12.516 Gbit/s Hardware UDP segmentation 18.308 Gbit/s 19.233 Gbit/s The equal-weight mean of the two directional results increased from 10.712 to 18.770 Gbit/s with hardware UDP segmentation, a 75.2% improvement. With segmentation performed in software, it increased to 12.697 Gbit/s, an 18.5% improvement. Signed-off-by: Ralf Lici --- No changes since v2 https://lore.kernel.org/openvpn-devel/9359f737627fc84be08ee7d301415c023ebadeb2.1789540779.git.ralf@mandelbit.com/ No changes since v1 https://lore.kernel.org/openvpn-devel/9359f737627fc84be08ee7d301415c023ebadeb2.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 85 ++++++++++- drivers/net/ovpn/crypto_aead.h | 4 + drivers/net/ovpn/io.c | 258 ++++++++++++++++++++++++++++++--- drivers/net/ovpn/skb.h | 27 +++- drivers/net/ovpn/stats.h | 16 +- drivers/net/ovpn/tcp.c | 4 +- drivers/net/ovpn/udp.c | 27 +++- 7 files changed, 382 insertions(+), 39 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 30299581422d..8eb76268dc3a 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -134,13 +134,17 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, struct sk_buff *skb, - unsigned int nents, u8 **iv) + unsigned int nents, + unsigned int extra, u8 **iv) { struct aead_request *req; void *tmp; - /* allocate IV, request and scatterlist entries in one block */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + /* allocate IV, request, scatterlist entries and caller scratch space + * in one block + */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents) + extra, + GFP_ATOMIC); if (unlikely(!tmp)) return ERR_PTR(-ENOMEM); @@ -217,7 +221,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, 0, &iv); if (IS_ERR(req)) return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); @@ -261,6 +265,79 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return crypto_aead_encrypt(req); } +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, + struct sk_buff *gso_skb, unsigned int offset) +{ + const unsigned int dst_nents = skb_shinfo(gso_skb)->nr_frags + 4; + const unsigned int src_nents = 2; + unsigned int nents, payload_off; + struct scatterlist *src, *dst; + struct aead_request *req; + int dst_idx, mapped, ret; + u8 *aad, *iv; + + /* each input records the shared peer and key for the common completion + * path but their references remain owned by the output aggregate + */ + ovpn_skb_cb(skb)->peer = peer; + ovpn_skb_cb(skb)->ks = ks; + + if (WARN_ON_ONCE(skb_is_nonlinear(skb))) + return -EINVAL; + + nents = src_nents + dst_nents; + req = ovpn_aead_request_alloc(ks->encrypt, skb, nents, OVPN_AAD_SIZE, + &iv); + if (IS_ERR(req)) + return PTR_ERR(req); + src = ovpn_aead_crypto_req_sg(ks->encrypt, req); + dst = src + src_nents; + aad = (u8 *)(dst + dst_nents); + + ret = ovpn_aead_encrypt_header(peer, ks, iv, aad); + if (unlikely(ret < 0)) + return ret; + + ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); + if (unlikely(ret < 0)) + return ret; + + /* encrypt out of place from the original segmented skb directly into + * its final range in the UDP GSO skb + */ + sg_init_table(src, src_nents); + sg_set_buf(src, aad, OVPN_AAD_SIZE); + sg_set_buf(src + 1, skb->data, skb->len); + + sg_init_table(dst, dst_nents); + dst_idx = skb_to_sgvec_nomark(gso_skb, dst, offset, OVPN_AAD_SIZE); + if (unlikely(dst_idx < 0)) + return dst_idx; + + payload_off = offset + OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE; + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, payload_off, + skb->len); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, + offset + OVPN_AAD_SIZE, + OVPN_AUTH_TAG_SIZE); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + sg_mark_end(&dst[dst_idx - 1]); + + aead_request_set_tfm(req, ks->encrypt); + aead_request_set_callback(req, 0, ovpn_encrypt_post, skb); + aead_request_set_crypt(req, src, dst, skb->len, iv); + aead_request_set_ad(req, OVPN_AAD_SIZE); + + return crypto_aead_encrypt(req); +} + int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { diff --git a/drivers/net/ovpn/crypto_aead.h b/drivers/net/ovpn/crypto_aead.h index fae3b585a43b..8b444744944e 100644 --- a/drivers/net/ovpn/crypto_aead.h +++ b/drivers/net/ovpn/crypto_aead.h @@ -17,6 +17,10 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + struct sk_buff *skb, struct sk_buff *gso_skb, + unsigned int offset); int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 112067ded401..3ad4cadeeb02 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include "ovpnpriv.h" #include "peer.h" @@ -32,6 +34,12 @@ const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE] = { 0x07, 0xed, 0x2d, 0x0a, 0x98, 0x1f, 0xc7, 0x48 }; +/* Leave room for the largest outer network header. The strict inequality in + * is_skb_forwardable also requires staying one byte below gso_max_size. + */ +#define OVPN_UDP_GSO_MAX_PAYLOAD (GSO_LEGACY_MAX_SIZE - \ + sizeof(struct ipv6hdr) - \ + sizeof(struct udphdr) - 1) /** * ovpn_is_keepalive - check if skb contains a keepalive message * @skb: packet to check @@ -237,11 +245,13 @@ void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb) void ovpn_encrypt_post(void *data, int ret) { + unsigned int orig_len, packets = 1; struct ovpn_crypto_key_slot *ks; struct sk_buff *skb = data; struct ovpn_socket *sock; + struct ovpn_cb *batch_cb; struct ovpn_peer *peer; - unsigned int orig_len; + struct sk_buff *batch; /* encryption is happening asynchronously. This function will be * called later by the crypto callback with a proper return value @@ -249,15 +259,19 @@ void ovpn_encrypt_post(void *data, int ret) if (unlikely(ret == -EINPROGRESS)) return; - ks = ovpn_skb_cb(skb)->ks; + /* ordinary encryption leaves batch zeroed; a GSO input uses it to find + * the aggregate whose lifetime is shared by all segment requests + */ + batch = ovpn_skb_cb(skb)->batch; peer = ovpn_skb_cb(skb)->peer; + ks = ovpn_skb_cb(skb)->ks; /* crypto is done, cleanup skb CB and its members */ kfree(ovpn_skb_cb(skb)->crypto_tmp); if (unlikely(ret == -ERANGE)) { /* we ran out of IVs and we must kill the key as it can't be - * use anymore + * used anymore */ netdev_warn(peer->ovpn->dev, "killing key %u for peer %u\n", ks->key_id, @@ -265,8 +279,30 @@ void ovpn_encrypt_post(void *data, int ret) if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) /* let userspace know so that a new key must be negotiated */ ovpn_nl_key_swap_notify(peer, ks->key_id); + } - goto err; + if (batch) { + batch_cb = ovpn_skb_cb(batch); + /* every segment publishes its result before releasing its + * pending count and only the final completion continues with + * the aggregate + */ + if (unlikely(ret < 0)) + atomic_set(&batch_cb->batch_state.failed, 1); + + kfree_skb(skb); + if (!atomic_dec_and_test(&batch_cb->batch_state.pending)) + return; + + skb = batch; + packets = skb_shinfo(batch)->gso_segs; + if (unlikely(atomic_read(&batch_cb->batch_state.failed))) + goto err; + + /* reaching the final callback with no sticky failure means + * every segment completed successfully + */ + ret = 0; } if (unlikely(ret < 0)) @@ -292,7 +328,7 @@ void ovpn_encrypt_post(void *data, int ret) goto err_unlock; } - ovpn_peer_stats_increment_tx(&peer->link_stats, orig_len); + ovpn_peer_stats_add_tx(&peer->link_stats, orig_len, packets); /* keep track of last sent packet for keepalive */ WRITE_ONCE(peer->last_sent, ktime_get_boottime_seconds()); /* skb passed down the stack - don't free it */ @@ -301,7 +337,7 @@ void ovpn_encrypt_post(void *data, int ret) rcu_read_unlock(); err: if (unlikely(skb)) - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, packets); kfree_skb(skb); if (likely(ks)) ovpn_crypto_key_slot_put(ks); @@ -309,29 +345,124 @@ void ovpn_encrypt_post(void *data, int ret) ovpn_peer_put(peer); } -static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +/* Hold the peer and its primary key for one encryption submission. + * The returned key and the peer each carry one reference which completion must + * release. + */ +static struct ovpn_crypto_key_slot * +ovpn_encrypt_refs_get(struct ovpn_peer *peer) { struct ovpn_crypto_key_slot *ks; /* get primary key to be used for encrypting data */ ks = ovpn_crypto_key_slot_primary(&peer->crypto); if (unlikely(!ks)) - return false; + return NULL; - /* take a reference to the peer because the crypto code may run async. - * ovpn_encrypt_post() will release it upon completion + /* the caller already owns a peer reference, so failure indicates a + * broken reference lifetime elsewhere */ if (unlikely(!ovpn_peer_hold(peer))) { DEBUG_NET_WARN_ON_ONCE(1); ovpn_crypto_key_slot_put(ks); - return false; + return NULL; } + return ks; +} + +static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct ovpn_crypto_key_slot *ks; + + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_encrypt_post(skb, ovpn_aead_encrypt(peer, ks, skb)); return true; } +static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, + struct ovpn_peer *peer, + struct sk_buff *batch, + unsigned int segments) +{ + unsigned int offset = 0, i, len; + struct ovpn_crypto_key_slot *ks; + struct sk_buff *skb; + + /* acquire all shared state before removing the first input skb so that + * failure can leave the queue intact for the ordinary transmit path + */ + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + + /* the aggregate owns these references until every sync or async crypto + * completion has finished + */ + memset(ovpn_skb_cb(batch), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(batch)->peer = peer; + ovpn_skb_cb(batch)->ks = ks; + atomic_set(&ovpn_skb_cb(batch)->batch_state.pending, segments); + atomic_set(&ovpn_skb_cb(batch)->batch_state.failed, 0); + + for (i = 0; i < segments; i++) { + skb = __skb_dequeue(skbs); + len = skb->len + OVPN_DATA_V2_OVERHEAD; + + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(skb)->batch = batch; + ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, + batch, offset)); + offset += len; + } + + return true; +} + +static struct sk_buff *ovpn_udp_gso_alloc(const struct sk_buff *first_segment, + unsigned int batch_len, + unsigned int segments) +{ + struct sk_buff *gso_skb; + int ret; + + gso_skb = alloc_skb_with_frags(OVPN_HEAD_ROOM, batch_len, + SKB_FRAG_PAGE_ORDER, &ret, GFP_ATOMIC); + if (unlikely(!gso_skb)) + return NULL; + + skb_reserve(gso_skb, OVPN_HEAD_ROOM); + gso_skb->len = batch_len; + gso_skb->data_len = batch_len; + gso_skb->priority = first_segment->priority; + + /* Segments retain the originating socket so we keep its send-buffer + * accounting active until the UDP GSO is transmitted. This also + * preserves its cached TX queue. + */ + if (first_segment->sk && is_skb_wmem(first_segment)) + skb_set_owner_w(gso_skb, first_segment->sk); + skb_copy_hash(gso_skb, first_segment); +#ifdef CONFIG_XPS + /* keep the aggregate on the TX queue selected for the original flow + * otherwise async crypto completion on another CPU could move the flow + * to a different queue and cause delay or reordering + */ + gso_skb->sender_cpu = first_segment->sender_cpu; +#endif + + skb_shinfo(gso_skb)->gso_type = SKB_GSO_UDP_L4; + skb_shinfo(gso_skb)->gso_size = first_segment->len + + OVPN_DATA_V2_OVERHEAD; + skb_shinfo(gso_skb)->gso_segs = segments; + + return gso_skb; +} + /* send skb to connected peer, if any */ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, struct ovpn_peer *peer) @@ -343,7 +474,7 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, */ skb_list_walk_safe(skb, curr, next) { if (unlikely(!ovpn_encrypt_one(peer, curr))) { - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); } } @@ -351,19 +482,96 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, ovpn_peer_put(peer); } +/* encrypt fixed-size input segments into one or more UDP GSO aggregates */ +static void ovpn_send_gso(struct sk_buff_head *skbs, struct ovpn_peer *peer) +{ + unsigned int max_segs = 1, seg_len, batch_len, segs; + struct sk_buff *batch; + + seg_len = skb_peek(skbs)->len + OVPN_DATA_V2_OVERHEAD; + max_segs = min_t(unsigned int, UDP_MAX_SEGMENTS, + OVPN_UDP_GSO_MAX_PAYLOAD / seg_len); + + /* if even two encrypted skbs cannot fit, leave the whole queue for the + * ordinary transmit path + */ + if (max_segs < 2) + return; + + /* An input GSO skb might be prduce more than max_segs segments so we + * consume as many as we can for each iteration. A final single skb, or + * the whole remainder after an allocation failure, stays queued and + * fallback to the ordinary transmit path. + */ + while (skb_queue_len(skbs) > 1) { + segs = min_t(unsigned int, skb_queue_len(skbs), max_segs); + + /* all but the final input skb have the same length, so start + * with the full-size calculation and adjust only the final + * group below + */ + batch_len = segs * (skb_peek(skbs)->len + + OVPN_DATA_V2_OVERHEAD); + if (segs == skb_queue_len(skbs)) + batch_len -= skb_peek(skbs)->len - + skb_peek_tail(skbs)->len; + + batch = ovpn_udp_gso_alloc(skb_peek(skbs), batch_len, segs); + if (unlikely(!batch)) + return; + + if (unlikely(!ovpn_encrypt_gso_queue(skbs, peer, + batch, segs))) { + kfree_skb(batch); + return; + } + } +} + +static bool ovpn_peer_supports_udp_gso(struct ovpn_peer *peer) +{ + struct ovpn_socket *sock; + bool udp_gso; + + rcu_read_lock(); + sock = rcu_dereference(peer->sock); + /* UDP GSO requires checksums. These socket settings can change after + * we decide to batch, but an already-built batch remains checksummed. + * Linux's ordinary UDP GSO path makes the same choice. + */ + udp_gso = sock && sock->sk->sk_protocol == IPPROTO_UDP && + !sock->sk->sk_no_check_tx && !udp_get_no_check6_tx(sock->sk); + rcu_read_unlock(); + + return udp_gso; +} + /* Send user data to the network */ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) { struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; + const bool gso_in = skb_is_gso(skb); struct sk_buff_head skb_list; netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; + bool gso_out; __be16 proto; int ret; + /* A frag-list GSO skb already stores complete segments as child skbs. + * Keep those children on the ordinary in-place encryption path instead + * of copying them into a replacement UDP GSO skb. + * + * GSO_BY_FRAGS input must also remain on that path because its variable + * segment sizes cannot be represented by one UDP GSO output size. + */ + gso_out = gso_in && + !(skb_shinfo(skb)->gso_type & SKB_GSO_FRAGLIST) && + skb_shinfo(skb)->gso_size != GSO_BY_FRAGS; + /* reset netfilter state */ nf_reset_ct(skb); @@ -392,13 +600,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) /* dst was needed for peer selection - it can now be dropped */ skb_dst_drop(skb); - if (skb_is_gso(skb)) { - /* force software segmentation, but keep ovpn's non-GSO feature - * bits so the generated segments can preserve non-linear skb - * data where possible + if (gso_in) { + /* force software segmentation into linear skbs and calculate + * each checksum while copying the segment */ features = netif_skb_features(skb); - segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + features &= ~(NETIF_F_GSO_MASK | NETIF_F_SG | + NETIF_F_CSUM_MASK); + segments = skb_gso_segment(skb, features); if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", @@ -420,7 +629,8 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) if (unlikely(!curr)) { net_err_ratelimited("%s: skb_share_check failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); + gso_out = false; continue; } @@ -429,8 +639,9 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_checksum_help(curr) < 0)) { net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); + gso_out = false; continue; } @@ -446,9 +657,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) ovpn_peer_put(peer); return NETDEV_TX_OK; } - skb_list.prev->next = NULL; ovpn_peer_stats_increment_tx(&peer->vpn_stats, tx_bytes); + + if (gso_out && skb_queue_len(&skb_list) > 1 && + ovpn_peer_supports_udp_gso(peer)) + ovpn_send_gso(&skb_list, peer); + + skb_list.prev->next = NULL; ovpn_send(ovpn, skb_list.next, peer); return NETDEV_TX_OK; @@ -456,7 +672,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) drop: ovpn_peer_put(peer); drop_no_peer: - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); skb_tx_error(skb); kfree_skb_list(skb); return NETDEV_TX_OK; diff --git a/drivers/net/ovpn/skb.h b/drivers/net/ovpn/skb.h index 4fb7ea025426..cca29479c038 100644 --- a/drivers/net/ovpn/skb.h +++ b/drivers/net/ovpn/skb.h @@ -10,6 +10,7 @@ #ifndef _NET_OVPN_SKB_H_ #define _NET_OVPN_SKB_H_ +#include #include #include #include @@ -20,19 +21,35 @@ /** * struct ovpn_cb - ovpn skb control block - * @peer: the peer this skb was received from/sent to - * @ks: the crypto key slot used to encrypt/decrypt this skb * @crypto_tmp: pointer to temporary memory used for crypto operations * containing the IV, the scatter gather list and the aead request + * @peer: peer used by this crypto operation or owned by this aggregate + * @ks: crypto key slot used by this operation or owned by this aggregate * @payload_offset: offset in the skb where the payload starts * @nosignal: whether this skb should be sent with the MSG_NOSIGNAL flag (TCP) + * @batch: UDP GSO aggregate receiving this input skb's encrypted payload + * @batch_state: completion state owned by a UDP GSO aggregate */ struct ovpn_cb { + void *crypto_tmp; struct ovpn_peer *peer; struct ovpn_crypto_key_slot *ks; - void *crypto_tmp; - unsigned int payload_offset; - bool nosignal; + + /* Ordinary encryption leaves this union zeroed. Decryption and TCP use + * their ordinary fields, a UDP GSO input stores its output aggregate, + * and that aggregate uses the same space to coordinate its completions. + */ + union { + struct { + unsigned int payload_offset; + bool nosignal; + }; + struct sk_buff *batch; + struct { + atomic_t pending; + atomic_t failed; + } batch_state; + }; }; static inline struct ovpn_cb *ovpn_skb_cb(struct sk_buff *skb) diff --git a/drivers/net/ovpn/stats.h b/drivers/net/ovpn/stats.h index 3a45b97c0056..b3fe006c01f6 100644 --- a/drivers/net/ovpn/stats.h +++ b/drivers/net/ovpn/stats.h @@ -40,16 +40,26 @@ static inline void ovpn_peer_stats_increment_rx(struct ovpn_peer_stats *stats, ovpn_peer_stats_increment(&stats->rx, n); } +static inline void ovpn_peer_stats_add_tx(struct ovpn_peer_stats *stats, + const unsigned int bytes, + unsigned int packets) +{ + atomic64_add(bytes, &stats->tx.bytes); + atomic64_add(packets, &stats->tx.packets); +} + static inline void ovpn_peer_stats_increment_tx(struct ovpn_peer_stats *stats, const unsigned int n) { - ovpn_peer_stats_increment(&stats->tx, n); + ovpn_peer_stats_add_tx(stats, n, 1); } -static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev) +static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev, + unsigned int packets) { local_bh_disable(); - dev_dstats_tx_dropped(dev); + while (packets--) + dev_dstats_tx_dropped(dev); local_bh_enable(); } diff --git a/drivers/net/ovpn/tcp.c b/drivers/net/ovpn/tcp.c index 8fe8a8e750a4..5cba35e4a8ee 100644 --- a/drivers/net/ovpn/tcp.c +++ b/drivers/net/ovpn/tcp.c @@ -332,7 +332,7 @@ static void ovpn_tcp_send_sock_skb(struct ovpn_peer *peer, struct sock *sk, ovpn_tcp_send_sock(peer, sk); if (peer->tcp.out_msg.skb) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); return; } @@ -354,7 +354,7 @@ void ovpn_tcp_send_skb(struct ovpn_peer *peer, struct sock *sk, if (sock_owned_by_user(sk)) { if (skb_queue_len(&peer->tcp.out_queue) >= READ_ONCE(net_hotdata.max_backlog)) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); goto unlock; } diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 7f69e8890b5b..4802d982de08 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -121,6 +121,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); + skb_mark_not_on_list(skb); ovpn_recv(peer, skb); return 0; @@ -196,9 +197,13 @@ static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind, dst_cache_set_ip4(cache, &rt->dst, fl.saddr); transmit: + /* an already-built UDP GSO needs a checksum seed even if the socket's + * no-check option changed while encryption was in flight + */ udp_tunnel_xmit_skb(rt, sk, skb, fl.saddr, fl.daddr, 0, ip4_dst_hoplimit(&rt->dst), 0, fl.fl4_sport, - fl.fl4_dport, false, sk->sk_no_check_tx, 0); + fl.fl4_dport, false, + !skb_is_gso(skb) && sk->sk_no_check_tx, 0); ret = 0; err: local_bh_enable(); @@ -271,9 +276,13 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind, * udp_tunnel_xmit_skb() */ skb->ignore_df = 1; + /* keep checksum offload enabled for an in-flight UDP GSO batch even if + * the socket's no-check option has changed since batch creation + */ udp_tunnel6_xmit_skb(dst, sk, skb, skb->dev, &fl.saddr, &fl.daddr, 0, ip6_dst_hoplimit(dst), 0, fl.fl6_sport, - fl.fl6_dport, udp_get_no_check6_tx(sk), 0); + fl.fl6_dport, + !skb_is_gso(skb) && udp_get_no_check6_tx(sk), 0); ret = 0; err: local_bh_enable(); @@ -344,8 +353,18 @@ void ovpn_udp_send_skb(struct ovpn_peer *peer, struct sock *sk, skb->dev = peer->ovpn->dev; skb->mark = READ_ONCE(sk->sk_mark); - /* no checksum performed at this layer */ - skb->ip_summed = CHECKSUM_NONE; + if (skb_is_gso(skb)) { + /* udp_tunnel_xmit_skb installs the outer UDP header after this + * function returns: point CHECKSUM_PARTIAL at that future + * header so both hw and sw UDP GSO can complete the checksum. + */ + skb->ip_summed = CHECKSUM_PARTIAL; + skb->csum_start = skb_headroom(skb) - sizeof(struct udphdr); + skb->csum_offset = offsetof(struct udphdr, check); + } else { + /* no checksum performed at this layer */ + skb->ip_summed = CHECKSUM_NONE; + } /* crypto layer -> transport (UDP) */ ret = ovpn_udp_output(peer, &peer->dst_cache, sk, skb); From patchwork Wed Sep 16 08:35:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5358 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5930059mag; Wed, 16 Sep 2026 01:35:49 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBzdEZFOskxIHukrBgYuQL1QilzTUrvVM0ZjjAmtfe0ij8XEeB9B7VB+VWeXw55gxw6BhTU6EM8gboA=@openvpn.net X-Received: by 2002:a05:6820:3098:b0:6ba:3635:7ffd with SMTP id 006d021491bc7-6c7d46e698fmr1745272eaf.64.1789547748605; Wed, 16 Sep 2026 01:35:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547748; cv=none; d=google.com; s=arc-20260327; b=oHp6SsWg/FVEWGABugjWk3KwEd+WO57CLwOm3/ywTlRGl/1R1u0tZgCq99Vvg8EjkJ N8JPXNhnwtPS/+7w5G++TlDD/K7qBuDRSADDEX3Ooz6zpQtkMTVkwYiwBejByMOLcKDg 7SmrvHZaKqC3yuE+ST+aZcajfZsyKj+FHW6P8tl1iJNN5IZMR6GtVeqIuAKWiKMECSFH trwvsUgTuDNVu6a8CylcPE6vM4IF7aoVZKj9PwR8kW/vb2hPfhTVK8pDgx+AyUlIcwLj b35TXBFKXuB5EfEgjroRLlEt1B7trhe9qMz5FmsjRB3MYqtUZa63ib02DNylr5EYv/j4 beVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=qTV71Sm4XsIGOGJbtGRIBCl4pGOf5iegvOsXhvTtVNU=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=qLnumFcMUyoLSU6eiee9zBO4KQps58Nsa27IauCvFWdaDAjRJ/Im12Igj9Kbe23QN+ o++01hhyQzIripnHKKAMRpIkNgsWVrkac/0IoKap2qkp7YsXTkSI85Elnhoq4ZjGVnf1 W9wB0rC6PmC8ynD+wRuqw8Zv+u+WERA6OphOqAJtDyLN/YyMPpYBn6IFVZCArvsYwX4O 4wgIobByE+Ik0184V7vnad+FQxy4RJqyxpMnW5Z7lPxxJny5b7SZi6/tZneCHC7J3HaU 7hYH6ZmDoyclJmjud4Ph9wPUM7Jm1hUac6WTCwZ73xDIdn2D/eR01nhvqPu1pueKDMrW jBng==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=lBCSqgWZ; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=ZKWyJF4a; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=VdP8kVkX; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=D+Iy1hn9; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-4842790ce65si2294629fac.25.2026.09.16.01.35.47 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:35:48 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=lBCSqgWZ; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=ZKWyJF4a; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=VdP8kVkX; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=D+Iy1hn9; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=qTV71Sm4XsIGOGJbtGRIBCl4pGOf5iegvOsXhvTtVNU=; b=lBCSqgWZvT8sih7eVJbEt5xm5r gV36RQxigjjxqigw/Mq5IMilpOqHccJeYvh/QFT8YE0N/61MI4f4Hd/NWZF/ZTpekz5NXinOzTs9i +6O/5e+uttX5ENxvjDt+s6bAHViSQ5XE4XPPYJbRdyd/8gLTOHm7lWk/Q1qfExyaOpUg=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7H-0004Xp-G4; Wed, 16 Sep 2026 08:35:44 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7G-0004Xe-AH for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Xcbn1IBV1wPJscBwKDfKRS+bBZAuL6cXU3ZhvYBlixY=; b=ZKWyJF4aPQSQbZXfAiiDMqn6G5 Og9Hi8MwMqwVWdC3QkSNnJMXFaOcMrvoHFQLMj2gd/3J1oGWAk6wZVtczAi57AnP65UGyv/QV6d6E +TbrZSGirpZxoGZ2W9gDQqaAuf45gcdFrpajLzNUBgQV3Db5qiMQDXFHsY4+PWxUOPi8=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=Xcbn1IBV1wPJscBwKDfKRS+bBZAuL6cXU3ZhvYBlixY=; b=VdP8kVkX45Zaef8Ot5dRjqFxsw vXrcMJVgcQHEsDdMX0cy1U5w2PNIhJvX+wHD0lEqK0X3AlO8uk99Qq7nfkymQZ6XOZpWLT2tmkiDM umdcRVddWSFyhZAc9l5hn7VsX//VdX9drRvxHPyDBWWt9TpqZ10mCTxUwPaAJyMATPjM=; Received: from mout-b-201.mailbox.org ([195.10.208.61]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7E-00053H-Dl for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:43 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:b231:465::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-201.mailbox.org (Postfix) with ESMTPS id 4hlBzS0wHFzLm5C for ; Wed, 16 Sep 2026 10:35:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547732; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xcbn1IBV1wPJscBwKDfKRS+bBZAuL6cXU3ZhvYBlixY=; b=D+Iy1hn9fqTjXxHtLlQZCD2SZeD0XgK3y4QAa/jLeOf/iYgCIo5KkarMavgy+rWeUxkfMf OHJbeLMKkgcr80qusSx1PMtsv3N1V08UcY8jSdUDs03quqoRjPeRPA50y6rhCFxOuk9Q6n aMnVMn0OZVu1X/2TRLU/62hocdPUk5hqPj7cUbx68NVBSjoluXrj8Qp5aUXL3qLqTwHPf7 oQjnG9dS9YV0Z3f6Q64axZQrwMAdoEg72kwhbz346Ly4i6p3nc8DR4PPrS0M1+b2Y501Xh U228rmLpcQlxITfReCNQA4aKI9/Ezzd1+pvr4LEeM0gg5tPKxi/7yas+GlDKvg== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::2 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:14 +0200 Message-ID: <903e2f55a68a1653f15014a91b0d7ac35ccabf11.1789546917.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hlBzS0wHFzLm5C X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path ca [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6l7E-00053H-Dl Subject: [Openvpn-devel] [RFC ovpn net-next v3 5/9] ovpn: coalesce UDP data records with GRO X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476820278284671 X-GMAIL-MSGID: 1876476820278284671 Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path can detach and authenticate records independently. Match the complete opcode/key/peer header and require compatible outer- network and checksum state. Flush on short records, differing segment geometry, existing GSO input, or the 64-record limit. Export skb_gro_receive_list, which is already shared by the core UDP and TCP frag-list GRO paths, so modular ovpn can use the same primitive instead of maintaining a local copy. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following throughput: Forward Reverse Without receive GRO 18.308 Gbit/s 19.233 Gbit/s With frag-list GRO 22.087 Gbit/s 22.962 Gbit/s The preceding UDP GSO transmit path and hardware UDP segmentation were enabled in both cases. The equal-weight mean of the two directional results increased from 18.770 to 22.524 Gbit/s, a 20.0% improvement. Signed-off-by: Ralf Lici --- Changes since v2 https://lore.kernel.org/openvpn-devel/76cb811457ac17b519d219b13fb2a4317a6d5e74.1789540779.git.ralf@mandelbit.com/ - Unclone GRO aggregates to avoid shared skb corruption in ovpn_udp_gro_detach. (Sashiko) Changes since v1 https://lore.kernel.org/openvpn-devel/dd08a7a2fe0dfc88509115b5d7ee17825020c012.1789485693.git.ralf@mandelbit.com/ - Preserve the outer network offset in ovpn_udp_gro_receive_fraglist. (Sashiko) - Detach frag_list only from a UDP-tunnel GSO aggregate. (Sashiko) drivers/net/ovpn/io.c | 7 +- drivers/net/ovpn/udp.c | 197 ++++++++++++++++++++++++++++++++++++++++- net/core/gro.c | 1 + net/ipv4/udp_offload.c | 3 +- 4 files changed, 201 insertions(+), 7 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 3ad4cadeeb02..11f7f16d7b79 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -70,11 +70,10 @@ static void ovpn_netdev_write(struct ovpn_peer *peer, struct sk_buff *skb) unsigned int pkt_len; int ret; - /* - * GSO state from the transport layer is not valid for the tunnel/data - * path. Reset all GSO fields to prevent any further GSO processing - * from entering an inconsistent state. + /* the transport encapsulation and its GSO metadata do not describe the + * decrypted inner packet */ + skb->encapsulation = 0; skb_gso_reset(skb); /* we can't guarantee the packet wasn't corrupted before entering the diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 4802d982de08..ee3b9d1aec25 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -11,8 +11,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -27,6 +29,192 @@ #include "socket.h" #include "udp.h" +/* like UDP and TCP frag-list GRO */ +#define OVPN_UDP_GRO_CNT_MAX 64 + +static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) +{ + const unsigned int offset = skb_gro_offset(skb); + + /* GRO replaces its frag0 pointer after holding an skb, so keep the + * openvpn header linear for later candidate comparisons + */ + if (!pskb_may_pull(skb, offset + OVPN_OPCODE_SIZE)) + return false; + + *header = get_unaligned_be32(skb->data + offset); + return true; +} + +static struct sk_buff *ovpn_udp_gro_receive_fraglist(struct sock *sk, + struct list_head *head, + struct sk_buff *skb) +{ + const unsigned int gso_size = skb_gro_len(skb); + struct sk_buff *p, *pp = NULL; + u32 header, header2; + int ret = 0, nhoff; + bool flush; + + if (!ovpn_udp_gro_header(skb, &header) || + FIELD_GET(OVPN_OPCODE_PKTTYPE_MASK, header) != OVPN_DATA_V2) { + NAPI_GRO_CB(skb)->flush = 1; + return NULL; + } + + /* do not nest an existing GSO packet in the record list */ + if (skb_is_gso(skb)) { + NAPI_GRO_CB(skb)->flush = 1; + return NULL; + } + + list_for_each_entry(p, head, list) { + if (!NAPI_GRO_CB(p)->same_flow) + continue; + + /* match opcode, key ID and peer ID */ + if (!ovpn_udp_gro_header(p, &header2) || header != header2) { + NAPI_GRO_CB(p)->same_flow = 0; + continue; + } + + /* GRO has already matched the outer addresses and UDP ports; + * check the remaining outer IP fields + */ + nhoff = skb_transport_offset(p) - + NAPI_GRO_CB(p)->network_offset; + flush = __gro_receive_network_flush(udp_hdr(skb), udp_hdr(p), p, + nhoff, false); + + /* The first record determines the nominal GSO size. A shorter + * final record may follow it, but a larger record cannot. + * Checksum metadata must also be uniform because the aggregate + * exposes only one checksum state. + */ + if (gso_size > skb_shinfo(p)->gso_size || flush || + skb->ip_summed != p->ip_summed || + skb->csum_level != p->csum_level) { + pp = p; + } else { + /* skb_gro_receive_list pulls the headers already + * processed by GRO before linking this skb to the + * record list so we have to manually preserve the + * outer network header location for later handling + */ + nhoff = NAPI_GRO_CB(skb)->network_offset; + skb_set_network_header(skb, nhoff); + ret = skb_gro_receive_list(p, skb); + } + + /* complete the aggregate if the append failed, or after + * appending a shorter final record, or after reaching the + * record-count limit + */ + if (ret || gso_size != skb_shinfo(p)->gso_size || + NAPI_GRO_CB(p)->count >= OVPN_UDP_GRO_CNT_MAX) + pp = p; + + return pp; + } + + return NULL; +} + +static int ovpn_udp_gro_complete(struct sock *sk, struct sk_buff *skb, + int nhoff) +{ + /* udp_gro_complete has already marked this as a UDP tunnel GSO packet. + * Keep that type so UDP passes the aggregate directly to the encap cb, + * where the original record skbs are detached. + */ + skb_shinfo(skb)->gso_segs = NAPI_GRO_CB(skb)->count; + + /* Each outer UDP checksum was either validated (or accepted in case of + * checksumless UDP) before its record was merged in + * skb_gro_checksum_validate_zero_check. + * The checksum in the aggregate cannot describe the concatenation of + * independent UDP payloads, so we preserve the validation result. + */ + skb->ip_summed = CHECKSUM_UNNECESSARY; + skb->csum_level = 0; + skb->csum_valid = 0; + + return 0; +} + +/* skb_gro_receive_list keeps the first openvpn record in 'skb' and links the + * remaining records through frag_list. Here we segment by detaching that list + * before delivering the records individually, and remove the child skbs from + * the head skb's length and memory accounting so the head describes only the + * first record again. + */ +static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) +{ + unsigned int data_len = 0, truesize = 0; + struct sk_buff *curr, *list; + int ret; + + /* IP reassembly may also use fraglist, but it is not a record batch */ + if (!skb_is_gso(skb) || + !(skb_shinfo(skb)->gso_type & + (SKB_GSO_UDP_TUNNEL | SKB_GSO_UDP_TUNNEL_CSUM))) + return NULL; + + if (unlikely(!skb_shinfo(skb)->frag_list)) + return NULL; + + /* packet taps may have cloned the aggregate before it reached UDP, so + * make shared info private before removing its frag_list + */ + ret = skb_unclone(skb, GFP_ATOMIC); + if (unlikely(ret)) + return ERR_PTR(ret); + + list = skb_shinfo(skb)->frag_list; + + for (curr = list; curr; curr = curr->next) { + data_len += curr->len; + truesize += curr->truesize; + } + + skb_shinfo(skb)->frag_list = NULL; + skb->len -= data_len; + skb->data_len -= data_len; + skb->truesize -= truesize; + + return list; +} + +static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct sk_buff *list, *next; + + list = ovpn_udp_gro_detach(skb); + if (IS_ERR(list)) { + ovpn_dev_dstats_rx_dropped(peer->ovpn->dev); + kfree_skb(skb); + ovpn_peer_put(peer); + return; + } + skb->next = list; + + skb_list_walk_safe(skb, skb, next) + { + skb_mark_not_on_list(skb); + + /* keep the current reference alive for the next record before + * handing this one to crypto + */ + if (next && unlikely(!ovpn_peer_hold(peer))) { + DEBUG_NET_WARN_ON_ONCE(1); + kfree_skb_list(next); + next = NULL; + } + + ovpn_recv(peer, skb); + } +} + /* Retrieve the corresponding ovpn object from a UDP socket * rcu_read_lock must be held on entry */ @@ -121,8 +309,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); - skb_mark_not_on_list(skb); - ovpn_recv(peer, skb); + ovpn_udp_recv(peer, skb); return 0; drop: @@ -408,6 +595,8 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock, .encap_type = UDP_ENCAP_OVPNINUDP, .encap_rcv = ovpn_udp_encap_recv, .encap_destroy = ovpn_udp_encap_destroy, + .gro_receive = ovpn_udp_gro_receive_fraglist, + .gro_complete = ovpn_udp_gro_complete, }; struct ovpn_socket *old_data; int ret; @@ -454,6 +643,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) { struct sock *sk = ovpn_sock->sk; + udp_tunnel_cleanup_gro(sk); + /* Re-enable multicast loopback */ inet_set_bit(MC_LOOP, sk); /* Disable CHECKSUM_UNNECESSARY to CHECKSUM_COMPLETE conversion */ @@ -462,6 +653,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) WRITE_ONCE(udp_sk(sk)->encap_type, 0); WRITE_ONCE(udp_sk(sk)->encap_rcv, NULL); WRITE_ONCE(udp_sk(sk)->encap_destroy, NULL); + WRITE_ONCE(udp_sk(sk)->gro_receive, NULL); + WRITE_ONCE(udp_sk(sk)->gro_complete, NULL); rcu_assign_sk_user_data(sk, NULL); } diff --git a/net/core/gro.c b/net/core/gro.c index 29b4d02bf519..b6acedc919f8 100644 --- a/net/core/gro.c +++ b/net/core/gro.c @@ -262,6 +262,7 @@ int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb) return 0; } +EXPORT_SYMBOL(skb_gro_receive_list); static void gro_complete(struct gro_node *gro, struct sk_buff *skb) { diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index cf07c3c6611a..187f108f3ee8 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -40,7 +40,8 @@ struct udp_tunnel_type_entry { #define UDP_MAX_TUNNEL_TYPES (IS_ENABLED(CONFIG_GENEVE) + \ IS_ENABLED(CONFIG_VXLAN) * 2 + \ IS_ENABLED(CONFIG_NET_FOU) * 2 + \ - IS_ENABLED(CONFIG_XFRM) * 2) + IS_ENABLED(CONFIG_XFRM) * 2 + \ + IS_ENABLED(CONFIG_OVPN)) DEFINE_STATIC_CALL(udp_tunnel_gro_rcv, dummy_gro_rcv); static DEFINE_STATIC_KEY_FALSE(udp_tunnel_static_call); From patchwork Wed Sep 16 08:35:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5361 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5930101mag; Wed, 16 Sep 2026 01:35:51 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBxuNPljEmvKl8qFC7ZGsvFe4oXLrrVziJ/JO3pxxHU2txAbuKReRE+KvpU6methxJJp/I6r+FWSS6k=@openvpn.net X-Received: by 2002:a05:6870:d38c:b0:465:fdf:bd44 with SMTP id 586e51a60fabf-48475e95f2cmr1748507fac.14.1789547751529; Wed, 16 Sep 2026 01:35:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547751; cv=none; d=google.com; s=arc-20260327; b=rEY35zUvXjvRkkxMmwgErdVpaz24iqG22tcW4q+S7Rg7XcKUcIaaFzERq9TPZf3PAF vE9OjdV50Y2oGWh2BBYzqKgBDZncXtccfrfuQpTNUFGhiM6lvDvEtPgxAq+qB+YhHafN DTv8pIuLWyisGK8yfu4RE1Ag12AG/rZcuurxEyiYcfqP5680VJNVboOyHo/i3a4VqZYx KiAYjTETp1BveG4NWWVGQ8bXM8qqSpX4vH7YSOnP3C3V5h9od9/Nyx6G11PkgGjLtJSI RJPFLGWH3cmqZxxRli9qrEWPCPBw6IB2d3hTFxeQVs/By5Wp8gq6c703PUhbdnZOefyj fcCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=dgL5SahhRczTVpp1QryyyKT6lpA50z1s7vl25gAfU7o=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=jQIRnRQCfe78Qnce/FKyGsTBC/8m7lw9SPA1C/q57EmRmbaxGy98PWqNW84qoWBUW5 Kj+PngNsTbQn/5mX/eaxPqf5Ooer6b0bJsCCT7iAsxWGd68JYLGEr1fy5w9fh1ao6/1R 4pTCDkNgEf0UyLa/bFQ0yMS6NsAnMzyv2us4TCqplZTqiqoXymZiuQVKJ4Svr/ZDKGVf DZPmSckgq07y3bsqT1CNyXnLsLxOUO6QvjvoLYRMmO/BdBl9LKfv2O7b6djW1EN/4HNY xkYfk6UkfyukVn06+xNaEGlJGlLX+XetQ17nzzP1q1vFWlpKocC2OZCHpX3/p+pPak4/ 2dfQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=DENpl97P; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=LrPrwbbf; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=lu0UUJIo; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=hnpiCM1j; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-4842a9bed66si2268977fac.300.2026.09.16.01.35.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:35:51 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=DENpl97P; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=LrPrwbbf; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=lu0UUJIo; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=hnpiCM1j; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dgL5SahhRczTVpp1QryyyKT6lpA50z1s7vl25gAfU7o=; b=DENpl97PqIYnujFgPgG3S3JPiY If+A/wNSZKbrF6zlBf3Tut8mTzbLXrU81Yd8lm1Cp9GccdxMD7KNjj6tVnfPyTFFvyiQF2vuEncPv bPGNidCEwOf3xyBMFNyORi+EIBTgIouu1m/byqqExgeT41CRMZxpu3tQzxQ3hjDqVrEU=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7M-0002Jx-2l; Wed, 16 Sep 2026 08:35:48 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7H-0002JS-S2 for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=uRUxsZoOGryxI6iEUVGA3A6clvp1g5IhYIPzQfHaZ9c=; b=LrPrwbbfw9lRNSFRF5wSqB9E/U qOX0X/NIuoypq6+68AY3I/UCxeHAbInudPjRnjye7glucXgDdtfwuFHBtj84G6+eWQJyetXcbYVzQ cRh/HvZ0iE56sfM+RjWhLjrVncsWaAo5X+UdCjXdLwjWsEzzLkEZWhYCV57cmEByKh84=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=uRUxsZoOGryxI6iEUVGA3A6clvp1g5IhYIPzQfHaZ9c=; b=lu0UUJIoOlz+Zi8lP8JrRezPpL NIw8iDV/bfELk49CW4/pP02uDeZzwrtt3Q5UzDbHR3ijnSLCA61hBcLoukBvz7k4LehBHuy62QWoT 0sXXfNq/OhTUcC3lvxSFn6GXY8iHMUMNBoueA5I91ljdZ+AbEFjZehoTySkVXD/lGULg=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7F-00053J-F8 for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:43 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:b231:465::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4hlBzT0c5Dz5wxq for ; Wed, 16 Sep 2026 10:35:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547733; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uRUxsZoOGryxI6iEUVGA3A6clvp1g5IhYIPzQfHaZ9c=; b=hnpiCM1j5aBy7S7pkavxmLzGwm66jzerUmSmsvbe5l+CAFYvAP6dC4tUmh6xFVPHQ7B9q9 0As0uPdWj72/uJncrhzWf/zVwpvY/uPp414EncNlJWxe0nwSNtcSeb+4SHxnFlpPVzFDuX xAHhGf578SxJwhI+VTV8rrV0oXrttZLBmPrgLiqrD9oXpmqP3nKxGWF1uHoc8qkcZpBej3 qGpwX+7vkL9gWo1nyhJyjom2OyXHB97eRz3/059zP+HIbpNC4s64gHddMArMo2fnFeQZko D++9rlGnoV3cI1gwrgNBbFAOjuAm2kCqwUSRpK8Rvi1JQqFD4X+k7orjsxyQYQ== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::2 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:15 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hlBzT0c5Dz5wxq X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (a [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6l7F-00053J-F8 Subject: [Openvpn-devel] [RFC ovpn net-next v3 6/9] ovpn: prefetch encrypted records before GRO batch decryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476823261224854 X-GMAIL-MSGID: 1876476823261224854 Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (and maintain the same distance throughout the batch), overlapping their memory access latency with the current AEAD operation. An ordinary single-record UDP receive has no later record and therefore skips the prefetch path. Only prefetch the linear part of each skb. Walking non-linear fragments here would duplicate the scatterlist walk performed by crypto and could cost more than the cache hint saves. A same-binary comparison using three 30-second samples per direction found distances one and two effectively tied forward, while distance two was 3.3% faster reverse and less variable in both directions. Profiling also measured slightly fewer decrypt cycles at distance two than at one, while wider distances provided no repeatable benefit. On a direct 100 Gbit/s ConnectX-5 link using one TCP stream, AES-128-GCM, a 1408-byte inner MTU and 8192-entry rings, five interleaved 60-second samples per direction increased throughput by 16.9% forward and 18.0% reverse. Signed-off-by: Ralf Lici --- No changes since v2 https://lore.kernel.org/openvpn-devel/4d8357e320fc7c8fd4ab6e42a0bc38b275b1ff5d.1789540779.git.ralf@mandelbit.com/ No changes since v1 https://lore.kernel.org/openvpn-devel/712708baf265c5509aed4f9d476abf514a242b8a.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/io.h | 14 ++++++++++++++ drivers/net/ovpn/udp.c | 21 ++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index 1a94f0fda1d1..49180214fe08 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -10,6 +10,9 @@ #ifndef _NET_OVPN_OVPN_H_ #define _NET_OVPN_OVPN_H_ +#include +#include + /* DATA_V2 header size with AEAD encryption */ #define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ @@ -21,6 +24,17 @@ #define OVPN_KEEPALIVE_SIZE 16 extern const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE]; +static inline void ovpn_skb_prefetchw(const struct sk_buff *skb) +{ + unsigned int offset; + + /* crypto overwrites data in place, so request write ownership of each + * linear cache line before the AEAD implementation reaches it + */ + for (offset = 0; offset < skb_headlen(skb); offset += L1_CACHE_BYTES) + prefetchw(skb->data + offset); +} + netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev); void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb); diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index ee3b9d1aec25..ce2584ecc934 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -32,6 +32,9 @@ /* like UDP and TCP frag-list GRO */ #define OVPN_UDP_GRO_CNT_MAX 64 +/* leave enough work between a cache hint and the record which consumes it */ +#define OVPN_UDP_GRO_PREFETCH_DISTANCE 2 + static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) { const unsigned int offset = skb_gro_offset(skb); @@ -187,7 +190,8 @@ static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) { - struct sk_buff *list, *next; + struct sk_buff *list, *next, *prefetch; + unsigned int i; list = ovpn_udp_gro_detach(skb); if (IS_ERR(list)) { @@ -198,8 +202,23 @@ static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) } skb->next = list; + /* a frag-list GRO aggregate makes later ciphertext visible before the + * current record is decrypted, so we prime the first two records, then + * keep the cache hints the same distance ahead while draining the list + */ + prefetch = skb->next ? skb : NULL; + for (i = 0; i < OVPN_UDP_GRO_PREFETCH_DISTANCE && prefetch; i++) { + ovpn_skb_prefetchw(prefetch); + prefetch = prefetch->next; + } + skb_list_walk_safe(skb, skb, next) { + if (prefetch) { + ovpn_skb_prefetchw(prefetch); + prefetch = prefetch->next; + } + skb_mark_not_on_list(skb); /* keep the current reference alive for the next record before From patchwork Wed Sep 16 08:35:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5364 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5930459mag; Wed, 16 Sep 2026 01:36:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBw7E/le4J/7UyD/D4GM9LvVw+EDZRNIWV8hC4TGz4rnwpx9oxCNfln3b4LhdP6NwMsnW1+j5U/Va6w=@openvpn.net X-Received: by 2002:a05:6830:378c:b0:800:a53f:707c with SMTP id 46e09a7af769-80b2c181571mr1824780a34.2.1789547777160; Wed, 16 Sep 2026 01:36:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547777; cv=none; d=google.com; s=arc-20260327; b=UQnhhMUxzG9beYOyCoePUGKV4xF8ttmC/720yO6Ppk1qTmHqGlu2jbLjDHdXL1XCpY HvjX98lUfT/dP3x5/qfdoil76n/Jdo/RdRp78jvSErSw2Slb/TmrLMM8gMYBTCd7gMW+ POxvipkLua6thLNXrKqr6oC5/eYg1O5qdEWMCfofjbVcZf8uAKajghLDAyHqvve7hRHF gS4smxRcWY0NxDTQnaRyP3zJ1Tkl37Yh0Ks2A1sn8bNNgnwQfrXI4muCGkIlNVhNOaro fyuxzwAU6atZ+cV8J6q6DZTzR6/mGplIihxXb3i/haNTyHyhZPUbztKQxpBXvze62sXF 4hjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=3nCZERF5M1HPfsFEXkxlIFnd16ySsbeCXUPBPQRggMg=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=hIagOseDaRwMoZEAeIkzaWV1r5o/k8pnKP2IyDH63QVl9pWAzeNIJRVg5iOaOKKNUp ZKKdWOfVQT94k7JIJZ01QbsYmIdEMtYNcM+YGBYVgkw3MJdcIC+jXfMCpG7CYT/xITqQ JnfnQ+UZqAcvGOGHTBUfGC5dYbk3CEazmK8SVCB8sv3Hn2ujQctcaNnsgZzzoo5/5pJ4 ZFhbV/JEavQwxrcLibC7+xPCCYeCIlIJSliv2+A6507abnNZR1M7hYIQc2ERFdaKwMAu W1u6PAlWyflzWnzdj9iTUX9CQH9XHlm0QdVZobHP7rQTwsRCIojQA89i6rgbKSxOF/TZ 8fLg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=i1dLJOXV; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="CGN/R75a"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=UjgSlJe7; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=dtdHOaqU; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-80b0860c624si2762876a34.94.2026.09.16.01.36.16 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:36:17 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=i1dLJOXV; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="CGN/R75a"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=UjgSlJe7; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=dtdHOaqU; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=3nCZERF5M1HPfsFEXkxlIFnd16ySsbeCXUPBPQRggMg=; b=i1dLJOXVlK4ErS17Xb1Sp+L6V8 +iABHE5utCtv5DzHqTWi9ETPvJahF1STo7JEGqv2oSJxK8iiHuEs5ez1AMdKN+vK1tDSyDQg5dsYf UbjRd4NCAqCRgnAH4LOKElAXiEJ7Wb6fPqCYcD9zCLgoX4NKoc7rm5jzEi0FDmCueEYI=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7o-0004l8-3x; Wed, 16 Sep 2026 08:36:13 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7L-0004cF-Bn for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=/HBAEcX+VIZvYPe/5m0G2nDlCHVefic+vzCRj3Yb0Ik=; b=CGN/R75aNujCf/e5YCdQM6lrRL 8ZfgLfCefusyTpQDX5DwQo+Epnjbz3D1qo8Yua3yFsz6UOGUx0rpM7SqhrjcosdaCU/JIqfYw/opd 9DjEe90cpEWJqIdorXBZ4gZsU+ZUsGS+VnVcd67NGNOPMzQvJEHUi5CKg0sJy3sVZ44M=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=/HBAEcX+VIZvYPe/5m0G2nDlCHVefic+vzCRj3Yb0Ik=; b=UjgSlJe7PFYMp25YvKqvY29fjh R9uRHoG6HOzNPCA09usU1TKyfgSr0zkiuEcie5JEqa98x5yzXxzOI1NrvbW7sB8HUXzKCZIXxwY6b biZcOtDYzA0jRlyFBawJckDPfJoIX3Z+vKC9nQSwEEJAzUCDzR2fuqfMSJzu3JBq1Q0o=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7F-00053K-Tq for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:44 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:b231:465::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4hlBzT57SNz5x0c for ; Wed, 16 Sep 2026 10:35:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547733; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/HBAEcX+VIZvYPe/5m0G2nDlCHVefic+vzCRj3Yb0Ik=; b=dtdHOaqUKCaalgRI0DRjK7Kxya+CoWvNNVfnEVAwAHeGPnBZjwEaFGbMFKRxRUFhN/60fu 47WPAsNnmzVqH3BvX5iaegVG7DUB5XEoDBWoWAvwPVznYOU2L+GAL3oXjBKMncfNEtFWz/ sAKuSCdKKjaXIfMAwFbDPgfOwsgEw4dS/fCVomcNOdSVFEHmXzbm/YLAVPa2b75eRu0oBz lPkWXXNC2cLc1iGQIrNTkMXPC1vCR+ffCk0tVbLokeUvGGxYo6ij38ConJSQefxrxP9gxL hvVEJCYUmhS+ebMm7zOH+XktVYElMXSKVAgD9LCrHF+xWt6VvYMOYXmMxNVl1Q== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::2 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:16 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hlBzT57SNz5x0c X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Software segmentation exposes the complete skb list before encryption begins. While encrypting each segment, request write ownership of the next linear segment cache line by cache line. A single skb n [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6l7F-00053K-Tq Subject: [Openvpn-devel] [RFC ovpn net-next v3 7/9] ovpn: prefetch GSO segments before in-place encryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476850095499086 X-GMAIL-MSGID: 1876476850095499086 Software segmentation exposes the complete skb list before encryption begins. While encrypting each segment, request write ownership of the next linear segment cache line by cache line. A single skb naturally skips the prefetch path. Signed-off-by: Ralf Lici --- No changes since v2 https://lore.kernel.org/openvpn-devel/3f903692549ee58887b1bb421f8b283059f66a1d.1789540779.git.ralf@mandelbit.com/ No changes since v1 https://lore.kernel.org/openvpn-devel/382879c187b1acca65198c467d1263266243ea50.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/io.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 11f7f16d7b79..cb7503a3e79c 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -472,6 +472,12 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, * independently */ skb_list_walk_safe(skb, curr, next) { + /* encrypting this segment can hide the cost of fetching the + * next segment's data into the cache + */ + if (next) + ovpn_skb_prefetchw(next); + if (unlikely(!ovpn_encrypt_one(peer, curr))) { ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); From patchwork Wed Sep 16 08:35:17 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5360 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5930087mag; Wed, 16 Sep 2026 01:35:51 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBw1C2DNozkGEwhJD5ZAu0PbyMhdV6hUy9SYp23gf0E2uvKXhA1GznStyrg0LPRDl2WnfFenmJIVKCc=@openvpn.net X-Received: by 2002:a05:6830:2b09:b0:7fb:547f:98bc with SMTP id 46e09a7af769-80b2cbb36a3mr2098864a34.5.1789547750942; Wed, 16 Sep 2026 01:35:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547750; cv=none; d=google.com; s=arc-20260327; b=ZV/M0jxw5rXf7BH8CDoBkqQClY7MvK3aw++FFx8G7cQ10KnLl2TSP+yVuuoCmJZQpK SfLnIVVesMdiZK9WOFmOpHnCxskiaZUalXWKOT3c5HkYCQ3wgS28pB7O/8I6e/vW/I9H nR372CdZ2ECNfQaioZsri19JPbp6L3TKl5QXSjHRewij1sKe/O6KfGpKRc53qLc6eOVq 4zPPRRnZ+3zVaQlbkhI5HpAMdGohFHEQsJN0gmNJAitrnIhZx+dUL8XgOQJp5r1xHznt rldirFX/5YB0o7pg0YSunYtIUH/KcgqtZohXHEMu+iyNINOqAKq26iCu9H5j6QsLHnk/ GAbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=7S+Ul8Z56nf547WODOD1LZBxk0eHivU+Cl620ao8WzE=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=aH3Kxd5P0qOJwilMnm35HFVtbVMPjMm7WAjym1mdT9QVPtJHPOyrQIrTuqhDL0PvUq vPiwhD+BAjwWbAo0lYDHRO6DuvTlHBFd3PLNrQYXyumM80Kajaex70YCW/rJXsamhcdX yaaIbff96P9g7iG4mbnOEqeBcwIp/Y1b5jAQN1ZYikKDA1/eJnneBvLdLjPvC0P1JDhZ jK6jKR2vNhc4HBeH6sfNk3uuD0vKHRRSTs0ECLvp4DTizLyOPrHiZ/0HAMb8lhMlpQMo MEsWq9vkyQPr1l082lFKXMa/XVqLuY8ITIlwmsWButP5zjl1AvwUPrqaOzTETvk4HmTq OxKQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=HYjwubwO; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=PLVRB2Wn; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b="ky/MJgr/"; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=onGn22K0; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-80b07ad8c8dsi3011367a34.46.2026.09.16.01.35.50 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:35:50 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=HYjwubwO; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=PLVRB2Wn; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b="ky/MJgr/"; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=onGn22K0; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=7S+Ul8Z56nf547WODOD1LZBxk0eHivU+Cl620ao8WzE=; b=HYjwubwOIKmoMN6TVRrGuj1y4X rk07b8ChdopretdMpqtq8a10eIm3apr0BIphXyzsGNMh5ZFs5lpUPLqsKoHkQpKLYQIyZLccL+xxN 82ucIbc5Dt7JeEza3g80YxcvDWd23EDb7JBI8DxHeRRYir2RYX7su1NGjbBGpB1/0D7g=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7K-0004YO-Ux; Wed, 16 Sep 2026 08:35:47 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7I-0004Y1-AB for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=a7gijAAEIUn/Ae6//xolhJZ6xhi2uWt9Sr+GRbs3slk=; b=PLVRB2WnCMTss7ID/yXRuWvE+0 +93LN+1k7GA1q1GsrDfHhHjhCvCDpa9/FkkRarquKWgRGPNjG1a4pbU4taOYdl6TKSi0GOFOvo1BO hFco7HPJLqALHeYxlH5NLEYhLObjfe7P/pJlHyhhxPYpAo6eJeW3G1eoG1YQ/tN1R/MA=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=a7gijAAEIUn/Ae6//xolhJZ6xhi2uWt9Sr+GRbs3slk=; b=ky/MJgr/0bYvhBQi//Y18gKPDR 3ZPKzNX52kzdXsd8aJTJx+vhCk56he7KxHihHKFcGURLwe3D6UfslNsHu3jyknhBiqqfljaM6uVm8 +v+z5nSb5lrF36zlIgcRuFTAqq7Neb3Igu5mAoCx5REopRXIk7Bwu363HvWOrtj2BUpg=; Received: from mout-b-206.mailbox.org ([195.10.208.51]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7H-0000m2-0r for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:45 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [IPv6:2001:67c:2050:b231:465::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-206.mailbox.org (Postfix) with ESMTPS id 4hlBzV46HRzS3 for ; Wed, 16 Sep 2026 10:35:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547734; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=a7gijAAEIUn/Ae6//xolhJZ6xhi2uWt9Sr+GRbs3slk=; b=onGn22K0fOCLlwSuCKJR/0qfZCIkKOecTl84b4pzPjKTJ0d2CXcDnmHc2JEXrrrGXW7thR uU4uzzxIVAvnzePHkcPwcVJAkeUlhKJyPGjVJmmpyX+Ud2zX8mQZF3Cf0b1uoY7sVp17Jg MRGQjB6aoJc7IYDQ9JkCQ4HaVxLoN+nipiIgitExowZK3G/PE977cZ4aebDq1byiaCXzKY MVQPr+R+J4pjRRJPpDo9jpf+CXwS3KsUo6/44Dt+Zyayarc5mijfQ4meBT8B/FCUnhexkI MMvo++xDf5uvZI25xtHntd877Ikqx77HJxayLATHcH30X8NFiKOSka0KO1s/7g== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::2 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:17 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hlBzV46HRzS3 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: XFRM's ESP GRO callbacks may consume an skb and return ERR_PTR(-EINPROGRESS) as an ownership marker. dev_gro_receive already recognizes this marker unconditionally and converts it to GRO_CONSUMED, so [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6l7H-0000m2-0r Subject: [Openvpn-devel] [RFC ovpn net-next v3 8/9] net: gro: honor skbs consumed by protocol callbacks X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476822641166509 X-GMAIL-MSGID: 1876476822641166509 XFRM's ESP GRO callbacks may consume an skb and return ERR_PTR(-EINPROGRESS) as an ownership marker. dev_gro_receive already recognizes this marker unconditionally and converts it to GRO_CONSUMED, so -EINPROGRESS is reserved by the generic GRO callback interface and cannot represent an ordinary callback error. The nested flush helpers currently avoid accessing a consumed skb only when XFRM offload is configured, because XFRM has so far been the sole user of the convention. Make the ownership check unconditional so other protocol callbacks can safely use the existing marker without acquiring an unrelated CONFIG_XFRM_OFFLOAD dependency. Callbacks which do not return the marker are unaffected. Signed-off-by: Ralf Lici --- No changes since v2 https://lore.kernel.org/openvpn-devel/3192bff7d69f958114e5fc9efe0dc5039c5be147.1789540779.git.ralf@mandelbit.com/ No changes since v1 https://lore.kernel.org/openvpn-devel/893f6c2b385f9df3e9617a9b7f547734061df195.1789485693.git.ralf@mandelbit.com/ include/net/gro.h | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/include/net/gro.h b/include/net/gro.h index 2300b6da05b2..20ddc5488789 100644 --- a/include/net/gro.h +++ b/include/net/gro.h @@ -361,9 +361,11 @@ static inline void skb_gro_remcsum_cleanup(struct sk_buff *skb, remcsum_unadjust((__sum16 *)ptr, grc->delta); } -#ifdef CONFIG_XFRM_OFFLOAD static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush) { + /* a GRO callback may consume skb and return this marker to prevent + * accessing the skb while unwinding through the enclosing GRO layers + */ if (PTR_ERR(pp) != -EINPROGRESS) NAPI_GRO_CB(skb)->flush |= flush; } @@ -378,21 +380,6 @@ static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb, skb->remcsum_offload = 0; } } -#else -static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush) -{ - NAPI_GRO_CB(skb)->flush |= flush; -} -static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb, - struct sk_buff *pp, - int flush, - struct gro_remcsum *grc) -{ - NAPI_GRO_CB(skb)->flush |= flush; - skb_gro_remcsum_cleanup(skb, grc); - skb->remcsum_offload = 0; -} -#endif INDIRECT_CALLABLE_DECLARE(struct sk_buff *ipv6_gro_receive(struct list_head *, struct sk_buff *)); From patchwork Wed Sep 16 08:35:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5365 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5931352mag; Wed, 16 Sep 2026 01:37:24 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvByb9HXvkO4PsSEqQ28gtltD3/7qwHpHE1dOc3htSFh70LG4h0i7cbpNgP3F6xqbehbSLHU2RrtbsAo=@openvpn.net X-Received: by 2002:a05:6820:616:b0:6be:354a:1ced with SMTP id 006d021491bc7-6c7d4fe7acfmr1916934eaf.64.1789547756852; Wed, 16 Sep 2026 01:35:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789547756; cv=none; d=google.com; s=arc-20260327; b=HjqGGFBiJlwEAEg+6rzkuZZplgmSHQlpw26bBKmZ3k+vSQvcOikojStL3PtCqrDT+P UOhcs253cA5cqXUbgb4pArM14bTt+VrqxWeIpOWb4Vzcf83vWNt2ltbKNaIZPL3yRTe3 kVrCPMmLgx3JWoeHZtdDsKRYdmAJjBDLGTO4sZUlCHBZSwqU5E6GSuFExB3qGBKC4DXX BAlAmZDdmWlooKHAwfgjbVHGzTYlWnHGMv9lJBgbGag5qEhw3OKtvT9d+rne7bX8AFQ3 tlIfB997qqDrmAVPEVOHO2tfSRUoIZkZxYzhhvShTYGFK4JfZXIe6jTTFwKKyt3/VIbW lrjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=ogTlPz317R3bfdrvU0rrRTeBkCjVROYjzczhkSxNL6Q=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=chqXyVg9vn0KTG5K3Db4p0RcF1BG+uLPnvtNUXwyMhe16U22BslDP2aANJueaK1av5 vB/CsCy9+OXrhcySjY8GqOs4/pxwM5YY6SgNGWV2pCx5BTWkp6ppXCCQD1NnTFbim0wD y6WwU005DWoow1Xd2/NLgnj38QYF1Qv1OoneOJjh6OF1Dcg5rnGaCQoYsObd0sUNvvTK UA1X/S+xYK9i582wFDR3HM5qFk4OyCIdjGq6EMJLl9iIQaUKAjLIDjCuWkFphBDX1krq UrSH8pT3/ONs41heVIyHCSUyPYrmqhncNVnsco7IvJWJ6iAoUW03vhTolXGPgL+S74G+ yCHg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=AHEIcfJF; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=NPxT1TpO; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=e7ZsxSyB; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=AxRr9l3B; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-80b080e398esi2816077a34.55.2026.09.16.01.35.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Sep 2026 01:35:56 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=AHEIcfJF; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=NPxT1TpO; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=e7ZsxSyB; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=AxRr9l3B; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ogTlPz317R3bfdrvU0rrRTeBkCjVROYjzczhkSxNL6Q=; b=AHEIcfJFbRRezu/2cA3jSae2jt VrV66uhhs5RUnO82x0JRZSHQrN+h57H4FBP61XfNx0HmipprEUB1Nz5eWq3V9FAwWg0I3/g2zRRHX bXp7eifVD4GHZh2k7DaWYQbZ1I739Gt6XJ+G/BwIFx9y8I9TOsliHFfXU5lPsK5KyN0Q=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6l7K-0003ld-Sz; Wed, 16 Sep 2026 08:35:47 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6l7J-0003lJ-5K for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:45 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=lBwD8hw741ubF2k3uaURKG/4AmoyhOE2/f7NsD5PoYI=; b=NPxT1TpOQk3zVbdXhB5Qc08v6q Bq5TZSTIdRS7PRfpmFwpESABWg5B+stfDDs49YMTj8LC8e/RgUrXa7CvZbD0h0Z8uJAMq79xagw4y uJLJdEtKAct5bxKn6EWGvx0mGwZC7evCW/VgUvt8qQiOaHOt9neSFtHhd1LcgM8QPLEQ=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=lBwD8hw741ubF2k3uaURKG/4AmoyhOE2/f7NsD5PoYI=; b=e7ZsxSyBUDA2wd3tZ3fMAMnIHt 9UKNz56qd1/DEJ2MTsELA0zIMwTerxpJDEGSp3RvoWoM8w/hLHOuzXGptohZPyrn9ZPlYgFfFQX5W CGF9cU4WY8mGM4qSSDGOJRoO/w/5awgBQUy4NAIK2dYGrZ9MigzKlkbAhTBRWZYkskOM=; Received: from mout-b-107.mailbox.org ([195.10.208.47]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6l7H-00053M-4J for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 08:35:45 +0000 Received: from smtp2.mailbox.org (smtp2.mailbox.org [10.196.197.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-107.mailbox.org (Postfix) with ESMTPS id 4hlBzW2R6Yz3y4g for ; Wed, 16 Sep 2026 10:35:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789547735; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lBwD8hw741ubF2k3uaURKG/4AmoyhOE2/f7NsD5PoYI=; b=AxRr9l3BQD2tNR1MMikjq/CPbYJ/XeyY/330ktRo8Vb0QthXp08WVgS+qbCJep9k6uCsdt SvlpoT7uvhFNQhAHVrI+3kP+Uoj99fYweXs6ScF2D9VrfZclJKEx3uYYDz0Zx8l98V857C lENa+Fp52H478TSjlEOZo5FQk1k7dVK0RE7gbVXqhm1OVeVxUKrg6RT2YvAe1fHHJa9UIJ SyqsxogwSskRUjY5XfNwVKxTBsnBfP3NuOA/jPLunpxyvg6PVYJhmDzJpd85lwyr6Ph1ui RD4neaF/L8LpnU7toOOxnWvkb2Z9vt67urS3rjGysJtrwIMwoceRdPt6Lcwn/g== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 10:35:18 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: The normal ovpn UDP GRO path retains compatible encrypted records in a frag-list and lets the completed aggregate traverse the outer IP and UDP receive stack before handing its records to the existing [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6l7H-00053M-4J Subject: [Openvpn-devel] [RFC ovpn net-next v3 9/9] ovpn: add opt-in direct GRO receive mode X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876476828464073258 X-GMAIL-MSGID: 1876476828464073258 The normal ovpn UDP GRO path retains compatible encrypted records in a frag-list and lets the completed aggregate traverse the outer IP and UDP receive stack before handing its records to the existing decryption path. Add an optional UDP GRO mode which instead consumes DATA_V2 packets from the UDP tunnel GRO callback and starts their existing per-record decryption immediately, similarly to xfrm. Control packets are flushed from GRO, restored, and continue through the normal stack. Select the mode through the immutable IFLA_OVPN_UDP_GRO_MODE link attribute. FULL_STACK remains the default for existing userspace, while DIRECT enables the new path. Keep this local receive policy per ovpn interface and select the corresponding GRO callback when each UDP socket is attached. This makes all sockets attached to the interface behave consistently without performing a mode lookup and dispatch for every packet. Account for both ovpn callbacks in the UDP tunnel GRO callback limit. The direct mode deliberately bypasses packet taps, TC ingress, outer IP validation and routing, netfilter hooks, the final UDP lookup, socket XFRM policy, and normal UDP receive accounting for DATA_V2. It is therefore an explicit operator choice for controlled transport interfaces rather than a transparent replacement for the full receive stack. Extend the UDP throughput selftest with a PRE_ROUTING nftables counter, verifying that full-stack aggregates reach the hook while direct-GRO aggregates bypass it. Before ciphertext prefetch was added to FULL_STACK, five interleaved single-flow AES-128-GCM runs per direction on two directly connected 100-Gbit/s mlx5 ports measured 22.087/22.962 Gbit/s forward/reverse in FULL_STACK and 24.315/25.313 Gbit/s in DIRECT, a 10.2% equal-weight gain. With the preceding prefetch commit enabled in FULL_STACK, later checks measured DIRECT within 1.3% forward and 0.7% reverse of FULL_STACK. Signed-off-by: Ralf Lici --- Changes since v2 https://lore.kernel.org/openvpn-devel/69c873c8837a0d8028c0427509aa384ee73be02b.1789540779.git.ralf@mandelbit.com/ - Copy the GRO mode into a byte-sized local before constructing the netlink attribute in ovpn-cli.c. (Sashiko) No changes since v1 https://lore.kernel.org/openvpn-devel/b4bd25d6c4c01a81447a31054abd095e0533fcdd.1789485693.git.ralf@mandelbit.com/ Documentation/netlink/specs/rt-link.yaml | 12 +++ drivers/net/ovpn/main.c | 15 ++- drivers/net/ovpn/ovpnpriv.h | 2 + drivers/net/ovpn/udp.c | 98 ++++++++++++++----- include/uapi/linux/if_link.h | 6 ++ net/ipv4/udp_offload.c | 2 +- tools/testing/selftests/net/ovpn/Makefile | 1 + tools/testing/selftests/net/ovpn/common.sh | 4 +- tools/testing/selftests/net/ovpn/ovpn-cli.c | 40 +++++++- .../selftests/net/ovpn/test-gro-direct.sh | 10 ++ tools/testing/selftests/net/ovpn/test.sh | 65 ++++++++++++ 11 files changed, 221 insertions(+), 34 deletions(-) create mode 100755 tools/testing/selftests/net/ovpn/test-gro-direct.sh diff --git a/Documentation/netlink/specs/rt-link.yaml b/Documentation/netlink/specs/rt-link.yaml index 7a72cd1b7e1e..d43e995c7f09 100644 --- a/Documentation/netlink/specs/rt-link.yaml +++ b/Documentation/netlink/specs/rt-link.yaml @@ -844,6 +844,14 @@ definitions: entries: - p2p - mp + - + name: ovpn-udp-gro-mode + enum-name: ovpn-udp-gro-mode + name-prefix: ovpn-udp-gro-mode + type: enum + entries: + - full-stack + - direct - name: br-stp-mode type: enum @@ -2365,6 +2373,10 @@ attribute-sets: name: mode type: u8 enum: ovpn-mode + - + name: udp-gro-mode + type: u8 + enum: ovpn-udp-gro-mode sub-messages: - diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index ac4e0d85e215..ecf27d1e2420 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -129,6 +129,9 @@ static const struct device_type ovpn_type = { static const struct nla_policy ovpn_policy[IFLA_OVPN_MAX + 1] = { [IFLA_OVPN_MODE] = NLA_POLICY_RANGE(NLA_U8, OVPN_MODE_P2P, OVPN_MODE_MP), + [IFLA_OVPN_UDP_GRO_MODE] = + NLA_POLICY_RANGE(NLA_U8, OVPN_UDP_GRO_MODE_FULL_STACK, + OVPN_UDP_GRO_MODE_DIRECT), }; /** @@ -200,6 +203,7 @@ static int ovpn_newlink(struct net_device *dev, struct rtnl_newlink_params *params, struct netlink_ext_ack *extack) { + enum ovpn_udp_gro_mode gro_mode = OVPN_UDP_GRO_MODE_FULL_STACK; struct ovpn_priv *ovpn = netdev_priv(dev); struct nlattr **data = params->data; enum ovpn_mode mode = OVPN_MODE_P2P; @@ -209,9 +213,14 @@ static int ovpn_newlink(struct net_device *dev, mode = nla_get_u8(data[IFLA_OVPN_MODE]); netdev_dbg(dev, "setting device mode: %u\n", mode); } + if (data && data[IFLA_OVPN_UDP_GRO_MODE]) { + gro_mode = nla_get_u8(data[IFLA_OVPN_UDP_GRO_MODE]); + netdev_dbg(dev, "setting UDP GRO mode: %u\n", gro_mode); + } ovpn->dev = dev; ovpn->mode = mode; + ovpn->gro_mode = gro_mode; spin_lock_init(&ovpn->lock); INIT_DELAYED_WORK(&ovpn->keepalive_work, ovpn_peer_keepalive_work); @@ -237,8 +246,8 @@ static int ovpn_newlink(struct net_device *dev, static size_t ovpn_get_size(const struct net_device *dev) { - /* IFLA_OVPN_MODE */ - return nla_total_size(sizeof(u8)); + /* IFLA_OVPN_MODE and IFLA_OVPN_UDP_GRO_MODE */ + return nla_total_size(sizeof(u8)) + nla_total_size(sizeof(u8)); } static int ovpn_fill_info(struct sk_buff *skb, const struct net_device *dev) @@ -247,6 +256,8 @@ static int ovpn_fill_info(struct sk_buff *skb, const struct net_device *dev) if (nla_put_u8(skb, IFLA_OVPN_MODE, ovpn->mode)) return -EMSGSIZE; + if (nla_put_u8(skb, IFLA_OVPN_UDP_GRO_MODE, ovpn->gro_mode)) + return -EMSGSIZE; return 0; } diff --git a/drivers/net/ovpn/ovpnpriv.h b/drivers/net/ovpn/ovpnpriv.h index 84499140e4bd..dc6210b99f4a 100644 --- a/drivers/net/ovpn/ovpnpriv.h +++ b/drivers/net/ovpn/ovpnpriv.h @@ -40,6 +40,7 @@ struct ovpn_peer_collection { * struct ovpn_priv - per ovpn interface state * @dev: the actual netdev representing the tunnel * @mode: device operation mode (i.e. p2p, mp, ..) + * @gro_mode: whether UDP data follows the full stack or is decrypted from GRO * @lock: protect this object * @peers: data structures holding multi-peer references * @peer: in P2P mode, this is the only remote peer @@ -49,6 +50,7 @@ struct ovpn_peer_collection { struct ovpn_priv { struct net_device *dev; enum ovpn_mode mode; + enum ovpn_udp_gro_mode gro_mode; spinlock_t lock; /* protect writing to the ovpn_priv object */ struct ovpn_peer_collection *peers; struct ovpn_peer __rcu *peer; diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index ce2584ecc934..f07b3e3d2a10 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -255,23 +255,12 @@ static struct ovpn_socket *ovpn_socket_from_udp_sock(struct sock *sk) return ovpn_sock; } -/** - * ovpn_udp_encap_recv - Start processing a received UDP packet. - * @sk: socket over which the packet was received - * @skb: the received packet - * - * If the first byte of the payload is: - * - DATA_V2 the packet is accepted for further processing, - * - DATA_V1 the packet is dropped as not supported, - * - anything else the packet is forwarded to the UDP stack for - * delivery to user space. - * - * Return: - * 0 if skb was consumed or dropped - * >0 if skb should be passed up to userspace as UDP (packet not consumed) - * <0 if skb should be resubmitted as proto -N (packet not consumed) +/* Process one packet after the caller has made its OpenVPN header visible at + * @payload_offset. A zero return means the skb was consumed. A positive return + * leaves a control packet for the UDP socket. */ -static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) +static int ovpn_udp_data_recv(struct sock *sk, struct sk_buff *skb, + unsigned int payload_offset) { struct ovpn_socket *ovpn_sock; struct ovpn_priv *ovpn; @@ -292,18 +281,16 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) goto drop_noovpn; } - /* Make sure the first 4 bytes of the skb data buffer after the UDP - * header are accessible. + /* Make sure the first 4 bytes of the OpenVPN header are accessible. * They are required to fetch the OP code, the key ID and the peer ID. */ - if (unlikely(!pskb_may_pull(skb, sizeof(struct udphdr) + - OVPN_OPCODE_SIZE))) { + if (unlikely(!pskb_may_pull(skb, payload_offset + OVPN_OPCODE_SIZE))) { net_dbg_ratelimited("%s: packet too small from UDP socket\n", netdev_name(ovpn->dev)); goto drop; } - opcode = ovpn_opcode_from_skb(skb, sizeof(struct udphdr)); + opcode = ovpn_opcode_from_skb(skb, payload_offset); if (unlikely(opcode != OVPN_DATA_V2)) { /* DATA_V1 is not supported */ if (opcode == OVPN_DATA_V1) @@ -313,7 +300,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) return 1; } - peer_id = ovpn_peer_id_from_skb(skb, sizeof(struct udphdr)); + peer_id = ovpn_peer_id_from_skb(skb, payload_offset); /* some OpenVPN server implementations send data packets with the * peer-id set to UNDEF. In this case we skip the peer lookup by peer-id * and we try with the transport address @@ -326,8 +313,10 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) if (unlikely(!peer)) goto drop; - /* pop off outer UDP header */ - __skb_pull(skb, sizeof(struct udphdr)); + /* the crypto receive path expects skb->data to begin at the OpenVPN + * header and takes ownership of the skb + */ + __skb_pull(skb, payload_offset); ovpn_udp_recv(peer, skb); return 0; @@ -338,6 +327,60 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) return 0; } +/* Consume DATA_V2 directly from UDP GRO. These packets deliberately bypass + * packet taps, TC ingress, the outer IP and netfilter receive paths, the final + * UDP lookup, and normal UDP accounting. Control packets are restored and + * continue through all of those layers normally. + */ +static struct sk_buff *ovpn_udp_gro_receive_direct(struct sock *sk, + struct list_head *head, + struct sk_buff *skb) +{ + unsigned int offset = skb_gro_offset(skb); + + /* if the OpenVPN header is not accessible, leave validation and drop + * handling to the ordinary UDP receive path + */ + if (unlikely(!pskb_pull(skb, offset))) + goto flush; + + /* tell UDP GRO not to touch the skb if it was consumed by the direct + * receive path + */ + if (likely(!ovpn_udp_data_recv(sk, skb, 0))) + return ERR_PTR(-EINPROGRESS); + + /* control packets still belongs to the socket so we restore the data + * pointer because the normal receive path expects the outer headers + */ + skb_push(skb, offset); + +flush: + NAPI_GRO_CB(skb)->same_flow = 0; + NAPI_GRO_CB(skb)->flush = 1; + return NULL; +} + +/** + * ovpn_udp_encap_recv - Start processing a received UDP packet. + * @sk: socket over which the packet was received + * @skb: the received packet + * + * If the first byte of the payload is: + * - DATA_V2 the packet is accepted for further processing, + * - DATA_V1 the packet is dropped as not supported, + * - anything else the packet is forwarded to the UDP stack for + * delivery to user space. + * + * Return: + * 0 if @skb was consumed or dropped + * 1 if @skb should continue through normal UDP delivery + */ +static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) +{ + return ovpn_udp_data_recv(sk, skb, sizeof(struct udphdr)); +} + /** * ovpn_udp4_output - send IPv4 packet over udp socket * @peer: the destination peer @@ -614,7 +657,12 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock, .encap_type = UDP_ENCAP_OVPNINUDP, .encap_rcv = ovpn_udp_encap_recv, .encap_destroy = ovpn_udp_encap_destroy, - .gro_receive = ovpn_udp_gro_receive_fraglist, + /* GRO mode cannot change after the interface is created so + * select the socket callback once at socket setup + */ + .gro_receive = ovpn->gro_mode == OVPN_UDP_GRO_MODE_DIRECT ? + ovpn_udp_gro_receive_direct : + ovpn_udp_gro_receive_fraglist, .gro_complete = ovpn_udp_gro_complete, }; struct ovpn_socket *old_data; diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index 245b36204525..2d7b320f83be 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -2067,9 +2067,15 @@ enum ovpn_mode { OVPN_MODE_MP, }; +enum ovpn_udp_gro_mode { + OVPN_UDP_GRO_MODE_FULL_STACK, + OVPN_UDP_GRO_MODE_DIRECT, +}; + enum { IFLA_OVPN_UNSPEC, IFLA_OVPN_MODE, + IFLA_OVPN_UDP_GRO_MODE, __IFLA_OVPN_MAX, }; diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index 187f108f3ee8..bfe23ec9dfca 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -41,7 +41,7 @@ struct udp_tunnel_type_entry { IS_ENABLED(CONFIG_VXLAN) * 2 + \ IS_ENABLED(CONFIG_NET_FOU) * 2 + \ IS_ENABLED(CONFIG_XFRM) * 2 + \ - IS_ENABLED(CONFIG_OVPN)) + IS_ENABLED(CONFIG_OVPN) * 2) DEFINE_STATIC_CALL(udp_tunnel_gro_rcv, dummy_gro_rcv); static DEFINE_STATIC_KEY_FALSE(udp_tunnel_static_call); diff --git a/tools/testing/selftests/net/ovpn/Makefile b/tools/testing/selftests/net/ovpn/Makefile index 169f0464ac3a..412a70abf739 100644 --- a/tools/testing/selftests/net/ovpn/Makefile +++ b/tools/testing/selftests/net/ovpn/Makefile @@ -37,6 +37,7 @@ TEST_PROGS := \ test-close-socket-tcp.sh \ test-close-socket.sh \ test-float.sh \ + test-gro-direct.sh \ test-large-mtu.sh \ test-mark.sh \ test-symmetric-id-float.sh \ diff --git a/tools/testing/selftests/net/ovpn/common.sh b/tools/testing/selftests/net/ovpn/common.sh index 2d844eb3aa6e..1467caa95168 100644 --- a/tools/testing/selftests/net/ovpn/common.sh +++ b/tools/testing/selftests/net/ovpn/common.sh @@ -10,6 +10,7 @@ source "$OVPN_COMMON_DIR/../../kselftest/ktap_helpers.sh" OVPN_UDP_PEERS_FILE=${OVPN_UDP_PEERS_FILE:-udp_peers.txt} OVPN_TCP_PEERS_FILE=${OVPN_TCP_PEERS_FILE:-tcp_peers.txt} OVPN_CLI=${OVPN_CLI:-${OVPN_COMMON_DIR}/ovpn-cli} +OVPN_UDP_GRO_MODE=${OVPN_UDP_GRO_MODE:-FULL_STACK} OVPN_YNL=${OVPN_YNL:-${OVPN_COMMON_DIR}/../../../../net/ynl/pyynl/cli.py} OVPN_ALG=${OVPN_ALG:-aes} OVPN_PROTO=${OVPN_PROTO:-UDP} @@ -162,7 +163,8 @@ ovpn_setup_ns() { done fi - ip netns exec "${peer}" ${OVPN_CLI} new_iface tun${1} $MODE + ip netns exec "${peer}" ${OVPN_CLI} new_iface tun${1} $MODE \ + "${OVPN_UDP_GRO_MODE}" ip -n "${peer}" addr add ${2} dev tun${1} # add a secondary IP to peer 1, to test a LAN behind a client if [ ${1} -eq 1 -a -n "${OVPN_LAN_IP}" ]; then diff --git a/tools/testing/selftests/net/ovpn/ovpn-cli.c b/tools/testing/selftests/net/ovpn/ovpn-cli.c index f4effa7580c0..5d3ae8fb1152 100644 --- a/tools/testing/selftests/net/ovpn/ovpn-cli.c +++ b/tools/testing/selftests/net/ovpn/ovpn-cli.c @@ -123,6 +123,8 @@ struct ovpn_ctx { char ifname[IFNAMSIZ]; enum ovpn_mode mode; bool mode_set; + enum ovpn_udp_gro_mode udp_gro_mode; + bool udp_gro_mode_set; int socket; int cli_sockets[MAX_PEERS]; @@ -1375,10 +1377,12 @@ static int ovpn_new_iface(struct ovpn_ctx *ovpn) { struct rtattr *linkinfo, *data; struct ovpn_link_req req = { 0 }; + uint8_t udp_gro_mode; int ret = -1; - fprintf(stdout, "Creating interface %s with mode %u\n", ovpn->ifname, - ovpn->mode); + fprintf(stdout, + "Creating interface %s with mode %u and UDP GRO mode %u\n", + ovpn->ifname, ovpn->mode, ovpn->udp_gro_mode); req.n.nlmsg_len = NLMSG_LENGTH(sizeof(req.i)); req.n.nlmsg_flags = NLM_F_REQUEST | NLM_F_CREATE | NLM_F_EXCL; @@ -1396,15 +1400,22 @@ static int ovpn_new_iface(struct ovpn_ctx *ovpn) strlen(OVPN_FAMILY_NAME) + 1) < 0) goto err; - if (ovpn->mode_set) { + if (ovpn->mode_set || ovpn->udp_gro_mode_set) { data = ovpn_nest_start(&req.n, sizeof(req), IFLA_INFO_DATA); if (!data) goto err; - if (ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_MODE, + if (ovpn->mode_set && + ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_MODE, &ovpn->mode, sizeof(uint8_t)) < 0) goto err; + udp_gro_mode = ovpn->udp_gro_mode; + if (ovpn->udp_gro_mode_set && + ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_UDP_GRO_MODE, + &udp_gro_mode, sizeof(udp_gro_mode)) < 0) + goto err; + ovpn_nest_end(&req.n, data); } @@ -1666,11 +1677,16 @@ static void usage(const char *cmd) cmd); fprintf(stderr, "where can be one of the following\n\n"); - fprintf(stderr, "* new_iface [mode]: create new ovpn interface\n"); + fprintf(stderr, + "* new_iface [mode] [udp-gro-mode]: create new ovpn interface\n"); fprintf(stderr, "\tiface: ovpn interface name\n"); fprintf(stderr, "\tmode:\n"); fprintf(stderr, "\t\t- P2P for peer-to-peer mode (i.e. client)\n"); fprintf(stderr, "\t\t- MP for multi-peer mode (i.e. server)\n"); + fprintf(stderr, "\tudp-gro-mode:\n"); + fprintf(stderr, "\t\t- FULL_STACK for the normal receive stack\n"); + fprintf(stderr, + "\t\t- DIRECT to decrypt data from the UDP GRO callback\n"); fprintf(stderr, "* del_iface : delete ovpn interface\n"); fprintf(stderr, "\tiface: ovpn interface name\n"); @@ -2206,6 +2222,20 @@ static int ovpn_parse_cmd_args(struct ovpn_ctx *ovpn, int argc, char *argv[]) return -1; } ovpn->mode_set = true; + + if (argc < 5) + break; + + if (!strcmp(argv[4], "FULL_STACK")) { + ovpn->udp_gro_mode = OVPN_UDP_GRO_MODE_FULL_STACK; + } else if (!strcmp(argv[4], "DIRECT")) { + ovpn->udp_gro_mode = OVPN_UDP_GRO_MODE_DIRECT; + } else { + fprintf(stderr, "Cannot parse UDP GRO mode: %s\n", + argv[4]); + return -1; + } + ovpn->udp_gro_mode_set = true; break; case CMD_DEL_IFACE: break; diff --git a/tools/testing/selftests/net/ovpn/test-gro-direct.sh b/tools/testing/selftests/net/ovpn/test-gro-direct.sh new file mode 100755 index 000000000000..f35db23eb425 --- /dev/null +++ b/tools/testing/selftests/net/ovpn/test-gro-direct.sh @@ -0,0 +1,10 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020-2025 OpenVPN, Inc. +# +# Author: Ralf Lici +# Antonio Quartulli + +OVPN_UDP_GRO_MODE="DIRECT" + +source test.sh diff --git a/tools/testing/selftests/net/ovpn/test.sh b/tools/testing/selftests/net/ovpn/test.sh index 9b5610837032..55cb4d4c3e3d 100755 --- a/tools/testing/selftests/net/ovpn/test.sh +++ b/tools/testing/selftests/net/ovpn/test.sh @@ -113,6 +113,67 @@ ovpn_run_lan_traffic() { ip netns exec ovpn_peer0 ping -qfc 100 -w 3 "${OVPN_LAN_IP}" } +ovpn_udp_gro_counter_add() { + [ "${OVPN_PROTO}" == "UDP" ] || return 0 + # A custom tunnel MTU can fragment outer packets before UDP GRO. + [ -z "${MTU:-}" ] || return 0 + + # Enable UDP forwarding GRO on the receiving endpoint so this test + # exercises the configured ovpn callback. + ovpn_cmd_ok "enable UDP GRO on the iperf receive path" \ + ip netns exec ovpn_peer0 ethtool -K veth1 gro on \ + rx-udp-gro-forwarding on + + ovpn_cmd_ok "create UDP GRO path counter table" \ + ip netns exec ovpn_peer0 nft add table inet ovpn_gro_test + ovpn_cmd_ok "create UDP GRO path counter chain" \ + ip netns exec ovpn_peer0 nft \ + "add chain inet ovpn_gro_test prerouting { type filter hook \ + prerouting priority filter; policy accept; }" + + # Count only aggregated outer packets after they enter the normal + # receive stack in peer0. Direct GRO consumes those DATA_V2 aggregates + # before this hook, while small packets which bypass veth's GRO path + # are deliberately ignored. + ovpn_cmd_ok "add UDP GRO path counter" \ + ip netns exec ovpn_peer0 nft add rule inet ovpn_gro_test \ + prerouting iifname "veth1" meta length gt 1500 udp dport 1 \ + counter +} + +ovpn_udp_gro_counter_check() { + local packets + + [ "${OVPN_PROTO}" == "UDP" ] || return 0 + [ -z "${MTU:-}" ] || return 0 + + packets=$(ip netns exec ovpn_peer0 nft list chain inet ovpn_gro_test \ + prerouting | sed -n \ + 's/.*counter packets \([0-9][0-9]*\) bytes.*/\1/p') + ovpn_cmd_ok "remove UDP GRO path counter table" \ + ip netns exec ovpn_peer0 nft delete table inet ovpn_gro_test + + if [ -z "${packets}" ]; then + printf '%s\n' "unable to read UDP GRO path counter" + return 1 + fi + + if [ "${OVPN_UDP_GRO_MODE}" == "FULL_STACK" ]; then + if [ "${packets}" -eq 0 ]; then + printf '%s\n' \ + "full-stack UDP GRO did not reach PRE_ROUTING" + return 1 + fi + return 0 + fi + + if [ "${packets}" -ne 0 ]; then + printf '%s\n' \ + "direct UDP GRO reached PRE_ROUTING ${packets} times" + return 1 + fi +} + ovpn_run_float_mode() { local p local peer_ns @@ -134,12 +195,16 @@ ovpn_run_float_mode() { ovpn_run_iperf() { local iperf_pid + ovpn_udp_gro_counter_add + ovpn_run_bg iperf_pid ip netns exec ovpn_peer0 iperf3 -1 -s sleep 1 ovpn_cmd_ok "run iperf throughput flow" \ ip netns exec ovpn_peer1 iperf3 -Z -t 3 -c 5.5.5.1 wait "${iperf_pid}" || return 1 + + ovpn_udp_gro_counter_check } ovpn_run_key_rollover() {