From patchwork Wed Sep 16 06:52:42 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5350 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853243mag; Tue, 15 Sep 2026 23:53:18 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBxs6QQJyMx9d9APqaY11F5lwX7GOPw0dVKpLrlYKCafulYCAjHQ6YzJ9uGe6CQWkIjOM9YcOFam7a0=@openvpn.net X-Received: by 2002:a05:6871:a872:b0:45e:daba:e3bc with SMTP id 586e51a60fabf-48475a7407emr1461772fac.15.1789541598025; Tue, 15 Sep 2026 23:53:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541598; cv=none; d=google.com; s=arc-20260327; b=P3/qb7OE6SxPEvTZ8tY53vXpEq0mVamHMyYQbSMj/cH3HKIdMa7LxsTbiPw0jNXCFb lCJdGpijrkJiUIWW72fhCJ8DOvKSRnKnIo0rze9BsICvMZ3VQWQjXIgqzCHIgkk4ewPJ xJ49hf+xKBJ6m1ZkO4bSXDJ43kpo8YDULvk5F5iCd4sTasVHg9JBkFtb43BrOw1UpDRs AWotMZV15fXS1L1hJzIoJ/yNB1IgResR80IMix+k7uVjg1itmwMGemvwcXOzlnIoMPBv 69yAQoumYIDO6LtFt+uVx3vqFLF6JLdCOJvSMLb/iHSMLOA6+2R5ctYeGXp6XF+Wk+aC 6Xfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=FkgqB8W+q5bnu0D/dCHJoSUoH9fwF7IIAOTsjJXD3RQ=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=SfIwAogHiWzltIjI6eitrH49MHW6jGE2Ff/gbOgo3dBiA3Ae0VOF0od3BSlzYVo1Hq q47QOD1pqzU0cOFZW8ZY45l3w5MsKF3HObSYCWBKa8RhmpiwiBRVLFcMjdSB58qSTPQK AO3XB7AMXHQMJFFIPFD2hCCWZW9FJpN4oslvOwiPK4uUYlgCdZ9RDwFLb1u5yGcbX/rI bsd4ZsWl1Ws+sS4FTalrkHB3MpLlsOSmN3/bJAAmCKv76UlGp0n5iT3YHdfkXX/tu2bD QRjPWuimNxaF92tZPmTBQK+vRSF4jp0YgQeGmqV28egDsJ85Lldxi2U2wK6IkNiYmEjz diDA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=BnB1dM3Z; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=BXVtwSnl; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=CqCtCkKu; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Z9iCxY7C; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-48429f20001si2049309fac.122.2026.09.15.23.53.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:18 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=BnB1dM3Z; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=BXVtwSnl; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=CqCtCkKu; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Z9iCxY7C; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=FkgqB8W+q5bnu0D/dCHJoSUoH9fwF7IIAOTsjJXD3RQ=; b=BnB1dM3Z75T6TkiYdt4liEVDhI IJDC9VT3JfhQJ+fAzFXhuCEQ3VXrebdfq1Gc2ejm+fM83A8NZFaN4K0zOHfwb8W33Y12Shj+L0d2a Ly/FqIm13L7bZZmpUHn0sAd1K/q22oWWv47L+vgrEMjUhGcS4vG6p1yaPkulxF4ghscQ=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW3-0001IO-Qi; Wed, 16 Sep 2026 06:53:12 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jVz-0001Hz-RN for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=fyQgrF5Ipkwull3dYlDG+VJucpyKg81I7KDThNvJq6Y=; b=BXVtwSnlzlmIY7sGmIesgZeeDZ R4/CGkt2WWtmns3ObOynDFI+rbKdVRmB/Zi906cI/Pq9KF8F+ecxSrbBfaldo9F0xb5ZuQEs8cJCe 1LOu4/6ixTqJwwHi+Bm0tG4zJ5JC1+7XzIDVmgBjLtKDvcRJW4RHNffV8+dGzEy2UG2A=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=fyQgrF5Ipkwull3dYlDG+VJucpyKg81I7KDThNvJq6Y=; b=CqCtCkKuSk3M7dFJGi0jS6GApM hQX/76TGzhZe2swLSb5JWYbJcVvHfhIbGMqdxSPxWOqwUo9xUMMEUB89Ns6f02fm8Mh0FMgQqHC+t u8JVSvEK/YLjzQgf+CRZFg1ZhdD3wL0wtvHsPO8Q45w8EltGhn4tv/Fxe3ZAngWJ2pSk=; Received: from mout-b-201.mailbox.org ([195.10.208.61]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jVy-000672-N0 for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:08 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:b231:465::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-201.mailbox.org (Postfix) with ESMTPS id 4hl8j61qPfzLlyR for ; Wed, 16 Sep 2026 08:52:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541578; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fyQgrF5Ipkwull3dYlDG+VJucpyKg81I7KDThNvJq6Y=; b=Z9iCxY7CBYNoUQITP9aTn2C1IL9kk4qvwiVc0PbWkGJ7XLrGhiH8saeDEsLdkWL83J/NHK /q3st0P48b2Ihu5g9dTCpDENnQ3N71pq3q4tiEbBZMF6C2CwrdRL2XlVyzdoFVLkV4c1kA DlP3+XA3RRR+zz81vkYeAuENdI/eAd3kb8il7D9g5YeyTHD4gCgORD5jJ0fvZgrbouvS2P RsCXHLP4UbukFJ4wVLGJ70vh6/pp+owWLaQVIne1oyf6lvFr1xGIShxrsVbwScGg600ijV /cbBPGIKd2GtBEbEGXqLvF0I2GUj3KTwhXgpSCSAiJAgYKYXCDW14bsosaxi5w== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::1 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:42 +0200 Message-ID: <1bbfde37488ade61928554db3a119525adf7b608.1789540779.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hl8j61qPfzLlyR X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1x6jVy-000672-N0 Subject: [Openvpn-devel] [RFC ovpn net-next v2 1/9] ovpn: advertise checksum offload for GSO packets X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470370864589144 X-GMAIL-MSGID: 1876470370864589144 ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before they reach ovpn because TCP GSO packets normally carry CHECKSUM_PARTIAL state. Advertise NETIF_F_HW_CSUM so the stack can pass such packets to ovpn. Complete partial checksums after any GSO segmentation and before submitting packets for encryption, since the inner packet checksum can no longer be fixed after the packet has been encrypted. Also pass the ovpn feature set to skb_gso_segment with GSO capabilities masked out: this forces software segmentation, but still lets the segmenter preserve supported non-GSO properties such as non-linear skb data instead of needlessly linearizing. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/1bbfde37488ade61928554db3a119525adf7b608.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/io.c | 20 ++++++++++++++++++-- drivers/net/ovpn/main.c | 2 +- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 9526f8096da6..112067ded401 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -358,6 +358,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; struct sk_buff_head skb_list; + netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; __be16 proto; @@ -392,8 +393,13 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_dst_drop(skb); if (skb_is_gso(skb)) { - segments = skb_gso_segment(skb, 0); - if (IS_ERR(segments)) { + /* force software segmentation, but keep ovpn's non-GSO feature + * bits so the generated segments can preserve non-linear skb + * data where possible + */ + features = netif_skb_features(skb); + segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", netdev_name(dev), ret); @@ -418,6 +424,16 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) continue; } + /* NETIF_F_HW_CSUM requires completing partial checksums */ + if (unlikely(curr->ip_summed == CHECKSUM_PARTIAL && + skb_checksum_help(curr) < 0)) { + net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", + netdev_name(dev)); + ovpn_dev_dstats_tx_dropped(ovpn->dev); + kfree_skb(curr); + continue; + } + /* only count what we actually send */ tx_bytes += curr->len; __skb_queue_tail(&skb_list, curr); diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 0708249e9607..28e1eb06e127 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -157,7 +157,7 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { - netdev_features_t feat = NETIF_F_SG | NETIF_F_GSO | + netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Wed Sep 16 06:52:43 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5355 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853283mag; Tue, 15 Sep 2026 23:53:20 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwb7jifxRG3cmVmdA0g/xh+/Vy4IEh+DM2l52/e5qif+N0Q0XvN9OrArhrxRaKuFfcaVq8ILAHmCMA=@openvpn.net X-Received: by 2002:a05:6820:c3d2:10b0:6b9:7d9e:5708 with SMTP id 006d021491bc7-6c7d15cebe7mr2822900eaf.3.1789541600281; Tue, 15 Sep 2026 23:53:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541600; cv=none; d=google.com; s=arc-20260327; b=PW3ZPux3TgEYVgq3nmPkgsEEQ0Ep7zJbOluto2zkKIlWFcAuQo+wTyeHeIKKjHszQz BKXDVF13wzrOYFHlvMbxEnNe9tV6bV32Dao53iwLZCRONqj2mmO2Q2u+yJBxsLAdJ0q1 WRW3humfWzVfw/y+k9ipoS6nnejI0U0nheHuqNChujUiUKnwpN5L/JV/wLPaK5p7CGL4 Krw09rzYVK+zr1/CXElnJ3jFrtOzMoQNHGgCa4PlYxCQcINfr31VsC0rPEcqXBzlvbkW KDo+YbS1IJnBwlSGtMPH+zFUy2Trpj4Wh0mgO6L6xWKexXxKI/1eGccbFw7UICvkRhqu pNNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=mxm8eVdBoVBQHbQC+vqCA0pgfKSu4owbZEC3wgxENsc=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=IH6XiGNON1GW3VmaLu7lQjUyZ9fjhq5gT5YH7pgka5sUD+ZCW5uQkbQAYp1MP9W2WN 0THlNQKSfWW452iNp4L2xVFQq7EHoqyRWdlnU20EvZexWlhLv6T+rcfRo4bEVhFj4sPN cz7PM0xQegs/XDVMqL0vGkQ2YDuE31EI7TW3mdd6zPUkuTk1K22AnAM02LJTwKWW7iPu W6Zx4/qDe9i6oNpAEX+2ouefEhmpfvaPV8SgJiPD6T8s66T/DBXzFE8HoNVtLKm06vk7 qJbvmU5fymTePzrb31b7cRNKq4L4iqrVn9CSQ57mzFHNxlIS9bvygy32Pnk91iP7uPTf /BbQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=Brj+QsjX; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=e8GRi46b; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=SyCGqdUO; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=dok8doYm; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-80b081ed126si2451919a34.76.2026.09.15.23.53.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:20 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=Brj+QsjX; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=e8GRi46b; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=SyCGqdUO; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=dok8doYm; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mxm8eVdBoVBQHbQC+vqCA0pgfKSu4owbZEC3wgxENsc=; b=Brj+QsjXkAfImItG8eGpvxBvUr o2lGcIQtiNUKXdWItg8GWclvVD2C1Jgvn8MUko1v2ULoz07S0sTyLJrtgmkE2sGtFXD5Z/55dhEU5 MpZ4PttKgErebvDMZ6OxwVJFlCf4M2HAEdnIAS4Nheoqg6udKKPlfZVj39Ggsybuomd4=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW1-00042N-WF; Wed, 16 Sep 2026 06:53:10 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jW0-000425-0G for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=MZVjiga8pyIY9u8NzfX8XEMiodR9+oFFncj7fLYshsA=; b=e8GRi46bmUK5YEN5GQ+LK5gmfo yc3lZ0442ghPbKh0qMZ/3KT+J6ygWarOWpAP1oZjZSut3K30klJECB1ZfrBXrav5sz0fWSmoCbPpw VutnJEMf+W4wr9Q0r88UKNxcNEE9tF+4Ty7XisvFQp+baJLqullr5LgBlJa/UTFgZdw0=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=MZVjiga8pyIY9u8NzfX8XEMiodR9+oFFncj7fLYshsA=; b=SyCGqdUOGp2U3JpzHFGtaIMk/t dqMsKSADty7CL+r17pYmgHg6YqQSeoEFaXbaw02a7s8JLi+u1uCM6wc3j2wC9RdWzLs4qI2uw/WuN YRtKZysTb0Dcp6NZ8zQ6Gler9QU/DJCVlhUQ8k08YlrijYlP6og0tHqQBdcOu3Dgwsh8=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jVz-0001sb-70 for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:08 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:b231:465::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4hl8j66Zhsz5wh1 for ; Wed, 16 Sep 2026 08:52:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541578; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MZVjiga8pyIY9u8NzfX8XEMiodR9+oFFncj7fLYshsA=; b=dok8doYm5tI2lyVG9WktR9opJIIFQBLnkxwEBo+DOvc919OSRhyrG/ZcDSDQSJvD+De4cn hS4bskrc6eLTZ6+xMCmQHL7KMfsP8Uk605/0Gd8vB6sVpJvHx1l0tM2dLjmH3cwTY48QJV u1Wf7s9QmVYoZ1RWKTrdLAHulW/n66o6/RQ6Mr0ripni8Od6LsaGabTizqIIuCraTlJVno uF5nkjAjgA8Un76vBeOnPtzMEAwYzl4zUdKTHwVjc+nYPZESjxM28HABTpRfJ2tK9KoXPp zEJpCjIE96+nwqYw6i/ezXTVx3458ikjfjH1BOUVv2vokOP/LRgRUFbsL2zN3w== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::1 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:43 +0200 Message-ID: <5baa1d94e29e7b109ffd412b4f1d5d113a319389.1789540779.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hl8j66Zhsz5wh1 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_ [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1x6jVz-0001sb-70 Subject: [Openvpn-devel] [RFC ovpn net-next v2 2/9] ovpn: accept frag-list GSO input X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470373092731855 X-GMAIL-MSGID: 1876470373092731855 Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_FRAGLIST, so generic transmit validation segments these aggregates before calling ovpn_net_xmit. That segmentation is functionally correct, but causes ovpn_net_xmit to be invoked separately for every resulting packet. Advertise frag-list storage so ovpn receives the aggregate intact and performs protocol validation and destination-to-peer lookup once before segmenting it. Frag-list GSO segmentation recovers the complete child skbs, which can then be encrypted in place and transmitted independently. Rebuilding those children into a replacement UDP GSO aggregate was found to add cost rather than improve throughput. The feature also admits non-GSO frag lists, which describe one packet split across several skbs. Let skb_cow_data preserve small lists directly. If a list exceeds the AEAD scatterlist limit, linearize it and continue rather than rejecting an otherwise valid packet. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/5baa1d94e29e7b109ffd412b4f1d5d113a319389.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 8 ++++++-- drivers/net/ovpn/main.c | 3 ++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 74eaf6fac2f5..2af493fd5735 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -168,8 +168,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, if (unlikely(nfrags < 0)) return nfrags; - if (unlikely(nfrags + 2 > (MAX_SKB_FRAGS + 2))) - return -ENOSPC; + if (unlikely(nfrags > MAX_SKB_FRAGS)) { + ret = skb_linearize(skb); + if (unlikely(ret)) + return ret; + nfrags = 1; + } /* allocate temporary memory for iv, sg and req */ tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 28e1eb06e127..ac4e0d85e215 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -158,7 +158,8 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | - NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; + NETIF_F_GSO_SOFTWARE | NETIF_F_FRAGLIST | + NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Wed Sep 16 06:52:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5356 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853316mag; Tue, 15 Sep 2026 23:53:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBxQDoDZtfTrpayJbT042FsbK7+TpCtf9txMfXnd0+X8+qBDsUIu+5+4hFVJJyZAzEh331nh/uHs9kM=@openvpn.net X-Received: by 2002:a05:6820:2221:b0:6c4:5f50:af03 with SMTP id 006d021491bc7-6c7ce737ce6mr1501466eaf.0.1789541603271; Tue, 15 Sep 2026 23:53:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541603; cv=none; d=google.com; s=arc-20260327; b=R1iNdrcT763vKMIUOUfe9n+sLPkEUAY+lionbRoh/j3I/5ISHhVRx+VNWXGWuxs64Z bpYabADIRR1uAQm+Q5dwXwAu4gM75ZUxkC3YVn50BwqOG3anwKxCpzGUTNY0mWX+tsEg CoW4Rju0rVr+U8cP04sv8XODxc2gZNXnmUIrUFUvS8iZwYdD+/ip7r2YJOQ26cLBw0Bw ZICSvJMUYSGs1aBJu0nIxtiilsYpEFXEdd2trHLAHJMVaXM4HLa/6+MLl+BqZFd51yZc vqyT09d5iWDbBTCQGZH6ApTsmLr2Snwmv7Kbaav2ZGC3wtg2+Q8OUrEVm9RiJTt6T1C6 WLqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=CVqqC9OMdHkMhi3mCOaI81goDCIBMB24cUbApKMlb5I=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=fQiC+n5B1ghJ0lwonaCUYMZy3MJ0BrvmPf13Ht0/fgEagl0t/SoG6P3OlazolMQi7F tDc4W4RfQmV88pOW7B0P2IUxm0PhErj2fa9JNfjEjwMR951Dngt4RyGWfGCfoewJbFLP /Mqz8QPIdrXL/+ixTn34pjzUiJt1EfqIBsdbm6SU2KzHNcKCe/IMQNkd2+KhAxbvPDsJ 1cqLBNnrZqRi5cUsARoHNuRV/VrMcx8LbBb++XAiX5iLsbar+A8PMCZcKwHU9AjP9NjA jHfUwrY6g/tqtS/I4LLTQ3MQC9Zr1THNCg1ZkN7jGa8yAeIy77vgDJ3BR7WMBWDji5jS AnHA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ZsAnbJg6; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=hLjj6Qtf; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=JxiIEJw9; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=y+jXXwDR; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-4842a13a265si2106906fac.177.2026.09.15.23.53.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:23 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ZsAnbJg6; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=hLjj6Qtf; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=JxiIEJw9; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=y+jXXwDR; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CVqqC9OMdHkMhi3mCOaI81goDCIBMB24cUbApKMlb5I=; b=ZsAnbJg6VxzCBpGvo5sGp8dkrt gMevVynt/66D3eGdjuKS1gdugZtKmxNJII88bJGxiMicS2owG2B7coUnK6uB5ZCZ0sg0yIw0u72oP ZmQNjjpZK7crWCPMJ9DNWfjzaJ2zWZn0XZq7xEtXrOCo6uH9XPxH3YERd2Heqm7cp1v8=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW6-00051G-EO; Wed, 16 Sep 2026 06:53:11 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jW3-000512-1h for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=A1UWsdQLXL9+dhF4kphq/thkQTF2NVP4hEL9I9Weq2U=; b=hLjj6QtfiZ2GzPTNBQ7TDeTlNY AZgksKWFIit6Na4QDsART3tWdOYLaRJ5hJwqzLA/dPcwpoofMAlvVM9kVRF68RFcSDfdv+CoxhdMp GjPHvn9Uyu4sge/+ge1bu2zlXJrWDMoGNqFu4hB1LbdiUkhbsbYOlY/wag0JjRf9WCto=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=A1UWsdQLXL9+dhF4kphq/thkQTF2NVP4hEL9I9Weq2U=; b=JxiIEJw9GFaCBU5qZExbSlMynu wXOJrZj3N2p9ZfzMrbL7+uAbJqnmm/STZEGVdgbZHmcFlLsxHqQ7jkNBR15hlymC88xXlOuLlWNdm b9Kx7JN+tFT5f8y4+LnCBtypO5J5FilWE4K0Qq4wdoniFOTBDYUbk7Izv/Wi36ihSttk=; Received: from mout-b-106.mailbox.org ([195.10.208.46]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jVz-000673-9G for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:08 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:b231:465::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-106.mailbox.org (Postfix) with ESMTPS id 4hl8j737hnzNlqB for ; Wed, 16 Sep 2026 08:52:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541579; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A1UWsdQLXL9+dhF4kphq/thkQTF2NVP4hEL9I9Weq2U=; b=y+jXXwDRMbgo2wutgCd7a/RJrviFoohF7XLoY2hvASOZbh78oL7UWsC9+cc5qL544y1oui 6ll8bIL2kZbhAi1DwQpIJyIy+h/ukEnoc/9X8ZTnXnqePZl0Nr8DFz9G/pZWNxUvvwS/nN FluGXLGkEXLNLMVM3TPqsQd5Txb8BP02u2ufOvrbuDeezTBkgJGbIwUKQcAMqk/zUGg4Be CvVlkswp3my9xwo2lwXmivJ3C/ex8oWMReDRmseAvtevnHLopxPBKzPYiIAiUE6oe4iA0A e/YOwDpOtvabY8h9/qU1APCSt9zFgBqdzdntT94w4bOEeeZmrOLwJVCRdI34UA== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::1 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:44 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hl8j737hnzNlqB X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1x6jVz-000673-9G Subject: [Openvpn-devel] [RFC ovpn net-next v2 3/9] ovpn: refactor AEAD encryption helpers X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470376003513813 X-GMAIL-MSGID: 1876470376003513813 Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without changing packet handling. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/f075b9782b14d64756c84e132c138482f08287b9.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 97 ++++++++++++++++++++-------------- drivers/net/ovpn/io.h | 3 +- drivers/net/ovpn/proto.h | 4 ++ 3 files changed, 63 insertions(+), 41 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 2af493fd5735..30299581422d 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -24,9 +24,6 @@ #include "proto.h" #include "skb.h" -#define OVPN_AUTH_TAG_SIZE 16 -#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE) - #define ALG_NAME_AES "gcm(aes)" #define ALG_NAME_CHACHAPOLY "rfc7539(chacha20,poly1305)" @@ -42,7 +39,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * an AEAD request structure with extra space for SG * and IV. * @tfm: the AEAD cipher handle - * @nfrags: the number of fragments in the skb + * @nents: the number of scatterlist entries * * This function calculates the size of a contiguous memory block that includes * the initialization vector (IV), the AEAD request, and an array of scatterlist @@ -54,7 +51,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * Return: the size of the temporary memory that needs to be allocated */ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, - const unsigned int nfrags) + const unsigned int nents) { unsigned int len = OVPN_NONCE_SIZE; @@ -70,8 +67,8 @@ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, /* round up to the next multiple of the scatterlist alignment */ len = ALIGN(len, __alignof__(struct scatterlist)); - /* add enough space for nfrags + 2 scatterlist entries */ - len += array_size(sizeof(struct scatterlist), nfrags + 2); + /* add enough space for the scatterlist entries */ + len += array_size(sizeof(struct scatterlist), nents); return len; } @@ -135,6 +132,53 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, __alignof__(struct scatterlist)); } +static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, + struct sk_buff *skb, + unsigned int nents, u8 **iv) +{ + struct aead_request *req; + void *tmp; + + /* allocate IV, request and scatterlist entries in one block */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + if (unlikely(!tmp)) + return ERR_PTR(-ENOMEM); + + ovpn_skb_cb(skb)->crypto_tmp = tmp; + *iv = ovpn_aead_crypto_tmp_iv(aead, tmp); + req = ovpn_aead_crypto_tmp_req(aead, *iv); + + return req; +} + +static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + u8 *iv, u8 *data) +{ + u32 pktid, op; + int ret; + + /* obtain packet ID, which is used both as a first + * 4 bytes of nonce and last 4 bytes of associated data. + */ + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + if (unlikely(ret < 0)) + return ret; + + /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes + * nonce + */ + ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); + + /* add the packet opcode and wire nonce as associated data */ + op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); + BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); + *(__force __be32 *)data = htonl(op); + memcpy(data + OVPN_OPCODE_SIZE, iv, OVPN_NONCE_WIRE_SIZE); + + return 0; +} + int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { @@ -143,8 +187,6 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *trailer; struct scatterlist *sg; int nfrags, ret; - u32 pktid, op; - void *tmp; u8 *iv; ovpn_skb_cb(skb)->peer = peer; @@ -175,16 +217,9 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), - GFP_ATOMIC); - if (unlikely(!tmp)) - return -ENOMEM; - - ovpn_skb_cb(skb)->crypto_tmp = tmp; - - iv = ovpn_aead_crypto_tmp_iv(ks->encrypt, tmp); - req = ovpn_aead_crypto_tmp_req(ks->encrypt, iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + if (IS_ERR(req)) + return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); /* sg table: @@ -206,28 +241,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, __skb_push(skb, tag_size); sg_set_buf(sg + ret + 1, skb->data, tag_size); - /* obtain packet ID, which is used both as a first - * 4 bytes of nonce and last 4 bytes of associated data. - */ - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + /* make space for the additional data and push it to the front */ + __skb_push(skb, OVPN_AAD_SIZE); + ret = ovpn_aead_encrypt_header(peer, ks, iv, skb->data); if (unlikely(ret < 0)) return ret; - /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes - * nonce - */ - ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); - - /* make space for packet id and push it to the front */ - __skb_push(skb, OVPN_NONCE_WIRE_SIZE); - memcpy(skb->data, iv, OVPN_NONCE_WIRE_SIZE); - - /* add packet op as head of additional data */ - op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); - __skb_push(skb, OVPN_OPCODE_SIZE); - BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); - *((__force __be32 *)skb->data) = htonl(op); - /* AEAD Additional data */ sg_set_buf(sg, skb->data, OVPN_AAD_SIZE); @@ -281,7 +300,7 @@ int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return -ENOSPC; /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags), + tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags + 2), GFP_ATOMIC); if (unlikely(!tmp)) return -ENOMEM; diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index db9e10f9077c..1a94f0fda1d1 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -11,8 +11,7 @@ #define _NET_OVPN_OVPN_H_ /* DATA_V2 header size with AEAD encryption */ -#define OVPN_HEAD_ROOM (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE + \ - 16 /* AEAD TAG length */ + \ +#define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ max(sizeof(struct ipv6hdr), sizeof(struct iphdr))) diff --git a/drivers/net/ovpn/proto.h b/drivers/net/ovpn/proto.h index b7d285b4d9c1..f3b305cbefe5 100644 --- a/drivers/net/ovpn/proto.h +++ b/drivers/net/ovpn/proto.h @@ -39,6 +39,10 @@ #define OVPN_NONCE_WIRE_SIZE (OVPN_NONCE_SIZE - OVPN_NONCE_TAIL_SIZE) #define OVPN_OPCODE_SIZE 4 /* DATA_V2 opcode size */ +#define OVPN_AUTH_TAG_SIZE 16 +#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + \ + OVPN_NONCE_WIRE_SIZE) +#define OVPN_DATA_V2_OVERHEAD (OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE) #define OVPN_OPCODE_KEYID_MASK 0x07000000 #define OVPN_OPCODE_PKTTYPE_MASK 0xF8000000 #define OVPN_OPCODE_PEERID_MASK 0x00FFFFFF From patchwork Wed Sep 16 06:52:45 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5354 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853266mag; Tue, 15 Sep 2026 23:53:19 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwK5ll3Dazr/JpOdLro+zYAjZSAhz/1EqQ/Z04SHpBcDXlwuDK9a/8KMNp8yWk+zjPXPjBe3wOdjAY=@openvpn.net X-Received: by 2002:a05:6808:150d:b0:4c3:ee9e:200c with SMTP id 5614622812f47-4ca499bfaddmr1621437b6e.1.1789541598865; Tue, 15 Sep 2026 23:53:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541598; cv=none; d=google.com; s=arc-20260327; b=sjcXUqcIuhXxis/HcEt2nVmkFhDqLn6D4m2WJbJaMPhYPBDha5HtfNDHjwCaXtdiTz RuRLCtACO0biH4Xw8ezF9VZOyAtM0u6cL/iAYfI7OY3VA6hrT040d82+KKh75hUQ9rbf WUH8A+qnyy6JJ5KIS4OByuPke7jY4ZB0Osn/vRTrtMSzLbah92ibulu2kKMgsCu9ZJeA UuZGk2oaxDvrKq4pVNY4oeKcUwyjKUdcwuPmTzksdN0vxfc/hV0B4W8si+BMUVloUKgi MtdGGXo7l5qg2rsMi8uGq6tFOXUGLbNRDgMBckXQr6JtKKeS2+CCZPsjR4L45FgYODup 33Kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=65oDHv+gxsp0O3QJUzUKKOzqv/bwJUzduhk/SsfpDkE=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=OeVTpy/eoDCvvPFRKPaD+BUBd0TDPt8C4UHtvz3A7qoARC2XaepzgRNf/o3cNskIe3 Uq8mDmIe/u1fc4bDkwCat5g2ezbP1ntPq3aWCyRHkCVmeJnc5kUDydgz1oj8SEAQ9TyY Xp9D2BPsyZ0U7rRu60oJfVDfGiSYPBQtfT58wHT7WjxiEsp7EOhFu5bbDSs3B4EdM4nM Z/QetZIWGzaKbCgLyKhc1rsEHFqaA1xP3keG0Mnj/djksp9w168aMxyS8DOpTkfd0eJW 3DMX9vdD2ODRs9EqubgdXF4K7zNmZgljjCwElr8rNN4TG9MmrjpfqyF18cinmriD86D2 488A==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ekFYdwUP; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=nHjnt2jL; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Qkyhy8GI; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=RwrMU1Zv; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5614622812f47-4ca26167054si2469566b6e.132.2026.09.15.23.53.18 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:18 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ekFYdwUP; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=nHjnt2jL; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Qkyhy8GI; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=RwrMU1Zv; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=65oDHv+gxsp0O3QJUzUKKOzqv/bwJUzduhk/SsfpDkE=; b=ekFYdwUP/hsv3IQ+dOzxJgrW/0 n1qquEahbrM41f3TuoeJkrrgLn/DQ9gppOqAjsMmJl2wlsPBZsjcKxUr2uQ8anuLl8WHC6w4L0NZv K7Q2JzytTMSCeb2Yg9gvI11Fg1IBJf988w8tTSIaxffX4K3EqLHJqIpFCVzupQt2APqk=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW4-0001IV-4b; Wed, 16 Sep 2026 06:53:12 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jW1-0001I7-OU for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=9xmVKnUhklyEnYhraJrWdOm1XOdAwykDpwfVcBT908E=; b=nHjnt2jLR24Xew0XNxB5btSiZE /YFXM4mf3S1U9/khl5Nv5FlMR4fpnKRXu/vVLqrxvOArSTYVetX83S4MQnstwwMAXS+hFudsRR2iB Rl5Onm3fPKBJA3gC2CcQgoKm7Q38XdXtCJ3Wq6V1ykFT/GXrCdw+PqnamAMrhZg09UB4=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=9xmVKnUhklyEnYhraJrWdOm1XOdAwykDpwfVcBT908E=; b=Qkyhy8GI/yhKno1TQZYre+0bei vd9ak3JR/+j7kZoZ+CIby2kdUK0+sMw99IK/StW2p3RpqNQKtS0NCKKSM4UO2CNeZ6OIEvexwJPAD wvb1gtLa3N4qciOY+gsqLkwQCQFbK5Bh6k22qZzMGFbBBTG12b6q7g+pVWzL0uYT/gec=; Received: from mout-b-107.mailbox.org ([195.10.208.47]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jW0-0001sg-AD for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:10 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:b231:465::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-107.mailbox.org (Postfix) with ESMTPS id 4hl8j80Hwxz3yDh for ; Wed, 16 Sep 2026 08:53:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541580; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9xmVKnUhklyEnYhraJrWdOm1XOdAwykDpwfVcBT908E=; b=RwrMU1ZvkUD+dK4PIJoH1GeJNllxfFs/ucXhT6+WtfUQT5ONri+DJFU194swseUZJssOv9 wilZsP1lmfAGk6AXINSGIyFg9fp5SjZyqeHskno/sNQyRoRFNq0oQ+q0zLdg5WlexLSgBI TyILsFPBepssl0/YxLeZBOxwzduqPD5FFrFpl+0NAIc44OuaiFgiLDH9nWSxxU2MttZpfA GozRQ4V1NhHO6ZCXA8wyp8xhF1q2fHKgZLZtUnulI+QuCQJh3k3ZWuxgx45EFxJQerPH/S 2hzxDMqMObfAxYoTGdkVnWCc1rh9sxbom/8p0eSLRlcf1e4YHSew7KPlO06x3A== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::1 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:45 +0200 Message-ID: <9359f737627fc84be08ee7d301415c023ebadeb2.1789540779.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hl8j80Hwxz3yDh X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb into linear packets while completing their checksums, then emit eligible fixed-size inputs as UDP GSO skb(s). Allocate one final page-backed aggregate per batch before submitting encryption and have each AEAD request write out of place directly into its record slot. Attempting in-place encryption would be com [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1x6jW0-0001sg-AD Subject: [Openvpn-devel] [RFC ovpn net-next v2 4/9] ovpn: convert GSO input into UDP GSO output X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470371778637442 X-GMAIL-MSGID: 1876470371778637442 Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb into linear packets while completing their checksums, then emit eligible fixed-size inputs as UDP GSO skb(s). Allocate one final page-backed aggregate per batch before submitting encryption and have each AEAD request write out of place directly into its record slot. Attempting in-place encryption would be complex (the OpenVPN wire layout adds a header and authentication tag to every segment) and not necessarily more performant: encrypting into individual skbs would still require assembling or copying those records into the UDP GSO skb. The destination allocation and lifetime are instead amortized over the whole batch. Keep the existing in-place path for ordinary packets, where allocating and retiring a separate output skb for every record would provide no aggregate construction benefit. Also retain that path for frag-list GSO input: it already stores complete segments as child skbs, and measurements show that copying those children into another aggregate is counterproductive. Transmit the aggregate as SKB_GSO_UDP_L4 only after every record succeeds, and discard it if any request fails. Split aggregates at the legacy GSO size limit and fall back to individual records when batching is unavailable or the segment geometry is unsuitable. Preserve the input priority, flow hash and sender CPU on the replacement aggregate. If the input has real write ownership, charge the aggregate to the same socket as well. This retains socket lifetime and write-memory accounting and lets lower-device queue selection use the socket's cached TX queue instead of choosing a new queue after crypto completion. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following throughput: Forward Reverse Before this change 11.522 Gbit/s 9.902 Gbit/s Software UDP segmentation 12.879 Gbit/s 12.516 Gbit/s Hardware UDP segmentation 18.308 Gbit/s 19.233 Gbit/s The equal-weight mean of the two directional results increased from 10.712 to 18.770 Gbit/s with hardware UDP segmentation, a 75.2% improvement. With segmentation performed in software, it increased to 12.697 Gbit/s, an 18.5% improvement. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/9359f737627fc84be08ee7d301415c023ebadeb2.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 85 ++++++++++- drivers/net/ovpn/crypto_aead.h | 4 + drivers/net/ovpn/io.c | 258 ++++++++++++++++++++++++++++++--- drivers/net/ovpn/skb.h | 27 +++- drivers/net/ovpn/stats.h | 16 +- drivers/net/ovpn/tcp.c | 4 +- drivers/net/ovpn/udp.c | 27 +++- 7 files changed, 382 insertions(+), 39 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 30299581422d..8eb76268dc3a 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -134,13 +134,17 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, struct sk_buff *skb, - unsigned int nents, u8 **iv) + unsigned int nents, + unsigned int extra, u8 **iv) { struct aead_request *req; void *tmp; - /* allocate IV, request and scatterlist entries in one block */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + /* allocate IV, request, scatterlist entries and caller scratch space + * in one block + */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents) + extra, + GFP_ATOMIC); if (unlikely(!tmp)) return ERR_PTR(-ENOMEM); @@ -217,7 +221,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, 0, &iv); if (IS_ERR(req)) return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); @@ -261,6 +265,79 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return crypto_aead_encrypt(req); } +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, + struct sk_buff *gso_skb, unsigned int offset) +{ + const unsigned int dst_nents = skb_shinfo(gso_skb)->nr_frags + 4; + const unsigned int src_nents = 2; + unsigned int nents, payload_off; + struct scatterlist *src, *dst; + struct aead_request *req; + int dst_idx, mapped, ret; + u8 *aad, *iv; + + /* each input records the shared peer and key for the common completion + * path but their references remain owned by the output aggregate + */ + ovpn_skb_cb(skb)->peer = peer; + ovpn_skb_cb(skb)->ks = ks; + + if (WARN_ON_ONCE(skb_is_nonlinear(skb))) + return -EINVAL; + + nents = src_nents + dst_nents; + req = ovpn_aead_request_alloc(ks->encrypt, skb, nents, OVPN_AAD_SIZE, + &iv); + if (IS_ERR(req)) + return PTR_ERR(req); + src = ovpn_aead_crypto_req_sg(ks->encrypt, req); + dst = src + src_nents; + aad = (u8 *)(dst + dst_nents); + + ret = ovpn_aead_encrypt_header(peer, ks, iv, aad); + if (unlikely(ret < 0)) + return ret; + + ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); + if (unlikely(ret < 0)) + return ret; + + /* encrypt out of place from the original segmented skb directly into + * its final range in the UDP GSO skb + */ + sg_init_table(src, src_nents); + sg_set_buf(src, aad, OVPN_AAD_SIZE); + sg_set_buf(src + 1, skb->data, skb->len); + + sg_init_table(dst, dst_nents); + dst_idx = skb_to_sgvec_nomark(gso_skb, dst, offset, OVPN_AAD_SIZE); + if (unlikely(dst_idx < 0)) + return dst_idx; + + payload_off = offset + OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE; + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, payload_off, + skb->len); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, + offset + OVPN_AAD_SIZE, + OVPN_AUTH_TAG_SIZE); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + sg_mark_end(&dst[dst_idx - 1]); + + aead_request_set_tfm(req, ks->encrypt); + aead_request_set_callback(req, 0, ovpn_encrypt_post, skb); + aead_request_set_crypt(req, src, dst, skb->len, iv); + aead_request_set_ad(req, OVPN_AAD_SIZE); + + return crypto_aead_encrypt(req); +} + int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { diff --git a/drivers/net/ovpn/crypto_aead.h b/drivers/net/ovpn/crypto_aead.h index fae3b585a43b..8b444744944e 100644 --- a/drivers/net/ovpn/crypto_aead.h +++ b/drivers/net/ovpn/crypto_aead.h @@ -17,6 +17,10 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + struct sk_buff *skb, struct sk_buff *gso_skb, + unsigned int offset); int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 112067ded401..3ad4cadeeb02 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include "ovpnpriv.h" #include "peer.h" @@ -32,6 +34,12 @@ const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE] = { 0x07, 0xed, 0x2d, 0x0a, 0x98, 0x1f, 0xc7, 0x48 }; +/* Leave room for the largest outer network header. The strict inequality in + * is_skb_forwardable also requires staying one byte below gso_max_size. + */ +#define OVPN_UDP_GSO_MAX_PAYLOAD (GSO_LEGACY_MAX_SIZE - \ + sizeof(struct ipv6hdr) - \ + sizeof(struct udphdr) - 1) /** * ovpn_is_keepalive - check if skb contains a keepalive message * @skb: packet to check @@ -237,11 +245,13 @@ void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb) void ovpn_encrypt_post(void *data, int ret) { + unsigned int orig_len, packets = 1; struct ovpn_crypto_key_slot *ks; struct sk_buff *skb = data; struct ovpn_socket *sock; + struct ovpn_cb *batch_cb; struct ovpn_peer *peer; - unsigned int orig_len; + struct sk_buff *batch; /* encryption is happening asynchronously. This function will be * called later by the crypto callback with a proper return value @@ -249,15 +259,19 @@ void ovpn_encrypt_post(void *data, int ret) if (unlikely(ret == -EINPROGRESS)) return; - ks = ovpn_skb_cb(skb)->ks; + /* ordinary encryption leaves batch zeroed; a GSO input uses it to find + * the aggregate whose lifetime is shared by all segment requests + */ + batch = ovpn_skb_cb(skb)->batch; peer = ovpn_skb_cb(skb)->peer; + ks = ovpn_skb_cb(skb)->ks; /* crypto is done, cleanup skb CB and its members */ kfree(ovpn_skb_cb(skb)->crypto_tmp); if (unlikely(ret == -ERANGE)) { /* we ran out of IVs and we must kill the key as it can't be - * use anymore + * used anymore */ netdev_warn(peer->ovpn->dev, "killing key %u for peer %u\n", ks->key_id, @@ -265,8 +279,30 @@ void ovpn_encrypt_post(void *data, int ret) if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) /* let userspace know so that a new key must be negotiated */ ovpn_nl_key_swap_notify(peer, ks->key_id); + } - goto err; + if (batch) { + batch_cb = ovpn_skb_cb(batch); + /* every segment publishes its result before releasing its + * pending count and only the final completion continues with + * the aggregate + */ + if (unlikely(ret < 0)) + atomic_set(&batch_cb->batch_state.failed, 1); + + kfree_skb(skb); + if (!atomic_dec_and_test(&batch_cb->batch_state.pending)) + return; + + skb = batch; + packets = skb_shinfo(batch)->gso_segs; + if (unlikely(atomic_read(&batch_cb->batch_state.failed))) + goto err; + + /* reaching the final callback with no sticky failure means + * every segment completed successfully + */ + ret = 0; } if (unlikely(ret < 0)) @@ -292,7 +328,7 @@ void ovpn_encrypt_post(void *data, int ret) goto err_unlock; } - ovpn_peer_stats_increment_tx(&peer->link_stats, orig_len); + ovpn_peer_stats_add_tx(&peer->link_stats, orig_len, packets); /* keep track of last sent packet for keepalive */ WRITE_ONCE(peer->last_sent, ktime_get_boottime_seconds()); /* skb passed down the stack - don't free it */ @@ -301,7 +337,7 @@ void ovpn_encrypt_post(void *data, int ret) rcu_read_unlock(); err: if (unlikely(skb)) - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, packets); kfree_skb(skb); if (likely(ks)) ovpn_crypto_key_slot_put(ks); @@ -309,29 +345,124 @@ void ovpn_encrypt_post(void *data, int ret) ovpn_peer_put(peer); } -static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +/* Hold the peer and its primary key for one encryption submission. + * The returned key and the peer each carry one reference which completion must + * release. + */ +static struct ovpn_crypto_key_slot * +ovpn_encrypt_refs_get(struct ovpn_peer *peer) { struct ovpn_crypto_key_slot *ks; /* get primary key to be used for encrypting data */ ks = ovpn_crypto_key_slot_primary(&peer->crypto); if (unlikely(!ks)) - return false; + return NULL; - /* take a reference to the peer because the crypto code may run async. - * ovpn_encrypt_post() will release it upon completion + /* the caller already owns a peer reference, so failure indicates a + * broken reference lifetime elsewhere */ if (unlikely(!ovpn_peer_hold(peer))) { DEBUG_NET_WARN_ON_ONCE(1); ovpn_crypto_key_slot_put(ks); - return false; + return NULL; } + return ks; +} + +static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct ovpn_crypto_key_slot *ks; + + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_encrypt_post(skb, ovpn_aead_encrypt(peer, ks, skb)); return true; } +static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, + struct ovpn_peer *peer, + struct sk_buff *batch, + unsigned int segments) +{ + unsigned int offset = 0, i, len; + struct ovpn_crypto_key_slot *ks; + struct sk_buff *skb; + + /* acquire all shared state before removing the first input skb so that + * failure can leave the queue intact for the ordinary transmit path + */ + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + + /* the aggregate owns these references until every sync or async crypto + * completion has finished + */ + memset(ovpn_skb_cb(batch), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(batch)->peer = peer; + ovpn_skb_cb(batch)->ks = ks; + atomic_set(&ovpn_skb_cb(batch)->batch_state.pending, segments); + atomic_set(&ovpn_skb_cb(batch)->batch_state.failed, 0); + + for (i = 0; i < segments; i++) { + skb = __skb_dequeue(skbs); + len = skb->len + OVPN_DATA_V2_OVERHEAD; + + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(skb)->batch = batch; + ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, + batch, offset)); + offset += len; + } + + return true; +} + +static struct sk_buff *ovpn_udp_gso_alloc(const struct sk_buff *first_segment, + unsigned int batch_len, + unsigned int segments) +{ + struct sk_buff *gso_skb; + int ret; + + gso_skb = alloc_skb_with_frags(OVPN_HEAD_ROOM, batch_len, + SKB_FRAG_PAGE_ORDER, &ret, GFP_ATOMIC); + if (unlikely(!gso_skb)) + return NULL; + + skb_reserve(gso_skb, OVPN_HEAD_ROOM); + gso_skb->len = batch_len; + gso_skb->data_len = batch_len; + gso_skb->priority = first_segment->priority; + + /* Segments retain the originating socket so we keep its send-buffer + * accounting active until the UDP GSO is transmitted. This also + * preserves its cached TX queue. + */ + if (first_segment->sk && is_skb_wmem(first_segment)) + skb_set_owner_w(gso_skb, first_segment->sk); + skb_copy_hash(gso_skb, first_segment); +#ifdef CONFIG_XPS + /* keep the aggregate on the TX queue selected for the original flow + * otherwise async crypto completion on another CPU could move the flow + * to a different queue and cause delay or reordering + */ + gso_skb->sender_cpu = first_segment->sender_cpu; +#endif + + skb_shinfo(gso_skb)->gso_type = SKB_GSO_UDP_L4; + skb_shinfo(gso_skb)->gso_size = first_segment->len + + OVPN_DATA_V2_OVERHEAD; + skb_shinfo(gso_skb)->gso_segs = segments; + + return gso_skb; +} + /* send skb to connected peer, if any */ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, struct ovpn_peer *peer) @@ -343,7 +474,7 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, */ skb_list_walk_safe(skb, curr, next) { if (unlikely(!ovpn_encrypt_one(peer, curr))) { - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); } } @@ -351,19 +482,96 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, ovpn_peer_put(peer); } +/* encrypt fixed-size input segments into one or more UDP GSO aggregates */ +static void ovpn_send_gso(struct sk_buff_head *skbs, struct ovpn_peer *peer) +{ + unsigned int max_segs = 1, seg_len, batch_len, segs; + struct sk_buff *batch; + + seg_len = skb_peek(skbs)->len + OVPN_DATA_V2_OVERHEAD; + max_segs = min_t(unsigned int, UDP_MAX_SEGMENTS, + OVPN_UDP_GSO_MAX_PAYLOAD / seg_len); + + /* if even two encrypted skbs cannot fit, leave the whole queue for the + * ordinary transmit path + */ + if (max_segs < 2) + return; + + /* An input GSO skb might be prduce more than max_segs segments so we + * consume as many as we can for each iteration. A final single skb, or + * the whole remainder after an allocation failure, stays queued and + * fallback to the ordinary transmit path. + */ + while (skb_queue_len(skbs) > 1) { + segs = min_t(unsigned int, skb_queue_len(skbs), max_segs); + + /* all but the final input skb have the same length, so start + * with the full-size calculation and adjust only the final + * group below + */ + batch_len = segs * (skb_peek(skbs)->len + + OVPN_DATA_V2_OVERHEAD); + if (segs == skb_queue_len(skbs)) + batch_len -= skb_peek(skbs)->len - + skb_peek_tail(skbs)->len; + + batch = ovpn_udp_gso_alloc(skb_peek(skbs), batch_len, segs); + if (unlikely(!batch)) + return; + + if (unlikely(!ovpn_encrypt_gso_queue(skbs, peer, + batch, segs))) { + kfree_skb(batch); + return; + } + } +} + +static bool ovpn_peer_supports_udp_gso(struct ovpn_peer *peer) +{ + struct ovpn_socket *sock; + bool udp_gso; + + rcu_read_lock(); + sock = rcu_dereference(peer->sock); + /* UDP GSO requires checksums. These socket settings can change after + * we decide to batch, but an already-built batch remains checksummed. + * Linux's ordinary UDP GSO path makes the same choice. + */ + udp_gso = sock && sock->sk->sk_protocol == IPPROTO_UDP && + !sock->sk->sk_no_check_tx && !udp_get_no_check6_tx(sock->sk); + rcu_read_unlock(); + + return udp_gso; +} + /* Send user data to the network */ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) { struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; + const bool gso_in = skb_is_gso(skb); struct sk_buff_head skb_list; netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; + bool gso_out; __be16 proto; int ret; + /* A frag-list GSO skb already stores complete segments as child skbs. + * Keep those children on the ordinary in-place encryption path instead + * of copying them into a replacement UDP GSO skb. + * + * GSO_BY_FRAGS input must also remain on that path because its variable + * segment sizes cannot be represented by one UDP GSO output size. + */ + gso_out = gso_in && + !(skb_shinfo(skb)->gso_type & SKB_GSO_FRAGLIST) && + skb_shinfo(skb)->gso_size != GSO_BY_FRAGS; + /* reset netfilter state */ nf_reset_ct(skb); @@ -392,13 +600,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) /* dst was needed for peer selection - it can now be dropped */ skb_dst_drop(skb); - if (skb_is_gso(skb)) { - /* force software segmentation, but keep ovpn's non-GSO feature - * bits so the generated segments can preserve non-linear skb - * data where possible + if (gso_in) { + /* force software segmentation into linear skbs and calculate + * each checksum while copying the segment */ features = netif_skb_features(skb); - segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + features &= ~(NETIF_F_GSO_MASK | NETIF_F_SG | + NETIF_F_CSUM_MASK); + segments = skb_gso_segment(skb, features); if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", @@ -420,7 +629,8 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) if (unlikely(!curr)) { net_err_ratelimited("%s: skb_share_check failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); + gso_out = false; continue; } @@ -429,8 +639,9 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_checksum_help(curr) < 0)) { net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); + gso_out = false; continue; } @@ -446,9 +657,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) ovpn_peer_put(peer); return NETDEV_TX_OK; } - skb_list.prev->next = NULL; ovpn_peer_stats_increment_tx(&peer->vpn_stats, tx_bytes); + + if (gso_out && skb_queue_len(&skb_list) > 1 && + ovpn_peer_supports_udp_gso(peer)) + ovpn_send_gso(&skb_list, peer); + + skb_list.prev->next = NULL; ovpn_send(ovpn, skb_list.next, peer); return NETDEV_TX_OK; @@ -456,7 +672,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) drop: ovpn_peer_put(peer); drop_no_peer: - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); skb_tx_error(skb); kfree_skb_list(skb); return NETDEV_TX_OK; diff --git a/drivers/net/ovpn/skb.h b/drivers/net/ovpn/skb.h index 4fb7ea025426..cca29479c038 100644 --- a/drivers/net/ovpn/skb.h +++ b/drivers/net/ovpn/skb.h @@ -10,6 +10,7 @@ #ifndef _NET_OVPN_SKB_H_ #define _NET_OVPN_SKB_H_ +#include #include #include #include @@ -20,19 +21,35 @@ /** * struct ovpn_cb - ovpn skb control block - * @peer: the peer this skb was received from/sent to - * @ks: the crypto key slot used to encrypt/decrypt this skb * @crypto_tmp: pointer to temporary memory used for crypto operations * containing the IV, the scatter gather list and the aead request + * @peer: peer used by this crypto operation or owned by this aggregate + * @ks: crypto key slot used by this operation or owned by this aggregate * @payload_offset: offset in the skb where the payload starts * @nosignal: whether this skb should be sent with the MSG_NOSIGNAL flag (TCP) + * @batch: UDP GSO aggregate receiving this input skb's encrypted payload + * @batch_state: completion state owned by a UDP GSO aggregate */ struct ovpn_cb { + void *crypto_tmp; struct ovpn_peer *peer; struct ovpn_crypto_key_slot *ks; - void *crypto_tmp; - unsigned int payload_offset; - bool nosignal; + + /* Ordinary encryption leaves this union zeroed. Decryption and TCP use + * their ordinary fields, a UDP GSO input stores its output aggregate, + * and that aggregate uses the same space to coordinate its completions. + */ + union { + struct { + unsigned int payload_offset; + bool nosignal; + }; + struct sk_buff *batch; + struct { + atomic_t pending; + atomic_t failed; + } batch_state; + }; }; static inline struct ovpn_cb *ovpn_skb_cb(struct sk_buff *skb) diff --git a/drivers/net/ovpn/stats.h b/drivers/net/ovpn/stats.h index 3a45b97c0056..b3fe006c01f6 100644 --- a/drivers/net/ovpn/stats.h +++ b/drivers/net/ovpn/stats.h @@ -40,16 +40,26 @@ static inline void ovpn_peer_stats_increment_rx(struct ovpn_peer_stats *stats, ovpn_peer_stats_increment(&stats->rx, n); } +static inline void ovpn_peer_stats_add_tx(struct ovpn_peer_stats *stats, + const unsigned int bytes, + unsigned int packets) +{ + atomic64_add(bytes, &stats->tx.bytes); + atomic64_add(packets, &stats->tx.packets); +} + static inline void ovpn_peer_stats_increment_tx(struct ovpn_peer_stats *stats, const unsigned int n) { - ovpn_peer_stats_increment(&stats->tx, n); + ovpn_peer_stats_add_tx(stats, n, 1); } -static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev) +static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev, + unsigned int packets) { local_bh_disable(); - dev_dstats_tx_dropped(dev); + while (packets--) + dev_dstats_tx_dropped(dev); local_bh_enable(); } diff --git a/drivers/net/ovpn/tcp.c b/drivers/net/ovpn/tcp.c index 8fe8a8e750a4..5cba35e4a8ee 100644 --- a/drivers/net/ovpn/tcp.c +++ b/drivers/net/ovpn/tcp.c @@ -332,7 +332,7 @@ static void ovpn_tcp_send_sock_skb(struct ovpn_peer *peer, struct sock *sk, ovpn_tcp_send_sock(peer, sk); if (peer->tcp.out_msg.skb) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); return; } @@ -354,7 +354,7 @@ void ovpn_tcp_send_skb(struct ovpn_peer *peer, struct sock *sk, if (sock_owned_by_user(sk)) { if (skb_queue_len(&peer->tcp.out_queue) >= READ_ONCE(net_hotdata.max_backlog)) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); goto unlock; } diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 7f69e8890b5b..4802d982de08 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -121,6 +121,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); + skb_mark_not_on_list(skb); ovpn_recv(peer, skb); return 0; @@ -196,9 +197,13 @@ static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind, dst_cache_set_ip4(cache, &rt->dst, fl.saddr); transmit: + /* an already-built UDP GSO needs a checksum seed even if the socket's + * no-check option changed while encryption was in flight + */ udp_tunnel_xmit_skb(rt, sk, skb, fl.saddr, fl.daddr, 0, ip4_dst_hoplimit(&rt->dst), 0, fl.fl4_sport, - fl.fl4_dport, false, sk->sk_no_check_tx, 0); + fl.fl4_dport, false, + !skb_is_gso(skb) && sk->sk_no_check_tx, 0); ret = 0; err: local_bh_enable(); @@ -271,9 +276,13 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind, * udp_tunnel_xmit_skb() */ skb->ignore_df = 1; + /* keep checksum offload enabled for an in-flight UDP GSO batch even if + * the socket's no-check option has changed since batch creation + */ udp_tunnel6_xmit_skb(dst, sk, skb, skb->dev, &fl.saddr, &fl.daddr, 0, ip6_dst_hoplimit(dst), 0, fl.fl6_sport, - fl.fl6_dport, udp_get_no_check6_tx(sk), 0); + fl.fl6_dport, + !skb_is_gso(skb) && udp_get_no_check6_tx(sk), 0); ret = 0; err: local_bh_enable(); @@ -344,8 +353,18 @@ void ovpn_udp_send_skb(struct ovpn_peer *peer, struct sock *sk, skb->dev = peer->ovpn->dev; skb->mark = READ_ONCE(sk->sk_mark); - /* no checksum performed at this layer */ - skb->ip_summed = CHECKSUM_NONE; + if (skb_is_gso(skb)) { + /* udp_tunnel_xmit_skb installs the outer UDP header after this + * function returns: point CHECKSUM_PARTIAL at that future + * header so both hw and sw UDP GSO can complete the checksum. + */ + skb->ip_summed = CHECKSUM_PARTIAL; + skb->csum_start = skb_headroom(skb) - sizeof(struct udphdr); + skb->csum_offset = offsetof(struct udphdr, check); + } else { + /* no checksum performed at this layer */ + skb->ip_summed = CHECKSUM_NONE; + } /* crypto layer -> transport (UDP) */ ret = ovpn_udp_output(peer, &peer->dst_cache, sk, skb); From patchwork Wed Sep 16 06:52:46 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5349 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853239mag; Tue, 15 Sep 2026 23:53:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBzGD8fxgk47gGiBlMypHlSELobXO7ilZ/FtEBi354CsbSYv7jNnIeclB3ou+xyNFWbbPNnum8O0g7M=@openvpn.net X-Received: by 2002:a05:6830:67d3:b0:804:b2ad:81c2 with SMTP id 46e09a7af769-80b28e56325mr1808084a34.0.1789541597755; Tue, 15 Sep 2026 23:53:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541597; cv=none; d=google.com; s=arc-20260327; b=aTH8Vy8J/GN24ESMbcUQld9Zso2/8f4k4v72xzhgYS0DFZi+11RA1Ke+BTVryfjAho DcgYBmy1QWwcDXXZlSYN5JlQysze6enY2+n3Z8vc0dzwtXrZajnYbBX8G5wghLoTmJ8g Xbl4jUYYAnP2IO/Kwd7nUTlOST8wzVsPkLCDvSxkBT6ZgRergM0wCw2F4vJgLGxrZtyc cdXIpqMgF4nY2rBJT6Yzat/MhvQWUQUqQFT2Mx1oG7TXRPgsNFHGcOr+ysbW1vcmMkHd mibalxBm7t+qC4mqBzcA+mo9SfnOVLv6PMzruD0rQJW+ahjE43NePkKgH8UHyWh6DXU9 wVHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=YDoVb8usf/pMXJ65JNNW7C4zjjBc9zcyVDVJSl2da3Q=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=mkRB10zjcW3nuNkbizuiiZotR92V9Vd/dhUSdROIbLPmwRhjWOw9YxBdRuMvUrIGcY 1CXSx2iLlHad5JGbshrmKhceWi7r4Mg16H0FrvcUmKUGvWE4DSTVaM2/TzjSYj9+6JwG 0PyH7H0dfHlmazIlqkOQTOCfgKNHbAldziSVo1VPZP8+NdkaV372TGiSh8Wx12hMjK3U xaCOu82De/bOezGYqR2L/RhWlfrkSd1wotpRrnk0YJSmpcRa/QK8xQpaiTbHd7AJlTs+ /Ht+p/mkuvp9nMREtVV29INrqNpuPDNOv5jeXZl7jccIo5qTg8uNRFbdtPt1hxshDkbS VEBQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=cMdVOIMO; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="hWzHvkt/"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=YcAleR7H; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=mXntJ4r2; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-80b084ea58bsi2477741a34.77.2026.09.15.23.53.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:17 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=cMdVOIMO; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="hWzHvkt/"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=YcAleR7H; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=mXntJ4r2; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YDoVb8usf/pMXJ65JNNW7C4zjjBc9zcyVDVJSl2da3Q=; b=cMdVOIMOk4fMQY1xOFlccGFEzq VS0/FTernFmp9FMVx+7CweH4CWycHGjCx/0sr/0tJVgamjtHAu/TknLi0JJkVQf4Otnii/wbaqfxS /z+MbPhEx/0ode5FZlZO3+UVtc9/b+thTcKhfU/hXvQ9rsVqMPNtGGUa+T89Zk5J2lmA=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW6-0008EQ-ER; Wed, 16 Sep 2026 06:53:14 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jW3-0008E0-2P for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=fEJRJ1ph8y1MZmPuHybv405vLE/5gaq2qjN7MYQuQsM=; b=hWzHvkt/oKL4l+Iw7Iu2Qc6BDm uWxbXW3dBb3TCDKPv0YRgadUzCWSUo5rt3iftqUiEoUkxwNcRRjyKCBSIe1JvVUVn0iUG6gCZO/+T 9O7NBFKO6D3khRxfywk6uluX8DVA80AJsatfzEsdYBVcbGymq1pFul5YJJWQWQi7s3vI=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=fEJRJ1ph8y1MZmPuHybv405vLE/5gaq2qjN7MYQuQsM=; b=YcAleR7HQ7FuoEVipBe5ZZzO0N mKaqzfyCEBp7W5PJqqafEdQMbvpL2jsXUc6ASz+UwsUXbIn2jEOdvm/sR5sSCkrQe2x1S840+9t8s GiWd5j9FnMUJGY/Qz9peHVKECVgXM4cs0VbGyogC348+VnzYpVL7M1N3AwMaI/fxeavE=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jW1-000674-7o for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:10 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:b231:465::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4hl8j84XLJz5wlM for ; Wed, 16 Sep 2026 08:53:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541580; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fEJRJ1ph8y1MZmPuHybv405vLE/5gaq2qjN7MYQuQsM=; b=mXntJ4r2MUqFONoc16e6lhjyBBxQMTLFu3oHrZoqsx5Lp305kCfgC20d159QeRP3+unEtZ sk1Cjned5jc1+RSEPGSLR0/KK0qHnJgwfDHlf19COgH6FtkbeKLO0Wp8IWzCtf4sAKMhU3 3Aw6HfajcBXaOoicSnIiB/fYkZlEN4ucY6c1+rsEZC00Yeg/47daATTwl2QsZRuypymvte r9eFK8i0PBbTLAA2i86e0AGN2FzJ7TUHTcQx7P0qtcNmlJVLSIeg2sKI5viphc/qFzWsG0 qv9ZlOcrSpG5Es3Jrpub2QLgZTInORj2mZDH8PzN31Y2iavINWv/IbIwXDhs0Q== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::1 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:46 +0200 Message-ID: <76cb811457ac17b519d219b13fb2a4317a6d5e74.1789540779.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hl8j84XLJz5wlM X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path ca [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1x6jW1-000674-7o Subject: [Openvpn-devel] [RFC ovpn net-next v2 5/9] ovpn: coalesce UDP data records with GRO X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470370713782263 X-GMAIL-MSGID: 1876470370713782263 Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path can detach and authenticate records independently. Match the complete opcode/key/peer header and require compatible outer- network and checksum state. Flush on short records, differing segment geometry, existing GSO input, or the 64-record limit. Export skb_gro_receive_list, which is already shared by the core UDP and TCP frag-list GRO paths, so modular ovpn can use the same primitive instead of maintaining a local copy. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following throughput: Forward Reverse Without receive GRO 18.308 Gbit/s 19.233 Gbit/s With frag-list GRO 22.087 Gbit/s 22.962 Gbit/s The preceding UDP GSO transmit path and hardware UDP segmentation were enabled in both cases. The equal-weight mean of the two directional results increased from 18.770 to 22.524 Gbit/s, a 20.0% improvement. Signed-off-by: Ralf Lici --- Changes since v1 https://lore.kernel.org/openvpn-devel/dd08a7a2fe0dfc88509115b5d7ee17825020c012.1789485693.git.ralf@mandelbit.com/ - Preserve the outer network offset in ovpn_udp_gro_receive_fraglist. (Sashiko) - Detach frag_list only from a UDP-tunnel GSO aggregate. (Sashiko) drivers/net/ovpn/io.c | 7 +- drivers/net/ovpn/udp.c | 181 ++++++++++++++++++++++++++++++++++++++++- net/core/gro.c | 1 + net/ipv4/udp_offload.c | 3 +- 4 files changed, 185 insertions(+), 7 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 3ad4cadeeb02..11f7f16d7b79 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -70,11 +70,10 @@ static void ovpn_netdev_write(struct ovpn_peer *peer, struct sk_buff *skb) unsigned int pkt_len; int ret; - /* - * GSO state from the transport layer is not valid for the tunnel/data - * path. Reset all GSO fields to prevent any further GSO processing - * from entering an inconsistent state. + /* the transport encapsulation and its GSO metadata do not describe the + * decrypted inner packet */ + skb->encapsulation = 0; skb_gso_reset(skb); /* we can't guarantee the packet wasn't corrupted before entering the diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 4802d982de08..adce9dd8629e 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -11,8 +11,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -27,6 +29,176 @@ #include "socket.h" #include "udp.h" +/* like UDP and TCP frag-list GRO */ +#define OVPN_UDP_GRO_CNT_MAX 64 + +static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) +{ + const unsigned int offset = skb_gro_offset(skb); + + /* GRO replaces its frag0 pointer after holding an skb, so keep the + * openvpn header linear for later candidate comparisons + */ + if (!pskb_may_pull(skb, offset + OVPN_OPCODE_SIZE)) + return false; + + *header = get_unaligned_be32(skb->data + offset); + return true; +} + +static struct sk_buff *ovpn_udp_gro_receive_fraglist(struct sock *sk, + struct list_head *head, + struct sk_buff *skb) +{ + const unsigned int gso_size = skb_gro_len(skb); + struct sk_buff *p, *pp = NULL; + u32 header, header2; + int ret = 0, nhoff; + bool flush; + + if (!ovpn_udp_gro_header(skb, &header) || + FIELD_GET(OVPN_OPCODE_PKTTYPE_MASK, header) != OVPN_DATA_V2) { + NAPI_GRO_CB(skb)->flush = 1; + return NULL; + } + + /* do not nest an existing GSO packet in the record list */ + if (skb_is_gso(skb)) { + NAPI_GRO_CB(skb)->flush = 1; + return NULL; + } + + list_for_each_entry(p, head, list) { + if (!NAPI_GRO_CB(p)->same_flow) + continue; + + /* match opcode, key ID and peer ID */ + if (!ovpn_udp_gro_header(p, &header2) || header != header2) { + NAPI_GRO_CB(p)->same_flow = 0; + continue; + } + + /* GRO has already matched the outer addresses and UDP ports; + * check the remaining outer IP fields + */ + nhoff = skb_transport_offset(p) - + NAPI_GRO_CB(p)->network_offset; + flush = __gro_receive_network_flush(udp_hdr(skb), udp_hdr(p), p, + nhoff, false); + + /* The first record determines the nominal GSO size. A shorter + * final record may follow it, but a larger record cannot. + * Checksum metadata must also be uniform because the aggregate + * exposes only one checksum state. + */ + if (gso_size > skb_shinfo(p)->gso_size || flush || + skb->ip_summed != p->ip_summed || + skb->csum_level != p->csum_level) { + pp = p; + } else { + /* skb_gro_receive_list pulls the headers already + * processed by GRO before linking this skb to the + * record list so we have to manually preserve the + * outer network header location for later handling + */ + nhoff = NAPI_GRO_CB(skb)->network_offset; + skb_set_network_header(skb, nhoff); + ret = skb_gro_receive_list(p, skb); + } + + /* complete the aggregate if the append failed, or after + * appending a shorter final record, or after reaching the + * record-count limit + */ + if (ret || gso_size != skb_shinfo(p)->gso_size || + NAPI_GRO_CB(p)->count >= OVPN_UDP_GRO_CNT_MAX) + pp = p; + + return pp; + } + + return NULL; +} + +static int ovpn_udp_gro_complete(struct sock *sk, struct sk_buff *skb, + int nhoff) +{ + /* udp_gro_complete has already marked this as a UDP tunnel GSO packet. + * Keep that type so UDP passes the aggregate directly to the encap cb, + * where the original record skbs are detached. + */ + skb_shinfo(skb)->gso_segs = NAPI_GRO_CB(skb)->count; + + /* Each outer UDP checksum was either validated (or accepted in case of + * checksumless UDP) before its record was merged in + * skb_gro_checksum_validate_zero_check. + * The checksum in the aggregate cannot describe the concatenation of + * independent UDP payloads, so we preserve the validation result. + */ + skb->ip_summed = CHECKSUM_UNNECESSARY; + skb->csum_level = 0; + skb->csum_valid = 0; + + return 0; +} + +/* skb_gro_receive_list keeps the first openvpn record in 'skb' and links the + * remaining records through frag_list. Here we segment by detaching that list + * before delivering the records individually, and remove the child skbs from + * the head skb's length and memory accounting so the head describes only the + * first record again. + */ +static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) +{ + struct sk_buff *curr, *list; + unsigned int data_len = 0, truesize = 0; + + /* IP reassembly may also use fraglist, but it is not a record batch */ + if (!skb_is_gso(skb) || + !(skb_shinfo(skb)->gso_type & + (SKB_GSO_UDP_TUNNEL | SKB_GSO_UDP_TUNNEL_CSUM))) + return NULL; + + list = skb_shinfo(skb)->frag_list; + if (unlikely(!list)) + return NULL; + + for (curr = list; curr; curr = curr->next) { + data_len += curr->len; + truesize += curr->truesize; + } + + skb_shinfo(skb)->frag_list = NULL; + skb->len -= data_len; + skb->data_len -= data_len; + skb->truesize -= truesize; + + return list; +} + +static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct sk_buff *next; + + skb->next = ovpn_udp_gro_detach(skb); + + skb_list_walk_safe(skb, skb, next) + { + skb_mark_not_on_list(skb); + + /* keep the current reference alive for the next record before + * handing this one to crypto + */ + if (next && unlikely(!ovpn_peer_hold(peer))) { + DEBUG_NET_WARN_ON_ONCE(1); + kfree_skb_list(next); + next = NULL; + } + + ovpn_recv(peer, skb); + } +} + /* Retrieve the corresponding ovpn object from a UDP socket * rcu_read_lock must be held on entry */ @@ -121,8 +293,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); - skb_mark_not_on_list(skb); - ovpn_recv(peer, skb); + ovpn_udp_recv(peer, skb); return 0; drop: @@ -408,6 +579,8 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock, .encap_type = UDP_ENCAP_OVPNINUDP, .encap_rcv = ovpn_udp_encap_recv, .encap_destroy = ovpn_udp_encap_destroy, + .gro_receive = ovpn_udp_gro_receive_fraglist, + .gro_complete = ovpn_udp_gro_complete, }; struct ovpn_socket *old_data; int ret; @@ -454,6 +627,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) { struct sock *sk = ovpn_sock->sk; + udp_tunnel_cleanup_gro(sk); + /* Re-enable multicast loopback */ inet_set_bit(MC_LOOP, sk); /* Disable CHECKSUM_UNNECESSARY to CHECKSUM_COMPLETE conversion */ @@ -462,6 +637,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) WRITE_ONCE(udp_sk(sk)->encap_type, 0); WRITE_ONCE(udp_sk(sk)->encap_rcv, NULL); WRITE_ONCE(udp_sk(sk)->encap_destroy, NULL); + WRITE_ONCE(udp_sk(sk)->gro_receive, NULL); + WRITE_ONCE(udp_sk(sk)->gro_complete, NULL); rcu_assign_sk_user_data(sk, NULL); } diff --git a/net/core/gro.c b/net/core/gro.c index 29b4d02bf519..b6acedc919f8 100644 --- a/net/core/gro.c +++ b/net/core/gro.c @@ -262,6 +262,7 @@ int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb) return 0; } +EXPORT_SYMBOL(skb_gro_receive_list); static void gro_complete(struct gro_node *gro, struct sk_buff *skb) { diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index cf07c3c6611a..187f108f3ee8 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -40,7 +40,8 @@ struct udp_tunnel_type_entry { #define UDP_MAX_TUNNEL_TYPES (IS_ENABLED(CONFIG_GENEVE) + \ IS_ENABLED(CONFIG_VXLAN) * 2 + \ IS_ENABLED(CONFIG_NET_FOU) * 2 + \ - IS_ENABLED(CONFIG_XFRM) * 2) + IS_ENABLED(CONFIG_XFRM) * 2 + \ + IS_ENABLED(CONFIG_OVPN)) DEFINE_STATIC_CALL(udp_tunnel_gro_rcv, dummy_gro_rcv); static DEFINE_STATIC_KEY_FALSE(udp_tunnel_static_call); From patchwork Wed Sep 16 06:52:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5351 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853250mag; Tue, 15 Sep 2026 23:53:18 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwTurSfWawg5zC9cDWbUi/S2WU2y5xrrByhGOKps9KzT064FjvFbcpIDqC2rPG5Ae9WyrElOdxV+tg=@openvpn.net X-Received: by 2002:a05:6808:1202:b0:4be:84c2:9836 with SMTP id 5614622812f47-4ca4ac0e54dmr2805705b6e.1.1789541598354; Tue, 15 Sep 2026 23:53:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541598; cv=none; d=google.com; s=arc-20260327; b=eMMnscpwVMCZkTdmQgWaWZgoOsNcPHJUrLtTjEeaeHOjAgDz5QN+XJdavwBB5Wglau PnR42f7TpHF45P8YDAnsgRfYD9wYp3NNPogB7+D4WGuVL7oKci1bfUkZkP845pIh2dWW dmyl2+j49+xD6BRdH9lbyrttIh/+k7UUMDFKOowrADiXD0v5x+38lbRAayKVVCYV4adu cdc/vKHTy9RHpOMAnc63R9hsRSiVRIoUwd7jusicmpLJTn2JkCBSM8wjhROPrAE1+keL VRO9MMMrSzOlEA8LDpaj0WSgxlaayhRcf8Cocso7NNrggfHWqs/hNEg3D5wlGy1Z+Loa zO9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=iei6ny9eGSMnCN+hQOO5H1GkevsRlg9PuuVOWUgcNQI=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=KjJKrWSBvoKJvMs+DrhJiR7yUaSeObGejN8C8jmZ+qvMEQa4Ea+qXbw/Zvn8xN7zOq 9Y9QozRrar2DN8an29cS0rIs9s/ehJmObPcnLoUUo4WKrAh3bCuXKSyzLdgEf2sVC9ZX 7fkLybzRlwpUCjKHuDbm/AhLFH50ih1D4Vyxr9ZxRXbh5uvP/X2QJZ88oyCjDCZz5+yH HhxSLYOR1PkRYNWFUR8rg1AIINsTMsfmtIJGiG6gSxzHhuO/LkEkJ5WoLEQa9NywzXM0 fXZzUkNoZ83u7LC8INE8AvNK8Uw78m/V8AlT5SBmBP1FfxHlKFrnlBDMleF0IuI9wr0m enng==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=TC5P41wl; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=frhoYRdE; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=N+vTwZSP; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=flk6s9Lj; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5614622812f47-4ca242e2bfdsi2528613b6e.27.2026.09.15.23.53.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:18 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=TC5P41wl; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=frhoYRdE; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=N+vTwZSP; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=flk6s9Lj; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=iei6ny9eGSMnCN+hQOO5H1GkevsRlg9PuuVOWUgcNQI=; b=TC5P41wlD+r4B1ah9dEWANBgmQ XXyb8g5TvlVdRHmm3ViTIjjqU9uhGOL+9dct6o4NVLe/nb/F+H4kIhfR23oxoTiGa+A76a40Ixr8E pRU7R+0GbcdRaZowI+jI4kKiWa9TKzVCm58McoL/DSjLsO4QdM9qGLPiHYTfyvqla26U=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW4-00042p-Ch; Wed, 16 Sep 2026 06:53:13 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jW2-00042V-DR for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=aU4jL3pubYAoO8V+Pm6EnvN0CS+ibHARwsKWzVcq5IU=; b=frhoYRdEIAehdVXl9HHpLIzJBz XXTxd8JHmfDF5HEwnsF5tn89kv0qNDOGpn2jtIh++vYciEm2Cn9UVCo8tSVEkwwiYicXpOcSPPP01 /DYJjNuGxQi4V2Pk4zRR1GC5Zuc9mVBmjAfmcssX/u1U6xJDsFyFtbmARApPkLVqVB7Y=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=aU4jL3pubYAoO8V+Pm6EnvN0CS+ibHARwsKWzVcq5IU=; b=N+vTwZSPcBWvf76h8Mky/y9vhV v+Uu3KqI1AlIgGkb7Lpfh4KaZ0W2ew7ABQKhFjN2mqASq9Ipk4rZY40UxST+KR2lSVuW/cKRX8tCC +cZ68Ny+6m1HYRkq0l9psYzkGGICWSTW+W/EZezPlY1dOsosib0luf+yJDBfxYpZxenU=; Received: from mout-b-210.mailbox.org ([195.10.208.40]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jW1-0001sh-Ke for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:11 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [IPv6:2001:67c:2050:b231:465::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-210.mailbox.org (Postfix) with ESMTPS id 4hl8j90zBVzFqwW for ; Wed, 16 Sep 2026 08:53:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541581; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aU4jL3pubYAoO8V+Pm6EnvN0CS+ibHARwsKWzVcq5IU=; b=flk6s9LjnGdNwiSQoZHSHxPEZrif3PMXOCEbDN9t45Wr9o0p8R1nqDpUuwzhslMJ6m6e8H rYLQotWI5G3PK0V/NGM9z8zJoHkXDLkmCW+4E+r6zaO40jlq1JacddAYZ4/d3SCQynEjbE qlyA2urd9Ugh+5IFL7+8aexPyw0JLH2E/Jup+yDvY93NJ9f3DQjZoOWwGiX73pgW04HSRa XuKLmGll5dTu1g7HHE+is9DpHGSChe8V+1VrPuY3/5hJksk+ArUMv9gbqIas9vGxVbmUkN RpLH7OepiUhH7OQX3v4GfQXI8ftPbL5G43Zt/6nJgFg/ekkOVMdVAlvygCWN7Q== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::1 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:47 +0200 Message-ID: <4d8357e320fc7c8fd4ab6e42a0bc38b275b1ff5d.1789540779.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hl8j90zBVzFqwW X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (a [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1x6jW1-0001sh-Ke Subject: [Openvpn-devel] [RFC ovpn net-next v2 6/9] ovpn: prefetch encrypted records before GRO batch decryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470371323254050 X-GMAIL-MSGID: 1876470371323254050 Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (and maintain the same distance throughout the batch), overlapping their memory access latency with the current AEAD operation. An ordinary single-record UDP receive has no later record and therefore skips the prefetch path. Only prefetch the linear part of each skb. Walking non-linear fragments here would duplicate the scatterlist walk performed by crypto and could cost more than the cache hint saves. A same-binary comparison using three 30-second samples per direction found distances one and two effectively tied forward, while distance two was 3.3% faster reverse and less variable in both directions. Profiling also measured slightly fewer decrypt cycles at distance two than at one, while wider distances provided no repeatable benefit. On a direct 100 Gbit/s ConnectX-5 link using one TCP stream, AES-128-GCM, a 1408-byte inner MTU and 8192-entry rings, five interleaved 60-second samples per direction increased throughput by 16.9% forward and 18.0% reverse. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/712708baf265c5509aed4f9d476abf514a242b8a.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/io.h | 14 ++++++++++++++ drivers/net/ovpn/udp.c | 21 ++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index 1a94f0fda1d1..49180214fe08 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -10,6 +10,9 @@ #ifndef _NET_OVPN_OVPN_H_ #define _NET_OVPN_OVPN_H_ +#include +#include + /* DATA_V2 header size with AEAD encryption */ #define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ @@ -21,6 +24,17 @@ #define OVPN_KEEPALIVE_SIZE 16 extern const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE]; +static inline void ovpn_skb_prefetchw(const struct sk_buff *skb) +{ + unsigned int offset; + + /* crypto overwrites data in place, so request write ownership of each + * linear cache line before the AEAD implementation reaches it + */ + for (offset = 0; offset < skb_headlen(skb); offset += L1_CACHE_BYTES) + prefetchw(skb->data + offset); +} + netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev); void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb); diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index adce9dd8629e..ca3344646a27 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -32,6 +32,9 @@ /* like UDP and TCP frag-list GRO */ #define OVPN_UDP_GRO_CNT_MAX 64 +/* leave enough work between a cache hint and the record which consumes it */ +#define OVPN_UDP_GRO_PREFETCH_DISTANCE 2 + static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) { const unsigned int offset = skb_gro_offset(skb); @@ -178,12 +181,28 @@ static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) { - struct sk_buff *next; + struct sk_buff *next, *prefetch; + unsigned int i; skb->next = ovpn_udp_gro_detach(skb); + /* a frag-list GRO aggregate makes later ciphertext visible before the + * current record is decrypted, so we prime the first two records, then + * keep the cache hints the same distance ahead while draining the list + */ + prefetch = skb->next ? skb : NULL; + for (i = 0; i < OVPN_UDP_GRO_PREFETCH_DISTANCE && prefetch; i++) { + ovpn_skb_prefetchw(prefetch); + prefetch = prefetch->next; + } + skb_list_walk_safe(skb, skb, next) { + if (prefetch) { + ovpn_skb_prefetchw(prefetch); + prefetch = prefetch->next; + } + skb_mark_not_on_list(skb); /* keep the current reference alive for the next record before From patchwork Wed Sep 16 06:52:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5348 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853236mag; Tue, 15 Sep 2026 23:53:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBzWDLEnVkAWOhWzsNWGGs0XJoPlcI7RcqGFM+8RPQrLzkiatJvOXl+BAvRJoxwqYzWLte0VM4rcKQI=@openvpn.net X-Received: by 2002:a05:6870:b013:b0:47b:cd37:f931 with SMTP id 586e51a60fabf-484771aacfdmr2336874fac.13.1789541597700; Tue, 15 Sep 2026 23:53:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541597; cv=none; d=google.com; s=arc-20260327; b=f+ZTK5aSe7YCgPh7Ksn/NGzufs/rLM1+4eJPz838+C/SWQrIhrEkWLY3/uZCT63bOK g5+iMzox6Kzopna3gwUnsokSZl69QgEsnAdSZl4SiU0ZfmTlaqhlJM5eX/kcS4wmMbcP sOeTr1FftJR4kS47RfqLwz7whKP/tW50JD5BvLoNdKR4S025MQD0bmUQGWD1rG9Kr9uT yvnjFipdu5BPmOMuhJYBp1uxxyjmM6+5HsfLGuy456cTmTyTRHTD0oXPdsIz0feDUrJd OAtXq2NNfUBpckR5xza9rO58R5H7ZLIYEUTOjwsuhn9p65njxosjLG3sc53GiH+/1IDQ 8X6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=b3dnzvtKs4Rh3Dy2ph/so0rk2itzZbhEer1AC5qZ7cw=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=N50Bl5p7gJG5u+hswkJjH0EE43W7csws6g8ITTSvnQ+A+6GuchBVi3kGjEX4S7ZIAT 9nPDqav7Mx4yXHF2ptJ51YMgZ92p/5aQ7YtyRKyXDX5GIVWk7X25K1B7HG8+rGoEn+oL IRYI2eVxrlhdXcHzNcHrXQUciS24HyZOR8OecQCvs6ado18Iw4iCnKAumbAQLY9SCORZ lo+FuS+VX1RGdNMVz3QCEJ3VoqXMODxqNXjAl6aRyi/+ZOS0gfg3NseE9L03SKK+ccNR 0ALwF2BSns6jQZ+wwGbdJZOXpizg3CRVZDGfuS/96StVi8Q2uOMIvK99ldFGUzwQtikc DaXA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=VVoD7Qdm; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=b4bPXity; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=CdVUguSc; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=0sCBNy4D; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-4842aac077dsi2234713fac.323.2026.09.15.23.53.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:17 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=VVoD7Qdm; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=b4bPXity; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=CdVUguSc; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=0sCBNy4D; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=b3dnzvtKs4Rh3Dy2ph/so0rk2itzZbhEer1AC5qZ7cw=; b=VVoD7QdmZ6rhGKNdf84rdwcEOa o7bhbO1g5WNVhteHIxulOzDYUe9rqwrNPaAzQASIM8YZrLa7jFSvvnfWutp9EBURGFZxhA67/bJUk w1L44/4ZQVAUozCxB3dftT7hYINuq/Z7e9v2m9qDMzG/u7Kc9sVoM5E9M1zzFVIL+Vcc=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW7-00051S-3h; Wed, 16 Sep 2026 06:53:12 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jW5-000519-Hj for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=lROSELap/mVvudmaNb+lHtUVa5vNhgpljj95LyEOrIk=; b=b4bPXityjPsvfuvwbpNZqeqera 4Nfk5wfDp+uOkwAk7zAhZhYqmIZ+D1U+G/Zyir2WqqBxIJ6U5q5eFlIYkQ5KyUMhYeVCEQ/gCme0n kW9fbjvFsFgpfooZr/VK13udu01I7aX5xLsiT9Un7+Pp8rsVd2NzKlA5ofnhAt8JbYDQ=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=lROSELap/mVvudmaNb+lHtUVa5vNhgpljj95LyEOrIk=; b=CdVUguSczsXGFkyGy4miH6JvIv EuISaZWjd60TFJe39sRsIq+Y+/eribj/dOhujgYPrlDWH86FymKB8B9h49yx5rnC/r1nNiMHImrJf /ASZ89cNADsE2toL3f7LA7hvDMtGQ3f1vGgD33rzSl+NEuAdOwq3O24cj4iVglUuYRP8=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jW1-000675-BV for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:10 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [10.196.197.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4hl8j95F2Nz5x0N for ; Wed, 16 Sep 2026 08:53:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541581; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lROSELap/mVvudmaNb+lHtUVa5vNhgpljj95LyEOrIk=; b=0sCBNy4DUlkPliCiZ7tTowTyS8XHqiwbvlqhdeWOov4SPWwQeMPpQjceZ5aoO7eNaPQopn BaWXwP1tbPu32jr1tyyXuY1VeIeeh7lSRku61fu1WFf2rrozIXEL3a8aprsD9PUUSOeKoN 74MBp+gTrLx7eQV73vL6eHLzmgcYdL5iZ2L9TSQJNFBoB/uHdTZNm0xq5i8vTVFUnf9C5t 1qatRmFkP1mE/el8vZ6GrRUm/O7IAKEJDuCIDQqdzIT6qThmgat1zv+4wlmgjGk73jV1x6 iRcwxsf8etCv4/1/y4ag1d1agaZRDi/yHuoxwvNhnN0Bvyd/UTXSqRixnQoHZQ== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:48 +0200 Message-ID: <3f903692549ee58887b1bb421f8b283059f66a1d.1789540779.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Software segmentation exposes the complete skb list before encryption begins. While encrypting each segment, request write ownership of the next linear segment cache line by cache line. A single skb n [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1x6jW1-000675-BV Subject: [Openvpn-devel] [RFC ovpn net-next v2 7/9] ovpn: prefetch GSO segments before in-place encryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470370224944013 X-GMAIL-MSGID: 1876470370224944013 Software segmentation exposes the complete skb list before encryption begins. While encrypting each segment, request write ownership of the next linear segment cache line by cache line. A single skb naturally skips the prefetch path. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/382879c187b1acca65198c467d1263266243ea50.1789485693.git.ralf@mandelbit.com/ drivers/net/ovpn/io.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 11f7f16d7b79..cb7503a3e79c 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -472,6 +472,12 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, * independently */ skb_list_walk_safe(skb, curr, next) { + /* encrypting this segment can hide the cost of fetching the + * next segment's data into the cache + */ + if (next) + ovpn_skb_prefetchw(next); + if (unlikely(!ovpn_encrypt_one(peer, curr))) { ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); From patchwork Wed Sep 16 06:52:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5353 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853273mag; Tue, 15 Sep 2026 23:53:19 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwcSZ40mHa//0Ak/rrUii5wuxbrGn8Iz4rePoZRpKQ/oeVvF/VuiFQ5aaxu/ZZ+xNmjDTEIPogG67M=@openvpn.net X-Received: by 2002:a05:6808:1903:b0:4b9:a88b:8887 with SMTP id 5614622812f47-4ca4c48f8bamr1585273b6e.29.1789541599315; Tue, 15 Sep 2026 23:53:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541599; cv=none; d=google.com; s=arc-20260327; b=OvwtbZLrzJAVQlgGiOkaw2DDcmocstdUNOUlZn8upfZJ66UOAAhaV5O2Sms3wpZPzv xi75q3UALQYMDwInYrenlBnP14vgb2qkvNKfkdlPcCvpEAHhadFP9by8FDne7HPdbqG1 /eb/Ye0W2AVMDPxy28DUDpiikRIaFb1JYNdZe5QLJKTFAoYKgwO1FuHPhlsrg6XmLWRA /ZWp7FtmosEG/aQ1aKtrNXpZ1EsDfm+k3vcJpSgr4LOCg7mnrJGY8jikifsIR6HVPHK7 eDLob+llkpKCMoTCGQMU6QQMnDpQ/3WIWxkJj++EOhZk5mjsyi2zCL0sw9lWioJoKrjC ZGGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=ike8dvFlCoCs4CZGHyKew6Gyk/IvuxpRZmP1jz3tlZY=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=TUVxxRCIinRd1i1s9cN2+VF4+G/A+eN32YfBgnqR4hsHHGkVwZOhNA7ztN1+9+D4P1 3M5yVNGpLw6K+S7ov8NL2SVx7J8zrsOeWo+hWdftmWBXygyST8hZjTUph3PLLNZ2voM4 ub/hqfAh1W3y3V6pZyYPs8dHPkJCP0didzl2XwZICntf0Puo4abMoArtAhXfdahg9fji SBzC8er8xJ2N7Bc+KhwPI8UEqwiCSU+ew6sICilcLosiOZv4abnXSVDWhy49ZrYk69sc f1SuO3gYPdRuufftgbP+JkCgTc6uklJ4Ox6Jx4GGY18QI70zo97wm0nHpm/mEbxvnEqh TJvw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=S9n3J8ky; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=dTorj9e5; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Y5yFJkHI; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=J+mDtb+u; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5614622812f47-4ca2615b876si2485352b6e.130.2026.09.15.23.53.19 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:19 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=S9n3J8ky; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=dTorj9e5; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Y5yFJkHI; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=J+mDtb+u; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ike8dvFlCoCs4CZGHyKew6Gyk/IvuxpRZmP1jz3tlZY=; b=S9n3J8kypW9ZsIGCKkQL9FWLUk YyTm7spOXlEeBiT10XcWX+ptkl4xr/xmNqab8EB92bO+5XBCw22giJ+bivaGcBvw4N6PjGwuNuHsY xDULlQkdTw7OrW5k2dRdgT4MU8IL7svMTgN82ubEviyN+N7xtyd99J9fAoXychUI+wMI=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW4-0001Ic-FL; Wed, 16 Sep 2026 06:53:13 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jW2-0001IF-Ml for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:11 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=CITsaCk4S8DnpvDeecUNjmmT1RHQUsnQPp+g5qBuw6g=; b=dTorj9e5kZUEQE6SOK9ticm3FO q31jKpDirq9r2TDhdbjN2GhcTAfQF2tvYtvfZyJwL+/NignmnFJzs6mxymJqDl5EMFnEN7bvR8XSb ZBQtlVxlCI47JxMG5atPF4taCKSMNUyNLll8S46AcEchuptTxUv2mEf8nqsJ2SNzPVeA=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=CITsaCk4S8DnpvDeecUNjmmT1RHQUsnQPp+g5qBuw6g=; b=Y5yFJkHIaKG1XFjlxbR3cKTXC/ XtlOmIY6enun+xQ4u8BmVdF5J6gAgzZjyhcTknYkDdZr0XnOUSPu3oP5YBMs3fK9pIbLRBc0jgM/c 3iD4d0xfN4EwGl7qgYvWXu75kxLY5xciEs1CjPzlnqrKg4+gaCe2aT0PQYUM0LJYrx8A=; Received: from mout-b-105.mailbox.org ([195.10.208.50]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jW2-0001sj-4w for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:11 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [10.196.197.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-105.mailbox.org (Postfix) with ESMTPS id 4hl8jB2C0cz9tLr for ; Wed, 16 Sep 2026 08:53:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541582; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CITsaCk4S8DnpvDeecUNjmmT1RHQUsnQPp+g5qBuw6g=; b=J+mDtb+uOSa01iDQlhLqH9nWNLjNkp/nhLEo8Tw4a/tTFDjyO2D6hlyLo7hi5jSNU9DEo4 cr3xQeu4bOFCcuXcs9tuWgnF1yhaE/ynciMdSQkJnBc1XdrJQ4frGZ+FSADFJJb4RkGKMe m+2zNlH5qzYrBeLrKR8CfVCkxbTx2wcV9mezifk4Yp3y4aj4oBpy/OZ59VER3iWp6seoU1 pMwv9NvovRb82IzU/2hydY+GI88XfbO8u/YGPH3BRCUS8kTmxl44U0jZRYu+fpAOFkefyY hW4x67jJBb9mYlEVCLly1Of4vbiXG39rOSAZ9IciE8xsOD7kj/m6NFuq2FP1XQ== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:49 +0200 Message-ID: <3192bff7d69f958114e5fc9efe0dc5039c5be147.1789540779.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: XFRM's ESP GRO callbacks may consume an skb and return ERR_PTR(-EINPROGRESS) as an ownership marker. dev_gro_receive already recognizes this marker unconditionally and converts it to GRO_CONSUMED, so [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1x6jW2-0001sj-4w Subject: [Openvpn-devel] [RFC ovpn net-next v2 8/9] net: gro: honor skbs consumed by protocol callbacks X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470372063788872 X-GMAIL-MSGID: 1876470372063788872 XFRM's ESP GRO callbacks may consume an skb and return ERR_PTR(-EINPROGRESS) as an ownership marker. dev_gro_receive already recognizes this marker unconditionally and converts it to GRO_CONSUMED, so -EINPROGRESS is reserved by the generic GRO callback interface and cannot represent an ordinary callback error. The nested flush helpers currently avoid accessing a consumed skb only when XFRM offload is configured, because XFRM has so far been the sole user of the convention. Make the ownership check unconditional so other protocol callbacks can safely use the existing marker without acquiring an unrelated CONFIG_XFRM_OFFLOAD dependency. Callbacks which do not return the marker are unaffected. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/893f6c2b385f9df3e9617a9b7f547734061df195.1789485693.git.ralf@mandelbit.com/ include/net/gro.h | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/include/net/gro.h b/include/net/gro.h index 2300b6da05b2..20ddc5488789 100644 --- a/include/net/gro.h +++ b/include/net/gro.h @@ -361,9 +361,11 @@ static inline void skb_gro_remcsum_cleanup(struct sk_buff *skb, remcsum_unadjust((__sum16 *)ptr, grc->delta); } -#ifdef CONFIG_XFRM_OFFLOAD static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush) { + /* a GRO callback may consume skb and return this marker to prevent + * accessing the skb while unwinding through the enclosing GRO layers + */ if (PTR_ERR(pp) != -EINPROGRESS) NAPI_GRO_CB(skb)->flush |= flush; } @@ -378,21 +380,6 @@ static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb, skb->remcsum_offload = 0; } } -#else -static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush) -{ - NAPI_GRO_CB(skb)->flush |= flush; -} -static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb, - struct sk_buff *pp, - int flush, - struct gro_remcsum *grc) -{ - NAPI_GRO_CB(skb)->flush |= flush; - skb_gro_remcsum_cleanup(skb, grc); - skb->remcsum_offload = 0; -} -#endif INDIRECT_CALLABLE_DECLARE(struct sk_buff *ipv6_gro_receive(struct list_head *, struct sk_buff *)); From patchwork Wed Sep 16 06:52:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5352 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5853246mag; Tue, 15 Sep 2026 23:53:18 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvByVLK2x2nXZLV3uo+KOFlNsIeX6gMDMM0PXDgIESW9KlWx8t9ul37jMrCBfYgii1yzynYjmH8cOE3o=@openvpn.net X-Received: by 2002:a05:6830:6f41:b0:807:70aa:b2ac with SMTP id 46e09a7af769-80b2e052eccmr1619235a34.22.1789541598119; Tue, 15 Sep 2026 23:53:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789541598; cv=none; d=google.com; s=arc-20260327; b=T1Rg8Zuu6dqKb1PGZGaXZ0TcWbiuuvhg3yfjthCvaRhmwzsmc+gXeKcbvbwNgvaAuk KZVjz76OtmeG9fjkXg4KciEJzLWA8GKzN0/oiHMlJnWIHt4KJpXhW7UgD3tEfUUSQ6P4 M8IDCjFER0n1Xe+U5gAhf2aaBPqQYZXsrOB5Hu4Qs5+TpvqC/dPQSR2ep4rFpAtsB2ob +EPJz71b/DGI0dBzrJkmLRA4A9BwyFrcz9lJX4Eo0Uva70f4L6e6QSYqQtu+RLoDfxuB nfhJP23BwsfBX72+3YQv2MA9ZHv0LJWoHEdJREiVfquDCQNFEl8pv5wsQ/ilzQPu0d7O yzVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=svC1muhATeIxXiEXlMqWcSOXT91jV0MAGHaOPLvaumg=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=nhWtajgE5UHpJitzoAfL5FOSKkdsj4iWfflJSsL+4ANJUvomMeu/ofd4Fi5tSTQVf5 Ll6pYYqcEqo8maimHAvFRXq81fWCwjXd/PUVFJDvCEAa9bDVhv5d5qkO2VjDBGtNVGu1 802JrdM1Vgnz8wPD68twZLExts9Ts40N+2BYj0ENps288oJnxyRXPtzLaoYP+7wIegLM vQ8wsYLjD2DYBCKKfAhSVoDnCV2v3DWZZC/eLL+afKut1+llA6cIYwNLlBm5EG7RRzol Pxyn0oMdW/Tq5BJjsMJaBwNVH4uM1/SNBG66Stiv53RRF5ZZIIVzG22p+sQVnij/H0LT g6Ag==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ISCfynEp; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=AVOXdmd4; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Bb3hLXZ0; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Km1M+01I; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-80b09022482si2450814a34.138.2026.09.15.23.53.17 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 23:53:18 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ISCfynEp; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=AVOXdmd4; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Bb3hLXZ0; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Km1M+01I; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=svC1muhATeIxXiEXlMqWcSOXT91jV0MAGHaOPLvaumg=; b=ISCfynEpoXLi6b2TB24r5Ua7e5 5xRWT/vF+jGHXAymtX5tpJnHrLXcki9Yx5v5+ulXr04J7276FyysLtB52aU4Rxl765KjPgxHXqzzM 4ismwZS/wvOBn6fbXDA7wWT1+bY2qtVp8AmDS1Dv2GdgeA2FmAnm1ElLuA9Dw+WC7duQ=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6jW7-0008Eg-2E; Wed, 16 Sep 2026 06:53:15 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6jW5-0008EB-TZ for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=uq29HG6oEnQiJD89rFVOT4w98lRfcEBCfZKBsB2SKJg=; b=AVOXdmd4R0JynOOxwi0rBepbX7 lRXY01SEiXR2AduA3iv8aZjQoij0U1Zf/7dLcnmjKFE37+OuExFs2q8xzlnapegK6bFRxm1EAND4n Ll3QE8ZZtsyrsFJP2XSn/CoHMdDk/0/C/nGCmLAH0Ffe6I8TzCKoILAwwOvb0C4jqo9Q=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=uq29HG6oEnQiJD89rFVOT4w98lRfcEBCfZKBsB2SKJg=; b=Bb3hLXZ0mfU3g4SQFHRYaRpuuA DOhIkH0xaNCwlSovS/zMtlo+Qv/P6IbKrnlT+QuxDW+Xhi8Z0Xqiz1RPK7Qi+l4PDrazuRiI71a07 6ux4lWTR8T1mqNrfvJQBsE46io7Aj4ExUcIpO2OCC1uMJ/VVM9mjEQ+JVrnKINwdi1VI=; Received: from mout-b-107.mailbox.org ([195.10.208.47]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6jW3-000677-72 for openvpn-devel@lists.sourceforge.net; Wed, 16 Sep 2026 06:53:13 +0000 Received: from smtp1.mailbox.org (smtp1.mailbox.org [10.196.197.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-107.mailbox.org (Postfix) with ESMTPS id 4hl8jB5xJdz3yFC for ; Wed, 16 Sep 2026 08:53:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789541582; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uq29HG6oEnQiJD89rFVOT4w98lRfcEBCfZKBsB2SKJg=; b=Km1M+01IY7g/Pt+q2ab0obQ/zBAmeFsT8oBReKzNwsisoK1SqEjNOpbvoNRr1l0YUZJdae Dsr4QbpzfOEGMY3zKEgHjn2NQYm99zA4wlcA6kURT7hi6KIaTio2b0wuCWtXqeXeQH9FpW p8CvzYjz4i+WaBegJmFb5ndQdjtsONKu0e/aUtxhobr3icK4K8JtLKcETpZZN2VHboO62a yHR4WPpXjBi9/LPZdnE2nHjIxgDQs59ypn1szeVPmgbsl/VXuAv2XvXf1nLSDkxPR9Llw0 3z/Rzgr16m+blQrGDOaH/DbPePM8lBWTcWBxQJXpOHCGoKjgRUK1yoHwu5uwJA== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Wed, 16 Sep 2026 08:52:50 +0200 Message-ID: <69c873c8837a0d8028c0427509aa384ee73be02b.1789540779.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: The normal ovpn UDP GRO path retains compatible encrypted records in a frag-list and lets the completed aggregate traverse the outer IP and UDP receive stack before handing its records to the existing [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1x6jW3-000677-72 Subject: [Openvpn-devel] [RFC ovpn net-next v2 9/9] ovpn: add opt-in direct GRO receive mode X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876470370526633043 X-GMAIL-MSGID: 1876470370526633043 The normal ovpn UDP GRO path retains compatible encrypted records in a frag-list and lets the completed aggregate traverse the outer IP and UDP receive stack before handing its records to the existing decryption path. Add an optional UDP GRO mode which instead consumes DATA_V2 packets from the UDP tunnel GRO callback and starts their existing per-record decryption immediately, similarly to xfrm. Control packets are flushed from GRO, restored, and continue through the normal stack. Select the mode through the immutable IFLA_OVPN_UDP_GRO_MODE link attribute. FULL_STACK remains the default for existing userspace, while DIRECT enables the new path. Keep this local receive policy per ovpn interface and select the corresponding GRO callback when each UDP socket is attached. This makes all sockets attached to the interface behave consistently without performing a mode lookup and dispatch for every packet. Account for both ovpn callbacks in the UDP tunnel GRO callback limit. The direct mode deliberately bypasses packet taps, TC ingress, outer IP validation and routing, netfilter hooks, the final UDP lookup, socket XFRM policy, and normal UDP receive accounting for DATA_V2. It is therefore an explicit operator choice for controlled transport interfaces rather than a transparent replacement for the full receive stack. Extend the UDP throughput selftest with a PRE_ROUTING nftables counter, verifying that full-stack aggregates reach the hook while direct-GRO aggregates bypass it. Before ciphertext prefetch was added to FULL_STACK, five interleaved single-flow AES-128-GCM runs per direction on two directly connected 100-Gbit/s mlx5 ports measured 22.087/22.962 Gbit/s forward/reverse in FULL_STACK and 24.315/25.313 Gbit/s in DIRECT, a 10.2% equal-weight gain. With the preceding prefetch commit enabled in FULL_STACK, later checks measured DIRECT within 1.3% forward and 0.7% reverse of FULL_STACK. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/b4bd25d6c4c01a81447a31054abd095e0533fcdd.1789485693.git.ralf@mandelbit.com/ Documentation/netlink/specs/rt-link.yaml | 12 +++ drivers/net/ovpn/main.c | 15 ++- drivers/net/ovpn/ovpnpriv.h | 2 + drivers/net/ovpn/udp.c | 98 ++++++++++++++----- include/uapi/linux/if_link.h | 6 ++ net/ipv4/udp_offload.c | 2 +- tools/testing/selftests/net/ovpn/Makefile | 1 + tools/testing/selftests/net/ovpn/common.sh | 4 +- tools/testing/selftests/net/ovpn/ovpn-cli.c | 38 ++++++- .../selftests/net/ovpn/test-gro-direct.sh | 10 ++ tools/testing/selftests/net/ovpn/test.sh | 65 ++++++++++++ 11 files changed, 219 insertions(+), 34 deletions(-) create mode 100755 tools/testing/selftests/net/ovpn/test-gro-direct.sh diff --git a/Documentation/netlink/specs/rt-link.yaml b/Documentation/netlink/specs/rt-link.yaml index 7a72cd1b7e1e..d43e995c7f09 100644 --- a/Documentation/netlink/specs/rt-link.yaml +++ b/Documentation/netlink/specs/rt-link.yaml @@ -844,6 +844,14 @@ definitions: entries: - p2p - mp + - + name: ovpn-udp-gro-mode + enum-name: ovpn-udp-gro-mode + name-prefix: ovpn-udp-gro-mode + type: enum + entries: + - full-stack + - direct - name: br-stp-mode type: enum @@ -2365,6 +2373,10 @@ attribute-sets: name: mode type: u8 enum: ovpn-mode + - + name: udp-gro-mode + type: u8 + enum: ovpn-udp-gro-mode sub-messages: - diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index ac4e0d85e215..ecf27d1e2420 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -129,6 +129,9 @@ static const struct device_type ovpn_type = { static const struct nla_policy ovpn_policy[IFLA_OVPN_MAX + 1] = { [IFLA_OVPN_MODE] = NLA_POLICY_RANGE(NLA_U8, OVPN_MODE_P2P, OVPN_MODE_MP), + [IFLA_OVPN_UDP_GRO_MODE] = + NLA_POLICY_RANGE(NLA_U8, OVPN_UDP_GRO_MODE_FULL_STACK, + OVPN_UDP_GRO_MODE_DIRECT), }; /** @@ -200,6 +203,7 @@ static int ovpn_newlink(struct net_device *dev, struct rtnl_newlink_params *params, struct netlink_ext_ack *extack) { + enum ovpn_udp_gro_mode gro_mode = OVPN_UDP_GRO_MODE_FULL_STACK; struct ovpn_priv *ovpn = netdev_priv(dev); struct nlattr **data = params->data; enum ovpn_mode mode = OVPN_MODE_P2P; @@ -209,9 +213,14 @@ static int ovpn_newlink(struct net_device *dev, mode = nla_get_u8(data[IFLA_OVPN_MODE]); netdev_dbg(dev, "setting device mode: %u\n", mode); } + if (data && data[IFLA_OVPN_UDP_GRO_MODE]) { + gro_mode = nla_get_u8(data[IFLA_OVPN_UDP_GRO_MODE]); + netdev_dbg(dev, "setting UDP GRO mode: %u\n", gro_mode); + } ovpn->dev = dev; ovpn->mode = mode; + ovpn->gro_mode = gro_mode; spin_lock_init(&ovpn->lock); INIT_DELAYED_WORK(&ovpn->keepalive_work, ovpn_peer_keepalive_work); @@ -237,8 +246,8 @@ static int ovpn_newlink(struct net_device *dev, static size_t ovpn_get_size(const struct net_device *dev) { - /* IFLA_OVPN_MODE */ - return nla_total_size(sizeof(u8)); + /* IFLA_OVPN_MODE and IFLA_OVPN_UDP_GRO_MODE */ + return nla_total_size(sizeof(u8)) + nla_total_size(sizeof(u8)); } static int ovpn_fill_info(struct sk_buff *skb, const struct net_device *dev) @@ -247,6 +256,8 @@ static int ovpn_fill_info(struct sk_buff *skb, const struct net_device *dev) if (nla_put_u8(skb, IFLA_OVPN_MODE, ovpn->mode)) return -EMSGSIZE; + if (nla_put_u8(skb, IFLA_OVPN_UDP_GRO_MODE, ovpn->gro_mode)) + return -EMSGSIZE; return 0; } diff --git a/drivers/net/ovpn/ovpnpriv.h b/drivers/net/ovpn/ovpnpriv.h index 84499140e4bd..dc6210b99f4a 100644 --- a/drivers/net/ovpn/ovpnpriv.h +++ b/drivers/net/ovpn/ovpnpriv.h @@ -40,6 +40,7 @@ struct ovpn_peer_collection { * struct ovpn_priv - per ovpn interface state * @dev: the actual netdev representing the tunnel * @mode: device operation mode (i.e. p2p, mp, ..) + * @gro_mode: whether UDP data follows the full stack or is decrypted from GRO * @lock: protect this object * @peers: data structures holding multi-peer references * @peer: in P2P mode, this is the only remote peer @@ -49,6 +50,7 @@ struct ovpn_peer_collection { struct ovpn_priv { struct net_device *dev; enum ovpn_mode mode; + enum ovpn_udp_gro_mode gro_mode; spinlock_t lock; /* protect writing to the ovpn_priv object */ struct ovpn_peer_collection *peers; struct ovpn_peer __rcu *peer; diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index ca3344646a27..fcdbb0dea284 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -239,23 +239,12 @@ static struct ovpn_socket *ovpn_socket_from_udp_sock(struct sock *sk) return ovpn_sock; } -/** - * ovpn_udp_encap_recv - Start processing a received UDP packet. - * @sk: socket over which the packet was received - * @skb: the received packet - * - * If the first byte of the payload is: - * - DATA_V2 the packet is accepted for further processing, - * - DATA_V1 the packet is dropped as not supported, - * - anything else the packet is forwarded to the UDP stack for - * delivery to user space. - * - * Return: - * 0 if skb was consumed or dropped - * >0 if skb should be passed up to userspace as UDP (packet not consumed) - * <0 if skb should be resubmitted as proto -N (packet not consumed) +/* Process one packet after the caller has made its OpenVPN header visible at + * @payload_offset. A zero return means the skb was consumed. A positive return + * leaves a control packet for the UDP socket. */ -static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) +static int ovpn_udp_data_recv(struct sock *sk, struct sk_buff *skb, + unsigned int payload_offset) { struct ovpn_socket *ovpn_sock; struct ovpn_priv *ovpn; @@ -276,18 +265,16 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) goto drop_noovpn; } - /* Make sure the first 4 bytes of the skb data buffer after the UDP - * header are accessible. + /* Make sure the first 4 bytes of the OpenVPN header are accessible. * They are required to fetch the OP code, the key ID and the peer ID. */ - if (unlikely(!pskb_may_pull(skb, sizeof(struct udphdr) + - OVPN_OPCODE_SIZE))) { + if (unlikely(!pskb_may_pull(skb, payload_offset + OVPN_OPCODE_SIZE))) { net_dbg_ratelimited("%s: packet too small from UDP socket\n", netdev_name(ovpn->dev)); goto drop; } - opcode = ovpn_opcode_from_skb(skb, sizeof(struct udphdr)); + opcode = ovpn_opcode_from_skb(skb, payload_offset); if (unlikely(opcode != OVPN_DATA_V2)) { /* DATA_V1 is not supported */ if (opcode == OVPN_DATA_V1) @@ -297,7 +284,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) return 1; } - peer_id = ovpn_peer_id_from_skb(skb, sizeof(struct udphdr)); + peer_id = ovpn_peer_id_from_skb(skb, payload_offset); /* some OpenVPN server implementations send data packets with the * peer-id set to UNDEF. In this case we skip the peer lookup by peer-id * and we try with the transport address @@ -310,8 +297,10 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) if (unlikely(!peer)) goto drop; - /* pop off outer UDP header */ - __skb_pull(skb, sizeof(struct udphdr)); + /* the crypto receive path expects skb->data to begin at the OpenVPN + * header and takes ownership of the skb + */ + __skb_pull(skb, payload_offset); ovpn_udp_recv(peer, skb); return 0; @@ -322,6 +311,60 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) return 0; } +/* Consume DATA_V2 directly from UDP GRO. These packets deliberately bypass + * packet taps, TC ingress, the outer IP and netfilter receive paths, the final + * UDP lookup, and normal UDP accounting. Control packets are restored and + * continue through all of those layers normally. + */ +static struct sk_buff *ovpn_udp_gro_receive_direct(struct sock *sk, + struct list_head *head, + struct sk_buff *skb) +{ + unsigned int offset = skb_gro_offset(skb); + + /* if the OpenVPN header is not accessible, leave validation and drop + * handling to the ordinary UDP receive path + */ + if (unlikely(!pskb_pull(skb, offset))) + goto flush; + + /* tell UDP GRO not to touch the skb if it was consumed by the direct + * receive path + */ + if (likely(!ovpn_udp_data_recv(sk, skb, 0))) + return ERR_PTR(-EINPROGRESS); + + /* control packets still belongs to the socket so we restore the data + * pointer because the normal receive path expects the outer headers + */ + skb_push(skb, offset); + +flush: + NAPI_GRO_CB(skb)->same_flow = 0; + NAPI_GRO_CB(skb)->flush = 1; + return NULL; +} + +/** + * ovpn_udp_encap_recv - Start processing a received UDP packet. + * @sk: socket over which the packet was received + * @skb: the received packet + * + * If the first byte of the payload is: + * - DATA_V2 the packet is accepted for further processing, + * - DATA_V1 the packet is dropped as not supported, + * - anything else the packet is forwarded to the UDP stack for + * delivery to user space. + * + * Return: + * 0 if @skb was consumed or dropped + * 1 if @skb should continue through normal UDP delivery + */ +static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) +{ + return ovpn_udp_data_recv(sk, skb, sizeof(struct udphdr)); +} + /** * ovpn_udp4_output - send IPv4 packet over udp socket * @peer: the destination peer @@ -598,7 +641,12 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock, .encap_type = UDP_ENCAP_OVPNINUDP, .encap_rcv = ovpn_udp_encap_recv, .encap_destroy = ovpn_udp_encap_destroy, - .gro_receive = ovpn_udp_gro_receive_fraglist, + /* GRO mode cannot change after the interface is created so + * select the socket callback once at socket setup + */ + .gro_receive = ovpn->gro_mode == OVPN_UDP_GRO_MODE_DIRECT ? + ovpn_udp_gro_receive_direct : + ovpn_udp_gro_receive_fraglist, .gro_complete = ovpn_udp_gro_complete, }; struct ovpn_socket *old_data; diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index 245b36204525..2d7b320f83be 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -2067,9 +2067,15 @@ enum ovpn_mode { OVPN_MODE_MP, }; +enum ovpn_udp_gro_mode { + OVPN_UDP_GRO_MODE_FULL_STACK, + OVPN_UDP_GRO_MODE_DIRECT, +}; + enum { IFLA_OVPN_UNSPEC, IFLA_OVPN_MODE, + IFLA_OVPN_UDP_GRO_MODE, __IFLA_OVPN_MAX, }; diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index 187f108f3ee8..bfe23ec9dfca 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -41,7 +41,7 @@ struct udp_tunnel_type_entry { IS_ENABLED(CONFIG_VXLAN) * 2 + \ IS_ENABLED(CONFIG_NET_FOU) * 2 + \ IS_ENABLED(CONFIG_XFRM) * 2 + \ - IS_ENABLED(CONFIG_OVPN)) + IS_ENABLED(CONFIG_OVPN) * 2) DEFINE_STATIC_CALL(udp_tunnel_gro_rcv, dummy_gro_rcv); static DEFINE_STATIC_KEY_FALSE(udp_tunnel_static_call); diff --git a/tools/testing/selftests/net/ovpn/Makefile b/tools/testing/selftests/net/ovpn/Makefile index 169f0464ac3a..412a70abf739 100644 --- a/tools/testing/selftests/net/ovpn/Makefile +++ b/tools/testing/selftests/net/ovpn/Makefile @@ -37,6 +37,7 @@ TEST_PROGS := \ test-close-socket-tcp.sh \ test-close-socket.sh \ test-float.sh \ + test-gro-direct.sh \ test-large-mtu.sh \ test-mark.sh \ test-symmetric-id-float.sh \ diff --git a/tools/testing/selftests/net/ovpn/common.sh b/tools/testing/selftests/net/ovpn/common.sh index 2d844eb3aa6e..1467caa95168 100644 --- a/tools/testing/selftests/net/ovpn/common.sh +++ b/tools/testing/selftests/net/ovpn/common.sh @@ -10,6 +10,7 @@ source "$OVPN_COMMON_DIR/../../kselftest/ktap_helpers.sh" OVPN_UDP_PEERS_FILE=${OVPN_UDP_PEERS_FILE:-udp_peers.txt} OVPN_TCP_PEERS_FILE=${OVPN_TCP_PEERS_FILE:-tcp_peers.txt} OVPN_CLI=${OVPN_CLI:-${OVPN_COMMON_DIR}/ovpn-cli} +OVPN_UDP_GRO_MODE=${OVPN_UDP_GRO_MODE:-FULL_STACK} OVPN_YNL=${OVPN_YNL:-${OVPN_COMMON_DIR}/../../../../net/ynl/pyynl/cli.py} OVPN_ALG=${OVPN_ALG:-aes} OVPN_PROTO=${OVPN_PROTO:-UDP} @@ -162,7 +163,8 @@ ovpn_setup_ns() { done fi - ip netns exec "${peer}" ${OVPN_CLI} new_iface tun${1} $MODE + ip netns exec "${peer}" ${OVPN_CLI} new_iface tun${1} $MODE \ + "${OVPN_UDP_GRO_MODE}" ip -n "${peer}" addr add ${2} dev tun${1} # add a secondary IP to peer 1, to test a LAN behind a client if [ ${1} -eq 1 -a -n "${OVPN_LAN_IP}" ]; then diff --git a/tools/testing/selftests/net/ovpn/ovpn-cli.c b/tools/testing/selftests/net/ovpn/ovpn-cli.c index f4effa7580c0..8e5f9c0986eb 100644 --- a/tools/testing/selftests/net/ovpn/ovpn-cli.c +++ b/tools/testing/selftests/net/ovpn/ovpn-cli.c @@ -123,6 +123,8 @@ struct ovpn_ctx { char ifname[IFNAMSIZ]; enum ovpn_mode mode; bool mode_set; + enum ovpn_udp_gro_mode udp_gro_mode; + bool udp_gro_mode_set; int socket; int cli_sockets[MAX_PEERS]; @@ -1377,8 +1379,9 @@ static int ovpn_new_iface(struct ovpn_ctx *ovpn) struct ovpn_link_req req = { 0 }; int ret = -1; - fprintf(stdout, "Creating interface %s with mode %u\n", ovpn->ifname, - ovpn->mode); + fprintf(stdout, + "Creating interface %s with mode %u and UDP GRO mode %u\n", + ovpn->ifname, ovpn->mode, ovpn->udp_gro_mode); req.n.nlmsg_len = NLMSG_LENGTH(sizeof(req.i)); req.n.nlmsg_flags = NLM_F_REQUEST | NLM_F_CREATE | NLM_F_EXCL; @@ -1396,15 +1399,21 @@ static int ovpn_new_iface(struct ovpn_ctx *ovpn) strlen(OVPN_FAMILY_NAME) + 1) < 0) goto err; - if (ovpn->mode_set) { + if (ovpn->mode_set || ovpn->udp_gro_mode_set) { data = ovpn_nest_start(&req.n, sizeof(req), IFLA_INFO_DATA); if (!data) goto err; - if (ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_MODE, + if (ovpn->mode_set && + ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_MODE, &ovpn->mode, sizeof(uint8_t)) < 0) goto err; + if (ovpn->udp_gro_mode_set && + ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_UDP_GRO_MODE, + &ovpn->udp_gro_mode, sizeof(uint8_t)) < 0) + goto err; + ovpn_nest_end(&req.n, data); } @@ -1666,11 +1675,16 @@ static void usage(const char *cmd) cmd); fprintf(stderr, "where can be one of the following\n\n"); - fprintf(stderr, "* new_iface [mode]: create new ovpn interface\n"); + fprintf(stderr, + "* new_iface [mode] [udp-gro-mode]: create new ovpn interface\n"); fprintf(stderr, "\tiface: ovpn interface name\n"); fprintf(stderr, "\tmode:\n"); fprintf(stderr, "\t\t- P2P for peer-to-peer mode (i.e. client)\n"); fprintf(stderr, "\t\t- MP for multi-peer mode (i.e. server)\n"); + fprintf(stderr, "\tudp-gro-mode:\n"); + fprintf(stderr, "\t\t- FULL_STACK for the normal receive stack\n"); + fprintf(stderr, + "\t\t- DIRECT to decrypt data from the UDP GRO callback\n"); fprintf(stderr, "* del_iface : delete ovpn interface\n"); fprintf(stderr, "\tiface: ovpn interface name\n"); @@ -2206,6 +2220,20 @@ static int ovpn_parse_cmd_args(struct ovpn_ctx *ovpn, int argc, char *argv[]) return -1; } ovpn->mode_set = true; + + if (argc < 5) + break; + + if (!strcmp(argv[4], "FULL_STACK")) { + ovpn->udp_gro_mode = OVPN_UDP_GRO_MODE_FULL_STACK; + } else if (!strcmp(argv[4], "DIRECT")) { + ovpn->udp_gro_mode = OVPN_UDP_GRO_MODE_DIRECT; + } else { + fprintf(stderr, "Cannot parse UDP GRO mode: %s\n", + argv[4]); + return -1; + } + ovpn->udp_gro_mode_set = true; break; case CMD_DEL_IFACE: break; diff --git a/tools/testing/selftests/net/ovpn/test-gro-direct.sh b/tools/testing/selftests/net/ovpn/test-gro-direct.sh new file mode 100755 index 000000000000..f35db23eb425 --- /dev/null +++ b/tools/testing/selftests/net/ovpn/test-gro-direct.sh @@ -0,0 +1,10 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020-2025 OpenVPN, Inc. +# +# Author: Ralf Lici +# Antonio Quartulli + +OVPN_UDP_GRO_MODE="DIRECT" + +source test.sh diff --git a/tools/testing/selftests/net/ovpn/test.sh b/tools/testing/selftests/net/ovpn/test.sh index 9b5610837032..55cb4d4c3e3d 100755 --- a/tools/testing/selftests/net/ovpn/test.sh +++ b/tools/testing/selftests/net/ovpn/test.sh @@ -113,6 +113,67 @@ ovpn_run_lan_traffic() { ip netns exec ovpn_peer0 ping -qfc 100 -w 3 "${OVPN_LAN_IP}" } +ovpn_udp_gro_counter_add() { + [ "${OVPN_PROTO}" == "UDP" ] || return 0 + # A custom tunnel MTU can fragment outer packets before UDP GRO. + [ -z "${MTU:-}" ] || return 0 + + # Enable UDP forwarding GRO on the receiving endpoint so this test + # exercises the configured ovpn callback. + ovpn_cmd_ok "enable UDP GRO on the iperf receive path" \ + ip netns exec ovpn_peer0 ethtool -K veth1 gro on \ + rx-udp-gro-forwarding on + + ovpn_cmd_ok "create UDP GRO path counter table" \ + ip netns exec ovpn_peer0 nft add table inet ovpn_gro_test + ovpn_cmd_ok "create UDP GRO path counter chain" \ + ip netns exec ovpn_peer0 nft \ + "add chain inet ovpn_gro_test prerouting { type filter hook \ + prerouting priority filter; policy accept; }" + + # Count only aggregated outer packets after they enter the normal + # receive stack in peer0. Direct GRO consumes those DATA_V2 aggregates + # before this hook, while small packets which bypass veth's GRO path + # are deliberately ignored. + ovpn_cmd_ok "add UDP GRO path counter" \ + ip netns exec ovpn_peer0 nft add rule inet ovpn_gro_test \ + prerouting iifname "veth1" meta length gt 1500 udp dport 1 \ + counter +} + +ovpn_udp_gro_counter_check() { + local packets + + [ "${OVPN_PROTO}" == "UDP" ] || return 0 + [ -z "${MTU:-}" ] || return 0 + + packets=$(ip netns exec ovpn_peer0 nft list chain inet ovpn_gro_test \ + prerouting | sed -n \ + 's/.*counter packets \([0-9][0-9]*\) bytes.*/\1/p') + ovpn_cmd_ok "remove UDP GRO path counter table" \ + ip netns exec ovpn_peer0 nft delete table inet ovpn_gro_test + + if [ -z "${packets}" ]; then + printf '%s\n' "unable to read UDP GRO path counter" + return 1 + fi + + if [ "${OVPN_UDP_GRO_MODE}" == "FULL_STACK" ]; then + if [ "${packets}" -eq 0 ]; then + printf '%s\n' \ + "full-stack UDP GRO did not reach PRE_ROUTING" + return 1 + fi + return 0 + fi + + if [ "${packets}" -ne 0 ]; then + printf '%s\n' \ + "direct UDP GRO reached PRE_ROUTING ${packets} times" + return 1 + fi +} + ovpn_run_float_mode() { local p local peer_ns @@ -134,12 +195,16 @@ ovpn_run_float_mode() { ovpn_run_iperf() { local iperf_pid + ovpn_udp_gro_counter_add + ovpn_run_bg iperf_pid ip netns exec ovpn_peer0 iperf3 -1 -s sleep 1 ovpn_cmd_ok "run iperf throughput flow" \ ip netns exec ovpn_peer1 iperf3 -Z -t 3 -c 5.5.5.1 wait "${iperf_pid}" || return 1 + + ovpn_udp_gro_counter_check } ovpn_run_key_rollover() {