From patchwork Mon Nov 2 16:43:43 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5557 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673329laa; Mon, 2 Nov 2026 08:44:21 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBw5Z4J+FkVBiIs56FA13ZB5uNB0H4kttA4ioMW1rTG32NBjLHGGdb09aTr65Rd5BC3Ch7C88k7ASHE=@openvpn.net X-Received: by 2002:a05:7022:3f0a:b0:1ac:2f6d:e1c6 with SMTP id a92af1059eb24-1ac56c9fffamr17427123c88.6.1793637860833; Mon, 02 Nov 2026 08:44:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637860; cv=none; d=google.com; s=arc-20260327; b=M4zYJ94FVSpng/vjcE5E7fLJ+QsgelpQZAJ7+1CFtz/V3HgJ0usyU4UR2sTkqlg3ge WZRUoKnqe8UE33zuTX/JdfD72FrL9lF/PPIruPzAIf9RyENh75S7iyd1/NOl21+Eegop ymGgYGX1XYyV+qU0arjHzncm2bsRFPC6HqfE273Boa4lCm6sDp5po7Dv1HzkdR4LqXF0 aoc4PHzY96k54Ujrql+rlWAN0ET7PX7sSAn9YXprHfg9rnpMObSkbwcJZukveVKk6n+g +fJnp2K5GMSMDOwUh4tnvbgcUFZVK6rrbB+NdLELZ9xyowE2IbtfZw8+6qd85eiSb+fp ABqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=NxDGLLPaPXfr4PzvcMu+gAUmuWe7burmlxu64YtTAMc=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=ddYLwwdPANYaDrsp07wBHBGQNQi1HZ1cANm0JkFy6KtZ8+6AWyyYyGaaYKG6BePs1+ CljDvVFWo4iHZYyxjuM8lqgO+2iqGZtel5onPAbX2BNVNwKTsr6eED/eMnUNaAr23imb yWW2R5oT0fYdTZS3BZGdSD6stBEknYLiY7hW892AzAK6oYiQPP+5ljtn00TWeDXJM3l6 zQO8ySvO1FBcLvcVA6OVNBd7d4Nb7vuoJZodrA6ebKK0NzBiw6cNjp7zMN8BcDKObXlx LNGe7f4FIynQW3LUqUhuNcYu/7dvgTIoz/XED/fAV4Q0hoi6frJctXXrpodh8QfkGLGu CMzQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=SV8K8iD4; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=CwuFDZlC; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=m4ebFmgd; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b="KKKo4y/w"; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id a92af1059eb24-1ac1a22c423si30314387c88.64.2026.11.02.08.44.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:20 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=SV8K8iD4; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=CwuFDZlC; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=m4ebFmgd; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b="KKKo4y/w"; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=NxDGLLPaPXfr4PzvcMu+gAUmuWe7burmlxu64YtTAMc=; b=SV8K8iD4wH2xEN cSaTFx/1jnuCKaKVxedNko1nE13rhSoKkT8e2k3mLd2ST7GGlDEqGcwXpTuRl36qWk+3PkXbw5GDP t6qibQYYnti9E6+Gq2y9OxwC1dLhWsxjz4eCsaSydsIVm04y4Tc2cu988Sg/w2u8VXACFNGObRIwi kJ8=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8t-000000001da-1jkx; Mon, 02 Nov 2026 16:44:19 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8h-000000001d7-3TuY for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=KKmHXrJ+3AaQZ632CRxiErDmYTghlL6pgUcIYhI2FkU=; b=CwuFDZlCBoxU8XKbgkTEx2x/jN diN9v70mnQHCcMABcDKrrW1rcR7+GAfytyCRh5lgeBRYJjIujM20pdUqcXXToeNfplksezg2oDUXN fVR3wuFPXzT4iy4xDjh2IAXD1/d5SyKYrf5DWiiN/thSRHOg79fkE4htC4ssVCxvJ8Hs=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=KKmHXrJ+3AaQZ632CRxiErDmYTghlL6pgUcIYhI2FkU=; b=m4ebFmgdnE5Rw6RBPvYmut5DmO Qa3hF7iQBn7N+Pu0Twz38glUaZhcVKFtZpIhAE3JiaKSHp3t9WO/Neff6wKAJDTqllk0kyHZJaQhc Sl05lojeTny3NW2WBq5Ph5Kuzl0x3ZLZdaVxyVfdlqpwjdAsvYLBJHzEmK372xmZad+k=; Received: from mout-b-106.mailbox.org ([195.10.208.46]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8g-000000005Ww-0aHO for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:07 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-106.mailbox.org (Postfix) with ESMTPS id 4jJHbK64G8zNkHq for ; Mon, 02 Nov 2026 17:43:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637837; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KKmHXrJ+3AaQZ632CRxiErDmYTghlL6pgUcIYhI2FkU=; b=KKKo4y/w+Sz80oIaKPAs+ukFV+3aTPN/dsq78vHtUZxuAhDWW7g9OlLairlBput4mngiQY v6zuOQSJXN4FJ8mDiCKSzxWLZ3bn3jmoLWoHr3guieiLAf25dlDuRdKvf7mSrHmadrIVxe oF/cKHjzqaAW0WmvOk+NaobGgRaKpkAuMnMmJxGb9DLTK6eZrme76qow8IsFSjS4u8YSU7 EE9Rv9GWAQF1zys/3ac1hvjFd0NJL98cglUWag+MOU5WjipFr/qLfv0GtYmUUgf3hkerbn QJE1Tbf7/oC8D1gLQOrL8ltxa77yCziUmunaZzVU+m0Mr3WKNiO2snAYRKiVWw== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:43 +0100 Message-ID: <20261102164353.459969-2-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbK64G8zNkHq X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1xNv8g-000000005Ww-0aHO Subject: [Openvpn-devel] [PATCH ovpn net-next v2 1/9] ovpn: advertise checksum offload for GSO packets X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765613394247352 X-GMAIL-MSGID: 1880765613394247352 ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before they reach ovpn because TCP GSO packets normally carry CHECKSUM_PARTIAL state. Advertise NETIF_F_HW_CSUM so the stack can pass such packets to ovpn. Complete partial checksums after any GSO segmentation and before submitting packets for encryption, since the inner packet checksum can no longer be fixed after the packet has been encrypted. Also pass the ovpn feature set to skb_gso_segment with GSO capabilities masked out: this forces software segmentation, but still lets the segmenter preserve supported non-GSO properties such as non-linear skb data instead of needlessly linearizing. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/1bbfde37488ade61928554db3a119525adf7b608.1790843153.git.ralf@mandelbit.com/ drivers/net/ovpn/io.c | 20 ++++++++++++++++++-- drivers/net/ovpn/main.c | 2 +- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index f14eb4e6c46a..39bbd464e437 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -356,6 +356,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; struct sk_buff_head skb_list; + netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; __be16 proto; @@ -390,8 +391,13 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_dst_drop(skb); if (skb_is_gso(skb)) { - segments = skb_gso_segment(skb, 0); - if (IS_ERR(segments)) { + /* force software segmentation, but keep ovpn's non-GSO feature + * bits so the generated segments can preserve non-linear skb + * data where possible + */ + features = netif_skb_features(skb); + segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", netdev_name(dev), ret); @@ -416,6 +422,16 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) continue; } + /* NETIF_F_HW_CSUM requires completing partial checksums */ + if (unlikely(curr->ip_summed == CHECKSUM_PARTIAL && + skb_checksum_help(curr) < 0)) { + net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", + netdev_name(dev)); + ovpn_dev_dstats_tx_dropped(ovpn->dev); + kfree_skb(curr); + continue; + } + /* only count what we actually send */ tx_bytes += curr->len; __skb_queue_tail(&skb_list, curr); diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 0708249e9607..28e1eb06e127 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -157,7 +157,7 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { - netdev_features_t feat = NETIF_F_SG | NETIF_F_GSO | + netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Mon Nov 2 16:43:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5556 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673322laa; Mon, 2 Nov 2026 08:44:21 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBxWutjDLn6Qyo+flMGqXz6+HNQIQuS0dVfbChItLQa6Cem8PIEtM/lYzlb/XTG/y4xEUjH7Ej+ujRA=@openvpn.net X-Received: by 2002:a05:7300:8e:b0:36a:119d:a099 with SMTP id 5a478bee46e88-36a422062f9mr17864516eec.5.1793637860694; Mon, 02 Nov 2026 08:44:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637860; cv=none; d=google.com; s=arc-20260327; b=boLXJL00oxLzqZDYFgTUoBkq4Ldd9M4DuM77vX56tKNNVVtsyDsgUS2UVJAgy+2neF TvSjQ8U22yH4Bzj1+nCKpgPm03S8rwLHdhJX8J/vxPA5IXHsQrEIhrjActd8t+rNI68U kr2vMryVM3hPG2TqmxPVYnXkRYDAwL0OaJbg+FmsLAnZd1J1ZE7ywcGR33zRKCQ/YmA8 SBzb4dQLfqzztimqdhFDYgLUiK5CqoDfyevV0xnxQS7HePYgcKOMkpNUETCS3TlvULby nzAnSmntKukkH2GSkMm0VHjTtx4rbiwJbUaUXOTSicTvnWvL0tnjonWknIbtEG+wDEyT /yvw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=wrWCMErb07Ah3a6eodUA9EOj2e9RsiEKOj+1s67K2t8=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=g+6FuTac56jm2mKroGjppCDx9q1cjmZX4lBPPNRNgM0Yj11toVx2KurDI2rS+VyM1L JjxNAwJ6uW6Y3ja7XIHrgYuDSMTlewEf1AhHrkp93ip3z7kuTPRt8hiTCiNxJDdo59C0 cZ0Bs1UAYyaT1PCT/5Qy2Szh9gm4MIYuo4khuW3H/3mN8VuRs9/w4MA4CSgbU7C4oAvC HSi3gDezdYARYAXRlpYaisu2xCLb7yIfjcjoTdth2V4kdJxt3oSfFvrrYwVwihaXd5Q/ DD38cIlcTEnVRLnp9Xv3krzcdqad1Rv7VLpKfcLqJbwXZZsfWRsvXsL8S31zBBGKfRk1 L9xA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=TS7trACD; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=ZExlec54; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=OMVvfGxc; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=UBdGMbGV; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5a478bee46e88-36a42a8d28esi31082336eec.15.2026.11.02.08.44.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:20 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=TS7trACD; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=ZExlec54; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=OMVvfGxc; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=UBdGMbGV; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=wrWCMErb07Ah3a6eodUA9EOj2e9RsiEKOj+1s67K2t8=; b=TS7trACDcIvKGS JFVX5VBh5bUCWwffc+3SCqsAZYrTeDL3bJgRzNvAfmaF5eLKOKdAHNLJUJ83Qr4lSUIfWx5wGD2nq iF0tM9csTEcTsqykTpqu6IfaIpuh3Y7/syHBXrfR6e7dmXFC/eJIi+4aOIIBDzWG/Rg+Qy7VxBuiC cbQ=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8s-000000000ai-3gy7; Mon, 02 Nov 2026 16:44:18 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8i-000000000Zs-3VRj for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=0ZH6lSFiR2jzEaIKqoSfpd/68ydb0IEIWJnWLxiWFD8=; b=ZExlec54Yoh4Ae2GqgfBSrzX5j 0ixsxhrOWwUVd1QhijxSLyLfmTn5+iqkdzyc7Hg0UfO3BbrGfkgyRA+grfBTxO0YU1qUDn0VeJeh9 gxQsBHLWNKB9I0GKx2HGW+D36WKWw2RLTSUFafE6W68qTn95ky9sOWqsurEi+LvSr0ms=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=0ZH6lSFiR2jzEaIKqoSfpd/68ydb0IEIWJnWLxiWFD8=; b=OMVvfGxcjzSAxW2evc6vl7Allw ed2sBskk9HVLielPrGI8Rb3iDWJyLRiSiEDlKWrGKZ7l4CzKCPGMx9qnJOV7oz0k2g3YonHKgjRa1 T4zJ1UZ/IZ/MPRfF43f62sAklB1pdGyRXzIYfNFRxkYhY6fJhdTZAiBJ6IlrNO/+aS0U=; Received: from mout-b-202.mailbox.org ([195.10.208.62]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8h-000000005hM-2ibq for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:08 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-202.mailbox.org (Postfix) with ESMTPS id 4jJHbL3px7zKn5v for ; Mon, 02 Nov 2026 17:43:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637838; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0ZH6lSFiR2jzEaIKqoSfpd/68ydb0IEIWJnWLxiWFD8=; b=UBdGMbGVw4/91a+ZYPENlIAH5/sr3EbE3d6s7VUDSTqJjBXRskXhXRV++ZwAvd9hCFThvD 8bOrCyXgP/hCiYL2dSfnpKFV9scqTeXcw8DGXa5Reips4V+O3FYwBVnrUaJqJy6XEOWit+ p4+pIEarRt1kwa77BMaTWDpFmPpU304hxNzZ6sVI42mTt7mDXxtcISUpcdtiLY3W9+VkpS zFOYpr/vyw6dxHmUgsdYpxpTm7FqQCdTwhkKlA6rVLFXKTjoLiphlo7OVieNw3lBkpmFpH 23Wr0yA2fY7/NeUdTkbzHOk8M1Rjb4IDsBmcGENv9Xf2cX0goUieLb5ImNlJwA== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:44 +0100 Message-ID: <20261102164353.459969-3-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbL3px7zKn5v X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_ [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1xNv8h-000000005hM-2ibq Subject: [Openvpn-devel] [PATCH ovpn net-next v2 2/9] ovpn: accept frag-list GSO input X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765613075278882 X-GMAIL-MSGID: 1880765613075278882 Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_FRAGLIST, so generic transmit validation segments these aggregates before calling ovpn_net_xmit. That segmentation is functionally correct, but causes ovpn_net_xmit to be invoked separately for every resulting packet. Advertise frag-list storage so ovpn receives the aggregate intact and performs protocol validation and destination-to-peer lookup once before segmenting it. Frag-list GSO segmentation recovers the complete child skbs, which can then be encrypted in place and transmitted independently. Rebuilding those children into a replacement UDP GSO aggregate was found to add cost rather than improve throughput. The feature also admits non-GSO frag lists, which describe one packet split across several skbs. Let skb_cow_data preserve small lists directly. If a list exceeds the AEAD scatterlist limit, linearize it and continue rather than rejecting an otherwise valid packet. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/5baa1d94e29e7b109ffd412b4f1d5d113a319389.1790843153.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 8 ++++++-- drivers/net/ovpn/main.c | 3 ++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 74eaf6fac2f5..2af493fd5735 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -168,8 +168,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, if (unlikely(nfrags < 0)) return nfrags; - if (unlikely(nfrags + 2 > (MAX_SKB_FRAGS + 2))) - return -ENOSPC; + if (unlikely(nfrags > MAX_SKB_FRAGS)) { + ret = skb_linearize(skb); + if (unlikely(ret)) + return ret; + nfrags = 1; + } /* allocate temporary memory for iv, sg and req */ tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 28e1eb06e127..ac4e0d85e215 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -158,7 +158,8 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | - NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; + NETIF_F_GSO_SOFTWARE | NETIF_F_FRAGLIST | + NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Mon Nov 2 16:43:45 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5562 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673445laa; Mon, 2 Nov 2026 08:44:25 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBwJvaXLdLAhiODL5c/NewyNV0/jSZyO9YfIyqFcHM3rm/rPYGPpAv9rH902MhC3kJ/8bX3gudeY6nM=@openvpn.net X-Received: by 2002:a05:7301:82a:b0:351:dbc:b663 with SMTP id 5a478bee46e88-36a4244f9f6mr18237159eec.31.1793637864448; Mon, 02 Nov 2026 08:44:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637864; cv=none; d=google.com; s=arc-20260327; b=CsqFVJEcb10SxYjIyoCqeZywONFY8jjAzIvY5DlVHQyvNA7am7qvSgqhOQ7AYIXYXj qAJmAUkdnRfY6bfuHX/3OtHlg60aEOFghDPe90xiOyM3njlUuYOeOhapAQuU4tkOhXBV R/Cw7etbhOxcPpmaCXUmvaRDmx/gd8CenU/28fsNGx3kuY5i5iplHU4iW2vwmsbEIFKd whZMXjgJtAUY8HGbkgEfzMrt/86u2qplpXM17WvAkwFsl97c5k3aP44tjFcbzN+S6aFr uRTiex0PFJetUkA0s2eyrTcW8QqDm9AZIKOXm6ldxiL/8NpVjd3qhUQndfI8kXfvwjNM qQuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=QmOSSPDxdjUM8aI4xxtE73p6A9YY8cweTZzzsH3/7Fk=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=BMtmp9Af8MV6kcpp/HhcPRmn6tCYDBdME8U1NlzBiKKly54/D9KGiVDeDV88gkPtET 2Dph5whqSqokqENlBueYK1pvqukg3vsPSPn/j07UKafvAR/rj8dNWAn0k/xt66zeIqxC hMu3x30ZRoVSVlF+9+HOTKBd4+ejlVQZ9ABS6M//KRAJGhoVAb7FZv69rMFw6TeNomkQ wE/RioVX0lcjQC9ocWqtVUa4Kc9zlnEL/n7WLVg81VgvI6dIRQa/UnOx+jOFwnW19qeF GAQWmksPZbjG6oKODSxPrwBbm1hEsLo6m/wxjf0wfdOFkXLHqSrMQDR79lxzzpRFqoQo fYXw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=HiBuZJiS; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Ajv2DLNZ; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=gc9Nl+Z+; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=CfJ39yxq; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5a478bee46e88-36a42bb8c10si29792599eec.96.2026.11.02.08.44.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:24 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=HiBuZJiS; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Ajv2DLNZ; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=gc9Nl+Z+; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=CfJ39yxq; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=QmOSSPDxdjUM8aI4xxtE73p6A9YY8cweTZzzsH3/7Fk=; b=HiBuZJiSyi7I21 WJbnvzXihGCoXyglMTc8lXeyTn1Prk3DVOLUa4vD5lljgFefh2xdccVnQbGmMHMMeu6EigtLtNvi9 SpMedpfV4bi4OmN2EBNrxlWL7bhZVOPXHxJYMBa07W53ueQSeXulQ03iOvk6M17lI7UCi8Lu6XbY2 Uag=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8x-000000002uZ-1GZs; Mon, 02 Nov 2026 16:44:23 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8i-000000002uA-3WMd for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=z5U7rARn4Gsd0vR6oS2qnc/XhmQqPkwvF4OkYTjFUOo=; b=Ajv2DLNZb+JpAO0NLZYhkql8Bz MKm3zKw2tIdvqhNiY986CcKydK2dIwHhXsvL/teKYTwM0j0JrUEOKo7sYuolaEBscRekQPt2Mb+RD UzbJVotZRavhPhWtATyNpnlC4qot2lGq2R/sL7PETwpe8GLmPTGFaytsUl7UeebvLWvM=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=z5U7rARn4Gsd0vR6oS2qnc/XhmQqPkwvF4OkYTjFUOo=; b=gc9Nl+Z+dDI/Q/cV66n3nzhNm/ c8u3GofJXSpU3a+RK6DmjdYcDwBxkCtcjCcIbZxdJlAiBcGGp1wrToKXuNmXpjxYwIRKqcE58/mZG UCIxcUPx8Hp0zrA5e2llZka2uxbnfTTQB0s4khlwgtYvOg5WXQUWF1HUwzmnSDM0yBtc=; Received: from mout-b-201.mailbox.org ([195.10.208.61]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8h-000000005Wx-1TEj for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:08 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-201.mailbox.org (Postfix) with ESMTPS id 4jJHbL6N9MzLmB8 for ; Mon, 02 Nov 2026 17:43:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637838; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z5U7rARn4Gsd0vR6oS2qnc/XhmQqPkwvF4OkYTjFUOo=; b=CfJ39yxqysJAyBR5mQrqPhJ0T/b8cOb0y7cxhOiX66vMlL05XKuSfaDBP2EGxZJqZjjnUl zx++rzUdsMfCOU8DZh2KYcSg0QtAECjg5IsPRWcAyLiatlYed/xUgucSfS9UpG7XmXmVDQ PV7615caMhecbZC2hkPmcmivzbuRTPyMYqReELWFA6BKHoFvOUEqfl68zpl4H/kPm29I3p sEkfTKHieV+m2Uo1/VQY3bBOralmeP126st2qvTBF2Ex77RMBVwwjfduT0kafMOF4LHfzt LKIlrZYZTaEI1vYia04jARM8/VnNd88eyqGhSFwaOEhqQAcTPDrK3mX1NL79Lg== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:45 +0100 Message-ID: <20261102164353.459969-4-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbL6N9MzLmB8 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xNv8h-000000005Wx-1TEj Subject: [Openvpn-devel] [PATCH ovpn net-next v2 3/9] ovpn: refactor AEAD encryption helpers X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765617293179026 X-GMAIL-MSGID: 1880765617293179026 Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without changing packet handling. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/f075b9782b14d64756c84e132c138482f08287b9.1790843153.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 97 ++++++++++++++++++++-------------- drivers/net/ovpn/io.h | 3 +- drivers/net/ovpn/proto.h | 4 ++ 3 files changed, 63 insertions(+), 41 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 2af493fd5735..30299581422d 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -24,9 +24,6 @@ #include "proto.h" #include "skb.h" -#define OVPN_AUTH_TAG_SIZE 16 -#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE) - #define ALG_NAME_AES "gcm(aes)" #define ALG_NAME_CHACHAPOLY "rfc7539(chacha20,poly1305)" @@ -42,7 +39,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * an AEAD request structure with extra space for SG * and IV. * @tfm: the AEAD cipher handle - * @nfrags: the number of fragments in the skb + * @nents: the number of scatterlist entries * * This function calculates the size of a contiguous memory block that includes * the initialization vector (IV), the AEAD request, and an array of scatterlist @@ -54,7 +51,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * Return: the size of the temporary memory that needs to be allocated */ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, - const unsigned int nfrags) + const unsigned int nents) { unsigned int len = OVPN_NONCE_SIZE; @@ -70,8 +67,8 @@ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, /* round up to the next multiple of the scatterlist alignment */ len = ALIGN(len, __alignof__(struct scatterlist)); - /* add enough space for nfrags + 2 scatterlist entries */ - len += array_size(sizeof(struct scatterlist), nfrags + 2); + /* add enough space for the scatterlist entries */ + len += array_size(sizeof(struct scatterlist), nents); return len; } @@ -135,6 +132,53 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, __alignof__(struct scatterlist)); } +static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, + struct sk_buff *skb, + unsigned int nents, u8 **iv) +{ + struct aead_request *req; + void *tmp; + + /* allocate IV, request and scatterlist entries in one block */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + if (unlikely(!tmp)) + return ERR_PTR(-ENOMEM); + + ovpn_skb_cb(skb)->crypto_tmp = tmp; + *iv = ovpn_aead_crypto_tmp_iv(aead, tmp); + req = ovpn_aead_crypto_tmp_req(aead, *iv); + + return req; +} + +static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + u8 *iv, u8 *data) +{ + u32 pktid, op; + int ret; + + /* obtain packet ID, which is used both as a first + * 4 bytes of nonce and last 4 bytes of associated data. + */ + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + if (unlikely(ret < 0)) + return ret; + + /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes + * nonce + */ + ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); + + /* add the packet opcode and wire nonce as associated data */ + op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); + BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); + *(__force __be32 *)data = htonl(op); + memcpy(data + OVPN_OPCODE_SIZE, iv, OVPN_NONCE_WIRE_SIZE); + + return 0; +} + int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { @@ -143,8 +187,6 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *trailer; struct scatterlist *sg; int nfrags, ret; - u32 pktid, op; - void *tmp; u8 *iv; ovpn_skb_cb(skb)->peer = peer; @@ -175,16 +217,9 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), - GFP_ATOMIC); - if (unlikely(!tmp)) - return -ENOMEM; - - ovpn_skb_cb(skb)->crypto_tmp = tmp; - - iv = ovpn_aead_crypto_tmp_iv(ks->encrypt, tmp); - req = ovpn_aead_crypto_tmp_req(ks->encrypt, iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + if (IS_ERR(req)) + return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); /* sg table: @@ -206,28 +241,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, __skb_push(skb, tag_size); sg_set_buf(sg + ret + 1, skb->data, tag_size); - /* obtain packet ID, which is used both as a first - * 4 bytes of nonce and last 4 bytes of associated data. - */ - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + /* make space for the additional data and push it to the front */ + __skb_push(skb, OVPN_AAD_SIZE); + ret = ovpn_aead_encrypt_header(peer, ks, iv, skb->data); if (unlikely(ret < 0)) return ret; - /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes - * nonce - */ - ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); - - /* make space for packet id and push it to the front */ - __skb_push(skb, OVPN_NONCE_WIRE_SIZE); - memcpy(skb->data, iv, OVPN_NONCE_WIRE_SIZE); - - /* add packet op as head of additional data */ - op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); - __skb_push(skb, OVPN_OPCODE_SIZE); - BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); - *((__force __be32 *)skb->data) = htonl(op); - /* AEAD Additional data */ sg_set_buf(sg, skb->data, OVPN_AAD_SIZE); @@ -281,7 +300,7 @@ int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return -ENOSPC; /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags), + tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags + 2), GFP_ATOMIC); if (unlikely(!tmp)) return -ENOMEM; diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index db9e10f9077c..1a94f0fda1d1 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -11,8 +11,7 @@ #define _NET_OVPN_OVPN_H_ /* DATA_V2 header size with AEAD encryption */ -#define OVPN_HEAD_ROOM (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE + \ - 16 /* AEAD TAG length */ + \ +#define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ max(sizeof(struct ipv6hdr), sizeof(struct iphdr))) diff --git a/drivers/net/ovpn/proto.h b/drivers/net/ovpn/proto.h index b7d285b4d9c1..f3b305cbefe5 100644 --- a/drivers/net/ovpn/proto.h +++ b/drivers/net/ovpn/proto.h @@ -39,6 +39,10 @@ #define OVPN_NONCE_WIRE_SIZE (OVPN_NONCE_SIZE - OVPN_NONCE_TAIL_SIZE) #define OVPN_OPCODE_SIZE 4 /* DATA_V2 opcode size */ +#define OVPN_AUTH_TAG_SIZE 16 +#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + \ + OVPN_NONCE_WIRE_SIZE) +#define OVPN_DATA_V2_OVERHEAD (OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE) #define OVPN_OPCODE_KEYID_MASK 0x07000000 #define OVPN_OPCODE_PKTTYPE_MASK 0xF8000000 #define OVPN_OPCODE_PEERID_MASK 0x00FFFFFF From patchwork Mon Nov 2 16:43:46 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5561 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673370laa; Mon, 2 Nov 2026 08:44:22 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBxJejX7XD3l1+C5yKgQDmLLhIZU037AOgn5KmSgLTdIlACaz0ZYY31j4WdRv3Y5nO6iR0l8C+2yDwg=@openvpn.net X-Received: by 2002:a05:701b:4287:20b0:1a8:b566:80c2 with SMTP id a92af1059eb24-1ac5d889488mr15697052c88.17.1793637861979; Mon, 02 Nov 2026 08:44:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637861; cv=none; d=google.com; s=arc-20260327; b=Z+P2FtDpz/UkgEsCffpAKrv9zy1U5TEwUTM7S+2l9R/KJcsZnyP3RxPRpTDDjYSHSQ zyHcXo0wiN6YmB2EzMzJHFpsNhMm/gEgidHd+Apilh1GLMTEswTQesyIDp61SHSM7WCl XZPimIWQjQYQnf0i7AnQuDgn9qh3TdDR6l1M90pS/g7CC48hi9JYSKFoLygkUS8jDIaL yN/fMSEpm2EFTqKQjE4iq8xTW4kPD1fHEsQD45Sje0JHKUStg/tO75xYsPy08Ur+rsOS vd6Q2yd4YVuVnCrAJnkJTVKthK1cLQ5SznJ8EEivt/dASvsSeHibpsjwqDlkk1T15uFo vFGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=1mQp9z4lkIlKPHQtt3HKIUyXJqr/cVJKG3t1oDUmNR8=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=KljmGtd71BiD5gJbudnoiw/9qLuY9SztF5SRDSdUJhYVwrlXp2/OrOZWaOIG1rlUdZ BqJ0WzBgoWJSsoiGZiyHJwbMU8OtkAvqC/eKvcnUW8a6V7RJ0b9fKXZhHyXxnmDc5p4G oBTjjjZigUAsfwHYw6DuVYkNrkY+R8Pk35DZZGQHDEseFOO01sfR+LEEaNsGD+qjAEG2 PIShtcoFvMLki8nZlGylHpEjxlJ1lPBHqWJbBbEk2qvgxFPhzRtiJIq0050IDABzgpdD ZcB/NWJc18y1WLs1TS2pL/f4dV2Bj9kypY2cm681hcPErfz/j2MxlaVuz9ZNPMI6hPAp w+iA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=WYXKHiBd; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="jWa+C/Y1"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=HIg6KDVR; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=XyHd091L; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id a92af1059eb24-1ac1a6022aasi30070828c88.66.2026.11.02.08.44.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:21 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=WYXKHiBd; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="jWa+C/Y1"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=HIg6KDVR; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=XyHd091L; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=1mQp9z4lkIlKPHQtt3HKIUyXJqr/cVJKG3t1oDUmNR8=; b=WYXKHiBdAMzUQH TS0Q5eNztIQoAZuVp/0v/wfT/k+aIMhH8nUwvLBaerQv84qK9AJ8q3ZZpITLzCWjWDyXzTpnO/bEZ G30Qsr/X8Rq4a6tioyo0ZaXJPK2xGU46rHEPi7XMtngkOv0Mu6QWT4j/M0XHLHGXjMTSofH0k/vdN Mf4=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8u-000000000iF-3LIn; Mon, 02 Nov 2026 16:44:20 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8k-000000000hV-3t7E for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=Kwx83emCCTnos/I+j/q8v2CuM/8gr873i/M2hnGL3w4=; b=jWa+C/Y1xK13bT36Zkot5ocZBO dUjhIlzIuf3Ti+aIZNJ+pnUWKaMYvCaKsDL6mkSxL0hK6fTOvAAAEq52cxALyb8jXTxlSoBwAx4LH C9kU0dMHOBftF12YpsZWQkWh0Zc2NtRYyV0t2/GYvIaKwQILNEw71ndfBMvkkGhw/Xv4=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Kwx83emCCTnos/I+j/q8v2CuM/8gr873i/M2hnGL3w4=; b=HIg6KDVR6ygGgJajugkmcatpUX Z25XBmcia8htoc8r1BT/H3S0xZneeITwULB0XKfRCbouYUIw8qG/b80rEqVAyDtbevpwm6ZTW5/lM gU6M/5wlzXVJiDkBkadbNUNirjMI6PM4wymaKdE8AD4jk/ys/KPRBWiZdCnypUv7nbOQ=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8h-000000005Wz-2gAQ for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:10 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4jJHbM1Zl2z5vMB for ; Mon, 02 Nov 2026 17:43:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Kwx83emCCTnos/I+j/q8v2CuM/8gr873i/M2hnGL3w4=; b=XyHd091LcQ6Lp9nvHasn5XwwSrIEvGf/5YtRDQ2rj/eO/Q7ylKHghGoAdw4QbK0SSgl5v2 Lli1Gy+mnG4MRDsZW7WANUj5fJbFb4lFAv9NckZcV3YFpqaUF5uMYqpC1TgTdw0gOHvJUS Zx9uYRLpjEY2Q8ILh8JWqy9KN+96jQ0gb+ffKCWCh6toJl6HdEW8dowxmXtoFVgiK+mUMb dLLCNQU+o1QGUzUJnTszwS8fpmSo/Tq84t95MggCyqF0+R4zic5QHGlV84v+z5zqEAQy5+ ss73ccQFQaaNn00n7yaJNbjRsL8s4Kva1J5z7sxBW+aNjVVRU9ObnVVdRkfw5w== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:46 +0100 Message-ID: <20261102164353.459969-5-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbM1Zl2z5vMB X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb while completing its checksums. Allow the segmentation code to retain safe page-backed payload fragments, then emit eligible [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1xNv8h-000000005Wz-2gAQ Subject: [Openvpn-devel] [PATCH ovpn net-next v2 4/9] ovpn: convert GSO input into UDP GSO output X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765614925679846 X-GMAIL-MSGID: 1880765614925679846 Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb while completing its checksums. Allow the segmentation code to retain safe page-backed payload fragments, then emit eligible fixed-size inputs as UDP GSO skb(s). Allocate one final page-backed aggregate per batch before submitting encryption. Map each segment's linear data and page fragments directly into the AEAD source scatterlist, and have each request write out of place into its final record slot. This avoids copying page-backed payload into a linear segment. The skb core can still linearize externally mutable fragments when needed to complete their checksums safely. Keep the existing in-place path for ordinary packets, where allocating and retiring a separate output skb for every record would provide no aggregate construction benefit. Also retain that path for frag-list GSO input: it already stores complete segments as child skbs, and measurements show that copying those children into another aggregate is counterproductive. Transmit the aggregate as SKB_GSO_UDP_L4 only after every record succeeds, and discard it if any request fails. Split aggregates at the legacy GSO size limit and fall back to individual records when batching is unavailable or the segment geometry is unsuitable. Preserve the input priority, flow hash and sender CPU on the replacement aggregate. If the input has real write ownership, charge the aggregate to the same socket as well. This retains socket lifetime and write-memory accounting and lets lower-device queue selection use the socket's cached TX queue instead of choosing a new queue after crypto completion. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without UDP GSO 12.053 Gbit/s 10.992 Gbit/s With UDP GSO 19.262 Gbit/s 19.718 Gbit/s This is a 59.8% improvement forward and a 79.4% improvement reverse. With lower-device UDP segmentation disabled, the GSO path raised median throughput from 12.386 to 15.489 Gbit/s forward and from 10.206 to 14.816 Gbit/s reverse, improvements of 25.1% and 45.2%, respectively. Signed-off-by: Ralf Lici --- Changes since v1 https://lore.kernel.org/openvpn-devel/47eee3045f402b2baf7db759bc9cfd35dee8f73d.1790843153.git.ralf@mandelbit.com/ - Release successfully encrypted source skbs with consume_skb after their data has been transferred to the output aggregate. (Antonio) drivers/net/ovpn/crypto_aead.c | 109 +++++++++++--- drivers/net/ovpn/crypto_aead.h | 4 + drivers/net/ovpn/io.c | 261 ++++++++++++++++++++++++++++++--- drivers/net/ovpn/skb.h | 27 +++- drivers/net/ovpn/stats.h | 16 +- drivers/net/ovpn/tcp.c | 4 +- drivers/net/ovpn/udp.c | 27 +++- 7 files changed, 394 insertions(+), 54 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 30299581422d..76028f048606 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -134,13 +134,17 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, struct sk_buff *skb, - unsigned int nents, u8 **iv) + unsigned int nents, + unsigned int extra, u8 **iv) { struct aead_request *req; void *tmp; - /* allocate IV, request and scatterlist entries in one block */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + /* allocate IV, request, scatterlist entries and caller scratch space + * in one block + */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents) + extra, + GFP_ATOMIC); if (unlikely(!tmp)) return ERR_PTR(-ENOMEM); @@ -151,19 +155,11 @@ static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, return req; } -static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, - struct ovpn_crypto_key_slot *ks, - u8 *iv, u8 *data) +static void ovpn_aead_encrypt_header(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + u32 pktid, u8 *iv, u8 *data) { - u32 pktid, op; - int ret; - - /* obtain packet ID, which is used both as a first - * 4 bytes of nonce and last 4 bytes of associated data. - */ - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); - if (unlikely(ret < 0)) - return ret; + u32 op; /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes * nonce @@ -175,8 +171,6 @@ static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); *(__force __be32 *)data = htonl(op); memcpy(data + OVPN_OPCODE_SIZE, iv, OVPN_NONCE_WIRE_SIZE); - - return 0; } int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, @@ -187,6 +181,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *trailer; struct scatterlist *sg; int nfrags, ret; + u32 pktid; u8 *iv; ovpn_skb_cb(skb)->peer = peer; @@ -217,7 +212,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, 0, &iv); if (IS_ERR(req)) return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); @@ -243,9 +238,11 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, /* make space for the additional data and push it to the front */ __skb_push(skb, OVPN_AAD_SIZE); - ret = ovpn_aead_encrypt_header(peer, ks, iv, skb->data); + /* obtain the packet ID used by both the nonce and associated data */ + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); if (unlikely(ret < 0)) return ret; + ovpn_aead_encrypt_header(peer, ks, pktid, iv, skb->data); /* AEAD Additional data */ sg_set_buf(sg, skb->data, OVPN_AAD_SIZE); @@ -261,6 +258,80 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return crypto_aead_encrypt(req); } +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, + struct sk_buff *gso_skb, unsigned int offset) +{ + const unsigned int dst_nents = skb_shinfo(gso_skb)->nr_frags + 4; + const unsigned int src_nents = skb_shinfo(skb)->nr_frags + 2; + struct scatterlist *src, *dst; + struct aead_request *req; + unsigned int payload_off; + int dst_idx, mapped, ret; + u32 pktid; + u8 *aad, *iv; + + /* each input records the shared peer and key for the common completion + * path but their references remain owned by the output aggregate + */ + ovpn_skb_cb(skb)->peer = peer; + ovpn_skb_cb(skb)->ks = ks; + + req = ovpn_aead_request_alloc(ks->encrypt, skb, src_nents + dst_nents, + OVPN_AAD_SIZE, &iv); + if (IS_ERR(req)) + return PTR_ERR(req); + src = ovpn_aead_crypto_req_sg(ks->encrypt, req); + dst = src + src_nents; + aad = (u8 *)(dst + dst_nents); + + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + if (unlikely(ret < 0)) + return ret; + ovpn_aead_encrypt_header(peer, ks, pktid, iv, aad); + + ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); + if (unlikely(ret < 0)) + return ret; + + /* encrypt out of place from the original segmented skb directly into + * its final range in the UDP GSO skb + */ + sg_init_table(src, src_nents); + sg_set_buf(src, aad, OVPN_AAD_SIZE); + mapped = skb_to_sgvec_nomark(skb, src + 1, 0, skb->len); + if (unlikely(mapped < 0)) + return mapped; + sg_mark_end(&src[mapped]); + + sg_init_table(dst, dst_nents); + dst_idx = skb_to_sgvec_nomark(gso_skb, dst, offset, OVPN_AAD_SIZE); + if (unlikely(dst_idx < 0)) + return dst_idx; + + payload_off = offset + OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE; + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, payload_off, + skb->len); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, + offset + OVPN_AAD_SIZE, + OVPN_AUTH_TAG_SIZE); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + sg_mark_end(&dst[dst_idx - 1]); + + aead_request_set_tfm(req, ks->encrypt); + aead_request_set_callback(req, 0, ovpn_encrypt_post, skb); + aead_request_set_crypt(req, src, dst, skb->len, iv); + aead_request_set_ad(req, OVPN_AAD_SIZE); + + return crypto_aead_encrypt(req); +} + int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { diff --git a/drivers/net/ovpn/crypto_aead.h b/drivers/net/ovpn/crypto_aead.h index fae3b585a43b..8b444744944e 100644 --- a/drivers/net/ovpn/crypto_aead.h +++ b/drivers/net/ovpn/crypto_aead.h @@ -17,6 +17,10 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + struct sk_buff *skb, struct sk_buff *gso_skb, + unsigned int offset); int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 39bbd464e437..dbf738cb5cbc 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include "ovpnpriv.h" #include "peer.h" @@ -32,6 +34,12 @@ const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE] = { 0x07, 0xed, 0x2d, 0x0a, 0x98, 0x1f, 0xc7, 0x48 }; +/* Leave room for the largest outer network header. The strict inequality in + * is_skb_forwardable also requires staying one byte below gso_max_size. + */ +#define OVPN_UDP_GSO_MAX_PAYLOAD (GSO_LEGACY_MAX_SIZE - \ + sizeof(struct ipv6hdr) - \ + sizeof(struct udphdr) - 1) /** * ovpn_is_keepalive - check if skb contains a keepalive message * @skb: packet to check @@ -236,11 +244,13 @@ void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb) void ovpn_encrypt_post(void *data, int ret) { + unsigned int orig_len, packets = 1; struct ovpn_crypto_key_slot *ks; struct sk_buff *skb = data; struct ovpn_socket *sock; + struct ovpn_cb *batch_cb; struct ovpn_peer *peer; - unsigned int orig_len; + struct sk_buff *batch; /* encryption is happening asynchronously. This function will be * called later by the crypto callback with a proper return value @@ -248,15 +258,19 @@ void ovpn_encrypt_post(void *data, int ret) if (unlikely(ret == -EINPROGRESS)) return; - ks = ovpn_skb_cb(skb)->ks; + /* ordinary encryption leaves batch zeroed; a GSO input uses it to find + * the aggregate whose lifetime is shared by all segment requests + */ + batch = ovpn_skb_cb(skb)->batch; peer = ovpn_skb_cb(skb)->peer; + ks = ovpn_skb_cb(skb)->ks; /* crypto is done, cleanup skb CB and its members */ kfree(ovpn_skb_cb(skb)->crypto_tmp); if (unlikely(ret == -ERANGE)) { /* we ran out of IVs and we must kill the key as it can't be - * use anymore + * used anymore */ netdev_warn(peer->ovpn->dev, "killing key %u for peer %u\n", ks->key_id, @@ -264,8 +278,33 @@ void ovpn_encrypt_post(void *data, int ret) if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) /* let userspace know so that a new key must be negotiated */ ovpn_nl_key_swap_notify(peer, ks->key_id); + } - goto err; + if (batch) { + batch_cb = ovpn_skb_cb(batch); + /* every segment publishes its result before releasing its + * pending count and only the final completion continues with + * the aggregate + */ + if (unlikely(ret < 0)) { + atomic_set(&batch_cb->batch_state.failed, 1); + kfree_skb(skb); + } else { + /* the encrypted data now lives in the aggregate */ + consume_skb(skb); + } + if (!atomic_dec_and_test(&batch_cb->batch_state.pending)) + return; + + skb = batch; + packets = skb_shinfo(batch)->gso_segs; + if (unlikely(atomic_read(&batch_cb->batch_state.failed))) + goto err; + + /* reaching the final callback with no sticky failure means + * every segment completed successfully + */ + ret = 0; } if (unlikely(ret < 0)) @@ -291,7 +330,7 @@ void ovpn_encrypt_post(void *data, int ret) goto err_unlock; } - ovpn_peer_stats_increment_tx(&peer->link_stats, orig_len); + ovpn_peer_stats_add_tx(&peer->link_stats, orig_len, packets); /* keep track of last sent packet for keepalive */ WRITE_ONCE(peer->last_sent, ktime_get_boottime_seconds()); /* skb passed down the stack - don't free it */ @@ -300,36 +339,131 @@ void ovpn_encrypt_post(void *data, int ret) rcu_read_unlock(); err: if (unlikely(skb)) - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, packets); kfree_skb(skb); if (likely(ks)) ovpn_crypto_key_slot_put(ks); ovpn_peer_put(peer); } -static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +/* Hold the peer and its primary key for one encryption submission. + * The returned key and the peer each carry one reference which completion must + * release. + */ +static struct ovpn_crypto_key_slot * +ovpn_encrypt_refs_get(struct ovpn_peer *peer) { struct ovpn_crypto_key_slot *ks; /* get primary key to be used for encrypting data */ ks = ovpn_crypto_key_slot_primary(&peer->crypto); if (unlikely(!ks)) - return false; + return NULL; - /* take a reference to the peer because the crypto code may run async. - * ovpn_encrypt_post() will release it upon completion + /* the caller already owns a peer reference, so failure indicates a + * broken reference lifetime elsewhere */ if (unlikely(!ovpn_peer_hold(peer))) { DEBUG_NET_WARN_ON_ONCE(1); ovpn_crypto_key_slot_put(ks); - return false; + return NULL; } + return ks; +} + +static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct ovpn_crypto_key_slot *ks; + + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_encrypt_post(skb, ovpn_aead_encrypt(peer, ks, skb)); return true; } +static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, + struct ovpn_peer *peer, + struct sk_buff *batch, + unsigned int segments) +{ + unsigned int offset = 0, i, len; + struct ovpn_crypto_key_slot *ks; + struct sk_buff *skb; + + /* acquire all shared state before removing the first input skb so that + * failure can leave the queue intact for the ordinary transmit path + */ + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + + /* the aggregate owns these references until every sync or async crypto + * completion has finished + */ + memset(ovpn_skb_cb(batch), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(batch)->peer = peer; + ovpn_skb_cb(batch)->ks = ks; + atomic_set(&ovpn_skb_cb(batch)->batch_state.pending, segments); + atomic_set(&ovpn_skb_cb(batch)->batch_state.failed, 0); + + for (i = 0; i < segments; i++) { + skb = __skb_dequeue(skbs); + len = skb->len + OVPN_DATA_V2_OVERHEAD; + + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(skb)->batch = batch; + ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, + batch, offset)); + offset += len; + } + + return true; +} + +static struct sk_buff *ovpn_udp_gso_alloc(const struct sk_buff *first_segment, + unsigned int batch_len, + unsigned int segments) +{ + struct sk_buff *gso_skb; + int ret; + + gso_skb = alloc_skb_with_frags(OVPN_HEAD_ROOM, batch_len, + SKB_FRAG_PAGE_ORDER, &ret, GFP_ATOMIC); + if (unlikely(!gso_skb)) + return NULL; + + skb_reserve(gso_skb, OVPN_HEAD_ROOM); + gso_skb->len = batch_len; + gso_skb->data_len = batch_len; + gso_skb->priority = first_segment->priority; + + /* Segments retain the originating socket so we keep its send-buffer + * accounting active until the UDP GSO is transmitted. This also + * preserves its cached TX queue. + */ + if (first_segment->sk && is_skb_wmem(first_segment)) + skb_set_owner_w(gso_skb, first_segment->sk); + skb_copy_hash(gso_skb, first_segment); +#ifdef CONFIG_XPS + /* keep the aggregate on the TX queue selected for the original flow + * otherwise async crypto completion on another CPU could move the flow + * to a different queue and cause delay or reordering + */ + gso_skb->sender_cpu = first_segment->sender_cpu; +#endif + + skb_shinfo(gso_skb)->gso_type = SKB_GSO_UDP_L4; + skb_shinfo(gso_skb)->gso_size = first_segment->len + + OVPN_DATA_V2_OVERHEAD; + skb_shinfo(gso_skb)->gso_segs = segments; + + return gso_skb; +} + /* send skb to connected peer, if any */ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, struct ovpn_peer *peer) @@ -341,7 +475,7 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, */ skb_list_walk_safe(skb, curr, next) { if (unlikely(!ovpn_encrypt_one(peer, curr))) { - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); } } @@ -349,19 +483,96 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, ovpn_peer_put(peer); } +/* encrypt fixed-size input segments into one or more UDP GSO aggregates */ +static void ovpn_send_gso(struct sk_buff_head *skbs, struct ovpn_peer *peer) +{ + unsigned int max_segs = 1, seg_len, batch_len, segs; + struct sk_buff *batch; + + seg_len = skb_peek(skbs)->len + OVPN_DATA_V2_OVERHEAD; + max_segs = min_t(unsigned int, UDP_MAX_SEGMENTS, + OVPN_UDP_GSO_MAX_PAYLOAD / seg_len); + + /* if even two encrypted skbs cannot fit, leave the whole queue for the + * ordinary transmit path + */ + if (max_segs < 2) + return; + + /* An input GSO skb might produce more than max_segs segments, so we + * consume as many as we can for each iteration. A final single skb, or + * the whole remainder after an allocation failure, stays queued and + * fallback to the ordinary transmit path. + */ + while (skb_queue_len(skbs) > 1) { + segs = min_t(unsigned int, skb_queue_len(skbs), max_segs); + + /* all but the final input skb have the same length, so start + * with the full-size calculation and adjust only the final + * group below + */ + batch_len = segs * (skb_peek(skbs)->len + + OVPN_DATA_V2_OVERHEAD); + if (segs == skb_queue_len(skbs)) + batch_len -= skb_peek(skbs)->len - + skb_peek_tail(skbs)->len; + + batch = ovpn_udp_gso_alloc(skb_peek(skbs), batch_len, segs); + if (unlikely(!batch)) + return; + + if (unlikely(!ovpn_encrypt_gso_queue(skbs, peer, + batch, segs))) { + kfree_skb(batch); + return; + } + } +} + +static bool ovpn_peer_supports_udp_gso(struct ovpn_peer *peer) +{ + struct ovpn_socket *sock; + bool udp_gso; + + rcu_read_lock(); + sock = rcu_dereference(peer->sock); + /* UDP GSO requires checksums. These socket settings can change after + * we decide to batch, but an already-built batch remains checksummed. + * Linux's ordinary UDP GSO path makes the same choice. + */ + udp_gso = sock && sock->sk->sk_protocol == IPPROTO_UDP && + !sock->sk->sk_no_check_tx && !udp_get_no_check6_tx(sock->sk); + rcu_read_unlock(); + + return udp_gso; +} + /* Send user data to the network */ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) { struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; + const bool gso_in = skb_is_gso(skb); struct sk_buff_head skb_list; netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; + bool gso_out; __be16 proto; int ret; + /* A frag-list GSO skb already stores complete segments as child skbs. + * Keep those children on the ordinary in-place encryption path instead + * of copying them into a replacement UDP GSO skb. + * + * GSO_BY_FRAGS input must remain on that path too because its variable + * segment sizes cannot be represented by one UDP GSO output size. + */ + gso_out = gso_in && + !(skb_shinfo(skb)->gso_type & SKB_GSO_FRAGLIST) && + skb_shinfo(skb)->gso_size != GSO_BY_FRAGS; + /* reset netfilter state */ nf_reset_ct(skb); @@ -390,13 +601,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) /* dst was needed for peer selection - it can now be dropped */ skb_dst_drop(skb); - if (skb_is_gso(skb)) { - /* force software segmentation, but keep ovpn's non-GSO feature - * bits so the generated segments can preserve non-linear skb - * data where possible + if (gso_in) { + /* force software segmentation and calculate each checksum + * while allowing safe page-backed payload fragments to remain + * non-linear */ features = netif_skb_features(skb); - segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + features &= ~(NETIF_F_GSO_MASK | NETIF_F_CSUM_MASK); + segments = skb_gso_segment(skb, features); if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", @@ -418,7 +630,8 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) if (unlikely(!curr)) { net_err_ratelimited("%s: skb_share_check failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); + gso_out = false; continue; } @@ -427,8 +640,9 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_checksum_help(curr) < 0)) { net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); + gso_out = false; continue; } @@ -444,9 +658,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) ovpn_peer_put(peer); return NETDEV_TX_OK; } - skb_list.prev->next = NULL; ovpn_peer_stats_increment_tx(&peer->vpn_stats, tx_bytes); + + if (gso_out && skb_queue_len(&skb_list) > 1 && + ovpn_peer_supports_udp_gso(peer)) + ovpn_send_gso(&skb_list, peer); + + skb_list.prev->next = NULL; ovpn_send(ovpn, skb_list.next, peer); return NETDEV_TX_OK; @@ -454,7 +673,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) drop: ovpn_peer_put(peer); drop_no_peer: - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); skb_tx_error(skb); kfree_skb_list(skb); return NETDEV_TX_OK; diff --git a/drivers/net/ovpn/skb.h b/drivers/net/ovpn/skb.h index 4fb7ea025426..cca29479c038 100644 --- a/drivers/net/ovpn/skb.h +++ b/drivers/net/ovpn/skb.h @@ -10,6 +10,7 @@ #ifndef _NET_OVPN_SKB_H_ #define _NET_OVPN_SKB_H_ +#include #include #include #include @@ -20,19 +21,35 @@ /** * struct ovpn_cb - ovpn skb control block - * @peer: the peer this skb was received from/sent to - * @ks: the crypto key slot used to encrypt/decrypt this skb * @crypto_tmp: pointer to temporary memory used for crypto operations * containing the IV, the scatter gather list and the aead request + * @peer: peer used by this crypto operation or owned by this aggregate + * @ks: crypto key slot used by this operation or owned by this aggregate * @payload_offset: offset in the skb where the payload starts * @nosignal: whether this skb should be sent with the MSG_NOSIGNAL flag (TCP) + * @batch: UDP GSO aggregate receiving this input skb's encrypted payload + * @batch_state: completion state owned by a UDP GSO aggregate */ struct ovpn_cb { + void *crypto_tmp; struct ovpn_peer *peer; struct ovpn_crypto_key_slot *ks; - void *crypto_tmp; - unsigned int payload_offset; - bool nosignal; + + /* Ordinary encryption leaves this union zeroed. Decryption and TCP use + * their ordinary fields, a UDP GSO input stores its output aggregate, + * and that aggregate uses the same space to coordinate its completions. + */ + union { + struct { + unsigned int payload_offset; + bool nosignal; + }; + struct sk_buff *batch; + struct { + atomic_t pending; + atomic_t failed; + } batch_state; + }; }; static inline struct ovpn_cb *ovpn_skb_cb(struct sk_buff *skb) diff --git a/drivers/net/ovpn/stats.h b/drivers/net/ovpn/stats.h index 3a45b97c0056..b3fe006c01f6 100644 --- a/drivers/net/ovpn/stats.h +++ b/drivers/net/ovpn/stats.h @@ -40,16 +40,26 @@ static inline void ovpn_peer_stats_increment_rx(struct ovpn_peer_stats *stats, ovpn_peer_stats_increment(&stats->rx, n); } +static inline void ovpn_peer_stats_add_tx(struct ovpn_peer_stats *stats, + const unsigned int bytes, + unsigned int packets) +{ + atomic64_add(bytes, &stats->tx.bytes); + atomic64_add(packets, &stats->tx.packets); +} + static inline void ovpn_peer_stats_increment_tx(struct ovpn_peer_stats *stats, const unsigned int n) { - ovpn_peer_stats_increment(&stats->tx, n); + ovpn_peer_stats_add_tx(stats, n, 1); } -static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev) +static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev, + unsigned int packets) { local_bh_disable(); - dev_dstats_tx_dropped(dev); + while (packets--) + dev_dstats_tx_dropped(dev); local_bh_enable(); } diff --git a/drivers/net/ovpn/tcp.c b/drivers/net/ovpn/tcp.c index 8fe8a8e750a4..5cba35e4a8ee 100644 --- a/drivers/net/ovpn/tcp.c +++ b/drivers/net/ovpn/tcp.c @@ -332,7 +332,7 @@ static void ovpn_tcp_send_sock_skb(struct ovpn_peer *peer, struct sock *sk, ovpn_tcp_send_sock(peer, sk); if (peer->tcp.out_msg.skb) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); return; } @@ -354,7 +354,7 @@ void ovpn_tcp_send_skb(struct ovpn_peer *peer, struct sock *sk, if (sock_owned_by_user(sk)) { if (skb_queue_len(&peer->tcp.out_queue) >= READ_ONCE(net_hotdata.max_backlog)) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); goto unlock; } diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 055cdb1bee13..a000401ed543 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -121,6 +121,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); + skb_mark_not_on_list(skb); ovpn_recv(peer, skb); return 0; @@ -288,9 +289,13 @@ static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind, spin_unlock_bh(&peer->lock); transmit: + /* an already-built UDP GSO needs a checksum seed even if the socket's + * no-check option changed while encryption was in flight + */ udp_tunnel_xmit_skb(rt, sk, skb, fl.saddr, fl.daddr, 0, ip4_dst_hoplimit(&rt->dst), 0, fl.fl4_sport, - fl.fl4_dport, false, sk->sk_no_check_tx, 0); + fl.fl4_dport, false, + !skb_is_gso(skb) && sk->sk_no_check_tx, 0); ret = 0; err: local_bh_enable(); @@ -388,9 +393,13 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind, * udp_tunnel_xmit_skb() */ skb->ignore_df = 1; + /* keep checksum offload enabled for an in-flight UDP GSO batch even if + * the socket's no-check option has changed since batch creation + */ udp_tunnel6_xmit_skb(dst, sk, skb, skb->dev, &fl.saddr, &fl.daddr, 0, ip6_dst_hoplimit(dst), 0, fl.fl6_sport, - fl.fl6_dport, udp_get_no_check6_tx(sk), 0); + fl.fl6_dport, + !skb_is_gso(skb) && udp_get_no_check6_tx(sk), 0); ret = 0; err: local_bh_enable(); @@ -467,8 +476,18 @@ void ovpn_udp_send_skb(struct ovpn_peer *peer, struct sock *sk, skb->dev = peer->ovpn->dev; skb->mark = key.mark; - /* no checksum performed at this layer */ - skb->ip_summed = CHECKSUM_NONE; + if (skb_is_gso(skb)) { + /* udp_tunnel_xmit_skb installs the outer UDP header after this + * function returns: point CHECKSUM_PARTIAL at that future + * header so both hw and sw UDP GSO can complete the checksum. + */ + skb->ip_summed = CHECKSUM_PARTIAL; + skb->csum_start = skb_headroom(skb) - sizeof(struct udphdr); + skb->csum_offset = offsetof(struct udphdr, check); + } else { + /* no checksum performed at this layer */ + skb->ip_summed = CHECKSUM_NONE; + } ovpn_dst_cache_check_key(peer, &peer->dst_cache, &key); From patchwork Mon Nov 2 16:43:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5560 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673367laa; Mon, 2 Nov 2026 08:44:22 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBzzqoV+3lpiFzvCJJETSFH1+CsnM+yxWEDK871aSELkve4z/R6CeTzucdBGAHr0f0C1Dy31nwHBh6A=@openvpn.net X-Received: by 2002:a05:701a:ca8a:b0:19a:e0eb:6916 with SMTP id a92af1059eb24-1ac5de540bbmr17310483c88.42.1793637861783; Mon, 02 Nov 2026 08:44:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637861; cv=none; d=google.com; s=arc-20260327; b=J+7hGTt9VCOCuQX3Yx20J1dFH7aSc8AtMtxK7T4KRyhvxQSTQzpO6pK4aEiMNIO9m2 0CZsyDK4LHspd7wGW1g4i4yqtXIicowA8ht+B5Ky2v5UXc74+l7bo83bd61TMsV2t17g IMxS/bHLKJ0dTfHRRq9T7aixGomcyEiPNDy8LOjyNzLKbrhOag7v6gG9smn1jfQmk0cU +pRHfxr0gWqEw1KV6zIV6gqmJtX7pmkwkjxpfnbDKGYoD47hETlcq76CDhqQIqbdE6QV g5blI4yGI5QW/L3BkQX1Lo3Ta8AzRIExtNnmq8srjuZbUKDLd/lfFU2I4e6ekebRA0zI aSCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=qBdmWVgpydFoSBXoK7KYNjjPctlFKvUh3tp8faLaZS0=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=QLWkRcO5SsAO0B5lp98bvtsrPp8Xr3cu7HPt0aBTeB/nDukpLIk53BMACDYDzjkXQo XNXpnqCfJSJbfwgY1eL02ZqYbXI292YDpHo0y4C5jc/RwpCiytUWAgMf/MWID/GgPgbn D+h0RjTnrEhn/fwQS8gneA0Di0UYI3YsNE4s4J/QUmLRWOnXtB56mkpl6uXojXrsDi6y h1DZRvEfVuTzQX2wDxXpwJowfrlRiz4t+IxyvxV010zniXScy4yvZdb3k2LltsE0xpY/ GWP+6MErvKZJbN4ZJqM3P5ueCXe3mbT3WLE47ot4ndp8gF4lDKsnF7cKFfQ08crVCVuY irQw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=TA2C5D27; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=hi1hY6KP; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=OaYMV7Hy; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=VxPCahtW; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id a92af1059eb24-1b21861d698si8945821c88.25.2026.11.02.08.44.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:21 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=TA2C5D27; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=hi1hY6KP; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=OaYMV7Hy; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=VxPCahtW; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=qBdmWVgpydFoSBXoK7KYNjjPctlFKvUh3tp8faLaZS0=; b=TA2C5D27wXA1P9 Zk+LIWdwWMVy8RgDL4pUcZkd9ut0zdslHCV30LlOjZgxwCowY60OqKmXDOUlaPwIRdFjpVtVu264M d7JsxTuqfDcNWcXUTEQb9rUg49gnYKK6FNYDiEnx5QB7OTK9hCFtoLsXy3xl8dLoQkX1iTTVyZBn6 t7U=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8u-000000000hu-2Yhd; Mon, 02 Nov 2026 16:44:20 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8i-000000000hI-49Yg for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=QJ2ZZ2KALVg267tFNw0uTTLIn//fsEjFdURO8gQVGPs=; b=hi1hY6KPq8675KAV8lEylTS4Li /BeTJztinGBWW8/7rBnwNJkvvad516waNjhs28nWQd0jRrDuJcDFnSGUlojhUFL6gI4jQwHXPe4a5 YQwtFrXuPEqVKw9SsRdmh8v7K4CphjQyAItn4geOCyZioz77orffMRvbzP8TwV0/OYRI=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=QJ2ZZ2KALVg267tFNw0uTTLIn//fsEjFdURO8gQVGPs=; b=OaYMV7HyvRXGZPknSat9esI+EC 6iaTOD62RQTmwPlq32h7iZfVwvt0jL+Tv0u04EWuamPZ1lLpXWem/LVojeZBXPjSkUEephjtmkNWB sSahPDd3zfbvnhGxsTtPEfNCzFyKf1CnIfivc/Hw8wyrQ0POiYCeA+5/S9KRItBce1nA=; Received: from mout-b-210.mailbox.org ([195.10.208.40]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8h-000000005hN-2mSW for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:08 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-210.mailbox.org (Postfix) with ESMTPS id 4jJHbM5217zFqY4 for ; Mon, 02 Nov 2026 17:43:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=QJ2ZZ2KALVg267tFNw0uTTLIn//fsEjFdURO8gQVGPs=; b=VxPCahtWBNucBQ3wSCS+srYU2eiXHwmJNVkIm2XZYryR++j5emEH/mP/9QUs2bvWiDOPTt Ebouy9azmfLgJ+LBsUuyD4l07VhpkRb44suZyrfcqA7C3z2uFogzubst9q0zmFhaH3Wefc HNSgbtvMjlfi6ad7AyR3PS53eRjsM4fh6RZzNzMw6erU9zCIAr0f63iBiO+W3kLPY4fVRV ZI7vMX3+F5+0A0L5ro8nCiTSD8yl13nlMRUdDndSn/cweegtzUf08b2fVPrpxbKHs5JGQX U/0SNuF1Z3EOYjRGR72gOzHNjzLsIYgO+7kTwIyjr4k3UsyyutCMowDBxHrxQw== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:47 +0100 Message-ID: <20261102164353.459969-6-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbM5217zFqY4 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: The UDP GSO path currently allocates one packet ID for every input segment. This performs one atomic update per segment even though the whole batch is prepared by a single caller. Reserve the complete consecutive range before dequeuing any input skb, then assign its IDs to segments in queue order. This reduces packet-ID allocation to one atomic operation per batch and leaves th [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xNv8h-000000005hN-2mSW Subject: [Openvpn-devel] [PATCH ovpn net-next v2 5/9] ovpn: reserve packet-ID ranges for UDP GSO batches X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765614316541918 X-GMAIL-MSGID: 1880765614316541918 The UDP GSO path currently allocates one packet ID for every input segment. This performs one atomic update per segment even though the whole batch is prepared by a single caller. Reserve the complete consecutive range before dequeuing any input skb, then assign its IDs to segments in queue order. This reduces packet-ID allocation to one atomic operation per batch and leaves the input queue intact when no complete range remains, allowing the ordinary transmit path to retry with a newly installed key. If the requested range would cross the end of the packet-ID space, publish exhaustion immediately. Packet IDs abandoned after a later encryption failure remain unused, so an AEAD IV is never reused. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/a7138d54319d501b6ba7f1d50dce65cb8d2c3e46.1790843153.git.ralf@mandelbit.com/ drivers/net/ovpn/crypto_aead.c | 7 ++--- drivers/net/ovpn/crypto_aead.h | 2 +- drivers/net/ovpn/io.c | 32 ++++++++++++++++++----- drivers/net/ovpn/pktid.h | 48 ++++++++++++++++++++++++++++++++++ 4 files changed, 76 insertions(+), 13 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 76028f048606..1c5387f83eca 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -260,7 +260,8 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, - struct sk_buff *gso_skb, unsigned int offset) + struct sk_buff *gso_skb, unsigned int offset, + u32 pktid) { const unsigned int dst_nents = skb_shinfo(gso_skb)->nr_frags + 4; const unsigned int src_nents = skb_shinfo(skb)->nr_frags + 2; @@ -268,7 +269,6 @@ int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, struct aead_request *req; unsigned int payload_off; int dst_idx, mapped, ret; - u32 pktid; u8 *aad, *iv; /* each input records the shared peer and key for the common completion @@ -285,9 +285,6 @@ int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, dst = src + src_nents; aad = (u8 *)(dst + dst_nents); - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); - if (unlikely(ret < 0)) - return ret; ovpn_aead_encrypt_header(peer, ks, pktid, iv, aad); ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); diff --git a/drivers/net/ovpn/crypto_aead.h b/drivers/net/ovpn/crypto_aead.h index 8b444744944e..92e8ed76fba6 100644 --- a/drivers/net/ovpn/crypto_aead.h +++ b/drivers/net/ovpn/crypto_aead.h @@ -20,7 +20,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, struct sk_buff *gso_skb, - unsigned int offset); + unsigned int offset, u32 pktid); int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index dbf738cb5cbc..0f877be9807b 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -242,6 +242,16 @@ void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb) ovpn_decrypt_post(skb, ovpn_aead_decrypt(peer, ks, skb)); } +static void ovpn_encrypt_key_exhausted(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks) +{ + netdev_warn(peer->ovpn->dev, "killing key %u for peer %u\n", + ks->key_id, peer->id); + if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) + /* let userspace know so that a new key must be negotiated */ + ovpn_nl_key_swap_notify(peer, ks->key_id); +} + void ovpn_encrypt_post(void *data, int ret) { unsigned int orig_len, packets = 1; @@ -272,12 +282,7 @@ void ovpn_encrypt_post(void *data, int ret) /* we ran out of IVs and we must kill the key as it can't be * used anymore */ - netdev_warn(peer->ovpn->dev, - "killing key %u for peer %u\n", ks->key_id, - peer->id); - if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) - /* let userspace know so that a new key must be negotiated */ - ovpn_nl_key_swap_notify(peer, ks->key_id); + ovpn_encrypt_key_exhausted(peer, ks); } if (batch) { @@ -393,6 +398,7 @@ static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, unsigned int offset = 0, i, len; struct ovpn_crypto_key_slot *ks; struct sk_buff *skb; + u32 first_pktid; /* acquire all shared state before removing the first input skb so that * failure can leave the queue intact for the ordinary transmit path @@ -401,6 +407,17 @@ static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, if (unlikely(!ks)) return false; + /* reserve the complete range before dequeuing any input skb so that a + * failure leaves the ordinary transmit path available as a fallback + */ + if (unlikely(ovpn_pktid_xmit_range(&ks->pid_xmit, segments, + &first_pktid))) { + ovpn_encrypt_key_exhausted(peer, ks); + ovpn_crypto_key_slot_put(ks); + ovpn_peer_put(peer); + return false; + } + /* the aggregate owns these references until every sync or async crypto * completion has finished */ @@ -417,7 +434,8 @@ static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_skb_cb(skb)->batch = batch; ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, - batch, offset)); + batch, offset, + first_pktid + i)); offset += len; } diff --git a/drivers/net/ovpn/pktid.h b/drivers/net/ovpn/pktid.h index 21845f353bc8..c922e11da6a0 100644 --- a/drivers/net/ovpn/pktid.h +++ b/drivers/net/ovpn/pktid.h @@ -68,6 +68,54 @@ static inline int ovpn_pktid_xmit_next(struct ovpn_pktid_xmit *pid, u32 *pktid) return 0; } +/** + * ovpn_pktid_xmit_range - reserve consecutive packet IDs + * @pid: transmit packet-ID state + * @count: non-zero number of IDs to reserve + * @pktid: first reserved packet ID + * + * The reservation is all-or-nothing. If the requested range would cross the + * end of the 32-bit packet-ID space, exhaust the state without returning the + * unused tail. Packet IDs can be skipped but never reused as an AEAD IV. + * + * Return: 0 on success or -ERANGE when no complete range remains + */ +static inline int ovpn_pktid_xmit_range(struct ovpn_pktid_xmit *pid, + unsigned int count, u32 *pktid) +{ + int old = atomic_read(&pid->seq_num); + u32 first, next; + bool fits; + + for (;;) { + first = (u32)old; + next = first + count; + + /* space is already exhausted */ + if (unlikely(!first)) + return -ERANGE; + + fits = count - 1 <= U32_MAX - first; + /* if it doesn't fit we still have to publish exhaustion (0) + * so concurrent allocators stop using this key + */ + if (unlikely(!fits)) + next = 0; + + /* if another CPU changed the state, we retry using the new + * seq_num value + */ + if (!atomic_try_cmpxchg(&pid->seq_num, &old, (int)next)) + continue; + + if (unlikely(!fits)) + return -ERANGE; + + *pktid = first; + return 0; + } +} + /* Write 12-byte AEAD IV to dest */ static inline void ovpn_pktid_aead_write(const u32 pktid, const u8 nt[], From patchwork Mon Nov 2 16:43:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5564 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673443laa; Mon, 2 Nov 2026 08:44:25 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBx6cSRjA2tuoID5oc1NW8pklY9K8ZqVc3+0dKJz9BdlEms2iY8+JGGqYl5LMh2TNZl8j9qIf31b3Eo=@openvpn.net X-Received: by 2002:a05:7301:fc02:b0:368:57fe:305c with SMTP id 5a478bee46e88-36a4238bb87mr17053334eec.16.1793637864628; Mon, 02 Nov 2026 08:44:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637864; cv=none; d=google.com; s=arc-20260327; b=LRLfQ/psw0rzPQcjxenhIR2txmXsRjYSxFSG8M3pi3clBD+/wid6Eq7uZK8GruZRFF J3pUu9PmyKLJgwqNO1WHVN/dVlyeo19qCdmsuqzeYcYVb+WS3uJmkJwuuvIHI53k8cWv G3QTSn+gs6XhalnyEQfCrUOaM0YWmpGX1WYEScUBDxHrM9HiEtQGqMktoUr4AkZ683qA wbCIBdP87KuM53no/OeMnzVFkOQaYgwwcxQmEXMC+cOWkaRzC8mpLZrKiAen8EIVKxAw vSLEsU3yQCaet8YJ6eMF2w4bUUw/kvzKPm4AQj5+/60ZILhrKK2RgpcToAIUE9nDcf4V HtVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=YFy7Apk4sI36o22QkbjWgIG+xSne4gcEp0roc7Bot1c=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=pbuYtu7ermuayErC2BRfDYLqD+jM4t2aus6OsCVouJxCmNBgQjERSkEnsyo5WHd8vD Z1/wfEeRIYgdcooxopF22ejToF3lcJhlXFoqxTPMfxgem/FvN1fDEsGGPBxQtIlrd5KG Vm7snHv+lRvs8h4BZtStpXcqT942chv6RnhseqHgpREi0wzgE3ie/x4KLYZHRW9nrPWD T1XVVnmQm3Athnl87ifo9QkBQJFdDDl0wEwhOuntwa69V4cd3bqHHO2RgxcLCnAK4uK/ za81qVkR8M4U56IvskWaGeKUn3R7JtOABOX97HESTCQfv1FMFvsXGpOa4p7vBRDVuGFk gC7A==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=RTwx8p8G; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=lrocqGd0; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=SSu3852m; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=ZaaxmICa; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5a478bee46e88-36a4281b0e1si31823300eec.12.2026.11.02.08.44.24 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:24 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=RTwx8p8G; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=lrocqGd0; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=SSu3852m; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=ZaaxmICa; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=YFy7Apk4sI36o22QkbjWgIG+xSne4gcEp0roc7Bot1c=; b=RTwx8p8GjNovnJ daB3e+l9GLiCEkPItQXgpg9tPIiUsLBR3EYmvbrM5tOuA6QIuvt0cU10YfXTJcYbE7sPq34SpqqIJ LGfdAnTWS/XSAQ2g5h/cf5iopTQ8FmLcMfdbEb2Gan8KUSvtjGrTCnR9KDariCFxbppRwbVXLr4xB JGs=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8x-000000002ut-25sU; Mon, 02 Nov 2026 16:44:23 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8k-000000002uI-1iA7 for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=F0fYOdr11VdzVW/OCy0NoSfqna0l63USk5YISmx+6eA=; b=lrocqGd0NSKOcqRNGV3G4NpJ13 xma2MAdtIhu1jPSz5rFxM8Y1QlszqVpA6cGTwgsGC98iETHnNEgX4urTyOvZQ2Qn3aXP+aqYlJgMp E3UOYVqh9GSTnqxRYCPp8DB1s2sgVsLOZrDPAZ+JxyOgFP+hPQs0SAIL4Mj/COqNFwYs=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=F0fYOdr11VdzVW/OCy0NoSfqna0l63USk5YISmx+6eA=; b=SSu3852mm9HLZcWQwM6D7uJybZ FkosX7CkUwBozJMff4BeI4mK1QA8VWZmXu/vnxb9yrgF27/xblY2WidCtK/tjr29oNtZE69r1crSM vg8Y3FhHsp9V8GpF0OVjLlI3dwqXZWJ8PuRvOmOz5pHseihGfSRNZ5TNXcR8kUFkxkhM=; Received: from mout-b-203.mailbox.org ([195.10.208.52]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8i-000000005hO-1m00 for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:10 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA512) (No client certificate requested) by mout-b-203.mailbox.org (Postfix) with ESMTPS id 4jJHbN0JNhzLlsX for ; Mon, 02 Nov 2026 17:44:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=F0fYOdr11VdzVW/OCy0NoSfqna0l63USk5YISmx+6eA=; b=ZaaxmICaxDH/9BU9/wAtuRbsjPTVejZ7Xj25hce7fj3VkEG3Tm6rlXsBKdx47Nke6xBMZg fiFTrZxfvq/fhYEWDAdxbhBH/W7nvmME6bU2j83088JhesyDPpy+VkT1Zd5BjJNmi2PRY0 asYPp9IFo01Vx2klJwQlRhnd09PonP7rMdyN+/Obw13cGYbdVUUxc8aQx8Vz+ps1i9NsxL 6YYQB4Ssbt+G9m1GyCkFArgIMdRo90jUJ7rbHGh09u4Z2+p5DdenVXaOfXGK/K81Z76imn g1qVRqSDNzrn4pM54QZGBJl7V+kzoYUvx6Ldp/u0VWbWbVutGSi2UnZRJfmBcg== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:48 +0100 Message-ID: <20261102164353.459969-7-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbN0JNhzLlsX X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path ca [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain X-Headers-End: 1xNv8i-000000005hO-1m00 Subject: [Openvpn-devel] [PATCH ovpn net-next v2 6/9] ovpn: coalesce UDP data records with GRO X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765617571921544 X-GMAIL-MSGID: 1880765617571921544 Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path can detach and authenticate records independently. Match the complete opcode/key/peer header and require compatible outer- network and checksum state. Flush on short records, differing segment geometry, existing GSO input, or the 64-record limit. Do not let a record which cannot join an aggregate overtake records of the same transport flow which GRO is still holding. Complete the held aggregate first; after an opcode, key ID or peer ID change, let the new record start the next aggregate. Export skb_gro_receive_list, which is already shared by the core UDP and TCP frag-list GRO paths, so modular ovpn can use the same primitive instead of maintaining a local copy. Packet taps may clone a completed aggregate before UDP receives it. Making the aggregate head private retains shared references to its child skbs, so also clone and unshare each shared child before detaching the list and handing its records to crypto. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without receive GRO 19.279 Gbit/s 19.819 Gbit/s With frag-list GRO 22.773 Gbit/s 22.796 Gbit/s The preceding UDP GSO transmit path and hardware UDP segmentation were enabled in both cases. Frag-list GRO improved throughput by 18.1% forward and 15.0% reverse. Signed-off-by: Ralf Lici --- Changes since v1 https://lore.kernel.org/openvpn-devel/0431607b16294b9521e908f4d29a524698e39bb7.1790843153.git.ralf@mandelbit.com/ - Complete a held GRO aggregate before delivering an unmergeable record, and let a DATA_V2 record with a changed header start the next aggregate. (Antonio) drivers/net/ovpn/io.c | 7 +- drivers/net/ovpn/udp.c | 216 ++++++++++++++++++++++++++++++++++++++++- net/core/gro.c | 1 + net/ipv4/udp_offload.c | 3 +- 4 files changed, 220 insertions(+), 7 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 0f877be9807b..debe3d4d5731 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -70,11 +70,10 @@ static void ovpn_netdev_write(struct ovpn_peer *peer, struct sk_buff *skb) unsigned int pkt_len; int ret; - /* - * GSO state from the transport layer is not valid for the tunnel/data - * path. Reset all GSO fields to prevent any further GSO processing - * from entering an inconsistent state. + /* the transport encapsulation and its GSO metadata do not describe the + * decrypted inner packet */ + skb->encapsulation = 0; skb_gso_reset(skb); /* we can't guarantee the packet wasn't corrupted before entering the diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index a000401ed543..0c3557d90a1e 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -11,8 +11,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -27,6 +29,211 @@ #include "socket.h" #include "udp.h" +/* like UDP and TCP frag-list GRO */ +#define OVPN_UDP_GRO_CNT_MAX 64 + +static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) +{ + const unsigned int offset = skb_gro_offset(skb); + + /* GRO replaces its frag0 pointer after holding an skb, so keep the + * openvpn header linear for later candidate comparisons + */ + if (!pskb_may_pull(skb, offset + OVPN_OPCODE_SIZE)) + return false; + + *header = get_unaligned_be32(skb->data + offset); + return true; +} + +static struct sk_buff *ovpn_udp_gro_receive_fraglist(struct sock *sk, + struct list_head *head, + struct sk_buff *skb) +{ + const unsigned int gso_size = skb_gro_len(skb); + struct sk_buff *p, *pp = NULL; + u32 header, header2; + int ret = 0, nhoff; + bool flush; + + /* A packet which cannot join an aggregate must not overtake records of + * the same flow which GRO is still holding. Complete those first, as + * the other GRO handlers do. + */ + if (!ovpn_udp_gro_header(skb, &header) || + FIELD_GET(OVPN_OPCODE_PKTTYPE_MASK, header) != OVPN_DATA_V2 || + skb_is_gso(skb)) { + NAPI_GRO_CB(skb)->flush = 1; + list_for_each_entry(p, head, list) + if (NAPI_GRO_CB(p)->same_flow) + return p; + return NULL; + } + + list_for_each_entry(p, head, list) { + if (!NAPI_GRO_CB(p)->same_flow) + continue; + + /* A different opcode, key ID or peer ID completes the held + * aggregate; this record can start the next one. + */ + if (!ovpn_udp_gro_header(p, &header2) || header != header2) + return p; + + /* GRO has already matched the outer addresses and UDP ports; + * check the remaining outer IP fields + */ + nhoff = skb_transport_offset(p) - + NAPI_GRO_CB(p)->network_offset; + flush = __gro_receive_network_flush(udp_hdr(skb), udp_hdr(p), p, + nhoff, false); + + /* The first record determines the nominal GSO size. A shorter + * final record may follow it, but a larger record cannot. + * Checksum metadata must also be uniform because the aggregate + * exposes only one checksum state. + */ + if (gso_size > skb_shinfo(p)->gso_size || flush || + skb->ip_summed != p->ip_summed || + skb->csum_level != p->csum_level) { + pp = p; + } else { + /* skb_gro_receive_list pulls the headers already + * processed by GRO before linking this skb to the + * record list so we have to manually preserve the + * outer network header location for later handling + */ + nhoff = NAPI_GRO_CB(skb)->network_offset; + skb_set_network_header(skb, nhoff); + ret = skb_gro_receive_list(p, skb); + } + + /* complete the aggregate if the append failed, or after + * appending a shorter final record, or after reaching the + * record-count limit + */ + if (ret || gso_size != skb_shinfo(p)->gso_size || + NAPI_GRO_CB(p)->count >= OVPN_UDP_GRO_CNT_MAX) + pp = p; + + return pp; + } + + return NULL; +} + +static int ovpn_udp_gro_complete(struct sock *sk, struct sk_buff *skb, + int nhoff) +{ + /* udp_gro_complete has already marked this as a UDP tunnel GSO packet. + * Keep that type so UDP passes the aggregate directly to the encap cb, + * where the original record skbs are detached. + */ + skb_shinfo(skb)->gso_segs = NAPI_GRO_CB(skb)->count; + + /* Each outer UDP checksum was either validated (or accepted in case of + * checksumless UDP) before its record was merged in + * skb_gro_checksum_validate_zero_check. + * The checksum in the aggregate cannot describe the concatenation of + * independent UDP payloads, so we preserve the validation result. + */ + skb->ip_summed = CHECKSUM_UNNECESSARY; + skb->csum_level = 0; + skb->csum_valid = 0; + + return 0; +} + +/* skb_gro_receive_list keeps the first openvpn record in 'skb' and links the + * remaining records through fraglist. Here we segment by detaching that list + * before delivering the records individually, and remove the child skbs from + * the head skb's length and memory accounting so the head describes only the + * first record again. + */ +static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) +{ + unsigned int data_len = 0, truesize = 0; + struct sk_buff *curr, *list; + int ret; + + /* IP reassembly may also use fraglist, but it is not a record batch */ + if (!skb_is_gso(skb) || + !(skb_shinfo(skb)->gso_type & + (SKB_GSO_UDP_TUNNEL | SKB_GSO_UDP_TUNNEL_CSUM))) + return NULL; + + if (unlikely(!skb_shinfo(skb)->frag_list)) + return NULL; + + /* packet taps may have cloned the aggregate before it reached UDP, so + * make shared info private before removing its fraglist + */ + ret = skb_unclone(skb, GFP_ATOMIC); + if (unlikely(ret)) + return ERR_PTR(ret); + + list = skb_shinfo(skb)->frag_list; + + for (curr = list; curr; curr = curr->next) { + data_len += curr->len; + truesize += curr->truesize; + } + + skb_shinfo(skb)->frag_list = NULL; + skb->len -= data_len; + skb->data_len -= data_len; + skb->truesize -= truesize; + + return list; +} + +static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct sk_buff *list, *next; + + list = ovpn_udp_gro_detach(skb); + if (IS_ERR(list)) { + ovpn_dev_dstats_rx_dropped(peer->ovpn->dev); + kfree_skb(skb); + ovpn_peer_put(peer); + return; + } + skb->next = list; + + skb_list_walk_safe(skb, skb, next) + { + /* skb_unclone can leave fraglist children shared with a packet + * tap, so make each skb header private before changing its + * list pointer or control block. skb_cow_data makes the packet + * data writable later in the decrypt path. + */ + skb = skb_share_check(skb, GFP_ATOMIC); + if (unlikely(!skb)) { + ovpn_dev_dstats_rx_dropped(peer->ovpn->dev); + + /* carry the current record's peer reference to the + * next record or release it at the end of the list + */ + if (!next) + ovpn_peer_put(peer); + continue; + } + + skb_mark_not_on_list(skb); + + /* keep the current reference alive for the next record before + * handing this one to crypto + */ + if (next && unlikely(!ovpn_peer_hold(peer))) { + DEBUG_NET_WARN_ON_ONCE(1); + kfree_skb_list(next); + next = NULL; + } + + ovpn_recv(peer, skb); + } +} + /* Retrieve the corresponding ovpn object from a UDP socket * rcu_read_lock must be held on entry */ @@ -121,8 +328,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); - skb_mark_not_on_list(skb); - ovpn_recv(peer, skb); + ovpn_udp_recv(peer, skb); return 0; drop: @@ -533,6 +739,8 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock, .encap_type = UDP_ENCAP_OVPNINUDP, .encap_rcv = ovpn_udp_encap_recv, .encap_destroy = ovpn_udp_encap_destroy, + .gro_receive = ovpn_udp_gro_receive_fraglist, + .gro_complete = ovpn_udp_gro_complete, }; struct ovpn_socket *old_data; int ret; @@ -579,6 +787,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) { struct sock *sk = ovpn_sock->sk; + udp_tunnel_cleanup_gro(sk); + /* Re-enable multicast loopback */ inet_set_bit(MC_LOOP, sk); /* Disable CHECKSUM_UNNECESSARY to CHECKSUM_COMPLETE conversion */ @@ -587,6 +797,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) WRITE_ONCE(udp_sk(sk)->encap_type, 0); WRITE_ONCE(udp_sk(sk)->encap_rcv, NULL); WRITE_ONCE(udp_sk(sk)->encap_destroy, NULL); + WRITE_ONCE(udp_sk(sk)->gro_receive, NULL); + WRITE_ONCE(udp_sk(sk)->gro_complete, NULL); rcu_assign_sk_user_data(sk, NULL); } diff --git a/net/core/gro.c b/net/core/gro.c index ff828a3dce3c..069df5d96075 100644 --- a/net/core/gro.c +++ b/net/core/gro.c @@ -266,6 +266,7 @@ int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb) return 0; } +EXPORT_SYMBOL(skb_gro_receive_list); static void gro_complete(struct gro_node *gro, struct sk_buff *skb) { diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index cf07c3c6611a..187f108f3ee8 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -40,7 +40,8 @@ struct udp_tunnel_type_entry { #define UDP_MAX_TUNNEL_TYPES (IS_ENABLED(CONFIG_GENEVE) + \ IS_ENABLED(CONFIG_VXLAN) * 2 + \ IS_ENABLED(CONFIG_NET_FOU) * 2 + \ - IS_ENABLED(CONFIG_XFRM) * 2) + IS_ENABLED(CONFIG_XFRM) * 2 + \ + IS_ENABLED(CONFIG_OVPN)) DEFINE_STATIC_CALL(udp_tunnel_gro_rcv, dummy_gro_rcv); static DEFINE_STATIC_KEY_FALSE(udp_tunnel_static_call); From patchwork Mon Nov 2 16:43:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5563 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673487laa; Mon, 2 Nov 2026 08:44:26 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBzfN/I1sT8XXfX+fLrZ8Ck1Tgx+5ge5tJniKTOT1kj6UHB6/P9zYY3/yZkkEo7bES5fOaffsooHzq0=@openvpn.net X-Received: by 2002:a05:7301:1828:b0:36a:10d:afab with SMTP id 5a478bee46e88-36a42423d10mr17412960eec.25.1793637866377; Mon, 02 Nov 2026 08:44:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637866; cv=none; d=google.com; s=arc-20260327; b=NIOce6GxJue9lG6jY0HwYTWmLmUEsJFNFeMTxdla3P61OD6GJj2Php86oAmg4P7S1Y Pg5mR20NSlJ3b2egB8wN9P98NU/SzPJVzuSe6R+bcH263X5I4g7Dyefb2TaV74Ryk8i4 SisfIJQaHaDjWlE6jCLhAT9KofhtEWStH7S1dRkD1n3ByzyaWavEtEgddHFurOq05kKZ 09lsRW6DgSDqhRrtGnE0wHNDrXKNSjn65rHX8flgcQtZFDgJm1pNCR/pcxKERfZNDWol Bda5NP+tWMD8kQkKCwaDAkIszcW26oDavgdn1LfCn1kK5ohcwMMWF6chg3wiXvek1VtN bDhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=Wz+qMSZjyMdih9zBPtaT5zkbvc5Ybol83woLmGM9pN0=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=XXFcHLaznYLKotxdUjsSv0g5zRcZGHvIGlKc6Hk7TVBdwCZL4sPybry2Bkk3ZzwmYI dREjpIS+h9hpGvA6MzexLSc8Ob9PYxV9mlW+mA1U/koy4HCaktWGUy1fUl2TNs6fMqX/ 8npcQSuYhdBHVZ1puHgJIuEKFPxTFS7UOhvOh9iR/p5E/vNKnAC8GqGTAE3v++XwTnOv beZ5vRdrLbud++vaFdd9EcXuNo7cSX9B8BNh1KF5frBbUyfbIqCahntUjyJLJfGkrPkD AxY4Zj6bDKh2kIHeRfk5RznLLdQNRqEqFkcuqLwthad15vtSFr9AGS968Nl0vL6TxCck ZyXw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=cKQ1PR0f; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=iRU3gbiR; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Bp6HiPdp; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=CdU4T3U7; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5a478bee46e88-36a42b1cc20si34359427eec.56.2026.11.02.08.44.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:26 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=cKQ1PR0f; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=iRU3gbiR; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Bp6HiPdp; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=CdU4T3U7; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=Wz+qMSZjyMdih9zBPtaT5zkbvc5Ybol83woLmGM9pN0=; b=cKQ1PR0fui/7+d m9Qr3xhvygRhARycEvWmfaggpAL4Fv/HS+Eskeb1QCXlt2m6LALpSUHbBjCp+FO72/RaE4DsDn3qb lI+C6/ZvW/zw2Uya3YN9eJokzJMoFhQY+1KapxU5HvahxcTcEq0UmvyoAqA0p6wHHSwoIhylgJSEl N8Q=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8z-000000000bz-17R8; Mon, 02 Nov 2026 16:44:25 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8k-000000000a6-1IgD for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=ernflziDPbi1ZnQJFgUjxF6sBdiHnzY9PIymRhaJdkE=; b=iRU3gbiRjmAyqz6kArbBYYSj2I mfz9BvvO4XLVhKDddI3g1HQeQd0SqNic7SgXzrfKWJwgFklaGk4HZmC7Sn0ZEk9g19MhvkxhG0DaK hS0+09FdgMDlVD8N47ieTCz7elUuYp9n5Z+scVQ9eZk2DM1jkr7CKs6FWCTYpup1Ffvc=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ernflziDPbi1ZnQJFgUjxF6sBdiHnzY9PIymRhaJdkE=; b=Bp6HiPdpM/Shp5Eyrv7czq9JZ6 Q+3TVhgeV2PWsA/VT2sis7sdqJKJTQ2CWYLp6gzQlKUYj7Gf9R51zes+uBH2adFA2IcNcqtFzy/cm 1hoLN6+XEPuAKKfM1pHdtufJmmQIWJAFz7Kb7TVM59JTgC1PXFydWTrT/z7beL88BPBM=; Received: from mout-b-206.mailbox.org ([195.10.208.51]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8j-000000005X5-26Q8 for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:10 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-206.mailbox.org (Postfix) with ESMTPS id 4jJHbN3CLXzXw for ; Mon, 02 Nov 2026 17:44:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ernflziDPbi1ZnQJFgUjxF6sBdiHnzY9PIymRhaJdkE=; b=CdU4T3U7oAFTrxNmPtRdB5O62Mu3mcKN8ohyJ6VaBdUxdSaWsdSlbtu9WLbtbLUSR45aPe azDUDSW31RAcODVE5OIR+bf4NCYmx1zXpRt9VDH4GuPS4KflAuWAeKE7nJ8w7nIin3YYxM CcsMJAevdK+PKovyUFEAU8h4c2UKncV5xWVtJa+goyH60qn3eKZ4XeCcNiRCXQJvnFB4rF CgQR6kPGTahPOqaVGf3CazaRW05ECRmdtYF3XlZZHaATrK/cF9qFFraTH8xvUwP5Hj6+4y QMV+Z7i6gEX2luqsclwh0tWzZUjVc681QSkTPuLolrUtntfxP8Dta0yTF3b4Yg== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:49 +0100 Message-ID: <20261102164353.459969-8-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbN3CLXzXw X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (a [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xNv8j-000000005X5-26Q8 Subject: [Openvpn-devel] [PATCH ovpn net-next v2 7/9] ovpn: prefetch encrypted records before GRO batch decryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765619111526878 X-GMAIL-MSGID: 1880765619111526878 Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (and maintain the same distance throughout the batch), overlapping their memory access latency with the current AEAD operation. An ordinary single-record UDP receive has no later record and therefore skips the prefetch path. Only prefetch the linear part of each skb. Walking non-linear fragments here would duplicate the scatterlist walk performed by crypto and could cost more than the cache hint saves. This optimization primarily targets synchronous software AEAD, where the CPU which issues the hint subsequently overwrites the prefetched lines. An asynchronous accelerator may instead access them through DMA and is not expected to benefit in the same way; the cache hint remains correctness-neutral. A same-binary comparison using three 30-second samples per direction found distances one and two effectively tied forward, while distance two was 3.3% faster reverse and less variable in both directions. Profiling also measured slightly fewer decrypt cycles at distance two than at one, while wider distances provided no repeatable benefit. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without RX prefetch 22.773 Gbit/s 22.796 Gbit/s With RX prefetch 24.028 Gbit/s 23.405 Gbit/s RX prefetch improved throughput by 5.5% forward and 2.7% reverse. Signed-off-by: Ralf Lici --- Changes since v1 https://lore.kernel.org/openvpn-devel/0651258e11d4a0c290152a1f286b9d8a45c5acd4.1790843153.git.ralf@mandelbit.com/ - Document that RX prefetch primarily targets synchronous software AEAD; asynchronous transforms are not expected to benefit in the same way. drivers/net/ovpn/io.h | 15 +++++++++++++++ drivers/net/ovpn/udp.c | 21 ++++++++++++++++++++- 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index 1a94f0fda1d1..10c568c02234 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -10,6 +10,9 @@ #ifndef _NET_OVPN_OVPN_H_ #define _NET_OVPN_OVPN_H_ +#include +#include + /* DATA_V2 header size with AEAD encryption */ #define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ @@ -21,6 +24,18 @@ #define OVPN_KEEPALIVE_SIZE 16 extern const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE]; +static inline void ovpn_skb_prefetchw_linear(const struct sk_buff *skb) +{ + unsigned long addr = (unsigned long)skb->data; + unsigned long end = addr + skb_headlen(skb); + + /* crypto overwrites data in place, so request write ownership of each + * linear cache line before the AEAD implementation reaches it + */ + for (; addr < end; addr = ALIGN(addr + 1, L1_CACHE_BYTES)) + prefetchw((void *)addr); +} + netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev); void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb); diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 0c3557d90a1e..e0aeb09d0e2d 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -32,6 +32,9 @@ /* like UDP and TCP frag-list GRO */ #define OVPN_UDP_GRO_CNT_MAX 64 +/* leave enough work between a cache hint and the record which consumes it */ +#define OVPN_UDP_GRO_PREFETCH_DISTANCE 2 + static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) { const unsigned int offset = skb_gro_offset(skb); @@ -189,7 +192,8 @@ static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) { - struct sk_buff *list, *next; + struct sk_buff *list, *next, *prefetch; + unsigned int i; list = ovpn_udp_gro_detach(skb); if (IS_ERR(list)) { @@ -200,8 +204,23 @@ static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) } skb->next = list; + /* a fraglist GRO aggregate makes later ciphertext visible before the + * current record is decrypted, so we prime the first two records, then + * keep the cache hints the same distance ahead while draining the list + */ + prefetch = skb->next ? skb : NULL; + for (i = 0; i < OVPN_UDP_GRO_PREFETCH_DISTANCE && prefetch; i++) { + ovpn_skb_prefetchw_linear(prefetch); + prefetch = prefetch->next; + } + skb_list_walk_safe(skb, skb, next) { + if (prefetch) { + ovpn_skb_prefetchw_linear(prefetch); + prefetch = prefetch->next; + } + /* skb_unclone can leave fraglist children shared with a packet * tap, so make each skb header private before changing its * list pointer or control block. skb_cow_data makes the packet From patchwork Mon Nov 2 16:43:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5559 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673339laa; Mon, 2 Nov 2026 08:44:21 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBzH4VTVnP9nyAusQuJluf1FhySzz3Hm64zj4mbV2xBNEhGBdE0szDXo0hG7LFIWAe2teFjRx8Fhp+w=@openvpn.net X-Received: by 2002:a05:7022:ea46:20b0:164:4e36:f956 with SMTP id a92af1059eb24-1ac5d88c619mr18000195c88.16.1793637860842; Mon, 02 Nov 2026 08:44:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637860; cv=none; d=google.com; s=arc-20260327; b=CV7yMfwd1M2HgFzYGbwU7h7JXKMRohXIrxeoBNmQf0XCymbVllzaMXK51jXvFxWltu 2gx78x75eNdAMbIZbZ1LqouH6SwIB6BCZSVQp3WnR4TUIFDgjR4zIF0mUxGhgf2wqYhl sxoQLjdgqGp3PZFHCLrHoCxiecMtaV/sVwUqzSokMCnTQMeDNamAwmevHgoHlgbet+M3 CtDyESbtALX+NrHbosZjnJ2eqQp2mZcHHEp58kDizk5j9iTtr3B8xLUQv0kVpZ78eBZA plJQQld9WBvyG60PbNvaOJNk0Ir2EhWM3evY4MSBWPRoDDHBa5/pExThWqzXpwaXPwJy p0gQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=FMX1zwz+aIxDT493eDAG2Prf04yIl83lF8Kr/cayk00=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=ZMTkec9v24rBOBmyiVdk41ueLOEVU9Rp0fjkZKWSsyLcTXKezzYfrLqtCm4DZ3GdYO W8R8VsXYLhWmkjVWcMy7sfpSibWseWhyrRsB5WVljq/WLlUw/DDSsVeZKnZLWM5MtSid HeuuqJSRVpU4q95Et4eFws92nxaR9sjNclqU9Mro6I087VZRLM1m7EF09vdecBh/1Wlh UrnTAlnuk8qZFKuE/FjChs1sW0ffXAkxNBr/dlZongMihCqcy4X8KP19ONPLEZen5NIq Vi6ByAsErXILDRQI+/iLUKnHhDNtlDi1FarC09rbmsUAzM7DMzaQJGYOy7h89Bs29DNH Zjfw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=CQCJ9mTA; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=ApLHyHsP; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=USIJXdme; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=nXRYrIXM; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id a92af1059eb24-1ac143c47f3si37903312c88.12.2026.11.02.08.44.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:20 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=CQCJ9mTA; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=ApLHyHsP; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=USIJXdme; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=nXRYrIXM; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=FMX1zwz+aIxDT493eDAG2Prf04yIl83lF8Kr/cayk00=; b=CQCJ9mTAsioDCY rAPV5NJryoxe81hRcxUFFLNtCGHR7jPEaZlmFI6REpmdEkKwPx1dq0EubPzTespQf8zHk981nFulu l1PjBTjEEzYnlT9+Ll6OFZ8f+msC4mxUcit6AwkI/arrIc6eNqb4yB1EzSTb0J/DyTzgQU01yg86e DXQ=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8t-000000001e0-2oy3; Mon, 02 Nov 2026 16:44:19 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8m-000000001dO-3A5J for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:12 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=vZzK0tKhbCGc2hchUM7ZeDD+tx4P+Ygd+Y483VXnj7g=; b=ApLHyHsP+KPSWv1jzF/qaePvNo QAGr6NYuTiG/TQAegwj07I7SSstbcUHQDk3NdwTmT53bzzejQfP/aVOmf8ngarb+FqQ0zquEHifOY EAdcFnI5Wn8WUoSOjgc8vZD9uBFCHELUHSnQecf0YVPRCBShblmLcPdgH2Os9OWaTTDE=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=vZzK0tKhbCGc2hchUM7ZeDD+tx4P+Ygd+Y483VXnj7g=; b=USIJXdmeua63RSSJAY2i+y/jJu GILs9erUslIrAm0NOy6A8PbagTZeRgqCf5tJtk6YFvSGEM0doYjg54642ToK4xDcd0JaLBX5oPudC 6BuNN0Pl+8iwQ3XBGE3cZkt5yiK+xbeBHGKGYHDCwCnNOM4/KNE95C6tldgjYCYswyto=; Received: from mout-b-106.mailbox.org ([195.10.208.46]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8k-000000005hU-03DT for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:12 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-106.mailbox.org (Postfix) with ESMTPS id 4jJHbN5QdlzNlb8 for ; Mon, 02 Nov 2026 17:44:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vZzK0tKhbCGc2hchUM7ZeDD+tx4P+Ygd+Y483VXnj7g=; b=nXRYrIXMkoAq5oNx8la0zmM536t7jM6MmGmHmxryZHslJ0Ztqz28d29l2jfW0PH61X0Ml/ uSquu6k4KkTpWoF//Rpv0GdJryF4YxEQaxMml4WD3kfdUIyUSpRq6W+5CdxE1iqzUdkAUk MWOw01oT/Jkf9mnYKVqzo/4Lx5P9/5S5kdGnz8bSHKG64lcIQLNM3kmU84z1+Bs3313XH8 avsFkQHoqWHfk5pCjJcOjxQCeI2HwsuNiNtOkE8avNd4K4YZ/oNbvIFR4Sg+2DzOqHeLnS RreWrOFOz/SuM+ulED9nzxR7jvfHNNnTsG1RZoq1QeTcgNcnVAVVbHc+sZ3eGw== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:50 +0100 Message-ID: <20261102164353.459969-9-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbN5QdlzNlb8 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Software segmentation exposes the complete skb list before encryption begins. Use encryption of the current segment to overlap fetching data needed by the next one. For segments encrypted in place, request write ownership of every cache line in the next segment's linear area. When converting ordinary GSO input into an out-of-place UDP GSO aggregate, the linear he [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xNv8k-000000005hU-03DT Subject: [Openvpn-devel] [PATCH ovpn net-next v2 8/9] ovpn: prefetch GSO segments before encryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765613411318533 X-GMAIL-MSGID: 1880765613411318533 Software segmentation exposes the complete skb list before encryption begins. Use encryption of the current segment to overlap fetching data needed by the next one. For segments encrypted in place, request write ownership of every cache line in the next segment's linear area. When converting ordinary GSO input into an out-of-place UDP GSO aggregate, the linear head has just been touched by segmentation while most payload remains in shared page fragments. Read-prefetch the first direct fragment without repeating the complete scatter-gather walk performed by crypto. Fully linear segments and fragments without a directly accessible kernel virtual address skip the fragment hint. A single skb naturally skips both forms of lookahead. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without TX prefetch 24.028 Gbit/s 23.405 Gbit/s With TX prefetch 24.166 Gbit/s 23.558 Gbit/s This is a modest 0.6% increase in both directions. Signed-off-by: Ralf Lici --- No changes since v1 https://lore.kernel.org/openvpn-devel/aabfba2c687333c940de22e33bc3181f7e63817e.1790843153.git.ralf@mandelbit.com/ drivers/net/ovpn/io.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index debe3d4d5731..3a58b7f767fa 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -389,6 +389,28 @@ static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) return true; } +static void ovpn_skb_prefetch_frag(const struct sk_buff *skb) +{ + unsigned long addr, end; + const skb_frag_t *frag; + + if (!skb_shinfo(skb)->nr_frags) + return; + + frag = &skb_shinfo(skb)->frags[0]; + /* prefetch requires a directly accessible kernel virtual address */ + addr = (unsigned long)skb_frag_address_safe(frag); + if (!addr) + return; + + end = addr + skb_frag_size(frag); + /* the linear head was just touched by segmentation, so prefetch the + * first shared payload frag without repeating crypto's full sg walk + */ + for (; addr < end; addr = ALIGN(addr + 1, L1_CACHE_BYTES)) + prefetch((void *)addr); +} + static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, struct ovpn_peer *peer, struct sk_buff *batch, @@ -430,6 +452,12 @@ static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, skb = __skb_dequeue(skbs); len = skb->len + OVPN_DATA_V2_OVERHEAD; + /* overlap fetching the next segment's shared payload with + * encryption of the current segment + */ + if (i + 1 < segments) + ovpn_skb_prefetch_frag(skb_peek(skbs)); + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_skb_cb(skb)->batch = batch; ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, @@ -491,6 +519,12 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, * independently */ skb_list_walk_safe(skb, curr, next) { + /* encrypting this segment can hide the cost of fetching the + * next segment's data into the cache + */ + if (next) + ovpn_skb_prefetchw_linear(next); + if (unlikely(!ovpn_encrypt_one(peer, curr))) { ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); From patchwork Mon Nov 2 16:43:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5558 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673332laa; Mon, 2 Nov 2026 08:44:21 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBzdf+bV3MtsOqKd4g31M60gFyBbj5LD8taz0Kpf0mCAkWn1wiCEFYo6qRIia7L8vyfXMrMkXGYG2uI=@openvpn.net X-Received: by 2002:a05:7301:6183:20b0:36a:1089:9836 with SMTP id 5a478bee46e88-36a423c8988mr15061982eec.15.1793637860781; Mon, 02 Nov 2026 08:44:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637860; cv=none; d=google.com; s=arc-20260327; b=Ww0i9ilWwq55TnZGFmLkgxOtPRnxBfb2kI2PWu+dlUEAFCxWGKy9McckZp/mp3f0Qs Ate9g9i6yv9OoPcuiyz5Pur5eHL6MvMzNcz/sW+1sARdP2Vh3tLJCMs9M9bnv9kEig28 XTvRRO6Rci79jLBHLDsHNHqiSR5zkJQtDdWMqzMMY5mmd+1+xb76zBATet1DpqeguZ9l tlm1DxxVgz5kyubU0dyEhX5lLWHPV43ycTnDlz2DRKeWAdLSHU1UtKUQsIOwe8cePSLi dKFApmp3xgoQMoM182sgbY5VnAPPMTLR48hyjeePuccp5DOwfkA13Li/lWv05uzcNRI+ MIAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=edzfY2+F0V1YvYdl/fys5QtAbC5n+D1z2lDXyVJjaoI=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=ekzvyCcz/rOYyi4EHatBX0/e4kI63/WmdsfRCbJiZzsy3HiP7Vo9ZtQQD5Y8OBHLkb JIhm/z2AIoEU5mIAX1FBxo7I7SnSbxwgyW4Y358UBZ76EBlHd3PsKrdO5Y7HMkEwfn5g eJDuG/BMylgBktk3SmxqE8tpIk5Qa7Wisli6ZKD3qVyI8WYMICrO5otxXtAvUMUPlHAr +8vNWPoFi1iQzz4i0h2tnaC+39Wt4iohTcdvyfpr+QPK+0r4dusRbYTWxmK+aW2Jsh7P uMOnOfAamlE1dEoVDSSvYgkcamU5Ic08vLEYFkWWITaLLTidAj6JmaPDYox9YDcRbIle ApEQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=X1PqTnhp; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="YJKs/REb"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Ti0dCi6V; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=AI+2Odb6; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5a478bee46e88-36a4281b081si32089985eec.13.2026.11.02.08.44.20 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:20 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=X1PqTnhp; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="YJKs/REb"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Ti0dCi6V; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=AI+2Odb6; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=edzfY2+F0V1YvYdl/fys5QtAbC5n+D1z2lDXyVJjaoI=; b=X1PqTnhpht07e1 P9vyfrUbCb+mBYrCduvFdgb/XgRA0wonyhI8uswYx56Nzc/uUgGA3WEe780ri+3wTgBC4GESPWSmN yjU+AvCHyuk2CENpi3pmQOBd0MUVekjlXdF3+yhm//yty7TB7pYPIijiXdngrzwrGOjGp+axOsY9j V+0=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8t-000000001dh-2Dzc; Mon, 02 Nov 2026 16:44:19 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8j-000000001dF-1n0h for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:09 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=qpRCXbqnv9GGKYsh7UGdNpah6jLYZpyuJAna/4QirRE=; b=YJKs/REbAuzV10Riuc59Q8b/Pa qSfiQVpi1BcN26pODr4xNJgGw4h2fCsl+E0I5Y7thrMlotVrpNjr+ZUtiZ6s6R+3xJsKM8rN4XzS4 HosQErZBcWUWMS2bQLH/5+lFy91lwr+T7fxbM4+zRQmsYI/hGrZQ+yIcTiu6BIyw7iBk=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=qpRCXbqnv9GGKYsh7UGdNpah6jLYZpyuJAna/4QirRE=; b=Ti0dCi6VjecFOD6OcmCIz4Tllo y4pCYr2DClW6BC1ui8qnOX9fffSicn0o2sCHjSZ6NFVor4kork3K4mh/A1ie42NSV6QO78lRxDDNY suhznxiLl4JcYZsmsiZDKNcC7hO4otkOPhR/DBs/d1kpauyi+rBebguq7KjkgysyDbdw=; Received: from mout-b-202.mailbox.org ([195.10.208.62]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8i-000000005X0-23Fh for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:09 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-202.mailbox.org (Postfix) with ESMTPS id 4jJHbP0szQzKnfg for ; Mon, 02 Nov 2026 17:44:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qpRCXbqnv9GGKYsh7UGdNpah6jLYZpyuJAna/4QirRE=; b=AI+2Odb6+8FAhlkDa1oGswRSdzB6DWBTGRklpizaRppoTGvazWyl1TvgP/7igi41uzas9J 6c7xJpBXcx3993gcj8Di3+IxPdxxpUiFOeE7oOL4hOqY7HhwL7yK0CKQK8sf7mGdPbvO0h rWGzrmMBlSsEEBsx2l5wCscjctvI3f7Oh1BkoCDm/gM7P48hUzJNjEtWIcyebPsADi0n2u O9rVHdzNlZsU3+n1hkl1kb8aatV9ws5u/SYvRONf5kZx5FcfEIIMDxYdmmhJpEiYfgNuiS 3T1olSJAc2d3EG8t8eciUXEw40fAT7FSq7FcoEYD1eIjX14tSSj28FvyRAFwKw== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:51 +0100 Message-ID: <20261102164353.459969-10-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbP0szQzKnfg X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Each encrypted GSO record currently builds its authenticated header in request-local scratch storage, copies it into the final UDP GSO aggregate and then maps that range again as destination AAD. Map the final output range first and, when it is represented by one base-page-contained scatterlist entry, construct the header directly there. Use the same entry as source and destination AAD since t [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xNv8i-000000005X0-23Fh Subject: [Openvpn-devel] [PATCH ovpn net-next v2 9/9] ovpn: place GSO record AAD directly in output X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765613189945700 X-GMAIL-MSGID: 1880765613189945700 Each encrypted GSO record currently builds its authenticated header in request-local scratch storage, copies it into the final UDP GSO aggregate and then maps that range again as destination AAD. Map the final output range first and, when it is represented by one base-page-contained scatterlist entry, construct the header directly there. Use the same entry as source and destination AAD since the AEAD interface does not write destination AAD and permits the two entries to alias. Retain the scratch-and-copy path as a fallback when the output range cannot be mapped suitably. In a three-run ConnectX-5 hairpin test with AES-128-GCM, one inner TCP stream over UDP, a 1408-byte tunnel MTU and a 1500-byte physical MTU, mean throughput increased from 24.487 to 25.002 Gbit/s (+2.11%). Cycles per byte decreased from 3.616 to 3.569 (-1.30%) and instructions per byte from 9.635 to 9.540 (-0.99%). Signed-off-by: Ralf Lici --- New in v2. drivers/net/ovpn/crypto_aead.c | 61 ++++++++++++++++++++++++++++------ 1 file changed, 51 insertions(+), 10 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 1c5387f83eca..7bbb4ea3d642 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -8,6 +8,7 @@ */ #include +#include #include #include #include @@ -173,6 +174,29 @@ static void ovpn_aead_encrypt_header(struct ovpn_peer *peer, memcpy(data + OVPN_OPCODE_SIZE, iv, OVPN_NONCE_WIRE_SIZE); } +static bool ovpn_aead_encrypt_header_direct(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + u32 pktid, u8 *iv, + struct scatterlist *sg) +{ + unsigned int offset, page_offset; + struct page *page; + void *addr; + + offset = sg->offset; + page_offset = offset_in_page(offset); + if (page_offset + OVPN_AAD_SIZE > PAGE_SIZE) + return false; + + /* an SG offset can address a later base page in a compound page */ + page = sg_page(sg) + (offset >> PAGE_SHIFT); + addr = kmap_local_page(page); + ovpn_aead_encrypt_header(peer, ks, pktid, iv, addr + page_offset); + kunmap_local(addr); + + return true; +} + int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { @@ -269,6 +293,7 @@ int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, struct aead_request *req; unsigned int payload_off; int dst_idx, mapped, ret; + bool direct_aad; u8 *aad, *iv; /* each input records the shared peer and key for the common completion @@ -285,26 +310,42 @@ int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, dst = src + src_nents; aad = (u8 *)(dst + dst_nents); - ovpn_aead_encrypt_header(peer, ks, pktid, iv, aad); - - ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); - if (unlikely(ret < 0)) - return ret; - /* encrypt out of place from the original segmented skb directly into * its final range in the UDP GSO skb */ sg_init_table(src, src_nents); - sg_set_buf(src, aad, OVPN_AAD_SIZE); + /* use the final output bytes as source AAD too when they can be safely + * written through one base-page mapping. + */ + mapped = skb_to_sgvec_nomark(gso_skb, src, offset, OVPN_AAD_SIZE); + if (unlikely(mapped < 0)) + return mapped; + direct_aad = mapped == 1 && + ovpn_aead_encrypt_header_direct(peer, ks, pktid, iv, src); + if (unlikely(!direct_aad)) { + ovpn_aead_encrypt_header(peer, ks, pktid, iv, aad); + ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); + if (unlikely(ret < 0)) + return ret; + sg_set_buf(src, aad, OVPN_AAD_SIZE); + } + mapped = skb_to_sgvec_nomark(skb, src + 1, 0, skb->len); if (unlikely(mapped < 0)) return mapped; sg_mark_end(&src[mapped]); sg_init_table(dst, dst_nents); - dst_idx = skb_to_sgvec_nomark(gso_skb, dst, offset, OVPN_AAD_SIZE); - if (unlikely(dst_idx < 0)) - return dst_idx; + if (likely(direct_aad)) { + /* destination AAD is not written and may alias source AAD */ + sg_set_page(dst, sg_page(src), src->length, src->offset); + dst_idx = 1; + } else { + dst_idx = skb_to_sgvec_nomark(gso_skb, dst, offset, + OVPN_AAD_SIZE); + if (unlikely(dst_idx < 0)) + return dst_idx; + } payload_off = offset + OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE; mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, payload_off,