From patchwork Tue Sep 15 15:23:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5346 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:30c4:b0:8a4:a7c2:8138 with SMTP id n4csp3642482maa; Tue, 15 Sep 2026 08:26:17 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBz11BIul28ZMn0ppKjIxTUJi3u1dYAVHo1PkFxM0/FVXFRXe5bX5/f1fOUhpOyBqRntj7SY/fEOaJI=@openvpn.net X-Received: by 2002:a05:6809:184:20b0:4b9:e65b:8c36 with SMTP id 5614622812f47-4c9ab4b3cf1mr905320b6e.36.1789485863014; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485863; cv=none; d=google.com; s=arc-20260327; b=ZNhbMH+86SXkPfyd083r4uO7ocagK5gg0vIwOgyClxj4pfJq+zxGxIviTodPOLXmoI hGTNQoJYR4Ih3FU04XFMRbHyxt4m1DZtWVBj1NwsFqulUsnGCAuO4h3dKpmLrL0XHtKW 7PyV8y2qmry4ZY8hP2OVv1quZS7b1adSYIBxeEfzX/m8UIZX12Uewsbln3AbaiL7T6+r BsraliequBiINSkIF038rOPTLqiSE3aq+8RFZWsNG0lfdPrB8nagaAmM0i4WatOWL8hc 1c+mZQX9SoV9dsTNuEADhmIOxZbdv8hWYaXHo48QdLh11fqpKGv/dtnaIDohASrCbaHy HzQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=FkCwVfT4Am+dQXmRyLA16Gsh4uWbF70yHJgi2lZYQOM=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=WKAcdxqENKbpxJyOWl1dBXi9AC/64TZC7dJysSbYUSNBfUU3UYzN0nemmfdVw3ExzP JmPyHoVyTAxFfB0mYckeZW7F49z4Al2SUSLZF1E91wmmyZmsbVqmsC9XVHZBSSPaFKLr oG5eG96EIzPLZ+zxRccGTNtisNHmlcx2T+xEz1e1yTaadGBUU+a2kLr/QePIbBFGAtp+ EMF/NUFoio1bzovvNBzbG7m3i5Ibm/EKdSmErLqJ/ZH04dO7NRsT2kgJ6ZIwd2Rv7dPB phnmiUdSabPjVQs6VjhO9rFJBr5/74756UNGB5ei4SOf8ncd+rLLkmX9v/cwi5kTRE9g 0jvQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=hipNzePq; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="DT2Qbg/p"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=juRVviO1; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Z61e+cM1; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5614622812f47-4c7b8fc1977si4692416b6e.92.2026.09.15.08.24.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:22 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=hipNzePq; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="DT2Qbg/p"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=juRVviO1; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=Z61e+cM1; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=FkCwVfT4Am+dQXmRyLA16Gsh4uWbF70yHJgi2lZYQOM=; b=hipNzePq5y+6v7m+PR1QHL4A4U yNwb02QKGi6yFUN42gQZpkMXbD9WER/mnGcpvED0eyMZeUOK3oH4uqAl/NOTxivoWt5zsmLYJNo2G zcwL1GoC6cegM/ZItGDKRcwjupC+xIlHTmkJYJWyndm0ZzFCkFmALFeSebPbUCL3SdLk=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V16-0000JL-JZ; Tue, 15 Sep 2026 15:24:17 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V11-0000IU-Al for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=VresLWD6BJvlpegALgISoARrJWJkgDuZoqGTBddC6EQ=; b=DT2Qbg/pTSPSbs6hjGbf4MUZL+ UjMEblrvb41zeNySE9uXPXQhm8J1OoVuctUyNxnqICoVcd86JuzEYbVcRhoX9+S9a/Y+irEHf/KnG 8HRMW9yPh2EB5U/pkHJQ69JDyJ3Qm42IxptwTq7MZRD+hAIviD68saenOGq/dsxeIyHQ=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=VresLWD6BJvlpegALgISoARrJWJkgDuZoqGTBddC6EQ=; b=juRVviO1sthhJJxwak3A30zm+N /gKK8ipxKii3R6NO/bxx4eC68gIHt9A8jFNaMeBAV5HKFILVkkg42olwd2328xF8je3FQWrfTgZxY GMbycb6wj9a/RcKj3IH6M0KYiONwnKRHbbo+SY1l/ClDi4PvkhHgzX0t2PLo2h9d0d3k=; Received: from mout-b-201.mailbox.org ([195.10.208.61]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V0z-00076y-TD for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:12 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-201.mailbox.org (Postfix) with ESMTPS id 4hkm5G0pthzLmDM for ; Tue, 15 Sep 2026 17:24:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VresLWD6BJvlpegALgISoARrJWJkgDuZoqGTBddC6EQ=; b=Z61e+cM1ttWIg149D3w2/0iG4nAjg/U7j6HtIXc1uOiB4IqGEkQM2Lg38Fl9J2rg9rcMQv Bee+IjvSxKQN+RSIhz//+R2ZpjYxeeMgc7R5vZrs0UY+4/vy6VuCWpr/N9MeQPqEwyAojA gQ3uRnutKp21clj9omVe4+KrhX216Q0xbfU5q+Kx6DDyZfFmXXNDhK9V6sw0ue0PnZCzea IwEqfuuLV4WwKge4fW2I6Xt1mK1rT6GuCkw4RykSnZe1iOrRkkxx6z8BzdCgFHl9up5JS1 idy6F/37Amg2D32SjxyF5DGZE+drZqDzbKZ2gy6vmKUD/3yNgPBzzrjLNo3DgA== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:49 +0200 Message-ID: <1bbfde37488ade61928554db3a119525adf7b608.1789485693.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hkm5G0pthzLmDM X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1x6V0z-00076y-TD Subject: [Openvpn-devel] [RFC ovpn net-next 1/9] ovpn: advertise checksum offload for GSO packets X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411928584655789 X-GMAIL-MSGID: 1876411928584655789 ovpn already advertises software GSO support and segments GSO skbs in its transmit path. However, without checksum offload in the device features, the networking core has to segment GSO packets before they reach ovpn because TCP GSO packets normally carry CHECKSUM_PARTIAL state. Advertise NETIF_F_HW_CSUM so the stack can pass such packets to ovpn. Complete partial checksums after any GSO segmentation and before submitting packets for encryption, since the inner packet checksum can no longer be fixed after the packet has been encrypted. Also pass the ovpn feature set to skb_gso_segment with GSO capabilities masked out: this forces software segmentation, but still lets the segmenter preserve supported non-GSO properties such as non-linear skb data instead of needlessly linearizing. Signed-off-by: Ralf Lici --- drivers/net/ovpn/io.c | 20 ++++++++++++++++++-- drivers/net/ovpn/main.c | 2 +- 2 files changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 9526f8096da6..112067ded401 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -358,6 +358,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; struct sk_buff_head skb_list; + netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; __be16 proto; @@ -392,8 +393,13 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_dst_drop(skb); if (skb_is_gso(skb)) { - segments = skb_gso_segment(skb, 0); - if (IS_ERR(segments)) { + /* force software segmentation, but keep ovpn's non-GSO feature + * bits so the generated segments can preserve non-linear skb + * data where possible + */ + features = netif_skb_features(skb); + segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", netdev_name(dev), ret); @@ -418,6 +424,16 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) continue; } + /* NETIF_F_HW_CSUM requires completing partial checksums */ + if (unlikely(curr->ip_summed == CHECKSUM_PARTIAL && + skb_checksum_help(curr) < 0)) { + net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", + netdev_name(dev)); + ovpn_dev_dstats_tx_dropped(ovpn->dev); + kfree_skb(curr); + continue; + } + /* only count what we actually send */ tx_bytes += curr->len; __skb_queue_tail(&skb_list, curr); diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 0708249e9607..28e1eb06e127 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -157,7 +157,7 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { - netdev_features_t feat = NETIF_F_SG | NETIF_F_GSO | + netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Tue Sep 15 15:23:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5345 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5064926mag; Tue, 15 Sep 2026 08:26:03 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwiTFbdSyuqY884FZvr21qbBaJyTaZ/x6jMPDcmosQds8hYrF8OfvHBsFuIA+wO5FKdqs1g8pcJF8s=@openvpn.net X-Received: by 2002:a05:6870:3c0d:b0:46a:e0c1:6b34 with SMTP id 586e51a60fabf-481f94edd6amr10661066fac.12.1789485861834; Tue, 15 Sep 2026 08:24:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485861; cv=none; d=google.com; s=arc-20260327; b=nY3KDby0ru4Wk7ZiuEnqYWhkO4P2gMk8ujR07+hxBRKL+GG1uxcu23xdYFyEEqyqNu KejJh077gDER3PQ8nsU7BD1yErmhXIAppC752Pd6tdU4ASJRWa8YQpOCjgSJzZQp0Zx3 NjylyKSguq0EFIQnYqRNKXarhFFq+E0ZARHahCYhABKW0WQ8hOs9jvecPPyA5cZsybXw RWQw7jSo1AvcWYV8YnGBfjwiGnS/JC04z5B0ZuqKmMgwIeL0cOfFbseTqq7OE7wfMYL6 MuUKewlvTgTiiNVmY6M/7nVImzDRIM+CNevU4cQiIY2GxtUEoWz+YM2eJANCaRa8Pkqs p/4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=PJFlxf6DvHbxep4kldQFnVa65oEFoZADIy767GjhKEA=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=mMt9Kdm//A7mAP4SlQd8ifCTzLCxh1ykKkIhe2b5jUudTiWhzbEXB+GbJn2RdJCW4E LHQU2Q3VTHEcCwORYV7zRDJrwNU1Ye6UT7Qxevs0JqHf5X5vC/TyBSw0wwpTnpXVkp/k bNTV2GuAxI7CexU2qLaY3lBAiltBMeG9KAJcFOh8iD52EQlWy/dk3ipcx5DT7YA5ottK t7FnkGCFvFkRkkDjFmtlU7607SOG9uoJHrKBCeIe9jOz8f9nQSoaZWk4k4+7D1MVVJTk Ey/mVPoCROME2KsyP1SbKZlTRX3zduPhIgCIBMNB7f0410LgriINqj5ycNJx1dp+Zx5N /mbg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ONdBfeRo; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=EuZouNdn; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=SFzPdMNA; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=O09c0CTs; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-47df961d79csi11323480fac.235.2026.09.15.08.24.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:21 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=ONdBfeRo; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=EuZouNdn; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=SFzPdMNA; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=O09c0CTs; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=PJFlxf6DvHbxep4kldQFnVa65oEFoZADIy767GjhKEA=; b=ONdBfeRoovy0tajPo218JKdlDf ktNA5ASIW/JFU+W2HZ+tvHF4WSLMY4lo0ULB3C6n5mLSVWjniPuXfu9ktZLnO0T3dtjSBkwvlEfKv fVI6hCQwwwEIaKK5K2kSqIhG+VEhRMxnldN4Trg+CtvcksF3gLuF2HErrPy1pGJ+Yrpo=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V15-0005pi-2T; Tue, 15 Sep 2026 15:24:15 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V12-0005pT-FX for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=GKJOL2XeKLv1y8rVhqwEbNqapdsR912FCXVnp63S2KI=; b=EuZouNdnCNkXVBaDjUiqjbQGh/ 07viJL2vfHv3hqmtAerc61++sDb/03SAoojnAAmic17ZBivlIwYFoivx0EDMVm3YoutnCLxuc1ArW 4z9t1mbWJwIX8WJ37gCzr4Ah6Wq1+4xASN67JwuLqVVot2RUUDldBeWH2sZ2xgHa3jtc=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=GKJOL2XeKLv1y8rVhqwEbNqapdsR912FCXVnp63S2KI=; b=SFzPdMNAiUg/m6A+mmkojulnLa 72id7ENpLtf6khDvUooMxA3UkRsOq3uIfP0O2GsMaSEctYDE006jsYHki8jZIZzvgp+nkp7KjgCIY wrXIX1pU4rfjRg8fAUgd9KmdPjYHxH3ZsolHczsBLvFfbWpfGRum6xqbCm/u+gIqYy8o=; Received: from mout-b-210.mailbox.org ([195.10.208.40]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V11-0003bO-GG for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:12 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-210.mailbox.org (Postfix) with ESMTPS id 4hkm5G4KvKzFqwg for ; Tue, 15 Sep 2026 17:24:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GKJOL2XeKLv1y8rVhqwEbNqapdsR912FCXVnp63S2KI=; b=O09c0CTsHAb9mdnk4nBD5O8upwTGGamA9Tnm1/fE15XxwGNIWTnNBc+NvYk+fHeEyK9c5z QeOsPzLenuHSD/lwj5CgCcY3tFVQpJrtQA1+FvyMte/O4tPDd+09BBjd5wHahCzsBE1XcQ PsWUKYKYOmUSGlEG4NmELtpPpBlnB6jCWTaakNBqvEBmrwXc6k/gBhnMenk6hIOZPduoBY lpE7or5Dd6nZ0nlf4BKoBK/N7EImRLYpa54+MGVOVkBeLp2rPuFsvPQU0W/hAjuXC4Yfug V0VBlArnATtUHRRKbpyy1vr4l2euukOPaHW4wuN00Kg8MI80EktjAqR+3Az3Yw== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:50 +0200 Message-ID: <5baa1d94e29e7b109ffd412b4f1d5d113a319389.1789485693.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_ [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6V11-0003bO-GG Subject: [Openvpn-devel] [RFC ovpn net-next 2/9] ovpn: accept frag-list GSO input X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411927015035984 X-GMAIL-MSGID: 1876411927015035984 Forwarded TCP traffic can be coalesced into SKB_GSO_FRAGLIST when the receiving host has no local TCP socket for the flow. Although ovpn segments every GSO input itself, it does not advertise NETIF_F_FRAGLIST, so generic transmit validation segments these aggregates before calling ovpn_net_xmit. That segmentation is functionally correct, but causes ovpn_net_xmit to be invoked separately for every resulting packet. Advertise frag-list storage so ovpn receives the aggregate intact and performs protocol validation and destination-to-peer lookup once before segmenting it. Frag-list GSO segmentation recovers the complete child skbs, which can then be encrypted in place and transmitted independently. Rebuilding those children into a replacement UDP GSO aggregate was found to add cost rather than improve throughput. The feature also admits non-GSO frag lists, which describe one packet split across several skbs. Let skb_cow_data preserve small lists directly. If a list exceeds the AEAD scatterlist limit, linearize it and continue rather than rejecting an otherwise valid packet. Signed-off-by: Ralf Lici --- drivers/net/ovpn/crypto_aead.c | 8 ++++++-- drivers/net/ovpn/main.c | 3 ++- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 74eaf6fac2f5..2af493fd5735 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -168,8 +168,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, if (unlikely(nfrags < 0)) return nfrags; - if (unlikely(nfrags + 2 > (MAX_SKB_FRAGS + 2))) - return -ENOSPC; + if (unlikely(nfrags > MAX_SKB_FRAGS)) { + ret = skb_linearize(skb); + if (unlikely(ret)) + return ret; + nfrags = 1; + } /* allocate temporary memory for iv, sg and req */ tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 28e1eb06e127..ac4e0d85e215 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -158,7 +158,8 @@ static const struct ethtool_ops ovpn_ethtool_ops = { static void ovpn_setup(struct net_device *dev) { netdev_features_t feat = NETIF_F_HW_CSUM | NETIF_F_SG | NETIF_F_GSO | - NETIF_F_GSO_SOFTWARE | NETIF_F_HIGHDMA; + NETIF_F_GSO_SOFTWARE | NETIF_F_FRAGLIST | + NETIF_F_HIGHDMA; dev->needs_free_netdev = true; From patchwork Tue Sep 15 15:23:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5339 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5063775mag; Tue, 15 Sep 2026 08:24:21 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBw30mBVdFwxLNV5IZ9qC5gKU3YX4bMCXpHmEcrSXLXCvSaJNAmrJUzfmXGsWKF9wXr8YcWShCZAnV8=@openvpn.net X-Received: by 2002:a05:6870:5cc8:b0:46a:c0f8:ad3a with SMTP id 586e51a60fabf-483d14d1890mr1297447fac.8.1789485861510; Tue, 15 Sep 2026 08:24:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485861; cv=none; d=google.com; s=arc-20260327; b=YKJMqRqsMmTgkPpEjxtmB2w4TxjuEOe02NwzBTNJQRaw+kLfvW0XxhdSXYA953i+hs /iqGn01llYcgIshcJdTZM7RXHQCZ2snaX8AIcIbgE8qiO0FpexeqmipcPW9RqZKUrbLX L5uBNdRL+YulGoMwFp00ZDMxfEasjnOa+u5dwlD55KDI7BMd9FIHvEsDjWfEQqikUMIO EMajy0gB9pKM4jzqUP14LfUDJpPRqOE9jFPHQC3hF1EmbBVo7TkYcppFcdg5pzGGH0+s YfgIzYcaIti+z+aVLge8plNbfpEQYN7SQCbJSP8114CyQgx187x3HC9WQDtSl5ym4i+N Uu8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=9lwBVrqNUMYPuwqdgoT1vXHzudMj33//V8OKMSrB/14=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=FLlw5oHH7wH/B70BD7mEWv8ei8Ca4RrtUwAe83VkhFvMK4iSYSmj6bi3M35xjyYy8v 8D0Q4cgedl1KIv8wX9aIgETTo0RQo1HMzSqeDB6YhG05kxYsOuKVGbiCN9PjSH1OvGhq x5VbW+nNmcKDV/YdOuzkE2cTALCY0bFcXJNOx59YROkx8ByfbUx2yj3yQzzNrIJqDpIw TSJq0cJxsxQ+nwzZCNuMV4a44FSWVt9X6Mq6BrNsFsS8ZlRnbVi6wrNW/Q8Xwpb5Cj/U DxbBqADg7mH463hfWR4UkcbWPF8egmE4F5AvdP+ApcywPOPiAhDqJ5YV7q8XKbNPCEGl JK2g==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="j/1jvYgp"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="OE/F7tPd"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=b4q6pQfM; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=yWcUa4Pv; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-47df951f67esi12564631fac.223.2026.09.15.08.24.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:21 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="j/1jvYgp"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="OE/F7tPd"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=b4q6pQfM; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=yWcUa4Pv; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=9lwBVrqNUMYPuwqdgoT1vXHzudMj33//V8OKMSrB/14=; b=j/1jvYgpFfmgwn6+HOF8OrjNEz pSfGk42ifXGHmOeUE2GdPwurQfWxNdNRMgKPrDVbmsg+91y4D6GFu57+WvD35vPDGBVFUp4gKTShn gWgDM/f537soiFrt+pZw8395B7APtrXpw5Y/hxqdaFwifkcEslCmah89TgdhWNJcxGvw=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V15-0007o9-EV; Tue, 15 Sep 2026 15:24:16 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V13-0007nq-RG for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=NRdb18BUnOsrqKuhyaWH/qtXDqTFOtMnEvqNURJE06s=; b=OE/F7tPdt4DEHTKWqHNNl5O2kF Yjq5nf300TvhVLxauq9RelAqDKIAJeZiXjacck3jAtKNevLPDkECDT8dWKh0dKypIMLKzuL+02L08 tEq5BBtqOCtCSCQDW217vt5h0EJjkWbwtSKBp6yfs9BL74b2IMjwdY7L0AuNrsVH7caU=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=NRdb18BUnOsrqKuhyaWH/qtXDqTFOtMnEvqNURJE06s=; b=b4q6pQfM3Fi+QkOcEy+I/afPdy EDlERkdU59F86pLp/9anUwyMuDOszfmhWge3ImlQiZ03LuVusbkyl7m6ehbUsMTb/QPultJadhxYJ uciHhLE7RMkExiOYuRxCe+gNVAgvNpM2bd0ITXVHm5F7ww59PnpeXlav+hg57qrFeJvE=; Received: from mout-b-206.mailbox.org ([195.10.208.51]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V10-0003bN-QW for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:14 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-206.mailbox.org (Postfix) with ESMTPS id 4hkm5H3SBfzH3 for ; Tue, 15 Sep 2026 17:24:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485843; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NRdb18BUnOsrqKuhyaWH/qtXDqTFOtMnEvqNURJE06s=; b=yWcUa4PvSj9OhZO5PY65wSxug5iwG6+bbh17kTTLX59qM2SPJkdf9vYOKYmZQw45Y+X4ZO 5PW1IhS/B83qDo1ilFEi5KiQeXX8sp/yErM3R8EqCImQ2YKSSKbBNojDnR64cJlIboslI/ KygnYbEVICvHXx76SvlM1lY3e/AqrhtRaJNnepGWBp3ivce12ra1396sN01f5Nb19nZy79 WnS2tDYXGF4yl7rBinA5Ehrbj8MX7RUcRiPY37R3dSIeeHS/8ag3fu9QEXC888Yqma9rTm EZkD9QlRZMU+wQokdAskXUwaLThGYMX2Edi+XMT7eHioQJEuacBxfebqFrTTWQ== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:51 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1x6V10-0003bN-QW Subject: [Openvpn-devel] [RFC ovpn net-next 3/9] ovpn: refactor AEAD encryption helpers X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411927006090884 X-GMAIL-MSGID: 1876411927006090884 Move DATA_V2 framing sizes to the protocol header and factor request allocation and header construction into helpers. This prepares the transmit path for an out-of-place encryption destination without changing packet handling. Signed-off-by: Ralf Lici --- drivers/net/ovpn/crypto_aead.c | 97 ++++++++++++++++++++-------------- drivers/net/ovpn/io.h | 3 +- drivers/net/ovpn/proto.h | 4 ++ 3 files changed, 63 insertions(+), 41 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 2af493fd5735..30299581422d 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -24,9 +24,6 @@ #include "proto.h" #include "skb.h" -#define OVPN_AUTH_TAG_SIZE 16 -#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE) - #define ALG_NAME_AES "gcm(aes)" #define ALG_NAME_CHACHAPOLY "rfc7539(chacha20,poly1305)" @@ -42,7 +39,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * an AEAD request structure with extra space for SG * and IV. * @tfm: the AEAD cipher handle - * @nfrags: the number of fragments in the skb + * @nents: the number of scatterlist entries * * This function calculates the size of a contiguous memory block that includes * the initialization vector (IV), the AEAD request, and an array of scatterlist @@ -54,7 +51,7 @@ static int ovpn_aead_encap_overhead(const struct ovpn_crypto_key_slot *ks) * Return: the size of the temporary memory that needs to be allocated */ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, - const unsigned int nfrags) + const unsigned int nents) { unsigned int len = OVPN_NONCE_SIZE; @@ -70,8 +67,8 @@ static unsigned int ovpn_aead_crypto_tmp_size(struct crypto_aead *tfm, /* round up to the next multiple of the scatterlist alignment */ len = ALIGN(len, __alignof__(struct scatterlist)); - /* add enough space for nfrags + 2 scatterlist entries */ - len += array_size(sizeof(struct scatterlist), nfrags + 2); + /* add enough space for the scatterlist entries */ + len += array_size(sizeof(struct scatterlist), nents); return len; } @@ -135,6 +132,53 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, __alignof__(struct scatterlist)); } +static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, + struct sk_buff *skb, + unsigned int nents, u8 **iv) +{ + struct aead_request *req; + void *tmp; + + /* allocate IV, request and scatterlist entries in one block */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + if (unlikely(!tmp)) + return ERR_PTR(-ENOMEM); + + ovpn_skb_cb(skb)->crypto_tmp = tmp; + *iv = ovpn_aead_crypto_tmp_iv(aead, tmp); + req = ovpn_aead_crypto_tmp_req(aead, *iv); + + return req; +} + +static int ovpn_aead_encrypt_header(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + u8 *iv, u8 *data) +{ + u32 pktid, op; + int ret; + + /* obtain packet ID, which is used both as a first + * 4 bytes of nonce and last 4 bytes of associated data. + */ + ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + if (unlikely(ret < 0)) + return ret; + + /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes + * nonce + */ + ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); + + /* add the packet opcode and wire nonce as associated data */ + op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); + BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); + *(__force __be32 *)data = htonl(op); + memcpy(data + OVPN_OPCODE_SIZE, iv, OVPN_NONCE_WIRE_SIZE); + + return 0; +} + int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { @@ -143,8 +187,6 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *trailer; struct scatterlist *sg; int nfrags, ret; - u32 pktid, op; - void *tmp; u8 *iv; ovpn_skb_cb(skb)->peer = peer; @@ -175,16 +217,9 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->encrypt, nfrags), - GFP_ATOMIC); - if (unlikely(!tmp)) - return -ENOMEM; - - ovpn_skb_cb(skb)->crypto_tmp = tmp; - - iv = ovpn_aead_crypto_tmp_iv(ks->encrypt, tmp); - req = ovpn_aead_crypto_tmp_req(ks->encrypt, iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + if (IS_ERR(req)) + return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); /* sg table: @@ -206,28 +241,12 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, __skb_push(skb, tag_size); sg_set_buf(sg + ret + 1, skb->data, tag_size); - /* obtain packet ID, which is used both as a first - * 4 bytes of nonce and last 4 bytes of associated data. - */ - ret = ovpn_pktid_xmit_next(&ks->pid_xmit, &pktid); + /* make space for the additional data and push it to the front */ + __skb_push(skb, OVPN_AAD_SIZE); + ret = ovpn_aead_encrypt_header(peer, ks, iv, skb->data); if (unlikely(ret < 0)) return ret; - /* concat 4 bytes packet id and 8 bytes nonce tail into 12 bytes - * nonce - */ - ovpn_pktid_aead_write(pktid, ks->nonce_tail_xmit, iv); - - /* make space for packet id and push it to the front */ - __skb_push(skb, OVPN_NONCE_WIRE_SIZE); - memcpy(skb->data, iv, OVPN_NONCE_WIRE_SIZE); - - /* add packet op as head of additional data */ - op = ovpn_opcode_compose(OVPN_DATA_V2, ks->key_id, peer->tx_id); - __skb_push(skb, OVPN_OPCODE_SIZE); - BUILD_BUG_ON(sizeof(op) != OVPN_OPCODE_SIZE); - *((__force __be32 *)skb->data) = htonl(op); - /* AEAD Additional data */ sg_set_buf(sg, skb->data, OVPN_AAD_SIZE); @@ -281,7 +300,7 @@ int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return -ENOSPC; /* allocate temporary memory for iv, sg and req */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags), + tmp = kmalloc(ovpn_aead_crypto_tmp_size(ks->decrypt, nfrags + 2), GFP_ATOMIC); if (unlikely(!tmp)) return -ENOMEM; diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index db9e10f9077c..1a94f0fda1d1 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -11,8 +11,7 @@ #define _NET_OVPN_OVPN_H_ /* DATA_V2 header size with AEAD encryption */ -#define OVPN_HEAD_ROOM (OVPN_OPCODE_SIZE + OVPN_NONCE_WIRE_SIZE + \ - 16 /* AEAD TAG length */ + \ +#define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ max(sizeof(struct ipv6hdr), sizeof(struct iphdr))) diff --git a/drivers/net/ovpn/proto.h b/drivers/net/ovpn/proto.h index b7d285b4d9c1..f3b305cbefe5 100644 --- a/drivers/net/ovpn/proto.h +++ b/drivers/net/ovpn/proto.h @@ -39,6 +39,10 @@ #define OVPN_NONCE_WIRE_SIZE (OVPN_NONCE_SIZE - OVPN_NONCE_TAIL_SIZE) #define OVPN_OPCODE_SIZE 4 /* DATA_V2 opcode size */ +#define OVPN_AUTH_TAG_SIZE 16 +#define OVPN_AAD_SIZE (OVPN_OPCODE_SIZE + \ + OVPN_NONCE_WIRE_SIZE) +#define OVPN_DATA_V2_OVERHEAD (OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE) #define OVPN_OPCODE_KEYID_MASK 0x07000000 #define OVPN_OPCODE_PKTTYPE_MASK 0xF8000000 #define OVPN_OPCODE_PEERID_MASK 0x00FFFFFF From patchwork Tue Sep 15 15:23:52 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5342 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5063806mag; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvByLibLtRH1AoiiCGX21w4xE/XvdEL/ycIq3wAemZC+GIOn3KX8Ga9nwAuVaeeyJDd3mDcjKNuh9JaU=@openvpn.net X-Received: by 2002:a05:6870:c2a5:b0:471:3325:80ca with SMTP id 586e51a60fabf-481f813e10cmr10739081fac.5.1789485863020; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485863; cv=none; d=google.com; s=arc-20260327; b=I6QlUZIumNhQ87AdWlCQPE2TXe4v1JdFCDlw7c1kjdHcPrQLeLG1fcRNv/gfbSgNnQ YEpJJWZMAqE6BtfG8UjkJwCNrAM3PBipM9fN7fMHZcwDcnFtMZrJGmea8J7pC4FpNUa2 xuv6WBVBGQEwuNOpH30VKStkswelGlnoWv8ygF+Ulp5BypeL+RP7lhzi1m0VvLMoOxoP G6J2MsrQNZ4r1LIZfGasTUJJSql86wfRlsCvktx7upuk+dU7eAQkN2bUD3n3AuLnOlMy 7sWGFeootL+yNmFnugdlw+zvp2CQc66RwuOW62HLC/Wa/oQWiVte/vxJH9rv4p5s6TMx eTIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=z+DCpb6b8Y4JHAev2GjyiuAbNCEeq/H75HX6FofoYq4=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=Ry4Mv7DztYHC4Q1jMNEq/5NQoAFiuEZwBJgAn64mmor3OLEa9gsz0HpZBqA/Z6Andi ki94Ij9mD9zTldsy4KR2S1nVjqGMdaZKdr3HmjxgdanabIzKe7HuRfF9yApZbzM0Lzpe /3/xe3ZKGgqRew2lDRRzRuen3HedGqlRVTsgIhMDjqr2jIT/qXPme7Q4sOt8FvFinsgK X/GfBU459qbN8vHfQgwg0tcwIOHuZu+e/STWwweZUGtH11GNfkbiVYcAsGAwq3TlO8YV 1WSxRWmEAs/QJUetVOb+PmQYikcqxRpH+szcdEf2w9rD7NcyaTrfgd8boQX+ncM7Udex cRGg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=MxzbbsrD; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=fj2uL5Rx; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=gZCoSRS7; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=sXEe4zwB; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-47df9d60ea8si12738962fac.333.2026.09.15.08.24.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:22 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=MxzbbsrD; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=fj2uL5Rx; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=gZCoSRS7; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=sXEe4zwB; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=z+DCpb6b8Y4JHAev2GjyiuAbNCEeq/H75HX6FofoYq4=; b=MxzbbsrDLoE8HKNeneJ35yEVFr zziN3bICeot2D/30KR/rQsvKU1FopHXXeey29sOkcdFRdEu4Pgau/sJdMf4+b95pARMookI/jz3Tk EsmlboJHO4dWCXXnl/3aBmatzkCtQnABToRp09/V009IWbbsQzMdRumMt3mq2548Ig38=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V19-0005pw-FK; Tue, 15 Sep 2026 15:24:19 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V14-0005pc-T8 for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=k31Tj1nt5VdSiJTe+ku2KhEjxiFFG1z0Dwyc1EXIjBk=; b=fj2uL5RxL+E6YKub/lJrUxhVTK BzIpURS+BiwUnkS1RXZGkTOJdDbP55p+KnCXx8/K7LAi1n77l6OeklOAyOX2j3BoPZtLukgPgEkOn umEJAUO07rPHcL6smZf60lr3niKYptPCFoYAU84HVTIVpRJ5wRkfG929pXlRz/iwHAfY=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=k31Tj1nt5VdSiJTe+ku2KhEjxiFFG1z0Dwyc1EXIjBk=; b=gZCoSRS7Yy/qUzqtiz6H8CvL1p IahJpvVDslIjNemJuiZU2mh4fBqBjdrUvNaHaEsDGCYKLanF4m2pmA8PC8cnVynpcmiCj+xA1MOst k+1Z6fUB5NIUmLtFsvpD7y7/mBY/n21t1F3ammQxSFHoMv9zqhQjwc+Z0qgP79iRa29Q=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V11-00076z-AP for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:14 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4hkm5H75cmz5x0D for ; Tue, 15 Sep 2026 17:24:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485844; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k31Tj1nt5VdSiJTe+ku2KhEjxiFFG1z0Dwyc1EXIjBk=; b=sXEe4zwBzSLoIjZuomNs70XNmndPH7R9zVzONAOguqAsAhE74dmY70jWEmCOs41Rpai+lL UxfHPu6LsYfKUQT8Ik/MB5vvNFGn65F1Fq8HyGGomr/I/2ZeXamw8LZuwaKFKH8jYVGWRW XaPjZwIfDDhzNac0P1XXsQjmA76ZH+hXCMViaQiWPfjg4xkUU8EHcvUEo0wAOm0Qn+1Nqr nAVxxA7+yan6Dror7JM54/BvHDAkwRP9IhOiTqOrrckHJ/LRa9xs/TXasoSyi2Zw0HAgak a6X97BFPcIFa1R3eTF4C/HFxcTWYhTAO9nVCdjQ70gohyitwLffr9t1Ex3wYsw== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:52 +0200 Message-ID: <9359f737627fc84be08ee7d301415c023ebadeb2.1789485693.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb into linear packets while completing their checksums, then emit eligible fixed-size inputs as UDP GSO skb(s). Allocate one final page-backed aggregate per batch before submitting encryption and have each AEAD request write out of place directly into its record slot. Attempting in-place encryption would be com [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6V11-00076z-AP Subject: [Openvpn-devel] [RFC ovpn net-next 4/9] ovpn: convert GSO input into UDP GSO output X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411928191643553 X-GMAIL-MSGID: 1876411928191643553 Whenever a GSO skb arrives at ovpn's ndo_start_xmit, segment the inner skb into linear packets while completing their checksums, then emit eligible fixed-size inputs as UDP GSO skb(s). Allocate one final page-backed aggregate per batch before submitting encryption and have each AEAD request write out of place directly into its record slot. Attempting in-place encryption would be complex (the OpenVPN wire layout adds a header and authentication tag to every segment) and not necessarily more performant: encrypting into individual skbs would still require assembling or copying those records into the UDP GSO skb. The destination allocation and lifetime are instead amortized over the whole batch. Keep the existing in-place path for ordinary packets, where allocating and retiring a separate output skb for every record would provide no aggregate construction benefit. Also retain that path for frag-list GSO input: it already stores complete segments as child skbs, and measurements show that copying those children into another aggregate is counterproductive. Transmit the aggregate as SKB_GSO_UDP_L4 only after every record succeeds, and discard it if any request fails. Split aggregates at the legacy GSO size limit and fall back to individual records when batching is unavailable or the segment geometry is unsuitable. Preserve the input priority, flow hash and sender CPU on the replacement aggregate. If the input has real write ownership, charge the aggregate to the same socket as well. This retains socket lifetime and write-memory accounting and lets lower-device queue selection use the socket's cached TX queue instead of choosing a new queue after crypto completion. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following throughput: Forward Reverse Before this change 11.522 Gbit/s 9.902 Gbit/s Software UDP segmentation 12.879 Gbit/s 12.516 Gbit/s Hardware UDP segmentation 18.308 Gbit/s 19.233 Gbit/s The equal-weight mean of the two directional results increased from 10.712 to 18.770 Gbit/s with hardware UDP segmentation, a 75.2% improvement. With segmentation performed in software, it increased to 12.697 Gbit/s, an 18.5% improvement. Signed-off-by: Ralf Lici --- drivers/net/ovpn/crypto_aead.c | 85 ++++++++++- drivers/net/ovpn/crypto_aead.h | 4 + drivers/net/ovpn/io.c | 258 ++++++++++++++++++++++++++++++--- drivers/net/ovpn/skb.h | 27 +++- drivers/net/ovpn/stats.h | 16 +- drivers/net/ovpn/tcp.c | 4 +- drivers/net/ovpn/udp.c | 27 +++- 7 files changed, 382 insertions(+), 39 deletions(-) diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 30299581422d..8eb76268dc3a 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -134,13 +134,17 @@ static struct scatterlist *ovpn_aead_crypto_req_sg(struct crypto_aead *aead, static struct aead_request *ovpn_aead_request_alloc(struct crypto_aead *aead, struct sk_buff *skb, - unsigned int nents, u8 **iv) + unsigned int nents, + unsigned int extra, u8 **iv) { struct aead_request *req; void *tmp; - /* allocate IV, request and scatterlist entries in one block */ - tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents), GFP_ATOMIC); + /* allocate IV, request, scatterlist entries and caller scratch space + * in one block + */ + tmp = kmalloc(ovpn_aead_crypto_tmp_size(aead, nents) + extra, + GFP_ATOMIC); if (unlikely(!tmp)) return ERR_PTR(-ENOMEM); @@ -217,7 +221,7 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, nfrags = 1; } - req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, &iv); + req = ovpn_aead_request_alloc(ks->encrypt, skb, nfrags + 2, 0, &iv); if (IS_ERR(req)) return PTR_ERR(req); sg = ovpn_aead_crypto_req_sg(ks->encrypt, req); @@ -261,6 +265,79 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, return crypto_aead_encrypt(req); } +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, struct sk_buff *skb, + struct sk_buff *gso_skb, unsigned int offset) +{ + const unsigned int dst_nents = skb_shinfo(gso_skb)->nr_frags + 4; + const unsigned int src_nents = 2; + unsigned int nents, payload_off; + struct scatterlist *src, *dst; + struct aead_request *req; + int dst_idx, mapped, ret; + u8 *aad, *iv; + + /* each input records the shared peer and key for the common completion + * path but their references remain owned by the output aggregate + */ + ovpn_skb_cb(skb)->peer = peer; + ovpn_skb_cb(skb)->ks = ks; + + if (WARN_ON_ONCE(skb_is_nonlinear(skb))) + return -EINVAL; + + nents = src_nents + dst_nents; + req = ovpn_aead_request_alloc(ks->encrypt, skb, nents, OVPN_AAD_SIZE, + &iv); + if (IS_ERR(req)) + return PTR_ERR(req); + src = ovpn_aead_crypto_req_sg(ks->encrypt, req); + dst = src + src_nents; + aad = (u8 *)(dst + dst_nents); + + ret = ovpn_aead_encrypt_header(peer, ks, iv, aad); + if (unlikely(ret < 0)) + return ret; + + ret = skb_store_bits(gso_skb, offset, aad, OVPN_AAD_SIZE); + if (unlikely(ret < 0)) + return ret; + + /* encrypt out of place from the original segmented skb directly into + * its final range in the UDP GSO skb + */ + sg_init_table(src, src_nents); + sg_set_buf(src, aad, OVPN_AAD_SIZE); + sg_set_buf(src + 1, skb->data, skb->len); + + sg_init_table(dst, dst_nents); + dst_idx = skb_to_sgvec_nomark(gso_skb, dst, offset, OVPN_AAD_SIZE); + if (unlikely(dst_idx < 0)) + return dst_idx; + + payload_off = offset + OVPN_AAD_SIZE + OVPN_AUTH_TAG_SIZE; + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, payload_off, + skb->len); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + + mapped = skb_to_sgvec_nomark(gso_skb, dst + dst_idx, + offset + OVPN_AAD_SIZE, + OVPN_AUTH_TAG_SIZE); + if (unlikely(mapped < 0)) + return mapped; + dst_idx += mapped; + sg_mark_end(&dst[dst_idx - 1]); + + aead_request_set_tfm(req, ks->encrypt); + aead_request_set_callback(req, 0, ovpn_encrypt_post, skb); + aead_request_set_crypt(req, src, dst, skb->len, iv); + aead_request_set_ad(req, OVPN_AAD_SIZE); + + return crypto_aead_encrypt(req); +} + int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb) { diff --git a/drivers/net/ovpn/crypto_aead.h b/drivers/net/ovpn/crypto_aead.h index fae3b585a43b..8b444744944e 100644 --- a/drivers/net/ovpn/crypto_aead.h +++ b/drivers/net/ovpn/crypto_aead.h @@ -17,6 +17,10 @@ int ovpn_aead_encrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); +int ovpn_aead_encrypt_gso(struct ovpn_peer *peer, + struct ovpn_crypto_key_slot *ks, + struct sk_buff *skb, struct sk_buff *gso_skb, + unsigned int offset); int ovpn_aead_decrypt(struct ovpn_peer *peer, struct ovpn_crypto_key_slot *ks, struct sk_buff *skb); diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 112067ded401..3ad4cadeeb02 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -13,6 +13,8 @@ #include #include #include +#include +#include #include "ovpnpriv.h" #include "peer.h" @@ -32,6 +34,12 @@ const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE] = { 0x07, 0xed, 0x2d, 0x0a, 0x98, 0x1f, 0xc7, 0x48 }; +/* Leave room for the largest outer network header. The strict inequality in + * is_skb_forwardable also requires staying one byte below gso_max_size. + */ +#define OVPN_UDP_GSO_MAX_PAYLOAD (GSO_LEGACY_MAX_SIZE - \ + sizeof(struct ipv6hdr) - \ + sizeof(struct udphdr) - 1) /** * ovpn_is_keepalive - check if skb contains a keepalive message * @skb: packet to check @@ -237,11 +245,13 @@ void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb) void ovpn_encrypt_post(void *data, int ret) { + unsigned int orig_len, packets = 1; struct ovpn_crypto_key_slot *ks; struct sk_buff *skb = data; struct ovpn_socket *sock; + struct ovpn_cb *batch_cb; struct ovpn_peer *peer; - unsigned int orig_len; + struct sk_buff *batch; /* encryption is happening asynchronously. This function will be * called later by the crypto callback with a proper return value @@ -249,15 +259,19 @@ void ovpn_encrypt_post(void *data, int ret) if (unlikely(ret == -EINPROGRESS)) return; - ks = ovpn_skb_cb(skb)->ks; + /* ordinary encryption leaves batch zeroed; a GSO input uses it to find + * the aggregate whose lifetime is shared by all segment requests + */ + batch = ovpn_skb_cb(skb)->batch; peer = ovpn_skb_cb(skb)->peer; + ks = ovpn_skb_cb(skb)->ks; /* crypto is done, cleanup skb CB and its members */ kfree(ovpn_skb_cb(skb)->crypto_tmp); if (unlikely(ret == -ERANGE)) { /* we ran out of IVs and we must kill the key as it can't be - * use anymore + * used anymore */ netdev_warn(peer->ovpn->dev, "killing key %u for peer %u\n", ks->key_id, @@ -265,8 +279,30 @@ void ovpn_encrypt_post(void *data, int ret) if (ovpn_crypto_kill_key(&peer->crypto, ks->key_id)) /* let userspace know so that a new key must be negotiated */ ovpn_nl_key_swap_notify(peer, ks->key_id); + } - goto err; + if (batch) { + batch_cb = ovpn_skb_cb(batch); + /* every segment publishes its result before releasing its + * pending count and only the final completion continues with + * the aggregate + */ + if (unlikely(ret < 0)) + atomic_set(&batch_cb->batch_state.failed, 1); + + kfree_skb(skb); + if (!atomic_dec_and_test(&batch_cb->batch_state.pending)) + return; + + skb = batch; + packets = skb_shinfo(batch)->gso_segs; + if (unlikely(atomic_read(&batch_cb->batch_state.failed))) + goto err; + + /* reaching the final callback with no sticky failure means + * every segment completed successfully + */ + ret = 0; } if (unlikely(ret < 0)) @@ -292,7 +328,7 @@ void ovpn_encrypt_post(void *data, int ret) goto err_unlock; } - ovpn_peer_stats_increment_tx(&peer->link_stats, orig_len); + ovpn_peer_stats_add_tx(&peer->link_stats, orig_len, packets); /* keep track of last sent packet for keepalive */ WRITE_ONCE(peer->last_sent, ktime_get_boottime_seconds()); /* skb passed down the stack - don't free it */ @@ -301,7 +337,7 @@ void ovpn_encrypt_post(void *data, int ret) rcu_read_unlock(); err: if (unlikely(skb)) - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, packets); kfree_skb(skb); if (likely(ks)) ovpn_crypto_key_slot_put(ks); @@ -309,29 +345,124 @@ void ovpn_encrypt_post(void *data, int ret) ovpn_peer_put(peer); } -static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +/* Hold the peer and its primary key for one encryption submission. + * The returned key and the peer each carry one reference which completion must + * release. + */ +static struct ovpn_crypto_key_slot * +ovpn_encrypt_refs_get(struct ovpn_peer *peer) { struct ovpn_crypto_key_slot *ks; /* get primary key to be used for encrypting data */ ks = ovpn_crypto_key_slot_primary(&peer->crypto); if (unlikely(!ks)) - return false; + return NULL; - /* take a reference to the peer because the crypto code may run async. - * ovpn_encrypt_post() will release it upon completion + /* the caller already owns a peer reference, so failure indicates a + * broken reference lifetime elsewhere */ if (unlikely(!ovpn_peer_hold(peer))) { DEBUG_NET_WARN_ON_ONCE(1); ovpn_crypto_key_slot_put(ks); - return false; + return NULL; } + return ks; +} + +static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct ovpn_crypto_key_slot *ks; + + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_encrypt_post(skb, ovpn_aead_encrypt(peer, ks, skb)); return true; } +static bool ovpn_encrypt_gso_queue(struct sk_buff_head *skbs, + struct ovpn_peer *peer, + struct sk_buff *batch, + unsigned int segments) +{ + unsigned int offset = 0, i, len; + struct ovpn_crypto_key_slot *ks; + struct sk_buff *skb; + + /* acquire all shared state before removing the first input skb so that + * failure can leave the queue intact for the ordinary transmit path + */ + ks = ovpn_encrypt_refs_get(peer); + if (unlikely(!ks)) + return false; + + /* the aggregate owns these references until every sync or async crypto + * completion has finished + */ + memset(ovpn_skb_cb(batch), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(batch)->peer = peer; + ovpn_skb_cb(batch)->ks = ks; + atomic_set(&ovpn_skb_cb(batch)->batch_state.pending, segments); + atomic_set(&ovpn_skb_cb(batch)->batch_state.failed, 0); + + for (i = 0; i < segments; i++) { + skb = __skb_dequeue(skbs); + len = skb->len + OVPN_DATA_V2_OVERHEAD; + + memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); + ovpn_skb_cb(skb)->batch = batch; + ovpn_encrypt_post(skb, ovpn_aead_encrypt_gso(peer, ks, skb, + batch, offset)); + offset += len; + } + + return true; +} + +static struct sk_buff *ovpn_udp_gso_alloc(const struct sk_buff *first_segment, + unsigned int batch_len, + unsigned int segments) +{ + struct sk_buff *gso_skb; + int ret; + + gso_skb = alloc_skb_with_frags(OVPN_HEAD_ROOM, batch_len, + SKB_FRAG_PAGE_ORDER, &ret, GFP_ATOMIC); + if (unlikely(!gso_skb)) + return NULL; + + skb_reserve(gso_skb, OVPN_HEAD_ROOM); + gso_skb->len = batch_len; + gso_skb->data_len = batch_len; + gso_skb->priority = first_segment->priority; + + /* Segments retain the originating socket so we keep its send-buffer + * accounting active until the UDP GSO is transmitted. This also + * preserves its cached TX queue. + */ + if (first_segment->sk && is_skb_wmem(first_segment)) + skb_set_owner_w(gso_skb, first_segment->sk); + skb_copy_hash(gso_skb, first_segment); +#ifdef CONFIG_XPS + /* keep the aggregate on the TX queue selected for the original flow + * otherwise async crypto completion on another CPU could move the flow + * to a different queue and cause delay or reordering + */ + gso_skb->sender_cpu = first_segment->sender_cpu; +#endif + + skb_shinfo(gso_skb)->gso_type = SKB_GSO_UDP_L4; + skb_shinfo(gso_skb)->gso_size = first_segment->len + + OVPN_DATA_V2_OVERHEAD; + skb_shinfo(gso_skb)->gso_segs = segments; + + return gso_skb; +} + /* send skb to connected peer, if any */ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, struct ovpn_peer *peer) @@ -343,7 +474,7 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, */ skb_list_walk_safe(skb, curr, next) { if (unlikely(!ovpn_encrypt_one(peer, curr))) { - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); } } @@ -351,19 +482,96 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, ovpn_peer_put(peer); } +/* encrypt fixed-size input segments into one or more UDP GSO aggregates */ +static void ovpn_send_gso(struct sk_buff_head *skbs, struct ovpn_peer *peer) +{ + unsigned int max_segs = 1, seg_len, batch_len, segs; + struct sk_buff *batch; + + seg_len = skb_peek(skbs)->len + OVPN_DATA_V2_OVERHEAD; + max_segs = min_t(unsigned int, UDP_MAX_SEGMENTS, + OVPN_UDP_GSO_MAX_PAYLOAD / seg_len); + + /* if even two encrypted skbs cannot fit, leave the whole queue for the + * ordinary transmit path + */ + if (max_segs < 2) + return; + + /* An input GSO skb might be prduce more than max_segs segments so we + * consume as many as we can for each iteration. A final single skb, or + * the whole remainder after an allocation failure, stays queued and + * fallback to the ordinary transmit path. + */ + while (skb_queue_len(skbs) > 1) { + segs = min_t(unsigned int, skb_queue_len(skbs), max_segs); + + /* all but the final input skb have the same length, so start + * with the full-size calculation and adjust only the final + * group below + */ + batch_len = segs * (skb_peek(skbs)->len + + OVPN_DATA_V2_OVERHEAD); + if (segs == skb_queue_len(skbs)) + batch_len -= skb_peek(skbs)->len - + skb_peek_tail(skbs)->len; + + batch = ovpn_udp_gso_alloc(skb_peek(skbs), batch_len, segs); + if (unlikely(!batch)) + return; + + if (unlikely(!ovpn_encrypt_gso_queue(skbs, peer, + batch, segs))) { + kfree_skb(batch); + return; + } + } +} + +static bool ovpn_peer_supports_udp_gso(struct ovpn_peer *peer) +{ + struct ovpn_socket *sock; + bool udp_gso; + + rcu_read_lock(); + sock = rcu_dereference(peer->sock); + /* UDP GSO requires checksums. These socket settings can change after + * we decide to batch, but an already-built batch remains checksummed. + * Linux's ordinary UDP GSO path makes the same choice. + */ + udp_gso = sock && sock->sk->sk_protocol == IPPROTO_UDP && + !sock->sk->sk_no_check_tx && !udp_get_no_check6_tx(sock->sk); + rcu_read_unlock(); + + return udp_gso; +} + /* Send user data to the network */ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) { struct ovpn_priv *ovpn = netdev_priv(dev); struct sk_buff *segments, *curr, *next; + const bool gso_in = skb_is_gso(skb); struct sk_buff_head skb_list; netdev_features_t features; unsigned int tx_bytes = 0; struct ovpn_peer *peer; + bool gso_out; __be16 proto; int ret; + /* A frag-list GSO skb already stores complete segments as child skbs. + * Keep those children on the ordinary in-place encryption path instead + * of copying them into a replacement UDP GSO skb. + * + * GSO_BY_FRAGS input must also remain on that path because its variable + * segment sizes cannot be represented by one UDP GSO output size. + */ + gso_out = gso_in && + !(skb_shinfo(skb)->gso_type & SKB_GSO_FRAGLIST) && + skb_shinfo(skb)->gso_size != GSO_BY_FRAGS; + /* reset netfilter state */ nf_reset_ct(skb); @@ -392,13 +600,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) /* dst was needed for peer selection - it can now be dropped */ skb_dst_drop(skb); - if (skb_is_gso(skb)) { - /* force software segmentation, but keep ovpn's non-GSO feature - * bits so the generated segments can preserve non-linear skb - * data where possible + if (gso_in) { + /* force software segmentation into linear skbs and calculate + * each checksum while copying the segment */ features = netif_skb_features(skb); - segments = skb_gso_segment(skb, features & ~NETIF_F_GSO_MASK); + features &= ~(NETIF_F_GSO_MASK | NETIF_F_SG | + NETIF_F_CSUM_MASK); + segments = skb_gso_segment(skb, features); if (IS_ERR_OR_NULL(segments)) { ret = PTR_ERR(segments); net_err_ratelimited("%s: cannot segment payload packet: %d\n", @@ -420,7 +629,8 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) if (unlikely(!curr)) { net_err_ratelimited("%s: skb_share_check failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); + gso_out = false; continue; } @@ -429,8 +639,9 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) skb_checksum_help(curr) < 0)) { net_err_ratelimited("%s: skb_checksum_help failed for payload packet\n", netdev_name(dev)); - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); + gso_out = false; continue; } @@ -446,9 +657,14 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) ovpn_peer_put(peer); return NETDEV_TX_OK; } - skb_list.prev->next = NULL; ovpn_peer_stats_increment_tx(&peer->vpn_stats, tx_bytes); + + if (gso_out && skb_queue_len(&skb_list) > 1 && + ovpn_peer_supports_udp_gso(peer)) + ovpn_send_gso(&skb_list, peer); + + skb_list.prev->next = NULL; ovpn_send(ovpn, skb_list.next, peer); return NETDEV_TX_OK; @@ -456,7 +672,7 @@ netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev) drop: ovpn_peer_put(peer); drop_no_peer: - ovpn_dev_dstats_tx_dropped(ovpn->dev); + ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); skb_tx_error(skb); kfree_skb_list(skb); return NETDEV_TX_OK; diff --git a/drivers/net/ovpn/skb.h b/drivers/net/ovpn/skb.h index 4fb7ea025426..cca29479c038 100644 --- a/drivers/net/ovpn/skb.h +++ b/drivers/net/ovpn/skb.h @@ -10,6 +10,7 @@ #ifndef _NET_OVPN_SKB_H_ #define _NET_OVPN_SKB_H_ +#include #include #include #include @@ -20,19 +21,35 @@ /** * struct ovpn_cb - ovpn skb control block - * @peer: the peer this skb was received from/sent to - * @ks: the crypto key slot used to encrypt/decrypt this skb * @crypto_tmp: pointer to temporary memory used for crypto operations * containing the IV, the scatter gather list and the aead request + * @peer: peer used by this crypto operation or owned by this aggregate + * @ks: crypto key slot used by this operation or owned by this aggregate * @payload_offset: offset in the skb where the payload starts * @nosignal: whether this skb should be sent with the MSG_NOSIGNAL flag (TCP) + * @batch: UDP GSO aggregate receiving this input skb's encrypted payload + * @batch_state: completion state owned by a UDP GSO aggregate */ struct ovpn_cb { + void *crypto_tmp; struct ovpn_peer *peer; struct ovpn_crypto_key_slot *ks; - void *crypto_tmp; - unsigned int payload_offset; - bool nosignal; + + /* Ordinary encryption leaves this union zeroed. Decryption and TCP use + * their ordinary fields, a UDP GSO input stores its output aggregate, + * and that aggregate uses the same space to coordinate its completions. + */ + union { + struct { + unsigned int payload_offset; + bool nosignal; + }; + struct sk_buff *batch; + struct { + atomic_t pending; + atomic_t failed; + } batch_state; + }; }; static inline struct ovpn_cb *ovpn_skb_cb(struct sk_buff *skb) diff --git a/drivers/net/ovpn/stats.h b/drivers/net/ovpn/stats.h index 3a45b97c0056..b3fe006c01f6 100644 --- a/drivers/net/ovpn/stats.h +++ b/drivers/net/ovpn/stats.h @@ -40,16 +40,26 @@ static inline void ovpn_peer_stats_increment_rx(struct ovpn_peer_stats *stats, ovpn_peer_stats_increment(&stats->rx, n); } +static inline void ovpn_peer_stats_add_tx(struct ovpn_peer_stats *stats, + const unsigned int bytes, + unsigned int packets) +{ + atomic64_add(bytes, &stats->tx.bytes); + atomic64_add(packets, &stats->tx.packets); +} + static inline void ovpn_peer_stats_increment_tx(struct ovpn_peer_stats *stats, const unsigned int n) { - ovpn_peer_stats_increment(&stats->tx, n); + ovpn_peer_stats_add_tx(stats, n, 1); } -static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev) +static inline void ovpn_dev_dstats_tx_dropped(struct net_device *dev, + unsigned int packets) { local_bh_disable(); - dev_dstats_tx_dropped(dev); + while (packets--) + dev_dstats_tx_dropped(dev); local_bh_enable(); } diff --git a/drivers/net/ovpn/tcp.c b/drivers/net/ovpn/tcp.c index 8fe8a8e750a4..5cba35e4a8ee 100644 --- a/drivers/net/ovpn/tcp.c +++ b/drivers/net/ovpn/tcp.c @@ -332,7 +332,7 @@ static void ovpn_tcp_send_sock_skb(struct ovpn_peer *peer, struct sock *sk, ovpn_tcp_send_sock(peer, sk); if (peer->tcp.out_msg.skb) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); return; } @@ -354,7 +354,7 @@ void ovpn_tcp_send_skb(struct ovpn_peer *peer, struct sock *sk, if (sock_owned_by_user(sk)) { if (skb_queue_len(&peer->tcp.out_queue) >= READ_ONCE(net_hotdata.max_backlog)) { - ovpn_dev_dstats_tx_dropped(peer->ovpn->dev); + ovpn_dev_dstats_tx_dropped(peer->ovpn->dev, 1); kfree_skb(skb); goto unlock; } diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 7f69e8890b5b..4802d982de08 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -121,6 +121,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); + skb_mark_not_on_list(skb); ovpn_recv(peer, skb); return 0; @@ -196,9 +197,13 @@ static int ovpn_udp4_output(struct ovpn_peer *peer, struct ovpn_bind *bind, dst_cache_set_ip4(cache, &rt->dst, fl.saddr); transmit: + /* an already-built UDP GSO needs a checksum seed even if the socket's + * no-check option changed while encryption was in flight + */ udp_tunnel_xmit_skb(rt, sk, skb, fl.saddr, fl.daddr, 0, ip4_dst_hoplimit(&rt->dst), 0, fl.fl4_sport, - fl.fl4_dport, false, sk->sk_no_check_tx, 0); + fl.fl4_dport, false, + !skb_is_gso(skb) && sk->sk_no_check_tx, 0); ret = 0; err: local_bh_enable(); @@ -271,9 +276,13 @@ static int ovpn_udp6_output(struct ovpn_peer *peer, struct ovpn_bind *bind, * udp_tunnel_xmit_skb() */ skb->ignore_df = 1; + /* keep checksum offload enabled for an in-flight UDP GSO batch even if + * the socket's no-check option has changed since batch creation + */ udp_tunnel6_xmit_skb(dst, sk, skb, skb->dev, &fl.saddr, &fl.daddr, 0, ip6_dst_hoplimit(dst), 0, fl.fl6_sport, - fl.fl6_dport, udp_get_no_check6_tx(sk), 0); + fl.fl6_dport, + !skb_is_gso(skb) && udp_get_no_check6_tx(sk), 0); ret = 0; err: local_bh_enable(); @@ -344,8 +353,18 @@ void ovpn_udp_send_skb(struct ovpn_peer *peer, struct sock *sk, skb->dev = peer->ovpn->dev; skb->mark = READ_ONCE(sk->sk_mark); - /* no checksum performed at this layer */ - skb->ip_summed = CHECKSUM_NONE; + if (skb_is_gso(skb)) { + /* udp_tunnel_xmit_skb installs the outer UDP header after this + * function returns: point CHECKSUM_PARTIAL at that future + * header so both hw and sw UDP GSO can complete the checksum. + */ + skb->ip_summed = CHECKSUM_PARTIAL; + skb->csum_start = skb_headroom(skb) - sizeof(struct udphdr); + skb->csum_offset = offsetof(struct udphdr, check); + } else { + /* no checksum performed at this layer */ + skb->ip_summed = CHECKSUM_NONE; + } /* crypto layer -> transport (UDP) */ ret = ovpn_udp_output(peer, &peer->dst_cache, sk, skb); From patchwork Tue Sep 15 15:23:53 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5341 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5063804mag; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvByYOqmZwzzE1sIrja5csLn8lKQdrK5ZTjJATidviLrj/2XscSoTAgnKupfoqsDimDw394sLolqGTfw=@openvpn.net X-Received: by 2002:a05:6871:e0f5:b0:475:a112:1269 with SMTP id 586e51a60fabf-481f9b11682mr5561617fac.22.1789485863018; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485863; cv=none; d=google.com; s=arc-20260327; b=ShcrE5691FKsKhymKGKH7xA6MG06Qu6HriuuTl2zzARPUzolAOMK53gncRV1sILppK MFHIdZNxkJhNHYEjoOCcN28JD/GkABzFU9gg1+Wb6FvTlyADmXrZY8Iay7a3YhtHDKqo TVNITgGFFhceOdFnIbv4A3polc/br9/4t8aGRi1HQl6W88gqrbFaDCxWINRIkcv3LM4E JQwY60AWsY0AaPZS0BkDBdnsiotj7/tmlMpkXruvfsjZZrGsPIAQJfaxR21MmYs+lZv7 KfKj+8GaTNgAgsOUGLPHHyjUIJMKCN15i6LqA10T54tkRGn1AEIKS2o0S+Z4U1DS0Mff 7mVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=hjWa4bUaHwlNNp2J/W+LHhiTvU+l0dE21bGwQLcklao=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=iWJYMyLzxsQRiVG0DHwUNuIYZTnn4inDQcDN+y3tEcdPA/FITHVNN+6hzsV1psu6wK 6wjnoCdvTcJJm7mgw7ScP5LsODj0dbEVMYw5JzG9Fr+BoogL+Qiaa1byXuQcJjAOjkn6 PHHG6lwP6Ze6MeGVcrlFhNJffd1KUslFlMyXsXrt8DXn5X/klgQWXMvw/jHXWKQylVo0 nJRgMy6+0Eot0oJOHQgPbta9NRMUM700pVKwU0f4EHkdIeI+kmnl0LsuVWdnVEuke+/1 u10b1yQCR4SCIzX+ny3DVEUjW2cN51mFkJ5WE87BX5DcX6pkvvxC7Rf2Aw9uvOdPL2t7 aKNA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=E+yorcHt; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=bn6nD2ba; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=cGiOFRug; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=rFx3yZec; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-47df8ff98adsi11071926fac.125.2026.09.15.08.24.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:22 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=E+yorcHt; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=bn6nD2ba; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=cGiOFRug; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=rFx3yZec; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hjWa4bUaHwlNNp2J/W+LHhiTvU+l0dE21bGwQLcklao=; b=E+yorcHtuqXzbLP+0zTkau7oWo HWN1xb5AowLVrMTQE1Ge8n0Hg2Fe2gyfXv3o87S/EvaKO0OVC0UwmlClhzcAsB5FbD+0ngVkhJtVz fGc6Ok8vUf6/yUY+f0lQAILQG7SsQ3FguMOvTtxxXB4g+cGbt5HoyeKPtZqgyP3nf6LA=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V1B-0007II-Hj; Tue, 15 Sep 2026 15:24:19 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V17-0007Hy-MQ for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Y6ZcSuLvQ6X5Ydek+ixWKUb5I/qvCbobOvM28/42oS8=; b=bn6nD2ba+Cz5CzCY/3uQ9kVKAc JegkYTkOx1DHuow+vRDh+ZVRV+fEpbpvQB4uHXeEAzdXV15V3PeUabK9qsUa4la//SiX51oG7tqke SkqqWiLSaxNhTXKaFo0GyLfUNfSuo3kI4Q3k8+UBacpJDZ2SJkfEuSjHVttYVRr54WH4=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=Y6ZcSuLvQ6X5Ydek+ixWKUb5I/qvCbobOvM28/42oS8=; b=cGiOFRugo5QOB28SX5Dv9RGAMK o1BgIns9BJz0HD9W14qyJA5TYHV6CUmzvhRAhkz3yevNyZNw0SXMjCZ9A+F41wc2VBWAvIXQSFCKd AlmzKfhCpEnanyAOvywXupH53ANgKQVvlif5jtOEFcW9FTzS1dcyaTE47oUw26qDy6mY=; Received: from mout-b-203.mailbox.org ([195.10.208.52]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V11-000771-Ru for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:15 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-203.mailbox.org (Postfix) with ESMTPS id 4hkm5J432DzLmF9 for ; Tue, 15 Sep 2026 17:24:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485844; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Y6ZcSuLvQ6X5Ydek+ixWKUb5I/qvCbobOvM28/42oS8=; b=rFx3yZec2cYmQjPU8lpec6q6atMoiwLr3g9Cdmod1xiUVzbyAAtwjGhky0INhN+NCdViFq 897csrGf4NfOmB82uNvWgHQiSQgtm8U5Sbml2AcK2FZBxof0ACDL5uTxuBMRTCDCz3yUMf psAn6M2RZBmBg5mdU4z8y28ukRvhon8T7s3x3svSbi6Bz3U2t7vN3z0g6ls8OovYzaiYTb QCLEoT7Iynbic4NbHHPSu+29gTAm5Ga9KG7LmNQhgxnMdtii7MeXnwQpFAZNFUzVRmOMwp Fgxjb4gWNJVPQ0zMp1B7pTGUoYv+vahbIr/lmjHpJr1mnAgL6YEXge2zxr4gDw== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:53 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path ca [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6V11-000771-Ru Subject: [Openvpn-devel] [RFC ovpn net-next 5/9] ovpn: coalesce UDP data records with GRO X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411928300197903 X-GMAIL-MSGID: 1876411928300197903 Register UDP tunnel GRO callbacks for ovpn data sockets and coalesce compatible DATA_V2 records from one transport flow. Keep each encrypted record as a separate frag-list entry so the receive path can detach and authenticate records independently. Match the complete opcode/key/peer header and require compatible outer- network and checksum state. Flush on short records, differing segment geometry, existing GSO input, or the 64-record limit. Export skb_gro_receive_list, which is already shared by the core UDP and TCP frag-list GRO paths, so modular ovpn can use the same primitive instead of maintaining a local copy. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following throughput: Forward Reverse Without receive GRO 18.308 Gbit/s 19.233 Gbit/s With frag-list GRO 22.087 Gbit/s 22.962 Gbit/s The preceding UDP GSO transmit path and hardware UDP segmentation were enabled in both cases. The equal-weight mean of the two directional results increased from 18.770 to 22.524 Gbit/s, a 20.0% improvement. Signed-off-by: Ralf Lici --- drivers/net/ovpn/io.c | 7 +- drivers/net/ovpn/udp.c | 174 ++++++++++++++++++++++++++++++++++++++++- net/core/gro.c | 1 + net/ipv4/udp_offload.c | 3 +- 4 files changed, 178 insertions(+), 7 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 3ad4cadeeb02..11f7f16d7b79 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -70,11 +70,10 @@ static void ovpn_netdev_write(struct ovpn_peer *peer, struct sk_buff *skb) unsigned int pkt_len; int ret; - /* - * GSO state from the transport layer is not valid for the tunnel/data - * path. Reset all GSO fields to prevent any further GSO processing - * from entering an inconsistent state. + /* the transport encapsulation and its GSO metadata do not describe the + * decrypted inner packet */ + skb->encapsulation = 0; skb_gso_reset(skb); /* we can't guarantee the packet wasn't corrupted before entering the diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 4802d982de08..dfb1aa10556d 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -11,8 +11,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -27,6 +29,169 @@ #include "socket.h" #include "udp.h" +/* like UDP and TCP frag-list GRO */ +#define OVPN_UDP_GRO_CNT_MAX 64 + +static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) +{ + const unsigned int offset = skb_gro_offset(skb); + + /* GRO replaces its frag0 pointer after holding an skb, so keep the + * openvpn header linear for later candidate comparisons + */ + if (!pskb_may_pull(skb, offset + OVPN_OPCODE_SIZE)) + return false; + + *header = get_unaligned_be32(skb->data + offset); + return true; +} + +static struct sk_buff *ovpn_udp_gro_receive_fraglist(struct sock *sk, + struct list_head *head, + struct sk_buff *skb) +{ + const unsigned int gso_size = skb_gro_len(skb); + struct sk_buff *p, *pp = NULL; + u32 header, header2; + int ret = 0, nhoff; + bool flush; + + if (!ovpn_udp_gro_header(skb, &header) || + FIELD_GET(OVPN_OPCODE_PKTTYPE_MASK, header) != OVPN_DATA_V2) { + NAPI_GRO_CB(skb)->flush = 1; + return NULL; + } + + /* do not nest an existing GSO packet in the record list */ + if (skb_is_gso(skb)) { + NAPI_GRO_CB(skb)->flush = 1; + return NULL; + } + + list_for_each_entry(p, head, list) { + if (!NAPI_GRO_CB(p)->same_flow) + continue; + + /* match opcode, key ID and peer ID */ + if (!ovpn_udp_gro_header(p, &header2) || header != header2) { + NAPI_GRO_CB(p)->same_flow = 0; + continue; + } + + /* GRO has already matched the outer addresses and UDP ports; + * check the remaining outer IP fields + */ + nhoff = skb_transport_offset(p) - + NAPI_GRO_CB(p)->network_offset; + flush = __gro_receive_network_flush(udp_hdr(skb), udp_hdr(p), p, + nhoff, false); + + /* The first record determines the nominal GSO size. A shorter + * final record may follow it, but a larger record cannot. + * Checksum metadata must also be uniform because the aggregate + * exposes only one checksum state. + */ + if (gso_size > skb_shinfo(p)->gso_size || flush || + skb->ip_summed != p->ip_summed || + skb->csum_level != p->csum_level) { + pp = p; + } else { + /* skb_gro_receive_list pulls the headers already + * processed by GRO before linking this skb to the + * record list so we have to manually preserve the + * outer network header location for later handling + */ + nhoff = skb_gro_receive_network_offset(skb); + skb_set_network_header(skb, nhoff); + ret = skb_gro_receive_list(p, skb); + } + + /* complete the aggregate if the append failed, or after + * appending a shorter final record, or after reaching the + * record-count limit + */ + if (ret || gso_size != skb_shinfo(p)->gso_size || + NAPI_GRO_CB(p)->count >= OVPN_UDP_GRO_CNT_MAX) + pp = p; + + return pp; + } + + return NULL; +} + +static int ovpn_udp_gro_complete(struct sock *sk, struct sk_buff *skb, + int nhoff) +{ + /* udp_gro_complete has already marked this as a UDP tunnel GSO packet. + * Keep that type so UDP passes the aggregate directly to the encap cb, + * where the original record skbs are detached. + */ + skb_shinfo(skb)->gso_segs = NAPI_GRO_CB(skb)->count; + + /* Each outer UDP checksum was either validated (or accepted in case of + * checksumless UDP) before its record was merged in + * skb_gro_checksum_validate_zero_check. + * The checksum in the aggregate cannot describe the concatenation of + * independent UDP payloads, so we preserve the validation result. + */ + skb->ip_summed = CHECKSUM_UNNECESSARY; + skb->csum_level = 0; + skb->csum_valid = 0; + + return 0; +} + +/* skb_gro_receive_list keeps the first openvpn record in 'skb' and links the + * remaining records through frag_list. Here we segment by detaching that list + * before delivering the records individually, and remove the child skbs from + * the head skb's length and memory accounting so the head describes only the + * first record again. + */ +static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) +{ + struct sk_buff *curr, *list = skb_shinfo(skb)->frag_list; + unsigned int data_len = 0, truesize = 0; + + if (!list) + return NULL; + + for (curr = list; curr; curr = curr->next) { + data_len += curr->len; + truesize += curr->truesize; + } + + skb_shinfo(skb)->frag_list = NULL; + skb->len -= data_len; + skb->data_len -= data_len; + skb->truesize -= truesize; + + return list; +} + +static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) +{ + struct sk_buff *next; + + skb->next = ovpn_udp_gro_detach(skb); + + skb_list_walk_safe(skb, skb, next) + { + skb_mark_not_on_list(skb); + + /* keep the current reference alive for the next record before + * handing this one to crypto + */ + if (next && unlikely(!ovpn_peer_hold(peer))) { + DEBUG_NET_WARN_ON_ONCE(1); + kfree_skb_list(next); + next = NULL; + } + + ovpn_recv(peer, skb); + } +} + /* Retrieve the corresponding ovpn object from a UDP socket * rcu_read_lock must be held on entry */ @@ -121,8 +286,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) /* pop off outer UDP header */ __skb_pull(skb, sizeof(struct udphdr)); - skb_mark_not_on_list(skb); - ovpn_recv(peer, skb); + ovpn_udp_recv(peer, skb); return 0; drop: @@ -408,6 +572,8 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock, .encap_type = UDP_ENCAP_OVPNINUDP, .encap_rcv = ovpn_udp_encap_recv, .encap_destroy = ovpn_udp_encap_destroy, + .gro_receive = ovpn_udp_gro_receive_fraglist, + .gro_complete = ovpn_udp_gro_complete, }; struct ovpn_socket *old_data; int ret; @@ -454,6 +620,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) { struct sock *sk = ovpn_sock->sk; + udp_tunnel_cleanup_gro(sk); + /* Re-enable multicast loopback */ inet_set_bit(MC_LOOP, sk); /* Disable CHECKSUM_UNNECESSARY to CHECKSUM_COMPLETE conversion */ @@ -462,6 +630,8 @@ void ovpn_udp_socket_detach(struct ovpn_socket *ovpn_sock) WRITE_ONCE(udp_sk(sk)->encap_type, 0); WRITE_ONCE(udp_sk(sk)->encap_rcv, NULL); WRITE_ONCE(udp_sk(sk)->encap_destroy, NULL); + WRITE_ONCE(udp_sk(sk)->gro_receive, NULL); + WRITE_ONCE(udp_sk(sk)->gro_complete, NULL); rcu_assign_sk_user_data(sk, NULL); } diff --git a/net/core/gro.c b/net/core/gro.c index 29b4d02bf519..b6acedc919f8 100644 --- a/net/core/gro.c +++ b/net/core/gro.c @@ -262,6 +262,7 @@ int skb_gro_receive_list(struct sk_buff *p, struct sk_buff *skb) return 0; } +EXPORT_SYMBOL(skb_gro_receive_list); static void gro_complete(struct gro_node *gro, struct sk_buff *skb) { diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index cf07c3c6611a..187f108f3ee8 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -40,7 +40,8 @@ struct udp_tunnel_type_entry { #define UDP_MAX_TUNNEL_TYPES (IS_ENABLED(CONFIG_GENEVE) + \ IS_ENABLED(CONFIG_VXLAN) * 2 + \ IS_ENABLED(CONFIG_NET_FOU) * 2 + \ - IS_ENABLED(CONFIG_XFRM) * 2) + IS_ENABLED(CONFIG_XFRM) * 2 + \ + IS_ENABLED(CONFIG_OVPN)) DEFINE_STATIC_CALL(udp_tunnel_gro_rcv, dummy_gro_rcv); static DEFINE_STATIC_KEY_FALSE(udp_tunnel_static_call); From patchwork Tue Sep 15 15:23:54 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5344 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5063811mag; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBwYbq7whhY9N/GWthtucHpFyROLe98NZOt1sgoHJEz3lcKKvBo4bGCj45OMPx/oCKYf3bIPTejHuAA=@openvpn.net X-Received: by 2002:a05:6870:720c:b0:47c:1cfe:5cd0 with SMTP id 586e51a60fabf-481f5144002mr9694630fac.0.1789485863020; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485863; cv=none; d=google.com; s=arc-20260327; b=SgwxhqMZGDZiNPHQFIjAxkKri7sl0yxisKKipQB8hJKkJJzYZr1H4ylJLy41FWxYTC XR2ho31yz78197OT5+yrfSlE4RLtDAvnJv+Vs/xl0w42kebkvx5BccxD9AOy+lBgTF2x 81HO4Kiu8eq8TFwnJKxGJPdaFOI5uN6578zcYCWT4zbSVTsroYvWURE3vPp1QX+Ord1r rLANUCVqDhayDWuancStDGZ25dY9H5EGVEZ1SuTBKuCmVBDLT+LL4HfhvTCueufuYSYl dZb2uEiJp1bCZeWzL4c0Vh3UHOUrIPhX3rP6vAhFVrCB8m6PnLPcZ83oGSTTfsE3ZtDB wrzw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=jMHjxBkKR1Pn2eSVg8SkFx3NnTrEHwm+LfaSL3OEnV0=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=ZXP/kSe0gauTbcl5g11jDmo9JAIbE3cZyYVA+xVmaXsr19g0k+eBsteNFoO9VpkN1U W+Y2NhFBJpAh1yRWxSllfNm2A2nrOnq/tkoMswYchJN2qwNrlaQg9yKTI6OZTmiS8VDA E/3pJ+NpLiHXTL7TPH39KvxrsveTYHA5pvyS3oFDfFLgE5w+O8n9vWBWqdkh8a6o1dVG e6amH5jmcRXd8HSJ4GEeoYu6SeGBzviaTCLGsUkH0BVaK7OK2nBRuTMHCauN88DFWJlT Nu9/BwCfnDgdSepDNxK95dlwad0RHBkaXL0oyhwkqMPJTfPPTMsAR21dbtcMjUXmi1SV C7Hg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=HQtMVXdi; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=mdnnDM3o; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=crqgyZdA; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=iKExSTCK; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-47df89c0fcesi11296985fac.82.2026.09.15.08.24.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:22 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=HQtMVXdi; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=mdnnDM3o; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=crqgyZdA; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=iKExSTCK; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=jMHjxBkKR1Pn2eSVg8SkFx3NnTrEHwm+LfaSL3OEnV0=; b=HQtMVXdirUT/Z8x8YHbSeGpzAy Js7TF0YBGn9nhyjD/jBPWBPX3+mpmDgpQ+D+XdQiyWkKxEFyqXKdkIEMsbpr/DryowuGNv8gkcRvR CeFQD80169ko9/2bG2ygeWfzZ+qv8MqRyNX+n5lkdP6YCN0qiq459IevdK7XWInntjXw=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V16-0000JT-V8; Tue, 15 Sep 2026 15:24:17 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V13-0000Ik-Iz for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Kut5kWrtVSzjUahQOQL0CLcLzWSFwxNf6jCq+jiq2QE=; b=mdnnDM3o1CKAmFmsG1R+xr+Kb1 i3Psjs3j5rYF9jJWsfSoqQ9qEAKUgaVweAO5wc3WuEKUOq4AqraN+LFVTgf7pyH2I4znrwwNNEvjT ipO8yHdqkXYsKbM+RFpW9eO7sAf7TvM9SaDKyW86NQ29DUS1D/CRrAwtnNm9khSXJDZY=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=Kut5kWrtVSzjUahQOQL0CLcLzWSFwxNf6jCq+jiq2QE=; b=crqgyZdA500QJs04MaI9psZfSc LDtLSQiw/hkENY/3yxFjI0UHE//lVQ/AEKKceYWA4dyfSvnpA1ss70+PgW71QskumBl9+VE+FER/2 ooWHPnn5OLB9D8oyH/FOQpYvmWqB2rXpzt2vlRk/jfwX0AhXKjaJVhvKxXRI7iOavd0U=; Received: from mout-b-112.mailbox.org ([195.10.208.42]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V12-0003bS-9q for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:14 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-112.mailbox.org (Postfix) with ESMTPS id 4hkm5K04FGz5vT4 for ; Tue, 15 Sep 2026 17:24:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Kut5kWrtVSzjUahQOQL0CLcLzWSFwxNf6jCq+jiq2QE=; b=iKExSTCKHi0V3Zwtbr9u7YYhO38J8u3y0aa/x7zVqq+nWWn2lZGZ7Hu/PW9X6yNk2Gs513 jnWkBNxAd3g0I6STK/y9K4D8F+SdxgT51EuywFJyiVtkGjszmYnlTBMLmtMYQmXpBAvP9n BP4Lc+eKUAk8Qbh/P0zNERF4h0LKW1+dorZt3FbpMrE8NIz82DZ5e8uZKGyn70fUpIbohN HvsjgwduL0oFXB0iU41GVjwwNdYmH43MhyLGRDbkKkyIDdv/XeY2hUJNUkuVSX2XfYoRKP +kYRSB1ZIlf43Z3w+/uLIINkwnizv4iSg3nNkaAApATgatJMgu9fgMDQZOBXRQ== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:54 +0200 Message-ID: <712708baf265c5509aed4f9d476abf514a242b8a.1789485693.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hkm5K04FGz5vT4 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (a [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6V12-0003bS-9q Subject: [Openvpn-devel] [RFC ovpn net-next 6/9] ovpn: prefetch encrypted records before GRO batch decryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411928268645580 X-GMAIL-MSGID: 1876411928268645580 Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (and maintain the same distance throughout the batch), overlapping their memory access latency with the current AEAD operation. An ordinary single-record UDP receive has no later record and therefore skips the prefetch path. Only prefetch the linear part of each skb. Walking non-linear fragments here would duplicate the scatterlist walk performed by crypto and could cost more than the cache hint saves. A same-binary comparison using three 30-second samples per direction found distances one and two effectively tied forward, while distance two was 3.3% faster reverse and less variable in both directions. Profiling also measured slightly fewer decrypt cycles at distance two than at one, while wider distances provided no repeatable benefit. On a direct 100 Gbit/s ConnectX-5 link using one TCP stream, AES-128-GCM, a 1408-byte inner MTU and 8192-entry rings, five interleaved 60-second samples per direction increased throughput by 16.9% forward and 18.0% reverse. Signed-off-by: Ralf Lici --- drivers/net/ovpn/io.h | 14 ++++++++++++++ drivers/net/ovpn/udp.c | 21 ++++++++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index 1a94f0fda1d1..49180214fe08 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -10,6 +10,9 @@ #ifndef _NET_OVPN_OVPN_H_ #define _NET_OVPN_OVPN_H_ +#include +#include + /* DATA_V2 header size with AEAD encryption */ #define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ @@ -21,6 +24,17 @@ #define OVPN_KEEPALIVE_SIZE 16 extern const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE]; +static inline void ovpn_skb_prefetchw(const struct sk_buff *skb) +{ + unsigned int offset; + + /* crypto overwrites data in place, so request write ownership of each + * linear cache line before the AEAD implementation reaches it + */ + for (offset = 0; offset < skb_headlen(skb); offset += L1_CACHE_BYTES) + prefetchw(skb->data + offset); +} + netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev); void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb); diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index dfb1aa10556d..31fae42e2990 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -32,6 +32,9 @@ /* like UDP and TCP frag-list GRO */ #define OVPN_UDP_GRO_CNT_MAX 64 +/* leave enough work between a cache hint and the record which consumes it */ +#define OVPN_UDP_GRO_PREFETCH_DISTANCE 2 + static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) { const unsigned int offset = skb_gro_offset(skb); @@ -171,12 +174,28 @@ static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) { - struct sk_buff *next; + struct sk_buff *next, *prefetch; + unsigned int i; skb->next = ovpn_udp_gro_detach(skb); + /* a frag-list GRO aggregate makes later ciphertext visible before the + * current record is decrypted, so we prime the first two records, then + * keep the cache hints the same distance ahead while draining the list + */ + prefetch = skb->next ? skb : NULL; + for (i = 0; i < OVPN_UDP_GRO_PREFETCH_DISTANCE && prefetch; i++) { + ovpn_skb_prefetchw(prefetch); + prefetch = prefetch->next; + } + skb_list_walk_safe(skb, skb, next) { + if (prefetch) { + ovpn_skb_prefetchw(prefetch); + prefetch = prefetch->next; + } + skb_mark_not_on_list(skb); /* keep the current reference alive for the next record before From patchwork Tue Sep 15 15:23:55 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5338 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5063801mag; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBz3fbJXXf8L/Y/qyvZlhsRC3jlkS4NhK4OZU5DMMNu2tS2ZecSQqc79HNxiHNAvrorZvwqTWS44x2g=@openvpn.net X-Received: by 2002:a05:6808:17aa:b0:4b9:a88b:8891 with SMTP id 5614622812f47-4c7b6175989mr6713180b6e.39.1789485862947; Tue, 15 Sep 2026 08:24:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485862; cv=none; d=google.com; s=arc-20260327; b=GH+6DNA8ZwVh8/y2dhc57Kad536g7aPQjeENl1LzlBpaDkiKV8OiUR5IjDHGc4pax4 +n0uUmEKWQLZ2CGhs/FQ5hVMMIyjWmyrqWPcApYfN7cZi75q+HokJNL7vcu/ojCTx6o7 jDJhBHgqa+3PU/LU6dzqiIqAm25zDInG22jbxj1pn9b/E6TUJgWjUG3U9gh6gUUbnuQ4 KfQM1Vj/qlNGE1iZJsJNlV+kZT9ctMWJZwKrDf+iDCWe3GPE9OhWaISkHP4BGv2Pz1d9 EhBT9x3b5jLNDxucR6XmnAMSXCFX1cw8j1sWUEs6+pDiUAQlbJwUIueTzjmuns/+va8J FN5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=huMbEKmhqtxWOwaC28zEkqwUuiUBlRseYdcerYoOj6U=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=eYyZFdE+6PoEkpLh2IikPocQgmfauUhdcrWcEGE86qG9Ynqu1lM7mC6an/ZaLH3X3a d2WtH1tPqgGfBQAJGLiaaeNDpEcylVI7LFDdLco+ICZABP1mayKr5bv+pK0FV5ce5VtZ 36GtOJDjtpZc/Z6NnZDTXPYgk/3LtKCaQNOgcPX+EJPJSeVESktl0+4uspWVAIfRHxAl zuciKP/2f25N9Jo9KTgcPVwhCJLbA/ELGQddm+18Dm97A2qlRZ97tZBm7uKiEYlZw1MP v1ztFm3iXpO3SdOV2Ih1gzgrY0eZ+tngeitwp8iVfVZiAA3AHRvVFkS3wBWtQLXs22N/ DtRg==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="JtS/ZMfG"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Dx9c+qtq; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=PtqMxYbL; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=PJn69eNj; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5614622812f47-4c9b72d9e99si738329b6e.125.2026.09.15.08.24.21 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:22 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="JtS/ZMfG"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=Dx9c+qtq; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=PtqMxYbL; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=PJn69eNj; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=huMbEKmhqtxWOwaC28zEkqwUuiUBlRseYdcerYoOj6U=; b=JtS/ZMfG3Dnu856W6r7xztkwdw 4Oh92ZbHtWOTNIuzm0EN73ZMncHwwYw2vlMZJ8QHvO2X/HN/H5BjiQ8M9BU4/JcjJl4U1wPApuBtD BDxigU2rsYcEas96qtlHXJ4p978rNFR/FLA3aoibAFOyF4niMtdUOOK6jhkp3vr5J054=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V1B-0007I8-6j; Tue, 15 Sep 2026 15:24:18 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V15-0007Hp-Se for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=Sp4yS6pMwtHpQ+TUjkzeCY+3UBvWernpFq7/9GyYFko=; b=Dx9c+qtqdk3BnuU0hYlYh1nsnm g098jhTaf1HUPwUjrdogE/nsalW3PEXhjJ5G7Qv+3eO11Ncj7xKB6oEM/8lRnw9V0py71tjR6oNc2 X03q1NWsmYSkGBYBv3IlZnzsFEo84FY+am3zgrgqkUABszowznkGX0avGvHeAJ49mS9s=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=Sp4yS6pMwtHpQ+TUjkzeCY+3UBvWernpFq7/9GyYFko=; b=PtqMxYbLgLbBeNstPEUbsboRpf 8v3xgPJfp7MZLWYwaOEKu22vmU5GWwx1ftVvHoi/eCC+/KDgB2dTNV3wCPI8U3JPAQUIPEg948/eU H1SS8bg5BE4OSDvHwT+qcOsxaxcfgLbTzdjB8mGx5ZTTjdUdAEJIklK0vLxA/uRlEyhQ=; Received: from mout-b-210.mailbox.org ([195.10.208.40]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V12-000773-SN for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:13 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-210.mailbox.org (Postfix) with ESMTPS id 4hkm5K3xYjzFqxF for ; Tue, 15 Sep 2026 17:24:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Sp4yS6pMwtHpQ+TUjkzeCY+3UBvWernpFq7/9GyYFko=; b=PJn69eNjbXtjlm8XWiz2Xx78DKUf3gOAxpPuBU87WbXpgnhV7D4rt+0NZaDpWdYnQFWLaH FSqa5EG9+yteKhqakENCbtnxKxsi8j3QCeuvfP/1wwoM4ms6gtIdw8XnbFiMtBsjQw+WyV NZht3SDDKJNrZ151kakHwmUV0I0dzC4uTSuXEQLGYHH8lWn/cVgD4pZbQ0LC8IIq49a0bU bDZRb5r2hsFgqCzw7ZzhRKaYb++lRys4BKlsRssdlvolRljDqyE77YB0RjEE7AREgyQSfC t9cfp0OUIVJR+BQcX7CwzfgEn8EfuZPi048w4FYPK3rvcru1k+kjw5QWVw/TCg== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:55 +0200 Message-ID: <382879c187b1acca65198c467d1263266243ea50.1789485693.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Software segmentation exposes the complete skb list before encryption begins. While encrypting each segment, request write ownership of the next linear segment cache line by cache line. A single skb n [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1x6V12-000773-SN Subject: [Openvpn-devel] [RFC ovpn net-next 7/9] ovpn: prefetch GSO segments before in-place encryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411927811963335 X-GMAIL-MSGID: 1876411927811963335 Software segmentation exposes the complete skb list before encryption begins. While encrypting each segment, request write ownership of the next linear segment cache line by cache line. A single skb naturally skips the prefetch path. Signed-off-by: Ralf Lici --- drivers/net/ovpn/io.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 11f7f16d7b79..cb7503a3e79c 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -472,6 +472,12 @@ static void ovpn_send(struct ovpn_priv *ovpn, struct sk_buff *skb, * independently */ skb_list_walk_safe(skb, curr, next) { + /* encrypting this segment can hide the cost of fetching the + * next segment's data into the cache + */ + if (next) + ovpn_skb_prefetchw(next); + if (unlikely(!ovpn_encrypt_one(peer, curr))) { ovpn_dev_dstats_tx_dropped(ovpn->dev, 1); kfree_skb(curr); From patchwork Tue Sep 15 15:23:56 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5340 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5063805mag; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBzWUlcf6pkXGg8lsBqjE9x0YgPOUfCr7X4M0ni59Br3yRSJznRveUgjldjns4WyTGRWoefylqPkH90=@openvpn.net X-Received: by 2002:a05:6830:71a5:b0:7f4:effa:a5ad with SMTP id 46e09a7af769-8089915d6acmr9218758a34.10.1789485862992; Tue, 15 Sep 2026 08:24:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485862; cv=none; d=google.com; s=arc-20260327; b=Ooptatqk0FYqsgCgtA/am867Mnf730zrDGW7h0H0efzBkPw5nHpapiS/MDhj0lTgkR mlQCjCtjg2jfCJ7JcXUbC+Z0voel7TaUD8bi3xfuaJKw2TsReOK//BJT1xix5y8GxFLU 35fsGtdy5lVUzuM8ZfA41Y1A+pOc7Rf4kcKGJ3+7tHuXURkfR25Wwy4s7CpZUDv4iVp6 d4a3waWBFEutwwV0ryj7MYbscLqA51LF/r2h9AEMHH0D3zEQR0vYdw2H5VAtgtrPH5cS EXmGhGbvaWbyPr/bFTg5tP7ZU1Id1HLo2uQS7lKn4o3BbLp8UNmnUwdLq2lfVVXlctru NG8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=s/Z301HEECxMvvZpiHXRdyorFo3O4axPkQM9w8v+0Uk=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=KT0wEivN1FQXRzWRaUlxk4SIBhUTMlhIa70euF3r6zRlhoGT65THFs7rPRqkshf/ep wT9buGtNavFmRFgR3kxI1YCxBVodx7UUyXSispfuyJX6dyu33QZCH5/EDI1RTrJXlOBw aeCzyFWh6elpjbNvM5YVTZINMymT4GvAYVOvMoSaH6eaS25OodMPeWWOLcioSsO9zUXY UKZt00UgWKJi9bUAd5PsgrEKS0itii1lqwpPLHSt3jsGNtPtzFzlmmCIZqCLFh118z27 fomOHgqKYrAsUOTw5tEbteOM6aOTGNlNelaPbgCRpnLnB0W8AG7dCKFL+YjQompXTbi3 ym4Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="EZ836/2x"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=E03EzAc6; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Osowae0N; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=P9cA14D0; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-803f724d082si14178401a34.69.2026.09.15.08.24.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:22 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b="EZ836/2x"; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=E03EzAc6; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Osowae0N; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=P9cA14D0; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=s/Z301HEECxMvvZpiHXRdyorFo3O4axPkQM9w8v+0Uk=; b=EZ836/2xMKRTMnCNq+v5OWTLG9 siQ35JsYVvLdOjg5U1Vzb/FsMA6o2LzpF+/IsfbaaHXA+Nr8Yfvhw4DhVZWchJgiKh2LF0hwrXXwl t3bzJ0MWVsCMi3ghl6CHz59Av8NkI5jO1qJpabq4SW080ZtQ5sgalMwP0IfJyf0jWAhA=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V17-0000Jd-E7; Tue, 15 Sep 2026 15:24:18 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V14-0000J2-HG for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:15 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=DC0eq//5IawQh4BTy4jFvKFEUqIze8wPibWReb452ew=; b=E03EzAc633uNTDaDBp5ZnnJK0p bAp/IqiEMFHS+hbRj9zAZB+Dw/NuH2PDyC9DXFsamDY93qZ4iIVxJ843fPGWBGDO8v9y5Hddz/naU fTqDiu9G87DEeXhhy+46dwZH1cwsLi9LmyZcyCVzG7L4FOrkO/fEXwLijgrIEt1HXQW8=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=DC0eq//5IawQh4BTy4jFvKFEUqIze8wPibWReb452ew=; b=Osowae0NJWHjsWrWX0jpNOcU7V 6BgsUY4PI4zo0NadkhjDj9n7u3fJ1/v+PCkQ3vsjTT/cSCOXwwIqFZ8FkuYADoYZwI7xv5lr6f5Pq eam64/02HaPTMeE/JPQ1gtBNkH7G129kvhfFQM3WmtoCoX+en2yNX6VJ/H5DCHFrv6C4=; Received: from mout-b-106.mailbox.org ([195.10.208.46]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V13-000774-P0 for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:15 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [10.196.197.102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-106.mailbox.org (Postfix) with ESMTPS id 4hkm5L0Fs5zNlKx for ; Tue, 15 Sep 2026 17:24:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DC0eq//5IawQh4BTy4jFvKFEUqIze8wPibWReb452ew=; b=P9cA14D0nxZGRbMTenDMzkKlcaFn2aLidAvo+LLGlm1hgpv7bAilcBnxyHNjfhVVo3ohkc ckLDa8QJ8r5joqlN2JlLKM/1YJSN9hqwVkkjeP4Q8ZU+7XwsB+t4B0j+obs3hSoczktdpz xEnVdNqbJO8+k5YdOlPPJY6ogcmg3TzLrHyM0P4uF5kleuRjWcLk4vajTZd/GcR5pM/uC9 KXtNGpAfQ4kYIgdrImerLSrzsH83EhT36VzLEUiERJtfjokuvrvhJWPYSQVQnFEsttaCJ+ mEBh9GH1JU1z1h/6QOzS7ltulwFdwv/hyZcbQAQrGPaHqCQ8OmbbFG1GIfvm+g== From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:56 +0200 Message-ID: <893f6c2b385f9df3e9617a9b7f547734061df195.1789485693.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: XFRM's ESP GRO callbacks may consume an skb and return ERR_PTR(-EINPROGRESS) as an ownership marker. dev_gro_receive already recognizes this marker unconditionally and converts it to GRO_CONSUMED, so [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6V13-000774-P0 Subject: [Openvpn-devel] [RFC ovpn net-next 8/9] net: gro: honor skbs consumed by protocol callbacks X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411928713150993 X-GMAIL-MSGID: 1876411928713150993 XFRM's ESP GRO callbacks may consume an skb and return ERR_PTR(-EINPROGRESS) as an ownership marker. dev_gro_receive already recognizes this marker unconditionally and converts it to GRO_CONSUMED, so -EINPROGRESS is reserved by the generic GRO callback interface and cannot represent an ordinary callback error. The nested flush helpers currently avoid accessing a consumed skb only when XFRM offload is configured, because XFRM has so far been the sole user of the convention. Make the ownership check unconditional so other protocol callbacks can safely use the existing marker without acquiring an unrelated CONFIG_XFRM_OFFLOAD dependency. Callbacks which do not return the marker are unaffected. Signed-off-by: Ralf Lici --- include/net/gro.h | 19 +++---------------- 1 file changed, 3 insertions(+), 16 deletions(-) diff --git a/include/net/gro.h b/include/net/gro.h index 2300b6da05b2..20ddc5488789 100644 --- a/include/net/gro.h +++ b/include/net/gro.h @@ -361,9 +361,11 @@ static inline void skb_gro_remcsum_cleanup(struct sk_buff *skb, remcsum_unadjust((__sum16 *)ptr, grc->delta); } -#ifdef CONFIG_XFRM_OFFLOAD static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush) { + /* a GRO callback may consume skb and return this marker to prevent + * accessing the skb while unwinding through the enclosing GRO layers + */ if (PTR_ERR(pp) != -EINPROGRESS) NAPI_GRO_CB(skb)->flush |= flush; } @@ -378,21 +380,6 @@ static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb, skb->remcsum_offload = 0; } } -#else -static inline void skb_gro_flush_final(struct sk_buff *skb, struct sk_buff *pp, int flush) -{ - NAPI_GRO_CB(skb)->flush |= flush; -} -static inline void skb_gro_flush_final_remcsum(struct sk_buff *skb, - struct sk_buff *pp, - int flush, - struct gro_remcsum *grc) -{ - NAPI_GRO_CB(skb)->flush |= flush; - skb_gro_remcsum_cleanup(skb, grc); - skb->remcsum_offload = 0; -} -#endif INDIRECT_CALLABLE_DECLARE(struct sk_buff *ipv6_gro_receive(struct list_head *, struct sk_buff *)); From patchwork Tue Sep 15 15:23:57 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5343 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp5063812mag; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBxXdVAOWvmtvQqk8Jtig++VJU2FNUi7/xgkyy+6SDk+WQ9uIlejgce2aER2Azv/yCzbSPLX7ggNJLQ=@openvpn.net X-Received: by 2002:a05:6871:56c1:b0:475:e0a7:9f32 with SMTP id 586e51a60fabf-481fa13a18amr5604552fac.32.1789485863017; Tue, 15 Sep 2026 08:24:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789485863; cv=none; d=google.com; s=arc-20260327; b=O3Xb8hzLL/P+xc0GHPSiODy638jQhwlRREEmL8pYdGQfKvFpQ357AJnWBK8U+AmE9v Of/ws4Iqc9nDgzIEU9TseGaxDZmLLyBl4ChSZRxEvltpRjxj1YEtZQ6xR2ns4uPnRfBf i63l7z866J80CFajSF71VfV++rh/F1qsvwNfg10B0lvUAjcg+J8Jc9BFWzjr7Lax//4z 6Q3TPSeOQvwy4Xo6IQN3XwmQf2ax58DPRYpgBppxybclsH/+ir2CWzfDXYNz5D5Q9IFh Jt0iajBkfj3MxTgt44om6jBxmYfn826B8M5uWuQ9lsdDKrlJlsuC/mzLUs9IMGQlzK5q 84IA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=v6aWBrzJ52AGAuTfA2r2j08YybpQC2j3SN7QDwQqh7k=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=fVLw64UMVB51PTtFVQ52u2Wdqhr1VBBnbWn5g5NYOOaUFry9EBNFPgSUmCmFkbrHN7 4qC01V+/21myy6cJXAbcZcTA8cv9jY+jFAUfjc+qJ635Gwn16LPWG8wXITYAWWi7K96S a1dwS3dgqhmULgJ4wQdIvxOVqPhsnjmDa+bbZFzpMzqim25KR2z8DoLbSRTSKNCs/K9N ZoHnr/mQcXw3ewe8MC6uj+9k4XC+vgo6XybT9cb0OF19eqVhCUZhxyR6j08odg2RFB6l glY41EfnHq8l22qZB9FaGRx4V9y0nMDKC5F0nzyOZTMCeoTo4LXMzHFjMQSjSWtdktWi C1nw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=YXO72CAJ; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="kltN/cHu"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=c7IQnZh5; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=IzfLu1ct; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-47dfa19e62dsi12631238fac.374.2026.09.15.08.24.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2026 08:24:22 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=YXO72CAJ; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b="kltN/cHu"; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=c7IQnZh5; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=IzfLu1ct; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=v6aWBrzJ52AGAuTfA2r2j08YybpQC2j3SN7QDwQqh7k=; b=YXO72CAJRLgQ858VQ/Eg5TenGZ K+SHdY58xaOMv2F9CDFupG8XYFW2N13zMKLTrHA8a52FZ8tdScsrZhhXmXFVrUDb4ez2/AILP9d96 qLayboAa11dce7fuLDvdmTdN7r2cZ2B/y+9Gpw2jZUiwWGiH5yN6WpskmOWJXiN+HoN0=; Received: from [127.0.0.1] (helo=sfs-ml-3.v29.lw.sourceforge.com) by sfs-ml-3.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x6V18-0007oV-Q5; Tue, 15 Sep 2026 15:24:19 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-3.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x6V17-0007oK-R9 for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:18 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ILSydsmfFTUwgW0MtyTe7SzjMzCFniSsnnFiJAkIgcw=; b=kltN/cHu5TDzzp9LbMlwsd5V05 yzgIKDYRTpjuqRVpHo3JvrtYN4hGDQCD8iIGKXo7fBEuPAx7Wkj6q174VlNIPh6D6mtU2p1Wou1Kn 8Wv9LtJFAYZ1K6BdPw84WtywSW0/jcGP3uPUkyLYH1cel8io7L/lVObVo34IBPAkhq3k=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ILSydsmfFTUwgW0MtyTe7SzjMzCFniSsnnFiJAkIgcw=; b=c7IQnZh5qcM6yke7VQlMJ8OPUr 1t/5ipH0DeD2gE0ZjSUwusIcGubEJbscA25vn1lRa4HkuIdP++LmD9iULr82/SKC79Vy42InaKoJE yf2+x+FvZ03KiTHeFG8mIEw3i8KfgxRS0fHPjRUPy/0P1+iF3IT+MQua0hgI45unxF+8=; Received: from mout-b-107.mailbox.org ([195.10.208.47]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x6V14-000776-A7 for openvpn-devel@lists.sourceforge.net; Tue, 15 Sep 2026 15:24:18 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange x25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-107.mailbox.org (Postfix) with ESMTPS id 4hkm5L3jgcz3y2W for ; Tue, 15 Sep 2026 17:24:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789485846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ILSydsmfFTUwgW0MtyTe7SzjMzCFniSsnnFiJAkIgcw=; b=IzfLu1ct8uNqGRZDcddo5SGX3usoI9JojsAY0bCOQnyrW8KeycoFHwCqqK2dGARlk/pUP+ bCjJ0X9IBWh1X5erewzvBTXARf0f8MEI/+/dj0lDN2DT9cwFHj5VqL0ijLejr9o9Ih6XYs 7i2F9caGNtWDiDIL98h9PXp6cUQZqtNdLlV9OoMdOVC8wO8JIwcK+MpDl5HR+GfczFnFAT j6dZNppNvx51/UcZLQj5d58Hbt40KrN95qO0ApbNCG5nRwTc4/WmH7JyQ9rm+X9EmjtLQM 1OpQhIjgfNmxWtGZ02MvKBGvVke52Eeq23WSMv0iQkdBh0ZqDVRCGsYJGp9Geg== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Tue, 15 Sep 2026 17:23:57 +0200 Message-ID: In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hkm5L3jgcz3y2W X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: The normal ovpn UDP GRO path retains compatible encrypted records in a frag-list and lets the completed aggregate traverse the outer IP and UDP receive stack before handing its records to the existing [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1x6V14-000776-A7 Subject: [Openvpn-devel] [RFC ovpn net-next 9/9] ovpn: add opt-in direct GRO receive mode X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876411928233272612 X-GMAIL-MSGID: 1876411928233272612 The normal ovpn UDP GRO path retains compatible encrypted records in a frag-list and lets the completed aggregate traverse the outer IP and UDP receive stack before handing its records to the existing decryption path. Add an optional UDP GRO mode which instead consumes DATA_V2 packets from the UDP tunnel GRO callback and starts their existing per-record decryption immediately, similarly to xfrm. Control packets are flushed from GRO, restored, and continue through the normal stack. Select the mode through the immutable IFLA_OVPN_UDP_GRO_MODE link attribute. FULL_STACK remains the default for existing userspace, while DIRECT enables the new path. Keep this local receive policy per ovpn interface and select the corresponding GRO callback when each UDP socket is attached. This makes all sockets attached to the interface behave consistently without performing a mode lookup and dispatch for every packet. Account for both ovpn callbacks in the UDP tunnel GRO callback limit. The direct mode deliberately bypasses packet taps, TC ingress, outer IP validation and routing, netfilter hooks, the final UDP lookup, socket XFRM policy, and normal UDP receive accounting for DATA_V2. It is therefore an explicit operator choice for controlled transport interfaces rather than a transparent replacement for the full receive stack. Extend the UDP throughput selftest with a PRE_ROUTING nftables counter, verifying that full-stack aggregates reach the hook while direct-GRO aggregates bypass it. Before ciphertext prefetch was added to FULL_STACK, five interleaved single-flow AES-128-GCM runs per direction on two directly connected 100-Gbit/s mlx5 ports measured 22.087/22.962 Gbit/s forward/reverse in FULL_STACK and 24.315/25.313 Gbit/s in DIRECT, a 10.2% equal-weight gain. With the preceding prefetch commit enabled in FULL_STACK, later checks measured DIRECT within 1.3% forward and 0.7% reverse of FULL_STACK. Signed-off-by: Ralf Lici --- Documentation/netlink/specs/rt-link.yaml | 12 +++ drivers/net/ovpn/main.c | 15 ++- drivers/net/ovpn/ovpnpriv.h | 2 + drivers/net/ovpn/udp.c | 98 ++++++++++++++----- include/uapi/linux/if_link.h | 6 ++ net/ipv4/udp_offload.c | 2 +- tools/testing/selftests/net/ovpn/Makefile | 1 + tools/testing/selftests/net/ovpn/common.sh | 4 +- tools/testing/selftests/net/ovpn/ovpn-cli.c | 38 ++++++- .../selftests/net/ovpn/test-gro-direct.sh | 10 ++ tools/testing/selftests/net/ovpn/test.sh | 65 ++++++++++++ 11 files changed, 219 insertions(+), 34 deletions(-) create mode 100755 tools/testing/selftests/net/ovpn/test-gro-direct.sh diff --git a/Documentation/netlink/specs/rt-link.yaml b/Documentation/netlink/specs/rt-link.yaml index 7a72cd1b7e1e..d43e995c7f09 100644 --- a/Documentation/netlink/specs/rt-link.yaml +++ b/Documentation/netlink/specs/rt-link.yaml @@ -844,6 +844,14 @@ definitions: entries: - p2p - mp + - + name: ovpn-udp-gro-mode + enum-name: ovpn-udp-gro-mode + name-prefix: ovpn-udp-gro-mode + type: enum + entries: + - full-stack + - direct - name: br-stp-mode type: enum @@ -2365,6 +2373,10 @@ attribute-sets: name: mode type: u8 enum: ovpn-mode + - + name: udp-gro-mode + type: u8 + enum: ovpn-udp-gro-mode sub-messages: - diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index ac4e0d85e215..ecf27d1e2420 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -129,6 +129,9 @@ static const struct device_type ovpn_type = { static const struct nla_policy ovpn_policy[IFLA_OVPN_MAX + 1] = { [IFLA_OVPN_MODE] = NLA_POLICY_RANGE(NLA_U8, OVPN_MODE_P2P, OVPN_MODE_MP), + [IFLA_OVPN_UDP_GRO_MODE] = + NLA_POLICY_RANGE(NLA_U8, OVPN_UDP_GRO_MODE_FULL_STACK, + OVPN_UDP_GRO_MODE_DIRECT), }; /** @@ -200,6 +203,7 @@ static int ovpn_newlink(struct net_device *dev, struct rtnl_newlink_params *params, struct netlink_ext_ack *extack) { + enum ovpn_udp_gro_mode gro_mode = OVPN_UDP_GRO_MODE_FULL_STACK; struct ovpn_priv *ovpn = netdev_priv(dev); struct nlattr **data = params->data; enum ovpn_mode mode = OVPN_MODE_P2P; @@ -209,9 +213,14 @@ static int ovpn_newlink(struct net_device *dev, mode = nla_get_u8(data[IFLA_OVPN_MODE]); netdev_dbg(dev, "setting device mode: %u\n", mode); } + if (data && data[IFLA_OVPN_UDP_GRO_MODE]) { + gro_mode = nla_get_u8(data[IFLA_OVPN_UDP_GRO_MODE]); + netdev_dbg(dev, "setting UDP GRO mode: %u\n", gro_mode); + } ovpn->dev = dev; ovpn->mode = mode; + ovpn->gro_mode = gro_mode; spin_lock_init(&ovpn->lock); INIT_DELAYED_WORK(&ovpn->keepalive_work, ovpn_peer_keepalive_work); @@ -237,8 +246,8 @@ static int ovpn_newlink(struct net_device *dev, static size_t ovpn_get_size(const struct net_device *dev) { - /* IFLA_OVPN_MODE */ - return nla_total_size(sizeof(u8)); + /* IFLA_OVPN_MODE and IFLA_OVPN_UDP_GRO_MODE */ + return nla_total_size(sizeof(u8)) + nla_total_size(sizeof(u8)); } static int ovpn_fill_info(struct sk_buff *skb, const struct net_device *dev) @@ -247,6 +256,8 @@ static int ovpn_fill_info(struct sk_buff *skb, const struct net_device *dev) if (nla_put_u8(skb, IFLA_OVPN_MODE, ovpn->mode)) return -EMSGSIZE; + if (nla_put_u8(skb, IFLA_OVPN_UDP_GRO_MODE, ovpn->gro_mode)) + return -EMSGSIZE; return 0; } diff --git a/drivers/net/ovpn/ovpnpriv.h b/drivers/net/ovpn/ovpnpriv.h index 84499140e4bd..dc6210b99f4a 100644 --- a/drivers/net/ovpn/ovpnpriv.h +++ b/drivers/net/ovpn/ovpnpriv.h @@ -40,6 +40,7 @@ struct ovpn_peer_collection { * struct ovpn_priv - per ovpn interface state * @dev: the actual netdev representing the tunnel * @mode: device operation mode (i.e. p2p, mp, ..) + * @gro_mode: whether UDP data follows the full stack or is decrypted from GRO * @lock: protect this object * @peers: data structures holding multi-peer references * @peer: in P2P mode, this is the only remote peer @@ -49,6 +50,7 @@ struct ovpn_peer_collection { struct ovpn_priv { struct net_device *dev; enum ovpn_mode mode; + enum ovpn_udp_gro_mode gro_mode; spinlock_t lock; /* protect writing to the ovpn_priv object */ struct ovpn_peer_collection *peers; struct ovpn_peer __rcu *peer; diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 31fae42e2990..ae14e7e4802f 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -232,23 +232,12 @@ static struct ovpn_socket *ovpn_socket_from_udp_sock(struct sock *sk) return ovpn_sock; } -/** - * ovpn_udp_encap_recv - Start processing a received UDP packet. - * @sk: socket over which the packet was received - * @skb: the received packet - * - * If the first byte of the payload is: - * - DATA_V2 the packet is accepted for further processing, - * - DATA_V1 the packet is dropped as not supported, - * - anything else the packet is forwarded to the UDP stack for - * delivery to user space. - * - * Return: - * 0 if skb was consumed or dropped - * >0 if skb should be passed up to userspace as UDP (packet not consumed) - * <0 if skb should be resubmitted as proto -N (packet not consumed) +/* Process one packet after the caller has made its OpenVPN header visible at + * @payload_offset. A zero return means the skb was consumed. A positive return + * leaves a control packet for the UDP socket. */ -static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) +static int ovpn_udp_data_recv(struct sock *sk, struct sk_buff *skb, + unsigned int payload_offset) { struct ovpn_socket *ovpn_sock; struct ovpn_priv *ovpn; @@ -269,18 +258,16 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) goto drop_noovpn; } - /* Make sure the first 4 bytes of the skb data buffer after the UDP - * header are accessible. + /* Make sure the first 4 bytes of the OpenVPN header are accessible. * They are required to fetch the OP code, the key ID and the peer ID. */ - if (unlikely(!pskb_may_pull(skb, sizeof(struct udphdr) + - OVPN_OPCODE_SIZE))) { + if (unlikely(!pskb_may_pull(skb, payload_offset + OVPN_OPCODE_SIZE))) { net_dbg_ratelimited("%s: packet too small from UDP socket\n", netdev_name(ovpn->dev)); goto drop; } - opcode = ovpn_opcode_from_skb(skb, sizeof(struct udphdr)); + opcode = ovpn_opcode_from_skb(skb, payload_offset); if (unlikely(opcode != OVPN_DATA_V2)) { /* DATA_V1 is not supported */ if (opcode == OVPN_DATA_V1) @@ -290,7 +277,7 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) return 1; } - peer_id = ovpn_peer_id_from_skb(skb, sizeof(struct udphdr)); + peer_id = ovpn_peer_id_from_skb(skb, payload_offset); /* some OpenVPN server implementations send data packets with the * peer-id set to UNDEF. In this case we skip the peer lookup by peer-id * and we try with the transport address @@ -303,8 +290,10 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) if (unlikely(!peer)) goto drop; - /* pop off outer UDP header */ - __skb_pull(skb, sizeof(struct udphdr)); + /* the crypto receive path expects skb->data to begin at the OpenVPN + * header and takes ownership of the skb + */ + __skb_pull(skb, payload_offset); ovpn_udp_recv(peer, skb); return 0; @@ -315,6 +304,60 @@ static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) return 0; } +/* Consume DATA_V2 directly from UDP GRO. These packets deliberately bypass + * packet taps, TC ingress, the outer IP and netfilter receive paths, the final + * UDP lookup, and normal UDP accounting. Control packets are restored and + * continue through all of those layers normally. + */ +static struct sk_buff *ovpn_udp_gro_receive_direct(struct sock *sk, + struct list_head *head, + struct sk_buff *skb) +{ + unsigned int offset = skb_gro_offset(skb); + + /* if the OpenVPN header is not accessible, leave validation and drop + * handling to the ordinary UDP receive path + */ + if (unlikely(!pskb_pull(skb, offset))) + goto flush; + + /* tell UDP GRO not to touch the skb if it was consumed by the direct + * receive path + */ + if (likely(!ovpn_udp_data_recv(sk, skb, 0))) + return ERR_PTR(-EINPROGRESS); + + /* control packets still belongs to the socket so we restore the data + * pointer because the normal receive path expects the outer headers + */ + skb_push(skb, offset); + +flush: + NAPI_GRO_CB(skb)->same_flow = 0; + NAPI_GRO_CB(skb)->flush = 1; + return NULL; +} + +/** + * ovpn_udp_encap_recv - Start processing a received UDP packet. + * @sk: socket over which the packet was received + * @skb: the received packet + * + * If the first byte of the payload is: + * - DATA_V2 the packet is accepted for further processing, + * - DATA_V1 the packet is dropped as not supported, + * - anything else the packet is forwarded to the UDP stack for + * delivery to user space. + * + * Return: + * 0 if @skb was consumed or dropped + * 1 if @skb should continue through normal UDP delivery + */ +static int ovpn_udp_encap_recv(struct sock *sk, struct sk_buff *skb) +{ + return ovpn_udp_data_recv(sk, skb, sizeof(struct udphdr)); +} + /** * ovpn_udp4_output - send IPv4 packet over udp socket * @peer: the destination peer @@ -591,7 +634,12 @@ int ovpn_udp_socket_attach(struct ovpn_socket *ovpn_sock, struct socket *sock, .encap_type = UDP_ENCAP_OVPNINUDP, .encap_rcv = ovpn_udp_encap_recv, .encap_destroy = ovpn_udp_encap_destroy, - .gro_receive = ovpn_udp_gro_receive_fraglist, + /* GRO mode cannot change after the interface is created so + * select the socket callback once at socket setup + */ + .gro_receive = ovpn->gro_mode == OVPN_UDP_GRO_MODE_DIRECT ? + ovpn_udp_gro_receive_direct : + ovpn_udp_gro_receive_fraglist, .gro_complete = ovpn_udp_gro_complete, }; struct ovpn_socket *old_data; diff --git a/include/uapi/linux/if_link.h b/include/uapi/linux/if_link.h index 245b36204525..2d7b320f83be 100644 --- a/include/uapi/linux/if_link.h +++ b/include/uapi/linux/if_link.h @@ -2067,9 +2067,15 @@ enum ovpn_mode { OVPN_MODE_MP, }; +enum ovpn_udp_gro_mode { + OVPN_UDP_GRO_MODE_FULL_STACK, + OVPN_UDP_GRO_MODE_DIRECT, +}; + enum { IFLA_OVPN_UNSPEC, IFLA_OVPN_MODE, + IFLA_OVPN_UDP_GRO_MODE, __IFLA_OVPN_MAX, }; diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index 187f108f3ee8..bfe23ec9dfca 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -41,7 +41,7 @@ struct udp_tunnel_type_entry { IS_ENABLED(CONFIG_VXLAN) * 2 + \ IS_ENABLED(CONFIG_NET_FOU) * 2 + \ IS_ENABLED(CONFIG_XFRM) * 2 + \ - IS_ENABLED(CONFIG_OVPN)) + IS_ENABLED(CONFIG_OVPN) * 2) DEFINE_STATIC_CALL(udp_tunnel_gro_rcv, dummy_gro_rcv); static DEFINE_STATIC_KEY_FALSE(udp_tunnel_static_call); diff --git a/tools/testing/selftests/net/ovpn/Makefile b/tools/testing/selftests/net/ovpn/Makefile index 169f0464ac3a..412a70abf739 100644 --- a/tools/testing/selftests/net/ovpn/Makefile +++ b/tools/testing/selftests/net/ovpn/Makefile @@ -37,6 +37,7 @@ TEST_PROGS := \ test-close-socket-tcp.sh \ test-close-socket.sh \ test-float.sh \ + test-gro-direct.sh \ test-large-mtu.sh \ test-mark.sh \ test-symmetric-id-float.sh \ diff --git a/tools/testing/selftests/net/ovpn/common.sh b/tools/testing/selftests/net/ovpn/common.sh index 2d844eb3aa6e..1467caa95168 100644 --- a/tools/testing/selftests/net/ovpn/common.sh +++ b/tools/testing/selftests/net/ovpn/common.sh @@ -10,6 +10,7 @@ source "$OVPN_COMMON_DIR/../../kselftest/ktap_helpers.sh" OVPN_UDP_PEERS_FILE=${OVPN_UDP_PEERS_FILE:-udp_peers.txt} OVPN_TCP_PEERS_FILE=${OVPN_TCP_PEERS_FILE:-tcp_peers.txt} OVPN_CLI=${OVPN_CLI:-${OVPN_COMMON_DIR}/ovpn-cli} +OVPN_UDP_GRO_MODE=${OVPN_UDP_GRO_MODE:-FULL_STACK} OVPN_YNL=${OVPN_YNL:-${OVPN_COMMON_DIR}/../../../../net/ynl/pyynl/cli.py} OVPN_ALG=${OVPN_ALG:-aes} OVPN_PROTO=${OVPN_PROTO:-UDP} @@ -162,7 +163,8 @@ ovpn_setup_ns() { done fi - ip netns exec "${peer}" ${OVPN_CLI} new_iface tun${1} $MODE + ip netns exec "${peer}" ${OVPN_CLI} new_iface tun${1} $MODE \ + "${OVPN_UDP_GRO_MODE}" ip -n "${peer}" addr add ${2} dev tun${1} # add a secondary IP to peer 1, to test a LAN behind a client if [ ${1} -eq 1 -a -n "${OVPN_LAN_IP}" ]; then diff --git a/tools/testing/selftests/net/ovpn/ovpn-cli.c b/tools/testing/selftests/net/ovpn/ovpn-cli.c index f4effa7580c0..8e5f9c0986eb 100644 --- a/tools/testing/selftests/net/ovpn/ovpn-cli.c +++ b/tools/testing/selftests/net/ovpn/ovpn-cli.c @@ -123,6 +123,8 @@ struct ovpn_ctx { char ifname[IFNAMSIZ]; enum ovpn_mode mode; bool mode_set; + enum ovpn_udp_gro_mode udp_gro_mode; + bool udp_gro_mode_set; int socket; int cli_sockets[MAX_PEERS]; @@ -1377,8 +1379,9 @@ static int ovpn_new_iface(struct ovpn_ctx *ovpn) struct ovpn_link_req req = { 0 }; int ret = -1; - fprintf(stdout, "Creating interface %s with mode %u\n", ovpn->ifname, - ovpn->mode); + fprintf(stdout, + "Creating interface %s with mode %u and UDP GRO mode %u\n", + ovpn->ifname, ovpn->mode, ovpn->udp_gro_mode); req.n.nlmsg_len = NLMSG_LENGTH(sizeof(req.i)); req.n.nlmsg_flags = NLM_F_REQUEST | NLM_F_CREATE | NLM_F_EXCL; @@ -1396,15 +1399,21 @@ static int ovpn_new_iface(struct ovpn_ctx *ovpn) strlen(OVPN_FAMILY_NAME) + 1) < 0) goto err; - if (ovpn->mode_set) { + if (ovpn->mode_set || ovpn->udp_gro_mode_set) { data = ovpn_nest_start(&req.n, sizeof(req), IFLA_INFO_DATA); if (!data) goto err; - if (ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_MODE, + if (ovpn->mode_set && + ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_MODE, &ovpn->mode, sizeof(uint8_t)) < 0) goto err; + if (ovpn->udp_gro_mode_set && + ovpn_addattr(&req.n, sizeof(req), IFLA_OVPN_UDP_GRO_MODE, + &ovpn->udp_gro_mode, sizeof(uint8_t)) < 0) + goto err; + ovpn_nest_end(&req.n, data); } @@ -1666,11 +1675,16 @@ static void usage(const char *cmd) cmd); fprintf(stderr, "where can be one of the following\n\n"); - fprintf(stderr, "* new_iface [mode]: create new ovpn interface\n"); + fprintf(stderr, + "* new_iface [mode] [udp-gro-mode]: create new ovpn interface\n"); fprintf(stderr, "\tiface: ovpn interface name\n"); fprintf(stderr, "\tmode:\n"); fprintf(stderr, "\t\t- P2P for peer-to-peer mode (i.e. client)\n"); fprintf(stderr, "\t\t- MP for multi-peer mode (i.e. server)\n"); + fprintf(stderr, "\tudp-gro-mode:\n"); + fprintf(stderr, "\t\t- FULL_STACK for the normal receive stack\n"); + fprintf(stderr, + "\t\t- DIRECT to decrypt data from the UDP GRO callback\n"); fprintf(stderr, "* del_iface : delete ovpn interface\n"); fprintf(stderr, "\tiface: ovpn interface name\n"); @@ -2206,6 +2220,20 @@ static int ovpn_parse_cmd_args(struct ovpn_ctx *ovpn, int argc, char *argv[]) return -1; } ovpn->mode_set = true; + + if (argc < 5) + break; + + if (!strcmp(argv[4], "FULL_STACK")) { + ovpn->udp_gro_mode = OVPN_UDP_GRO_MODE_FULL_STACK; + } else if (!strcmp(argv[4], "DIRECT")) { + ovpn->udp_gro_mode = OVPN_UDP_GRO_MODE_DIRECT; + } else { + fprintf(stderr, "Cannot parse UDP GRO mode: %s\n", + argv[4]); + return -1; + } + ovpn->udp_gro_mode_set = true; break; case CMD_DEL_IFACE: break; diff --git a/tools/testing/selftests/net/ovpn/test-gro-direct.sh b/tools/testing/selftests/net/ovpn/test-gro-direct.sh new file mode 100755 index 000000000000..f35db23eb425 --- /dev/null +++ b/tools/testing/selftests/net/ovpn/test-gro-direct.sh @@ -0,0 +1,10 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2020-2025 OpenVPN, Inc. +# +# Author: Ralf Lici +# Antonio Quartulli + +OVPN_UDP_GRO_MODE="DIRECT" + +source test.sh diff --git a/tools/testing/selftests/net/ovpn/test.sh b/tools/testing/selftests/net/ovpn/test.sh index 9b5610837032..55cb4d4c3e3d 100755 --- a/tools/testing/selftests/net/ovpn/test.sh +++ b/tools/testing/selftests/net/ovpn/test.sh @@ -113,6 +113,67 @@ ovpn_run_lan_traffic() { ip netns exec ovpn_peer0 ping -qfc 100 -w 3 "${OVPN_LAN_IP}" } +ovpn_udp_gro_counter_add() { + [ "${OVPN_PROTO}" == "UDP" ] || return 0 + # A custom tunnel MTU can fragment outer packets before UDP GRO. + [ -z "${MTU:-}" ] || return 0 + + # Enable UDP forwarding GRO on the receiving endpoint so this test + # exercises the configured ovpn callback. + ovpn_cmd_ok "enable UDP GRO on the iperf receive path" \ + ip netns exec ovpn_peer0 ethtool -K veth1 gro on \ + rx-udp-gro-forwarding on + + ovpn_cmd_ok "create UDP GRO path counter table" \ + ip netns exec ovpn_peer0 nft add table inet ovpn_gro_test + ovpn_cmd_ok "create UDP GRO path counter chain" \ + ip netns exec ovpn_peer0 nft \ + "add chain inet ovpn_gro_test prerouting { type filter hook \ + prerouting priority filter; policy accept; }" + + # Count only aggregated outer packets after they enter the normal + # receive stack in peer0. Direct GRO consumes those DATA_V2 aggregates + # before this hook, while small packets which bypass veth's GRO path + # are deliberately ignored. + ovpn_cmd_ok "add UDP GRO path counter" \ + ip netns exec ovpn_peer0 nft add rule inet ovpn_gro_test \ + prerouting iifname "veth1" meta length gt 1500 udp dport 1 \ + counter +} + +ovpn_udp_gro_counter_check() { + local packets + + [ "${OVPN_PROTO}" == "UDP" ] || return 0 + [ -z "${MTU:-}" ] || return 0 + + packets=$(ip netns exec ovpn_peer0 nft list chain inet ovpn_gro_test \ + prerouting | sed -n \ + 's/.*counter packets \([0-9][0-9]*\) bytes.*/\1/p') + ovpn_cmd_ok "remove UDP GRO path counter table" \ + ip netns exec ovpn_peer0 nft delete table inet ovpn_gro_test + + if [ -z "${packets}" ]; then + printf '%s\n' "unable to read UDP GRO path counter" + return 1 + fi + + if [ "${OVPN_UDP_GRO_MODE}" == "FULL_STACK" ]; then + if [ "${packets}" -eq 0 ]; then + printf '%s\n' \ + "full-stack UDP GRO did not reach PRE_ROUTING" + return 1 + fi + return 0 + fi + + if [ "${packets}" -ne 0 ]; then + printf '%s\n' \ + "direct UDP GRO reached PRE_ROUTING ${packets} times" + return 1 + fi +} + ovpn_run_float_mode() { local p local peer_ns @@ -134,12 +195,16 @@ ovpn_run_float_mode() { ovpn_run_iperf() { local iperf_pid + ovpn_udp_gro_counter_add + ovpn_run_bg iperf_pid ip netns exec ovpn_peer0 iperf3 -1 -s sleep 1 ovpn_cmd_ok "run iperf throughput flow" \ ip netns exec ovpn_peer1 iperf3 -Z -t 3 -c 5.5.5.1 wait "${iperf_pid}" || return 1 + + ovpn_udp_gro_counter_check } ovpn_run_key_rollover() {