From patchwork Mon Nov 2 16:43:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5563 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:6022:3119:b0:118:2077:975e with SMTP id e25csp5673487laa; Mon, 2 Nov 2026 08:44:26 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBzfN/I1sT8XXfX+fLrZ8Ck1Tgx+5ge5tJniKTOT1kj6UHB6/P9zYY3/yZkkEo7bES5fOaffsooHzq0=@openvpn.net X-Received: by 2002:a05:7301:1828:b0:36a:10d:afab with SMTP id 5a478bee46e88-36a42423d10mr17412960eec.25.1793637866377; Mon, 02 Nov 2026 08:44:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793637866; cv=none; d=google.com; s=arc-20260327; b=NIOce6GxJue9lG6jY0HwYTWmLmUEsJFNFeMTxdla3P61OD6GJj2Php86oAmg4P7S1Y Pg5mR20NSlJ3b2egB8wN9P98NU/SzPJVzuSe6R+bcH263X5I4g7Dyefb2TaV74Ryk8i4 SisfIJQaHaDjWlE6jCLhAT9KofhtEWStH7S1dRkD1n3ByzyaWavEtEgddHFurOq05kKZ 09lsRW6DgSDqhRrtGnE0wHNDrXKNSjn65rHX8flgcQtZFDgJm1pNCR/pcxKERfZNDWol Bda5NP+tWMD8kQkKCwaDAkIszcW26oDavgdn1LfCn1kK5ohcwMMWF6chg3wiXvek1VtN bDhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=Wz+qMSZjyMdih9zBPtaT5zkbvc5Ybol83woLmGM9pN0=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=XXFcHLaznYLKotxdUjsSv0g5zRcZGHvIGlKc6Hk7TVBdwCZL4sPybry2Bkk3ZzwmYI dREjpIS+h9hpGvA6MzexLSc8Ob9PYxV9mlW+mA1U/koy4HCaktWGUy1fUl2TNs6fMqX/ 8npcQSuYhdBHVZ1puHgJIuEKFPxTFS7UOhvOh9iR/p5E/vNKnAC8GqGTAE3v++XwTnOv beZ5vRdrLbud++vaFdd9EcXuNo7cSX9B8BNh1KF5frBbUyfbIqCahntUjyJLJfGkrPkD AxY4Zj6bDKh2kIHeRfk5RznLLdQNRqEqFkcuqLwthad15vtSFr9AGS968Nl0vL6TxCck ZyXw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=cKQ1PR0f; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=iRU3gbiR; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Bp6HiPdp; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=CdU4T3U7; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5a478bee46e88-36a42b1cc20si34359427eec.56.2026.11.02.08.44.26 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Nov 2026 08:44:26 -0800 (PST) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=cKQ1PR0f; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=iRU3gbiR; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=Bp6HiPdp; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=CdU4T3U7; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Unsubscribe-Post: List-Owner; bh=Wz+qMSZjyMdih9zBPtaT5zkbvc5Ybol83woLmGM9pN0=; b=cKQ1PR0fui/7+d m9Qr3xhvygRhARycEvWmfaggpAL4Fv/HS+Eskeb1QCXlt2m6LALpSUHbBjCp+FO72/RaE4DsDn3qb lI+C6/ZvW/zw2Uya3YN9eJokzJMoFhQY+1KapxU5HvahxcTcEq0UmvyoAqA0p6wHHSwoIhylgJSEl N8Q=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.100.1) (envelope-from ) id 1xNv8z-000000000bz-17R8; Mon, 02 Nov 2026 16:44:25 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.100.1) (envelope-from ) id 1xNv8k-000000000a6-1IgD for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:10 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Unsubscribe-Post:List-Subscribe:List-Post:List-Owner:List-Archive; bh=ernflziDPbi1ZnQJFgUjxF6sBdiHnzY9PIymRhaJdkE=; b=iRU3gbiRjmAyqz6kArbBYYSj2I mfz9BvvO4XLVhKDddI3g1HQeQd0SqNic7SgXzrfKWJwgFklaGk4HZmC7Sn0ZEk9g19MhvkxhG0DaK hS0+09FdgMDlVD8N47ieTCz7elUuYp9n5Z+scVQ9eZk2DM1jkr7CKs6FWCTYpup1Ffvc=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Unsubscribe-Post: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ernflziDPbi1ZnQJFgUjxF6sBdiHnzY9PIymRhaJdkE=; b=Bp6HiPdpM/Shp5Eyrv7czq9JZ6 Q+3TVhgeV2PWsA/VT2sis7sdqJKJTQ2CWYLp6gzQlKUYj7Gf9R51zes+uBH2adFA2IcNcqtFzy/cm 1hoLN6+XEPuAKKfM1pHdtufJmmQIWJAFz7Kb7TVM59JTgC1PXFydWTrT/z7beL88BPBM=; Received: from mout-b-206.mailbox.org ([195.10.208.51]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.100.1) id 1xNv8j-000000005X5-26Q8 for openvpn-devel@lists.sourceforge.net; Mon, 02 Nov 2026 16:44:10 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-206.mailbox.org (Postfix) with ESMTPS id 4jJHbN3CLXzXw for ; Mon, 02 Nov 2026 17:44:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1793637840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ernflziDPbi1ZnQJFgUjxF6sBdiHnzY9PIymRhaJdkE=; b=CdU4T3U7oAFTrxNmPtRdB5O62Mu3mcKN8ohyJ6VaBdUxdSaWsdSlbtu9WLbtbLUSR45aPe azDUDSW31RAcODVE5OIR+bf4NCYmx1zXpRt9VDH4GuPS4KflAuWAeKE7nJ8w7nIin3YYxM CcsMJAevdK+PKovyUFEAU8h4c2UKncV5xWVtJa+goyH60qn3eKZ4XeCcNiRCXQJvnFB4rF CgQR6kPGTahPOqaVGf3CazaRW05ECRmdtYF3XlZZHaATrK/cF9qFFraTH8xvUwP5Hj6+4y QMV+Z7i6gEX2luqsclwh0tWzZUjVc681QSkTPuLolrUtntfxP8Dta0yTF3b4Yg== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Mon, 2 Nov 2026 17:43:49 +0100 Message-ID: <20261102164353.459969-8-ralf@mandelbit.com> X-Mailer: git-send-email 2.56.0 In-Reply-To: <20261102164353.459969-1-ralf@mandelbit.com> References: <20261102164353.459969-1-ralf@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4jJHbN3CLXzXw X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (a [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain X-Headers-End: 1xNv8j-000000005X5-26Q8 Subject: [Openvpn-devel] [PATCH ovpn net-next v2 7/9] ovpn: prefetch encrypted records before GRO batch decryption X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1880765619111526878 X-GMAIL-MSGID: 1880765619111526878 Frag-list GRO exposes later encrypted records before the receive path decrypts the current one. Use this lookahead to request write ownership of linear ciphertext cache lines two records in advance (and maintain the same distance throughout the batch), overlapping their memory access latency with the current AEAD operation. An ordinary single-record UDP receive has no later record and therefore skips the prefetch path. Only prefetch the linear part of each skb. Walking non-linear fragments here would duplicate the scatterlist walk performed by crypto and could cost more than the cache hint saves. This optimization primarily targets synchronous software AEAD, where the CPU which issues the hint subsequently overwrites the prefetched lines. An asynchronous accelerator may instead access them through DMA and is not expected to benefit in the same way; the cache hint remains correctness-neutral. A same-binary comparison using three 30-second samples per direction found distances one and two effectively tied forward, while distance two was 3.3% faster reverse and less variable in both directions. Profiling also measured slightly fewer decrypt cycles at distance two than at one, while wider distances provided no repeatable benefit. On two directly connected 100-Gbit/s mlx5 ports, five interleaved iperf3 -t 60 -O 10 single-flow AES-128-GCM runs in each direction produced the following mean throughput: Forward Reverse Without RX prefetch 22.773 Gbit/s 22.796 Gbit/s With RX prefetch 24.028 Gbit/s 23.405 Gbit/s RX prefetch improved throughput by 5.5% forward and 2.7% reverse. Signed-off-by: Ralf Lici --- Changes since v1 https://lore.kernel.org/openvpn-devel/0651258e11d4a0c290152a1f286b9d8a45c5acd4.1790843153.git.ralf@mandelbit.com/ - Document that RX prefetch primarily targets synchronous software AEAD; asynchronous transforms are not expected to benefit in the same way. drivers/net/ovpn/io.h | 15 +++++++++++++++ drivers/net/ovpn/udp.c | 21 ++++++++++++++++++++- 2 files changed, 35 insertions(+), 1 deletion(-) diff --git a/drivers/net/ovpn/io.h b/drivers/net/ovpn/io.h index 1a94f0fda1d1..10c568c02234 100644 --- a/drivers/net/ovpn/io.h +++ b/drivers/net/ovpn/io.h @@ -10,6 +10,9 @@ #ifndef _NET_OVPN_OVPN_H_ #define _NET_OVPN_OVPN_H_ +#include +#include + /* DATA_V2 header size with AEAD encryption */ #define OVPN_HEAD_ROOM (OVPN_DATA_V2_OVERHEAD + \ max(sizeof(struct udphdr), sizeof(struct tcphdr)) +\ @@ -21,6 +24,18 @@ #define OVPN_KEEPALIVE_SIZE 16 extern const unsigned char ovpn_keepalive_message[OVPN_KEEPALIVE_SIZE]; +static inline void ovpn_skb_prefetchw_linear(const struct sk_buff *skb) +{ + unsigned long addr = (unsigned long)skb->data; + unsigned long end = addr + skb_headlen(skb); + + /* crypto overwrites data in place, so request write ownership of each + * linear cache line before the AEAD implementation reaches it + */ + for (; addr < end; addr = ALIGN(addr + 1, L1_CACHE_BYTES)) + prefetchw((void *)addr); +} + netdev_tx_t ovpn_net_xmit(struct sk_buff *skb, struct net_device *dev); void ovpn_recv(struct ovpn_peer *peer, struct sk_buff *skb); diff --git a/drivers/net/ovpn/udp.c b/drivers/net/ovpn/udp.c index 0c3557d90a1e..e0aeb09d0e2d 100644 --- a/drivers/net/ovpn/udp.c +++ b/drivers/net/ovpn/udp.c @@ -32,6 +32,9 @@ /* like UDP and TCP frag-list GRO */ #define OVPN_UDP_GRO_CNT_MAX 64 +/* leave enough work between a cache hint and the record which consumes it */ +#define OVPN_UDP_GRO_PREFETCH_DISTANCE 2 + static bool ovpn_udp_gro_header(struct sk_buff *skb, u32 *header) { const unsigned int offset = skb_gro_offset(skb); @@ -189,7 +192,8 @@ static struct sk_buff *ovpn_udp_gro_detach(struct sk_buff *skb) static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) { - struct sk_buff *list, *next; + struct sk_buff *list, *next, *prefetch; + unsigned int i; list = ovpn_udp_gro_detach(skb); if (IS_ERR(list)) { @@ -200,8 +204,23 @@ static void ovpn_udp_recv(struct ovpn_peer *peer, struct sk_buff *skb) } skb->next = list; + /* a fraglist GRO aggregate makes later ciphertext visible before the + * current record is decrypted, so we prime the first two records, then + * keep the cache hints the same distance ahead while draining the list + */ + prefetch = skb->next ? skb : NULL; + for (i = 0; i < OVPN_UDP_GRO_PREFETCH_DISTANCE && prefetch; i++) { + ovpn_skb_prefetchw_linear(prefetch); + prefetch = prefetch->next; + } + skb_list_walk_safe(skb, skb, next) { + if (prefetch) { + ovpn_skb_prefetchw_linear(prefetch); + prefetch = prefetch->next; + } + /* skb_unclone can leave fraglist children shared with a packet * tap, so make each skb header private before changing its * list pointer or control block. skb_cow_data makes the packet