From patchwork Thu Sep 17 15:29:56 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ralf Lici X-Patchwork-Id: 5379 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:6446:b0:8a0:ea1f:253a with SMTP id n6csp7554753mag; Thu, 17 Sep 2026 08:30:31 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBxkJPOffjRsXTnD+bjRLEq1943QW2UCG8EJy1uOQgyILWljMMqmMGnwsBO1mPJ7CF1g6iw64/qc7k8=@openvpn.net X-Received: by 2002:a4a:e841:0:b0:6b7:83d6:292d with SMTP id 006d021491bc7-6c7d3fe09b2mr6932882eaf.48.1789659031434; Thu, 17 Sep 2026 08:30:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789659031; cv=none; d=google.com; s=arc-20260327; b=cJZ1TrebJmqlUvzb2XDpykTJxAmBU2PxLHMS4EHg/PLoRW8x1UD0oIAXuNKGQTzuco iwy5mRxeFH/T4m8OQNNIqkjNkizb2rlJ2duR4o9mP5Kg8ubXkuRKlCSMYMdMAGHZ0Xv2 dlLgCTuw4gTRU+4BAqj8MBkAt5nR27eJAC9ISLwZYwb/UVAE42ZBBWuEQyPIVcxkd827 +rAFbq6xPA+ei7FDjWdJVsN0Mhye/f2xVp8NpYzgnjQkLiKEfq03UpPxU2OKa2gCYmpE SQj/pES7Otef2Y0IE7vSOfo5yBoiNoy9It541jFQ1M6n2jh2MbxtPW6yu5Gcr3QDL8Ju 9Mgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=wJimdI5h6dPYH8fNvbnM9flCIEpiRu/LLTfU+WDhyI0=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=H79HBu2ptZnpM4Cu5+AA51EJl+2GZDOj/Iexnv1hlIr5v8ln/eU/5sH4b4ttbAQkdS dwOE35VXym4gwgleT+w5+RGF3RtwIyIKHwWWVKHZtLkLOPuf5FIzz9uD5dIAipf+Lxy9 vlvefrmeVj25uchq5Rv0O4TPcoiMQ72cgx5bkAK1iZvufJnIeXnElnNwjPO35USuxkci 4vUYqhMoxjV8rx7IYAHXT68ycH6v7YUrzX3y8YwUBhiQgWVh6mRFKHhOIhFVlophy5f+ faNyq3h79Mg4LKaqBHpOnyuJrjFz6Jk7nf1NKB7LGInrqqZWbdBPT0+LdtpwrMIlHfF/ AW1Q==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=IU+JzFx3; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=RpA8KqDk; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=RBWe+Hc0; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=mKHAtGxe; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-486ab04fad8si247646fac.90.2026.09.17.08.30.31 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 17 Sep 2026 08:30:31 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=IU+JzFx3; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=RpA8KqDk; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=RBWe+Hc0; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=mKHAtGxe; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=wJimdI5h6dPYH8fNvbnM9flCIEpiRu/LLTfU+WDhyI0=; b=IU+JzFx3zGnkM4iVXshf96jFqJ 0fePhT2CU9QlvQZpjswvuAVi6fkFxEx6TFPgYKNiPQPaRSLmULiOyZY2M3aDqBr62KtG0AJ8a0PS+ 8JEgO7o26sVFevQ3rr/RHQS+g48OUjCGPOL9DRLfAT/Fh1sf4rDEltzldBsj2PGNZS30=; Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1x7E48-00015L-S2; Thu, 17 Sep 2026 15:30:24 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1x7E45-00015D-0G for openvpn-devel@lists.sourceforge.net; Thu, 17 Sep 2026 15:30:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=hBdxZjtHts8nK9raX5QZTeLbt7Tda72d2ZPfgf4p59w=; b=RpA8KqDkejbWhyHiQ4RO2i3+/U vb10QTemctZiajCq+sYZTLnMQCN+nD0uwVlm35YixsOlcq0TYfeJ+ZdkEytJ+qs03ygUC82F7koe2 jnJlSGthsvIa8lbde367LaKL7BdphXA/Wc0yHsMB7+ox5J1IuYvKfliD9E6h4XtMTO2Q=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=hBdxZjtHts8nK9raX5QZTeLbt7Tda72d2ZPfgf4p59w=; b=RBWe+Hc0GOHRtLzI2glYkQ7Tbc 00fqQ/QowX67NsbJUXNgIms2hvdslp21sJTjR9PxNDKKMeE3LHkhDgbt4OBitQKuz2m3sYzVwZW6x 9vka+Gz7h2LTTeShb2ce0lyOcrpBSmWkJdM7XDYyTZ2yeIjviPxvdtALP9KlINSIeOao=; Received: from mout-b-107.mailbox.org ([195.10.208.47]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1x7E43-0005Ym-Gr for openvpn-devel@lists.sourceforge.net; Thu, 17 Sep 2026 15:30:20 +0000 Received: from smtp202.mailbox.org (smtp202.mailbox.org [IPv6:2001:67c:2050:b231:465::202]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-107.mailbox.org (Postfix) with ESMTPS id 4hm07R4sj2z3yQs for ; Thu, 17 Sep 2026 17:30:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1789659011; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hBdxZjtHts8nK9raX5QZTeLbt7Tda72d2ZPfgf4p59w=; b=mKHAtGxedEPJXTHqbF9e8+SkFgdj9MJdfhWZbXpd8ZWcNRip9+Afx7nuLgX1el12eX0A8V JvYcs4v3phwpgtW7Z7Osea/EPvVtP8hE7GRRirZFMKLEx/fJoXeRib2yyG7pcQuuBMLAOW RMHDfqSmqr6oNf9LIvIK1UC+jRAbgSzAIH+T14PUtTHMstb1M8zU01W9nqVExKCEus/AjJ utDBvLwJe9XOErcHwk1OMJtDNoRKfwF54HcZqJnpjjJQ3k1XcpnUBq7KIpYmEpUEtsxzxj uRXC2PweqfXFiko60sOaXjYWSZjYY8rPxGgyhjhC7s4/yl0f9yvkAeIR/Y6ioQ== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of ralf@mandelbit.com designates 2001:67c:2050:b231:465::202 as permitted sender) smtp.mailfrom=ralf@mandelbit.com From: Ralf Lici To: openvpn-devel@lists.sourceforge.net Date: Thu, 17 Sep 2026 17:29:56 +0200 Message-ID: <9551c9c842e4ac926089badc81f445becca513e5.1789658051.git.ralf@mandelbit.com> In-Reply-To: References: MIME-Version: 1.0 X-Rspamd-Queue-Id: 4hm07R4sj2z3yQs X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Packet processing holds a key slot until asynchronous crypto completion. Parallel traffic using one key consequently updates the same kref cache line for every packet, even though key slots are normal [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1x7E43-0005Ym-Gr Subject: [Openvpn-devel] [PATCH ovpn net-next 2/2] ovpn: use percpu references for key slots X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1876593508333388155 X-GMAIL-MSGID: 1876593508333388155 Packet processing holds a key slot until asynchronous crypto completion. Parallel traffic using one key consequently updates the same kref cache line for every packet, even though key slots are normally long-lived. Replace the key-slot kref with percpu_ref. Kill the initial reference after atomically unpublishing a slot, while ordinary packet completions only put their live references. The release callback retains the existing rcu_work teardown so lockless readers receive an RCU grace period and crypto transforms are freed from workqueue context. Release the percpu_ref storage from that final worker. Initialize the reference only after the rest of the key slot is ready, allowing the existing destruction path to handle allocation failure. After the peer conversion, perf c2c still identified the key-slot kref cacheline, with 15 sampled HITM loads attributed to its locked reference update. This change removed that line from the shared-cacheline profile as well. In a set interleaved 32-flow iperf3 runs, the combined peer and key conversion moved throughput to 8.812 Gbit/s, against 7.734 Gbit/s for the baseline, a +13.95% improvement. Single-stream control found no reproducible regression. Signed-off-by: Ralf Lici --- drivers/net/ovpn/crypto.c | 14 +++++++------- drivers/net/ovpn/crypto.h | 15 +++++++++++---- drivers/net/ovpn/crypto_aead.c | 7 ++++++- 3 files changed, 24 insertions(+), 12 deletions(-) diff --git a/drivers/net/ovpn/crypto.c b/drivers/net/ovpn/crypto.c index 7e545428900a..23502aa125db 100644 --- a/drivers/net/ovpn/crypto.c +++ b/drivers/net/ovpn/crypto.c @@ -18,11 +18,11 @@ #include "crypto_aead.h" #include "crypto.h" -void ovpn_crypto_key_slot_release(struct kref *kref) +void ovpn_crypto_key_slot_release(struct percpu_ref *ref) { struct ovpn_crypto_key_slot *ks; - ks = container_of(kref, struct ovpn_crypto_key_slot, refcount); + ks = container_of(ref, struct ovpn_crypto_key_slot, refcount); queue_rcu_work(ovpn_wq, &ks->free_work); } @@ -36,13 +36,13 @@ void ovpn_crypto_state_release(struct ovpn_crypto_state *cs) ks = rcu_access_pointer(cs->slots[0]); if (ks) { RCU_INIT_POINTER(cs->slots[0], NULL); - ovpn_crypto_key_slot_put(ks); + ovpn_crypto_key_slot_kill(ks); } ks = rcu_access_pointer(cs->slots[1]); if (ks) { RCU_INIT_POINTER(cs->slots[1], NULL); - ovpn_crypto_key_slot_put(ks); + ovpn_crypto_key_slot_kill(ks); } } @@ -66,7 +66,7 @@ bool ovpn_crypto_kill_key(struct ovpn_crypto_state *cs, u8 key_id) spin_unlock_bh(&cs->lock); if (ks) - ovpn_crypto_key_slot_put(ks); + ovpn_crypto_key_slot_kill(ks); /* let the caller know if a key was actually killed */ return ks; @@ -104,7 +104,7 @@ int ovpn_crypto_state_reset(struct ovpn_crypto_state *cs, spin_unlock_bh(&cs->lock); if (old) - ovpn_crypto_key_slot_put(old); + ovpn_crypto_key_slot_kill(old); return 0; } @@ -141,7 +141,7 @@ void ovpn_crypto_key_slot_delete(struct ovpn_crypto_state *cs, } pr_debug("deleting key slot %u, key_id=%u\n", slot, ks->key_id); - ovpn_crypto_key_slot_put(ks); + ovpn_crypto_key_slot_kill(ks); } void ovpn_crypto_key_slots_swap(struct ovpn_crypto_state *cs) diff --git a/drivers/net/ovpn/crypto.h b/drivers/net/ovpn/crypto.h index e3feb16d5498..bd4056570d54 100644 --- a/drivers/net/ovpn/crypto.h +++ b/drivers/net/ovpn/crypto.h @@ -10,6 +10,7 @@ #ifndef _NET_OVPN_OVPNCRYPTO_H_ #define _NET_OVPN_OVPNCRYPTO_H_ +#include #include #include "pktid.h" @@ -48,7 +49,7 @@ struct ovpn_crypto_key_slot { struct ovpn_pktid_recv pid_recv ____cacheline_aligned_in_smp; struct ovpn_pktid_xmit pid_xmit ____cacheline_aligned_in_smp; struct rcu_work free_work; - struct kref refcount; + struct percpu_ref refcount; }; struct ovpn_crypto_state { @@ -61,7 +62,7 @@ struct ovpn_crypto_state { static inline bool ovpn_crypto_key_slot_hold(struct ovpn_crypto_key_slot *ks) { - return kref_get_unless_zero(&ks->refcount); + return percpu_ref_tryget_live_rcu(&ks->refcount); } static inline void ovpn_crypto_state_init(struct ovpn_crypto_state *cs) @@ -121,11 +122,17 @@ ovpn_crypto_key_slot_primary(const struct ovpn_crypto_state *cs) return ks; } -void ovpn_crypto_key_slot_release(struct kref *kref); +void ovpn_crypto_key_slot_release(struct percpu_ref *ref); static inline void ovpn_crypto_key_slot_put(struct ovpn_crypto_key_slot *ks) { - kref_put(&ks->refcount, ovpn_crypto_key_slot_release); + percpu_ref_put(&ks->refcount); +} + +static inline void +ovpn_crypto_key_slot_kill(struct ovpn_crypto_key_slot *ks) +{ + percpu_ref_kill(&ks->refcount); } int ovpn_crypto_state_reset(struct ovpn_crypto_state *cs, diff --git a/drivers/net/ovpn/crypto_aead.c b/drivers/net/ovpn/crypto_aead.c index 74eaf6fac2f5..414e5d7d32bf 100644 --- a/drivers/net/ovpn/crypto_aead.c +++ b/drivers/net/ovpn/crypto_aead.c @@ -393,6 +393,7 @@ static void ovpn_aead_crypto_key_slot_free_work(struct work_struct *work) ks = container_of(to_rcu_work(work), struct ovpn_crypto_key_slot, free_work); + percpu_ref_exit(&ks->refcount); ovpn_aead_crypto_key_slot_free(ks); kfree(ks); } @@ -428,7 +429,6 @@ ovpn_aead_crypto_key_slot_new(const struct ovpn_key_config *kc) ks->encrypt = NULL; ks->decrypt = NULL; INIT_RCU_WORK(&ks->free_work, ovpn_aead_crypto_key_slot_free_work); - kref_init(&ks->refcount); ks->key_id = kc->key_id; ks->encrypt = ovpn_aead_init("encrypt", alg_name, @@ -458,6 +458,11 @@ ovpn_aead_crypto_key_slot_new(const struct ovpn_key_config *kc) ovpn_pktid_xmit_init(&ks->pid_xmit); ovpn_pktid_recv_init(&ks->pid_recv); + ret = percpu_ref_init(&ks->refcount, ovpn_crypto_key_slot_release, 0, + GFP_KERNEL); + if (ret < 0) + goto destroy_ks; + return ks; destroy_ks: