From patchwork Fri Oct 9 15:27:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marco Baffo X-Patchwork-Id: 5454 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:32d1:b0:8d1:cccb:4552 with SMTP id y17csp3255444mad; Fri, 9 Oct 2026 08:28:12 -0700 (PDT) X-Forwarded-Encrypted: i=2; AKwUvBy3NctNt5zXPFTadrLuWlhcQ6eix2dPOotLPWf4qVU4cA8fDPZHf8XQQTPUqtYccBTtmEHksjnibTQ=@openvpn.net X-Received: by 2002:a05:6820:310c:b0:6d8:d121:a970 with SMTP id 006d021491bc7-6ef11f1c4c7mr1476429eaf.76.1791559692335; Fri, 09 Oct 2026 08:28:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1791559692; cv=none; d=google.com; s=arc-20260327; b=V2wzXn/Dv6sXVsMBMyswcQmTlTtreBvz+5J838IRUnHVZlX+ip8eB2t4V4MCOn2/7r tzVw5bcaC06Nws3ciJbs5uxyvZdGfQZ7QLTKiOE5uyU3cEL/pC8FAZRtx1p6wzvhNdhT dGYIBSjRiekdw29hChl1nQjgTfMKKiZgf51CPweyztJLHmrCcT7aZYirh/SQlnlkA9KZ TKgWk6Y700Ka5wG3N5q8uEuHWBckANBf8osRL6CWCm531HrSwyyotaucQixE/wdeEsKj hhiBY10+uoAYCe7zqJxSgZkg+fZ8wxglYCdP3xQJfsc0DrEnzXJsqxyzovv39RlTLavM DIQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=lZDXe5knLt/f3yoAHQJDqTTJQuYEIKoJ5dzJ1WF78xs=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=MnWUPeShQF1leeW1hsIX8qkhF06pgtHbbPIXd1Pn7IzjmPVawsfvuNA/99Si1ofAZS yQ/GFyuVD7qHZniVDiK9g8MHqsI+NK4wG7EYDwicGQN+rFoAOmD6zoOBD2XsMzBJDdDP IDX/S7beenxnAtcyTWQBL33lrIutfzqtcp6DigorTqnkdyQT5MnnIyVo7lbYwftlJkID v+rTXkHGc86n1EF+qdUxe4E1TfL+P3r0B/kd9jkWlo7aDc4tEjp1JU2wKjtrHvr5epm7 kKNGyIrt8dMMEm+HJwADOQt2JcrfAnO0XURVHhp3fEOlyr2Skzg6OuKRPr80NkMmopFJ GMzA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=Uk7nXa99; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=a1MI8g3y; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=bk8Uw1tj; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=qxFVf2ZQ; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 46e09a7af769-8303aaffed1si3503570a34.106.2026.10.09.08.28.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 09 Oct 2026 08:28:12 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=Uk7nXa99; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=a1MI8g3y; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=bk8Uw1tj; dkim=neutral (body hash did not verify) header.i=@mandelbit.com header.s=MBO0001 header.b=qxFVf2ZQ; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=lZDXe5knLt/f3yoAHQJDqTTJQuYEIKoJ5dzJ1WF78xs=; b=Uk7nXa99S2Wa5YbaONYjW7pZm+ pR6pG8coQr42wUMENarnAhOJDEwrdf6cv8zkklO6ja+SdKi0ZvwEJ+9TGX4SpQlZLvbnzlnFszsyB W+6M49TzxgSZltaskeOgXJINTXjI0YHr0XJrpQbYUAgJDWTvdRnMgN4AL5aR6V4NITXk=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1xFCVz-0007it-UP; Fri, 09 Oct 2026 15:28:08 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1xFCVz-0007ik-BP for openvpn-devel@lists.sourceforge.net; Fri, 09 Oct 2026 15:28:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=5j4W9Rql+N2/OQC6coiJAzzsa84IYKxuCKmISW1SUE8=; b=a1MI8g3y/bC/w2cGFxMNeG22SK /2cv9NPdnVmmygLurYaEtMNCymOkbdDvKmyOPyf8GDPyOvS9m5zvqD4d7XaemnUbhASsg6ReK4s4a AQidIYMOiVPp7tDM73oz4wDPlKXKUApItyKDGqp840j3cR5CM0tmPxNkRPaLfPXcfvOo=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=5j4W9Rql+N2/OQC6coiJAzzsa84IYKxuCKmISW1SUE8=; b=bk8Uw1tj2Lyav2bEMQnrXyt3Nc HHJFpY7GoOxrdaxvCP6+/bCvj1vT0BiQVddH0+lOslEECT/V0NjPsMlMF/nPVSCT+NnJOQ2Ei7OT2 oe1250w1lVUH26oXGal0zYqPyO8hNQU54fJqROSHcElOCwBe/FQ4zlXkm+yU8sLZk7ro=; Received: from mout-b-201.mailbox.org ([195.10.208.61]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1xFCVy-0003sC-3A for openvpn-devel@lists.sourceforge.net; Fri, 09 Oct 2026 15:28:08 +0000 Received: from smtp102.mailbox.org (smtp102.mailbox.org [IPv6:2001:67c:2050:b231:465::102]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519MLKEM768 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by mout-b-201.mailbox.org (Postfix) with ESMTPS id 4j1W2k0wdxzLmK8; Fri, 09 Oct 2026 17:27:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mandelbit.com; s=MBO0001; t=1791559678; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5j4W9Rql+N2/OQC6coiJAzzsa84IYKxuCKmISW1SUE8=; b=qxFVf2ZQP0HENHCpSv1uBvZ3dWzSvpc53s5Aw9b0kth6Y6biY3Vk/rGgcHr2M1sdqDDY/+ SPXLNaTNSPhlB5QUVi9SXwbJxt4wXmgbIT+U1Ck1+/0ile57hwuxByw23mefW9oxFBaO8F Le5DQRjLnrT2Khsw+iCxzHuUElbjiGiq7idpzqWl9JgNjaGxJNHYhfB+ZL0FFFVg5LyGEM SQLaFRoz76HoqhjHhAeh9hUk6wvJpLJkprubjbOOWQLR7ZeMWiBr5/NMAIm5eMKGTK8+eW 1vLgZS7oaTENYh+C9zUi2ZIi/dfuHNjERtuqZ3eHEez7KI60/Sd7urzBbtlD5w== Authentication-Results: outgoing_mbo_mout; dkim=none; spf=pass (outgoing_mbo_mout: domain of marco@mandelbit.com designates 2001:67c:2050:b231:465::102 as permitted sender) smtp.mailfrom=marco@mandelbit.com From: Marco Baffo To: openvpn-devel@lists.sourceforge.net Date: Fri, 9 Oct 2026 17:27:47 +0200 Message-ID: <20261009152750.740958-2-marco@mandelbit.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20261009152750.740958-1-marco@mandelbit.com> References: <20261009152750.740958-1-marco@mandelbit.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 4j1W2k0wdxzLmK8 X-Spam-Score: -0.2 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Ralf Lici Each packet holds its peer across packet processing and asynchronous crypto completion. When traffic for one peer is spread across several CPUs, the shared kref cache line therefore moves between thos [...] Content analysis details: (-0.2 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1xFCVy-0003sC-3A Subject: [Openvpn-devel] [PATCH ovpn net-next v2 1/3] ovpn: use percpu references for peers X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1878586495908159556 X-GMAIL-MSGID: 1878586495908159556 From: Ralf Lici Each packet holds its peer across packet processing and asynchronous crypto completion. When traffic for one peer is spread across several CPUs, the shared kref cache line therefore moves between those CPUs for every packet. Replace the peer kref with percpu_ref and kill the initial reference exactly once after the peer has been removed from its lookup structures and detached from its socket. Keep the existing RCU-delayed destruction and release the percpu_ref storage in the final RCU callback. Use an unconditional percpu_ref_get where the caller already owns a peer reference or otherwise excludes teardown. This includes TCP callbacks that observe an ovpn_socket under RCU: socket teardown clears sk_user_data and waits for RCU readers before killing the peer. RCU-protected peer lookup paths instead use percpu_ref_tryget_live_rcu, so they stop admitting new users as soon as teardown starts. percpu_ref_init can fail, so propagate allocation failure from peer creation. This trades a per-CPU counter allocation for a cheaper live data path on these long-lived, heavily shared objects. Specifically, in terms of memory, this costs: 8 bytes per possible CPU for the counters, a 56-byte control object, and 8 additional bytes in struct ovpn_peer. In a 32-stream test using one peer and one key, perf c2c placed the peer kref cacheline second among system-wide shared cachelines, with 37 sampled HITM loads attributed to the locked reference update. After this change that shared reference line was no longer sampled, while the untouched key-slot kref addressed by the next commit remained visible. RX used a single receive queue, so only TX exercised CPU fan-out in this test. Signed-off-by: Ralf Lici --- Changes in v2: - Use unconditional peer holds in TCP socket callbacks, as observing an ovpn_socket under RCU guarantees that peer teardown has not reached percpu_ref_kill. - Document the socket-detachment and peer-kill ordering in the commit message and ovpn_tcp_close. drivers/net/ovpn/io.c | 6 +----- drivers/net/ovpn/netlink.c | 6 +++--- drivers/net/ovpn/peer.c | 37 ++++++++++++++++++++++-------------- drivers/net/ovpn/peer.h | 37 ++++++++++++++++++++++++++++++------ drivers/net/ovpn/tcp.c | 39 +++++++++++++++++++------------------- 5 files changed, 77 insertions(+), 48 deletions(-) diff --git a/drivers/net/ovpn/io.c b/drivers/net/ovpn/io.c index 9526f8096da60..15072ce960fa4 100644 --- a/drivers/net/ovpn/io.c +++ b/drivers/net/ovpn/io.c @@ -321,11 +321,7 @@ static bool ovpn_encrypt_one(struct ovpn_peer *peer, struct sk_buff *skb) /* take a reference to the peer because the crypto code may run async. * ovpn_encrypt_post() will release it upon completion */ - if (unlikely(!ovpn_peer_hold(peer))) { - DEBUG_NET_WARN_ON_ONCE(1); - ovpn_crypto_key_slot_put(ks); - return false; - } + ovpn_peer_hold(peer); memset(ovpn_skb_cb(skb), 0, sizeof(struct ovpn_cb)); ovpn_encrypt_post(skb, ovpn_aead_encrypt(peer, ks, skb)); diff --git a/drivers/net/ovpn/netlink.c b/drivers/net/ovpn/netlink.c index 4dad852941982..8cdf46b611425 100644 --- a/drivers/net/ovpn/netlink.c +++ b/drivers/net/ovpn/netlink.c @@ -463,11 +463,11 @@ int ovpn_nl_peer_new_doit(struct sk_buff *skb, struct genl_info *info) ovpn_socket_release(peer); peer_release: /* For UDP, the peer is unreachable until added to the hashtables, so - * dropping the initial reference is enough. For TCP, the peer may be + * killing the initial reference is enough. For TCP, the peer may be * concurrently reachable via sk_user_data->peer until - * ovpn_socket_release() detaches; rely on the refcount. + * ovpn_socket_release() detaches; rely on the percpu reference. */ - ovpn_peer_put(peer); + ovpn_peer_kill(peer); return ret; } diff --git a/drivers/net/ovpn/peer.c b/drivers/net/ovpn/peer.c index c95656ca7c357..970ca7232ae45 100644 --- a/drivers/net/ovpn/peer.c +++ b/drivers/net/ovpn/peer.c @@ -22,6 +22,8 @@ #include "peer.h" #include "socket.h" +static void ovpn_peer_release_ref(struct percpu_ref *ref); + static void unlock_ovpn(struct ovpn_priv *ovpn, struct llist_head *release_list) __releases(&ovpn->lock) @@ -33,7 +35,7 @@ static void unlock_ovpn(struct ovpn_priv *ovpn, llist_for_each_entry_safe(peer, next, release_list->first, release_entry) { ovpn_socket_release(peer); - ovpn_peer_put(peer); + ovpn_peer_kill(peer); } } @@ -113,7 +115,6 @@ struct ovpn_peer *ovpn_peer_new(struct ovpn_priv *ovpn, u32 id) RCU_INIT_POINTER(peer->bind, NULL); ovpn_crypto_state_init(&peer->crypto); spin_lock_init(&peer->lock); - kref_init(&peer->refcount); ovpn_peer_stats_init(&peer->vpn_stats); ovpn_peer_stats_init(&peer->link_stats); INIT_WORK(&peer->keepalive_work, ovpn_peer_keepalive_send); @@ -127,6 +128,14 @@ struct ovpn_peer *ovpn_peer_new(struct ovpn_priv *ovpn, u32 id) return ERR_PTR(ret); } + ret = percpu_ref_init(&peer->refcount, ovpn_peer_release_ref, 0, + GFP_KERNEL); + if (ret < 0) { + dst_cache_destroy(&peer->dst_cache); + kfree(peer); + return ERR_PTR(ret); + } + netdev_hold(ovpn->dev, &peer->dev_tracker, GFP_KERNEL); return peer; @@ -348,6 +357,7 @@ static void ovpn_peer_release_rcu(struct rcu_head *head) * perform it in the RCU callback, when all contexts are done */ dst_cache_destroy(&peer->dst_cache); + percpu_ref_exit(&peer->refcount); kfree(peer); } @@ -366,12 +376,12 @@ static void ovpn_peer_release(struct ovpn_peer *peer) } /** - * ovpn_peer_release_kref - callback for kref_put - * @kref: the kref object belonging to the peer + * ovpn_peer_release_ref - callback for the peer percpu reference + * @ref: the percpu_ref object belonging to the peer */ -void ovpn_peer_release_kref(struct kref *kref) +static void ovpn_peer_release_ref(struct percpu_ref *ref) { - struct ovpn_peer *peer = container_of(kref, struct ovpn_peer, refcount); + struct ovpn_peer *peer = container_of(ref, struct ovpn_peer, refcount); ovpn_peer_release(peer); } @@ -569,7 +579,7 @@ ovpn_peer_get_by_transp_addr_p2p(struct ovpn_priv *ovpn, rcu_read_lock(); tmp = rcu_dereference(ovpn->peer); if (likely(tmp && ovpn_peer_transp_match(tmp, ss) && - ovpn_peer_hold(tmp))) + ovpn_peer_hold_rcu(tmp))) peer = tmp; rcu_read_unlock(); @@ -610,7 +620,7 @@ struct ovpn_peer *ovpn_peer_get_by_transp_addr(struct ovpn_priv *ovpn, if (!ovpn_peer_transp_match(tmp, &ss)) continue; - if (!ovpn_peer_hold(tmp)) + if (!ovpn_peer_hold_rcu(tmp)) continue; peer = tmp; @@ -641,7 +651,7 @@ static struct ovpn_peer *ovpn_peer_get_by_id_p2p(struct ovpn_priv *ovpn, rcu_read_lock(); tmp = rcu_dereference(ovpn->peer); - if (likely(tmp && tmp->id == peer_id && ovpn_peer_hold(tmp))) + if (likely(tmp && tmp->id == peer_id && ovpn_peer_hold_rcu(tmp))) peer = tmp; rcu_read_unlock(); @@ -671,7 +681,7 @@ struct ovpn_peer *ovpn_peer_get_by_id(struct ovpn_priv *ovpn, u32 peer_id) if (tmp->id != peer_id) continue; - if (!ovpn_peer_hold(tmp)) + if (!ovpn_peer_hold_rcu(tmp)) continue; peer = tmp; @@ -745,7 +755,7 @@ struct ovpn_peer *ovpn_peer_get_by_dst(struct ovpn_priv *ovpn, if (ovpn->mode == OVPN_MODE_P2P) { rcu_read_lock(); peer = rcu_dereference(ovpn->peer); - if (unlikely(peer && !ovpn_peer_hold(peer))) + if (unlikely(peer && !ovpn_peer_hold_rcu(peer))) peer = NULL; rcu_read_unlock(); return peer; @@ -763,7 +773,7 @@ struct ovpn_peer *ovpn_peer_get_by_dst(struct ovpn_priv *ovpn, break; } - if (unlikely(peer && !ovpn_peer_hold(peer))) + if (unlikely(peer && !ovpn_peer_hold_rcu(peer))) peer = NULL; rcu_read_unlock(); @@ -1369,8 +1379,7 @@ static time64_t ovpn_peer_keepalive_work_single(struct ovpn_peer *peer, netdev_dbg(peer->ovpn->dev, "sending keepalive to peer %u\n", peer->id); - if (WARN_ON(!ovpn_peer_hold(peer))) - return 0; + ovpn_peer_hold(peer); if (!queue_work(ovpn_wq, &peer->keepalive_work)) ovpn_peer_put(peer); } diff --git a/drivers/net/ovpn/peer.h b/drivers/net/ovpn/peer.h index dfa5c0037e02b..489bf26466847 100644 --- a/drivers/net/ovpn/peer.h +++ b/drivers/net/ovpn/peer.h @@ -10,6 +10,7 @@ #ifndef _NET_OVPN_OVPNPEER_H_ #define _NET_OVPN_OVPNPEER_H_ +#include #include #include @@ -110,7 +111,7 @@ struct ovpn_peer { struct ovpn_peer_stats link_stats; enum ovpn_del_peer_reason delete_reason; spinlock_t lock; /* protects bind and keepalive* */ - struct kref refcount; + struct percpu_ref refcount; struct rcu_head rcu; struct llist_node release_entry; struct work_struct keepalive_work; @@ -120,14 +121,26 @@ struct ovpn_peer { * ovpn_peer_hold - increase reference counter * @peer: the peer whose counter should be increased * - * Return: true if the counter was increased or false if it was zero already + * The caller must already own a peer reference or otherwise guarantee that + * teardown cannot complete while the new reference is acquired. */ -static inline bool ovpn_peer_hold(struct ovpn_peer *peer) +static inline void ovpn_peer_hold(struct ovpn_peer *peer) { - return kref_get_unless_zero(&peer->refcount); + percpu_ref_get(&peer->refcount); } -void ovpn_peer_release_kref(struct kref *kref); +/** + * ovpn_peer_hold_rcu - acquire a live peer reference under RCU + * @peer: peer to acquire + * + * The caller must hold rcu_read_lock. + * + * Return: true if the reference was acquired, false if teardown has started + */ +static inline bool ovpn_peer_hold_rcu(struct ovpn_peer *peer) +{ + return percpu_ref_tryget_live_rcu(&peer->refcount); +} /** * ovpn_peer_put - decrease reference counter @@ -135,7 +148,19 @@ void ovpn_peer_release_kref(struct kref *kref); */ static inline void ovpn_peer_put(struct ovpn_peer *peer) { - kref_put(&peer->refcount, ovpn_peer_release_kref); + percpu_ref_put(&peer->refcount); +} + +/** + * ovpn_peer_kill - drop the peer's initial reference + * @peer: peer which has been unpublished and is being destroyed + * + * This must be called exactly once, after the peer is no longer reachable + * through its owning ovpn instance. + */ +static inline void ovpn_peer_kill(struct ovpn_peer *peer) +{ + percpu_ref_kill(&peer->refcount); } struct ovpn_peer *ovpn_peer_new(struct ovpn_priv *ovpn, u32 id); diff --git a/drivers/net/ovpn/tcp.c b/drivers/net/ovpn/tcp.c index 8fe8a8e750a40..47af3a62f3551 100644 --- a/drivers/net/ovpn/tcp.c +++ b/drivers/net/ovpn/tcp.c @@ -140,21 +140,18 @@ static void ovpn_tcp_rcv(struct strparser *strp, struct sk_buff *skb) /* hold reference to peer as required by ovpn_recv(). * * NOTE: in this context we should already be holding a reference to - * this peer, therefore ovpn_peer_hold() is not expected to fail + * this peer */ - if (WARN_ON(!ovpn_peer_hold(peer))) - goto err_nopeer; + ovpn_peer_hold(peer); ovpn_recv(peer, skb); return; err: - /* take reference for deferred peer deletion. should never fail */ - if (WARN_ON(!ovpn_peer_hold(peer))) - goto err_nopeer; + /* take reference for deferred peer deletion */ + ovpn_peer_hold(peer); if (!queue_work(ovpn_wq, &peer->tcp.defer_del_work)) ovpn_peer_put(peer); ovpn_dev_dstats_rx_dropped(peer->ovpn->dev); -err_nopeer: kfree_skb(skb); } @@ -168,11 +165,12 @@ static int ovpn_tcp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, rcu_read_lock(); sock = rcu_dereference_sk_user_data(sk); - if (unlikely(!sock || !sock->peer || !ovpn_peer_hold(sock->peer))) { + if (unlikely(!sock || !sock->peer)) { rcu_read_unlock(); return -EBADF; } peer = sock->peer; + ovpn_peer_hold(peer); rcu_read_unlock(); skb = __skb_recv_datagram(sk, &peer->tcp.user_queue, flags, &off, &err); @@ -385,10 +383,11 @@ static void ovpn_tcp_release(struct sock *sk) /* during initialization this function is called before * assigning sock->peer */ - if (unlikely(!peer || !ovpn_peer_hold(peer))) { + if (unlikely(!peer)) { rcu_read_unlock(); return; } + ovpn_peer_hold(peer); rcu_read_unlock(); __skb_queue_head_init(&queue); @@ -411,13 +410,14 @@ static int ovpn_tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) lock_sock(sk); rcu_read_lock(); sock = rcu_dereference_sk_user_data(sk); - if (unlikely(!sock || !sock->peer || !ovpn_peer_hold(sock->peer))) { + if (unlikely(!sock || !sock->peer)) { rcu_read_unlock(); release_sock(sk); return -EIO; } - rcu_read_unlock(); peer = sock->peer; + ovpn_peer_hold(peer); + rcu_read_unlock(); if (msg->msg_flags & ~(MSG_DONTWAIT | MSG_NOSIGNAL)) { ret = -EOPNOTSUPP; @@ -588,10 +588,12 @@ static void ovpn_tcp_close(struct sock *sk, long timeout) } peer = sock->peer; - if (!peer || !ovpn_peer_hold(peer)) { + if (!peer) { rcu_read_unlock(); return; } + /* socket teardown waits for RCU readers before killing the peer */ + ovpn_peer_hold(peer); rcu_read_unlock(); ovpn_peer_del(peer, OVPN_DEL_PEER_REASON_TRANSPORT_DISCONNECT); @@ -604,7 +606,7 @@ static __poll_t ovpn_tcp_poll(struct file *file, struct socket *sock, { struct sk_buff_head *queue = &sock->sk->sk_receive_queue; struct ovpn_socket *ovpn_sock; - struct ovpn_peer *peer = NULL; + struct ovpn_peer *peer; __poll_t mask; rcu_read_lock(); @@ -619,16 +621,13 @@ static __poll_t ovpn_tcp_poll(struct file *file, struct socket *sock, return 0; } - if (ovpn_peer_hold(ovpn_sock->peer)) { - peer = ovpn_sock->peer; - queue = &peer->tcp.user_queue; - } + peer = ovpn_sock->peer; + ovpn_peer_hold(peer); + queue = &peer->tcp.user_queue; rcu_read_unlock(); mask = datagram_poll_queue(file, sock, wait, queue); - - if (peer) - ovpn_peer_put(peer); + ovpn_peer_put(peer); return mask; }