From patchwork Mon Sep 22 11:06:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marek Mietus X-Patchwork-Id: 4432 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:c08a:b0:72f:f16c:e055 with SMTP id jr10csp113140mab; Mon, 22 Sep 2025 04:07:41 -0700 (PDT) X-Forwarded-Encrypted: i=2; AJvYcCUrIsDRf6YAKC3KjL9gbDSR2IIhhMWUXVV4x9b2NwL2AHIaAKfZFyVQoOVa5NiyCfFgLzzkk2C3Dlo=@openvpn.net X-Google-Smtp-Source: AGHT+IF2ogL4kFgZmuyUnHPTqrtGlBDELr0f9yJeA2/Eghon7kKhb+46B2O9WeiCF3bXILYek0VH X-Received: by 2002:a05:6808:4f54:b0:43d:2bbe:bfba with SMTP id 5614622812f47-43d6c2453camr6712547b6e.23.1758539261648; Mon, 22 Sep 2025 04:07:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1758539261; cv=none; d=google.com; s=arc-20240605; b=V3CZ7C8hTUDwYz7E6vbblQcqm5PnL/SWujM1H9m9fe9CsggMT/lqgflwDhSbIOIFtf dwU7/1DSnku315D24vFWMbulD+yGxEi+AFwPl8GBln+fjqiyrKHVkpI9C59PGV0hjcHm oEMbs30m+BCd8chp5XOXGFQEWLmUf3dJF9FyY3CpEuXBPu0KhA2Asnedc1TnlXA/czHd 831LQ13LEcJcvB1XU3rcXK+v787NV5ridJFjSBtBN8tY3BMSi1Rwr2N9hJEjqrIDqile Oo4Etj/nM/R7eVUM7hIYjLLcg7/l6sAzRPlTWTZ1ArZcrPuM+MQaGlEkhomfakLOxLm2 vCFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=errors-to:content-transfer-encoding:cc:reply-to:from:list-subscribe :list-help:list-post:list-archive:list-unsubscribe:list-id :precedence:subject:mime-version:references:in-reply-to:message-id :date:to:dkim-signature:dkim-signature:dkim-signature:dkim-signature; bh=np6NcisKpYbaShltEb1tpDmJUGrddQn4joCJefGWsPo=; fh=6EbcJF2Br7Ckqvxmbzy/aBW5ZDkZ8Oeawieh63Sh7bg=; b=DvVq4DnfY9zXB14rUUdpZvrafRfHn3oa6QUoMFav/mZiAM6sFXLwvXd59Drm/iQhUS HjkARqFBRkYWY+NhYVwhcHK/j7sLLyFqbN1lEZ6VqmSk3UWkbroYZxYfGi3LreVNupN9 lf4kiO4zXWJdA3EcpjwYsCZ/O6gfzTQr4vr3+l1I8jeGFo605cjVdz6l6kZ08jPDUHiy q0qBeshbyfxF6wd0CKBKFVubgiisluT1tvvxPDMPydOCx7USDNgtGWz9S52xyG3K5izA 7TlVUPEUubULhPkPcIYu8HZqyDc9c0qE9FK32SyMKLB/IJH0uZRDH/oGZDgSWwXUY2wr Y8wQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=nIEjS0WR; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=HkaoBGsU; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=LEchjKqq; dkim=neutral (body hash did not verify) header.i=@yahoo.com header.s=s2048 header.b=QDJUnFGE; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=lists.sourceforge.net Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 5614622812f47-43f26644a8dsi6659b6e.85.2025.09.22.04.07.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 22 Sep 2025 04:07:41 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=nIEjS0WR; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=HkaoBGsU; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=LEchjKqq; dkim=neutral (body hash did not verify) header.i=@yahoo.com header.s=s2048 header.b=QDJUnFGE; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=lists.sourceforge.net DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type:Cc: Reply-To:From:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Subject:MIME-Version:References:In-Reply-To: Message-ID:Date:To:Sender:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=np6NcisKpYbaShltEb1tpDmJUGrddQn4joCJefGWsPo=; b=nIEjS0WRlL1ziik3cqpBhZ69nt hUNGCxzOIFRwyarERmTP+vPrdzXumOxNXeuv6vQFVoOtRN4h/nETausOBv4T9SVAkFJanWCHjdPnx Sq4rFQJfCIonOHBKHx1KwPqk9MbpikqqpJmBaa1fWlxaERZFT1wr+WHqSFFQqC6V+N5I=; Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1v0eOQ-0000Mv-8z; Mon, 22 Sep 2025 11:07:39 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1v0eOO-0000Me-LA for openvpn-devel@lists.sourceforge.net; Mon, 22 Sep 2025 11:07:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=hM0Deb59IC+cO6gRLu6B7UyScwoIMYSKWqYo/8GeKRU=; b=HkaoBGsUwdBUINjoYWrNuaSJs9 se3kY2uGWceCWbhH3SJKnlS0LAuZtLOVSeauQ1z7rToBhtWyxmSzo5TnHuCZLE6rmnU6YoG2Wqgnr 8XmJWF5Xc/1aj0lVj3Jkq+KGNTcZPN3IdcHitp4kKc/spZjk3q5IlFPcRYue8TWSugcU=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=hM0Deb59IC+cO6gRLu6B7UyScwoIMYSKWqYo/8GeKRU=; b=LEchjKqqOQlloffrwfvlK/os23 FxAIXn6EJzMIq16BHGmTJVLWzVIZqUY+zsNJAWQpgqjDIDj+mt8QVbbqmZfHncc4Y6tOzOSfhjbIU stV8qOjqLFJW4yIQxbza+kOsqN7qSX48FqQYWZ68CCCD3SL3YFesu95aIml5DwCPIHNc=; Received: from sonic315-20.consmr.mail.ne1.yahoo.com ([66.163.190.146]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.95) id 1v0eON-00050K-JA for openvpn-devel@lists.sourceforge.net; Mon, 22 Sep 2025 11:07:37 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1758539249; bh=hM0Deb59IC+cO6gRLu6B7UyScwoIMYSKWqYo/8GeKRU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From:Subject:Reply-To; b=QDJUnFGE9BjYJ2yfd+TocxJdxnaa0FxskOLE5ZRT+fRQWZbJCKUO4e7S6UFhMw9v/ua/9GL4loFqDFWk3qQPID2wf3QtSIqcMSJzYojkznBz0kw9IGYR2Z7+qGI4bMRMVoxXkbxfVIMVQYvr8lTTkdTq1YRUSFjIMObeZQoe5KpZpICyBu7BP+GZkV6YjGiQu/XTYzky76LI4JGMsqs/ChSOzpQtPJUBoTJ05ZxxLpLIA4Gyf3OINDXfFhai9Yo5o+VvfHmvdf7BrYbSmmePHjfAXm+J5RwS42klkc1OUVfotOr1Ln+XPxPuj1+oQY71nDm8ykXzmcKKIfIwtUPjcw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1758539249; bh=cCvLKV/D8v2vWBPa55b/1Wl1nr/qFHWqFwuu1HdrDCh=; h=X-Sonic-MF:From:To:Subject:Date:From:Subject; b=bqYBK8OabtggHZyrI0Am2RJDgOC1a/d+mQsivl0ITiN+BDtGcxj3HgrfdXUT93lq9KscDvh3yOQsHoUuEv6T4fi7WU/oVeswHyyL9pII8765k7ORV7VFTx6ydxEAQiQENvP8om/uQPbANvLL3minxCiPjTXg3EYUe13SzAkPq6Xjimre6QDhY3B3HwItUMm158+8D7FOP7xpthQEwZ40HNNo8JieSjfGQ2IJJ21KuOplDVRJSn+IQrOpQ84s3vGXjZa9dDWA07fIYEnaGlfQpVndceP7b5uliNs2jK84pMf3sCKfmWeHiz1iEJJrrZVHkPaVweDK/cAZL7oK39dHbA== X-YMail-OSG: m69wDWgVM1nCp7ETJA.lsO9F7znnyHraAHOc5En_hIHveTpwy8BFU69K5QdRO9v iLsAsXfCh9YIkCNyUEfixPg4gkYIfABGxIirq.CrdbhOHqPVMP35c8D2vL_Qok.HvQkvtwmF_pnb fXflf2bPIljb.Zkml6eLa_WR8rn8yMMH6Uts4WMTJHHPNHY5T3EYnleZAIRO3KQICGWnn777_RPc 9inBn5XfugmOrMMQZWlZA4E11TFmCYHzwrub6no1PVWYEOJjONdy1a2Yp9WuvZ_2RRkHLzEHEUWO KFAQ4yp9iMu0ruPQYKymMb3_HnIPORsrHzUogqwhTvk3Jp8AYjsnGBJ3denygvvsl8yC1YQsGQq3 3dEzjwdevfi_rJcwpKzY6weALpS3GuPePiuq.HdkipHApB9Ia4liw3ozTVU7YEzhsnBMq39E9gc2 ri6czUor6Q9vVUnjlP60vAmlWeDjQc4X.6m9bbuvgpUNZPKp50fyq.hBtp27egpmGbakMgv_6PAR T1ANJTEipx3CUyGB6kmdFPtPVFml3gymfmByEzZVQ.ddiJULMp.4ObQd3I9rmNybZBdUx1dYhljT NuXnkI2_f0957b.mwAgrW7PwYCB0Oj7OaQf.lfbmBbkYpBCxPv3qHB8ZWQ9WfoyvqNJKu6Ungmm. t2EifGVuv7xR8LrT_xFrr9L3pHtd0P36e4bVv7Sx8TLfCnAQe7g3J7ajOQKnLI7a6.iBw6E1nP_u mgj4m6I70PaSuMGfUTq9ZiRrfAO0NHOQQxfAPOEDcxxqChBgDCEzAoIXLgHTEd035tS0o5eDhyHB cWbs19FiCFcibljrduteBvS8s1G3y2oowJk8g7ncHea7EzQAp_tIwmLxD7Mc2MjmyY18esQtAJZV gTLKya4ZsacZLk6iTOfD2Ph6DMn9vhDkt5Heh5SxMtuU8gBpmKn19QPXSR8VKnFnAzjztVZHJXFp s3eKFlkLG.HpT30DR0XJ7f4Ln29kaJqztwubKrSN3qz3MNKtNuvNP.JvCTs051KnwdRBYnav4Zxg w2UlDOSu4W71wuOZxEr.R4CH5tu9mV5ycqR1baUi8PM4bjDKjMn9SAPpo66oqE0IL.V9QrUId.7I mC4WJi0CU.bQUVJMJIR.fynDa_bl29n810G61TfDG1G0eXGGXB.3vB7go4aQE92JzWvg.kFyvP_5 vp4fJEC0LDNDC1UYzmr_kDUnzzUU__.x6u61DfW4aoxQ7jPlLciZTLgF5IigqgRDMv1IEC5mXAO1 dlZWxtQ6E2GJRHUuF8Qq8w62vl25rvQtl70ZUeI5WouH_Y.qf9AIecQyIyZGaROSALvtpDJ.Hn5N iQcVUu4t_4pZo10fTdBIm5XK_PxcLeOv0VoOJyAKmVxsoVik6GuzOqkXDEY6Blq8glGjDWhkBuII D7DrHPC2F5tal27hRBYd_84wZtEoYozJBZMyL..6H2ZqJX_K4qt0o36R3dEPUHzqBYW7BW1u0Glc 8IPbtG8nygZNlF3BPrWhzGlZxMD3pxe1DJQCzQ9Egw8raIlggbANq4lL3p0Jkx9MCJxTR1cc_02j gnVYmKU91ImhIET1DHuMaze_i6qdt80rKoKULbt5oj31LooAvoDNVQhhENp5lYG349Cgm0HOaOBr ONfiYf1kPk9d6el9UX.jSY2IB3mi5f5W.kOFYDahZvLEuasz2xKdEb8GNQOt38up1XquCjgZYlox byJY089bv20_QY161jMk.oAL7u6Od35m62AAD7W8rFM4BT9.sOGT9ekwz.3RdcnrgwlrH9LOKKtm h08AjPdMtsXoUv1.qsT8VTVzVKVK2O_JzySuTVrheYuKHv.2lQ0lL2hsbYomJIuIx7dOX2PHO3tJ evOs53LFBw.ZKlI6nDOCvmW1BISfXDVbRINiEoIEDs5eKjWernrtwA1_iCrw9wNdX3jn.Qs9Yyfg HCd8qsUBVTRFg3FSv8cHW0lunynoeNxJxgMBeCAEfShDx.BtCoNYubSwE4W_yvHOrjNeLtqp23kt z6ga_mvDbWra9eQgwvgPz1d8BMr4dab8MzzwNt0G387lCWWlEvJYsvj4u2BUDuaHLZXc7su.aqJE 5dusDXkg_dli38x0be7gBuWZrCnYfcMlgCMxQsaOuTuKwCvKOfRb2U7.0KDH9eIrgqPDpVwJQ8LU xGnSkduVHIjd8E71uxq6s4Mib5W.9Zg.rHyFnkO5844s7lg-- X-Sonic-MF: X-Sonic-ID: 186ca35c-a907-4a8f-8d36-8f54763c1d8f Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.ne1.yahoo.com with HTTP; Mon, 22 Sep 2025 11:07:29 +0000 Received: by hermes--production-ir2-74585cff4f-4sjhz (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ca5bfbd718d89396be3325d48d68935b; Mon, 22 Sep 2025 11:07:06 +0000 (UTC) To: netdev@vger.kernel.org, sd@queasysnail.net, antonio@openvpn.net Date: Mon, 22 Sep 2025 13:06:20 +0200 Message-ID: <20250922110622.10368-2-mmietus97@yahoo.com> X-Mailer: git-send-email 2.51.0 In-Reply-To: <20250922110622.10368-1-mmietus97@yahoo.com> References: <20250922110622.10368-1-mmietus97@yahoo.com> MIME-Version: 1.0 Content-Length: 8824 X-Spam-Score: 0.1 (/) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-2.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Implement RCU variants for existing dst_cache helpers interacting with dst_entry. The new helpers avoid referencing the dst_entry, sparing some unnecessary atomic operations. They should only be used in flows that are already guaranteed to be inside a RCU read-side critical context [...] Content analysis details: (0.1 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from envelope-from domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider [mmietus97(at)yahoo.com] 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in digit [mmietus97(at)yahoo.com] 0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [66.163.190.146 listed in wl.mailspike.net] X-Headers-End: 1v0eON-00050K-JA Subject: [Openvpn-devel] [PATCH net-next v3 1/3] net: dst_cache: implement RCU variants for dst_cache helpers X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Marek Mietus via Openvpn-devel From: Marek Mietus Reply-To: Marek Mietus Cc: openvpn-devel@lists.sourceforge.net Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: =?utf-8?q?1843962064877773464?= X-GMAIL-MSGID: =?utf-8?q?1843962064877773464?= Implement RCU variants for existing dst_cache helpers interacting with dst_entry. The new helpers avoid referencing the dst_entry, sparing some unnecessary atomic operations. They should only be used in flows that are already guaranteed to be inside a RCU read-side critical context. Signed-off-by: Marek Mietus --- include/net/dst_cache.h | 54 ++++++++++++++++++ net/core/dst_cache.c | 119 ++++++++++++++++++++++++++++++++++++---- 2 files changed, 163 insertions(+), 10 deletions(-) diff --git a/include/net/dst_cache.h b/include/net/dst_cache.h index 1961699598e2..2c98451469ef 100644 --- a/include/net/dst_cache.h +++ b/include/net/dst_cache.h @@ -32,6 +32,21 @@ struct dst_entry *dst_cache_get(struct dst_cache *dst_cache); */ struct rtable *dst_cache_get_ip4(struct dst_cache *dst_cache, __be32 *saddr); +/** + * dst_cache_get_ip4_rcu - lookup cache and ipv4 source under RCU + * @dst_cache: the cache + * @saddr: return value for the retrieved source address + * + * Perform cache lookup and fetch ipv4 source without taking a + * reference on the dst. + * Must be called with local BH disabled, and within an rcu read side + * critical section. + * + * Return: Pointer to retrieved rtable if cache is initialized and + * cached dst is valid, NULL otherwise. + */ +struct rtable *dst_cache_get_ip4_rcu(struct dst_cache *dst_cache, __be32 *saddr); + /** * dst_cache_set_ip4 - store the ipv4 dst into the cache * @dst_cache: the cache @@ -43,6 +58,17 @@ struct rtable *dst_cache_get_ip4(struct dst_cache *dst_cache, __be32 *saddr); void dst_cache_set_ip4(struct dst_cache *dst_cache, struct dst_entry *dst, __be32 saddr); +/** + * dst_cache_steal_ip4 - store the ipv4 dst into the cache and steal its + * reference + * @dst_cache: the cache + * @dst: the entry to be cached whose reference will be stolen + * @saddr: the source address to be stored inside the cache + * + * local BH must be disabled + */ +void dst_cache_steal_ip4(struct dst_cache *dst_cache, struct dst_entry *dst, + __be32 saddr); #if IS_ENABLED(CONFIG_IPV6) /** @@ -56,6 +82,18 @@ void dst_cache_set_ip4(struct dst_cache *dst_cache, struct dst_entry *dst, void dst_cache_set_ip6(struct dst_cache *dst_cache, struct dst_entry *dst, const struct in6_addr *saddr); +/** + * dst_cache_steal_ip6 - store the ipv6 dst into the cache and steal its + * reference + * @dst_cache: the cache + * @dst: the entry to be cached whose reference will be stolen + * @saddr: the source address to be stored inside the cache + * + * local BH must be disabled + */ +void dst_cache_steal_ip6(struct dst_cache *dst_cache, struct dst_entry *dst, + const struct in6_addr *saddr); + /** * dst_cache_get_ip6 - perform cache lookup and fetch ipv6 source address * @dst_cache: the cache @@ -65,6 +103,22 @@ void dst_cache_set_ip6(struct dst_cache *dst_cache, struct dst_entry *dst, */ struct dst_entry *dst_cache_get_ip6(struct dst_cache *dst_cache, struct in6_addr *saddr); + +/** + * dst_cache_get_ip6_rcu - lookup cache and ipv6 source under RCU + * @dst_cache: the cache + * @saddr: return value for the retrieved source address + * + * Perform cache lookup and fetch ipv6 source without taking a + * reference on the dst. + * Must be called with local BH disabled, and within an rcu read side + * critical section. + * + * Return: Pointer to retrieved dst_entry if cache is initialized and + * cached dst is valid, NULL otherwise. + */ +struct dst_entry *dst_cache_get_ip6_rcu(struct dst_cache *dst_cache, + struct in6_addr *saddr); #endif /** diff --git a/net/core/dst_cache.c b/net/core/dst_cache.c index 9ab4902324e1..7d00745ac7d3 100644 --- a/net/core/dst_cache.c +++ b/net/core/dst_cache.c @@ -25,20 +25,27 @@ struct dst_cache_pcpu { }; }; -static void dst_cache_per_cpu_dst_set(struct dst_cache_pcpu *dst_cache, - struct dst_entry *dst, u32 cookie) +static void __dst_cache_per_cpu_dst_set(struct dst_cache_pcpu *dst_cache, + struct dst_entry *dst, u32 cookie) { DEBUG_NET_WARN_ON_ONCE(!in_softirq()); dst_release(dst_cache->dst); - if (dst) - dst_hold(dst); dst_cache->cookie = cookie; dst_cache->dst = dst; } -static struct dst_entry *dst_cache_per_cpu_get(struct dst_cache *dst_cache, - struct dst_cache_pcpu *idst) +static void dst_cache_per_cpu_dst_set(struct dst_cache_pcpu *dst_cache, + struct dst_entry *dst, u32 cookie) +{ + if (dst) + dst_hold(dst); + + __dst_cache_per_cpu_dst_set(dst_cache, dst, cookie); +} + +static struct dst_entry *__dst_cache_per_cpu_get(struct dst_cache *dst_cache, + struct dst_cache_pcpu *idst) { struct dst_entry *dst; @@ -47,14 +54,10 @@ static struct dst_entry *dst_cache_per_cpu_get(struct dst_cache *dst_cache, if (!dst) goto fail; - /* the cache already hold a dst reference; it can't go away */ - dst_hold(dst); - if (unlikely(!time_after(idst->refresh_ts, READ_ONCE(dst_cache->reset_ts)) || (READ_ONCE(dst->obsolete) && !dst->ops->check(dst, idst->cookie)))) { dst_cache_per_cpu_dst_set(idst, NULL, 0); - dst_release(dst); goto fail; } return dst; @@ -64,6 +67,18 @@ static struct dst_entry *dst_cache_per_cpu_get(struct dst_cache *dst_cache, return NULL; } +static struct dst_entry *dst_cache_per_cpu_get(struct dst_cache *dst_cache, + struct dst_cache_pcpu *idst) +{ + struct dst_entry *dst; + + dst = __dst_cache_per_cpu_get(dst_cache, idst); + if (dst) + /* the cache already hold a dst reference; it can't go away */ + dst_hold(dst); + return dst; +} + struct dst_entry *dst_cache_get(struct dst_cache *dst_cache) { struct dst_entry *dst; @@ -100,6 +115,28 @@ struct rtable *dst_cache_get_ip4(struct dst_cache *dst_cache, __be32 *saddr) } EXPORT_SYMBOL_GPL(dst_cache_get_ip4); +struct rtable *dst_cache_get_ip4_rcu(struct dst_cache *dst_cache, __be32 *saddr) +{ + struct dst_cache_pcpu *idst; + struct dst_entry *dst; + + if (!dst_cache->cache) + return NULL; + + local_lock_nested_bh(&dst_cache->cache->bh_lock); + idst = this_cpu_ptr(dst_cache->cache); + dst = __dst_cache_per_cpu_get(dst_cache, idst); + if (!dst) { + local_unlock_nested_bh(&dst_cache->cache->bh_lock); + return NULL; + } + + *saddr = idst->in_saddr.s_addr; + local_unlock_nested_bh(&dst_cache->cache->bh_lock); + return dst_rtable(dst); +} +EXPORT_SYMBOL_GPL(dst_cache_get_ip4_rcu); + void dst_cache_set_ip4(struct dst_cache *dst_cache, struct dst_entry *dst, __be32 saddr) { @@ -116,6 +153,24 @@ void dst_cache_set_ip4(struct dst_cache *dst_cache, struct dst_entry *dst, } EXPORT_SYMBOL_GPL(dst_cache_set_ip4); +void dst_cache_steal_ip4(struct dst_cache *dst_cache, struct dst_entry *dst, + __be32 saddr) +{ + struct dst_cache_pcpu *idst; + + if (!dst_cache->cache) { + dst_release(dst); + return; + } + + local_lock_nested_bh(&dst_cache->cache->bh_lock); + idst = this_cpu_ptr(dst_cache->cache); + __dst_cache_per_cpu_dst_set(idst, dst, 0); + idst->in_saddr.s_addr = saddr; + local_unlock_nested_bh(&dst_cache->cache->bh_lock); +} +EXPORT_SYMBOL_GPL(dst_cache_steal_ip4); + #if IS_ENABLED(CONFIG_IPV6) void dst_cache_set_ip6(struct dst_cache *dst_cache, struct dst_entry *dst, const struct in6_addr *saddr) @@ -135,6 +190,26 @@ void dst_cache_set_ip6(struct dst_cache *dst_cache, struct dst_entry *dst, } EXPORT_SYMBOL_GPL(dst_cache_set_ip6); +void dst_cache_steal_ip6(struct dst_cache *dst_cache, struct dst_entry *dst, + const struct in6_addr *saddr) +{ + struct dst_cache_pcpu *idst; + + if (!dst_cache->cache) { + dst_release(dst); + return; + } + + local_lock_nested_bh(&dst_cache->cache->bh_lock); + + idst = this_cpu_ptr(dst_cache->cache); + __dst_cache_per_cpu_dst_set(idst, dst, + rt6_get_cookie(dst_rt6_info(dst))); + idst->in6_saddr = *saddr; + local_unlock_nested_bh(&dst_cache->cache->bh_lock); +} +EXPORT_SYMBOL_GPL(dst_cache_steal_ip6); + struct dst_entry *dst_cache_get_ip6(struct dst_cache *dst_cache, struct in6_addr *saddr) { @@ -158,6 +233,30 @@ struct dst_entry *dst_cache_get_ip6(struct dst_cache *dst_cache, return dst; } EXPORT_SYMBOL_GPL(dst_cache_get_ip6); + +struct dst_entry *dst_cache_get_ip6_rcu(struct dst_cache *dst_cache, + struct in6_addr *saddr) +{ + struct dst_cache_pcpu *idst; + struct dst_entry *dst; + + if (!dst_cache->cache) + return NULL; + + local_lock_nested_bh(&dst_cache->cache->bh_lock); + + idst = this_cpu_ptr(dst_cache->cache); + dst = __dst_cache_per_cpu_get(dst_cache, idst); + if (!dst) { + local_unlock_nested_bh(&dst_cache->cache->bh_lock); + return NULL; + } + + *saddr = idst->in6_saddr; + local_unlock_nested_bh(&dst_cache->cache->bh_lock); + return dst; +} +EXPORT_SYMBOL_GPL(dst_cache_get_ip6_rcu); #endif int dst_cache_init(struct dst_cache *dst_cache, gfp_t gfp)