From patchwork Tue Aug 25 15:53:19 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gert Doering X-Patchwork-Id: 5278 Return-Path: Delivered-To: patchwork@openvpn.net Received: by 2002:a05:7000:4306:b0:87d:ab56:3700 with SMTP id q6csp5988317mae; Tue, 25 Aug 2026 08:53:45 -0700 (PDT) X-Forwarded-Encrypted: i=2; AHgh+RrdBYk81jFmBBhh8sO8hmEPTFC28qQ/yarSmlRT2qqdkWzxPLNLEFmjtRk4vx2APENwMEoDXJ34hho=@openvpn.net X-Received: by 2002:a05:6820:2906:b0:6a3:e297:2f14 with SMTP id 006d021491bc7-6b1903861a4mr6534973eaf.8.1787673225570; Tue, 25 Aug 2026 08:53:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1787673225; cv=none; d=google.com; s=arc-20260327; b=M+EBXR8gB4Y5xfmtjHBV4ET3FAD0rs2cF7WKcTRvVKaHG27OC42+RH9TmPLgm4pHF1 7Dq9NXaI8cQHPPVtstyh+CskMabr2cXdRxdYRzJY5c/IkNW4vJLz/P8CO1ImPP9YbdIY RaoWxMprY4qiHcWAvsxmjem+kAgNCQRw2A7dSkZxIK/EHubnic/f/hz/fL0JiBAsiByR DvZutvVr01AbgASRYZV59prrNr3qKMA9kbCY2RovW8jl5rlYWThyJeS02qp+3zGnBeRK JyFBIcPHGdacklQiFhhSyGlU0ocmPnxKP5Z61pG4FHwV7pWG3ILwE3TWSog73RneNrFv 4mIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:dkim-signature:dkim-signature; bh=NL22E/Yi2Vb+ZkkHuxYlor+IA0qIDgbbCkwnR/Fyz5U=; fh=4NbAC/LsuMLI0S0hprUlLSLCiHwg6SCAifhH718Jh0Q=; b=JNqBK02BuvLzg3aFOZMU03TTkoTFTa+dRX5eDsNey/4RCn2s8eSSyuVn1qcWbk1dyi jRRFICJxYVt7UMaKcR9y0peZ9o74xCFhQlYso0jCVr3XYz183Ja1temvnJXJg61fc/Fl +oHdmXPM11XgM/vf5BcyhdFMhlsQYlym7yin+JTPDPxMUnSxy8w6Id+YysmIZJyT9OcH NUpcgFg/XXgf27KgsIBRBHesxxKqjCnVhzxSpx735e3caD8MvF8WKNMECTczePomSRVC Pvmww4i0XxnETNxGFqFC6T6HjZi0ThDQ0ncsfD7M6VPiZx64nzGjhVy7ooqu6pfrV8lT 8zrA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=aQjs5r93; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=EFj3gSDS; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=TcJfaQzM; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=muc.de Received: from lists.sourceforge.net (lists.sourceforge.net. [216.105.38.7]) by mx.google.com with ESMTPS id 586e51a60fabf-463836e0a6fsi13700925fac.342.2026.08.25.08.53.45 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 25 Aug 2026 08:53:45 -0700 (PDT) Received-SPF: pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) client-ip=216.105.38.7; Authentication-Results: mx.google.com; dkim=pass header.i=@lists.sourceforge.net header.s=beta header.b=aQjs5r93; dkim=neutral (body hash did not verify) header.i=@sourceforge.net header.s=x header.b=EFj3gSDS; dkim=neutral (body hash did not verify) header.i=@sf.net header.s=x header.b=TcJfaQzM; spf=pass (google.com: domain of openvpn-devel-bounces@lists.sourceforge.net designates 216.105.38.7 as permitted sender) smtp.mailfrom=openvpn-devel-bounces@lists.sourceforge.net; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=muc.de DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.sourceforge.net; s=beta; h=Content-Transfer-Encoding:Content-Type: List-Subscribe:List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id: Subject:MIME-Version:References:In-Reply-To:Message-ID:Date:To:From:Sender: Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NL22E/Yi2Vb+ZkkHuxYlor+IA0qIDgbbCkwnR/Fyz5U=; b=aQjs5r93hH82FJb1MXb5A/zixh NxP6iXPe+rEzeScLXz1bYDbibkXOEgmg7GxK/T48dmb/XCH4sgY+pzcYH3VG7UURX6zN3VMUbU8cE F0OFxI4tSrpHLTMXWcUtYCw3k807zgW8x0kj2MVWUnBjCMpFgWxA1Pka18aPkXdz9g2g=; Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1wytT6-0002Dq-Dt; Tue, 25 Aug 2026 15:53:41 +0000 Received: from [172.30.29.66] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1wytSy-0002Df-F6 for openvpn-devel@lists.sourceforge.net; Tue, 25 Aug 2026 15:53:34 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=LrFBpYjCctKc0Sp44UY5IsQZJpMkzKP1vwDWADL26/8=; b=EFj3gSDSDcUv3uNiCK534oDvlK iPHj+ex2a2pvxy/ZCKsN4QFsild7lbjxjFypIFIiuUhjXqW8kApqnLrBHE/IlYL+xCcBliSpcNOUm m6ZU8UYMw/hE30svmg46/JFpMggP44HUnNBC0k4vpkx9GtodaiKpjB+N6Nav912Gfi/Q=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=LrFBpYjCctKc0Sp44UY5IsQZJpMkzKP1vwDWADL26/8=; b=TcJfaQzMvY0asCvQEP8wIapilX 1mu3FF0qJfi8Mk0jIzTqmQBhn2/bzRuw5me6DUZX9p2AeMJL96rKcANTRihBYwTcB//b5hfuWq4Q6 VuB2SMRDqc4h1i0Qd23ypAWn6uv/ULq/dE/1zVP+re0SAex9uKOp87Dl62+r54rbTZZM=; Received: from [193.149.48.129] (helo=blue.greenie.muc.de) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1wytSt-0002Px-VS for openvpn-devel@lists.sourceforge.net; Tue, 25 Aug 2026 15:53:33 +0000 Received: from blue.greenie.muc.de (localhost [127.0.0.1]) by blue.greenie.muc.de (8.18.1/8.18.1) with ESMTP id 67PFrPvr004972 for ; Tue, 25 Aug 2026 17:53:25 +0200 Received: (from gert@localhost) by blue.greenie.muc.de (8.18.2/8.18.1/Submit) id 67PFrPiP004971 for openvpn-devel@lists.sourceforge.net; Tue, 25 Aug 2026 17:53:25 +0200 From: Gert Doering To: openvpn-devel@lists.sourceforge.net Date: Tue, 25 Aug 2026 17:53:19 +0200 Message-ID: <20260825155325.4960-1-gert@greenie.muc.de> X-Mailer: git-send-email 2.53.0 In-Reply-To: References: MIME-Version: 1.0 X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "sfi-spamd-1.hosts.colo.sdot.me", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: From: Frank Lichtenheld Allow to mark tests that must fail. The EXPECT_FAIL message must contain a message that can be found in the log output. OpenVPN must not start up successfully, so this does not allow to test for failu [...] Content analysis details: (1.3 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.3 RDNS_NONE Delivered to internal network by a host with no rDNS X-Headers-End: 1wytSt-0002Px-VS Subject: [Openvpn-devel] [PATCH v5] t_client.sh.in: Add support for expected test failures X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox X-GMAIL-THRID: 1874511239947876029 X-GMAIL-MSGID: 1874511239947876029 From: Frank Lichtenheld Allow to mark tests that must fail. The EXPECT_FAIL message must contain a message that can be found in the log output. OpenVPN must not start up successfully, so this does not allow to test for failures that only affect the traffic over the link. This is based on work by Gert Doering for the t_server test infrastructure. Change-Id: I6c51bcdda5666e0dab2adcdaff28408c9bfcb844 Signed-off-by: Frank Lichtenheld Gerrit URL: https://gerrit.openvpn.net/c/openvpn/+/1755 --- This change was reviewed on Gerrit and approved by at least one developer. I request to merge it to master. Gerrit URL: https://gerrit.openvpn.net/c/openvpn/+/1755 This mail reflects revision 5 of this Change. Acked-by according to Gerrit (reflected above): diff --git a/tests/t_client.rc-sample b/tests/t_client.rc-sample index a35d185..04714ef 100644 --- a/tests/t_client.rc-sample +++ b/tests/t_client.rc-sample @@ -65,6 +65,7 @@ # EXPECT_IFCONFIG6_x = "this IPv6 address needs to show up in ifconfig" # PING4_HOSTS_x = "these hosts musts ping when openvpn is up (IPv4 fping)" # PING6_HOSTS_x = "these hosts musts ping when openvpn is up (IPv6 fping6)" +# EXPECT_FAIL_x = "instead of waiting for connection wait for this error message" # # hook test options: # diff --git a/tests/t_client.sh.in b/tests/t_client.sh.in index 230a273..dd7f743 100755 --- a/tests/t_client.sh.in +++ b/tests/t_client.sh.in @@ -321,6 +321,7 @@ eval expect_ifconfig6=\"\$EXPECT_IFCONFIG6_$SUF\" eval ping4_hosts=\"\$PING4_HOSTS_$SUF\" eval ping6_hosts=\"\$PING6_HOSTS_$SUF\" + eval expect_fail=\"\$EXPECT_FAIL_$SUF\" eval fping_args=\"\$FPING_EXTRA_ARGS \$FPING_ARGS_$SUF\" # If EXCEPT_IFCONFIG* variables for this test are missing, run an --up @@ -332,6 +333,9 @@ fi output_start "### test run $SUF: '$test_run_title' ###" + if [ -n "$expect_fail" ]; then + output "### expect failure: '$expect_fail'" + fi fail_count=0 if [ -n "$test_check_skip" ]; then @@ -380,23 +384,62 @@ ovpn_init_check=30 ovpn_init_success=0 while [ $ovpn_init_check -gt 0 ]; do - sleep 1 # Wait for OpenVPN to initialize and have had time to write the pid file - grep "Initialization Sequence Completed" $LOGDIR/$SUF:openvpn.log >/dev/null - if [ $? -eq 0 ]; then - ovpn_init_check=0 - ovpn_init_success=1 + sleep 1 # Wait for OpenVPN to initialize and to write the pid file + if [ -n "$expect_fail" ]; then + grep "$expect_fail" $LOGDIR/$SUF:openvpn.log >/dev/null + if [ $? -eq 0 ]; then + ovpn_init_check=0 + ovpn_init_success=1 + sleep 5 # give openvpn time to quit + fi + else + grep "Initialization Sequence Completed" $LOGDIR/$SUF:openvpn.log >/dev/null + if [ $? -eq 0 ]; then + ovpn_init_check=0 + ovpn_init_success=1 + fi fi ovpn_init_check=$((ovpn_init_check - 1)) done - opid=$(cat $pidfile) + opid=$([ -e $pidfile ] && cat $pidfile) if [ -n "$opid" ]; then output " OpenVPN running with PID $opid" else - output " Could not read OpenVPN PID file" + if [ -z "$expect_fail" ]; then # print this only if unexpected + output " Could not read OpenVPN PID file" + fi fi - # If OpenVPN did not start + # did we expect a failure? + if [ -n "$expect_fail" ]; then + if [ -n "$opid" ]; then # OpenVPN did start! + output "$0: OpenVPN did start up, expected failure" + $RUN_SUDO $KILL_EXEC $opid $sudopid + output "tail -5 $SUF:openvpn.log" + output "$(tail -5 $LOGDIR/$SUF:openvpn.log)" + fail "skip rest of sub-tests for test run $SUF." + trap - 0 1 2 3 15 + SUMMARY_FAIL="$SUMMARY_FAIL $SUF" + exit_code=32 + printf "$outbuf" + continue + elif [ $ovpn_init_success -eq 0 ]; then + output "$0: OpenVPN failure did not match expected failure" + output "tail -5 $SUF:openvpn.log" + output "$(tail -5 $LOGDIR/$SUF:openvpn.log)" + SUMMARY_FAIL="$SUMMARY_FAIL $SUF" + exit_code=33 + printf "$outbuf" + continue + else + output "test run $SUF: all tests OK (saw expected failure)." + SUMMARY_OK="$SUMMARY_OK $SUF" + continue + fi + fi + + # If OpenVPN did not start but should have if [ $ovpn_init_success -ne 1 -o -z "$opid" ]; then output "$0: OpenVPN did not initialize in a reasonable time" if [ -n "$opid" ]; then