From patchwork Tue Sep 20 03:23:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gert Doering X-Patchwork-Id: 2780 Return-Path: Delivered-To: patchwork@openvpn.net Delivered-To: patchwork@openvpn.net Received: from director8.mail.ord1d.rsapps.net ([172.31.255.6]) by backend30.mail.ord1d.rsapps.net with LMTP id MMIjOi6/KWOCdwAAIUCqbw (envelope-from ) for ; Tue, 20 Sep 2022 09:25:02 -0400 Received: from proxy2.mail.iad3b.rsapps.net ([172.31.255.6]) by director8.mail.ord1d.rsapps.net with LMTP id AIBfOS6/KWMpKAAAfY0hYg (envelope-from ) for ; Tue, 20 Sep 2022 09:25:02 -0400 Received: from smtp14.gate.iad3b ([172.31.255.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by proxy2.mail.iad3b.rsapps.net with LMTPS id 8NQbMy6/KWMAcAAAvAZTew (envelope-from ) for ; Tue, 20 Sep 2022 09:25:02 -0400 X-Spam-Threshold: 95 X-Spam-Score: 0 X-Spam-Flag: NO X-Virus-Scanned: OK X-Orig-To: openvpnslackdevel@openvpn.net X-Originating-Ip: [216.105.38.7] Authentication-Results: smtp14.gate.iad3b.rsapps.net; iprev=pass policy.iprev="216.105.38.7"; spf=pass smtp.mailfrom="openvpn-devel-bounces@lists.sourceforge.net" smtp.helo="lists.sourceforge.net"; dkim=fail (signature verification failed) header.d=sourceforge.net; dkim=fail (signature verification failed) header.d=sf.net; dmarc=none (p=nil; dis=none) header.from=greenie.muc.de X-Suspicious-Flag: YES X-Classification-ID: a154c6b6-38e7-11ed-8330-52540057873d-1-1 Received: from [216.105.38.7] ([216.105.38.7:55874] helo=lists.sourceforge.net) by smtp14.gate.iad3b.rsapps.net (envelope-from ) (ecelerity 4.2.38.62370 r(:)) with ESMTPS (cipher=DHE-RSA-AES256-GCM-SHA384) id C9/10-05902-D2FB9236; Tue, 20 Sep 2022 09:25:02 -0400 Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1oadEH-0005aQ-OW; Tue, 20 Sep 2022 13:24:01 +0000 Received: from [172.30.20.202] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oadEF-0005a4-MK for openvpn-devel@lists.sourceforge.net; Tue, 20 Sep 2022 13:23:59 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=DmGWsev6Tdh6yBo6tpHze/UXHAIAG7kFSO7XltQL05I=; b=F7wpwnKabH0Mazb88r0HaI42Gw OJt+WbeoCuMhuEyA2fp3uHuPu73f11jWg1LhcTZCX3sWflgxbI4r7qbQ/h/AoiysWTVBz6y3i0zsO 9LWi//FgHfO1UFw7/WS3pNkiWJ03RAVFrzIZru+OVg07v0n6aAKupfsCS7cOq1n6Hwkc=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=DmGWsev6Tdh6yBo6tpHze/UXHAIAG7kFSO7XltQL05I=; b=YXM1RNJPRxNZzqS58NG8LVYZoG SoCfMMiAEUCtK3zuBhs+rYWQuZl7kY/CrqMqSt5t/ZZWUmCQF/emHM89mLK9DZ+CaJvAVDMlnfHRq nQSgveFU+YSeBEnS5ZSfzHmT6X4MPU7yZy94T3VF2VzlRaSw3f/7seNJh34pJYZPSgXo=; Received: from vmail1.greenie.net ([195.30.8.66]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1oadEE-0007zH-AQ for openvpn-devel@lists.sourceforge.net; Tue, 20 Sep 2022 13:23:59 +0000 Received: from gentoo.ov.greenie.net (gentoo.ov.greenie.net [IPv6:2001:608:0:814:0:0:f000:11]) by vmail1.greenie.net (8.17.1/8.16.1) with SMTP id 28KDNp78079165 for ; Tue, 20 Sep 2022 15:23:51 +0200 (CEST) Received: (nullmailer pid 27727 invoked by uid 1000); Tue, 20 Sep 2022 13:23:51 -0000 From: Gert Doering To: openvpn-devel@lists.sourceforge.net Date: Tue, 20 Sep 2022 15:23:51 +0200 Message-Id: <20220920132351.27718-1-gert@greenie.muc.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220920114255.23639-1-gert@greenie.muc.de> References: <20220920114255.23639-1-gert@greenie.muc.de> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (vmail1.greenie.net [IPv6:2001:608:1:995a:20c:29ff:feb8:10eb]); Tue, 20 Sep 2022 15:23:51 +0200 (CEST) X-Spam-Report: Spam detection software, running on the system "util-spamd-2.v13.lw.sourceforge.com", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: If t_client.sh is run interactively, more verbose output is useful to quickly see what it is doing. If run from a CI environment, going through lots of output for successful tests just to find the one [...] Content analysis details: (-2.0 points, 6.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [195.30.8.66 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.0 SPF_NONE SPF: sender does not publish an SPF Record 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different X-Headers-End: 1oadEE-0007zH-AQ Subject: [Openvpn-devel] [PATCH v3] introduce V= level to manage t_client.sh output verbosity X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox If t_client.sh is run interactively, more verbose output is useful to quickly see what it is doing. If run from a CI environment, going through lots of output for successful tests just to find the one that failed is non-useful. Introduce V= environment variable to control output verbosity V=0 - do not print any per-test output at all, just overall summary V=1 - print single header line for each successful test print full output for failing tests V=99 - print full output, always, as before default is V=1 now Signed-off-by: Gert Doering v2: fix erroneous test on "-n" do not accumulate extra "\n" in outbuf (V=1) fix missing "-e" at "test failures. FAIL." message fix missing "\n" when including "diff" output fix missing "-n" when printing outbuf (= extra newline) (and more newlines being shuffled around) v3: fix quoting on inclusion of "ifconfig/route diff", with newlines... Acked-By: Frank Lichtenheld Acked-by: Antonio Quartulli --- tests/t_client.sh.in | 91 +++++++++++++++++++++++++++++--------------- 1 file changed, 60 insertions(+), 31 deletions(-) diff --git a/tests/t_client.sh.in b/tests/t_client.sh.in index 76ac9b22..d90fe638 100755 --- a/tests/t_client.sh.in +++ b/tests/t_client.sh.in @@ -121,17 +121,43 @@ else exit 1 fi +# verbosity, defaults to "1" +V="${V:-1}" + exit_code=0 # ---------------------------------------------------------- # helper functions # ---------------------------------------------------------- +# output progress information +# depending on verbosity level, collect & print only on failure +output_start() +{ + case $V in + 0) outbuf="" ;; # no per-test output at all + 1) echo -e "$@" # compact, details only on failure + outbuf="\n" ;; + *) echo -e "\n$@\n" ;; # print all, with a bit formatting + esac +} + +output() +{ + NO_NL=''; if [ "X$1" = "X-n" ] ; then NO_NL=$1 ; shift ; fi + case $V in + 0) ;; # no per-test output at all + 1) outbuf="$outbuf$@" # print details only on failure + test -z "$NO_NL" && outbuf="$outbuf\n" + ;; + *) echo -e $NO_NL "$@" ;; # print everything + esac +} + # print failure message, increase FAIL counter fail() { - echo "" - echo "FAIL: $@" >&2 + output "FAIL: $@\n" fail_count=$(( $fail_count + 1 )) } @@ -236,7 +262,7 @@ run_ping_tests() for bytes in $sizes_list do - echo "run IPv$proto ping tests ($want), $bytes byte packets..." + output "run IPv$proto ping tests ($want), $bytes byte packets..." echo "$cmd -b $bytes -C 20 -p 250 -q $FPING_EXTRA_ARGS $targetlist" >>$LOGDIR/$SUF:fping.out $cmd -b $bytes -C 20 -p 250 -q $FPING_EXTRA_ARGS $targetlist >>$LOGDIR/$SUF:fping.out 2>&1 @@ -287,32 +313,32 @@ do up="" fi - echo -e "\n### test run $SUF: '$test_run_title' ###\n" + output_start "### test run $SUF: '$test_run_title' ###" fail_count=0 if [ -n "$test_prep" ]; then - echo -e "running preparation: '$test_prep'" + output "running preparation: '$test_prep'" eval $test_prep fi - echo "save pre-openvpn ifconfig + route" + output "save pre-openvpn ifconfig + route" get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route_pre.txt - echo -e "\nrun pre-openvpn ping tests - targets must not be reachable..." + output "\nrun pre-openvpn ping tests - targets must not be reachable..." run_ping_tests 4 want_fail "$ping4_hosts" run_ping_tests 6 want_fail "$ping6_hosts" if [ "$fail_count" = 0 ] ; then - echo -e "OK.\n" + output "OK.\n" else - echo -e "FAIL: make sure that ping hosts are ONLY reachable via VPN, SKIP test $SUF". + fail "make sure that ping hosts are ONLY reachable via VPN, SKIP test $SUF." SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=31 - continue + echo -e "$outbuf" ; continue fi pidfile="${top_builddir}/tests/$LOGDIR/openvpn-$SUF.pid" openvpn_conf="$openvpn_conf --writepid $pidfile $up" - echo " run openvpn $openvpn_conf" + output " run openvpn $openvpn_conf" echo "# src/openvpn/openvpn $openvpn_conf" >$LOGDIR/$SUF:openvpn.log umask 022 $RUN_SUDO "${top_builddir}/src/openvpn/openvpn" $openvpn_conf >>$LOGDIR/$SUF:openvpn.log & @@ -335,25 +361,25 @@ do opid=`cat $pidfile` if [ -n "$opid" ]; then - echo " OpenVPN running with PID $opid" + output " OpenVPN running with PID $opid" else - echo " Could not read OpenVPN PID file" >&2 + output " Could not read OpenVPN PID file" fi # If OpenVPN did not start if [ $ovpn_init_success -ne 1 -o -z "$opid" ]; then - echo "$0: OpenVPN did not initialize in a reasonable time" >&2 + output "$0: OpenVPN did not initialize in a reasonable time" if [ -n "$opid" ]; then $RUN_SUDO $KILL_EXEC $opid fi $RUN_SUDO $KILL_EXEC $sudopid - echo "tail -5 $SUF:openvpn.log" >&2 - tail -5 $LOGDIR/$SUF:openvpn.log >&2 - echo -e "\nFAIL. skip rest of sub-tests for test run $SUF.\n" >&2 + output "tail -5 $SUF:openvpn.log" + output "`tail -5 $LOGDIR/$SUF:openvpn.log`" + fail "skip rest of sub-tests for test run $SUF." trap - 0 1 2 3 15 SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=30 - continue + echo -e "$outbuf" ; continue fi # make sure openvpn client is terminated in case shell exits @@ -361,21 +387,21 @@ do trap "$RUN_SUDO $KILL_EXEC $opid ; trap - 0 ; exit 1" 1 2 3 15 # compare whether anything changed in ifconfig/route setup? - echo "save ifconfig+route" + output "save ifconfig+route" get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route.txt - echo -n "compare pre-openvpn ifconfig+route with current values..." + output -n "compare pre-openvpn ifconfig+route with current values..." if diff $LOGDIR/$SUF:ifconfig_route_pre.txt \ $LOGDIR/$SUF:ifconfig_route.txt >/dev/null then fail "no differences between ifconfig/route before OpenVPN start and now." else - echo -e " OK!\n" + output " OK!\n" fi # post init script needed? if [ -n "$test_postinit" ]; then - echo -e "running post-init cmd: '$test_postinit'" + output "running post-init cmd: '$test_postinit'" eval $test_postinit fi @@ -385,9 +411,9 @@ do run_ping_tests 4 want_ok "$ping4_hosts" run_ping_tests 6 want_ok "$ping6_hosts" - echo -e "ping tests done.\n" + output "ping tests done.\n" - echo "stopping OpenVPN" + output "stopping OpenVPN" $RUN_SUDO $KILL_EXEC $opid wait $! rc=$? @@ -395,23 +421,26 @@ do fail "OpenVPN return code $rc, expect 0" fi - echo -e "\nsave post-openvpn ifconfig + route..." + output "\nsave post-openvpn ifconfig + route..." get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route_post.txt - echo -n "compare pre- and post-openvpn ifconfig + route..." + output -n "compare pre- and post-openvpn ifconfig + route..." if diff $LOGDIR/$SUF:ifconfig_route_pre.txt \ $LOGDIR/$SUF:ifconfig_route_post.txt >$LOGDIR/$SUF:ifconfig_route_diff.txt then - echo -e " OK.\n" + output " OK.\n" else - cat $LOGDIR/$SUF:ifconfig_route_diff.txt >&2 - fail "differences between pre- and post-ifconfig/route" + output "\n\n" "`cat $LOGDIR/$SUF:ifconfig_route_diff.txt`" "\n" + fail "differences between pre- and post-ifconfig/route." fi if [ "$fail_count" = 0 ] ; then - echo -e "test run $SUF: all tests OK.\n" + output "test run $SUF: all tests OK.\n" SUMMARY_OK="$SUMMARY_OK $SUF" else - echo -e "test run $SUF: $fail_count test failures. FAIL.\n"; + if [ "$V" -gt 0 ] ; then + echo -e -n "$outbuf" + echo -e "test run $SUF: $fail_count test failures. FAIL.\n" + fi SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=30 fi