From patchwork Sat Sep 17 01:41:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gert Doering X-Patchwork-Id: 2762 Return-Path: Delivered-To: patchwork@openvpn.net Delivered-To: patchwork@openvpn.net Received: from director8.mail.ord1d.rsapps.net ([172.27.255.50]) by backend30.mail.ord1d.rsapps.net with LMTP id YI7vNZ66JWMPfAAAIUCqbw (envelope-from ) for ; Sat, 17 Sep 2022 08:16:30 -0400 Received: from proxy21.mail.iad3a.rsapps.net ([172.27.255.50]) by director8.mail.ord1d.rsapps.net with LMTP id 6BuyNZ66JWM3dQAAfY0hYg (envelope-from ) for ; Sat, 17 Sep 2022 08:16:30 -0400 Received: from smtp34.gate.iad3a ([172.27.255.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by proxy21.mail.iad3a.rsapps.net with LMTPS id WKl7LZ66JWP5XAAASBQwCQ (envelope-from ) for ; Sat, 17 Sep 2022 08:16:30 -0400 X-Spam-Threshold: 95 X-Spam-Score: 0 X-Spam-Flag: NO X-Virus-Scanned: OK X-Orig-To: openvpnslackdevel@openvpn.net X-Originating-Ip: [216.105.38.7] Authentication-Results: smtp34.gate.iad3a.rsapps.net; iprev=pass policy.iprev="216.105.38.7"; spf=pass smtp.mailfrom="openvpn-devel-bounces@lists.sourceforge.net" smtp.helo="lists.sourceforge.net"; dkim=fail (signature verification failed) header.d=sourceforge.net; dkim=fail (signature verification failed) header.d=sf.net; dmarc=none (p=nil; dis=none) header.from=greenie.muc.de X-Suspicious-Flag: YES X-Classification-ID: 8f1dbadc-3682-11ed-bae5-525400865cc7-1-1 Received: from [216.105.38.7] ([216.105.38.7:43502] helo=lists.sourceforge.net) by smtp34.gate.iad3a.rsapps.net (envelope-from ) (ecelerity 4.2.38.62370 r(:)) with ESMTPS (cipher=DHE-RSA-AES256-GCM-SHA384) id 2C/87-18403-D9AB5236; Sat, 17 Sep 2022 08:16:30 -0400 Received: from [127.0.0.1] (helo=sfs-ml-2.v29.lw.sourceforge.com) by sfs-ml-2.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1oZWjf-00081K-1u; Sat, 17 Sep 2022 12:15:51 +0000 Received: from [172.30.20.202] (helo=mx.sourceforge.net) by sfs-ml-2.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oZWjd-00081E-P0 for openvpn-devel@lists.sourceforge.net; Sat, 17 Sep 2022 12:15:49 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:Message-Id: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=VY9emkTweO3nk7oZ5t6uHKjM5NU7eD1KLfw4F3llAFY=; b=CDXyoZovtzh7dY10iMQjMkuZ7+ RFduQGzeKhlucVNY5K3S5+m+ydUNf2ezWg+eNK76EcprxAwPouzWgynN6HnSPuVqqD0Ht9q1V9/Ui XD9/otMAdAxkI6Ke0/78VT8P56GpuboNMZSkEYSUfnMxogLGZXEfVP975GGkeiB9LDSQ=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:Message-Id:Date:Subject:To:From: Sender:Reply-To:Cc:Content-Type:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: References:List-Id:List-Help:List-Unsubscribe:List-Subscribe:List-Post: List-Owner:List-Archive; bh=VY9emkTweO3nk7oZ5t6uHKjM5NU7eD1KLfw4F3llAFY=; b=T p/Ig2FDWLzddF3ivnlZs7nO5ykix/Y537WUPER93Ig611wm4cf7GXktHFVItEgyuclsft9rBtoAQi 5kctKXfehnIu+tEHjudeB/0Axm1szUVsgPTMekk49VnutyA071sDyLwfGxCyvYPKkBH8BQT4csja8 jV0qvIwn1AX8O3gg=; Received: from vmail1.greenie.net ([195.30.8.66]) by sfi-mx-1.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1oZWjc-00BCGE-1m for openvpn-devel@lists.sourceforge.net; Sat, 17 Sep 2022 12:15:49 +0000 Received: from gentoo.ov.greenie.net (gentoo.ov.greenie.net [IPv6:2001:608:0:814:0:0:f000:11]) by vmail1.greenie.net (8.17.1/8.16.1) with SMTP id 28HBfwgc051818 for ; Sat, 17 Sep 2022 13:41:58 +0200 (CEST) Received: (nullmailer pid 20696 invoked by uid 1000); Sat, 17 Sep 2022 11:41:58 -0000 From: Gert Doering To: openvpn-devel@lists.sourceforge.net Date: Sat, 17 Sep 2022 13:41:58 +0200 Message-Id: <20220917114158.20687-1-gert@greenie.muc.de> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (vmail1.greenie.net [IPv6:2001:608:1:995a:20c:29ff:feb8:10eb]); Sat, 17 Sep 2022 13:41:58 +0200 (CEST) X-Spam-Report: Spam detection software, running on the system "util-spamd-2.v13.lw.sourceforge.com", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: If t_client.sh is run interactively, more verbose output is useful to quickly see what it is doing. If run from a CI environment, going through lots of output for successful tests just to find the one [...] Content analysis details: (-2.0 points, 6.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: suf.pid] 0.0 SPF_NONE SPF: sender does not publish an SPF Record -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [195.30.8.66 listed in list.dnswl.org] 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different X-Headers-End: 1oZWjc-00BCGE-1m Subject: [Openvpn-devel] [PATCH] introduce V= level to manage t_client.sh output verbosity X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox If t_client.sh is run interactively, more verbose output is useful to quickly see what it is doing. If run from a CI environment, going through lots of output for successful tests just to find the one that failed is non-useful. Introduce V= environment variable to control output verbosity V=0 - do not print any per-test output at all, just overall summary V=1 - print single header line for each successful test print full output for failing tests V=99 - print full output, always, as before default is V=1 now Signed-off-by: Gert Doering Acked-by: Antonio Quartulli --- tests/t_client.sh.in | 89 +++++++++++++++++++++++++++++--------------- 1 file changed, 59 insertions(+), 30 deletions(-) diff --git a/tests/t_client.sh.in b/tests/t_client.sh.in index 76ac9b22..4c217645 100755 --- a/tests/t_client.sh.in +++ b/tests/t_client.sh.in @@ -121,17 +121,43 @@ else exit 1 fi +# verbosity, defaults to "1" +V="${V:-1}" + exit_code=0 # ---------------------------------------------------------- # helper functions # ---------------------------------------------------------- +# output progress information +# depending on verbosity level, collect & print only on failure +output_start() +{ + outbuf="\n"; + case $V in + 0) outbuf="" ;; # no per-test output at all + 1) echo -e "$@" ;; # compact, details only on failure + *) echo -e "\n$@\n" ;; # print all, with a bit formatting + esac +} + +output() +{ + NO_NL=''; if [ "X$1" = "-n" ] ; then NO_NL=$1 ; shift ; fi + case $V in + 0) ;; # no per-test output at all + 1) outbuf="$outbuf\n$@" # print details only on failure + test -z "$NO_NL" && outbuf="$outbuf\n" + ;; + *) echo -e $NO_NL "$@" ;; # print everything + esac +} + # print failure message, increase FAIL counter fail() { - echo "" - echo "FAIL: $@" >&2 + output "\nFAIL: $@" fail_count=$(( $fail_count + 1 )) } @@ -236,7 +262,7 @@ run_ping_tests() for bytes in $sizes_list do - echo "run IPv$proto ping tests ($want), $bytes byte packets..." + output "run IPv$proto ping tests ($want), $bytes byte packets..." echo "$cmd -b $bytes -C 20 -p 250 -q $FPING_EXTRA_ARGS $targetlist" >>$LOGDIR/$SUF:fping.out $cmd -b $bytes -C 20 -p 250 -q $FPING_EXTRA_ARGS $targetlist >>$LOGDIR/$SUF:fping.out 2>&1 @@ -287,32 +313,32 @@ do up="" fi - echo -e "\n### test run $SUF: '$test_run_title' ###\n" + output_start "### test run $SUF: '$test_run_title' ###" fail_count=0 if [ -n "$test_prep" ]; then - echo -e "running preparation: '$test_prep'" + output "running preparation: '$test_prep'" eval $test_prep fi - echo "save pre-openvpn ifconfig + route" + output "save pre-openvpn ifconfig + route" get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route_pre.txt - echo -e "\nrun pre-openvpn ping tests - targets must not be reachable..." + output "\nrun pre-openvpn ping tests - targets must not be reachable..." run_ping_tests 4 want_fail "$ping4_hosts" run_ping_tests 6 want_fail "$ping6_hosts" if [ "$fail_count" = 0 ] ; then - echo -e "OK.\n" + output "OK.\n" else - echo -e "FAIL: make sure that ping hosts are ONLY reachable via VPN, SKIP test $SUF". + fail "make sure that ping hosts are ONLY reachable via VPN, SKIP test $SUF." SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=31 - continue + echo -e "$outbuf" ; continue fi pidfile="${top_builddir}/tests/$LOGDIR/openvpn-$SUF.pid" openvpn_conf="$openvpn_conf --writepid $pidfile $up" - echo " run openvpn $openvpn_conf" + output " run openvpn $openvpn_conf" echo "# src/openvpn/openvpn $openvpn_conf" >$LOGDIR/$SUF:openvpn.log umask 022 $RUN_SUDO "${top_builddir}/src/openvpn/openvpn" $openvpn_conf >>$LOGDIR/$SUF:openvpn.log & @@ -335,25 +361,25 @@ do opid=`cat $pidfile` if [ -n "$opid" ]; then - echo " OpenVPN running with PID $opid" + output " OpenVPN running with PID $opid" else - echo " Could not read OpenVPN PID file" >&2 + output " Could not read OpenVPN PID file" fi # If OpenVPN did not start if [ $ovpn_init_success -ne 1 -o -z "$opid" ]; then - echo "$0: OpenVPN did not initialize in a reasonable time" >&2 + output "$0: OpenVPN did not initialize in a reasonable time" if [ -n "$opid" ]; then $RUN_SUDO $KILL_EXEC $opid fi $RUN_SUDO $KILL_EXEC $sudopid - echo "tail -5 $SUF:openvpn.log" >&2 - tail -5 $LOGDIR/$SUF:openvpn.log >&2 - echo -e "\nFAIL. skip rest of sub-tests for test run $SUF.\n" >&2 + output "tail -5 $SUF:openvpn.log" + output "`tail -5 $LOGDIR/$SUF:openvpn.log`" + fail "skip rest of sub-tests for test run $SUF." trap - 0 1 2 3 15 SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=30 - continue + echo -e "$outbuf" ; continue fi # make sure openvpn client is terminated in case shell exits @@ -361,21 +387,21 @@ do trap "$RUN_SUDO $KILL_EXEC $opid ; trap - 0 ; exit 1" 1 2 3 15 # compare whether anything changed in ifconfig/route setup? - echo "save ifconfig+route" + output "save ifconfig+route" get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route.txt - echo -n "compare pre-openvpn ifconfig+route with current values..." + output -n "compare pre-openvpn ifconfig+route with current values..." if diff $LOGDIR/$SUF:ifconfig_route_pre.txt \ $LOGDIR/$SUF:ifconfig_route.txt >/dev/null then fail "no differences between ifconfig/route before OpenVPN start and now." else - echo -e " OK!\n" + output " OK!\n" fi # post init script needed? if [ -n "$test_postinit" ]; then - echo -e "running post-init cmd: '$test_postinit'" + output "running post-init cmd: '$test_postinit'" eval $test_postinit fi @@ -385,9 +411,9 @@ do run_ping_tests 4 want_ok "$ping4_hosts" run_ping_tests 6 want_ok "$ping6_hosts" - echo -e "ping tests done.\n" + output "ping tests done.\n" - echo "stopping OpenVPN" + output "stopping OpenVPN" $RUN_SUDO $KILL_EXEC $opid wait $! rc=$? @@ -395,23 +421,26 @@ do fail "OpenVPN return code $rc, expect 0" fi - echo -e "\nsave post-openvpn ifconfig + route..." + output "\nsave post-openvpn ifconfig + route..." get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route_post.txt - echo -n "compare pre- and post-openvpn ifconfig + route..." + output -n "compare pre- and post-openvpn ifconfig + route..." if diff $LOGDIR/$SUF:ifconfig_route_pre.txt \ $LOGDIR/$SUF:ifconfig_route_post.txt >$LOGDIR/$SUF:ifconfig_route_diff.txt then - echo -e " OK.\n" + output " OK.\n" else - cat $LOGDIR/$SUF:ifconfig_route_diff.txt >&2 + output `cat $LOGDIR/$SUF:ifconfig_route_diff.txt` fail "differences between pre- and post-ifconfig/route" fi if [ "$fail_count" = 0 ] ; then - echo -e "test run $SUF: all tests OK.\n" + output "test run $SUF: all tests OK.\n" SUMMARY_OK="$SUMMARY_OK $SUF" else - echo -e "test run $SUF: $fail_count test failures. FAIL.\n"; + if [ "$V" -gt 0 ] ; then + echo -e "$outbuf" + echo "test run $SUF: $fail_count test failures. FAIL.\n" + fi SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=30 fi