From patchwork Tue Sep 20 01:42:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gert Doering X-Patchwork-Id: 2779 Return-Path: Delivered-To: patchwork@openvpn.net Delivered-To: patchwork@openvpn.net Received: from director11.mail.ord1d.rsapps.net ([172.30.191.6]) by backend30.mail.ord1d.rsapps.net with LMTP id 0I1UEXKnKWMzKQAAIUCqbw (envelope-from ) for ; Tue, 20 Sep 2022 07:43:46 -0400 Received: from proxy9.mail.ord1d.rsapps.net ([172.30.191.6]) by director11.mail.ord1d.rsapps.net with LMTP id eE3yEHKnKWM1NQAAvGGmqA (envelope-from ) for ; Tue, 20 Sep 2022 07:43:46 -0400 Received: from smtp17.gate.ord1d ([172.30.191.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by proxy9.mail.ord1d.rsapps.net with LMTPS id 0NehEHKnKWNEVQAA7h+8OQ (envelope-from ) for ; Tue, 20 Sep 2022 07:43:46 -0400 X-Spam-Threshold: 95 X-Spam-Score: 0 X-Spam-Flag: NO X-Virus-Scanned: OK X-Orig-To: openvpnslackdevel@openvpn.net X-Originating-Ip: [216.105.38.7] Authentication-Results: smtp17.gate.ord1d.rsapps.net; iprev=pass policy.iprev="216.105.38.7"; spf=pass smtp.mailfrom="openvpn-devel-bounces@lists.sourceforge.net" smtp.helo="lists.sourceforge.net"; dkim=fail (signature verification failed) header.d=sourceforge.net; dkim=fail (signature verification failed) header.d=sf.net; dmarc=none (p=nil; dis=none) header.from=greenie.muc.de X-Suspicious-Flag: YES X-Classification-ID: 7b86c03c-38d9-11ed-af5c-5254008de1cb-1-1 Received: from [216.105.38.7] ([216.105.38.7:33776] helo=lists.sourceforge.net) by smtp17.gate.ord1d.rsapps.net (envelope-from ) (ecelerity 4.2.38.62370 r(:)) with ESMTPS (cipher=DHE-RSA-AES256-GCM-SHA384) id 9B/3A-22475-177A9236; Tue, 20 Sep 2022 07:43:45 -0400 Received: from [127.0.0.1] (helo=sfs-ml-1.v29.lw.sourceforge.com) by sfs-ml-1.v29.lw.sourceforge.com with esmtp (Exim 4.95) (envelope-from ) id 1oabeg-00054k-90; Tue, 20 Sep 2022 11:43:10 +0000 Received: from [172.30.20.202] (helo=mx.sourceforge.net) by sfs-ml-1.v29.lw.sourceforge.com with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1oabee-00054N-BQ for openvpn-devel@lists.sourceforge.net; Tue, 20 Sep 2022 11:43:08 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Content-Transfer-Encoding:MIME-Version:References: In-Reply-To:Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=rwa7BU+PZvhAv2rhAEWNSWN7vHA6sm9QEIz56VlzaK4=; b=NGQyjoOyLD7ETeEOczJtjUOwvB Qa3pk9ft2dSztE6aW4ahd/i4QCs12ufh/y1xYGmXEqBd0anzYoXlm3esWAZXYdjXO+PBQPIu1Uzk8 o9Y46eI5lshU5tUSFaQXxzfM7UDU3RJGv/Ae3uz3d0d2JP9NHNnJdk0m1hqGHVnzMwJY=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-Id: Date:Subject:To:From:Sender:Reply-To:Cc:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=rwa7BU+PZvhAv2rhAEWNSWN7vHA6sm9QEIz56VlzaK4=; b=Hp/hiwMKrgosd6MELZEKBil6Mc lZaAzJTdTz4j16zfrODfcVMpj9klNAQa+sofriOeQ3h/RDzxTF67po7AxqD8UVippUnvwnIpJKpXf ZrdNWHIlU585gPEaiUyoVoHteSHTqulfj6ByZS64NDLwseDXm+vgtTm88FTtT35w2adM=; Received: from vmail1.greenie.net ([195.30.8.66]) by sfi-mx-2.v28.lw.sourceforge.com with esmtps (TLS1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.95) id 1oabec-0002J0-Nx for openvpn-devel@lists.sourceforge.net; Tue, 20 Sep 2022 11:43:08 +0000 Received: from gentoo.ov.greenie.net (gentoo.ov.greenie.net [IPv6:2001:608:0:814:0:0:f000:11]) by vmail1.greenie.net (8.17.1/8.16.1) with SMTP id 28KBgtD7071849 for ; Tue, 20 Sep 2022 13:42:55 +0200 (CEST) Received: (nullmailer pid 23648 invoked by uid 1000); Tue, 20 Sep 2022 11:42:55 -0000 From: Gert Doering To: openvpn-devel@lists.sourceforge.net Date: Tue, 20 Sep 2022 13:42:55 +0200 Message-Id: <20220920114255.23639-1-gert@greenie.muc.de> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220917114158.20687-1-gert@greenie.muc.de> References: <20220917114158.20687-1-gert@greenie.muc.de> MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.4 (vmail1.greenie.net [IPv6:2001:608:1:995a:20c:29ff:feb8:10eb]); Tue, 20 Sep 2022 13:42:55 +0200 (CEST) X-Spam-Report: Spam detection software, running on the system "util-spamd-1.v13.lw.sourceforge.com", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: If t_client.sh is run interactively, more verbose output is useful to quickly see what it is doing. If run from a CI environment, going through lots of output for successful tests just to find the one [...] Content analysis details: (-2.0 points, 6.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [195.30.8.66 listed in list.dnswl.org] 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 SPF_NONE SPF: sender does not publish an SPF Record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record X-Headers-End: 1oabec-0002J0-Nx Subject: [Openvpn-devel] [PATCH v2] introduce V= level to manage t_client.sh output verbosity X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox If t_client.sh is run interactively, more verbose output is useful to quickly see what it is doing. If run from a CI environment, going through lots of output for successful tests just to find the one that failed is non-useful. Introduce V= environment variable to control output verbosity V=0 - do not print any per-test output at all, just overall summary V=1 - print single header line for each successful test print full output for failing tests V=99 - print full output, always, as before default is V=1 now Signed-off-by: Gert Doering v2: fix erroneous test on "-n" do not accumulate extra "\n" in outbuf (V=1) fix missing "-e" at "test failures. FAIL." message fix missing "\n" when including "diff" output fix missing "-n" when printing outbuf (= extra newline) (and more newlines being shuffled around) --- tests/t_client.sh.in | 91 +++++++++++++++++++++++++++++--------------- 1 file changed, 60 insertions(+), 31 deletions(-) diff --git a/tests/t_client.sh.in b/tests/t_client.sh.in index 76ac9b22..a3581905 100755 --- a/tests/t_client.sh.in +++ b/tests/t_client.sh.in @@ -121,17 +121,43 @@ else exit 1 fi +# verbosity, defaults to "1" +V="${V:-1}" + exit_code=0 # ---------------------------------------------------------- # helper functions # ---------------------------------------------------------- +# output progress information +# depending on verbosity level, collect & print only on failure +output_start() +{ + case $V in + 0) outbuf="" ;; # no per-test output at all + 1) echo -e "$@" # compact, details only on failure + outbuf="\n" ;; + *) echo -e "\n$@\n" ;; # print all, with a bit formatting + esac +} + +output() +{ + NO_NL=''; if [ "X$1" = "X-n" ] ; then NO_NL=$1 ; shift ; fi + case $V in + 0) ;; # no per-test output at all + 1) outbuf="$outbuf$@" # print details only on failure + test -z "$NO_NL" && outbuf="$outbuf\n" + ;; + *) echo -e $NO_NL "$@" ;; # print everything + esac +} + # print failure message, increase FAIL counter fail() { - echo "" - echo "FAIL: $@" >&2 + output "FAIL: $@\n" fail_count=$(( $fail_count + 1 )) } @@ -236,7 +262,7 @@ run_ping_tests() for bytes in $sizes_list do - echo "run IPv$proto ping tests ($want), $bytes byte packets..." + output "run IPv$proto ping tests ($want), $bytes byte packets..." echo "$cmd -b $bytes -C 20 -p 250 -q $FPING_EXTRA_ARGS $targetlist" >>$LOGDIR/$SUF:fping.out $cmd -b $bytes -C 20 -p 250 -q $FPING_EXTRA_ARGS $targetlist >>$LOGDIR/$SUF:fping.out 2>&1 @@ -287,32 +313,32 @@ do up="" fi - echo -e "\n### test run $SUF: '$test_run_title' ###\n" + output_start "### test run $SUF: '$test_run_title' ###" fail_count=0 if [ -n "$test_prep" ]; then - echo -e "running preparation: '$test_prep'" + output "running preparation: '$test_prep'" eval $test_prep fi - echo "save pre-openvpn ifconfig + route" + output "save pre-openvpn ifconfig + route" get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route_pre.txt - echo -e "\nrun pre-openvpn ping tests - targets must not be reachable..." + output "\nrun pre-openvpn ping tests - targets must not be reachable..." run_ping_tests 4 want_fail "$ping4_hosts" run_ping_tests 6 want_fail "$ping6_hosts" if [ "$fail_count" = 0 ] ; then - echo -e "OK.\n" + output "OK.\n" else - echo -e "FAIL: make sure that ping hosts are ONLY reachable via VPN, SKIP test $SUF". + fail "make sure that ping hosts are ONLY reachable via VPN, SKIP test $SUF." SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=31 - continue + echo -e "$outbuf" ; continue fi pidfile="${top_builddir}/tests/$LOGDIR/openvpn-$SUF.pid" openvpn_conf="$openvpn_conf --writepid $pidfile $up" - echo " run openvpn $openvpn_conf" + output " run openvpn $openvpn_conf" echo "# src/openvpn/openvpn $openvpn_conf" >$LOGDIR/$SUF:openvpn.log umask 022 $RUN_SUDO "${top_builddir}/src/openvpn/openvpn" $openvpn_conf >>$LOGDIR/$SUF:openvpn.log & @@ -335,25 +361,25 @@ do opid=`cat $pidfile` if [ -n "$opid" ]; then - echo " OpenVPN running with PID $opid" + output " OpenVPN running with PID $opid" else - echo " Could not read OpenVPN PID file" >&2 + output " Could not read OpenVPN PID file" fi # If OpenVPN did not start if [ $ovpn_init_success -ne 1 -o -z "$opid" ]; then - echo "$0: OpenVPN did not initialize in a reasonable time" >&2 + output "$0: OpenVPN did not initialize in a reasonable time" if [ -n "$opid" ]; then $RUN_SUDO $KILL_EXEC $opid fi $RUN_SUDO $KILL_EXEC $sudopid - echo "tail -5 $SUF:openvpn.log" >&2 - tail -5 $LOGDIR/$SUF:openvpn.log >&2 - echo -e "\nFAIL. skip rest of sub-tests for test run $SUF.\n" >&2 + output "tail -5 $SUF:openvpn.log" + output "`tail -5 $LOGDIR/$SUF:openvpn.log`" + fail "skip rest of sub-tests for test run $SUF." trap - 0 1 2 3 15 SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=30 - continue + echo -e "$outbuf" ; continue fi # make sure openvpn client is terminated in case shell exits @@ -361,21 +387,21 @@ do trap "$RUN_SUDO $KILL_EXEC $opid ; trap - 0 ; exit 1" 1 2 3 15 # compare whether anything changed in ifconfig/route setup? - echo "save ifconfig+route" + output "save ifconfig+route" get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route.txt - echo -n "compare pre-openvpn ifconfig+route with current values..." + output -n "compare pre-openvpn ifconfig+route with current values..." if diff $LOGDIR/$SUF:ifconfig_route_pre.txt \ $LOGDIR/$SUF:ifconfig_route.txt >/dev/null then fail "no differences between ifconfig/route before OpenVPN start and now." else - echo -e " OK!\n" + output " OK!\n" fi # post init script needed? if [ -n "$test_postinit" ]; then - echo -e "running post-init cmd: '$test_postinit'" + output "running post-init cmd: '$test_postinit'" eval $test_postinit fi @@ -385,9 +411,9 @@ do run_ping_tests 4 want_ok "$ping4_hosts" run_ping_tests 6 want_ok "$ping6_hosts" - echo -e "ping tests done.\n" + output "ping tests done.\n" - echo "stopping OpenVPN" + output "stopping OpenVPN" $RUN_SUDO $KILL_EXEC $opid wait $! rc=$? @@ -395,23 +421,26 @@ do fail "OpenVPN return code $rc, expect 0" fi - echo -e "\nsave post-openvpn ifconfig + route..." + output "\nsave post-openvpn ifconfig + route..." get_ifconfig_route >$LOGDIR/$SUF:ifconfig_route_post.txt - echo -n "compare pre- and post-openvpn ifconfig + route..." + output -n "compare pre- and post-openvpn ifconfig + route..." if diff $LOGDIR/$SUF:ifconfig_route_pre.txt \ $LOGDIR/$SUF:ifconfig_route_post.txt >$LOGDIR/$SUF:ifconfig_route_diff.txt then - echo -e " OK.\n" + output " OK.\n" else - cat $LOGDIR/$SUF:ifconfig_route_diff.txt >&2 - fail "differences between pre- and post-ifconfig/route" + output "\n" `cat $LOGDIR/$SUF:ifconfig_route_diff.txt` "\n" + fail "differences between pre- and post-ifconfig/route." fi if [ "$fail_count" = 0 ] ; then - echo -e "test run $SUF: all tests OK.\n" + output "test run $SUF: all tests OK.\n" SUMMARY_OK="$SUMMARY_OK $SUF" else - echo -e "test run $SUF: $fail_count test failures. FAIL.\n"; + if [ "$V" -gt 0 ] ; then + echo -e -n "$outbuf" + echo -e "test run $SUF: $fail_count test failures. FAIL.\n" + fi SUMMARY_FAIL="$SUMMARY_FAIL $SUF" exit_code=30 fi