[Openvpn-devel,v2] tests: disable automake serial_tests
Commit Message
Serial mode is the old one and offers much less options for
running the tests. Generally our tests seem to work fine
with the newer parallel mode. The only reason we stuck with
serial_tests seems to be that we didn't like that it doesn't
output the test output by default. We could fix that with a
custom test driver. But will put that into a separate commit.
Change-Id: Ic7265d89142637b0963a6847c6beb06d9163bbb1
Signed-off-by: Frank Lichtenheld <frank@lichtenheld.com>
Acked-by: Arne Schwabe <arne-openvpn@rfc2549.org>
---
This change was reviewed on Gerrit and approved by at least one
developer. I request to merge it to master.
Gerrit URL: https://gerrit.openvpn.net/c/openvpn/+/477
This mail reflects revision 2 of this Change.
Acked-by according to Gerrit (reflected above):
Arne Schwabe <arne-openvpn@rfc2549.org>
Comments
Simplification of the autoconf logic is definitely welcome... tested
on FreeBSD 13.2 with autoconf 2.71 / automake 1.16, and it seems to
nicely run all the tests, with a nice and short summary
PASS: argv_testdriver
PASS: buffer_testdriver
PASS: crypto_testdriver
PASS: packet_id_testdriver
PASS: auth_token_testdriver
PASS: ncp_testdriver
PASS: misc_testdriver
PASS: pkt_testdriver
PASS: tls_crypt_testdriver
PASS: provider_testdriver
============================================================================
Testsuite summary for OpenVPN 2.7_git Unit-Tests
============================================================================
# TOTAL: 10
# PASS: 10
(for the auth_pam_testdriver, the output now explodes from "one test
run" to "and now a 10-line summary", but well...)
I look forward to the promised improvements for the test driver, though
- after
make check-TESTS
SKIP: t_client.sh
PASS: t_lpback.sh
.. the thing just sits there for 2 minutes, with no indication what is
happening and why it's not finished yet (did it crash?) and then it
finally reaches
PASS: t_cltsrv.sh
============================================================================
Testsuite summary for OpenVPN 2.7_git System Tests
============================================================================
# TOTAL: 3
# PASS: 2
# SKIP: 1
.. and "make check V=99" does not improve things here, so this definitely
needs work. Much worse if t_client.sh tests actually run, because then it
just stops at "make check-TESTS" for a looong time...
Your patch has been applied to the master branch.
commit 55d73959cc98241d0ef10dbd091ecf0061b175fe
Author: Frank Lichtenheld
Date: Thu Dec 14 12:16:35 2023 +0100
tests: disable automake serial_tests
Signed-off-by: Frank Lichtenheld <frank@lichtenheld.com>
Acked-by: Arne Schwabe <arne-openvpn@rfc2549.org>
Message-Id: <20231214111635.237429-1-frank@lichtenheld.com>
URL: https://www.mail-archive.com/openvpn-devel@lists.sourceforge.net/msg27812.html
Signed-off-by: Gert Doering <gert@greenie.muc.de>
--
kind regards,
Gert Doering
@@ -41,20 +41,6 @@
AC_CONFIG_SRCDIR([src/openvpn/syshead.h])
AC_CONFIG_MACRO_DIR([m4])
-dnl Initialize automake. automake < 1.12 didn't have serial-tests and
-dnl gives an error if it sees this, but for automake >= 1.13
-dnl serial-tests is required so we have to include it. Solution is to
-dnl test for the version of automake (by running an external command)
-dnl and provide it if necessary. Note we have to do this entirely using
-dnl m4 macros since automake queries this macro by running
-dnl 'autoconf --trace ...'.
-m4_define([serial_tests], [
- m4_esyscmd([automake --version |
- head -1 |
- awk '{split ($NF,a,"."); if (a[1] == 1 && a[2] >= 12) { print "serial-tests" }}'
- ])
-])
-
dnl Automake 1.14+ warns if sources are in sub-directories but subdir-objects
dnl options is not enabled. However, automake before 1.15a has a bug that causes
dnl variable expansion to fail in foo_SOURCES when this option is used.
@@ -69,7 +55,7 @@
# This foreign option prevents autoreconf from overriding our COPYING and
# INSTALL targets:
-AM_INIT_AUTOMAKE(foreign serial_tests subdir_objects 1.9) dnl NB: Do not [quote] this parameter.
+AM_INIT_AUTOMAKE(foreign subdir_objects 1.9) dnl NB: Do not [quote] this parameter.
AC_CANONICAL_HOST
AC_USE_SYSTEM_EXTENSIONS
@@ -14,6 +14,8 @@
SUBDIRS = unit_tests
+AM_TESTSUITE_SUMMARY_HEADER = ' for $(PACKAGE_STRING) System Tests'
+
if !WIN32
test_scripts = t_client.sh t_lpback.sh t_cltsrv.sh
if HAVE_SITNL
@@ -1,5 +1,7 @@
AUTOMAKE_OPTIONS = foreign
+AM_TESTSUITE_SUMMARY_HEADER = ' for $(PACKAGE_STRING) example Unit-Tests'
+
check_PROGRAMS = example_testdriver example2_testdriver
if !CROSS_COMPILING
@@ -1,5 +1,7 @@
AUTOMAKE_OPTIONS = foreign
+AM_TESTSUITE_SUMMARY_HEADER = ' for $(PACKAGE_STRING) Unit-Tests'
+
test_binaries=
if HAVE_LD_WRAP_SUPPORT
@@ -1,5 +1,7 @@
AUTOMAKE_OPTIONS = foreign
+AM_TESTSUITE_SUMMARY_HEADER = ' for $(PACKAGE_STRING) auth_pam Plugin Unit-Tests'
+
if ENABLE_PLUGIN_AUTH_PAM
check_PROGRAMS = auth_pam_testdriver
TESTS = $(check_PROGRAMS)