[Openvpn-devel,v2] travis-ci: add arm64, s390x builds.

Message ID 20200201135026.18074-1-chipitsine@gmail.com
State Superseded
Headers show
Series [Openvpn-devel,v2] travis-ci: add arm64, s390x builds. | expand

Commit Message

Ilya Shipitsin Feb. 1, 2020, 2:50 a.m. UTC
From: Ilya Shipitsin <chipitsine@gmail.com>

as described on https://docs.travis-ci.com/user/multi-cpu-architectures travis-ci
now supports amd64, ppcle, arm64, s390 architectures. Add arm64 and s390x.

travis-ci images were upgraded to bionic.

"sudo" is deprecated, let us remove it, also "matrix" is deprecated in favour of "jobs".

LD_LIBRARY_PATH was replaced by using "rpath" in LDFLAGS, which is more elegant way of linking.

also, dependencies were upgraded to the latest versions.

travis_wait was added for long openssl builds.

cmocka was added to linux and osx builds.
---

v2 rebased against proper commit

 .travis.yml               | 42 +++++++++++++++++++++------------------
 .travis/build-check.sh    | 10 +---------
 .travis/build-deps.sh     | 10 ++++++----
 .travis/run-build-deps.sh | 10 ----------
 4 files changed, 30 insertions(+), 42 deletions(-)
 delete mode 100755 .travis/run-build-deps.sh

Comments

Lev Stipakov Feb. 2, 2020, 8:40 p.m. UTC | #1
Hi,

Could you provide a link to a travis build with your changes?
Ilya Shipitsin Feb. 2, 2020, 8:48 p.m. UTC | #2
пн, 3 февр. 2020 г. в 12:40, Lev Stipakov <lstipakov@gmail.com>:

> Hi,
>
> Could you provide a link to a travis build with your changes?
>


https://travis-ci.org/chipitsine/openvpn/builds/645173716


there's at least issue regarding "travis_wait 30", as I can see, windows
builds waits for 10 minutes.

I'll fix it in "v3" (after all reviews)

>
> --
> -Lev
>
<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">пн, 3 февр. 2020 г. в 12:40, Lev Stipakov &lt;<a href="mailto:lstipakov@gmail.com">lstipakov@gmail.com</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi,<div><br></div><div>Could you provide a link to a travis build with your changes?</div></div></div></blockquote><div><br></div><div><br></div><div><a href="https://travis-ci.org/chipitsine/openvpn/builds/645173716">https://travis-ci.org/chipitsine/openvpn/builds/645173716</a></div><div><br></div><div><br></div><div>there&#39;s at least issue regarding &quot;travis_wait 30&quot;, as I can see, windows builds waits for 10 minutes.</div><div><br></div><div>I&#39;ll fix it in &quot;v3&quot; (after all reviews)<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr" class="gmail_attr"><br></div></div>-- <br><div dir="ltr">-Lev</div></div>
</blockquote></div></div>
Ilya Shipitsin Feb. 2, 2020, 8:49 p.m. UTC | #3
https://travis-ci.org/chipitsine/openvpn/jobs/645173717#L62

I expected it to be 30 minutes wait.

пн, 3 февр. 2020 г. в 12:48, Илья Шипицин <chipitsine@gmail.com>:

>
>
> пн, 3 февр. 2020 г. в 12:40, Lev Stipakov <lstipakov@gmail.com>:
>
>> Hi,
>>
>> Could you provide a link to a travis build with your changes?
>>
>
>
> https://travis-ci.org/chipitsine/openvpn/builds/645173716
>
>
> there's at least issue regarding "travis_wait 30", as I can see, windows
> builds waits for 10 minutes.
>
> I'll fix it in "v3" (after all reviews)
>
>>
>> --
>> -Lev
>>
>
<div dir="ltr"><div><a href="https://travis-ci.org/chipitsine/openvpn/jobs/645173717#L62">https://travis-ci.org/chipitsine/openvpn/jobs/645173717#L62</a></div><div><br></div><div>I expected it to be 30 minutes wait.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">пн, 3 февр. 2020 г. в 12:48, Илья Шипицин &lt;<a href="mailto:chipitsine@gmail.com">chipitsine@gmail.com</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">пн, 3 февр. 2020 г. в 12:40, Lev Stipakov &lt;<a href="mailto:lstipakov@gmail.com" target="_blank">lstipakov@gmail.com</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi,<div><br></div><div>Could you provide a link to a travis build with your changes?</div></div></div></blockquote><div><br></div><div><br></div><div><a href="https://travis-ci.org/chipitsine/openvpn/builds/645173716" target="_blank">https://travis-ci.org/chipitsine/openvpn/builds/645173716</a></div><div><br></div><div><br></div><div>there&#39;s at least issue regarding &quot;travis_wait 30&quot;, as I can see, windows builds waits for 10 minutes.</div><div><br></div><div>I&#39;ll fix it in &quot;v3&quot; (after all reviews)<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr" class="gmail_attr"><br></div></div>-- <br><div dir="ltr">-Lev</div></div>
</blockquote></div></div>
</blockquote></div>
Ilya Shipitsin Feb. 2, 2020, 9:04 p.m. UTC | #4
also, ARM64 builds are flaky. maybe we should add them as allow_failures.

пн, 3 февр. 2020 г. в 12:49, Илья Шипицин <chipitsine@gmail.com>:

> https://travis-ci.org/chipitsine/openvpn/jobs/645173717#L62
>
> I expected it to be 30 minutes wait.
>
> пн, 3 февр. 2020 г. в 12:48, Илья Шипицин <chipitsine@gmail.com>:
>
>>
>>
>> пн, 3 февр. 2020 г. в 12:40, Lev Stipakov <lstipakov@gmail.com>:
>>
>>> Hi,
>>>
>>> Could you provide a link to a travis build with your changes?
>>>
>>
>>
>> https://travis-ci.org/chipitsine/openvpn/builds/645173716
>>
>>
>> there's at least issue regarding "travis_wait 30", as I can see, windows
>> builds waits for 10 minutes.
>>
>> I'll fix it in "v3" (after all reviews)
>>
>>>
>>> --
>>> -Lev
>>>
>>
<div dir="ltr">also, ARM64 builds are flaky. maybe we should add them as allow_failures.<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">пн, 3 февр. 2020 г. в 12:49, Илья Шипицин &lt;<a href="mailto:chipitsine@gmail.com">chipitsine@gmail.com</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><a href="https://travis-ci.org/chipitsine/openvpn/jobs/645173717#L62" target="_blank">https://travis-ci.org/chipitsine/openvpn/jobs/645173717#L62</a></div><div><br></div><div>I expected it to be 30 minutes wait.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">пн, 3 февр. 2020 г. в 12:48, Илья Шипицин &lt;<a href="mailto:chipitsine@gmail.com" target="_blank">chipitsine@gmail.com</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">пн, 3 февр. 2020 г. в 12:40, Lev Stipakov &lt;<a href="mailto:lstipakov@gmail.com" target="_blank">lstipakov@gmail.com</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi,<div><br></div><div>Could you provide a link to a travis build with your changes?</div></div></div></blockquote><div><br></div><div><br></div><div><a href="https://travis-ci.org/chipitsine/openvpn/builds/645173716" target="_blank">https://travis-ci.org/chipitsine/openvpn/builds/645173716</a></div><div><br></div><div><br></div><div>there&#39;s at least issue regarding &quot;travis_wait 30&quot;, as I can see, windows builds waits for 10 minutes.</div><div><br></div><div>I&#39;ll fix it in &quot;v3&quot; (after all reviews)<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div dir="ltr" class="gmail_attr"><br></div></div>-- <br><div dir="ltr">-Lev</div></div>
</blockquote></div></div>
</blockquote></div>
</blockquote></div>
Steffan Karger Feb. 2, 2020, 10:51 p.m. UTC | #5
On 03-02-2020 09:04, Илья Шипицин wrote:
> also, ARM64 builds are flaky. maybe we should add them as allow_failures.

What is flaky about the ARM64 builds? Is it our build? Is it the travis
infra?

-Steffan
Ilya Shipitsin Feb. 2, 2020, 10:59 p.m. UTC | #6
пн, 3 февр. 2020 г. в 14:51, Steffan Karger <steffan.karger@foxcrypto.com>:

> On 03-02-2020 09:04, Илья Шипицин wrote:
> > also, ARM64 builds are flaky. maybe we should add them as allow_failures.
>
> What is flaky about the ARM64 builds? Is it our build? Is it the travis
> infra?
>

in travis-ci terms "our" failure means "build failed", travis-ci infra
means "build errored"

they run arm64 on linaro cloud, it fails (sometimes) on infra level (I
guess cloud is overloaded or like that). I did not seen yet tests failure


>
> -Steffan
>
>
> _______________________________________________
> Openvpn-devel mailing list
> Openvpn-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/openvpn-devel
>
<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">пн, 3 февр. 2020 г. в 14:51, Steffan Karger &lt;<a href="mailto:steffan.karger@foxcrypto.com">steffan.karger@foxcrypto.com</a>&gt;:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 03-02-2020 09:04, Илья Шипицин wrote:<br>
&gt; also, ARM64 builds are flaky. maybe we should add them as allow_failures.<br>
<br>
What is flaky about the ARM64 builds? Is it our build? Is it the travis<br>
infra?<br></blockquote><div><br></div><div>in travis-ci terms &quot;our&quot; failure means &quot;build failed&quot;, travis-ci infra means &quot;build errored&quot;</div><div><br></div><div>they run arm64 on linaro cloud, it fails (sometimes) on infra level (I guess cloud is overloaded or like that). I did not seen yet tests failure<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
-Steffan<br>
<br>
<br>
_______________________________________________<br>
Openvpn-devel mailing list<br>
<a href="mailto:Openvpn-devel@lists.sourceforge.net" target="_blank">Openvpn-devel@lists.sourceforge.net</a><br>
<a href="https://lists.sourceforge.net/lists/listinfo/openvpn-devel" rel="noreferrer" target="_blank">https://lists.sourceforge.net/lists/listinfo/openvpn-devel</a><br>
</blockquote></div></div>
Arne Schwabe Feb. 12, 2020, 10:07 p.m. UTC | #7
Am 03.02.20 um 10:59 schrieb Илья Шипицин:
> 
> 
> пн, 3 февр. 2020 г. в 14:51, Steffan Karger
> <steffan.karger@foxcrypto.com <mailto:steffan.karger@foxcrypto.com>>:
> 
>     On 03-02-2020 09:04, Илья Шипицин wrote:
>     > also, ARM64 builds are flaky. maybe we should add them as
>     allow_failures.
> 
>     What is flaky about the ARM64 builds? Is it our build? Is it the travis
>     infra?
> 
> 
> in travis-ci terms "our" failure means "build failed", travis-ci infra
> means "build errored"
> 
> they run arm64 on linaro cloud, it fails (sometimes) on infra level (I
> guess cloud is overloaded or like that). I did not seen yet tests failure
>  

Hm. If we ignore the failure then the test is almost worthless since I
think most people only look at the failures if there is an error. If we
build and ignore it, we can also ignore it completely.

Just my 2c.

Arne

Patch

diff --git a/.travis.yml b/.travis.yml
index 40296d87..9f9db954 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,5 +1,4 @@ 
-sudo: required
-dist: xenial
+dist: bionic
 
 os: linux
 
@@ -11,18 +10,18 @@  env:
     - PREFIX="${HOME}/opt"
     - TAP_WINDOWS_VERSION=9.23.3
     - LZO_VERSION=2.10
-    - PKCS11_HELPER_VERSION=1.25.1
-    - MBEDTLS_VERSION=2.16.1
+    - PKCS11_HELPER_VERSION=1.26
+    - MBEDTLS_VERSION=2.16.4
     - MBEDTLS_CFLAGS="-I${PREFIX}/include"
     - MBEDTLS_LIBS="-L${PREFIX}/lib -lmbedtls -lmbedx509 -lmbedcrypto"
-    - OPENSSL_VERSION=1.0.2s
+    - OPENSSL_VERSION=1.0.2u
     - OPENSSL_CFLAGS="-I${PREFIX}/include"
     - OPENSSL_LIBS="-L${PREFIX}/lib -lssl -lcrypto"
     # The next declaration is the encrypted COVERITY_SCAN_TOKEN, created
     #   via the "travis encrypt" command using the project repo's public key
     - secure: "l9mSnEW4LJqjxftH5i1NdDaYfGmQB1mPXnSB3DXnsjzkCWZ+yJLfBemfQ0tx/wS7chBYxqUaUIMT0hw4zJVp/LANFJo2vfh//ymTS6h0uApRY1ofg9Pp1BFcV1laG6/u8pwSZ2EBy/GhCd3DS436oE8sYBRaFM9FU62L/oeQBfJ7r4ID/0eB1b8bqlbD4paty9MHui2P8EZJwR+KAD84prtfpZOcrSMxPh9OUhJxzxUvvVoP4s4+lZ5Kgg1bBQ3yzKGDqe8VOgK2BWCEuezqhMMc8oeKmAe7CUkoz5gsGYH++k3I9XzP9Z4xeJKoQnC/82qi4xkJmlaOxdionej9bHIcjfRt7D8j1J0U+wOj4p8VrDy7yHaxuN2fi0K5MGa/CaXQSrkna8dePniCng+xQ2MY/zxuRX2gA6xPNLUyQLU9LqIug7wj4z84Hk9iWib4L20MoPjeEo+vAUNq8FtjOPxMuHNpv4iGGx6kgJm7RXl5vC5hxfK6MprrnYe2U5Mcd8jpzagKBaKHL3zV2FxX9k0jRO9Mccz7M2WnaV0MQ6zcngzTN4+s0kCjhfGKd2F2ANT2Gkhj3Me36eNHfuE0dBbvYCMh4b3Mgd7b/OuXwQWdJ8PjJ1WHXjSOw5sHw1suaV6cEO2Meyz5j1tOkyOi0M9QF+LFenQ9vLH4sBCww8U="
 
-matrix:
+jobs:
   include:
     - env:
       - SSLLIB="openssl"
@@ -37,11 +36,21 @@  matrix:
     - env: SSLLIB="openssl" OPENSSL_VERSION="1.0.1u"
       os: linux
       compiler: gcc
-    - env: SSLLIB="openssl" OPENSSL_VERSION="1.1.1c"
+    - env: SSLLIB="openssl" OPENSSL_VERSION="1.1.1d"
       os: linux
+      arch: amd64
       compiler: gcc
-    - env: SSLLIB="openssl" OPENSSL_VERSION="1.1.1c" LABEL="linux-ppc64le"
-      os: linux-ppc64le
+    - env: SSLLIB="openssl" OPENSSL_VERSION="1.1.1d"
+      os: linux
+      arch: ppc64le
+      compiler: gcc
+    - env: SSLLIB="openssl" OPENSSL_VERSION="1.1.1d"
+      os: linux
+      arch: arm64
+      compiler: gcc
+    - env: SSLLIB="openssl" OPENSSL_VERSION="1.1.1d"
+      os: linux
+      arch: s390x
       compiler: gcc
     - env: SSLLIB="openssl" EXTRA_CONFIG="--enable-iproute2"
       os: linux
@@ -49,7 +58,7 @@  matrix:
     - env: SSLLIB="openssl" CFLAGS="-fsanitize=address" CC=clang-9
       os: linux
       compiler: clang
-    - env: SSLLIB="openssl" OPENSSL_VERSION="1.1.1c" CC=clang-9
+    - env: SSLLIB="openssl" OPENSSL_VERSION="1.1.1d" CC=clang-9
       os: linux
       compiler: clang
     - env: SSLLIB="mbedtls"
@@ -67,7 +76,7 @@  matrix:
     - env: SSLLIB="openssl" CHOST=x86_64-w64-mingw32 OPENSSL_VERSION="1.0.1u"
       os: linux
       compiler: ": Win64 build only"
-    - env: SSLLIB="openssl" CHOST=x86_64-w64-mingw32 OPENSSL_VERSION="1.1.1c"
+    - env: SSLLIB="openssl" CHOST=x86_64-w64-mingw32 OPENSSL_VERSION="1.1.1d"
       os: linux
       compiler: ": Win64 build only"
     - env: SSLLIB="openssl" CHOST=i686-w64-mingw32
@@ -82,15 +91,11 @@  matrix:
 
 addons:
   apt:
-    sources:
-      - ubuntu-toolchain-r-test
-      - sourceline: 'deb https://apt.llvm.org/xenial/ llvm-toolchain-xenial-9 main'
-        key_url: 'https://apt.llvm.org/llvm-snapshot.gpg.key'
     update: true
-    packages: [ liblzo2-dev, libpam0g-dev, liblz4-dev, linux-libc-dev, man2html, mingw-w64 ]
+    packages: [ liblzo2-dev, libpam0g-dev, liblz4-dev, linux-libc-dev, man2html, mingw-w64, clang-9, libcmocka-dev ]
   homebrew:
     update: true
-    packages: [ lzo ]
+    packages: [ lzo, lz4, cmocka ]
 
 cache:
   directories:
@@ -99,9 +104,8 @@  cache:
   - ${HOME}/Library/Caches/Homebrew
 
 install:
-  - if [ "$TRAVIS_OS_NAME" = "linux" ] && [ "$TRAVIS_CPU_ARCH" != "ppc64le" ]; then sudo apt-get install clang-9; fi
   - if [ ! -z "${CHOST}" ]; then unset CC; fi
-  - .travis/run-build-deps.sh
+  - travis_wait 30 .travis/build-deps.sh > build-deps.log 2>&1 || (cat build-deps.log && exit 1)
 
 before_script:
   - .travis/coverity.sh
diff --git a/.travis/build-check.sh b/.travis/build-check.sh
index f794704c..4e1b7243 100755
--- a/.travis/build-check.sh
+++ b/.travis/build-check.sh
@@ -6,14 +6,6 @@  if [ "${TRAVIS_OS_NAME}" = "windows" ]; then
 	MSBuild.exe openvpn.sln //p:Platform=x64 && exit 0
 fi
 
-if [ "${TRAVIS_OS_NAME}" = "linux" ]; then
-	export LD_LIBRARY_PATH="${PREFIX}/lib:${LD_LIBRARY_PATH:-}"
-fi
-
-if [ "${TRAVIS_OS_NAME}" = "osx"   ]; then
-	export DYLD_LIBRARY_PATH="${PREFIX}/lib:${DYLD_LIBRARY_PATH:-}"
-fi
-
 autoreconf -vi
 
 if [ -z ${CHOST+x} ]; then
@@ -21,7 +13,7 @@  if [ -z ${CHOST+x} ]; then
 		export EXTRA_CONFIG="${EXTRA_CONFIG:-} --enable-werror"
 	fi
 	./configure --with-crypto-library="${SSLLIB}" ${EXTRA_CONFIG:-} || (cat config.log && exit 1)
-	make -j$JOBS
+	make LDFLAGS="-Wl,-rpath,${PREFIX}/lib" -j$JOBS
 	src/openvpn/openvpn --version || true
 	if [ "${TRAVIS_OS_NAME}" = "linux" ]; then
 		ldd src/openvpn/openvpn;
diff --git a/.travis/build-deps.sh b/.travis/build-deps.sh
index 4c5cb05e..0c8d69b8 100755
--- a/.travis/build-deps.sh
+++ b/.travis/build-deps.sh
@@ -44,17 +44,19 @@  build_lzo () {
 }
 
 download_pkcs11_helper () {
-    if [ ! -f "pkcs11-helper-${PKCS11_HELPER_VERSION}.tar.bz2" ]; then
+    if [ ! -f "pkcs11-helper-${PKCS11_HELPER_VERSION}.tar.gz" ]; then
         wget -P download-cache/ \
-            "https://github.com/OpenSC/pkcs11-helper/releases/download/pkcs11-helper-${PKCS11_HELPER_VERSION}/pkcs11-helper-${PKCS11_HELPER_VERSION}.tar.bz2"
+            "https://github.com/OpenSC/pkcs11-helper/archive/pkcs11-helper-${PKCS11_HELPER_VERSION}.tar.gz"
     fi
 }
 
 build_pkcs11_helper () {
     if [ "$(cat ${PREFIX}/.pkcs11_helper-version)" != "${PKCS11_HELPER_VERSION}" ]; then
-        tar jxf download-cache/pkcs11-helper-${PKCS11_HELPER_VERSION}.tar.bz2
+        tar xf download-cache/pkcs11-helper-${PKCS11_HELPER_VERSION}.tar.gz
         (
-            cd "pkcs11-helper-${PKCS11_HELPER_VERSION}"
+            cd "pkcs11-helper-pkcs11-helper-${PKCS11_HELPER_VERSION}"
+
+            autoreconf -iv
 
             ./configure --host=${CHOST} --program-prefix='' --libdir=${PREFIX}/lib \
                  --prefix=${PREFIX} --build=x86_64-pc-linux-gnu \
diff --git a/.travis/run-build-deps.sh b/.travis/run-build-deps.sh
deleted file mode 100755
index b8eb41c5..00000000
--- a/.travis/run-build-deps.sh
+++ /dev/null
@@ -1,10 +0,0 @@ 
-#!/bin/sh
-set -eux
-
-if [ "${TRAVIS_OS_NAME}" = "windows" ]; then
-    # for windows we need to print output since openssl build
-    # might take more than 10 minutes, which causes build abort
-    .travis/build-deps.sh
-else
-    .travis/build-deps.sh > build-deps.log 2>&1 || (cat build-deps.log && exit 1)
-fi