From patchwork Fri Oct 2 11:51:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vladislav Grishenko X-Patchwork-Id: 1502 Return-Path: Delivered-To: patchwork@openvpn.net Delivered-To: patchwork@openvpn.net Received: from director8.mail.ord1d.rsapps.net ([172.31.255.6]) by backend30.mail.ord1d.rsapps.net with LMTP id kPwAIkOhd182ZQAAIUCqbw (envelope-from ) for ; Fri, 02 Oct 2020 17:53:07 -0400 Received: from proxy10.mail.iad3b.rsapps.net ([172.31.255.6]) by director8.mail.ord1d.rsapps.net with LMTP id qNDdIUOhd18ZPgAAfY0hYg (envelope-from ) for ; Fri, 02 Oct 2020 17:53:07 -0400 Received: from smtp40.gate.iad3b ([172.31.255.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) by proxy10.mail.iad3b.rsapps.net with LMTPS id ADOUGkOhd19QSQAA/F5p9A (envelope-from ) for ; Fri, 02 Oct 2020 17:53:07 -0400 X-Spam-Threshold: 95 X-Spam-Score: 0 X-Spam-Flag: NO X-Virus-Scanned: OK X-Orig-To: openvpnslackdevel@openvpn.net X-Originating-Ip: [216.105.38.7] Authentication-Results: smtp40.gate.iad3b.rsapps.net; iprev=pass policy.iprev="216.105.38.7"; spf=pass smtp.mailfrom="openvpn-devel-bounces@lists.sourceforge.net" smtp.helo="lists.sourceforge.net"; dkim=fail (signature verification failed) header.d=sourceforge.net; dkim=fail (signature verification failed) header.d=sf.net; dkim=fail (signature verification failed) header.d=yandex-team.ru; dmarc=fail (p=none; dis=none) header.from=yandex-team.ru X-Suspicious-Flag: YES X-Classification-ID: a6f84b16-04f9-11eb-869b-5254000cc6d4-1-1 Received: from [216.105.38.7] ([216.105.38.7:43858] helo=lists.sourceforge.net) by smtp40.gate.iad3b.rsapps.net (envelope-from ) (ecelerity 4.2.38.62370 r(:)) with ESMTPS (cipher=DHE-RSA-AES256-GCM-SHA384) id 18/C8-29889-241A77F5; Fri, 02 Oct 2020 17:53:06 -0400 Received: from [127.0.0.1] (helo=sfs-ml-4.v29.lw.sourceforge.com) by sfs-ml-4.v29.lw.sourceforge.com with esmtp (Exim 4.90_1) (envelope-from ) id 1kOSyC-0000KG-Fv; Fri, 02 Oct 2020 21:52:04 +0000 Received: from [172.30.20.202] (helo=mx.sourceforge.net) by sfs-ml-4.v29.lw.sourceforge.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kOSyA-0000K8-RD for openvpn-devel@lists.sourceforge.net; Fri, 02 Oct 2020 21:52:02 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sourceforge.net; s=x; h=Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc: MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=lSEdd1j22mszycyggBM7BN5bYL308FNxAlgGDVbYNN0=; b=hnS839LtMiQyQW0RSOcoDvftA/ RHECt8cDrcNeJ9+M7B71heKUerE/y9efVez8FLQUSGuGkd/F7MyUPOK3XQUvg9M97pSkyMVc7Vh29 L2LtMABZLbN5kz5olpTvtUf++f+V43ricqLra//SoPkW0EHNJqkIpiErtQM6Tz3P3Pek=; DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sf.net; s=x ; h=Message-Id:Date:Subject:To:From:Sender:Reply-To:Cc:MIME-Version: Content-Type:Content-Transfer-Encoding:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=lSEdd1j22mszycyggBM7BN5bYL308FNxAlgGDVbYNN0=; b=CyZ4KIy3idgA2LIP2HdlCga7jw WRGbdb/YZOzJ99kvBMiY7cbBxMOz/49lpz49GtYTk9rKgrqTgWFfFYXdvvUNCsSm2Q+mXMDoYbVdC WDSft3gO+jJIbvoehLnsOq7+zGVWdRcFdM1VLxFx/1DYQiCEWjy99RrZ7KdBGWaehtcE=; Received: from forwardcorp1o.mail.yandex.net ([95.108.205.193]) by sfi-mx-3.v28.lw.sourceforge.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.92.2) id 1kOSy5-00Ed4B-Ei for openvpn-devel@lists.sourceforge.net; Fri, 02 Oct 2020 21:52:02 +0000 Received: from iva8-d077482f1536.qloud-c.yandex.net (iva8-d077482f1536.qloud-c.yandex.net [IPv6:2a02:6b8:c0c:2f26:0:640:d077:482f]) by forwardcorp1o.mail.yandex.net (Yandex) with ESMTP id 464A92E164A for ; Sat, 3 Oct 2020 00:51:50 +0300 (MSK) Received: from iva4-7c3d9abce76c.qloud-c.yandex.net (iva4-7c3d9abce76c.qloud-c.yandex.net [2a02:6b8:c0c:4e8e:0:640:7c3d:9abc]) by iva8-d077482f1536.qloud-c.yandex.net (mxbackcorp/Yandex) with ESMTP id 4HTIbMIgyL-pow0RMNb; Sat, 03 Oct 2020 00:51:50 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex-team.ru; s=default; t=1601675510; bh=lSEdd1j22mszycyggBM7BN5bYL308FNxAlgGDVbYNN0=; h=Message-Id:Date:Subject:To:From; b=xssAw7T+IcRAd6iYjUEa3umYC7adb+cC/srxWqDH8mNRtvKaHfMU+V5sxKOL/zQXz PQQV8P/OBDxNeOstIaXcnUj4FMHekXgYa9J6y50nnwGI3pNM+EQa66yl2StwNg+6YV YjpYhaSeGaGHtsYO32/eIPuQHmKp6+6qgHqUGIGQ= Received: from 37.9.72.13-iva.dhcp.yndx.net (37.9.72.13-iva.dhcp.yndx.net [37.9.72.13]) by iva4-7c3d9abce76c.qloud-c.yandex.net (smtpcorp/Yandex) with ESMTPSA id vio97PRevh-pnmi0IvR; Sat, 03 Oct 2020 00:51:50 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) From: Vladislav Grishenko To: openvpn-devel@lists.sourceforge.net Date: Sat, 3 Oct 2020 02:51:46 +0500 Message-Id: <20201002215146.31324-1-themiron@yandex-team.ru> X-Mailer: git-send-email 2.17.1 X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: yandex-team.ru] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-Headers-End: 1kOSy5-00Ed4B-Ei Subject: [Openvpn-devel] [PATCH] Add CRL extractor script for --crl-verify dir mode X-BeenThere: openvpn-devel@lists.sourceforge.net X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: openvpn-devel-bounces@lists.sourceforge.net X-getmail-retrieved-from-mailbox: Inbox When --crl-verify is enabled, specified CRL file gets reloaded on every client connection. With huge CRL files it may take a significant amount of time - seconds and tens of seconds, during which OpenVPN is blocked and can't serve existing and/or incoming connections due its singlethread nature. In alternative mode --crl-verify option takes directory containing files named as decimal serial numbers of the revoked certificates and 'dir' flag, revoked certificate check is being done by checking the presence of client's certificate number in that directory. This script allow to perform incremental extraction of revoked serial numbers from CRL by adding absent ones and removing excess ones. Usage example: extractcrl.py -f pem /path/to/crl.pem /path/to/outdir extractcrl.py -f der /path/to/crl.crl /path/to/outdir cat /path/to/crl.pem | extractcrl.py -f pem - /path/to/outdir cat /path/to/crl.crl | extractcrl.py -f der - /path/to/outdir Output example: Loaded: 309797 revoked certs in 4.136s Scanned: 312006 files in 0.61s Created: 475 files in 0.05s Removed: 2684 files in 0.116s Signed-off-by: Vladislav Grishenko Acked-by: Gert Doering --- contrib/extract-crl/extractcrl.py | 138 ++++++++++++++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100755 contrib/extract-crl/extractcrl.py diff --git a/contrib/extract-crl/extractcrl.py b/contrib/extract-crl/extractcrl.py new file mode 100755 index 00000000..441464e9 --- /dev/null +++ b/contrib/extract-crl/extractcrl.py @@ -0,0 +1,138 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +''' +Helper script for CRL (certificate revocation list) file extraction +to a directory containing files named as decimal serial numbers of +the revoked certificates, to be used with OpenVPN CRL directory +verify mode. To enable this mode, directory and 'dir' flag needs to +be specified as parameters of '--crl-verify' option. +For more information refer OpenVPN tls-options.rst. + +Usage example: + extractcrl.py -f pem /path/to/crl.pem /path/to/outdir + extractcrl.py -f der /path/to/crl.crl /path/to/outdir + cat /path/to/crl.pem | extractcrl.py -f pem - /path/to/outdir + cat /path/to/crl.crl | extractcrl.py -f der - /path/to/outdir + +Output example: + Loaded: 309797 revoked certs in 4.136s + Scanned: 312006 files in 0.61s + Created: 475 files in 0.05s + Removed: 2684 files in 0.116s +''' + +import argparse +import os +import sys +import time +from subprocess import check_output + +FILETYPE_PEM = 'PEM' +FILETYPE_DER = 'DER' + +def measure_time(method): + def elapsed(*args, **kwargs): + start = time.time() + result = method(*args, **kwargs) + return result, round(time.time() - start, 3) + return elapsed + +@measure_time +def load_crl(filename, format): + + def try_openssl_module(filename, format): + from OpenSSL import crypto + types = { + FILETYPE_PEM: crypto.FILETYPE_PEM, + FILETYPE_DER: crypto.FILETYPE_ASN1 + } + if filename == '-': + crl = crypto.load_crl(types[format], sys.stdin.buffer.read()) + else: + with open(filename, 'rb') as f: + crl = crypto.load_crl(types[format], f.read()) + return set(int(r.get_serial(), 16) for r in crl.get_revoked()) + + def try_openssl_exec(filename, format): + args = ['openssl', 'crl', '-inform', format, '-text'] + if filename != '-': + args += ['-in', filename] + serials = set() + for line in check_output(args, universal_newlines=True).splitlines(): + _, _, serial = line.partition('Serial Number:') + if serial: + serials.add(int(serial.strip(), 16)) + return serials + + try: + return try_openssl_module(filename, format) + except ImportError: + return try_openssl_exec(filename, format) + +@measure_time +def scan_dir(dirname): + _, _, files = next(os.walk(dirname)) + return set(int(f) for f in files if f.isdigit()) + +@measure_time +def create_new_files(dirname, newset, oldset): + addset = newset.difference(oldset) + for serial in addset: + try: + with open(os.path.join(dirname, str(serial)), 'xb'): pass + except FileExistsError: + pass + return addset + +@measure_time +def remove_old_files(dirname, newset, oldset): + delset = oldset.difference(newset) + for serial in delset: + try: + os.remove(os.path.join(dirname, str(serial))) + except FileNotFoundError: + pass + return delset + +def check_crlfile(arg): + if arg == '-' or os.path.isfile(arg): + return arg + raise argparse.ArgumentTypeError('No such file "{}"'.format(arg)) + +def check_outdir(arg): + if os.path.isdir(arg): + return arg + raise argparse.ArgumentTypeError('No such directory: "{}"'.format(arg)) + +def main(): + parser = argparse.ArgumentParser(description='OpenVPN CRL extractor') + parser.add_argument('-f', '--format', + type=str.upper, + default=FILETYPE_PEM, choices=[FILETYPE_PEM, FILETYPE_DER], + help='input CRL format - default {}'.format(FILETYPE_PEM) + ) + parser.add_argument('crlfile', metavar='CRLFILE|-', + type=lambda x: check_crlfile(x), + help='input CRL file or "-" for stdin' + ) + parser.add_argument('outdir', metavar='OUTDIR', + type=lambda x: check_outdir(x), + help='output directory for serials numbers' + ) + args = parser.parse_args() + + certs, t = load_crl(args.crlfile, args.format) + print('Loaded: {} revoked certs in {}s'.format(len(certs), t)) + + files, t = scan_dir(args.outdir) + print('Scanned: {} files in {}s'.format(len(files), t)) + + created, t = create_new_files(args.outdir, certs, files) + print('Created: {} files in {}s'.format(len(created), t)) + + removed, t = remove_old_files(args.outdir, certs, files) + print('Removed: {} files in {}s'.format(len(removed), t)) + +if __name__ == "__main__": + main()