mbox series

[Openvpn-devel,0/4] Transport API: offload traffic manipulation to plugins

Message ID 20181230112901.29241-1-a@unstable.cc
Headers show
Series Transport API: offload traffic manipulation to plugins | expand

Message

Antonio Quartulli Dec. 30, 2018, 12:28 a.m. UTC 
From: Antonio Quartulli <antonio@openvpn.net>

Dear all,

This patchset implements a new API called "Transport API" which allows the
core codebase to offload traffic/socket manipulations to an external plugin.

To understand its potential, imagine you want to apply a specific
"mutation" on any packet going out and apply the its inverse, on the receiver
side, for any incoming packet.
Without this patchset, achieving this result would require a big surgery
through the OpenVPN code, and it would be the same for any new mutation one
wants to implements. Thanks to this new API the deemed effort
is drastically reduced.

In a few words, OpenVPN becomes extendible on a new front, by cutting the
need to touch the core codebase any longer.

An easy to imagine use case for this new API would be any "traffic obfuscation
technique": instead of patching the core code (like the famous "xor
patch"), a developer is now allowed to implement an external obfuscation
plugin and distribute/maintain it indepdendently.

This patchset comes with a sample plugin (obfs-test) which can be used as
starting point for new implementations.

Any feedback is welcome!

Note: the code has gone through the following tests with positive outcome:
- uncrustify
- GitLab-CI (with my configs)
- openvpn community buildbots

Since it is quite a change (even though most of the new code is fairly
isolated in its own module - transport.c/h), I expect more
comments/discussion/bugs to spark from here.

In any case, please consider myself as direct contact point to discuss
further changes and improvements (even though I am not the signer of the
patches). I'll be in responsible for ensuring this code gets merged sooner
rather than later.

Thanks a lot!!

== Credits ==

This code has been developed by the Operator Foundation[1], under the
umbrella project called "Jigsaw"[2], mainly promoted/sponsored by Google, Inc.

Brandon, reading this email in CC, has been the contact point on
the Operator Foundation's side, while Justin (in CC as well), managed the
task on Google's behalf.


OpenVPN, Inc. has offered its support to the cause by allowing its devs to
allocate a number of hours to follow the project and ensure it could
get all the guidance it required.

[1] https://operatorfoundation.org/
[2] https://jigsaw.google.com/


Best Regards,



Robin Tarsiger (4):
  transport: introduce tranport API plugin codebase
  socket: introduce INDIRECT transport protocol abstraction
  options: add support for --transport-plugin
  transport-plugin: add sample obfs-test plugin

 configure.ac                              |   9 +
 doc/openvpn.8                             |  40 ++
 include/Makefile.am                       |   1 +
 include/openvpn-plugin.h.in               |  31 +-
 include/openvpn-transport.h               | 240 +++++++++
 src/openvpn/Makefile.am                   |   1 +
 src/openvpn/forward.c                     |   5 +
 src/openvpn/init.c                        |   1 +
 src/openvpn/options.c                     |  31 ++
 src/openvpn/options.h                     |   1 +
 src/openvpn/plugin.c                      |   4 +
 src/openvpn/plugin.h                      |   1 +
 src/openvpn/socket.c                      | 148 +++++-
 src/openvpn/socket.h                      |  74 +++
 src/openvpn/transport.c                   | 303 +++++++++++
 src/openvpn/transport.h                   |  99 ++++
 src/plugins/Makefile.am                   |   2 +-
 src/plugins/obfs-test/Makefile.am         |  29 ++
 src/plugins/obfs-test/README.obfs-test    |  26 +
 src/plugins/obfs-test/obfs-test-args.c    |  60 +++
 src/plugins/obfs-test/obfs-test-munging.c | 129 +++++
 src/plugins/obfs-test/obfs-test-posix.c   | 207 ++++++++
 src/plugins/obfs-test/obfs-test-win32.c   | 579 ++++++++++++++++++++++
 src/plugins/obfs-test/obfs-test.c         |  94 ++++
 src/plugins/obfs-test/obfs-test.exports   |   4 +
 src/plugins/obfs-test/obfs-test.h         |  42 ++
 26 files changed, 2155 insertions(+), 6 deletions(-)
 create mode 100644 include/openvpn-transport.h
 create mode 100644 src/openvpn/transport.c
 create mode 100644 src/openvpn/transport.h
 create mode 100644 src/plugins/obfs-test/Makefile.am
 create mode 100644 src/plugins/obfs-test/README.obfs-test
 create mode 100644 src/plugins/obfs-test/obfs-test-args.c
 create mode 100644 src/plugins/obfs-test/obfs-test-munging.c
 create mode 100644 src/plugins/obfs-test/obfs-test-posix.c
 create mode 100644 src/plugins/obfs-test/obfs-test-win32.c
 create mode 100644 src/plugins/obfs-test/obfs-test.c
 create mode 100644 src/plugins/obfs-test/obfs-test.exports
 create mode 100644 src/plugins/obfs-test/obfs-test.h

Comments

Kristof Provost via Openvpn-devel Jan. 28, 2019, 7:09 a.m. UTC | #1 
Hi all,
I just wanted to bump this to see if there is any feedback on the API?

Thanks!
Justin

Justin Henck
Product Manager
212-565-9811
google.com/jigsaw

PGP: EA8E 8C27 2D75 974D B357 482B 1039 9F2D 869A 117B


On Sun, Dec 30, 2018 at 6:30 AM Antonio Quartulli <a@unstable.cc> wrote:

> From: Antonio Quartulli <antonio@openvpn.net>
>
> Dear all,
>
> This patchset implements a new API called "Transport API" which allows the
> core codebase to offload traffic/socket manipulations to an external
> plugin.
>
> To understand its potential, imagine you want to apply a specific
> "mutation" on any packet going out and apply the its inverse, on the
> receiver
> side, for any incoming packet.
> Without this patchset, achieving this result would require a big surgery
> through the OpenVPN code, and it would be the same for any new mutation one
> wants to implements. Thanks to this new API the deemed effort
> is drastically reduced.
>
> In a few words, OpenVPN becomes extendible on a new front, by cutting the
> need to touch the core codebase any longer.
>
> An easy to imagine use case for this new API would be any "traffic
> obfuscation
> technique": instead of patching the core code (like the famous "xor
> patch"), a developer is now allowed to implement an external obfuscation
> plugin and distribute/maintain it indepdendently.
>
> This patchset comes with a sample plugin (obfs-test) which can be used as
> starting point for new implementations.
>
> Any feedback is welcome!
>
> Note: the code has gone through the following tests with positive outcome:
> - uncrustify
> - GitLab-CI (with my configs)
> - openvpn community buildbots
>
> Since it is quite a change (even though most of the new code is fairly
> isolated in its own module - transport.c/h), I expect more
> comments/discussion/bugs to spark from here.
>
> In any case, please consider myself as direct contact point to discuss
> further changes and improvements (even though I am not the signer of the
> patches). I'll be in responsible for ensuring this code gets merged sooner
> rather than later.
>
> Thanks a lot!!
>
> == Credits ==
>
> This code has been developed by the Operator Foundation[1], under the
> umbrella project called "Jigsaw"[2], mainly promoted/sponsored by Google,
> Inc.
>
> Brandon, reading this email in CC, has been the contact point on
> the Operator Foundation's side, while Justin (in CC as well), managed the
> task on Google's behalf.
>
>
> OpenVPN, Inc. has offered its support to the cause by allowing its devs to
> allocate a number of hours to follow the project and ensure it could
> get all the guidance it required.
>
> [1] https://operatorfoundation.org/
> [2] https://jigsaw.google.com/
>
>
> Best Regards,
>
>
>
> Robin Tarsiger (4):
>   transport: introduce tranport API plugin codebase
>   socket: introduce INDIRECT transport protocol abstraction
>   options: add support for --transport-plugin
>   transport-plugin: add sample obfs-test plugin
>
>  configure.ac                              |   9 +
>  doc/openvpn.8                             |  40 ++
>  include/Makefile.am                       |   1 +
>  include/openvpn-plugin.h.in               |  31 +-
>  include/openvpn-transport.h               | 240 +++++++++
>  src/openvpn/Makefile.am                   |   1 +
>  src/openvpn/forward.c                     |   5 +
>  src/openvpn/init.c                        |   1 +
>  src/openvpn/options.c                     |  31 ++
>  src/openvpn/options.h                     |   1 +
>  src/openvpn/plugin.c                      |   4 +
>  src/openvpn/plugin.h                      |   1 +
>  src/openvpn/socket.c                      | 148 +++++-
>  src/openvpn/socket.h                      |  74 +++
>  src/openvpn/transport.c                   | 303 +++++++++++
>  src/openvpn/transport.h                   |  99 ++++
>  src/plugins/Makefile.am                   |   2 +-
>  src/plugins/obfs-test/Makefile.am         |  29 ++
>  src/plugins/obfs-test/README.obfs-test    |  26 +
>  src/plugins/obfs-test/obfs-test-args.c    |  60 +++
>  src/plugins/obfs-test/obfs-test-munging.c | 129 +++++
>  src/plugins/obfs-test/obfs-test-posix.c   | 207 ++++++++
>  src/plugins/obfs-test/obfs-test-win32.c   | 579 ++++++++++++++++++++++
>  src/plugins/obfs-test/obfs-test.c         |  94 ++++
>  src/plugins/obfs-test/obfs-test.exports   |   4 +
>  src/plugins/obfs-test/obfs-test.h         |  42 ++
>  26 files changed, 2155 insertions(+), 6 deletions(-)
>  create mode 100644 include/openvpn-transport.h
>  create mode 100644 src/openvpn/transport.c
>  create mode 100644 src/openvpn/transport.h
>  create mode 100644 src/plugins/obfs-test/Makefile.am
>  create mode 100644 src/plugins/obfs-test/README.obfs-test
>  create mode 100644 src/plugins/obfs-test/obfs-test-args.c
>  create mode 100644 src/plugins/obfs-test/obfs-test-munging.c
>  create mode 100644 src/plugins/obfs-test/obfs-test-posix.c
>  create mode 100644 src/plugins/obfs-test/obfs-test-win32.c
>  create mode 100644 src/plugins/obfs-test/obfs-test.c
>  create mode 100644 src/plugins/obfs-test/obfs-test.exports
>  create mode 100644 src/plugins/obfs-test/obfs-test.h
>
> --
> 2.19.2
>
>
<div dir="ltr">Hi all,<div>I just wanted to bump this to see if there is any feedback on the API?</div><div><br>Thanks!</div><div>Justin<br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><br><table cellspacing="0" cellpadding="0" style="font-family:&quot;open sans&quot;,sans-serif"><tbody><tr style="color:rgb(102,102,102);font-family:sans-serif;font-size:small"><td nowrap valign="top" style="border:none;padding-right:22px"><img src="https://www.gstatic.com/jigsaw/Jigsaw_logo.png" height="45" width="45" style="height:45px;width:45px"></td><td nowrap style="border:none"><span style="font-weight:bold">Justin Henck</span> <br><span>Product Manager</span><span></span> <br><div style="display:inline"><span>212-565-9811</span> <br></div><a href="https://google.com/jigsaw" style="color:rgb(102,102,102)" target="_blank">google.com/jigsaw</a></td></tr></tbody></table></div><div dir="ltr"><div dir="auto"><div><span><font color="#666666"><br></font></span></div><div><span><font color="#666666">PGP: EA8E 8C27 2D75 974D B357 482B 1039 9F2D 869A 117B</font></span></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Dec 30, 2018 at 6:30 AM Antonio Quartulli &lt;a@unstable.cc&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">From: Antonio Quartulli &lt;<a href="mailto:antonio@openvpn.net" target="_blank">antonio@openvpn.net</a>&gt;<br>
<br>
Dear all,<br>
<br>
This patchset implements a new API called &quot;Transport API&quot; which allows the<br>
core codebase to offload traffic/socket manipulations to an external plugin.<br>
<br>
To understand its potential, imagine you want to apply a specific<br>
&quot;mutation&quot; on any packet going out and apply the its inverse, on the receiver<br>
side, for any incoming packet.<br>
Without this patchset, achieving this result would require a big surgery<br>
through the OpenVPN code, and it would be the same for any new mutation one<br>
wants to implements. Thanks to this new API the deemed effort<br>
is drastically reduced.<br>
<br>
In a few words, OpenVPN becomes extendible on a new front, by cutting the<br>
need to touch the core codebase any longer.<br>
<br>
An easy to imagine use case for this new API would be any &quot;traffic obfuscation<br>
technique&quot;: instead of patching the core code (like the famous &quot;xor<br>
patch&quot;), a developer is now allowed to implement an external obfuscation<br>
plugin and distribute/maintain it indepdendently.<br>
<br>
This patchset comes with a sample plugin (obfs-test) which can be used as<br>
starting point for new implementations.<br>
<br>
Any feedback is welcome!<br>
<br>
Note: the code has gone through the following tests with positive outcome:<br>
- uncrustify<br>
- GitLab-CI (with my configs)<br>
- openvpn community buildbots<br>
<br>
Since it is quite a change (even though most of the new code is fairly<br>
isolated in its own module - transport.c/h), I expect more<br>
comments/discussion/bugs to spark from here.<br>
<br>
In any case, please consider myself as direct contact point to discuss<br>
further changes and improvements (even though I am not the signer of the<br>
patches). I&#39;ll be in responsible for ensuring this code gets merged sooner<br>
rather than later.<br>
<br>
Thanks a lot!!<br>
<br>
== Credits ==<br>
<br>
This code has been developed by the Operator Foundation[1], under the<br>
umbrella project called &quot;Jigsaw&quot;[2], mainly promoted/sponsored by Google, Inc.<br>
<br>
Brandon, reading this email in CC, has been the contact point on<br>
the Operator Foundation&#39;s side, while Justin (in CC as well), managed the<br>
task on Google&#39;s behalf.<br>
<br>
<br>
OpenVPN, Inc. has offered its support to the cause by allowing its devs to<br>
allocate a number of hours to follow the project and ensure it could<br>
get all the guidance it required.<br>
<br>
[1] <a href="https://operatorfoundation.org/" rel="noreferrer" target="_blank">https://operatorfoundation.org/</a><br>
[2] <a href="https://jigsaw.google.com/" rel="noreferrer" target="_blank">https://jigsaw.google.com/</a><br>
<br>
<br>
Best Regards,<br>
<br>
<br>
<br>
Robin Tarsiger (4):<br>
  transport: introduce tranport API plugin codebase<br>
  socket: introduce INDIRECT transport protocol abstraction<br>
  options: add support for --transport-plugin<br>
  transport-plugin: add sample obfs-test plugin<br>
<br>
 <a href="http://configure.ac" rel="noreferrer" target="_blank">configure.ac</a>                              |   9 +<br>
 doc/openvpn.8                             |  40 ++<br>
 include/Makefile.am                       |   1 +<br>
 include/<a href="http://openvpn-plugin.h.in" rel="noreferrer" target="_blank">openvpn-plugin.h.in</a>               |  31 +-<br>
 include/openvpn-transport.h               | 240 +++++++++<br>
 src/openvpn/Makefile.am                   |   1 +<br>
 src/openvpn/forward.c                     |   5 +<br>
 src/openvpn/init.c                        |   1 +<br>
 src/openvpn/options.c                     |  31 ++<br>
 src/openvpn/options.h                     |   1 +<br>
 src/openvpn/plugin.c                      |   4 +<br>
 src/openvpn/plugin.h                      |   1 +<br>
 src/openvpn/socket.c                      | 148 +++++-<br>
 src/openvpn/socket.h                      |  74 +++<br>
 src/openvpn/transport.c                   | 303 +++++++++++<br>
 src/openvpn/transport.h                   |  99 ++++<br>
 src/plugins/Makefile.am                   |   2 +-<br>
 src/plugins/obfs-test/Makefile.am         |  29 ++<br>
 src/plugins/obfs-test/README.obfs-test    |  26 +<br>
 src/plugins/obfs-test/obfs-test-args.c    |  60 +++<br>
 src/plugins/obfs-test/obfs-test-munging.c | 129 +++++<br>
 src/plugins/obfs-test/obfs-test-posix.c   | 207 ++++++++<br>
 src/plugins/obfs-test/obfs-test-win32.c   | 579 ++++++++++++++++++++++<br>
 src/plugins/obfs-test/obfs-test.c         |  94 ++++<br>
 src/plugins/obfs-test/obfs-test.exports   |   4 +<br>
 src/plugins/obfs-test/obfs-test.h         |  42 ++<br>
 26 files changed, 2155 insertions(+), 6 deletions(-)<br>
 create mode 100644 include/openvpn-transport.h<br>
 create mode 100644 src/openvpn/transport.c<br>
 create mode 100644 src/openvpn/transport.h<br>
 create mode 100644 src/plugins/obfs-test/Makefile.am<br>
 create mode 100644 src/plugins/obfs-test/README.obfs-test<br>
 create mode 100644 src/plugins/obfs-test/obfs-test-args.c<br>
 create mode 100644 src/plugins/obfs-test/obfs-test-munging.c<br>
 create mode 100644 src/plugins/obfs-test/obfs-test-posix.c<br>
 create mode 100644 src/plugins/obfs-test/obfs-test-win32.c<br>
 create mode 100644 src/plugins/obfs-test/obfs-test.c<br>
 create mode 100644 src/plugins/obfs-test/obfs-test.exports<br>
 create mode 100644 src/plugins/obfs-test/obfs-test.h<br>
<br>
-- <br>
2.19.2<br>
<br>
</blockquote></div>