libressl's AEAD api

[ThomasWaldmann]

it seems that they don't support these ciphers via the same EVP api as openssl:

• AES-OCB
• CHACHA20-POLY1305

For borg 1.3+ it seems like a good idea if we would deprecate the crypto modes based on AES-CTR and recommend the new AEAD modes.

So, how's the situation on OpenBSD? Are there other platforms requiring LibreSSL support?

• Do we need to support LibreSSL to run on OpenBSD or could we use OpenSSL there also?
• If we can neither use OpenSSL there nor some compatible API in LibreSSL, do we want to add some code for LibreSSL to our borg.crypto.low_level module that uses LibreSSL's EVP_AEAD api (which does not exist in OpenSSL)? https://man.openbsd.org/EVP_AEAD_CTX_init.3
• seems like we only get AES-GCM and chacha20-poly1305 via that api. there were objections against gcm, so we went for ocb rather. so guess that would mean with LibreSSL it would only be chacha20-poly1305 (which is fine as it is quite fast).

[bket]

@ThomasWaldmann

So, how's the situation on OpenBSD? Are there other platforms requiring LibreSSL support?

• Do we need to support LibreSSL to run on OpenBSD or could we use OpenSSL there also?

Some background:

A lot of people, including the LibreSSL people, OpenBSD's porters and upstream projects, invested a lot of effort to get LibreSSL to work with the hundreds of tools that exist. OpenBSD for example has ~11000 ports (including flavors and subpackages) of which ~800 use LibreSSL and only 6 explicitly require OpenSSL.

Concerning your questions: Strictly taken it is possible to use OpenSSL on OpenBSD, though it is not preferred. I'm not sure about requirements from other platforms.

• If we can neither use OpenSSL there nor some compatible API in LibreSSL, do we want to add some code for LibreSSL to our borg.crypto.low_level module that uses LibreSSL's EVP_AEAD api (which does not exist in OpenSSL)? https://man.openbsd.org/EVP_AEAD_CTX_init.3

EVP_aead seems to be borrowed from BoringSSL, and the API for EVP_aead is completely incompatible with what Borg currently uses.
I'm hoping that the EVP things for some of the djb ciphers will be eventually implemented in LibreSSL. AES-OCB is rarely used and was patent encumbered, which does not not help discussion on implementation.

It would be really cool if Borg keeps on supporting LibreSSL. However, in the end it is all about effort needed, and return on effort...

• seems like we only get AES-GCM and chacha20-poly1305 via that api. there were objections against gcm, so we went for ocb rather. so guess that would mean with LibreSSL it would only be chacha20-poly1305 (which is fine as it is quite fast).

I do not want to start a discussion on crypto as I do not understand it. I'm wondering though why OCB is better than GCM. Is it because OCB is a bit less fragile than GCM against repeated nonces? Are there other reasons? Is there a hard reason not to use GCM?

[ThomasWaldmann]

I am roughly aware about LibreSSL and other non-OpenSSL efforts (and had my own issues when working with OpenSSL).
But we need something that supports the borg legacy (== fast AES256-CTR / HMAC-SHA256) as well as a better future (AEAD ciphers: OCB and CHPO).

The OCB patents issue is long resolved for FOSS, see: https://en.wikipedia.org/wiki/OCB_mode and as a recent update also this: https://mailarchive.ietf.org/arch/msg/cfrg/qLTveWOdTJcLn4HP3ev-vrj05Vg/

I can not give you the details about the GCM concerns, but when searching for usable AEAD ciphers I got the impression that quite some cryptographers feel bad about GCM. Considering that OCB is also faster made it clear for me to choose OCB. Years ago, there was the availability issue, but this is also resolved for most platforms since openssl 1.1.0 (and all openssl versions without AES-OCB not receiving security support any more).

So, if openssl is working on openbsd, maybe we should switch to that for borg 1.3+?

Alternatively, maybe some openbsd python/cython developer who is into crypto stuff could work on borg.crypto.low_level for the alternative code needed for LibreSSL? As long as they don't have OCB, one can't read such repos on LibreSSL though.

[ThomasWaldmann]

good news: i have a vagrant based openbsd-current (future 7.1) test box now. all master tests passing currently.

[ThomasWaldmann]

Update: The AES-OCB and chacha20-poly1305 based crypto is now in master branch (was #6463).

[ThomasWaldmann]

I put this into 1.3.0b1 milestone (first beta release).

@bket When the alpha releases get successful testing on openbsd with openssl, I guess we can close this because we'll just use openssl then. In case libressl catches up later regarding API and available algorithms, we can reopen this.

[ThomasWaldmann]

@bket could you help fixing the Vagrantfile in master branch, so it works based on openssl 1.1.1?

Currently it tries building on libressl, which will fail the tests.

[bket]

@ThomasWaldmann, the Vagrantfile needs a small addition:

diff --git Vagrantfile Vagrantfile
index eea43d8c..359ae6d6 100644
--- Vagrantfile
+++ Vagrantfile
@@ -78,6 +78,7 @@ def packages_openbsd
     pkg_add lz4
     pkg_add zstd
     pkg_add git  # no fakeroot
+    pkg_add openssl%1.1
     pkg_add py3-pip
     pkg_add py3-virtualenv
   EOF

Unfortunately this is not enough as we need to explicity tell the build to link against libcrypto from the installed package. I have the diff below sitting in my tree, which passes some flags. Setting BORG_OPENSSL_PREFIX is not going to cut it because of some renaming. With this diff pkgconfig does the right thing, and everything builds/tests OK. Still thinking about an alternative solution.

Index: setup.py
--- setup.py.orig
+++ setup.py
@@ -161,12 +161,13 @@ if not on_rtd:
             pc, 'BORG_OPENSSL_PREFIX', 'libcrypto', 'libcrypto', '>=1.1.1', lib_subdir='')
     else:
         crypto_ext_lib = lib_ext_kwargs(
-            pc, 'BORG_OPENSSL_PREFIX', 'crypto', 'libcrypto', '>=1.1.1')
+            pc, 'BORG_OPENSSL_PREFIX', 'crypto', 'libecrypto11', '>=1.1.1')
 
     crypto_ext_kwargs = members_appended(
         dict(sources=[crypto_ll_source, crypto_helpers]),
         crypto_ext_lib,
         dict(extra_compile_args=cflags),
+        dict(extra_link_args=['-Wl,-rpath,/usr/local/lib/eopenssl11']),
     )
 
     compress_ext_kwargs = members_appended(

[ThomasWaldmann]

thanks for the info!

pkg_add: ah, it needs a percent separator. I tried - and @ and ..., but didn't find that.

the renaming also explains why i did not even manually find the right crypto.pc file...

setup.py: if there is no easier way, guess we need to add a platform check there and just switch these values based on the platform.