Hi all,
Here are a couple of small patches for nettle-benchmark: - removes the deprecated OpenSSL hash API - adds more OpenSSL sha2 hashes into the mix
Kindly merge or let me know of your concerns :-)
Related question:
As you can see from the second patch, nettle performance is a little low wrt OpenSSL - ~55% for sha1, and ~65% for sha2.
Is that normal, or there is something off with my system/build?
I'm using the default "--enable-assembler" and "-O2" as seen in the configure.ac, plus my processor lacks the SHA_NI ISA.
Thanks Emil P.S. More misc patches coming shortly, so stay tuned :-P
Emil Velikov (2): examples: don't use deprecated OpenSSL hashing API external: add more openssl sha2 digests to the benchmark
examples/nettle-benchmark.c | 6 +- examples/nettle-openssl.c | 112 ++++++++++++++---------------------- nettle-internal.h | 3 + 3 files changed, 50 insertions(+), 71 deletions(-)
The direct $HASH_{Init,Update,Final} has been discouraged for a while. With the upcoming OpenSSL 3.0 it will be officially deprecated.
Add a handy macro, to avoid repetition and mistakes like in the current code. Namely - we're using SHA cblock/digest_len for md5 :-\
The macro will also make it easier to add more, as seen with next patch.
Signed-off-by: Emil Velikov emil.l.velikov@gmail.com --- examples/nettle-openssl.c | 109 ++++++++++++++------------------------ 1 file changed, 40 insertions(+), 69 deletions(-)
diff --git a/examples/nettle-openssl.c b/examples/nettle-openssl.c index bb2e6627..19da06fd 100644 --- a/examples/nettle-openssl.c +++ b/examples/nettle-openssl.c @@ -383,76 +383,47 @@ nettle_openssl_cast128 = {
/* Hash functions */
-/* md5 */ -static nettle_hash_init_func openssl_md5_init; -static void -openssl_md5_init(void *ctx) -{ - MD5_Init(ctx); -} - -static nettle_hash_update_func openssl_md5_update; -static void -openssl_md5_update(void *ctx, - size_t length, - const uint8_t *src) -{ - MD5_Update(ctx, src, length); -} - -static nettle_hash_digest_func openssl_md5_digest; -static void -openssl_md5_digest(void *ctx, - size_t length, uint8_t *dst) -{ - assert(length == SHA_DIGEST_LENGTH); - MD5_Final(dst, ctx); - MD5_Init(ctx); -} - -const struct nettle_hash -nettle_openssl_md5 = { - "openssl md5", sizeof(SHA_CTX), - SHA_DIGEST_LENGTH, SHA_CBLOCK, - openssl_md5_init, - openssl_md5_update, - openssl_md5_digest +#define OPENSSL_HASH(NAME, name) \ +static nettle_hash_init_func openssl_##name##_init; \ +static void \ +openssl_##name##_init(void *ctx) \ +{ \ + if ((*(EVP_MD_CTX **)ctx = EVP_MD_CTX_new()) == NULL) \ + return; \ + \ + EVP_DigestInit(*(EVP_MD_CTX **)ctx, EVP_##name()); \ +} \ + \ +static nettle_hash_update_func openssl_##name##_update; \ +static void \ +openssl_##name##_update(void *ctx, \ + size_t length, \ + const uint8_t *src) \ +{ \ + EVP_DigestUpdate(*(EVP_MD_CTX **)ctx, src, length); \ +} \ + \ +static nettle_hash_digest_func openssl_##name##_digest; \ +static void \ +openssl_##name##_digest(void *ctx, \ + size_t length, uint8_t *dst) \ +{ \ + assert(length == NAME##_DIGEST_LENGTH); \ + \ + EVP_DigestFinal(*(EVP_MD_CTX **)ctx, dst, NULL); \ + EVP_DigestInit(*(EVP_MD_CTX **)ctx, EVP_##name()); \ +} \ + \ +const struct nettle_hash \ +nettle_openssl_##name = { \ + "openssl " #name, sizeof(NAME##_CTX), \ + NAME##_DIGEST_LENGTH, NAME##_CBLOCK, \ + openssl_##name##_init, \ + openssl_##name##_update, \ + openssl_##name##_digest \ };
-/* sha1 */ -static nettle_hash_init_func openssl_sha1_init; -static void -openssl_sha1_init(void *ctx) -{ - SHA1_Init(ctx); -} - -static nettle_hash_update_func openssl_sha1_update; -static void -openssl_sha1_update(void *ctx, - size_t length, - const uint8_t *src) -{ - SHA1_Update(ctx, src, length); -} - -static nettle_hash_digest_func openssl_sha1_digest; -static void -openssl_sha1_digest(void *ctx, - size_t length, uint8_t *dst) -{ - assert(length == SHA_DIGEST_LENGTH); - SHA1_Final(dst, ctx); - SHA1_Init(ctx); -} +OPENSSL_HASH(MD5, md5) +OPENSSL_HASH(SHA, sha1)
-const struct nettle_hash -nettle_openssl_sha1 = { - "openssl sha1", sizeof(SHA_CTX), - SHA_DIGEST_LENGTH, SHA_CBLOCK, - openssl_sha1_init, - openssl_sha1_update, - openssl_sha1_digest -}; - #endif /* WITH_OPENSSL */
Emil Velikov emil.l.velikov@gmail.com writes:
The direct $HASH_{Init,Update,Final} has been discouraged for a while. With the upcoming OpenSSL 3.0 it will be officially deprecated.
Add a handy macro, to avoid repetition and mistakes like in the current code. Namely - we're using SHA cblock/digest_len for md5 :-\
Thanks. Reducing code duplication with the macro is nice, but I think the handling of the context is a bit confusing. In the definiton of the nettle_hash structs, you use sizeof (NAME##_CTX), but in the functions you cast like (EVP_MD_CTX **)ctx. I think it will de clearer with something like
struct openssl_hash_ctx { EVP_MD_CTX *evp; };
analogous to struct openssl_cipher_ctx, and use that for all the hash functions. And it looks like the update function can be the same for all hashes, no need to have the macro define it.
Regards, /Niels
On Thu, 7 May 2020 at 07:51, Niels Möller nisse@lysator.liu.se wrote:
Emil Velikov emil.l.velikov@gmail.com writes:
The direct $HASH_{Init,Update,Final} has been discouraged for a while. With the upcoming OpenSSL 3.0 it will be officially deprecated.
Add a handy macro, to avoid repetition and mistakes like in the current code. Namely - we're using SHA cblock/digest_len for md5 :-\
Thanks. Reducing code duplication with the macro is nice, but I think the handling of the context is a bit confusing. In the definiton of the nettle_hash structs, you use sizeof (NAME##_CTX), but in the functions you cast like (EVP_MD_CTX **)ctx. I think it will de clearer with something like
struct openssl_hash_ctx { EVP_MD_CTX *evp; };
analogous to struct openssl_cipher_ctx, and use that for all the hash functions. And it looks like the update function can be the same for all hashes, no need to have the macro define it.
Thanks for the suggestions Niels. Will try to update the series in a couple of days.
-Emil
In particular, this commit adds sha256, sha384 and sha512.
Rough numbers from my system:
Algorithm mode Mbyte/s sha1 update 462.05 openssl sha1 update 804.90 sha256 update 222.32 openssl sha256 update 369.49 sha384 update 355.34 openssl sha384 update 536.05 sha512 update 355.90 openssl sha512 update 546.07
Signed-off-by: Emil Velikov emil.l.velikov@gmail.com --- examples/nettle-benchmark.c | 6 ++++-- examples/nettle-openssl.c | 3 +++ nettle-internal.h | 3 +++ 3 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/examples/nettle-benchmark.c b/examples/nettle-benchmark.c index 5d0e649e..b6e550cf 100644 --- a/examples/nettle-benchmark.c +++ b/examples/nettle-benchmark.c @@ -912,8 +912,10 @@ main(int argc, char **argv) &nettle_md2, &nettle_md4, &nettle_md5, OPENSSL(&nettle_openssl_md5) &nettle_sha1, OPENSSL(&nettle_openssl_sha1) - &nettle_sha224, &nettle_sha256, - &nettle_sha384, &nettle_sha512, + &nettle_sha224, + &nettle_sha256, OPENSSL(&nettle_openssl_sha256) + &nettle_sha384, OPENSSL(&nettle_openssl_sha384) + &nettle_sha512, OPENSSL(&nettle_openssl_sha512) &nettle_sha512_224, &nettle_sha512_256, &nettle_sha3_224, &nettle_sha3_256, &nettle_sha3_384, &nettle_sha3_512, diff --git a/examples/nettle-openssl.c b/examples/nettle-openssl.c index 19da06fd..f92d77a4 100644 --- a/examples/nettle-openssl.c +++ b/examples/nettle-openssl.c @@ -425,5 +425,8 @@ nettle_openssl_##name = { \
OPENSSL_HASH(MD5, md5) OPENSSL_HASH(SHA, sha1) +OPENSSL_HASH(SHA256, sha256) +OPENSSL_HASH(SHA512, sha384) // NOTE: SHA512 here is not a typo +OPENSSL_HASH(SHA512, sha512)
#endif /* WITH_OPENSSL */ diff --git a/nettle-internal.h b/nettle-internal.h index dc379f1f..f1a27f49 100644 --- a/nettle-internal.h +++ b/nettle-internal.h @@ -114,6 +114,9 @@ extern const struct nettle_aead nettle_openssl_arcfour128;
extern const struct nettle_hash nettle_openssl_md5; extern const struct nettle_hash nettle_openssl_sha1; +extern const struct nettle_hash nettle_openssl_sha256; +extern const struct nettle_hash nettle_openssl_sha384; +extern const struct nettle_hash nettle_openssl_sha512;
extern const struct nettle_hash * const _nettle_hashes[];
Emil Velikov emil.l.velikov@gmail.com writes:
As you can see from the second patch, nettle performance is a little low wrt OpenSSL - ~55% for sha1, and ~65% for sha2.
I think Nettle's assembly code for sha1 is quite old, and hasn'd been tuned for current processors with lots of parallelism.
If you analyze the data dependencies of sha1 carefully (I haven't looked into that for quite a while, though), I think the critical dependency path requires only 2-3 cycles per round, if instruction issue and execution can keep up with doing all the instructions not on the critical path in parallel.
If you look at round function,
C e += a <<< 5 + f( b, c, d ) + k + w; C b <<<= 30
what this means in practice is that we ought to identify the one of the a, b, c, d inputs which was updated in the previous round, and arrange the computation of the round function to use that input last, to minimize the chain of depending instructions from using that input until the new e is ready, since that will be the critical path of the round.
If you want to play with that, I would start experimenting with the C implementation and see what improvements can be made there, and then use a similar organization for the the assembly implementation(s).
Regards, /Niels
nettle-bugs@lists.lysator.liu.se
-
Emil Velikov -
nisse＠lysator.liu.se