Hanno Böck hanno@hboeck.de writes:
If the modulus is even / divisible by 2 then mpz_powm_sec() will crash with a floating point exception. Quite frankly, I think this is bad from GMPs side as well. At the very least it shouldn't crash for invalid inputs, but return an error.
I think the GMP design is to leave any easy sanity checks to the application. While for more complex conditions, like modular inversion (which fails if inputs have any non-trivial common factor), gmp checks for this and returns an error indication.
(Ideally it would just work for all inputs that mathematically make sense). But nettle has to handle things with gmp as they are.
I think it should be easy and reasonable to add code to rsa_public_key_prepare and rsa_private_key_prepare to check that the modulo is odd. What do you think? It's reasonable to have those functions do enough key validation to be able to handle the key without crashing.
Attached is a certificate + key where I manually changed the modulus to be even (P.S.: This tool [2] fis very useful for such cases). The certificate is therefore obviously bogus, but that doesn't matter in our case.
If you can transform this into a nettle testcase, that would be nice. I think it would fit best either with the high-level tests examples/rsa-sign-test and examples/rsa-verify-test, or as unit tests in testsuite/rsa-test.c.
(P.S.: The autoconf gmp detection part of this patch breaks on my system and nettle gets built without gmp,
With the patch as posted to the list, or using the master branch of the repo? I'm aware of configure bugs in the former but not the latter.
Thanks for the review, /Niels