nisse@lysator.liu.se (Niels Möller) writes:
On second look, it can't be rsa_compute_root, since that function has no return value. Is it sufficient for gnutls to do this check in rsa_compute_root_tr instead?
I also note that a check is needed in dsa_sign, which otherwise would crash if the group is invalid, with an even p.
I've comitted additional checks to dsa_sign and rsa_compute_root_tr.
I have one remaining question: Should there be additional sanity checks in the rsa_*_prepare functions, to reject keys with negative or out-of-range parameters? Out-of-range parameters will not, as far as I am aware, result in any crashes, only in bogus outputs.
Regards, /Niels