Nikos Mavrogiannopoulos n.mavrogiannopoulos@gmail.com writes:
A fix that could suit the master branch is attached. That adds the possibility to return an error if the maximum sizes are exceeded.
For the functions involving bignums, I wonder if it's better to allocate the needed storage, either with malloc or with gmp allocation functions. gmp is going to be allocating things anyway, so there's little point in avoiding malloc (which is a different situation than, e.g, ctr.c). And it's not really the right place for arbitrary limits.
2.7.x though this can be handled by an abort which is less than ideal, as it looks trivial to exploit for DoS.
I'm not too concerned about DoS here. An application receiving an RSA key from an untrusted source should impose some reasonable limit on keysize before it is used. If I send you a public RSA key with n and e of 100000 bits, and some signature for you to verify, and you don't impose any limit on key size, I'll hog your cpu for quite a while.
And the application really has to choose the limit, that's not Nettle's job.
That said, Nettle shouldn't do unbounded stack allocations in this case, it ought to use malloc, or abort or fail in some other *reliable* fashion. (I think having some documented limit on keysize would be acceptable, but I'm leaning towards saying that it's better to just use heap allcoation).
Do you agree?
Even worse, nettle_mpz_set_str_256*() does not enforce the maximum limits on big numbers and doesn't even have the ability to return an error.
I'm not entirely sure what problem you see here.
Regards, /Niels