Re: sec_powm.c:293: GNU MP assertion failed: enb >= windowsize

Jeffrey Walton noloader@gmail.com writes:

I'm trying to build Nettle 3.4.1 on Fedora 29, x64 (fully patched). Self tests are failing at:

PASS: rsa-sec-decrypt
sec_powm.c:293: GNU MP assertion failed: enb >= windowsize
../run-tests: line 57: 22997 Aborted (core dumped) "$1" $testflags
FAIL: rsa-compute-root
PASS: dsa
PASS: dsa-keygen

This is an ASSERT_ALWAYS in gmp's mpn/generic/sec_powm.c, checking the return value from the win_size function. To debug, it would be helpful to know

1. The value of enb and windowsize at the crash.

2. The definition of POWM_SEC_TABLE (defined in the platform specific gmp-mparam.h), and used by the win_size function in this file.

To me, it looks like the assert validates that POWM_SEC_TABLE confoems to expectations.

It might also be a bug in the book-keeping of scratch space, leading to some memory overwrite. Try running the test program under valgrind and/or asan?

Regards, /Niels