Re: sec_powm.c:293: GNU MP assertion failed: enb >= windowsize

On Sun, Dec 30, 2018 at 5:27 PM Niels Möller nisse@lysator.liu.se wrote:

Jeffrey Walton noloader@gmail.com writes:

...

I'm trying to build Nettle 3.4.1 on Fedora 29, x64 (fully patched). Self tests are failing at:

PASS: rsa-sec-decrypt
sec_powm.c:293: GNU MP assertion failed: enb >= windowsize
../run-tests: line 57: 22997 Aborted (core dumped) "$1" $testflags
FAIL: rsa-compute-root
PASS: dsa
PASS: dsa-keygen

This is an ASSERT_ALWAYS in gmp's mpn/generic/sec_powm.c, checking the return value from the win_size function. To debug, it would be helpful to know

Here are the options I used to build GMP 6.1.2 and Nettle 3.1.4. It looks fairly easy to duplicate on 64-bit platforms.

BUILD_BITS: 64-bits INSTX_PREFIX: /usr/local INSTX_LIBDIR: /usr/local/lib64

PKGCONFPATH: /usr/local/lib64/pkgconfig CPPFLAGS: -I/usr/local/include -DNDEBUG CFLAGS: -g2 -O2 -march=native -fPIC CXXFLAGS: -g2 -O2 -march=native -fPIC LDFLAGS: -L/usr/local/lib64 -Wl,-R,/usr/local/lib64 -Wl,--enable-new-dtags LDLIBS: -ldl -lpthread

Then, I run configure. There's nothing special about what I am doing. I'm just getting into a Release Build state.

1. The value of enb and windowsize at the crash.

2. The definition of POWM_SEC_TABLE (defined in the platform specific gmp-mparam.h), and used by the win_size function in this file.

To me, it looks like the assert validates that POWM_SEC_TABLE conforms to expectations.

It might also be a bug in the book-keeping of scratch space, leading to some memory overwrite. Try running the test program under valgrind and/or asan?

Thanks Niels.

I'll try to get to it later tonight or tomorrow.

In the meantime you might provide your test data to GMP. It looks like it would make a good test case for them.

Jeff