Hanno Böck hanno@hboeck.de writes:
It's a multiplication of these coordinates 23000000000000000000000000000000000000000000000000110011C2DD0000000000000000000 46BE3FEF75FCA4BD52CE28EC3F1483A05EE154965B05282F9029E14277409908C0EBAAD2CA5449FFA61FEC78473816BC with this scalar 23000000000000C1DD3FF800E83E2CACA1010A21
I've tracked this down to a miscomputation in the x86_64 assembly implementation of ecc_384_mod. If I add a testcase for the problematic value, the failure looks like
m->mod p failed: bit_size = 384 a = 4c9000000000000000000000000000000000000000000000004a604db486e000000000000000000000000000000000000000121025be29575adb2c8ffffffffffffffffffffffffffffffffffffffffffffffffffffffff t = fffffffffb37000004c90121025be29575b258cddb4d1404e0116e00098d59fb29853804d67f6e000000000004c8ffff (bad) ref = fffffffffb37000004c90121025be29575b258cddb4d1404e0116e00098d59fa29853803d67f6e000000000104c8fffe Aborted (core dumped) FAIL: ecc-mod
So most likely an unlikely carry which is mishandled. I'll dig further.
I don't know what tool you use to search for these problems, but if you are able to run it in additional configurations, the following x86_64 nettle configurations would be helpful (assuming 64-bit you're doing now is the compiler's default).
./configure ./configure --disable-assembler ./configure CC='gcc -m32' CXX='g++ -m32'
(If you reuse the nettle build tree, remember make distclean before you reconfigure).
Also testing on (32-bit) ARM would be helpful, to exercise the corresponding ARM assembly code.
Regards, /Niels