Ted Zlatanov tzz@lifelogs.com writes:
;; valid signature, valid key ; doesn't work; generated with "openssl rsautl -sign -in /tmp/test -inkey /tmp/privkey.pem" but contains other baggage (should (nettle-rsa-verify "Test the Nettle RSA signature" (decode-hex-string "abf710d920de0a210167e62995d5cb06fb0ff6a3f81e2f1965dd3f4716883ab61b7dec40d1ebde89b0657473a434d0333177f183f71a9f4b84a49781b1e4bc440e042f2eb4441000ba07168cdb190c5aebba8c433420f6fc28b6997cbfee061170210bfa65294199e6d6c8c5e1a16421942371f6115d77263b859a75645b6b70d56f14ad378c8499318ff05eda9d24a61d854a3d7f6b67b037abb8d25e4b11ca3e42bdb823cfac34c70057ecd55cbb8449346c0824b46f6c668d14f1744bad7d05470953981df32fde24d2a1f27e58bf9e7d99b20b39b25844c53945dcbbd8b406e78bc0b8aee48c0ec8a26e70301eeeb12ba733e0baf7b82c8e25ac3ee89291") key-bitstring "sha1")))
#+end_src
Take a look at the second test above; this is the signed text but it contains baggage (including the original data).
Are you sure it includes the data? If the hex string is interpreted as a raw bignum, it's 2048 bits, which sounds plausible. I guess "Test the Nettle RSA signature" is the mesage being signed? Maybe double check that you didn't get a trailing newline or something when generating the openssl signture.
I'm not very familiar with the openssl tools. What's the correct command line to let openssl verify the signature using the given public key?
Regards, /Niels