Hi, I've received a bug report (from someone at Opera) about a crash when using Crypto.AES.CCM. Example
int main() { mixed state1 = Crypto.AES.CCM.State();
state1->set_encrypt_key(String.hex2string("bedcfb5a011ebc84600fcb296c15af0d")); state1->set_iv(String.hex2string("438a547a94ea88dce46c6c85")); state1->update(String.hex2string(""));
string ct = state1->crypt(String.hex2string("")); state1->digest(); return 0; }
When I try it (with Pike v8.0, installed as a debian package on x86_64) it segfaults in nettle_memxor3. As far as I can tell from 5 minutes of printf debugging, crash is inside the call to ->digest().
The set_iv method looks a bit suspicious to me, it doesn't include the required arguments for the nettle function ccm_set_nonce (and ccm is generally a bit messier to setup than most other modes). See http://www.lysator.liu.se/~nisse/nettle/nettle.html#index-ccm_005fset_005fno...
There may of course be some problem in the nettle library too, in particular, I would be happier if api misuse resulted in an assert failure rather than a segfault.