It seems that in the GCM macros nettle uses casts to functions. That is
pretty dangerous with the changes of parameters in functions in nettle
3. The issue is the compiler will not warn for serious errors such as
different function type. An example macro is GCM_ENCRYPT.
#define GCM_ENCRYPT(ctx, encrypt, length, dst, src) \
(0 ? (encrypt)(&(ctx)->cipher, 0, (void *)0, (void *)0) \
: gcm_encrypt(&(ctx)->gcm, &(ctx)->key, &(ctx)->cipher, \
(nettle_cipher_func *) (encrypt), \
(length), (dst), (src)))
I don't think that nettle should be casting functions. The issue seems
to be present in ctr, eax and cbc modes as well.
regards,
Nikos