I'm working towards adding ECDH and ECDSA support to gnutls. I've currently ported libtomcrypt to use gmp directly and nettle's types. The code is under development, and can be found at: http://git.savannah.gnu.org/gitweb/?p=gnutls.git;a=tree;f=lib/nettle;h=23d9e...
The ECC functionality is confined to the ecc* files. It would be nice if nettle included it or something similar.
btw. Is there any plan on an 2.2 release?
regards, Nikos
Nikos Mavrogiannopoulos nmav@gnutls.org writes:
I'm working towards adding ECDH and ECDSA support to gnutls.
I don't have time at the moment to read the code. Could you summarize which curves/algorithms/specifications you're implementing?
The ECC functionality is confined to the ecc* files. It would be nice if nettle included it or something similar.
Contributions are welcome ;-) We'll see if I get any time to look into it myself.
btw. Is there any plan on an 2.2 release?
I guess I should put up a release TODO list on the web. Issues I'm aware of and which I think should be completed before the next release are:
* Fix the libdir defaults for linux.
* Port libgrypt's LGPL:ed serpent implementation to nettle.
* Overall license change to LGPL. Still not decided if it should be LGPLv2+, LGPLv3+, or dual license GPLv2+ and LGPLv3+ (in this notation, "+" means "or any later version, of the user's choice").
If it's going to take a long time to sort these things out, I could make a relase sooner and postpone some or all of the above to a later release. Main new feature since the previous release is the gcm support, I think.
Regards, /Niels
On 05/21/2011 01:04 PM, Niels Möller wrote:
I'm working towards adding ECDH and ECDSA support to gnutls.
I don't have time at the moment to read the code. Could you summarize which curves/algorithms/specifications you're implementing?
For gnutls I care only about the SECG curves (SECP224R1, SECP256R1, SECP384R1, SECP521R1) and with those I have tested the ECDH support. The code was fixed for the curves y^2=x^3-3x+b. I tried to make it apply to generic y^2=x^3+ax+b, but I haven't test that functionality (nor plan to) because all of the above curves are of the a=-3 format.
libtomcrypt's code style is similar to nettle's thus the changes required to port this to nettle would not be much, but always depends on what you want to include there.
- Port libgrypt's LGPL:ed serpent implementation to nettle. * Overall
license change to LGPL. Still not decided if it should be LGPLv2+, LGPLv3+, or dual license GPLv2+ and LGPLv3+ (in this notation, "+" means "or any later version, of the user's choice").
May I suggest an intermediate release with GCM under LGPLv2+? That would ensure that current applications using gnutls can use that code.
regards, Nikos
Nikos Mavrogiannopoulos nmav@gnutls.org writes:
May I suggest an intermediate release with GCM under LGPLv2+? That would ensure that current applications using gnutls can use that code.
An all-LGPL release is not possible until the serpent code is replaced, so any releases before that would use the current licensing terms (GPL as a whole, certain parts under more permissive license).
So I'm not sure exactly what you are suggesting.
Regards, /Niels
On 05/30/2011 10:36 AM, Niels Möller wrote:
Nikos Mavrogiannopoulos nmav@gnutls.org writes:
May I suggest an intermediate release with GCM under LGPLv2+? That would ensure that current applications using gnutls can use that code.
An all-LGPL release is not possible until the serpent code is replaced, so any releases before that would use the current licensing terms (GPL as a whole, certain parts under more permissive license). So I'm not sure exactly what you are suggesting.
I thought the GPL part applies only to the parts that are GPL and not to the whole library. In any case would you suggest gnutls using nettle a la lsh, i.e. include it as a static library? That way we can eliminate licensing issues by removing the GPL parts.
regards, Nikos
Nikos Mavrogiannopoulos nmav@gnutls.org writes:
I thought the GPL part applies only to the parts that are GPL and not to the whole library.
The manual says the following:
Nettle is distributed under the GNU General Public License (GPL) (see the file COPYING for details). However, most of the individual files are dual licensed under less restrictive licenses like the GNU Lesser General Public License (LGPL), or are in the public domain. This means that if you don't use the parts of nettle that are GPL-only, you have the option to use the Nettle library just as if it were licensed under the LGPL. To find the current status of particular files, you have to read the copyright notices at the top of the files.
Things will get simpler once the serpent code is replaced.
In any case would you suggest gnutls using nettle a la lsh, i.e. include it as a static library? That way we can eliminate licensing issues by removing the GPL parts.
My understanding is that as long as gnutls doesn't use the serpent code (the only remaining GPL-only feature) and has no references to it, it's ok no matter if you use static or dynamic linking. You can pretend that the serpent code was never built and included into the library.
Regards, /Niels
I looked at reviving my serpent port from libgcrypt. As a first step in testing it, it would help if Nettle had the same test vector as libgcrypt. Please consider this patch.
/Simon
2011-05-30 Simon Josefsson simon@josefsson.org
* testsuite/serpent-test.c (test_main): Add test vectors from libgcrypt.
--- serpent-test.c.~1.3.~ 2011-05-20 23:02:54.000000000 +0200 +++ serpent-test.c 2011-05-30 22:46:50.837183923 +0200 @@ -25,6 +25,26 @@ int test_main(void) { + /* From libgcrypt */ + test_cipher(&nettle_serpent128, + HL("0000000000000000 0000000000000000"), + HL("D29D576FCEA3A3A7 ED9099F29273D78E"), + H("B2288B968AE8B086 48D1CE9606FD992D")); + test_cipher(&nettle_serpent192, + HL("0000000000000000 0000000000000000 0000000000000000"), + HL("D29D576FCEABA3A7 ED9899F2927BD78E"), + H("130E353E1037C224 05E8FAEFB2C3C3E9")); + test_cipher(&nettle_serpent256, + HL("0000000000000000 0000000000000000" + "0000000000000000 0000000000000000"), + HL("D095576FCEA3E3A7 ED98D9F29073D78E"), + H("B90EE5862DE69168 F2BDD5125B45472B")); + test_cipher(&nettle_serpent256, + HL("0000000000000000 0000000000000000" + "0000000000000000 0000000000000000"), + HL("0000000001000000 0200000003000000"), + H("2061A42782BD52EC 691EC383B03BA77C")); + /* The first test for each key size from the ecb_vk.txt and ecb_vt.txt * files in the serpent package. */
Simon Josefsson simon@josefsson.org writes:
I looked at reviving my serpent port from libgcrypt. As a first step in testing it, it would help if Nettle had the same test vector as libgcrypt. Please consider this patch.
Thanks, checked in now. And the current (i.e., to-be-replaced) code passes these tests.
/Niels
Fixing a small bug in my 2010-12-07 port of serpent.c from libgcrypt to nettle was all that was required to make it work. Please consider this work, applied as follows:
* Add ChangeLog entry as below. * Remove serpent_sboxes.h. * Apply patch to Makefile.in as below. * Add attached serpent.c in place of the current code.
I have not compared the new file to the old one in Nettle, I'll leave that to you.
/Simon
2011-05-30 Simon Josefsson simon@josefsson.org
* serpent.c: Rewrite, based on libgcrypt code. License changed from GPL to LGPL. * serpent_sboxes.h: Removed. * Makefile.in: Drop serpent_sboxes.h.
Index: Makefile.in =================================================================== RCS file: /cvsroot/lsh/lsh/nettle/Makefile.in,v retrieving revision 1.32 diff -u -p -r1.32 Makefile.in --- Makefile.in 23 Mar 2011 10:34:46 -0000 1.32 +++ Makefile.in 30 May 2011 21:07:01 -0000 @@ -128,7 +128,7 @@ DISTFILES = $(SOURCES) $(HEADERS) .boots README AUTHORS COPYING COPYING.LIB INSTALL NEWS TODO ChangeLog \ memxor.c $(des_headers) descore.README \ aes-internal.h camellia-internal.h cast128_sboxes.h desinfo.h desCode.h \ - serpent_sboxes.h nettle-internal.h nettle-write.h prime-list.h \ + nettle-internal.h nettle-write.h prime-list.h \ asm.m4 \ nettle.texinfo nettle.info nettle.html nettle.pdf sha-example.c
Btw, I have tested it on a core i7 (little endien) and a PPC machine (big endian) and it worked.
(However, the 'gcm' self test crashed on PPC.)
/Simon
Simon Josefsson simon@josefsson.org writes:
(However, the 'gcm' self test crashed on PPC.)
Here is a backtrace.
espresso:~/nettle-2.2/testsuite jas$ gdb ./gcm-test GNU gdb 6.3.50-20050815 (Apple version gdb-696) (Sat Oct 20 18:20:28 GMT 2007) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "powerpc-apple-darwin"... warning: --arch option not supported in this gdb. Reading symbols for shared libraries .. done
(gdb) b main Breakpoint 1 at 0x3ddc: file testutils.c, line 149. (gdb) r Starting program: /Users/jas/nettle-2.2/testsuite/gcm-test Reading symbols for shared libraries . done
Breakpoint 1, main (argc=1, argv=0xbffffb34) at testutils.c:149 149 if (argc > 1) (gdb) c Continuing.
Program received signal EXC_BAD_ACCESS, Could not access memory. Reason: KERN_PROTECTION_FAILURE at address: 0x00000000 0x00000000 in ?? () (gdb) bt #0 0x00000000 in ?? () Cannot access memory at address 0x0 Cannot access memory at address 0x0 Cannot access memory at address 0x0 Cannot access memory at address 0x0 Cannot access memory at address 0x0 #1 0x000049f0 in test_aead (aead=Cannot access memory at address 0x0 Cannot access memory at address 0x0 Cannot access memory at address 0x0 Cannot access memory at address 0x0 Cannot access memory at address 0x0 Cannot access memory at address 0x0 0x80f0, key_length=16, key=0x3000f0 "", auth_length=0, authtext=0x300100 "", length=0, cleartext=0x300110 "", ciphertext=0x300120 "", iv_length=12, iv=0x300130 "", digest=0x300140 "X����~0a6\035W��EZ") at testutils.c:384 Cannot access memory at address 0x0 #2 0x000029ec in test_main () at gcm-test.c:14 Cannot access memory at address 0x0 Cannot access memory at address 0x0 Cannot access memory at address 0x0 #3 0x00003e30 in main (argc=1, argv=0xbffffb34) at testutils.c:161 (gdb)
What is fishy is that nettle_gcm_aes128 doesn't seem to have been initialized properly:
espresso:~/nettle-2.2/testsuite jas$ gdb ./gcm-test GNU gdb 6.3.50-20050815 (Apple version gdb-696) (Sat Oct 20 18:20:28 GMT 2007) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "powerpc-apple-darwin"... warning: --arch option not supported in this gdb. Reading symbols for shared libraries .. done
(gdb) p nettle_gcm_aes128 $1 = { name = 0x0, context_size = 0, block_size = 0, key_size = 0, set_key = 0, set_iv = 0, update = 0, encrypt = 0, decrypt = 0, digest = 0 } (gdb)
Even stranger is that it is defined properly in other tests:
espresso:~/nettle-2.2/testsuite jas$ gdb ./blowfish-test GNU gdb 6.3.50-20050815 (Apple version gdb-696) (Sat Oct 20 18:20:28 GMT 2007) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "powerpc-apple-darwin"... warning: --arch option not supported in this gdb. Reading symbols for shared libraries .. done
(gdb) p nettle_gcm_aes128 $1 = { name = 0xfabc "gcm-aes128", context_size = 4404, block_size = 16, key_size = 16, set_key = 0xc308 <nettle_gcm_aes_set_key>, set_iv = 0xc37c <nettle_gcm_aes_set_iv>, update = 0xc3d4 <nettle_gcm_aes_update>, encrypt = 0xc42c <nettle_gcm_aes_encrypt>, decrypt = 0xc4a8 <nettle_gcm_aes_decrypt>, digest = 0xc524 <nettle_gcm_aes_digest> } (gdb)
Possibly something could be broken on this machine.
/Simon
Simon Josefsson simon@josefsson.org writes:
What is fishy is that nettle_gcm_aes128 doesn't seem to have been initialized properly:
Did you solve this problem? To me it seems like it could be
* setup of the shared-library magic, for this particular object file, somehow broken.
* bugs in the macos port of gcc (I don't know if you use apples "xcode" or some other gcc port, and I don't really know the difference either). According to the gmp people, there have been *lots* of problems with apple's linker and maybe also with other related tools.
* some other random hardware or software problem.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
Simon Josefsson simon@josefsson.org writes:
What is fishy is that nettle_gcm_aes128 doesn't seem to have been initialized properly:
Did you solve this problem? To me it seems like it could be
setup of the shared-library magic, for this particular object file, somehow broken.
bugs in the macos port of gcc (I don't know if you use apples "xcode" or some other gcc port, and I don't really know the difference either). According to the gmp people, there have been *lots* of problems with apple's linker and maybe also with other related tools.
some other random hardware or software problem.
The problem still occurs, but I'd be inclined to write it off as a compiler bug because the system is running pretty old software (even if it is latest published from Apple for this hardware).
espresso:~ jas$ gcc --version powerpc-apple-darwin8-gcc-4.0.1 (GCC) 4.0.1 (Apple Computer, Inc. build 5370) Copyright (C) 2005 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
espresso:~ jas$
/Simon
nisse@lysator.liu.se (Niels Möller) writes:
Did you solve this problem? To me it seems like it could be
setup of the shared-library magic, for this particular object file, somehow broken.
bugs in the macos port of gcc (I don't know if you use apples "xcode" or some other gcc port, and I don't really know the difference either). According to the gmp people, there have been *lots* of problems with apple's linker and maybe also with other related tools.
some other random hardware or software problem.
Several people have reported a bug of the first type: in nettle-internal.h, the const structs nettle_gcm_* ought to be declared extern. I guess I will have to issue a bugfix release soon. I.e.,
-const struct nettle_aead nettle_gcm_aes128; +extern const struct nettle_aead nettle_gcm_aes128;
etc.
Fix checked in now. Simon: It would be nice if you could test if this also solves the ppc problem.
Regards, /Niels
Simon Josefsson simon@josefsson.org writes:
Fixing a small bug in my 2010-12-07 port of serpent.c from libgcrypt to nettle was all that was required to make it work. Please consider this work, applied as follows:
Great!
I'll try to get this integrated reasonably soon. Have you compared the performance of the old and new code?
Some minor things (which I think I can take care of myself):
/* Serpent works on 128 bit blocks. */ typedef uint32_t serpent_block_t[4];
/* Serpent key, provided by the user. If the original key is shorter than 256 bits, it is padded. */ typedef uint32_t serpent_key_t[8];
I dislike array typedefs.
#define byte_swap_32(x) \ (0 \ | (((x) & 0xff000000) >> 24) | (((x) & 0x00ff0000) >> 8) \ | (((x) & 0x0000ff00) << 8) | (((x) & 0x000000ff) << 24))
This and the endian test where it is used should be replaced by using LE_READ_UINT32.
/* Convert the user provided key KEY of KEY_LENGTH bytes into the internally used format. */ static void serpent_key_prepare (const uint8_t * key, unsigned int key_length, serpent_key_t key_prepared)
This function seems to assume that key is aligned on a four-byte boundary, and that key_length is a multiple of four. The nettle interface specifies no alignment requirement on the key. And the old serpent code is supposed to support any key size (although unfortunately I don't have any testcases for sizes other than 16, 24 and 32 bytes).
After this code is in, I'd like to try to do serpent with two blocks at a time in parallel, for machines with native 64-bit registers (and change at least the ctr code to do a couple of blocks at a time). I think that might be about as fast as aes or camellia.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
Simon Josefsson simon@josefsson.org writes:
Fixing a small bug in my 2010-12-07 port of serpent.c from libgcrypt to nettle was all that was required to make it work. Please consider this work, applied as follows:
Great!
I'll try to get this integrated reasonably soon. Have you compared the performance of the old and new code?
Old: jas@latte:~/src/lsh/nettle/examples$ ./nettle-benchmark serpent sha1_compress: 1139.20 cycles
benchmark call overhead: 0.002529 us Algorithm mode Mbyte/s
serpent256 ECB encrypt 28.69 serpent256 ECB decrypt 30.94 serpent256 CBC encrypt 26.99 serpent256 CBC decrypt 30.82 jas@latte:~/src/lsh/nettle/examples$
New: jas@latte:~/src/lsh/nettle/examples$ ./nettle-benchmark serpent sha1_compress: 1140.00 cycles
benchmark call overhead: 0.002425 us Algorithm mode Mbyte/s
serpent256 ECB encrypt 14.26 serpent256 ECB decrypt 16.34 serpent256 CBC encrypt 13.93 serpent256 CBC decrypt 15.93 jas@latte:~/src/lsh/nettle/examples$
Some minor things (which I think I can take care of myself):
/* Serpent works on 128 bit blocks. */ typedef uint32_t serpent_block_t[4];
/* Serpent key, provided by the user. If the original key is shorter than 256 bits, it is padded. */ typedef uint32_t serpent_key_t[8];
I dislike array typedefs.
#define byte_swap_32(x) \ (0 \ | (((x) & 0xff000000) >> 24) | (((x) & 0x00ff0000) >> 8) \ | (((x) & 0x0000ff00) << 8) | (((x) & 0x000000ff) << 24))
This and the endian test where it is used should be replaced by using LE_READ_UINT32.
Sounds good.
/* Convert the user provided key KEY of KEY_LENGTH bytes into the internally used format. */ static void serpent_key_prepare (const uint8_t * key, unsigned int key_length, serpent_key_t key_prepared)
This function seems to assume that key is aligned on a four-byte boundary, and that key_length is a multiple of four. The nettle interface specifies no alignment requirement on the key. And the old serpent code is supposed to support any key size (although unfortunately I don't have any testcases for sizes other than 16, 24 and 32 bytes).
Maybe throw an error for non-16/24/32 key sizes? I'm not sure how useful it is to support that.
After this code is in, I'd like to try to do serpent with two blocks at a time in parallel, for machines with native 64-bit registers (and change at least the ctr code to do a couple of blocks at a time). I think that might be about as fast as aes or camellia.
Did the old code do that? In any case, it looks like the performance of serpent has a long way to go to be comparable to aes or camellia:
jas@latte:~/src/lsh/nettle/examples$ ./nettle-benchmark camellia sha1_compress: 1139.20 cycles
benchmark call overhead: 0.002421 us Algorithm mode Mbyte/s
camellia128 ECB encrypt 146.18 camellia128 ECB decrypt 146.37 camellia128 CBC encrypt 127.87 camellia128 CBC decrypt 142.89
camellia192 ECB encrypt 110.03 camellia192 ECB decrypt 109.84 camellia192 CBC encrypt 98.50 camellia192 CBC decrypt 108.31
camellia256 ECB encrypt 109.72 camellia256 ECB decrypt 110.03 camellia256 CBC encrypt 98.46 camellia256 CBC decrypt 108.29 jas@latte:~/src/lsh/nettle/examples$ ./nettle-benchmark aes sha1_compress: 1149.60 cycles
benchmark call overhead: 0.002420 us Algorithm mode Mbyte/s
aes128 ECB encrypt 181.84 aes128 ECB decrypt 181.58 aes128 CBC encrypt 149.30 aes128 CBC decrypt 178.05
aes192 ECB encrypt 153.40 aes192 ECB decrypt 155.35 aes192 CBC encrypt 132.48 aes192 CBC decrypt 153.33
aes256 ECB encrypt 135.98 aes256 ECB decrypt 135.14 aes256 CBC encrypt 119.54 aes256 CBC decrypt 134.13
openssl aes128 ECB encrypt 203.50 openssl aes128 ECB decrypt 190.98
openssl aes192 ECB encrypt 171.97 openssl aes192 ECB decrypt 162.42
openssl aes256 ECB encrypt 149.75 openssl aes256 ECB decrypt 142.56 jas@latte:~/src/lsh/nettle/examples$
/Simon
Simon Josefsson simon@josefsson.org writes:
nisse@lysator.liu.se (Niels Möller) writes:
I'll try to get this integrated reasonably soon. Have you compared the performance of the old and new code?
Old: serpent256 ECB encrypt 28.69 serpent256 ECB decrypt 30.94 serpent256 CBC encrypt 26.99 serpent256 CBC decrypt 30.82
New: serpent256 ECB encrypt 14.26 serpent256 ECB decrypt 16.34 serpent256 CBC encrypt 13.93 serpent256 CBC decrypt 15.93
Then there's clearly some room for optimization.
Maybe throw an error for non-16/24/32 key sizes? I'm not sure how useful it is to support that.
Not terribly useful, I guess, but since it's well defined by the serpent spec, I think it should be supported.
After this code is in, I'd like to try to do serpent with two blocks at a time in parallel, for machines with native 64-bit registers (and change at least the ctr code to do a couple of blocks at a time). I think that might be about as fast as aes or camellia.
Did the old code do that?
No, I didn't want to do that work with the old code.
In any case, it looks like the performance of serpent has a long way to go to be comparable to aes or camellia:
On my Intel SU4100 laptop (64 bit), I get (still with the old serpent code):
aes256 ECB encrypt 54.72 aes256 ECB decrypt 54.36
camellia256 ECB encrypt 43.10 camellia256 ECB decrypt 43.09
serpent256 ECB encrypt 22.47 serpent256 ECB decrypt 26.89
I would expect that the two-block-in-parallel trick can almost double serpent performance (for ecb, ctr, cbc-decrypt, but not cbc-encrypt). And then all three algorithms are definitely in the same ballpark.
(And of these algorithms, only aes uses handwritten x86_64 assembly code).
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
Maybe throw an error for non-16/24/32 key sizes? I'm not sure how useful it is to support that.
Not terribly useful, I guess, but since it's well defined by the serpent spec, I think it should be supported.
Which spec? According to wikipedia
https://secure.wikimedia.org/wikipedia/en/wiki/Serpent_%28cipher%29
this is the official serpent homepage:
http://www.cl.cam.ac.uk/~rja14/serpent.html
and the link for "algorithm specification" is
http://www.cl.cam.ac.uk/~rja14/Papers/serpent.pdf
which says that
The user key length is variable, but for the purposes of this submission we fix it at 128, 192 or 256 bits
That is a bit vague, either it is variable or it isn't.
The Serpent-0 specification says
The user key can be of any length between 64 and 256 bits.[5]
[5] We define key lengths to be multiples of 32 bits for ease of implementation, and ...
Thus in any case, it seems, the key size is not completely variable and always a multiple of 32 bits (which resolves the key alignment issue).
My impression is that Serpent is a bit confusingly specified, and it is unclear which specification is actually the right one. The test vector confusion reinforced my impression here.
/Simon
Simon Josefsson simon@josefsson.org writes:
and the link for "algorithm specification" is
I think this is the authoritative spec, and same document which was included in the submission package (floppy5/serpent.pdf).
The user key length is variable, but for the purposes of this submission we fix it at 128, 192 or 256 bits
That is a bit vague, either it is variable or it isn't.
I agree, this is not as definitive as I remembered it. Anyway, it defines how short keys of any bitsize <= 256 bits should be padded, in case anybody ever wants to use it, and gives 40-bit keys as an example.
The Serpent-0 specification says
The user key can be of any length between 64 and 256 bits.[5]
[5] We define key lengths to be multiples of 32 bits for ease of implementation, and ...
I don't think this is relevant.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
I'll try to get this integrated reasonably soon. Have you compared the performance of the old and new code?
I've checked in the new code now, and I've fixed serpent_key_prepare to use LE_READ_UINT32 (with no alignment requirements) and with support for arbitrary key sizes up to 32 bytes.
This has been really straight forward.
On my development laptop, I get about the same performance for the new and the old code (x86_64, gcc-4.4.5).
I'd like to get rid of the serpent_block_t typedef (I think the use of an array may mess up register allcoation with some compilers, and that might be the source of the large performance difference on Simon's machine), and it's really silly to have a BLOCK_COPY in the round function.
So I think I'll want to spend some time on reorganizing the code (and there's also some purely stylistic things, like using uppercase for all macros) before adding anything really new, like the two-block trick.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
Nikos Mavrogiannopoulos nmav@gnutls.org writes:
May I suggest an intermediate release with GCM under LGPLv2+? That would ensure that current applications using gnutls can use that code.
An all-LGPL release is not possible until the serpent code is replaced, so any releases before that would use the current licensing terms (GPL as a whole, certain parts under more permissive license).
The serpent code is replaced, any other issues that we are waiting for?
A new Nettle release would be nice to make GnuTLS with Nettle easier to install. I guess anyone can prepare a snapshot, but an official release is better.
/Simon
Simon Josefsson simon@josefsson.org writes:
The serpent code is replaced, any other issues that we are waiting for?
* Fix the libdir defaults for linux x86_64 (I think the following should work in most cases for for both debian and fhs conventions: When building for $ABI (32 or 64), check if /usr/lib$ABI exists. If so, install there, otherwise in $prefix/lib. Possibly bail out with an error if neither /usr/lib32 nor /usr/lib64 exists).
* Decide which flavor of LGPL licensing should be used. Options: LGPLv2+, LGPLv3+ or dual license LGPLv3+ and GPLv2+.
* Update NEWS and documentation (regarding serpent and licensing).
Comments appreciated. Anything else?
In the mean time, I'm also hacking further on the serpent code. The double-block trick in C gave a modest improvement. I think it should work great with sse2 assembly to do four blocks at a time. Primarily for x86_64, but maybe also for plain x86 (but then maybe one can's use sse2 unconditionally). But that doesn't have to be completed before release.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
Simon Josefsson simon@josefsson.org writes:
The serpent code is replaced, any other issues that we are waiting for?
- Fix the libdir defaults for linux x86_64 (I think the following should work in most cases for for both debian and fhs conventions: When building for $ABI (32 or 64), check if /usr/lib$ABI exists. If so, install there, otherwise in $prefix/lib. Possibly bail out with an error if neither /usr/lib32 nor /usr/lib64 exists).
I don't understand why this is a Nettle problem, since I don't recall having to do anything like that for any of the libraries I maintain. Could it be because I use automake and libtool? If automake would solve this issue, how about using it in Nettle?
- Decide which flavor of LGPL licensing should be used. Options: LGPLv2+, LGPLv3+ or dual license LGPLv3+ and GPLv2+.
Is there a reason to change from the current LGPLv2+ before the next release?
- Update NEWS and documentation (regarding serpent and licensing).
I can help with that, if you want. I did a diff between 2.1 and the current code, and it is a fairly small set of changes.
Comments appreciated. Anything else?
Not that I can think of.
Using UTF-8 encoding for the files instead of ISO-8859-1 could be one.
In the mean time, I'm also hacking further on the serpent code. The double-block trick in C gave a modest improvement. I think it should work great with sse2 assembly to do four blocks at a time. Primarily for x86_64, but maybe also for plain x86 (but then maybe one can's use sse2 unconditionally). But that doesn't have to be completed before release.
Right.
/Simon
Simon Josefsson simon@josefsson.org writes:
I don't understand why this is a Nettle problem, since I don't recall having to do anything like that for any of the libraries I maintain. Could it be because I use automake and libtool?
Sure, it's not a very nettle-specific problem. Maybe current automake and/or libtool does the right thing, but I doubt it.
I think I've tried to explain what the problem is a couple of times before, but appearantly I failed at that. I'll try one more time: Please try out the the default behaviour of
./configure && make && make install
with your libraries (or any random library using autoconf and default GNU installation conventions). I would expect it to break badly on, e.g., a x86_64 redhat-based system, with an installed gcc which defaults to 64-bit objects (-m64 enabled by default). Then make install will install in /usr/local/lib, destroying any previous 32-bit installation of the same library. Correct install location on this system, which follows the linux fhs, is /usr/local/lib64.
Similar problems would occur on the (a bit less likely) combinations of
64-bit Solaris (sparc or x86_64), if you have installed a gcc to default to 64-bit.
freebsd or debian gnu/linux on x86_64, if for some reason you have installed a compiler which defaults to 32-bit.
I want the default behavior to not break things, that's all. And it's no big deal. The bug in the current configure logic should be easy to fix (I've probably spent about as much time on this email as it will take to fix the problem).
If automake would solve this issue, how about using it in Nettle?
No thanks. But I'd be happy to reuse any decent autoconf macros solving the problem, from automake or elsewhere. If there are any.
- Decide which flavor of LGPL licensing should be used. Options: LGPLv2+, LGPLv3+ or dual license LGPLv3+ and GPLv2+.
Is there a reason to change from the current LGPLv2+ before the next release?
Maybe not. Moving to LGPLv2+ is definitely the easiest and smallest change.
- Update NEWS and documentation (regarding serpent and licensing).
I can help with that, if you want. I did a diff between 2.1 and the current code, and it is a fairly small set of changes.
That would be appreciated. I started on the NEWS entry a while back, I just checked in my current version. I don't remember off the top of my head what the last item, "Interface cleanup", refers to. (I usually review the ChangeLog rather than actual diffs when updating the NEWS file).
Using UTF-8 encoding for the files instead of ISO-8859-1 could be one.
That should be done some time, but I don't think I'll bother about it now. E.g., the move away from cvs seems more important, and that will also have to wait until after release.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
- Fix the libdir defaults for linux x86_64 (I think the following should work in most cases for for both debian and fhs conventions: When building for $ABI (32 or 64), check if /usr/lib$ABI exists. If so, install there, otherwise in $prefix/lib. Possibly bail out with an error if neither /usr/lib32 nor /usr/lib64 exists).
I've checked in a fix for this now, and I also wrote up a web page with the release TODO list (organization copied from corresponding pages for recent GMP releases). See http://www.lysator.liu.se/~nisse/nettle/plan.html
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
I think everything on this list is done now. I also checked in the new serpent sse2 code, and I've looked through my older changes for memxor and cbs decrypt (the latter can avoid a memcpy by using memxor3 with overlapping operands), and I think they're ok.
So I think the 2.2 release is near. But I'm travelling with somewhat limited hacking time and spotty connectivity.
I'm checking http://www.lysator.liu.se/~nisse/xeno-lsh/latest.html (which also builds nettle). Any additional last minute testing of the current cvs tree is appreciated.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
So I think the 2.2 release is near.
Great!
Any additional last minute testing of the current cvs tree is appreciated.
It builds and passes self checks on debian squeeze x86 and a Solaris 2.10 box (without gmp though) I have access to.
Clang-analyze on the code did not raise any errors in the core library, but some other minor issues in other code:
Bug Group Bug Type File Line
API Argument with 'nonnull' attribute passed null tools/sexp-conv.c 274
Dead store Dead assignment examples/io.c 88
Dead store Dead assignment examples/io.c 90
Logic error Dereference of null pointer tools/nettle-hash.c 204
Logic error Dereference of null pointer tools/nettle-hash.c 205
Logic error Dereference of null pointer examples/io.c 123
/Simon
Simon Josefsson simon@josefsson.org writes:
Clang-analyze on the code did not raise any errors in the core library, but some other minor issues in other code:
Thanks, I've reviewed the code in questions:
API Argument with 'nonnull' attribute passed null tools/sexp-conv.c 274
This is a call to strtol.
int width = strtol(optarg, &end , 0);
optarg shouldn't be NULL, but that's not really nettle's responsibility (-w has a required argument in the getopt_long call). Warning seems to be a false positive.
Dead store Dead assignment examples/io.c 88
Dead store Dead assignment examples/io.c 90
This is a minor real bug.
Logic error Dereference of null pointer tools/nettle-hash.c 204
Logic error Dereference of null pointer tools/nettle-hash.c 205
This is
length = alg->digest_size; else if (length > alg->digest_size)
and there's an if (!alg) die(...) earlier, where (at least for gcc) die is declared as __attribute((__noreturn__)). Warning seems to be a false positive.
But there's a related copy&paste error, since werror has the same declaration and it *does* return,
Logic error Dereference of null pointer examples/io.c 123
buffer[done] = '\0';
buffer is return value from a realloc call with a NULL check and return. The logic using feof and ferror looks a bit suspicious, but as long as feof is false just after fopen, things should be ok (and feof should not be set until one attempts to read).
Thanks, /Niels
nettle-bugs@lists.lysator.liu.se
-
Nikos Mavrogiannopoulos -
nisse＠lysator.liu.se -
Simon Josefsson