From eliz at gnu.org Sat Jan 7 21:08:48 2012 From: eliz at gnu.org (Eli Zaretskii) Date: Sat, 07 Jan 2012 22:08:48 +0200 Subject: Building libnettle on MS-Windows Message-ID: <837h13ff73.fsf@gnu.org> About a week ago, I built libnettle 2.4 on MS-Windows using the MinGW tools. While doing that, I found a few problems; this message reports them. For the record, I configured using " --disable-openssl --enable-shared". This was suggested by the GnuTLS README file; as I needed libnettle to build GnuTLS, I followed that advice. Problem #1: Compiler warnings while compiling cbc.c: cbc.c: In function `nettle_cbc_decrypt': cbc.c:101: warning: implicit declaration of function `alloca' cbc.c:101: warning: nested extern declaration of `alloca' I fixed this by adding this to config.h.in: /* Needed for alloca on MinGW */ # if HAVE_MALLOC_H # include # endif malloc.h in MinGW defines alloca to the GCC builtin; if the above is absent, the compiler falls back on "char *alloca()" which causes the warnings. Problem #2: "make install" incorrectly copies the DLL files into lib/*.dll.a. *.dll.a are the Windows import libraries for using during the link stage; the *.dll dynamic libraries themselves should be copied into the $(bindir) directory. My solution was to add commands to corresponding recipes that cater to Windows, where *_SONAME variables are empty: install-shared-hogweed: $(LIBHOGWEED_FORLINK) $(MKDIR_P) $(DESTDIR)$(libdir) $(MKDIR_P) $(DESTDIR)$(bindir) [ -z "$(LIBHOGWEED_SONAME)" ] \ && $(INSTALL_DATA) $(LIBHOGWEED_FORLINK) $(DESTDIR)$(bindir) [ -z "$(LIBHOGWEED_SONAME)" ] \ && $(INSTALL_DATA) $(LIBHOGWEED_FILE) $(DESTDIR)$(libdir) [ -z "$(LIBHOGWEED_SONAME)" ] \ || $(INSTALL_DATA) $(LIBHOGWEED_FORLINK) $(DESTDIR)$(libdir)/$(LIBHOGWEED_FILE) [ -z "$(LIBHOGWEED_SONAME)" ] \ || (cd $(DESTDIR)$(libdir) \ && $(LN_S) $(LIBHOGWEED_FILE) $(LIBHOGWEED_SONAME) \ && $(LN_S) $(LIBHOGWEED_FILE) $(LIBHOGWEED_FORLINK)) Problem #3: Tests crash because they don't find libhogweed-2-1.dll. This is because Makefile uses "ln -sf" to create .lib/*.dll. Solution: add LN_S variable to Makefile.in, and then re-run the build with "make LN_S='cp -prf'. Also, add this command to the recipes that build shared libraries: [ -z "$(LIBNETTLE_SONAME)" ] && (cd .lib \ && $(LN_S) ../$(LIBNETTLE_FORLINK) $(LIBNETTLE_FORLINK)) so that the DLLs are copied to .lib/ under the same name. Problem #4: Tests in `examples' crash, cannot find libhogweed-2-1. Solution: add ../.lib to PATH, like testsuite/Makefile.in does. Problem #5: testsuite/pkcs1-conv fails due to CRLF vs LF issue in compared files (MinGW programs produce text files where each line ends in CRLF, while the test expects the Unix-style LF-only end-of-line format). Solution: add --strip-trailing-cr switch to diff. Problem #6: testsuite/sexp-conv fails due to CRLF vs LF issue in compared files. Solution: use "diff --strip-trailing-cr" instead of "cmp". Problem #7: examples/setup-env fails because it tries to read from rsa-decrypt, which doesn't exist. It should instead read from rsa-decrypt.exe. Solution: fix setup-env to check which program file is actually present and read from that. A similar problem is in examples/rsa-encrypt-test. Problem #8: examples/rsa-encrypt-test fails because rsa-encrypt.c and rsa-decrypt.c read/write binary data from stdin to stdout, but do not set these streams to binary mode. To solve this, I added code to the test programs to use binary mode. Let me know if you want me to submit patches for that. Last, but not least: thanks for developing and maintaining libnettle! From martin at martin.st Sat Jan 7 21:16:27 2012 From: martin at martin.st (=?ISO-8859-15?Q?Martin_Storsj=F6?=) Date: Sat, 7 Jan 2012 22:16:27 +0200 (EET) Subject: Building libnettle on MS-Windows In-Reply-To: <837h13ff73.fsf@gnu.org> References: <837h13ff73.fsf@gnu.org> Message-ID: Hi Eli, On Sat, 7 Jan 2012, Eli Zaretskii wrote: > About a week ago, I built libnettle 2.4 on MS-Windows using the MinGW > tools. While doing that, I found a few problems; this message reports > them. Many of these issues have been fixed in the latest version in CVS - please retry with that version and see which ones of them still apply. Of these, we've at least worked on problem #2, #5, #6, #7, #8, and improved the assembler code to work on windows. It's been mostly tested when building under linux though (and running tests through wine), so there might perhaps still be some issues when running under real windows. // Martin From nisse at lysator.liu.se Sat Jan 7 22:08:30 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sat, 07 Jan 2012 22:08:30 +0100 Subject: Building libnettle on MS-Windows In-Reply-To: <837h13ff73.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 07 Jan 2012 22:08:48 +0200") References: <837h13ff73.fsf@gnu.org> Message-ID: Eli Zaretskii writes: > About a week ago, I built libnettle 2.4 on MS-Windows using the MinGW > tools. While doing that, I found a few problems; this message reports > them. Thanks for testing. As Martin wrote, there has been some progress on windows support since the release. It would be great if you could test the cvs version (sorry it's a little cumbersome; you need to check out the lsh tree and follow the instructions on http://www.lysator.liu.se/~nisse/nettle/). > Problem #1: Compiler warnings while compiling cbc.c: > > cbc.c: In function `nettle_cbc_decrypt': > cbc.c:101: warning: implicit declaration of function `alloca' > cbc.c:101: warning: nested extern declaration of `alloca' Not fixed, as far as I know. I don't see this warning when cross compiling, so I guess this reflects a real difference in native builds. > Problem #2: "make install" incorrectly copies the DLL files into > lib/*.dll.a. *.dll.a are the Windows import libraries for using > during the link stage; the *.dll dynamic libraries themselves should > be copied into the $(bindir) directory. Should be fixed, thanks to Martin. > Problem #3: Tests crash because they don't find libhogweed-2-1.dll. This is > because Makefile uses "ln -sf" to create .lib/*.dll. Not fixed. Using an LN_S make variable makes sense to me, but there should also be a corresponding configure check (hopefully there's some standard autoconf test?). Patch appreciated. > Problem #4: Tests in `examples' crash, cannot find libhogweed-2-1. > > Solution: add ../.lib to PATH, like testsuite/Makefile.in does. Seems right. Checked in now. > Problem #5: testsuite/pkcs1-conv fails due to CRLF vs LF issue in compared > files (MinGW programs produce text files where each line ends in CRLF, > while the test expects the Unix-style LF-only end-of-line format). > > Solution: add --strip-trailing-cr switch to diff. Fixed, I think (and also #6). Files are massaged with tr as needed, since that's more portable than relying on GNU diff. > Problem #7: examples/setup-env fails because it tries to read from > rsa-decrypt, which doesn't exist. It should instead read from > rsa-decrypt.exe. > > Solution: fix setup-env to check which program file is actually > present and read from that. Fixed, by passing $EXEEXT in the environment and using it in setup-env and also in other test scripts. > Problem #8: examples/rsa-encrypt-test fails because rsa-encrypt.c and > rsa-decrypt.c read/write binary data from stdin to stdout, but do > not set these streams to binary mode. > > To solve this, I added code to the test programs to use binary > mode. Let me know if you want me to submit patches for that. These files now do #ifdef WIN32 setmode(0, O_BINARY); setmode(1, O_BINARY); #endif Do you think this is a correct solution? Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From eliz at gnu.org Sat Jan 7 22:24:17 2012 From: eliz at gnu.org (Eli Zaretskii) Date: Sat, 07 Jan 2012 23:24:17 +0200 Subject: Building libnettle on MS-Windows In-Reply-To: References: <837h13ff73.fsf@gnu.org> Message-ID: <8362gnfbpa.fsf@gnu.org> > From: nisse at lysator.liu.se (Niels M?ller) > Cc: nettle-bugs at lists.lysator.liu.se > Date: Sat, 07 Jan 2012 22:08:30 +0100 > > Thanks for testing. As Martin wrote, there has been some progress on > windows support since the release. It would be great if you could test > the cvs version (sorry it's a little cumbersome; you need to check out > the lsh tree and follow the instructions on > http://www.lysator.liu.se/~nisse/nettle/). I will try that when I have time, if a new release is too far to wait until then. Doesn't building from CVS require additional tools, like Autoconf and Automake? > > Problem #3: Tests crash because they don't find libhogweed-2-1.dll. This is > > because Makefile uses "ln -sf" to create .lib/*.dll. > > Not fixed. Using an LN_S make variable makes sense to me, but there > should also be a corresponding configure check (hopefully there's some > standard autoconf test?). Patch appreciated. LN_S is a standard autoconf thing, so it should be easy. Sorry, I don't know about autoconf enough to send a patch. However, by looking at random configure scripts, I found that having this in configure.ac: AC_PROG_LN_S is probably all you need. > > Problem #8: examples/rsa-encrypt-test fails because rsa-encrypt.c and > > rsa-decrypt.c read/write binary data from stdin to stdout, but do > > not set these streams to binary mode. > > > > To solve this, I added code to the test programs to use binary > > mode. Let me know if you want me to submit patches for that. > > These files now do > > #ifdef WIN32 > setmode(0, O_BINARY); > setmode(1, O_BINARY); > #endif > > Do you think this is a correct solution? Yes, except that I would use _setmode, as that will also work for Windows compilers other than MinGW. Thanks. From nisse at lysator.liu.se Sat Jan 7 23:05:42 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sat, 07 Jan 2012 23:05:42 +0100 Subject: Building libnettle on MS-Windows In-Reply-To: <8362gnfbpa.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 07 Jan 2012 23:24:17 +0200") References: <837h13ff73.fsf@gnu.org> <8362gnfbpa.fsf@gnu.org> Message-ID: Eli Zaretskii writes: > Doesn't building from CVS require additional tools, like Autoconf and > Automake? Autoconf is needed. So if you don't have autoconf on the windows box, what you'd need to do is check it out to a unix/gnu/linux-system, run the .bootstrap scripts (as per the instructions), and do a ./configure && make && make dist in the nettle directory. And then copy the resulting .tar.gz to the windows machine. So it's not very convenient. When the lsh xenofarm is up and running, you can also cheat and get a recent lsh snapshot from http://www.lysator.liu.se/xenofarm/lsh/builds/latest. At the moment, the most recent snapshot there is from October, but if things goes well a current one should be prepared within an hour or so. I'm not sure when I'll get around to making a new release. > AC_PROG_LN_S > > is probably all you need. [...] > Yes, except that I would use _setmode, as that will also work for > Windows compilers other than MinGW. Noted. I'll see what I can do. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Tue Jan 17 13:58:37 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 17 Jan 2012 13:58:37 +0100 Subject: Building libnettle on MS-Windows In-Reply-To: <8362gnfbpa.fsf@gnu.org> (Eli Zaretskii's message of "Sat, 07 Jan 2012 23:24:17 +0200") References: <837h13ff73.fsf@gnu.org> <8362gnfbpa.fsf@gnu.org> Message-ID: Eli Zaretskii writes: >I found that having this in configure.ac: > > AC_PROG_LN_S > > is probably all you need. I have added this now. > Yes, except that I would use _setmode, as that will also work for > Windows compilers other than MinGW. And this. I also found out that dlls apperantly weren't linked into the .lib directory, so I tried to fix that as well. This directory is added to LD_LIBRARY_PATH (for ELF systems) and PATH (for windows), when running the testsuite. Unfortunately, I couldn't figure out how to get wine to add the .lib directory to the initial PATH. So for now, as a workaround, I create symlinks also in the testsuite and examples subdirectories in the build tree, when cross compiling for windows. With that change, ./configure '--host=i586-mingw32msvc' '--enable-shared' make make check works for me. I just put up a prerelease at http://www.lysator.liu.se/~nisse/archive/nettle-2.5-pre.tar.gz (and .sig), any testing (in particular on windows) is appreciated. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From simon at josefsson.org Tue Jan 17 14:09:30 2012 From: simon at josefsson.org (Simon Josefsson) Date: Tue, 17 Jan 2012 14:09:30 +0100 Subject: Building libnettle on MS-Windows In-Reply-To: ("Niels \=\?iso-8859-1\?Q\?M\=F6ller\=22's\?\= message of "Tue, 17 Jan 2012 13:58:37 +0100") References: <837h13ff73.fsf@gnu.org> <8362gnfbpa.fsf@gnu.org> Message-ID: <87d3aiqxv9.fsf@latte.josefsson.org> nisse at lysator.liu.se (Niels M?ller) writes: > I just put up a prerelease at > http://www.lysator.liu.se/~nisse/archive/nettle-2.5-pre.tar.gz (and > .sig), any testing (in particular on windows) is appreciated. With MinGW-64 I got the following error when building: make[1]: Entering directory `/home/jas/src/nettle-2.5-pre/testsuite' i686-w64-mingw32-g++ -g -O2 -L.. cxx-test.o testutils.o -lnettle -o cxx-test.exe /home/jas/src/mingw-w64-32/build/root/lib/gcc/i686-w64-mingw32/4.5.1/../../../../i686-w64-mingw32/bin/ld: cannot find -lgcc_s /home/jas/src/mingw-w64-32/build/root/lib/gcc/i686-w64-mingw32/4.5.1/../../../../i686-w64-mingw32/bin/ld: cannot find -lgcc_s collect2: ld returnerade avslutningsstatus 1 make[1]: *** [cxx-test.exe] Fel 1 It may be that my MinGW-64 g++ installation is not working properly. When I passed 'CXX=false' to ./configure it built without errors. Most self-checks works, except if fails at the end: PASS: sexp-conv SKIP: pkcs1-conv PASS: symbols =================== All 36 tests passed =================== rm: argument saknas F?rs?k med "rm --help" f?r mer information. FAIL: ./teardown-env make[1]: *** [check] Fel 1 The call to find in teardown-env turns out empty. This seems to be the case when I build with --enable-shared as well -- then there is only one hit (./.lib/libnettle-4-3.dll), but it is not under testsuite/. Should there be more *.dll files? /Simon From nisse at lysator.liu.se Tue Jan 17 14:37:11 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 17 Jan 2012 14:37:11 +0100 Subject: Building libnettle on MS-Windows In-Reply-To: <87d3aiqxv9.fsf@latte.josefsson.org> (Simon Josefsson's message of "Tue, 17 Jan 2012 14:09:30 +0100") References: <837h13ff73.fsf@gnu.org> <8362gnfbpa.fsf@gnu.org> <87d3aiqxv9.fsf@latte.josefsson.org> Message-ID: Simon Josefsson writes: > With MinGW-64 I got the following error when building: For clarity: Is this MinGW-64 *natively* on a windows machine, or a cross compiler? > make[1]: Entering directory `/home/jas/src/nettle-2.5-pre/testsuite' > i686-w64-mingw32-g++ -g -O2 -L.. cxx-test.o testutils.o -lnettle -o cxx-test.exe > /home/jas/src/mingw-w64-32/build/root/lib/gcc/i686-w64-mingw32/4.5.1/../../../../i686-w64-mingw32/bin/ld: cannot find -lgcc_s > make[1]: *** [cxx-test.exe] Fel 1 > > It may be that my MinGW-64 g++ installation is not working properly. Seems likely; I can't see any reason why the compilation command line should give that error. > FAIL: ./teardown-env > make[1]: *** [check] Fel 1 > > The call to find in teardown-env turns out empty. Ooops. I should replace that backtick expression with an -exec argument to find. > This seems to be the case when I build with --enable-shared as well -- > then there is only one hit (./.lib/libnettle-4-3.dll), but it is not > under testsuite/. Should there be more *.dll files? For a native build: That's how it should be. The .lib directory is added to PATH, which should be sufficient for running the testprograms. The additional symlinks in the testsuite and example directories should be there only when cross compiling (detected by $EMULATOR matching wine*). Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From simon at josefsson.org Tue Jan 17 14:47:46 2012 From: simon at josefsson.org (Simon Josefsson) Date: Tue, 17 Jan 2012 14:47:46 +0100 Subject: Building libnettle on MS-Windows In-Reply-To: ("Niels \=\?iso-8859-1\?Q\?M\=F6ller\=22's\?\= message of "Tue, 17 Jan 2012 14:37:11 +0100") References: <837h13ff73.fsf@gnu.org> <8362gnfbpa.fsf@gnu.org> <87d3aiqxv9.fsf@latte.josefsson.org> Message-ID: <8739bephj1.fsf@latte.josefsson.org> nisse at lysator.liu.se (Niels M?ller) writes: > Simon Josefsson writes: > >> With MinGW-64 I got the following error when building: > > For clarity: Is this MinGW-64 *natively* on a windows machine, or a > cross compiler? Cross-compiler, running on a Debian Squeeze machine. >> This seems to be the case when I build with --enable-shared as well -- >> then there is only one hit (./.lib/libnettle-4-3.dll), but it is not >> under testsuite/. Should there be more *.dll files? > > For a native build: That's how it should be. The .lib directory is added > to PATH, which should be sufficient for running the testprograms. The > additional symlinks in the testsuite and example directories should be > there only when cross compiling (detected by $EMULATOR matching wine*). I have the binfmt-support package installed, so an emulator is not strictly required -- i.e., running ./foo.exe will work directly and using 'wine ./foo.exe' is not necessary. Will $EMULATOR be empty in this case? /Simon From nisse at lysator.liu.se Tue Jan 17 15:06:20 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 17 Jan 2012 15:06:20 +0100 Subject: Building libnettle on MS-Windows In-Reply-To: <8739bephj1.fsf@latte.josefsson.org> (Simon Josefsson's message of "Tue, 17 Jan 2012 14:47:46 +0100") References: <837h13ff73.fsf@gnu.org> <8362gnfbpa.fsf@gnu.org> <87d3aiqxv9.fsf@latte.josefsson.org> <8739bephj1.fsf@latte.josefsson.org> Message-ID: Simon Josefsson writes: > I have the binfmt-support package installed, so an emulator is not > strictly required -- i.e., running ./foo.exe will work directly and > using 'wine ./foo.exe' is not necessary. > Will $EMULATOR be empty in > this case? It will most likely have been set automatically to wine64. Check the substituted variable in config.make. Do you think some additional configure tests are needed? (Alternatively, one could do something similar to automake's "make check TESTS_ENVIRONMENT=wine", but that's not as general since some of the test programs are shell scripts, not windows executables). But I think I have found out why the links weren't created. I forgot to set the executable bit on testsuite/setup-env. And since this is still cvs, that's a bit painful to get fixed. *sigh* Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From eliz at gnu.org Tue Jan 17 18:38:46 2012 From: eliz at gnu.org (Eli Zaretskii) Date: Tue, 17 Jan 2012 19:38:46 +0200 Subject: Building libnettle on MS-Windows In-Reply-To: References: <837h13ff73.fsf@gnu.org> <8362gnfbpa.fsf@gnu.org> Message-ID: <83boq25ivt.fsf@gnu.org> > From: nisse at lysator.liu.se (Niels M?ller) > Cc: nettle-bugs at lists.lysator.liu.se > Date: Tue, 17 Jan 2012 13:58:37 +0100 > > I also found out that dlls apperantly weren't linked into the .lib > directory, so I tried to fix that as well. This directory is added to > LD_LIBRARY_PATH (for ELF systems) and PATH (for windows), when running > the testsuite. > > Unfortunately, I couldn't figure out how to get wine to add the .lib > directory to the initial PATH. Is that a wine-specific problem, or a general Windows problem? If the latter, maybe I can help if you describe the difficulty. > I just put up a prerelease at > http://www.lysator.liu.se/~nisse/archive/nettle-2.5-pre.tar.gz (and > .sig), any testing (in particular on windows) is appreciated. Thanks, I will try to do this as soon as I have time. From nisse at lysator.liu.se Tue Jan 17 20:49:29 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 17 Jan 2012 20:49:29 +0100 Subject: Building libnettle on MS-Windows In-Reply-To: <83boq25ivt.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 17 Jan 2012 19:38:46 +0200") References: <837h13ff73.fsf@gnu.org> <8362gnfbpa.fsf@gnu.org> <83boq25ivt.fsf@gnu.org> Message-ID: Eli Zaretskii writes: >> Unfortunately, I couldn't figure out how to get wine to add the .lib >> directory to the initial PATH. > > Is that a wine-specific problem, or a general Windows problem? That's wine specific. Wine copies most environment variables from the unix environment it is started from into the environment of the executed windows program. With some exceptions for "special" environment variables, including PATH. After a quick look at the source code, it looks one should be able to set WINEPATH when starting wine, and have that copied to the windows path (possibly without any automagic translation of the filenames). But I couldn't get that to work. Anyway, it's a wine-specific problem, and I have a workaround. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From senvey at gmail.com Sat Jan 21 07:59:19 2012 From: senvey at gmail.com (Senvey Lee) Date: Sat, 21 Jan 2012 06:59:19 +0000 (UTC) Subject: make issues References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com> Message-ID: Niels M?ller writes: > > Anna Smith writes: > > > the link provided is broken :( can you repost? > > Works for me. Maybe you accidentally copied the "," at the end? That was > not intended as a part of the url. It should be just > > http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD > > Regards, > /Niels > Another success! Thanks. But got an ensuing error: md5-compress.asm:100:suffix or operands invalid for `push' Could someone help one this? Thanks. From nisse at lysator.liu.se Tue Jan 24 10:01:01 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 24 Jan 2012 10:01:01 +0100 Subject: Migration from CVS to GIT Message-ID: I'm finally looking into conversion of the lsh repository from CVS to GIT, and this naturally also includes nettle. A public test repository can be found at lysator's gitorious installation, at http://git.lysator.liu.se/lsh/test-2. The conversion was done by the pcvs2git.pike program (see git://pike-git.lysator.liu.se/pcvs2git.git), and I got some help from Henrik Grubbstr?m to write a config file to handle the few peculiarities in the lsh repository. The intention is that the new git repository should include all branches and tags from the old cvs repository, and record major merge events, like the 2006-05-16 merge from the experimental branch to the trunk. Please test, and if all goes well I'll rename this repository to "lsh" (or recreate, if I can't figure out how to do repository renames with gitorious). If this first steps works out ok, step two is to do a some cleanups (switching to utf-8 for the files, deleting old $Id$ tags, etc). And then step three is to extract the nettle subdirectory as an independent project and repository, using git subtree of git filter-branch or so (I'm not sure what's the best tool for that job). Other sub-projects, e.g., the argp implementation, can be split out later, if desired. For the few "common files" (e.g., misc/run-tests), they'll simply have to be duplicated in several repositories. If I still want to bundle nettle with the lsh distribution, I'll handle that by setting up some symlink in my working tree. git submodule is probably not a solution, and git subtree is most likely overkill. The old CVS repository can be considered read-only now. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From simon at josefsson.org Tue Jan 24 10:25:50 2012 From: simon at josefsson.org (Simon Josefsson) Date: Tue, 24 Jan 2012 10:25:50 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Niels \=\?iso-8859-1\?Q\?M\=F6ller\=22's\?\= message of "Tue, 24 Jan 2012 10:01:01 +0100") References: Message-ID: <8762g178pt.fsf@latte.josefsson.org> nisse at lysator.liu.se (Niels M?ller) writes: > I'm finally looking into conversion of the lsh repository from CVS to > GIT, and this naturally also includes nettle. That is great. I find that testing and preparing patches is easier with git than CVS. I have been wanting to propose patches for adding the new SHA2/x hashes, and this will be easier for me now. > Please test, and if all goes well I'll rename this repository to "lsh" > (or recreate, if I can't figure out how to do repository renames with > gitorious). The nettle part worked fine for me. Let me suggest to populate .gitignore with some files indicate by 'git status' after building Nettle once. For me the output was as below. /Simon # INSTALL # aclocal.m4 # aes-decrypt-internal.asm # aes-encrypt-internal.asm # camellia-crypt-internal.asm # config.guess # config.sub # examples/getopt.c # examples/getopt.h # examples/getopt1.c # examples/next-prime # examples/random-prime # examples/run-tests # gcmdata # hogweed.pc # install-sh # machine.m4 # memxor.asm # nettle.pc # serpent-decrypt.asm # serpent-encrypt.asm # sha1-compress.asm # testsuite/camellia-test # testsuite/gcm-test # testsuite/memxor-test # testsuite/meta-armor-test # testsuite/meta-cipher-test # testsuite/meta-hash-test # testsuite/random-prime-test # testsuite/ripemd160-test # testsuite/run-tests # testsuite/sha224-test # testsuite/sha384-test # testsuite/sha512-test # texinfo.tex # tools/getopt.c # tools/getopt.h # tools/getopt1.c # tools/nettle-hash From martin at martin.st Tue Jan 24 10:27:44 2012 From: martin at martin.st (=?ISO-8859-15?Q?Martin_Storsj=F6?=) Date: Tue, 24 Jan 2012 11:27:44 +0200 (EET) Subject: Migration from CVS to GIT In-Reply-To: References: Message-ID: Hi, On Tue, 24 Jan 2012, Niels M?ller wrote: > I'm finally looking into conversion of the lsh repository from CVS to > GIT, and this naturally also includes nettle. This is great news! > If this first steps works out ok, step two is to do a some cleanups > (switching to utf-8 for the files, deleting old $Id$ tags, etc). And then > step three is to extract the nettle subdirectory as an independent > project and repository, using git subtree of git filter-branch or so > (I'm not sure what's the best tool for that job). Other sub-projects, > e.g., the argp implementation, can be split out later, if desired. I'd hope for a round of filter-branch to rewrite author names in git to the original author of patches. I haven't done it myself, but I've seen it done on other projects. One can do some kind of more or less heuristic shell script that e.g. greps the commit message for known strings (either known contributors or strings like "From:" or "contributed by") and based on these override the author name. If you don't feel up to doing this yourself, I can try to give it a go. // Martin From nisse at lysator.liu.se Tue Jan 24 13:15:08 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 24 Jan 2012 13:15:08 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Martin =?iso-8859-1?Q?Storsj=F6=22's?= message of "Tue, 24 Jan 2012 11:27:44 +0200 (EET)") References: Message-ID: Martin Storsj? writes: > I'd hope for a round of filter-branch to rewrite author names in git > to the original author of patches. I was thinking that it's good enough to convert the information recorded by cvs. But it's certainly possible to add specified author names for certain commits, as a part of the conversion process. The config/Pike-real-authors file (from the git://pike-git.lysator.liu.se/pcvs2git.git repository) is an example of what that input file should look like. Since I haven't used it I'm not sure how it works, but I could try to find out. But it's going to be some manual work, going through the ChangeLog files looking for contributed changes, and then looking at the cvs log to identify affected revisions and writing the input file for --contributors. > If you don't feel up to doing this yourself, I can try to give it a go. If you want to give it a try, I'll explain what's needed. 1. Get pike 7.8 release 550 or later, http://pike.ida.liu.se/pub/pike/beta/7.8.550/Pike-v7.8.550.tar.gz Annoyingly enough, it doesn't support the most recent version of nettle, resulting in compilation errors on the DES glue. I had to #if out the DES and DES3 ciphers from the Nettle module. Alternatively, there should be some configure argument to force Pike to build with the bundled version of nettle, rather than the installed version. 2. Get git://pike-git.lysator.liu.se/pcvs2git.git. This includes Henrik's configuration file in config/lsh.pcvs2git. 3. Get the CVS repository. I'm putting a copy at http://www.lysator.liu.se/~nisse/misc/lsh-cvsroot.tar.gz 4. Create an authors.txt file, containing nisse=Niels M?ller uid444=Niels M?ller _cvs_pont=Pontus Freyhult 5. To do the conversion, I used the command line pike ../pcvs2git/pcvs2git.pike -c ../pcvs2git/config/lsh.pcvs2git \ -A authors.txt -k -l -C lsh.git -d lsh-cvsroot/lsh It spawns a git fast-import process which populates the new repository. The result should be almost identical to the current "test-2" repository (Henrik has committed some tweaks for the generated .gitattributes files). To use a contributors file, add --contributors contributors.txt early on the above command line (the -d option and argument has to come after other options). Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Tue Jan 31 15:39:11 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 31 Jan 2012 15:39:11 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Tue, 24 Jan 2012 10:01:01 +0100") References: Message-ID: Now I've went through the history, back from the previous millennium, startig from the authors and contributors files that Martin was kind enough to prepare. It's taken a few days. The early ChangeLog entries were a bit messy. The new repository is now git://git.lysator.liu.se/lsh/test-3.git And I also found out some things which I had forgotten. E.g., * The very first version of the lsh-export-key program was written by Jean-Pierre Stierlin. * Dan Egnor wrote the base64 code in Nettle, with much effort spent on finding the right interface design. * Pieces of Bal?zs Scheidler's now obsolete lsh_proxy program still survives, since current gateway_channel.c evolved from his proxy_channel.c. The recorded authors are listed below. These are all people I have been able to find who have contributed a larger or smaller change to lsh or nettle (not counting bug reports or sugestions for changes). Change sets which mix a contributed patch with other changes also don't get the contributor listed as author. So we have 37 authors over the years: Adam Langley Andres Mejia Bal?zs Scheidler Dagobert Michelsen Dan Egnor Daniel Kahn Gillmor David Hoyt Gordon Matzigkeit Grant Robinson Henrik Grubbstr?m J.H.M. Dassen James Ralston Jean-Pierre Stierlin Joseph Galbraith Kalle Olavi Niemitalo Karl Berry Keresztfalvi Gabor Agoston Luiz Eduardo Gava Magnus Holmgren Martin Storsj? Meilof Veeningen Niels M?ller Nikos Mavrogiannopoulos Pavel Roskin Per Cederqvist Pontus Freyhult Pontus Sk?ld Rafael Sevilla Rafal Maszkowski Ruud de Rooij Ryan Schmidt Sebastian Reitenbach Simon Josefsson Stefan Pfetzing Thayne Harbaugh Vincent Torri Volker Zell (Some statistics on first and latest contribution for each author, lines contributed, contributed lines still surviving in the tree, etc, would be interesting). Testing and general browsing of the history is appreciated. In a few days, I think I'll consider the conversion final. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From martin at martin.st Tue Jan 31 15:50:48 2012 From: martin at martin.st (=?ISO-8859-15?Q?Martin_Storsj=F6?=) Date: Tue, 31 Jan 2012 16:50:48 +0200 (EET) Subject: Migration from CVS to GIT In-Reply-To: References: Message-ID: On Tue, 31 Jan 2012, Niels M?ller wrote: > Testing and general browsing of the history is appreciated. In a few > days, I think I'll consider the conversion final. The author mapping looks good to me at least, looking forward to the final repo (including separated nettle)! Thanks for your work on this! // Martin From dkg at fifthhorseman.net Tue Jan 31 16:07:28 2012 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Tue, 31 Jan 2012 10:07:28 -0500 Subject: Migration from CVS to GIT In-Reply-To: References: Message-ID: <4F2803B0.30301@fifthhorseman.net> On 01/31/2012 09:50 AM, Martin Storsj? wrote: > On Tue, 31 Jan 2012, Niels M?ller wrote: > >> Testing and general browsing of the history is appreciated. In a few >> days, I think I'll consider the conversion final. > > The author mapping looks good to me at least, looking forward to the > final repo (including separated nettle)! Thanks for your work on this! It also looks good to me. Thanks for doing this conversion work, Niels. --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1030 bytes Desc: OpenPGP digital signature URL: From nisse at lysator.liu.se Fri Feb 3 10:04:30 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Fri, 03 Feb 2012 10:04:30 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Martin =?iso-8859-1?Q?Storsj=F6=22's?= message of "Tue, 31 Jan 2012 16:50:48 +0200 (EET)") References: Message-ID: Martin Storsj? writes: > The author mapping looks good to me at least, looking forward to the > final repo Now "final". Repo url git://git.lysator.liu.se/lsh/lsh.git. I renamed the old gitorious "project" to get the various test repositories out of the way. After the conversion, I have pushed some tags to all branches to mark the conversion. On master only, I have eliminatd $Id tags in files, and converted all textfiles to utf-8 (encoding marks in texinfo and html not yet updated, though). > (including separated nettle)! Not there yet. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Fri Feb 3 22:35:41 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Fri, 03 Feb 2012 22:35:41 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Martin =?iso-8859-1?Q?Storsj=F6=22's?= message of "Tue, 31 Jan 2012 16:50:48 +0200 (EET)") References: Message-ID: Martin Storsj? writes: > The author mapping looks good to me at least, looking forward to the > final repo (including separated nettle)! Thanks for your work on this! I now have a first nettle test repo at git://git.lysator.liu.se/lsh-test/nettle-test.git It was created by first cloning the lsh repository, all branches. Then git filter-branch --subdirectory-filter nettle --prune-empty -- --all followed by git update-ref -d on all refs which the previous command warned about being unchanged. Including branches other than master is probably overkill. One file (nettle/tools/nettle-lfib-stream.c) seems to have originated on the lsh-1.4.2 branch, but I think it was copied manually to the main trunk. However, this conversion loses quite a lot of history, including all nettle-1.15 releases. To actually build old versions, one would need to use the lsh repo (due to the shared files linked by lsh's ./.bootstrap), but it would be nice to still have the history in the nettle repo. I think the following approach would make some sense, using git filter-branch --tree-filter or possibly --index-filter: 1. Delete everything except nettle, src/nettle and src/symmetric. 2. If nettle exists, move its contents up one level and delete nettle. 3. Otherwise, if src/nettle exists (and is non-empty), move its contents up two levels and delete the original directory. 4. Otherwise, if src/symmetric exists (and is non-empty), move its contents up two levels and delete the original directory. 5. Otherwise, the tree should be empty. It's from the first few weeks of lsh development, not relevant to nettle. Help from any git guru on the list is appreciated. In particular if the more efficient --index-filter method is to be used, it's not obvious to me how to examine the index and do the right tests and deletions. One could also try to extract all src/symmetric ChangeLog entries from lsh's ChangeLog and copy to nettle's (prior to spring 2001 or so) . But I think that can be done after the git migration; it's not much use to modify earlier revisions of the ChangeLog. Regards, /nisse -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From simon at josefsson.org Sat Feb 4 09:42:36 2012 From: simon at josefsson.org (Simon Josefsson) Date: Sat, 04 Feb 2012 09:42:36 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Niels \=\?iso-8859-1\?Q\?M\=F6ller\=22's\?\= message of "Fri, 03 Feb 2012 10:04:30 +0100") References: Message-ID: <87ipjnkn0j.fsf@latte.josefsson.org> nisse at lysator.liu.se (Niels M?ller) writes: > Martin Storsj? writes: > >> The author mapping looks good to me at least, looking forward to the >> final repo > > Now "final". Repo url git://git.lysator.liu.se/lsh/lsh.git. I renamed > the old gitorious "project" to get the various test repositories out of > the way. Great! I added it to my batch job to create statistics for, see output here: http://gitstats.josefsson.org/lsh/ /Simon From simon at josefsson.org Mon Feb 6 19:42:07 2012 From: simon at josefsson.org (Simon Josefsson) Date: Mon, 06 Feb 2012 19:42:07 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Niels \=\?iso-8859-1\?Q\?M\=F6ller\=22's\?\= message of "Fri, 03 Feb 2012 22:35:41 +0100") References: Message-ID: <87d39rajnk.fsf@latte.josefsson.org> nisse at lysator.liu.se (Niels M?ller) writes: > Martin Storsj? writes: > >> The author mapping looks good to me at least, looking forward to the >> final repo (including separated nettle)! Thanks for your work on this! > > I now have a first nettle test repo at > git://git.lysator.liu.se/lsh-test/nettle-test.git It doesn't build: jas at latte:~/src/nettle-test master$ ./.bootstrap jas at latte:~/src/nettle-test master$ ./configure configure: error: cannot find install-sh, install.sh, or shtool in . "."/. jas at latte:~/src/nettle-test master$ However perhaps that is one of the unresolved issues. > However, this conversion loses quite a lot of history, including all > nettle-1.15 releases. To actually build old versions, one would need to > use the lsh repo (due to the shared files linked by lsh's ./.bootstrap), > but it would be nice to still have the history in the nettle repo. > > I think the following approach would make some sense, using git > filter-branch --tree-filter or possibly --index-filter: > > 1. Delete everything except nettle, src/nettle and src/symmetric. > > 2. If nettle exists, move its contents up one level and delete nettle. > > 3. Otherwise, if src/nettle exists (and is non-empty), move its contents > up two levels and delete the original directory. > > 4. Otherwise, if src/symmetric exists (and is non-empty), move its > contents up two levels and delete the original directory. > > 5. Otherwise, the tree should be empty. It's from the first few weeks of > lsh development, not relevant to nettle. As for these advanced git features, I have no idea. However, I'm not sure it is that important to make the git repo usable for building older nettle releases -- for anyone that really wants to go back in time and build old things can always look at the 'lsh' repository. That reflects how nettle was developed before, and consequently how everyone need to work with those releases. For most people, working on master and/or the latest stable release is what's interesting. So I'd say do a git conversion and then make a release from git and then work forward from there. /Simon From dkg at fifthhorseman.net Mon Feb 6 20:04:08 2012 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Mon, 06 Feb 2012 14:04:08 -0500 Subject: Migration from CVS to GIT In-Reply-To: <87d39rajnk.fsf@latte.josefsson.org> References: <87d39rajnk.fsf@latte.josefsson.org> Message-ID: <4F302428.3080400@fifthhorseman.net> On 02/06/2012 01:42 PM, Simon Josefsson wrote: > For most people, working on master and/or the latest stable release is > what's interesting. So I'd say do a git conversion and then make a > release from git and then work forward from there. I'd agree with this. people who are providing longer-term support for older releases can also use the nettle repo to create a branch for the version they're supporting, and cherry-pick any cleanup commits from stable or master that will make the code generally buildable again. --dkg From nisse at lysator.liu.se Mon Feb 6 20:39:47 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 06 Feb 2012 20:39:47 +0100 Subject: make issues In-Reply-To: (Senvey Lee's message of "Sat, 21 Jan 2012 06:59:19 +0000 (UTC)") References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com> Message-ID: Senvey Lee writes: > Another success! Thanks. > > But got an ensuing error: > md5-compress.asm:100:suffix or operands invalid for `push' Sorry for the late reply. > Could someone help one this? Thanks. The md5-compress.asm file exists only for the 32-bit x86 target. I'm assuming you're building for x86_64? Then either configure and config.guess stil doesn't recognize your platform correctly, or the md5.comress.asm link is a left-over from an earlier run of configure. In the latter case, run make distclean to clean up (with gnu conventions, this is the make target to use to undo everything done by configure), and rerun configure. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Mon Feb 6 20:53:26 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 06 Feb 2012 20:53:26 +0100 Subject: Migration from CVS to GIT In-Reply-To: <87d39rajnk.fsf@latte.josefsson.org> (Simon Josefsson's message of "Mon, 06 Feb 2012 19:42:07 +0100") References: <87d39rajnk.fsf@latte.josefsson.org> Message-ID: Simon Josefsson writes: > configure: error: cannot find install-sh, install.sh, or shtool in . "."/. > jas at latte:~/src/nettle-test master$ > > However perhaps that is one of the unresolved issues. This is one of the files that was setup using symlinks in the lsh repo. I'll add this and other files to the nettle repo, *after* the conversion of the history is satisfactory. > However, I'm not sure it is that important to make the git repo usable > for building older nettle releases I mostly agree here; you won't be able to build from the nettle repo out of the box (just like you currently can't build HEAD out of the box). But I still think it's desirable to have older history, before directory renames, available for git log. Another approach I've been considering is to do a conversion which purges unrelated files (most of src except src/nettle and src/symmetric, most of misc, all contents in doc, argp, sftp, spki, most top-level files). And then use version controlled operations for the remaining cleanup and renames. The aim would then be to make it possible to check out old versions (with their different directory layout) and build them. But at the moment, I'm leaning towards moving directories in the conversion, since I think that will be less confusing with git log and diff. E.g., I'd expect git diff master nettle_1.7_release_20030311 -- configure.ac to show a diff of nettle's configure.ac in different revisions, not a diff between nettle's current configure.ac and a historic configure.ac from lsh. /nisse -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Wed Feb 8 22:07:51 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Wed, 08 Feb 2012 22:07:51 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Fri, 03 Feb 2012 22:35:41 +0100") References: Message-ID: nisse at lysator.liu.se (Niels M?ller) writes: > However, this conversion loses quite a lot of history, including all > nettle-1.15 releases. To actually build old versions, one would need to > use the lsh repo (due to the shared files linked by lsh's ./.bootstrap), > but it would be nice to still have the history in the nettle repo. Now I have a new test repo. Not created using git filter-branch on the lsh git repository, but converted from the same old cvs files, using pcvs2git and a new configuration file written primarily by Henrik Grubbstr?m. The conversion moves files from nettle, src/nettle and src/symmetric (whichever is found first) to the top level. Also tries to keep the shared files, also putting them at top-level in the new repo (getopt*, config.guess, misc/run-tests, and a few more). Some symlinking still needs to be added to the .bootstrap script before one can build. See git://git.lysator.liu.se/lsh-test/nettle-test-2.git /nisse -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From simon at josefsson.org Thu Feb 9 10:05:15 2012 From: simon at josefsson.org (Simon Josefsson) Date: Thu, 09 Feb 2012 10:05:15 +0100 Subject: Migration from CVS to GIT In-Reply-To: References: Message-ID: <1328778315.24581.18.camel@latte.josefsson.org> ons 2012-02-08 klockan 22:07 +0100 skrev Niels M?ller: > git://git.lysator.liu.se/lsh-test/nettle-test-2.git Looks good to me. It is probably easier to preserve history by going from a cvs repository directly to git. Maybe the CVS repository could even be tweaked somewhat before the conversion to make it look more like a "nettle" cvs repository than a lsh repository... /Simon From nisse at lysator.liu.se Tue Feb 14 21:27:31 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 14 Feb 2012 21:27:31 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Wed, 08 Feb 2012 22:07:51 +0100") References: Message-ID: nisse at lysator.liu.se (Niels M?ller) writes: > Now I have a new test repo. Not created using git filter-branch on the > lsh git repository, but converted from the same old cvs files, using > pcvs2git and a new configuration file written primarily by Henrik > Grubbstr?m. Yet another try, and unless I discover any problems with this soon, it's going to be the final one. git://git.lysator.liu.se/lsh-test/nettle-test-3.git Difference from the previous one is that I've hacked the pcvs2git.pike program to keep track of the original path to each RCS file (as it moves directories up one or two levels), and use that in the generated Rev: comments. E.g, for git log -- twofish.c (a file at the top-level in the new git repo), git log includes the following commits (among others): commit f9ec0110676f16e263688b208c3a2e8735723e7d Author: Niels M?ller Date: 2010-07-07 21:32:03 +0200 * aes.h (aes_encrypt, aes_decrypt): Declare ctx argument as const. Also updated implementation. * blowfish.h (blowfish_encrypt, blowfish_decrypt): Likewise. * cast128.h (cast128_encrypt, cast128_decrypt): Likewise. * serpent.h (serpent_encrypt, serpent_decrypt): Likewise. * twofish.h (twofish_encrypt, twofish_decrypt): Likewise. Rev: nettle/ChangeLog:1.91 Rev: nettle/aes-decrypt.c:1.2 Rev: nettle/aes-encrypt.c:1.2 Rev: nettle/aes.h:1.2 Rev: nettle/blowfish.c:1.3 Rev: nettle/blowfish.h:1.3 Rev: nettle/cast128.c:1.2 Rev: nettle/cast128.h:1.2 Rev: nettle/serpent.c:1.2 Rev: nettle/serpent.h:1.2 Rev: nettle/twofish.c:1.2 Rev: nettle/twofish.h:1.2 commit dd06afa4b49913037dabe6cab40ae1b9d49a0344 Author: Niels M?ller Date: 2004-10-05 23:46:33 +0200 (q_table): Use a const pointer array. Rev: src/nettle/twofish.c:1.7 commit 7b6ff2dcdaef64e194c9b83eed3354a5acda9168 Author: Niels M?ller Date: 1999-03-08 00:52:36 +0100 Moved UNUSED attribute to be compatible with gcc-2.x, x<8. Rev: src/symmetric/twofish.c:1.3 Note that the Rev: comments refer to different RCS files, and this information is needed to locate the corresponding change in the old CVS repo. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Wed Feb 15 20:32:33 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Wed, 15 Feb 2012 20:32:33 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Tue, 14 Feb 2012 21:27:31 +0100") References: Message-ID: nisse at lysator.liu.se (Niels M?ller) writes: > Yet another try, and unless I discover any problems with this soon, it's > going to be the final one. Turns out I had broken the handling of deleted files. So another try, git://git.lysator.liu.se/lsh-test/nettle-test-4.git Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Sat Feb 18 22:19:36 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sat, 18 Feb 2012 22:19:36 +0100 Subject: Migration from CVS to GIT In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Wed, 15 Feb 2012 20:32:33 +0100") References: Message-ID: I think the conversion can be declared final now. New repo at git://git.lysator.liu.se/nettle/nettle.git I have fixed the Makefiles so that they work in the new setting. After cloning, you need to run the ./.bootstrap script, then ./configure && make as usual. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From yyy at yyy.id.lv Sun Mar 18 11:42:05 2012 From: yyy at yyy.id.lv (yyy) Date: Sun, 18 Mar 2012 12:42:05 +0200 Subject: GNU MP not found (mingw) Message-ID: <504866837.20120318124205@yyy.id.lv> Configure fails to find gmp GMP version 5.0.4 nettle version 2.4 mingw on windows xp sp3 config.log did not contain any recognizable links to missing files (there were multiple references to "-lgmp", but it was not a file) (entire config.log is ~35KB, should I post it here? Compiling and building GMP was successful, it produced file libgmp.a with size of 648 236 bytes. Leaving that file in default lib directory, did not work. Putting it in nettle directory did not work either. Renaming that file to lgmp.a, libgmp.a.dll or lgmp.a.dll and copying to nettle directory did not work either. Where nettle's configure script is looking for lgmp and what is the required filename? Could it be, that nettle 2.4 does not work with GMP 5.0.4 and requires older version? From simon at josefsson.org Thu Mar 22 08:57:14 2012 From: simon at josefsson.org (Simon Josefsson) Date: Thu, 22 Mar 2012 08:57:14 +0100 Subject: Salsa20 Message-ID: <87sjh1xe3p.fsf@latte.josefsson.org> Hi! Please find attached a port of DJB's public domain code for Salsa20 to nettle. The patch is not meant to be applied as-is but to start a discussion. Review of the code itself is welcome. While writing this I noticed a sub-optimal design of Nettle, in that it overloads the meaning of struct nettle_cipher->block_size as a flag of whether the cipher is a stream cipher or not. Before arcfour was the only stream cipher in nettle, and using block_size=0 to signal that it is a stream cipher worked fine. However, Salsa20 has a block size of 8 bytes. Another odd thing is that Salsa20 requires an IV. So some changes to the nettle_cipher struct are required. This could be resolved better than I have done in the patch below. I see two reasonable alternatives: 1) Introduce a new struct 'struct nettle_stream_cipher' that applies to stream ciphers and use that. 2) Add a flag in 'struct nettle_cipher' to tell whether the cipher is a stream cipher or not. I started out strongly prefering 1) because it appeared cleaner. After thinking about the changes required to implement that, I am now leaning towards 2) because of: A) a lot of code would have to be duplicated if we have both 'struct nettle_cipher' and 'struct nettle_stream_cipher' for apparently little gain. Otherwise, most code could be the same, and where the distinction matters there could be a simple if-test. B) it allows for full backwards compatibility in a clean way. With 1) we would have a transition period where people switch arcfour usage from nettle_cipher to nettle_stream_cipher. With 2) arcfour would continue to have block_size==0 but it would also set the new stream cipher flag. External code could incrementally be updated to check the new flag instead of block_size==0 (if it does that at all) but that is something that doesn't uglify nettle. What do you think? I'd be happy to propose a patch to implement 2) and then a follow-on patch to add support for Salsa20. /Simon >From 600ae2815634ad95c4d2d93821316ce20df19ae3 Mon Sep 17 00:00:00 2001 From: Simon Josefsson Date: Wed, 21 Mar 2012 17:06:13 +0100 Subject: [PATCH] Implement Salsa20. --- Makefile.in | 3 +- examples/nettle-benchmark.c | 3 + nettle-meta-ciphers.c | 2 + nettle-meta.h | 8 ++- nettle-types.h | 5 + salsa20-meta.c | 49 ++++++++++++ salsa20.c | 172 ++++++++++++++++++++++++++++++++++++++++++ salsa20.h | 71 +++++++++++++++++ testsuite/.gitignore | 1 + testsuite/.test-rules.make | 3 + testsuite/Makefile.in | 1 + testsuite/meta-cipher-test.c | 2 + testsuite/salsa20-test.c | 49 ++++++++++++ testsuite/testutils.c | 36 +++++++-- testsuite/testutils.h | 10 +++ 15 files changed, 407 insertions(+), 8 deletions(-) create mode 100644 salsa20-meta.c create mode 100644 salsa20.c create mode 100644 salsa20.h create mode 100644 testsuite/salsa20-test.c diff --git a/Makefile.in b/Makefile.in index 1813a0d..6e2ec60 100644 --- a/Makefile.in +++ b/Makefile.in @@ -79,6 +79,7 @@ nettle_SOURCES = aes-decrypt-internal.c aes-decrypt.c \ md2.c md2-meta.c md4.c md4-meta.c \ md5.c md5-compress.c md5-compat.c md5-meta.c \ ripemd160.c ripemd160-compress.c ripemd160-meta.c \ + salsa20.c salsa20-meta.c \ sha1.c sha1-compress.c sha1-meta.c \ sha256.c sha256-compress.c sha224-meta.c sha256-meta.c \ sha512.c sha512-compress.c sha384-meta.c sha512-meta.c \ @@ -113,7 +114,7 @@ hogweed_SOURCES = sexp.c sexp-format.c \ pgp-encode.c rsa2openpgp.c \ der-iterator.c der2rsa.c der2dsa.c -HEADERS = aes.h arcfour.h arctwo.h asn1.h bignum.h blowfish.h \ +HEADERS = aes.h arcfour.h salsa20.h arctwo.h asn1.h bignum.h blowfish.h \ base16.h base64.h buffer.h camellia.h cast128.h \ cbc.h ctr.h gcm.h \ des.h des-compat.h dsa.h \ diff --git a/examples/nettle-benchmark.c b/examples/nettle-benchmark.c index 424ef19..44fca4e 100644 --- a/examples/nettle-benchmark.c +++ b/examples/nettle-benchmark.c @@ -40,6 +40,7 @@ #include "aes.h" #include "arcfour.h" +#include "salsa20.h" #include "blowfish.h" #include "cast128.h" #include "cbc.h" @@ -612,6 +613,8 @@ main(int argc, char **argv) OPENSSL(&nettle_openssl_aes192) OPENSSL(&nettle_openssl_aes256) &nettle_arcfour128, OPENSSL(&nettle_openssl_arcfour128) + &nettle_salsa20_128, + &nettle_salsa20_256, &nettle_blowfish128, OPENSSL(&nettle_openssl_blowfish128) &nettle_camellia128, &nettle_camellia192, &nettle_camellia256, &nettle_cast128, OPENSSL(&nettle_openssl_cast128) diff --git a/nettle-meta-ciphers.c b/nettle-meta-ciphers.c index 1f07595..aae90f9 100644 --- a/nettle-meta-ciphers.c +++ b/nettle-meta-ciphers.c @@ -36,6 +36,8 @@ const struct nettle_cipher * const nettle_ciphers[] = { &nettle_camellia192, &nettle_camellia256, &nettle_cast128, + &nettle_salsa20_128, + &nettle_salsa20_256, &nettle_serpent128, &nettle_serpent192, &nettle_serpent256, diff --git a/nettle-meta.h b/nettle-meta.h index 54cbbd2..c8aabda 100644 --- a/nettle-meta.h +++ b/nettle-meta.h @@ -5,6 +5,7 @@ /* nettle, low-level cryptographics library * + * Copyright (C) 2012 Simon Josefsson * Copyright (C) 2002 Niels M?ller * * The nettle library is free software; you can redistribute it and/or modify @@ -39,7 +40,7 @@ struct nettle_cipher unsigned context_size; - /* Zero for stream ciphers */ + /* Block size; 0 for byte-oriented stream ciphers. */ unsigned block_size; /* Suggested key size; other sizes are sometimes possible. */ @@ -50,6 +51,8 @@ struct nettle_cipher nettle_crypt_func *encrypt; nettle_crypt_func *decrypt; + + nettle_set_iv_func *set_iv; }; #define _NETTLE_CIPHER(name, NAME, key_size) { \ @@ -105,6 +108,9 @@ extern const struct nettle_cipher nettle_aes256; extern const struct nettle_cipher nettle_arcfour128; +extern const struct nettle_cipher nettle_salsa20_128; +extern const struct nettle_cipher nettle_salsa20_256; + extern const struct nettle_cipher nettle_camellia128; extern const struct nettle_cipher nettle_camellia192; extern const struct nettle_cipher nettle_camellia256; diff --git a/nettle-types.h b/nettle-types.h index b694332..e68cacb 100644 --- a/nettle-types.h +++ b/nettle-types.h @@ -2,6 +2,7 @@ /* nettle, low-level cryptographics library * + * Copyright (C) 2012 Simon Josefsson * Copyright (C) 2005 Niels M?ller * * The nettle library is free software; you can redistribute it and/or modify @@ -44,6 +45,10 @@ typedef void nettle_set_key_func(void *ctx, unsigned length, const uint8_t *key); +typedef void nettle_set_iv_func(void *ctx, + unsigned length, + const uint8_t *iv); + /* Uses a void * for cipher contexts. For block ciphers it would make sense with a const void * for the diff --git a/salsa20-meta.c b/salsa20-meta.c new file mode 100644 index 0000000..8278e09 --- /dev/null +++ b/salsa20-meta.c @@ -0,0 +1,49 @@ +/* salsa20-meta.c */ + +/* nettle, low-level cryptographics library + * + * Copyright (C) 2012 Simon Josefsson + * + * The nettle library is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or (at your + * option) any later version. + * + * The nettle library is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public + * License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the nettle library; see the file COPYING.LIB. If not, write to + * the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, + * MA 02111-1307, USA. + */ + +#if HAVE_CONFIG_H +# include "config.h" +#endif + +#include "nettle-meta.h" + +#include "salsa20.h" + +const struct nettle_cipher nettle_salsa20_128 = + { "salsa20-128", sizeof(struct salsa20_ctx), + 8, 16, + (nettle_set_key_func *) salsa20_set_key, + (nettle_set_key_func *) salsa20_set_key, + (nettle_crypt_func *) salsa20_crypt, + (nettle_crypt_func *) salsa20_crypt, + (nettle_set_iv_func *) salsa20_set_iv + }; + +const struct nettle_cipher nettle_salsa20_256 = + { "salsa20-256", sizeof(struct salsa20_ctx), + 8, 32, + (nettle_set_key_func *) salsa20_set_key, + (nettle_set_key_func *) salsa20_set_key, + (nettle_crypt_func *) salsa20_crypt, + (nettle_crypt_func *) salsa20_crypt, + (nettle_set_iv_func *) salsa20_set_iv + }; diff --git a/salsa20.c b/salsa20.c new file mode 100644 index 0000000..7885199 --- /dev/null +++ b/salsa20.c @@ -0,0 +1,172 @@ +/* salsa20.c + * + * The Salsa20 stream cipher. + */ + +/* nettle, low-level cryptographics library + * + * Copyright (C) 2012 Simon Josefsson + * + * The nettle library is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or (at your + * option) any later version. + * + * The nettle library is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public + * License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the nettle library; see the file COPYING.LIB. If not, write to + * the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, + * MA 02111-1307, USA. + */ + +#if HAVE_CONFIG_H +# include "config.h" +#endif + +#include + +#include "salsa20.h" + +#define ROTL32(x,n) ((((x))<<(n)) | (((x))>>(32-(n)))) + +#define SWAP32(v) \ + ((ROTL32(v, 8) & 0x00FF00FFUL) | \ + (ROTL32(v, 24) & 0xFF00FF00UL)) + +#ifdef WORDS_BIGENDIAN +#define U32TO32_LITTLE(v) SWAP32(v) +#else +#define U32TO32_LITTLE(v) (v) +#endif + +#define U8TO32_LITTLE(p) U32TO32_LITTLE(((uint32_t*)(p))[0]) +#define U32TO8_LITTLE(p, v) (((uint32_t*)(p))[0] = U32TO32_LITTLE(v)) + +/* +salsa20-ref.c version 20051118 +D. J. Bernstein +Public domain. +*/ + +#define ROTATE(v,c) (ROTL32(v,c)) +#define XOR(v,w) ((v) ^ (w)) +#define PLUS(v,w) ((v) + (w)) +#define PLUSONE(v) (PLUS((v),1)) + +static void salsa20_wordtobyte(uint8_t output[64],const uint32_t input[16]) +{ + uint32_t x[16]; + int i; + + for (i = 0;i < 16;++i) x[i] = input[i]; + for (i = 20;i > 0;i -= 2) { + x[ 4] = XOR(x[ 4],ROTATE(PLUS(x[ 0],x[12]), 7)); + x[ 8] = XOR(x[ 8],ROTATE(PLUS(x[ 4],x[ 0]), 9)); + x[12] = XOR(x[12],ROTATE(PLUS(x[ 8],x[ 4]),13)); + x[ 0] = XOR(x[ 0],ROTATE(PLUS(x[12],x[ 8]),18)); + x[ 9] = XOR(x[ 9],ROTATE(PLUS(x[ 5],x[ 1]), 7)); + x[13] = XOR(x[13],ROTATE(PLUS(x[ 9],x[ 5]), 9)); + x[ 1] = XOR(x[ 1],ROTATE(PLUS(x[13],x[ 9]),13)); + x[ 5] = XOR(x[ 5],ROTATE(PLUS(x[ 1],x[13]),18)); + x[14] = XOR(x[14],ROTATE(PLUS(x[10],x[ 6]), 7)); + x[ 2] = XOR(x[ 2],ROTATE(PLUS(x[14],x[10]), 9)); + x[ 6] = XOR(x[ 6],ROTATE(PLUS(x[ 2],x[14]),13)); + x[10] = XOR(x[10],ROTATE(PLUS(x[ 6],x[ 2]),18)); + x[ 3] = XOR(x[ 3],ROTATE(PLUS(x[15],x[11]), 7)); + x[ 7] = XOR(x[ 7],ROTATE(PLUS(x[ 3],x[15]), 9)); + x[11] = XOR(x[11],ROTATE(PLUS(x[ 7],x[ 3]),13)); + x[15] = XOR(x[15],ROTATE(PLUS(x[11],x[ 7]),18)); + x[ 1] = XOR(x[ 1],ROTATE(PLUS(x[ 0],x[ 3]), 7)); + x[ 2] = XOR(x[ 2],ROTATE(PLUS(x[ 1],x[ 0]), 9)); + x[ 3] = XOR(x[ 3],ROTATE(PLUS(x[ 2],x[ 1]),13)); + x[ 0] = XOR(x[ 0],ROTATE(PLUS(x[ 3],x[ 2]),18)); + x[ 6] = XOR(x[ 6],ROTATE(PLUS(x[ 5],x[ 4]), 7)); + x[ 7] = XOR(x[ 7],ROTATE(PLUS(x[ 6],x[ 5]), 9)); + x[ 4] = XOR(x[ 4],ROTATE(PLUS(x[ 7],x[ 6]),13)); + x[ 5] = XOR(x[ 5],ROTATE(PLUS(x[ 4],x[ 7]),18)); + x[11] = XOR(x[11],ROTATE(PLUS(x[10],x[ 9]), 7)); + x[ 8] = XOR(x[ 8],ROTATE(PLUS(x[11],x[10]), 9)); + x[ 9] = XOR(x[ 9],ROTATE(PLUS(x[ 8],x[11]),13)); + x[10] = XOR(x[10],ROTATE(PLUS(x[ 9],x[ 8]),18)); + x[12] = XOR(x[12],ROTATE(PLUS(x[15],x[14]), 7)); + x[13] = XOR(x[13],ROTATE(PLUS(x[12],x[15]), 9)); + x[14] = XOR(x[14],ROTATE(PLUS(x[13],x[12]),13)); + x[15] = XOR(x[15],ROTATE(PLUS(x[14],x[13]),18)); + } + for (i = 0;i < 16;++i) x[i] = PLUS(x[i],input[i]); + for (i = 0;i < 16;++i) U32TO8_LITTLE(output + 4 * i,x[i]); +} + +static const char sigma[16] = "expand 32-byte k"; +static const char tau[16] = "expand 16-byte k"; + +void +salsa20_set_key(struct salsa20_ctx *ctx, + unsigned length, const uint8_t *key) +{ + const char *constants; + + assert (length == SALSA20_MIN_KEY_SIZE || length == SALSA20_MAX_KEY_SIZE); + + ctx->input[1] = U8TO32_LITTLE(key + 0); + ctx->input[2] = U8TO32_LITTLE(key + 4); + ctx->input[3] = U8TO32_LITTLE(key + 8); + ctx->input[4] = U8TO32_LITTLE(key + 12); + if (length == SALSA20_MAX_KEY_SIZE) { /* recommended */ + key += 16; + constants = sigma; + } else { /* kbits == 128 */ + constants = tau; + } + ctx->input[11] = U8TO32_LITTLE(key + 0); + ctx->input[12] = U8TO32_LITTLE(key + 4); + ctx->input[13] = U8TO32_LITTLE(key + 8); + ctx->input[14] = U8TO32_LITTLE(key + 12); + ctx->input[0] = U8TO32_LITTLE(constants + 0); + ctx->input[5] = U8TO32_LITTLE(constants + 4); + ctx->input[10] = U8TO32_LITTLE(constants + 8); + ctx->input[15] = U8TO32_LITTLE(constants + 12); +} + +void +salsa20_set_iv(struct salsa20_ctx *ctx, unsigned length, const uint8_t *iv) +{ + assert (length == SALSA20_IV_SIZE); + + ctx->input[6] = U8TO32_LITTLE(iv + 0); + ctx->input[7] = U8TO32_LITTLE(iv + 4); + ctx->input[8] = 0; + ctx->input[9] = 0; +} + +void +salsa20_crypt(struct salsa20_ctx *ctx, + unsigned length, + uint8_t *c, + const uint8_t *m) +{ + uint8_t output[64]; + unsigned i; + + if (!length) return; + for (;;) { + salsa20_wordtobyte(output,ctx->input); + ctx->input[8] = PLUSONE(ctx->input[8]); + if (!ctx->input[8]) { + ctx->input[9] = PLUSONE(ctx->input[9]); + /* stopping at 2^70 length per nonce is user's responsibility */ + } + if (length <= 64) { + for (i = 0;i < length;++i) c[i] = m[i] ^ output[i]; + return; + } + for (i = 0;i < 64;++i) c[i] = m[i] ^ output[i]; + length -= 64; + c += 64; + m += 64; + } +} diff --git a/salsa20.h b/salsa20.h new file mode 100644 index 0000000..79f1505 --- /dev/null +++ b/salsa20.h @@ -0,0 +1,71 @@ +/* salsa20.h + * + * The Salsa20 stream cipher. + */ + +/* nettle, low-level cryptographics library + * + * Copyright (C) 2012 Simon Josefsson + * Copyright (C) 2001 Niels M?ller + * + * The nettle library is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or (at your + * option) any later version. + * + * The nettle library is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public + * License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the nettle library; see the file COPYING.LIB. If not, write to + * the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, + * MA 02111-1307, USA. + */ + +#ifndef NETTLE_SALSA20_H_INCLUDED +#define NETTLE_SALSA20_H_INCLUDED + +#include "nettle-types.h" + +#ifdef __cplusplus +extern "C" { +#endif + +/* Name mangling */ +#define salsa20_set_key nettle_salsa20_set_key +#define salsa20_set_iv nettle_salsa20_set_iv +#define salsa20_crypt nettle_salsa20_crypt + +/* Minimum and maximum keysizes, and a reasonable default. In + * octets.*/ +#define SALSA20_MIN_KEY_SIZE 16 +#define SALSA20_MAX_KEY_SIZE 32 +#define SALSA20_KEY_SIZE 32 + +#define SALSA20_IV_SIZE 8 + +struct salsa20_ctx +{ + uint32_t input[16]; +}; + +void +salsa20_set_key(struct salsa20_ctx *ctx, + unsigned length, const uint8_t *key); + +void +salsa20_set_iv(struct salsa20_ctx *ctx, + unsigned length, const uint8_t *iv); + +void +salsa20_crypt(struct salsa20_ctx *ctx, + unsigned length, uint8_t *dst, + const uint8_t *src); + +#ifdef __cplusplus +} +#endif + +#endif /* NETTLE_SALSA20_H_INCLUDED */ diff --git a/testsuite/.gitignore b/testsuite/.gitignore index 1147cfb..c9f4698 100644 --- a/testsuite/.gitignore +++ b/testsuite/.gitignore @@ -36,6 +36,7 @@ /rsa-keygen-test /rsa-test /rsa2sexp-test +/salsa20-test /serpent-test /sexp-format-test /sexp-test diff --git a/testsuite/.test-rules.make b/testsuite/.test-rules.make index e8f6650..10c993f 100644 --- a/testsuite/.test-rules.make +++ b/testsuite/.test-rules.make @@ -49,6 +49,9 @@ memxor-test$(EXEEXT): memxor-test.$(OBJEXT) ripemd160-test$(EXEEXT): ripemd160-test.$(OBJEXT) $(LINK) ripemd160-test.$(OBJEXT) $(TEST_OBJS) -o ripemd160-test$(EXEEXT) +salsa20-test$(EXEEXT): salsa20-test.$(OBJEXT) + $(LINK) salsa20-test.$(OBJEXT) $(TEST_OBJS) -o salsa20-test$(EXEEXT) + sha1-test$(EXEEXT): sha1-test.$(OBJEXT) $(LINK) sha1-test.$(OBJEXT) $(TEST_OBJS) -o sha1-test$(EXEEXT) diff --git a/testsuite/Makefile.in b/testsuite/Makefile.in index 8dfc62b..10ffae9 100644 --- a/testsuite/Makefile.in +++ b/testsuite/Makefile.in @@ -18,6 +18,7 @@ TS_NETTLE_SOURCES = aes-test.c arcfour-test.c arctwo-test.c \ md2-test.c md4-test.c md5-test.c md5-compat-test.c \ memxor-test.c \ ripemd160-test.c \ + salsa20-test.c \ sha1-test.c sha224-test.c sha256-test.c \ sha384-test.c sha512-test.c \ serpent-test.c twofish-test.c \ diff --git a/testsuite/meta-cipher-test.c b/testsuite/meta-cipher-test.c index 1bb74d8..50a02ba 100644 --- a/testsuite/meta-cipher-test.c +++ b/testsuite/meta-cipher-test.c @@ -14,6 +14,8 @@ const char* ciphers[] = { "camellia192", "camellia256", "cast128", + "salsa20-128", + "salsa20-256", "serpent128", "serpent192", "serpent256", diff --git a/testsuite/salsa20-test.c b/testsuite/salsa20-test.c new file mode 100644 index 0000000..e1c13dd --- /dev/null +++ b/testsuite/salsa20-test.c @@ -0,0 +1,49 @@ +#include "testutils.h" +#include "salsa20.h" + +int +test_main(void) +{ + /* http://www.ecrypt.eu.org/stream/svn/viewcvs.cgi/ecrypt/trunk/submissions/salsa20/full/verified.test-vectors?logsort=rev&rev=210&view=markup */ + + test_cipher_iv(&nettle_salsa20_128, + HL("80000000 00000000 00000000 00000000"), + HL("00000000 00000000"), + HL("00000000 00000000"), + H("4DFA5E48 1DA23EA0")); + + test_cipher_iv(&nettle_salsa20_128, + HL("00000000 00000000 00000000 00000000"), + HL("80000000 00000000"), + HL("00000000 00000000"), + H("B66C1E44 46DD9557")); + + test_cipher_iv(&nettle_salsa20_128, + HL("0053A6F94C9FF24598EB3E91E4378ADD"), + HL("0D74DB42A91077DE"), + HL("00000000 00000000"), + H("05E1E7BE B697D999")); + + test_cipher_iv(&nettle_salsa20_256, + HL("80000000 00000000 00000000 00000000" + "00000000 00000000 00000000 00000000"), + HL("00000000 00000000"), + HL("00000000 00000000"), + H("E3BE8FDD 8BECA2E3")); + + test_cipher_iv(&nettle_salsa20_256, + HL("00000000 00000000 00000000 00000000" + "00000000 00000000 00000000 00000000"), + HL("80000000 00000000"), + HL("00000000 00000000"), + H("2ABA3DC45B494700")); + + test_cipher_iv(&nettle_salsa20_256, + HL("0053A6F94C9FF24598EB3E91E4378ADD" + "3083D6297CCF2275C81B6EC11467BA0D"), + HL("0D74DB42A91077DE"), + HL("00000000 00000000"), + H("F5FAD53F 79F9DF58")); + + SUCCESS(); +} diff --git a/testsuite/testutils.c b/testsuite/testutils.c index d77bb7e..8f5dab0 100644 --- a/testsuite/testutils.c +++ b/testsuite/testutils.c @@ -162,17 +162,21 @@ main(int argc, char **argv) } void -test_cipher(const struct nettle_cipher *cipher, - unsigned key_length, - const uint8_t *key, - unsigned length, - const uint8_t *cleartext, - const uint8_t *ciphertext) +test_cipher_iv(const struct nettle_cipher *cipher, + unsigned key_length, + const uint8_t *key, + unsigned iv_length, + const uint8_t *iv, + unsigned length, + const uint8_t *cleartext, + const uint8_t *ciphertext) { void *ctx = xalloc(cipher->context_size); uint8_t *data = xalloc(length); cipher->set_encrypt_key(ctx, key_length, key); + if (iv_length > 0 && iv) + cipher->set_iv(ctx, iv_length, iv); cipher->encrypt(ctx, length, data, cleartext); if (!MEMEQ(length, data, ciphertext)) @@ -187,6 +191,8 @@ test_cipher(const struct nettle_cipher *cipher, FAIL(); } cipher->set_decrypt_key(ctx, key_length, key); + if (iv_length > 0 && iv) + cipher->set_iv(ctx, iv_length, iv); cipher->decrypt(ctx, length, data, data); if (!MEMEQ(length, data, cleartext)) @@ -206,6 +212,24 @@ test_cipher(const struct nettle_cipher *cipher, } void +test_cipher(const struct nettle_cipher *cipher, + unsigned key_length, + const uint8_t *key, + unsigned length, + const uint8_t *cleartext, + const uint8_t *ciphertext) +{ + test_cipher_iv(cipher, + key_length, + key, + 0, + NULL, + length, + cleartext, + ciphertext); +} + +void test_cipher_cbc(const struct nettle_cipher *cipher, unsigned key_length, const uint8_t *key, diff --git a/testsuite/testutils.h b/testsuite/testutils.h index d7ced9a..9d13dff 100644 --- a/testsuite/testutils.h +++ b/testsuite/testutils.h @@ -91,6 +91,16 @@ test_cipher(const struct nettle_cipher *cipher, const uint8_t *ciphertext); void +test_cipher_iv(const struct nettle_cipher *cipher, + unsigned key_length, + const uint8_t *key, + unsigned iv_length, + const uint8_t *iv, + unsigned length, + const uint8_t *cleartext, + const uint8_t *ciphertext); + +void test_cipher_cbc(const struct nettle_cipher *cipher, unsigned key_length, const uint8_t *key, -- 1.7.2.5 From nisse at lysator.liu.se Thu Mar 29 11:39:56 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Thu, 29 Mar 2012 11:39:56 +0200 Subject: GNU MP not found (mingw) In-Reply-To: <504866837.20120318124205@yyy.id.lv> (yyy@yyy.id.lv's message of "Sun, 18 Mar 2012 12:42:05 +0200") References: <504866837.20120318124205@yyy.id.lv> Message-ID: Sorry for the late reply. yyy writes: > config.log did not contain any recognizable links to missing files > (there were multiple references to "-lgmp", but it was not a file) > (entire config.log is ~35KB, should I post it here? In the config.log there should be a line like configure:6551: checking for __gmpz_getlimbn in -lgmp (possibly with a different line number). If that test fails, you should find the test program, the compiler command line, and error messages, close by. Have you tried writing any testprogram of your own to link to gmp? One possible issue is if you have compiled gmp in 64-bit mode (gmp's configure does this by default, whenever possible), but you're compiling nettle in 32-bit mode. If so, try ./configure CC="gcc -m64" when building nettle. > Compiling and building GMP was successful, it produced file libgmp.a > with size of 648 236 bytes. So it's a static library. May make things simpler. > Where nettle's configure script is looking for lgmp and what is the > required filename? The answer to those questions are "whatever the used compiler and linker does.". To get nettle's configure to look in different directories for libraries (and include files! It also needs gmp.h), either use the -with-library-path and --with-include-path flags, or set LDFLAGS and CPPFLAGS on the configure command line. > Could it be, that nettle 2.4 does not work with > GMP 5.0.4 and requires older version? I think it should work fine with the most recent gmp version. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Thu Mar 29 12:30:59 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Thu, 29 Mar 2012 12:30:59 +0200 Subject: Salsa20 In-Reply-To: <87sjh1xe3p.fsf@latte.josefsson.org> (Simon Josefsson's message of "Thu, 22 Mar 2012 08:57:14 +0100") References: <87sjh1xe3p.fsf@latte.josefsson.org> Message-ID: Simon Josefsson writes: > Hi! Please find attached a port of DJB's public domain code for Salsa20 > to nettle. The patch is not meant to be applied as-is but to start a > discussion. I'm not familiar with this cipher... Could you explain briefly why one may want to use it? > While writing this I noticed a sub-optimal design of Nettle, in that it > overloads the meaning of struct nettle_cipher->block_size as a flag of > whether the cipher is a stream cipher or not. Before arcfour was the > only stream cipher in nettle, and using block_size=0 to signal that it > is a stream cipher worked fine. However, Salsa20 has a block size of 8 > bytes. If Salsa20 doesn't fit in the nettle_cipher abstraction, then I think we should start by implementing Salsa20 *without* trying to get it to fit there. To me, a "stream cipher" with a block size and iv seems a bit like a contradiction. In general, I don't think it makes much sense to include algorithms which behave very differently under the "nettle_cipher" umbrella, since the point of that abstraction is to collect algorithms which are easily exchangable. Code accepting a nettle_cipher are not expected to check flags or behave fundamentally different for different algorithms. Including the arcfour stream cipher there may well be a mistake. One annoying problem with the current design is that for block ciphers, the ctx argument of the encrypt and decrypt functions naturally is const, but due arcfour being fittted in the same nettle_cipher abstraction, the function typedef nettle_crypt_func can't use a const ctx. Anyway, I'd prefer to treat redesign in that area as an independent problem from adding support for Salsa20. Then there's also the tentative (nettle-internal.h) nettle_aead abstraction. Salsa20 could perhaps fit there, if we allow algorithms with no authentication (NULL digest function pointer). > What do you think? I'd be happy to propose a patch to implement 2) and > then a follow-on patch to add support for Salsa20. I'd prefer the opposite order... Some quick comments on the code and algorithm below. This is the first I hear about it, so I may well be misunderstanding how it is supposed to work. > --- /dev/null > +++ b/salsa20.c > +#define ROTL32(x,n) ((((x))<<(n)) | (((x))>>(32-(n)))) There are several different variants of that macro. It would be nice with a unified one in macros.h. > +#define SWAP32(v) \ > + ((ROTL32(v, 8) & 0x00FF00FFUL) | \ > + (ROTL32(v, 24) & 0xFF00FF00UL)) > + > +#ifdef WORDS_BIGENDIAN > +#define U32TO32_LITTLE(v) SWAP32(v) > +#else > +#define U32TO32_LITTLE(v) (v) > +#endif > + > +#define U8TO32_LITTLE(p) U32TO32_LITTLE(((uint32_t*)(p))[0]) > +#define U32TO8_LITTLE(p, v) (((uint32_t*)(p))[0] = U32TO32_LITTLE(v)) That's a clever byte swapping trick (at least if a true rot instruction is available). In Nettle conversion between bytes and integers are usually done with READ_UINT32, LE_READ_UINT32 and friends. It's usually not very performance critical, and it deals naturally with any alignment. I suspect U8TO32_LITTLE above breaks if the input is unaligned but the architecture doesn't allow unaligned word reads. > +/* > +salsa20-ref.c version 20051118 > +D. J. Bernstein > +Public domain. > +*/ Shouldn't that be moved up closer to the (C) header? > +#define ROTATE(v,c) (ROTL32(v,c)) > +#define XOR(v,w) ((v) ^ (w)) > +#define PLUS(v,w) ((v) + (w)) > +#define PLUSONE(v) (PLUS((v),1)) I'm not sure these macros improve readability... > +static void salsa20_wordtobyte(uint8_t output[64],const uint32_t input[16]) Despite the name, this is the main encryption function. Or rather, a (hopefully) one-way function? "input" is really the context struct, which contains some mix of the key and the iv? > + x[ 4] = XOR(x[ 4],ROTATE(PLUS(x[ 0],x[12]), 7)); And what's wrong with writing that as x[ 4] ^= ROTATE(x[ 0] + x[12], 7) > +void > +salsa20_set_key(struct salsa20_ctx *ctx, > + unsigned length, const uint8_t *key) > +{ Ok, this initializes the input arrray at indices 0-5, and 10-15. > +void > +salsa20_set_iv(struct salsa20_ctx *ctx, unsigned length, const uint8_t *iv) > +{ > + assert (length == SALSA20_IV_SIZE); And this initializes indices 6-9. I'd remove the length argument, if only one value is allowed anyway. > +void > +salsa20_crypt(struct salsa20_ctx *ctx, > + unsigned length, > + uint8_t *c, > + const uint8_t *m) > +{ > + uint8_t output[64]; > + unsigned i; > + > + if (!length) return; > + for (;;) { > + salsa20_wordtobyte(output,ctx->input); That's applying a oneway function to the input, > + ctx->input[8] = PLUSONE(ctx->input[8]); > + if (!ctx->input[8]) { > + ctx->input[9] = PLUSONE(ctx->input[9]); > + /* stopping at 2^70 length per nonce is user's responsibility */ > + } followed by an increment of the iv, where the second half appearantly is used as a counter. > + if (length <= 64) { > + for (i = 0;i < length;++i) c[i] = m[i] ^ output[i]; > + return; > + } > + for (i = 0;i < 64;++i) c[i] = m[i] ^ output[i]; Followed by xoring the generated key stream to get the ciphertext. Do I understand it correctly that there's nothing really non-linear anywhere in this cipher? Only a mix of + and ^ which both are linear, but over different rings. So the security aginst a known plaintext attack would rest on the carry propagation, together with the large state space of 512 bits (448 bits of expanded key material, and 64 bits counter). Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Thu Mar 29 16:27:03 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Thu, 29 Mar 2012 16:27:03 +0200 Subject: Salsa20 In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Thu, 29 Mar 2012 12:30:59 +0200") References: <87sjh1xe3p.fsf@latte.josefsson.org> Message-ID: nisse at lysator.liu.se (Niels M?ller) writes: > I'm not familiar with this cipher... Could you explain briefly why one > may want to use it? I've done some more homework, reading some of djb's papers and the wikipedia page. Some questions: 1. Any protocols that specify use of salsa20? And in combination with which MAC? With claimed cycle numbers, hmac with sha1 or sha2 is going to take much more time than the encryption. 2. What about the later "ChaCha" cipher? As far as I see, it has not been studied as much as Salsa20. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Thu Mar 29 22:20:36 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Thu, 29 Mar 2012 22:20:36 +0200 Subject: Salsa20 In-Reply-To: <87sjh1xe3p.fsf@latte.josefsson.org> (Simon Josefsson's message of "Thu, 22 Mar 2012 08:57:14 +0100") References: <87sjh1xe3p.fsf@latte.josefsson.org> Message-ID: Simon Josefsson writes: > Please find attached a port of DJB's public domain code for Salsa20 > to nettle. The patch is not meant to be applied as-is but to start a > discussion. Nevertheless, I have now commited salsa20.[ch] and the testcase, postponing remaining work for later. ;-) Needed: * Support in nettle-benchmark in some way (can probably get away with a stupid nettle_cipher setting the iv to some fixed value). * A bit of cleanup of the code. * Try optimization of the C code for 64-bit machines. One ought to be able to do two column or row operations in parallel by putting two salsa20 words into a uint64_t variable. May need some tricks to avoid carry propagation between the words, but I suspect it may be a win due to lower register pressure. A bit similar to the HAVE_NATIVE_64_BIT in camellia-crypt-internal.c. * Try an sse2 assembly implementation (the djb:s papers outline how to do that). Or copy some existing implementation. * Documentation. As for the salsa20_* interface: * It would be possible to change the interface to not expose the block size, doing a little buffering instead. But I think it's better to not do that, and follow what the ctr code and gcm code does. * One advertised feature of the cipher is random access. I think we should have something like a salsa20_set_pos, taking a block count as argument. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Fri Mar 30 22:12:24 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Fri, 30 Mar 2012 22:12:24 +0200 Subject: Salsa20 In-Reply-To: <87sjh1xe3p.fsf@latte.josefsson.org> (Simon Josefsson's message of "Thu, 22 Mar 2012 08:57:14 +0100") References: <87sjh1xe3p.fsf@latte.josefsson.org> Message-ID: And now I've added support for salsa20 benchmarking as well. Performance is pretty good, even for this reference implementation. Seems to run at 12 cycles per byte on my laptop, a little faster than 128-bit aes. To get that working, I added a very kludgy const struct nettle_cipher nettle_salsa20 = {...}; to nettle-internal.c, for benchmarking only. I also arranged in the Makefile so that nettle-internal.o no longer is included in the nettle library. Will that cause any trouble for anybody? Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Sun Apr 1 00:13:20 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sun, 01 Apr 2012 00:13:20 +0200 Subject: Salsa20 In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Fri, 30 Mar 2012 22:12:24 +0200") References: <87sjh1xe3p.fsf@latte.josefsson.org> Message-ID: nisse at lysator.liu.se (Niels M?ller) writes: > And now I've added support for salsa20 benchmarking as well. Performance > is pretty good, even for this reference implementation. Seems to run at > 12 cycles per byte on my laptop, a little faster than 128-bit aes. Now I've done some cleanup and reorganization work. Using memxor3 for the actual xoring saves almost 3 cycles per byte, so it's a considerable speedup in this case (and then memxor3 can maybe be sped up a bit more on this platform, from Nikos' work a while ago). I wonder if there's any potential for simd-style parallel processing using plain 64-bit operations (no sse2 assembly). If one puts two 32-bit values in a 64-bit variable, parallel xor is trivial, but we also need 32-bit rotations and adds. Parallel rotations are possible as two shifts and some extra masking, so it might be slightly faster than two 32-bit rotations of two shifts each. For parallel mod 2^32 additions, one need to cancel one of the carries. Something like uint64_t x, y, z; z = x + y; z += (- (uint64_t) ((uint32_t) z < (uint32_t) y)) << 32; This can hardly be faster than two 32-bit additions (two lea, cmp, cmov on x86_64, or alternatively, one lea, cmp, sbb, shl, add, depending on code genereration). Any speedup depends on the relative speedup or slowdown of each primitive, the cost of permuting values as needed, and then the effects of reduced register pressure on top of that. For x86_64 (and maybe x86_32), assembly implementation also seems attractive, but I'd like to do any general or 64-bit specific optimizations in the C code first. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nmav at gnutls.org Sun Apr 1 20:42:01 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sun, 01 Apr 2012 20:42:01 +0200 Subject: comparison between crypto libraries Message-ID: <4F78A179.2000101@gnutls.org> Hello, Actually this comparison is pretty old, but was unknown to me until today. It compares the various cryptographic libraries on cipher performance. https://idlebox.net/2008/0714-cryptography-speedtest-comparison/ Since then many things might have changed, but I also noticed the AES performance difference on a 32-bit processor. However, in RSA and DH nettle/gmp seem to offer much better performance than openssl [0]. regards, Nikos [0]. http://nikmav.blogspot.com/2012/04/in-some-embedded-systems-space-may.html https://bitbucket.org/yarosla/nxweb/wiki/Benchmarks From nmav at gnutls.org Fri Apr 6 14:39:53 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Fri, 6 Apr 2012 14:39:53 +0200 Subject: rsa blinding patch Message-ID: Hello, Attached you'll find an initial patch that adds timing resistant versions of rsa_compute_root() and rsa_decrypt(). regards, Nikos From dkg at fifthhorseman.net Fri Apr 6 15:12:18 2012 From: dkg at fifthhorseman.net (Daniel Kahn Gillmor) Date: Fri, 06 Apr 2012 09:12:18 -0400 Subject: rsa blinding patch In-Reply-To: References: Message-ID: <4F7EEBB2.5060902@fifthhorseman.net> On 04/06/2012 08:39 AM, Nikos Mavrogiannopoulos wrote: > Attached you'll find an initial patch that adds timing resistant > versions of rsa_compute_root() and rsa_decrypt(). No patch was attached to the version of the message i received from the list. Maybe try it again? --dkg -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1030 bytes Desc: OpenPGP digital signature URL: From nmav at gnutls.org Fri Apr 6 16:46:17 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Fri, 6 Apr 2012 16:46:17 +0200 Subject: rsa blinding patch In-Reply-To: <4F7EEBB2.5060902@fifthhorseman.net> References: <4F7EEBB2.5060902@fifthhorseman.net> Message-ID: On Fri, Apr 6, 2012 at 3:12 PM, Daniel Kahn Gillmor wrote: > No patch was attached to the version of the message i received from the > list. Maybe try it again? I can see it on my sent folder. Could it be that the list manager removed it? I've put it at: http://homes.esat.kuleuven.be/~nikos/0001-Added-timing-resistant-versions-of-rsa_compute_root-.patch regards, Nikos From nisse at lysator.liu.se Sat Apr 7 10:47:47 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sat, 07 Apr 2012 10:47:47 +0200 Subject: rsa blinding patch In-Reply-To: (Nikos Mavrogiannopoulos's message of "Fri, 6 Apr 2012 16:46:17 +0200") References: <4F7EEBB2.5060902@fifthhorseman.net> Message-ID: Nikos Mavrogiannopoulos writes: > I can see it on my sent folder. Could it be that the list manager removed it? Possibly. The mailman config is known to be a bit picky (and I'm not very good at configuring it), but I thought it would usually bounce messages rather than silently remove attachments. > I've put it at: > http://homes.esat.kuleuven.be/~nikos/0001-Added-timing-resistant-versions-of-rsa_compute_root-.patch Thanks! I've had a quick look. A few comments: I'm not sure if the low-level rsa_compute_root should be aware of the blinding, or if it should be the responsibility of its callers (using _rsa_blind, _rsa_unblind helper functions, put in rsa_blind.c or so). Support for blinding is desirable not only for rsa_decrypt, but also for the various rsa_*_sign functions, right? The blinding function should probably use nettle_mpz_random, which provides for *almost* uniform distribution mod n by generating a few extra bits before the mpz_fdiv_r. As for interface, I have been considering to also add some kind of deterministic dsa signatures, substituting something like HMAC(private key, message) for the random input, and if possible, it would be nice to have a consistent interface for {rsa, dsa} signatures {with, without} randomness input. Not sure if we should have separate functions for operation with and without blidning, or a single function with an optional randomness source as argument. If we have separate functions, we have to decide on the name (I don't quite like "_timing": If the name is supposed to describe intended use, it needs to be more verbose, maybe "_timing_resistant". I think it may be more handy to rather describe what it *does*, something like "_blinding" or "_randomized" or so). Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nmav at gnutls.org Sat Apr 7 11:34:54 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sat, 07 Apr 2012 11:34:54 +0200 Subject: rsa blinding patch In-Reply-To: References: <4F7EEBB2.5060902@fifthhorseman.net> Message-ID: <4F800A3E.6000002@gnutls.org> On 04/07/2012 10:47 AM, Niels M?ller wrote: > Thanks! I've had a quick look. A few comments: > I'm not sure if the low-level rsa_compute_root should be aware of the > blinding, or if it should be the responsibility of its callers (using > _rsa_blind, _rsa_unblind helper functions, put in rsa_blind.c or so). As a user of the library I'd prefer a low level function that provides the algorithm operation in constant time, so that I don't need to understand the details of blinding. It might be that in later version nettle doesn't use blinding but something different to achieve constant time, so it would be nice existing applications to change automatically. > Support for blinding is desirable not only for rsa_decrypt, but also for > the various rsa_*_sign functions, right? Indeed. I changed rsa_compute_root() because I don't use the *_sign() functions. They were not very flexible for my needs. I will try updating making a constant time counterpart of them, but it will expand the interface considerably. > The blinding function should probably use nettle_mpz_random, which > provides for *almost* uniform distribution mod n by generating a few > extra bits before the mpz_fdiv_r. I'll update it for that. > Not sure if we should have separate functions for operation with and > without blidning, or a single function with an optional randomness > source as argument. If we have separate functions, we have to decide on > the name (I don't quite like "_timing": If the name is supposed to > describe intended use, it needs to be more verbose, maybe > "_timing_resistant". I think it may be more handy to rather describe > what it *does*, something like "_blinding" or "_randomized" or so). What about _ct for constant time? The _blinding is really specific on the method used to achieve constant time. regards, Nikos From nisse at lysator.liu.se Sat Apr 7 15:30:16 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sat, 07 Apr 2012 15:30:16 +0200 Subject: rsa blinding patch In-Reply-To: <4F800A3E.6000002@gnutls.org> (Nikos Mavrogiannopoulos's message of "Sat, 07 Apr 2012 11:34:54 +0200") References: <4F7EEBB2.5060902@fifthhorseman.net> <4F800A3E.6000002@gnutls.org> Message-ID: Nikos Mavrogiannopoulos writes: > What about _ct for constant time? The _blinding is really specific > on the method used to achieve constant time. But it's not really constant time, is it? Rather, timing is random but independent of the inputs which are under control of the attacker. While without RSA blinding, timing depends on the secret key and on data provided by the attacker, which is a bad combination. The function mpz_powm_sec in recent GMP is supposed to really be "constant time" (assuming the underlying multiply-instruction doesn't have data-dependent timing). I.e., the instructions executed and the memory access pattern should depend only on the sizes of the input operands, not on the actual values. Hence it should be resistant both to timing attacks, and attacks manipulating or observing the memory cache hit rate. But RSA operations uses a couple of additional functions, which doesn't yet have constant-time counterparts, so I don't yet use that function yet. BTW, what's a good reference for the recommendation to use RSA blinding? Is it in Handbook of applied cryptography? (I think pointers to papers on attacks have been posted previously, but that's describing the problem, not the solution...). Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nmav at gnutls.org Sat Apr 7 17:55:58 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sat, 07 Apr 2012 17:55:58 +0200 Subject: rsa blinding patch In-Reply-To: References: <4F7EEBB2.5060902@fifthhorseman.net> <4F800A3E.6000002@gnutls.org> Message-ID: <4F80638E.6060701@gnutls.org> On 04/07/2012 03:30 PM, Niels M?ller wrote: >> What about _ct for constant time? The _blinding is really specific >> on the method used to achieve constant time. > But it's not really constant time, is it? Rather, timing is random but > independent of the inputs which are under control of the attacker. While > without RSA blinding, timing depends on the secret key and on data > provided by the attacker, which is a bad combination. Indeed. So is _timing_resistant or _tr better? > BTW, what's a good reference for the recommendation to use RSA blinding? > Is it in Handbook of applied cryptography? (I think pointers to papers > on attacks have been posted previously, but that's describing the > problem, not the solution...). I've seen it there: ftp://ftp.rsasecurity.com/pub/pdfs/bull-2.pdf regards, Nikos From simon at josefsson.org Sat Apr 7 15:38:07 2012 From: simon at josefsson.org (Simon Josefsson) Date: Sat, 07 Apr 2012 15:38:07 +0200 Subject: rsa blinding patch In-Reply-To: References: <4F7EEBB2.5060902@fifthhorseman.net> <4F800A3E.6000002@gnutls.org> Message-ID: <1333805887.3570.3.camel@latte.josefsson.org> l?r 2012-04-07 klockan 15:30 +0200 skrev Niels M?ller: > Nikos Mavrogiannopoulos writes: > > > What about _ct for constant time? The _blinding is really specific > > on the method used to achieve constant time. > > But it's not really constant time, is it? Rather, timing is random but > independent of the inputs which are under control of the attacker. While > without RSA blinding, timing depends on the secret key and on data > provided by the attacker, which is a bad combination. Maybe a better term to use is "reduced side channel" or something. Not easy to shorten though. The generic problem adressed here is side channels. /Simon From nmav at gnutls.org Sat Apr 7 20:28:19 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sat, 07 Apr 2012 20:28:19 +0200 Subject: rsa blinding patch In-Reply-To: References: <4F7EEBB2.5060902@fifthhorseman.net> Message-ID: <4F808743.2070900@gnutls.org> On 04/07/2012 10:47 AM, Niels M?ller wrote: > I'm not sure if the low-level rsa_compute_root should be aware of the > blinding, or if it should be the responsibility of its callers (using > _rsa_blind, _rsa_unblind helper functions, put in rsa_blind.c or so). > > Support for blinding is desirable not only for rsa_decrypt, but also for > the various rsa_*_sign functions, right? > The blinding function should probably use nettle_mpz_random, which > provides for *almost* uniform distribution mod n by generating a few > extra bits before the mpz_fdiv_r. I've updated the patch to account for that and other issues. http://homes.esat.kuleuven.be/~nikos/0001-Added-timing-resistant-versions-of-rsa_compute_root-.patch What I haven't done is to duplicate all the rsa_(hash)_sign functions. They are quite many and it doesn't make much sense duplicating all of them. Do you have any suggestion on compacting it? > As for interface, I have been considering to also add some kind of > deterministic dsa signatures, substituting something like HMAC(private > key, message) for the random input, and if possible, it would be nice to > have a consistent interface for {rsa, dsa} signatures {with, without} > randomness input. I don't really think that the random number in DSA relates to RSA blinding. The random number in DSA is part of the signature scheme, while the RSA blinding random number is just to prevent timing attacks. regards, Nikos From nmav at gnutls.org Sat Apr 7 20:30:03 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sat, 07 Apr 2012 20:30:03 +0200 Subject: rsa blinding patch In-Reply-To: <1333805887.3570.3.camel@latte.josefsson.org> References: <4F7EEBB2.5060902@fifthhorseman.net> <4F800A3E.6000002@gnutls.org> <1333805887.3570.3.camel@latte.josefsson.org> Message-ID: <4F8087AB.2000703@gnutls.org> On 04/07/2012 03:38 PM, Simon Josefsson wrote: >>> What about _ct for constant time? The _blinding is really specific >>> on the method used to achieve constant time. >> But it's not really constant time, is it? Rather, timing is random but >> independent of the inputs which are under control of the attacker. While >> without RSA blinding, timing depends on the secret key and on data >> provided by the attacker, which is a bad combination. > > Maybe a better term to use is "reduced side channel" or something. Not > easy to shorten though. The generic problem adressed here is side > channels. Indeed, but side channels also contain the issues due to power analysis, and I don't know if the same techniques that make an algorithm timing analysis resistant, also apply for power analysis. regards, Nikos From simon at josefsson.org Sun Apr 8 07:49:28 2012 From: simon at josefsson.org (Simon Josefsson) Date: Sun, 08 Apr 2012 07:49:28 +0200 Subject: rsa blinding patch In-Reply-To: <4F8087AB.2000703@gnutls.org> References: <4F7EEBB2.5060902@fifthhorseman.net> <4F800A3E.6000002@gnutls.org> <1333805887.3570.3.camel@latte.josefsson.org> <4F8087AB.2000703@gnutls.org> Message-ID: <1333864168.3570.5.camel@latte.josefsson.org> l?r 2012-04-07 klockan 20:30 +0200 skrev Nikos Mavrogiannopoulos: > On 04/07/2012 03:38 PM, Simon Josefsson wrote: > > >>> What about _ct for constant time? The _blinding is really specific > >>> on the method used to achieve constant time. > >> But it's not really constant time, is it? Rather, timing is random but > >> independent of the inputs which are under control of the attacker. While > >> without RSA blinding, timing depends on the secret key and on data > >> provided by the attacker, which is a bad combination. > > > > Maybe a better term to use is "reduced side channel" or something. Not > > easy to shorten though. The generic problem adressed here is side > > channels. > > > Indeed, but side channels also contain the issues due to power analysis, > and I don't know if the same techniques that make an > algorithm timing analysis resistant, also apply for power analysis. Right, but maybe the API name could be general so that it could implement mitigation against power analysis as well in the future, if needed. Or some other side channel attack (cache misses, etc). /Simon From nisse at lysator.liu.se Mon Apr 9 16:29:50 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 09 Apr 2012 16:29:50 +0200 Subject: rsa blinding patch In-Reply-To: <4F808743.2070900@gnutls.org> (Nikos Mavrogiannopoulos's message of "Sat, 07 Apr 2012 20:28:19 +0200") References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> Message-ID: Nikos Mavrogiannopoulos writes: > I've updated the patch to account for that and other issues. > http://homes.esat.kuleuven.be/~nikos/0001-Added-timing-resistant-versions-of-rsa_compute_root-.patch Nice. I'm still not sure how to best do this. I've been looking at it for some time today, and I consider the below patch (with _rsa_blind and _rsa_unblind helper functions in aseparate file). But on second thought maybe it's better to do it like you do with an rsa_compute_root_tr function. In any case, I think it does make sense to have a separate pkcs1_decrypt function for the common processing of rsa_decrypt and rsa_decrypt_tr. > What I haven't done is to duplicate all the rsa_(hash)_sign functions. > They are quite many and it doesn't make much sense duplicating all of > them. Do you have any suggestion on compacting it? One way may be to *always* do blinding. But I hesitate before requiring the user to setup and provide a randomness function. I note that the document you pointed to, ftp://ftp.rsasecurity.com/pub/pdfs/bull-2.pdf, suggests using something similar to HMAC(private-key, input) to generate the random number to be used for blidning. Doing that (and maybe have an *optional* randomness source) could make sense. The needed randomness info has to go either as argument to all private key operations, or be put somewhere in the private key struct. And either way makes things more complicated. > I don't really think that the random number in DSA relates to RSA > blinding. The random number in DSA is part of the signature scheme, > while the RSA blinding random number is just to prevent timing attacks. The purpose is very different, but the interface issues (assuming we do add deterministic DSA signatures) are very similar: You have a signature method were a randomness function is optional for all private key operations. I think it makes some sense to implement rsa_decrypt_tr now, with no advertised internals, and give the signature interface more time. BTW, do you have any test cases? It would be useful both to have test cases that check that the _tr operations give the correct result (that should ba a simple addition to testsuite/rsa-encrypt-test, I guess), and some way to quantitatively check how the timing depends on the inputs, for both rsa_compute_root and rsa_compute_root_tr. Regard, /Niels diff --git a/Makefile.in b/Makefile.in index 4d3c89a..bad8103 100644 --- a/Makefile.in +++ b/Makefile.in @@ -99,7 +99,7 @@ hogweed_SOURCES = sexp.c sexp-format.c \ bignum.c bignum-next-prime.c \ bignum-random.c bignum-random-prime.c \ sexp2bignum.c \ - pkcs1.c pkcs1-rsa-md5.c pkcs1-rsa-sha1.c \ + pkcs1.c pkcs1-decrypt.c pkcs1-rsa-md5.c pkcs1-rsa-sha1.c \ pkcs1-rsa-sha256.c pkcs1-rsa-sha512.c \ rsa.c rsa-sign.c rsa-verify.c \ rsa-md5-sign.c rsa-md5-verify.c \ @@ -107,6 +107,7 @@ hogweed_SOURCES = sexp.c sexp-format.c \ rsa-sha256-sign.c rsa-sha256-verify.c \ rsa-sha512-sign.c rsa-sha512-verify.c \ rsa-encrypt.c rsa-decrypt.c \ + rsa-blind.c rsa-decrypt-tr.c \ rsa-keygen.c rsa-compat.c \ rsa2sexp.c sexp2rsa.c \ dsa.c dsa-sign.c dsa-verify.c dsa-keygen.c \ diff --git a/pkcs1-decrypt.c b/pkcs1-decrypt.c new file mode 100644 index 0000000..bd21f88 --- /dev/null +++ b/pkcs1-decrypt.c @@ -0,0 +1,72 @@ +/* pkcs1-decrypt.c + * + */ + +/* nettle, low-level cryptographics library + * + * Copyright (C) 2001, 2012 Niels M??ller + * + * The nettle library is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or (at your + * option) any later version. + * + * The nettle library is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public + * License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the nettle library; see the file COPYING.LIB. If not, write to + * the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, + * MA 02111-1307, USA. + */ + +#if HAVE_CONFIG_H +# include "config.h" +#endif + +#include + +#include "pkcs1.h" + +#include "bignum.h" +#include "nettle-internal.h" + +int +pkcs1_decrypt (unsigned key_size, + const mpz_t m, + unsigned *length, uint8_t *message) +{ + TMP_DECL(em, uint8_t, NETTLE_MAX_BIGNUM_BITS / 8); + uint8_t *terminator; + unsigned padding; + unsigned message_length; + + TMP_ALLOC(em, key_size); + nettle_mpz_get_str_256(key_size, em, m); + + /* Check format */ + if (em[0] || em[1] != 2) + return 0; + + terminator = memchr(em + 2, 0, key_size - 2); + + if (!terminator) + return 0; + + padding = terminator - (em + 2); + if (padding < 8) + return 0; + + message_length = key_size - 3 - padding; + + if (*length < message_length) + return 0; + + memcpy(message, terminator + 1, message_length); + *length = message_length; + + return 1; +} + diff --git a/pkcs1.h b/pkcs1.h index 732d0ed..95a6a83 100644 --- a/pkcs1.h +++ b/pkcs1.h @@ -43,6 +43,7 @@ extern "C" { #define pkcs1_rsa_sha256_encode_digest nettle_pkcs1_rsa_sha256_encode_digest #define pkcs1_rsa_sha512_encode nettle_pkcs1_rsa_sha512_encode #define pkcs1_rsa_sha512_encode_digest nettle_pkcs1_rsa_sha512_encode_digest +#define pkcs1_decrypt nettle_pkcs1_decrypt struct md5_ctx; struct sha1_ctx; @@ -57,6 +58,11 @@ pkcs1_signature_prefix(unsigned size, unsigned digest_size); int +pkcs1_decrypt (unsigned key_size, + const mpz_t m, + unsigned *length, uint8_t *message); + +int pkcs1_rsa_md5_encode(mpz_t m, unsigned length, struct md5_ctx *hash); int diff --git a/rsa-blind.c b/rsa-blind.c new file mode 100644 index 0000000..f66c169 --- /dev/null +++ b/rsa-blind.c @@ -0,0 +1,65 @@ +/* rsa-blind.c + * + */ + +/* nettle, low-level cryptographics library + * + * Copyright (C) 2012, Nikos Mavrogiannopoulos, Niels M??ller + * + * The nettle library is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or (at your + * option) any later version. + * + * The nettle library is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public + * License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the nettle library; see the file COPYING.LIB. If not, write to + * the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, + * MA 02111-1307, USA. + */ + +#if HAVE_CONFIG_H +# include "config.h" +#endif + +#include "rsa.h" +#include "bignum.h" + +void +_rsa_blind (const mpz_t n, const mpz_t e, + void *random_ctx, nettle_random_func random, + mpz_t c, mpz_t ri) +{ + mpz_t r; + + mpz_init(r); + + /* c = c*(r^e) + * ri = r^(-1) + */ + do + { + nettle_mpz_random(r, random_ctx, random, n); + /* invert r */ + } + while (!mpz_invert (ri, r, n)); + + /* c = c*(r^e) mod n */ + mpz_powm(r, r, e, n); + mpz_mul(c, c, r); + mpz_fdiv_r(c, c, n); + + mpz_clear(r); +} + +void +_rsa_unblind (const mpz_t n, mpz_t c, const mpz_t ri) +{ + mpz_mul(c, c, ri); + mpz_fdiv_r(c, c, n); +} + diff --git a/rsa-decrypt-tr.c b/rsa-decrypt-tr.c new file mode 100644 index 0000000..5951a12 --- /dev/null +++ b/rsa-decrypt-tr.c @@ -0,0 +1,55 @@ +/* rsa-decrypt-tr.c + * + * The RSA publickey algorithm. PKCS#1 encryption. + */ + +/* nettle, low-level cryptographics library + * + * Copyright (C) 2001, 2012 Niels M??ller + * + * The nettle library is free software; you can redistribute it and/or modify + * it under the terms of the GNU Lesser General Public License as published by + * the Free Software Foundation; either version 2.1 of the License, or (at your + * option) any later version. + * + * The nettle library is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY + * or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public + * License for more details. + * + * You should have received a copy of the GNU Lesser General Public License + * along with the nettle library; see the file COPYING.LIB. If not, write to + * the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, + * MA 02111-1307, USA. + */ + +#if HAVE_CONFIG_H +# include "config.h" +#endif + +#include "rsa.h" + +#include "pkcs1.h" + +int +rsa_decrypt_tr(const struct rsa_public_key *public, + const struct rsa_private_key *key, + void *random_ctx, nettle_random_func random, + unsigned *length, uint8_t *message, + const mpz_t gibberish) +{ + mpz_t m, ri; + int res; + + mpz_init_set(m, gibberish); + mpz_init (ri); + + rsa_blind (public->n, public->e, random_ctx, random, + m, ri); + rsa_compute_root(key, m, m); + rsa_unblind (public->n, m, ri); + + res = pkcs1_decrypt (key->size, m, length, message); + mpz_clear(m); + return res; +} diff --git a/rsa-decrypt.c b/rsa-decrypt.c index fe6de23..cde0d3c 100644 --- a/rsa-decrypt.c +++ b/rsa-decrypt.c @@ -1,11 +1,11 @@ -/* rsa_decrypt.c +/* rsa-decrypt.c * * The RSA publickey algorithm. PKCS#1 encryption. */ /* nettle, low-level cryptographics library * - * Copyright (C) 2001 Niels M??ller + * Copyright (C) 2001, 2012 Niels M??ller * * The nettle library is free software; you can redistribute it and/or modify * it under the terms of the GNU Lesser General Public License as published by @@ -27,54 +27,22 @@ # include "config.h" #endif -#include -#include -#include - #include "rsa.h" -#include "bignum.h" -#include "nettle-internal.h" +#include "pkcs1.h" int rsa_decrypt(const struct rsa_private_key *key, unsigned *length, uint8_t *message, const mpz_t gibberish) { - TMP_DECL(em, uint8_t, NETTLE_MAX_BIGNUM_BITS / 8); - uint8_t *terminator; - unsigned padding; - unsigned message_length; - mpz_t m; + int res; mpz_init(m); rsa_compute_root(key, m, gibberish); - TMP_ALLOC(em, key->size); - nettle_mpz_get_str_256(key->size, em, m); + res = pkcs1_decrypt (key->size, m, length, message); mpz_clear(m); - - /* Check format */ - if (em[0] || em[1] != 2) - return 0; - - terminator = memchr(em + 2, 0, key->size - 2); - - if (!terminator) - return 0; - - padding = terminator - (em + 2); - if (padding < 8) - return 0; - - message_length = key->size - 3 - padding; - - if (*length < message_length) - return 0; - - memcpy(message, terminator + 1, message_length); - *length = message_length; - - return 1; + return res; } diff --git a/rsa.h b/rsa.h index a4ef835..75eaafc 100644 --- a/rsa.h +++ b/rsa.h @@ -32,9 +32,6 @@ #include "md5.h" #include "sha.h" -/* For nettle_random_func */ -#include "nettle-meta.h" - #ifdef __cplusplus extern "C" { #endif @@ -64,7 +61,10 @@ extern "C" { #define rsa_sha512_verify_digest nettle_rsa_sha512_verify_digest #define rsa_encrypt nettle_rsa_encrypt #define rsa_decrypt nettle_rsa_decrypt +#define rsa_decrypt_tr nettle_rsa_decrypt_re #define rsa_compute_root nettle_rsa_compute_root +#define _rsa_blind _nettle_rsa_blind +#define _rsa_unblind _nettle_rsa_unblind #define rsa_generate_keypair nettle_rsa_generate_keypair #define rsa_keypair_to_sexp nettle_rsa_keypair_to_sexp #define rsa_keypair_from_sexp_alist nettle_rsa_keypair_from_sexp_alist @@ -260,7 +260,7 @@ rsa_sha512_verify_digest(const struct rsa_public_key *key, int rsa_encrypt(const struct rsa_public_key *key, /* For padding */ - void *random_ctx, nettle_random_func random, + void *random_ctx, nettle_random_func *random, unsigned length, const uint8_t *cleartext, mpz_t cipher); @@ -274,12 +274,31 @@ rsa_decrypt(const struct rsa_private_key *key, unsigned *length, uint8_t *cleartext, const mpz_t ciphertext); +/* Timing-resistant version, using randomized RSA blinding. */ +int +rsa_decrypt_tr(const struct rsa_public_key *public, + const struct rsa_private_key *key, + void *random_ctx, nettle_random_func random, + unsigned *length, uint8_t *message, + const mpz_t gibberish); + /* Compute x, the e:th root of m. Calling it with x == m is allowed. */ void rsa_compute_root(const struct rsa_private_key *key, mpz_t x, const mpz_t m); +/* Blinds the c, by computing c *= r^e (mod n), for a random r. Also + returns the inverse (ri), for use by rsa_unblind. */ +void +_rsa_blind (const mpz_t n, const mpz_t e, + void *random_ctx, nettle_random_func *random, + mpz_t c, mpz_t ri); + +/* c *= ri mod n */ +void +_rsa_unblind (const mpz_t n, mpz_t c, const mpz_t ri); + /* Key generation */ /* Note that the key structs must be initialized first. */ @@ -287,8 +306,8 @@ int rsa_generate_keypair(struct rsa_public_key *pub, struct rsa_private_key *key, - void *random_ctx, nettle_random_func random, - void *progress_ctx, nettle_progress_func progress, + void *random_ctx, nettle_random_func *random, + void *progress_ctx, nettle_progress_func *progress, /* Desired size of modulo, in bits */ unsigned n_size, -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Mon Apr 9 22:10:07 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 09 Apr 2012 22:10:07 +0200 Subject: rsa blinding patch In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Mon, 09 Apr 2012 16:29:50 +0200") References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> Message-ID: nisse at lysator.liu.se (Niels M?ller) writes: > I think it makes some sense to implement rsa_decrypt_tr now, with no > advertised internals, and give the signature interface more time. I've committed an rsa_decrypt_tr now, based on Nikos' code. I reordered the arguments a bit, in an attempt to be more consistent with other rsa functions, so the prototype is now /* Timing-resistant version, using randomized RSA blinding. */ int rsa_decrypt_tr(const struct rsa_public_key *pub, const struct rsa_private_key *key, void *random_ctx, nettle_random_func *random, unsigned *length, uint8_t *message, const mpz_t gibberish); Does this make sense? If you are using the rsa encrypt/decrypt functions, and the interface is ok, I guess we should also document them. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nmav at gnutls.org Mon Apr 9 22:39:05 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Mon, 09 Apr 2012 22:39:05 +0200 Subject: rsa blinding patch In-Reply-To: References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> Message-ID: <4F8348E9.5080806@gnutls.org> On 04/09/2012 04:29 PM, Niels M?ller wrote: > One way may be to *always* do blinding. But I hesitate before requiring > the user to setup and provide a randomness function. I note that the > document you pointed to, ftp://ftp.rsasecurity.com/pub/pdfs/bull-2.pdf, > suggests using something similar to HMAC(private-key, input) to generate > the random number to be used for blidning. Doing that (and maybe have an > *optional* randomness source) could make sense. Although the HMAC avoids the need for randomness, you need to get a key somehow, so the gain might not be much. > The needed randomness info has to go either as argument to all private > key operations, or be put somewhere in the private key struct. And > either way makes things more complicated. Indeed. > I think it makes some sense to implement rsa_decrypt_tr now, with no > advertised internals, and give the signature interface more time. What about the rsa_compute_root? This is the only function I can use from nettle for RSA signatures (TLS is peculiar on its signatures thus the exported interface isn't sufficient). If there was an rsa_pkcs1_sign() and rsa_pkcs1_verify() with similar interface to encrypt/decrypt, I could use those instead. > BTW, do you have any test cases? It would be useful both to have test > cases that check that the _tr operations give the correct result (that > should ba a simple addition to testsuite/rsa-encrypt-test, I guess), and > some way to quantitatively check how the timing depends on the inputs, > for both rsa_compute_root and rsa_compute_root_tr. I had modified the rsa-encrypt-test.c to include a test for the timing resistant version as well. Other than that I have no other test cases. regards, Nikos From nisse at lysator.liu.se Mon Apr 9 22:57:25 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 09 Apr 2012 22:57:25 +0200 Subject: rsa blinding patch In-Reply-To: <4F8348E9.5080806@gnutls.org> (Nikos Mavrogiannopoulos's message of "Mon, 09 Apr 2012 22:39:05 +0200") References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> <4F8348E9.5080806@gnutls.org> Message-ID: Nikos Mavrogiannopoulos writes: > Although the HMAC avoids the need for randomness, you need to get a key > somehow, so the gain might not be much. The idea is to reuse (hash of) the private RSA key as the HMAC key. And similarly for deterministic DSA. > What about the rsa_compute_root? This is the only function I can use > from nettle for RSA signatures I wasn't aware of that. That's an argument for an rsa_compute_root_tr (or alternatively, public rsa_blind and rsa_unblind helpers). Can you explain briefly what special signatures are used by tls? (It was more then 10 years since I wrote an implementation, then it was ssl version 3). > If there was an rsa_pkcs1_sign() and rsa_pkcs1_verify() with similar > interface to encrypt/decrypt, I could use those instead. Can you propose such an interface? Currently, rsa_md5_sign calls pkcs1_rsa_md5_encode followed by rsa_compute_root. If it's easy for you to use rsa_compute_root in the same way, then I guess there's no need to introduce new low-level primitives, but maybe it could be rearranged in some better way? Or, since tls is an important application, it may make sense to directly add tls-style signatures to Nettle. > I had modified the rsa-encrypt-test.c to include a test for the > timing resistant version as well. Other than that I have no other test > cases. I see. It would really be helpful with some tools for measuring the input dependence in the timing of rsa_compute_root. GMP's mpz_powm may well behave quite differently from openssl's. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nmav at gnutls.org Tue Apr 10 12:04:05 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Tue, 10 Apr 2012 12:04:05 +0200 Subject: rsa blinding patch In-Reply-To: References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> <4F8348E9.5080806@gnutls.org> Message-ID: On Mon, Apr 9, 2012 at 10:57 PM, Niels M?ller wrote: >> What about the rsa_compute_root? This is the only function I can use >> from nettle for RSA signatures > I wasn't aware of that. That's an argument for an rsa_compute_root_tr > (or alternatively, public rsa_blind and rsa_unblind helpers). > Can you explain briefly what special signatures are used by tls? (It was > more then 10 years since I wrote an implementation, then it was ssl > version 3). Out of memory TLS 1.0 uses a concatenation of md5 and sha1. I don't think however that, this mode should be added in nettle. It would complicate the API for no real reason (if a generic pkcs1, 1.5 signing function is available). >> If there was an rsa_pkcs1_sign() and rsa_pkcs1_verify() with similar >> interface to encrypt/decrypt, I could use those instead. > Can you propose such an interface? Currently, rsa_md5_sign calls > pkcs1_rsa_md5_encode followed by rsa_compute_root. If it's easy for you > to use rsa_compute_root in the same way, then I guess there's no need to > introduce new low-level primitives, but maybe it could be rearranged in > some better way? I do the pkcs1 1.5 encoding in gnutls, and you also do it in the high level functions in nettle, that I cannot use. It would be nice if we can save some code and reduce error risk by having a common pkcs1 1.5 signing function. I'll try to propose one the next few days. > Or, since tls is an important application, it may make sense to directly > add tls-style signatures to Nettle. I don't think it is a good idea. What might be needed at some point later is pkcs 1 2.0 signatures. I've come across some passports signed by certificates using rsa-pss :( regards, Nikos From nisse at lysator.liu.se Tue Apr 10 12:23:08 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 10 Apr 2012 12:23:08 +0200 Subject: rsa blinding patch In-Reply-To: (Nikos Mavrogiannopoulos's message of "Tue, 10 Apr 2012 12:04:05 +0200") References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> <4F8348E9.5080806@gnutls.org> Message-ID: Nikos Mavrogiannopoulos writes: > I do the pkcs1 1.5 encoding in gnutls, and you also do it in the high > level functions in nettle, that I cannot use. It would be nice if we > can save some code and reduce error risk by having a common pkcs1 1.5 > signing function. I'll try to propose one the next few days. Have you looked at pkcs1_signature_prefix? It does part of the work, so maybe it's a good starting point. The reason it leaves space for the actual digest rather than copying it in place, is to avoid extra copies for the rsa_md5_sign-style functions. If this could be simplified, to reduce the amount of duplicated code to implement signatures for each supported hash function, while keeping low overhead for applications using only a single variant, that would be nice. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nmav at gnutls.org Fri Apr 13 23:39:40 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Fri, 13 Apr 2012 23:39:40 +0200 Subject: rsa blinding patch In-Reply-To: References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> <4F8348E9.5080806@gnutls.org> Message-ID: <4F889D1C.3000202@gnutls.org> On 04/10/2012 12:23 PM, Niels M?ller wrote: >> I do the pkcs1 1.5 encoding in gnutls, and you also do it in the high >> level functions in nettle, that I cannot use. It would be nice if we >> can save some code and reduce error risk by having a common pkcs1 1.5 >> signing function. I'll try to propose one the next few days. > Have you looked at pkcs1_signature_prefix? It does part of the work, so > maybe it's a good starting point. The reason it leaves space for the > actual digest rather than copying it in place, is to avoid extra copies > for the rsa_md5_sign-style functions. pkcs1_signature_prefix() is very low level to be used by gnutls, as it assumes no ASN.1 encoder. The most suitable for me would be something like: /* RSA signatures, using PKCS#1 1.5 * Input should be a BER encoded DigestInfo */ int rsa_digest_info_sign(const struct rsa_private_key *key, unsigned length, const uint8_t *digest_info, mpz_t signature); int rsa_digest_info_verify(const struct rsa_public_key *key, unsigned length, const uint8_t *digest_info, const mpz_t signature); Those two could also be used in place of all the individual rsa_(hash)_sign and rsa_(hash)_verify. It requires though another function is available translate from a digest to the digest_info (this is not needed for gnutls as we already have that, just a suggestion to fit those two in nettle). E.g. digest_to_digest_info(hash_id, unsigned length, const uint8_t* digest, unsigned* digest_info_length, uint8_t* digest_info); The cost is having a single function that will contain the OIDs for all the supported hash algorithms. The benefit is that there is no need to have separate sign and verify functions for each supported hash. regards, Nikos From nisse at lysator.liu.se Sat Apr 14 09:28:33 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sat, 14 Apr 2012 09:28:33 +0200 Subject: rsa blinding patch In-Reply-To: <4F889D1C.3000202@gnutls.org> (Nikos Mavrogiannopoulos's message of "Fri, 13 Apr 2012 23:39:40 +0200") References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> <4F8348E9.5080806@gnutls.org> <4F889D1C.3000202@gnutls.org> Message-ID: Nikos Mavrogiannopoulos writes: > /* RSA signatures, using PKCS#1 1.5 > * Input should be a BER encoded DigestInfo > */ > int > rsa_digest_info_sign(const struct rsa_private_key *key, > unsigned length, const uint8_t *digest_info, > mpz_t signature); > > int > rsa_digest_info_verify(const struct rsa_public_key *key, > unsigned length, const uint8_t *digest_info, > const mpz_t signature); That seems reasonable (if and how the implementation of nettle's other rsa sign/verify functions could be reorganized to make use of that, I'm not sure). Minor points: It should really be DER encoded, right? And I guess you'd also want a _sign_tr variant. As for naming, maybe I'd want to have "pkcs1" int he name, not sure. A question for all of you: Nettle currently has two flavors of sign functions for each algorithm. E.g., int rsa_sha1_sign(const struct rsa_private_key *key, struct sha1_ctx *hash, mpz_t signature); int rsa_sha1_sign_digest(const struct rsa_private_key *key, const uint8_t *digest, mpz_t s); The former could be implemented in terms of the latter (currently it's not, to avoid an unnecessary copy, which may be a pointless optimization in this context). The point of the first function is convenience, where you can create a signature simply by sha1_init sha1_update rsa_sha1_sign without bothering with allocating and writing the actual digest. Do you think this design makes sense? Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nmav at gnutls.org Sat Apr 14 15:19:53 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sat, 14 Apr 2012 15:19:53 +0200 Subject: rsa blinding patch In-Reply-To: References: <4F7EEBB2.5060902@fifthhorseman.net> <4F808743.2070900@gnutls.org> <4F8348E9.5080806@gnutls.org> <4F889D1C.3000202@gnutls.org> Message-ID: <4F897979.6000305@gnutls.org> On 04/14/2012 09:28 AM, Niels M?ller wrote: >> /* RSA signatures, using PKCS#1 1.5 >> * Input should be a BER encoded DigestInfo >> */ >> int >> rsa_digest_info_sign(const struct rsa_private_key *key, >> unsigned length, const uint8_t *digest_info, >> mpz_t signature); >> >> int >> rsa_digest_info_verify(const struct rsa_public_key *key, >> unsigned length, const uint8_t *digest_info, >> const mpz_t signature); > That seems reasonable (if and how the implementation of nettle's other > rsa sign/verify functions could be reorganized to make use of that, I'm > not sure). > Minor points: It should really be DER encoded, right? PKCS #1 1.5 says BER and there were quite some implementations that used BER for encoding instead of DER. > And I guess you'd also want a _sign_tr variant. Indeed. I only listed the basic interface for review. > As for naming, maybe I'd want to have > "pkcs1" int he name, not sure. I wanted to use PKCS1 but it might be misleading as PKCS #1 1.5 has a different padding method than PKCS #1 2.1 which uses OAEP. I don't know if rsa_pkcs1_1_5_sign is reasonable... > A question for all of you: Nettle currently has two flavors of sign > functions for each algorithm. E.g., > int > rsa_sha1_sign(const struct rsa_private_key *key, > struct sha1_ctx *hash, > mpz_t signature); > > int > rsa_sha1_sign_digest(const struct rsa_private_key *key, > const uint8_t *digest, > mpz_t s); > The former could be implemented in terms of the latter (currently it's > not, to avoid an unnecessary copy, which may be a pointless > optimization in this context). > The point of the first function is convenience, where you can create a > signature simply by > > sha1_init > sha1_update > rsa_sha1_sign > without bothering with allocating and writing the actual digest. > Do you think this design makes sense? IMO it makes sense, but I think the drawbacks, which are maintaining one extra function per hash, do not really outweigh the benefit which is not really noticeable in the RSA operation. regards, Nikos From nisse at lysator.liu.se Wed Apr 18 21:47:14 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Wed, 18 Apr 2012 21:47:14 +0200 Subject: Salsa20 In-Reply-To: ("Niels =?iso-8859-1?Q?M=F6ller=22's?= message of "Sun, 01 Apr 2012 00:13:20 +0200") References: <87sjh1xe3p.fsf@latte.josefsson.org> Message-ID: nisse at lysator.liu.se (Niels M?ller) writes: > For x86_64 (and maybe x86_32), assembly implementation also seems > attractive, I just push an x86_64 implementation of salsa20_crypt. Runs at 6.6 cycles/byte on my laptop (or 189 Mbyte/s), which is more than twice the speed of aes128, and slightly faster than arcfour. It's likely possible to squeeze out a cycle or two more, by doing two blocks in parallel (I think djb's x86_64 code does that, but I found it very hard to read), or by other micro-optimizations. Do any of you know of any protocols which specify use of salsa20? Is it usually combined with some *fast* MAC algorithm? Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From mann.ern.kang at gmail.com Thu Apr 26 02:49:17 2012 From: mann.ern.kang at gmail.com (Mann Ern Kang) Date: Thu, 26 Apr 2012 08:49:17 +0800 Subject: AES assembler not compatible with Windows x64 Message-ID: Hi all, An issue arose while testing gnutls, which has a dependency on nettle, on Windows x64. The x86_64 aes assembler code in nettle is not compatible with the Windows calling convention, leading to crashes. The related gnutls item is here: https://savannah.gnu.org/support/?108038 This can be worked around for now with --disable-assembler during ./configure. Going forward, will there be any plans to support assembler for x64 Windows? Cheers, Mann Ern From martin at martin.st Thu Apr 26 09:21:09 2012 From: martin at martin.st (=?ISO-8859-15?Q?Martin_Storsj=F6?=) Date: Thu, 26 Apr 2012 10:21:09 +0300 (EEST) Subject: AES assembler not compatible with Windows x64 In-Reply-To: References: Message-ID: Hi, On Thu, 26 Apr 2012, Mann Ern Kang wrote: > This can be worked around for now with --disable-assembler during > ./configure. Going forward, will there be any plans to support assembler > for x64 Windows? This should already be fixed in the latest git version (and latest cvs version prior to the conversion), and in the 2.5-prerelease at http://www.lysator.liu.se/~nisse/archive/nettle-2.5-pre.tar.gz. // Martin From nisse at lysator.liu.se Thu Apr 26 10:10:06 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Thu, 26 Apr 2012 10:10:06 +0200 Subject: AES assembler not compatible with Windows x64 In-Reply-To: ("Martin =?iso-8859-1?Q?Storsj=F6=22's?= message of "Thu, 26 Apr 2012 10:21:09 +0300 (EEST)") References: Message-ID: Martin Storsj? writes: > This should already be fixed in the latest git version (and latest cvs > version prior to the conversion), and in the 2.5-prerelease at > http://www.lysator.liu.se/~nisse/archive/nettle-2.5-pre.tar.gz. Guess I should make a 2.5 release soon. I had planned to get xenofarm up and running for the new repository before release, but that has turned out to be harder than I expected. So maybe the release shouldn't have to wait for that. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From simon at josefsson.org Thu Apr 26 14:29:43 2012 From: simon at josefsson.org (Simon Josefsson) Date: Thu, 26 Apr 2012 14:29:43 +0200 Subject: Salsa20 In-Reply-To: ("Niels \=\?iso-8859-1\?Q\?M\=F6ller\=22's\?\= message of "Thu, 29 Mar 2012 12:30:59 +0200") References: <87sjh1xe3p.fsf@latte.josefsson.org> Message-ID: <87ty06oevc.fsf@latte.josefsson.org> Hi! Sorry for not following up on the Salsa20 thread, other things needed my attention... primarily my reason for saying this shouldn't be committed directly was that it used the first reference implementation from: http://cr.yp.to/snuffle.html and that page contains other implementations as well, which may be faster. However speed can always be improved later on, I suppose the important aspect is to get the API right from the start. I'm replying to all your replies in this e-mail, see below. nisse at lysator.liu.se (Niels M?ller) writes: > Simon Josefsson writes: > >> Hi! Please find attached a port of DJB's public domain code for Salsa20 >> to nettle. The patch is not meant to be applied as-is but to start a >> discussion. > > I'm not familiar with this cipher... Could you explain briefly why one > may want to use it? I don't know of any reason today, e.g., no important Internet applications uses it. It has some interesting properties though, so maybe there is uptake. >> While writing this I noticed a sub-optimal design of Nettle, in that it >> overloads the meaning of struct nettle_cipher->block_size as a flag of >> whether the cipher is a stream cipher or not. Before arcfour was the >> only stream cipher in nettle, and using block_size=0 to signal that it >> is a stream cipher worked fine. However, Salsa20 has a block size of 8 >> bytes. > > If Salsa20 doesn't fit in the nettle_cipher abstraction, then I think we > should start by implementing Salsa20 *without* trying to get it to fit > there. To me, a "stream cipher" with a block size and iv seems a bit > like a contradiction. Somewhat, although I think the distinction between stream ciphers and block ciphers is not a good one to make in an API. Usually applications want to use ciphers in some mode, and some modes used with block ciphers make them essentially stream ciphers. I think an good API should be modeled around "symmetric encryption" as the concept. Possibly just "encryption", but I think public-key and symmetric encryption are such different beasts that they shouldn't be exposed through the same API. > In general, I don't think it makes much sense to include algorithms > which behave very differently under the "nettle_cipher" umbrella, since > the point of that abstraction is to collect algorithms which are easily > exchangable. Code accepting a nettle_cipher are not expected to check > flags or behave fundamentally different for different algorithms. Sure. > Including the arcfour stream cipher there may well be a mistake. One > annoying problem with the current design is that for block ciphers, the > ctx argument of the encrypt and decrypt functions naturally is const, > but due arcfour being fittted in the same nettle_cipher abstraction, the > function typedef nettle_crypt_func can't use a const ctx. I always found rc4 the odd cipher in nettle. > Anyway, I'd prefer to treat redesign in that area as an independent > problem from adding support for Salsa20. That is a good idea. > Then there's also the tentative (nettle-internal.h) nettle_aead > abstraction. Salsa20 could perhaps fit there, if we allow algorithms > with no authentication (NULL digest function pointer). Possibly... or just have one abstract "symmetric encryption" that embodies all these variants. Or does that lead to other disadvantages? >> --- /dev/null >> +++ b/salsa20.c >> +#define ROTL32(x,n) ((((x))<<(n)) | (((x))>>(32-(n)))) > > There are several different variants of that macro. It would be nice > with a unified one in macros.h. I agree. >> +#define SWAP32(v) \ >> + ((ROTL32(v, 8) & 0x00FF00FFUL) | \ >> + (ROTL32(v, 24) & 0xFF00FF00UL)) >> + >> +#ifdef WORDS_BIGENDIAN >> +#define U32TO32_LITTLE(v) SWAP32(v) >> +#else >> +#define U32TO32_LITTLE(v) (v) >> +#endif >> + >> +#define U8TO32_LITTLE(p) U32TO32_LITTLE(((uint32_t*)(p))[0]) >> +#define U32TO8_LITTLE(p, v) (((uint32_t*)(p))[0] = U32TO32_LITTLE(v)) > > That's a clever byte swapping trick (at least if a true rot instruction > is available). In Nettle conversion between bytes and integers are > usually done with READ_UINT32, LE_READ_UINT32 and friends. It's usually > not very performance critical, and it deals naturally with any > alignment. I suspect U8TO32_LITTLE above breaks if the input is > unaligned but the architecture doesn't allow unaligned word reads. Using READ_UINT32 etc is probably better. >> +/* >> +salsa20-ref.c version 20051118 >> +D. J. Bernstein >> +Public domain. >> +*/ > > Shouldn't that be moved up closer to the (C) header? I put the majority of my changes above that header, to make it easy to sync and compare the code >> +#define ROTATE(v,c) (ROTL32(v,c)) >> +#define XOR(v,w) ((v) ^ (w)) >> +#define PLUS(v,w) ((v) + (w)) >> +#define PLUSONE(v) (PLUS((v),1)) > > I'm not sure these macros improve readability... They are from the reference code. >> +static void salsa20_wordtobyte(uint8_t output[64],const uint32_t input[16]) > > Despite the name, this is the main encryption function. Or rather, a > (hopefully) one-way function? "input" is really the context struct, > which contains some mix of the key and the iv? I must admit I haven't looked into the details of the code. >> + x[ 4] = XOR(x[ 4],ROTATE(PLUS(x[ 0],x[12]), 7)); > > And what's wrong with writing that as > > x[ 4] ^= ROTATE(x[ 0] + x[12], 7) That's clearer to me. >> +void >> +salsa20_set_iv(struct salsa20_ctx *ctx, unsigned length, const uint8_t *iv) >> +{ >> + assert (length == SALSA20_IV_SIZE); > > And this initializes indices 6-9. I'd remove the length argument, if > only one value is allowed anyway. I added it only for compatibility with the internal nettle interface. > I've done some more homework, reading some of djb's papers and the > wikipedia page. Some questions: > > 1. Any protocols that specify use of salsa20? And in combination with > which MAC? With claimed cycle numbers, hmac with sha1 or sha2 is > going to take much more time than the encryption. I don't know of any protocols right now. > 2. What about the later "ChaCha" cipher? As far as I see, it has not > been studied as much as Salsa20. Yeah, I looked at it too, but preferred something that people seems more familiar with. > * A bit of cleanup of the code. > > * Try optimization of the C code for 64-bit machines. One ought to be > able to do two column or row operations in parallel by putting two > salsa20 words into a uint64_t variable. May need some tricks to avoid > carry propagation between the words, but I suspect it may be a win due > to lower register pressure. A bit similar to the HAVE_NATIVE_64_BIT in > camellia-crypt-internal.c. > > * Try an sse2 assembly implementation (the djb:s papers outline how to > do that). Or copy some existing implementation. Take a look at the link above, most likely there exists something. I'm not sure how important it is though. > * Documentation. Yes. > As for the salsa20_* interface: > > * It would be possible to change the interface to not expose the block > size, doing a little buffering instead. But I think it's better to not > do that, and follow what the ctr code and gcm code does. I agree. > * One advertised feature of the cipher is random access. I think we > should have something like a salsa20_set_pos, taking a block count as > argument. Yes. > I also arranged in the Makefile so that nettle-internal.o no longer is > included in the nettle library. Will that cause any trouble for anybody? Didn't GnuTLS use some thought-to-be-internal-functions-but-apparently-not? Maybe it was just the macros.h header file though. > Do any of you know of any protocols which specify use of salsa20? Is it > usually combined with some *fast* MAC algorithm? I suspect people who like Salsa are inclined to also like CubeHash, which could be used in HMAC variants. CubeHash is fast with optimistic parameters, but the "default" is pretty conservative making it not so fast. /Simon From nisse at lysator.liu.se Thu Apr 26 20:54:55 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Thu, 26 Apr 2012 20:54:55 +0200 Subject: Salsa20 In-Reply-To: <87ty06oevc.fsf@latte.josefsson.org> (Simon Josefsson's message of "Thu, 26 Apr 2012 14:29:43 +0200") References: <87sjh1xe3p.fsf@latte.josefsson.org> <87ty06oevc.fsf@latte.josefsson.org> Message-ID: Simon Josefsson writes: > I don't know of any reason today, e.g., no important Internet > applications uses it. It has some interesting properties though, so > maybe there is uptake. It seems the main selling point is speed. Internally, it seems a bit unorthodox, * hash function in counter mode (when everybody seems to move to block ciphers in counter mode, the invertibility of the underlying block cipher is an irrelevant feature). * No sboxes or other "large" non-linearities. Just mix of xor and mod 2^32 add, and ten rotations to speed up diffusion. Everybody else seems to think non-linear sboxes (or things like the majority function in sha hashes) are essential. Possibly with the exception of idea? Which uses similar primitives, iirc, but with mod (2^16 + 1) multiplies instead of simple rotations to help the bit diffusion. > Somewhat, although I think the distinction between stream ciphers and > block ciphers is not a good one to make in an API. Usually applications > want to use ciphers in some mode, and some modes used with block ciphers > make them essentially stream ciphers. I think an good API should be > modeled around "symmetric encryption" as the concept. Such an API would make some sense, but I don't think it's a substitute for a block cipher API. E.g., in ssh, the application has to be aware of the block size, and the message padding is application specific (isn't it the same in tls?). An API for encrypting a stream would not be of much help here. And when using cbc, I think it's not even possible. >> Including the arcfour stream cipher there may well be a mistake. One >> annoying problem with the current design is that for block ciphers, the >> ctx argument of the encrypt and decrypt functions naturally is const, >> but due arcfour being fittted in the same nettle_cipher abstraction, the >> function typedef nettle_crypt_func can't use a const ctx. > > I always found rc4 the odd cipher in nettle. That the nettle_cipher abstraction treats the stream ciphers (arcfour being the only supported one) as a block cipher with block size zero goes back to my and Henrik Grubbstr?m's design for Pike's crypto library, some 15 years ago... Is that what you're finding odd, or is there anything else which is strange with arcfour? >> Then there's also the tentative (nettle-internal.h) nettle_aead >> abstraction. Salsa20 could perhaps fit there, if we allow algorithms >> with no authentication (NULL digest function pointer). > > Possibly... or just have one abstract "symmetric encryption" that > embodies all these variants. Or does that lead to other disadvantages? In general, it's not much point to have a general interface, if the application has to query particulars before using it (e.g, does this mechanism provide any authentication, or do I need to combine it with some other MAC?). So I don't think what I sketched is a good idea. >>> --- /dev/null >>> +++ b/salsa20.c >>> +#define ROTL32(x,n) ((((x))<<(n)) | (((x))>>(32-(n)))) >> >> There are several different variants of that macro. It would be nice >> with a unified one in macros.h. > > I agree. Done. >>> +#define SWAP32(v) \ >>> + ((ROTL32(v, 8) & 0x00FF00FFUL) | \ >>> + (ROTL32(v, 24) & 0xFF00FF00UL)) [...] >> That's a clever byte swapping trick (at least if a true rot instruction >> is available). In Nettle conversion between bytes and integers are >> usually done with READ_UINT32, LE_READ_UINT32 and friends. It's usually >> not very performance critical, and it deals naturally with any >> alignment. I suspect U8TO32_LITTLE above breaks if the input is >> unaligned but the architecture doesn't allow unaligned word reads. > > Using READ_UINT32 etc is probably better. I ended up keeping that kind of byte swapping, to be able to stick to word accesses to memory until the final plaintext/cryptotext xor. > I put the majority of my changes above that header, to make it easy to > sync and compare the code I'm afraid that not easy any more... I made quite a lot of changes. >> * Try an sse2 assembly implementation (the djb:s papers outline how to >> do that). Or copy some existing implementation. > > Take a look at the link above, most likely there exists something. I'm > not sure how important it is though. I had a look, but I found that assembly code hard to read (apparantly automatically generated by some tool of djb's). >> * One advertised feature of the cipher is random access. I think we >> should have something like a salsa20_set_pos, taking a block count as >> argument. > > Yes. Suggestions for name? I think set_iv should still set the count to zero, so you need to use the new function only if you want to do seeeks in the data. >> Do any of you know of any protocols which specify use of salsa20? Is it >> usually combined with some *fast* MAC algorithm? > > I suspect people who like Salsa are inclined to also like CubeHash, > which could be used in HMAC variants. CubeHash is fast with optimistic > parameters, but the "default" is pretty conservative making it not so > fast. Maybe it's possible to do something in the style of ofb, to get authentication cheaply as a side effect (haven't looked very closely at ofb, though, and ofb itself seems to be unusable in Nettle due to patents). Maybe I should mail djb and ask. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Fri Apr 27 14:54:48 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Fri, 27 Apr 2012 14:54:48 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> (Dagobert Michelsen's message of "Fri, 27 Apr 2012 14:23:13 +0200") References: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> Message-ID: Dagobert Michelsen writes: > I just verified that the problem persists with nettle 2.4 and gnutls-3.0.19. As you seem to have > access to the Solaris buildfarm it would be cool if you could finally fix this as it blocks > new releases of gnutls for the OpenCSW distribution. It's some time since I looked in to this. My understaning of the problem is that 1. This Solaris version lack both and . (A comment in AX_CREATE_STDINT_H seems to imply that it does have a , should we use that?) 2. Nettle defines uint32_t and friends using the AX_CREATE_STDINT_H autoconf macro. 3. gnulib defines them in a different way. 4. The multiple definitions of, e.g., uint32_t, agree and are no problem. 5. The definitions of some other (unused) types are incompatible, e.g., int_fast8_t, and cause the compilation errors. 6. The definitions from AX_CREATE_STDINT_H is compatible with the definitions of these types in Solaris 10. gnulib's definitions aren't, so any code using them can potentially break if linked together with code compiled on Solaris 10. Hence I think it ought to be fixed in gnulib. Simon (and any other gnulib people on these lists), do you agree? To address (1), maybe you can try diff --git a/configure.ac b/configure.ac index 6bf2b8b..77bb9b1 100644 --- a/configure.ac +++ b/configure.ac @@ -518,7 +518,7 @@ LSH_GCC_ATTRIBUTES # According to Simon Josefsson, looking for uint32_t and friends in # sys/types.h is needed on some systems, in particular cygwin. -AX_CREATE_STDINT_H([nettle-stdint.h], [sys/types.h]) +AX_CREATE_STDINT_H([nettle-stdint.h], [sys/types.h], [sys/inttypes.h]) # Check for file locking. We (AC_PROG_CC?) have already checked for # sys/types.h and unistd.h. but I suspect you would still get compilation errors due to incompatible definitions in gnulib and sys/inttypes.h. Maybe the easiest nettle workaround would be to somehow disable the definitions of the problematic types, which we don't use anyway. Could you try something like the following? diff --git a/nettle-types.h b/nettle-types.h index b694332..d0aa332 100644 --- a/nettle-types.h +++ b/nettle-types.h @@ -23,6 +23,9 @@ #ifndef NETTLE_TYPES_H #define NETTLE_TYPES_H +/* Pretend this type always exists. Nettle doesn't use it. */ +#define _STDINT_HAVE_INT_FAST32_T 1 + #include "nettle-stdint.h" #ifdef __cplusplus Have a look into the generated nettle-stdint.h (or the template in aclocal.m4) for other potentially useful defines. The closest Solaris 5.8 box I used to have acccess to (hanna.lysator.liu.se) seems to have been decomissioned. Regardss, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From simon at josefsson.org Fri Apr 27 15:46:18 2012 From: simon at josefsson.org (Simon Josefsson) Date: Fri, 27 Apr 2012 15:46:18 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: References: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> Message-ID: <1335534378.6649.2.camel@latte> fre 2012-04-27 klockan 14:54 +0200 skrev Niels M?ller: > Dagobert Michelsen writes: > > > I just verified that the problem persists with nettle 2.4 and gnutls-3.0.19. As you seem to have > > access to the Solaris buildfarm it would be cool if you could finally fix this as it blocks > > new releases of gnutls for the OpenCSW distribution. > > It's some time since I looked in to this. My understaning of the problem > is that > > 1. This Solaris version lack both and . (A > comment in AX_CREATE_STDINT_H seems to imply that it does have a > , should we use that?) Using system headers if available seems like a good idea. > 2. Nettle defines uint32_t and friends using the AX_CREATE_STDINT_H > autoconf macro. > > 3. gnulib defines them in a different way. What is the difference? This sounds bad. > 5. The definitions of some other (unused) types are incompatible, e.g., > int_fast8_t, and cause the compilation errors. If they are unused, we could remove the definitions. > 6. The definitions from AX_CREATE_STDINT_H is compatible with the > definitions of these types in Solaris 10. gnulib's definitions > aren't, so any code using them can potentially break if linked > together with code compiled on Solaris 10. > > Hence I think it ought to be fixed in gnulib. Simon (and any other > gnulib people on these lists), do you agree? Yes. Understanding what the differences is, and which one is right, seems useful first though. If the problem is in gnulib, we can fix that and GnuTLS will have the new variant soon. /Simon From nisse at lysator.liu.se Fri Apr 27 17:24:20 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Fri, 27 Apr 2012 17:24:20 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <1335534378.6649.2.camel@latte> (Simon Josefsson's message of "Fri, 27 Apr 2012 15:46:18 +0200") References: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> <1335534378.6649.2.camel@latte> Message-ID: Simon Josefsson writes: > Yes. Understanding what the differences is, and which one is right, > seems useful first though. If the problem is in gnulib, we can fix > that and GnuTLS will have the new variant soon. Quoting my investigation from an old (off-list) email thread, from November last year: And the problem seems to be that inttypes.h doesn't define int_fast8_t types. And then AX_CREATE_STDINT_H (used by nettle) and gnulib (used by gnutls) both decide to define them, and they do that in an incompatible way. --- From a quick look at gl/stdint.h.in and on posted error messages, I think glib uses long for all of int_fast8_t, int_fast16_t and int_fast32_t, and int64_t for int_fast64_t, while nettle uses int8_t, int, int32_t and int64_t. --- >> Actually, one ought to have a look at the definitions in the stdint.h >> provided by gcc (and at the definitions in the solaris10 stdint.h), and >> make sure the definitions of the int_fast* types are equivalent. >> Otherwise one might end up with subtle ABI differences and errors when >> linking together code compiled by different compilers. > > Agreed. I did a quick check. The following seems to be the correct sizes: int_fast8_t: 1 int_fast16_t: 4 int_fast32_t: 4 int_fast64_t: 8 This is using sizeof on the types defined in stdint.h, trying: gcc -m32 on solaris 9 gcc -m64 on solaris 9 cc -m32 on solaris 10 cc -m64 on solaris 10 Remarkably enough, they all agree. I also get the same sizes from nettle-stdint.h, for both 32 bit and 64 bit configurations. So I think nettle (or really, AX_CREATE_STDINT_H) gets this right. While if I understand gl correctly, it uses 4,4,4,8 for a 32-bit build and 8,8,8,8 for a 64-bit build (the size of long is different). This seems wrong. Sure, these are perfectly ok sizes according to C99, but they differ from the sparc conventions (and possibly conventions for other architectures as well). Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Mon Apr 30 09:10:58 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 30 Apr 2012 09:10:58 +0200 Subject: Release plans Message-ID: I've updated the todo page http://www.lysator.liu.se/~nisse/nettle/plan.html (also as misc/plan.html in the git repo). Any missing items? Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Mon Apr 30 11:16:32 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 30 Apr 2012 11:16:32 +0200 Subject: RSA private key abstraction Message-ID: I'm thinking about how to represent private keys. The context is the desire to support RSA blinding. I'd like to avoid having to duplicate all the signature functions (of which there's already a large number). This email is an expansion of a 3.0 entry in plan.txt. One way would be to pass a function pointer (and some optional ctx) for doing the private key operation. This would then be used wherever the current code calls _rsa_compute_root, and it would represent an RSA private key oracle. If we stick to software implementation, one could pass it as an extra argument, or one could extend the private key struct like struct rsa_private_key { mpz_t d; ... rsa_compute_root_func *f; }; and one could then "subclass" that as struct rsa_private_key_blinding { struct rsa_private_key super; struct whatever_random_ctx_needed ctx; }; (not sure this is the best design, but it's one possibility). Now I have a question: Would it be feasible to support smartcards (and similar hardware) at the same time? Then the secret key wouldn't be available at all to nettle, and a signature function might have an interface like int rsa_sha1_sign(rsa_compute_root_func *f, void *private, struct sha1_ctx *hash, mpz_t signature); where the actual private key data is totally dependent on the smartcard and its interface software. But what interfaces do typical smartcards use? Do they allow you to compute an arbitrary rsa root, or do they only more specific operations like "pkcs#1 v1.5 decrypt", "pkcs#1 v1.5 sha1 sign", #pkcs#1 v2.0 sha256 sign"? I want to figure out if smartcard support at this level is at all feasible. If not, we have to leave that for applications and libraries on top of nettle, and then the nettle interface should be tailored for software keys only. I've also had a quick look at the ssh-agent and gpgagent protocols. Both seems a bit too application specific, so I don't think it's feasible to hook in those as general "private key oracles" here. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From n.mavrogiannopoulos at gmail.com Mon Apr 30 18:53:14 2012 From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos) Date: Tue, 1 May 2012 01:53:14 +0900 Subject: Release plans In-Reply-To: References: Message-ID: A comment on the last point. I don't think a smart-card interface suits nettle. It is a software (low-level) library. Smart card support could be added only if nettle supported a higher level API, but even then I don't know how useful it would be as pkcs11 is already there for that reason. regards, Nikos On Mon, Apr 30, 2012 at 4:10 PM, Niels M?ller wrote: > I've updated the todo page > http://www.lysator.liu.se/~nisse/nettle/plan.html > (also as misc/plan.html in the git repo). > > Any missing items? > > Regards, > /Niels > > -- > Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. > Internet email is subject to wholesale government surveillance. > > _______________________________________________ > nettle-bugs mailing list > nettle-bugs at lists.lysator.liu.se > http://lists.lysator.liu.se/mailman/listinfo/nettle-bugs From nmav at gnutls.org Mon Apr 30 19:08:08 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Tue, 1 May 2012 02:08:08 +0900 Subject: RSA private key abstraction In-Reply-To: References: Message-ID: On Mon, Apr 30, 2012 at 6:16 PM, Niels M?ller wrote: > and one could then "subclass" that as > ?struct rsa_private_key_blinding > ?{ > ? ?struct rsa_private_key super; > ? ?struct whatever_random_ctx_needed ctx; > ?}; It might be too complicated for a low level library. If all those were hidden from applications it would be all right, but this complication will lead to a moment where something has to be changed/updated/added. I'd suggest to keep the low level interface as simple. > (not sure this is the best design, but it's one possibility). > Now I have a question: Would it be feasible to support smartcards (and > similar hardware) at the same time? Then the secret key wouldn't be > available at all to nettle, and a signature function might have an > interface like> > ?int > ?rsa_sha1_sign(rsa_compute_root_func *f, void *private, > ? ? ? ? ? ? ? ?struct sha1_ctx *hash, > ? ? ? ? ? ? ? ?mpz_t signature); Then you'd need to cope with smart card drivers and all. Opensc does all that and provides a pkcs11 interface. All (or most) commercial smart cards also provide a pkcs11 interface. So I'd expect that applications that do that, already use it (we do it in gnutls and the high level functions abstract between keys in software -> nettle and keys in hardware -> pkcs11). > where the actual private key data is totally dependent on the smartcard > and its interface software. > But what interfaces do typical smartcards use? Do they allow you to > compute an arbitrary rsa root, or do they only more specific operations > like "pkcs#1 v1.5 decrypt", "pkcs#1 v1.5 sha1 sign", #pkcs#1 v2.0 sha256 > sign"? They can do either, depending on the capabilities of the card. > I want to figure out if smartcard support at this level is at all > feasible. If not, we have to leave that for applications and libraries > on top of nettle, and then the nettle interface should be tailored for > software keys only. I think it would make sense for a high level interface. regards, Nikos From nisse at lysator.liu.se Sat May 5 21:19:09 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sat, 05 May 2012 21:19:09 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: (Dagobert Michelsen's message of "Sat, 5 May 2012 19:56:47 +0200") References: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> <1335534378.6649.2.camel@latte> Message-ID: Dagobert Michelsen writes: > I see different possibilities on how this can be resolved: > > 1. nettle uses the gnulib definitions I currently have no plans to switch to using gnulib. As far as I'm aware, there's no problem with the types which nettle actually uses. So it would be a reasonable workaround to simply not define the int_fast*_t types; then they won't collide with the incompatible definitions gnutls gets from gnulib. Question is just what's the least intrusive way to do that (*if* it's easy to avoid, I'd prefer not to modify the definition of AX_CREATE_STDINT_H). I would appreciate if you can try the workarounds I suggested the other day and let me know how it works. > 2. gnulib changes the definitions to conform to nettle The problem with gnulib is not primarily that it doesn't "conform to nettle", but that it defines some types in a way that differs both from on later releases on the OS (where the types are included), and from the definition in the stdint.h which comes with gcc. So code which uses the int_fast*_t types and relies on gnulib will get a slightly different ABI if compiled on SunOS-5.9 (or earlier) using Sun's compiler, than if compiled with gcc on the same system, or compiled with Sun's compiler on any later release of the OS. I think that's what needs to be fixed in gnulib. Do the gnulib folks agree? Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Sun May 6 09:13:43 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sun, 06 May 2012 09:13:43 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <4FA605DC.5020400@cs.ucla.edu> (Paul Eggert's message of "Sat, 05 May 2012 22:02:20 -0700") References: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> <1335534378.6649.2.camel@latte> <4FA605DC.5020400@cs.ucla.edu> Message-ID: Paul Eggert writes: > Generally speaking, it's unwise to use > the int_fast*_t types in a public header file. Why (feel free to point to the relevant section of gnulib docs, if it's explained well there)? Maybe nobody uses these types so it's an academic issue, but if these types are used in the interface to any library (possibly some system library), then it's important that applications agree on the definition. To me, it seems like the authoritative definition of int_fast*_t ought to be the system's ABI specification. And in the case of SunOS-5.9, where the types are missing, I think it makes the most sense to adopt the ABI of SunOS-5.10. Which also seems to be what gcc does. I'm a bit confused by your statement. I had the impression that gnulib *does* fall back on the definitions in public headers like and , for the types which *are* defined there. And that's why nettle and gnutls seem to work together on SunOS-10. Correct? > This is a documented issue in Gnulib. > It's hard to imagine a general fix for this. One possibility might be to not define the types at all, unless the gnulib application *explicitly* asks for them. Lots of applications want uint32_t, and I imagine a lot fewer have any need for int_fast*_t. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From dam at opencsw.org Sun May 6 09:47:16 2012 From: dam at opencsw.org (Dagobert Michelsen) Date: Sun, 6 May 2012 09:47:16 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <4FA605DC.5020400@cs.ucla.edu> References: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> <1335534378.6649.2.camel@latte> <4FA605DC.5020400@cs.ucla.edu> Message-ID: Hi Niels, Am 06.05.2012 um 07:02 schrieb Paul Eggert: > On 05/05/2012 12:19 PM, Niels M?ller wrote: >> So code which uses the int_fast*_t types and relies on gnulib will get a >> slightly different ABI > > Generally speaking, it's unwise to use > the int_fast*_t types in a public header file. > This is a documented issue in Gnulib. > It's hard to imagine a general fix for this. Do you see a chance to adjust the nettle header files to avoid the fast-types? The current situation makes gnutls uncompilable on platforms like Solaris. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From nmav at gnutls.org Sun May 6 12:49:07 2012 From: nmav at gnutls.org (Nikos Mavrogiannopoulos) Date: Sun, 06 May 2012 12:49:07 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <5405085.L6fiXlP7Mg@linuix> References: <4FA605DC.5020400@cs.ucla.edu> <5405085.L6fiXlP7Mg@linuix> Message-ID: <4FA65723.8000209@gnutls.org> On 05/06/2012 12:34 PM, Bruno Haible wrote: > Niels M?ller wrote: >> One possibility might be to not define the types at all, unless the >> gnulib application explicitly asks for them. Lots of applications want >> uint32_t, and I imagine a lot fewer have any need for int_fast*_t. > Yes. For this reason, libunistring installs a trimmed-down version of > gnulib's stdint.h, that defines only the int{8,16,32}_t types and not > all the rest (int64_t, int_least*_t, int_fast*_t, etc.). [1] Is there a gnulib module that only provides the basic types? regards, Nikos From nisse at lysator.liu.se Sun May 6 16:52:37 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sun, 06 May 2012 16:52:37 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <3244661.D8EYbvb0ce@linuix> (Bruno Haible's message of "Sun, 06 May 2012 16:38:04 +0200") References: <5405085.L6fiXlP7Mg@linuix> <4FA65723.8000209@gnutls.org> <3244661.D8EYbvb0ce@linuix> Message-ID: Bruno Haible writes: > It doesn't exist so far. But it is feasible. - On the other hand, it > looks like nettle-stdint.h can already be easily trimmed in this way, > by undefining _STDINT_NEED_INT_LEAST_T and _STDINT_NEED_INT_FAST_T ? I think that makes sense. I just need someone (Dagobert Michelsen?) who can try it out. Nettle uses the AX_CREATE_STDINT_H macro to generate that file. If it's easy, I'd prefer not to touch that and just add some extra #define in nettle-types.h before it includes nettle-stdint.h. Or if that doesn't work out, it's fine to hack away in the definition in aclocal.m4 to take out the unneded types. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Sun May 6 21:38:47 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Sun, 06 May 2012 21:38:47 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <4FA6B161.6050408@cs.ucla.edu> (Paul Eggert's message of "Sun, 06 May 2012 10:14:09 -0700") References: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> <1335534378.6649.2.camel@latte> <4FA605DC.5020400@cs.ucla.edu> <4FA6B161.6050408@cs.ucla.edu> Message-ID: Paul Eggert writes: > Intuitively, it's because the int_fast* types are so fragile: they > depend so much on choice of compiler and compiler option. Short version: I think gnulib should either make an effort to define int_fast*_t in the common way on each system (always doing the same as gcc would be very close to the Right Thing), or by default generate a stdint.h without these types and define the int_fast*_t types only when explicitly asked to. Longer version (sorry I fail to be more concise): I think I misread "public headers" in your previous mail, I thought you meant system header files, rather than public header files provided by portable non-system libraries. Sorry for the confusion. I agree one can expect these types to be a fragile and compiler dependent whenever they're left undefined by system headers and ABI spec, which will be the usual assumption for portable code. But in the particular case of SunOS, the types are defined by system headers in release 5.10 (and presumably nailed down by the ABI spec), but missing in release 5.9. Do you see any good reason for gnulib to not define the types in the same way as on 5.10, if building on 5.9? To me, that really looks like the Right Thing to do, and it also looks like the kind of job gnulib is supposed to do. Defining them in a way which differs from SunOS-5.10 definitely seems broken. In the case of gnutls and nettle, one gets compilation errors when another packed defines the same types, and does it using a configure hack which *is* compatible with SunOS-5.10. Of course, that fix doesn't magically make int_fast*_t universally non-fragile, but it definitely improves compatibility for the system in question. As for nettle, I think it's best to not define these types, which aren't used. On Solaris, it happens that AX_CREATE_STDINT_H defines them more correctly (in my view) than gnulib, but it may well cause problems on other system, so it seems more robust to not try to define them at all. But please also consider the hypothetical situation that nettle *did* use those types, even if only internally. Then I couldn't simply omit the definitions. To not cause problems for gnulib users I'd be forced to either change my definitions to prefer compatibility with gnulib over compatibility with SunOS-5.10, which seems really backwards, or introduce additional complexity in nettle-stdint.h to omit the definitions when my public headers need uint32_t. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Mon May 7 19:13:02 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 07 May 2012 19:13:02 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <4FA6EDA1.9060902@cs.ucla.edu> (Paul Eggert's message of "Sun, 06 May 2012 14:31:13 -0700") References: <6F92CE78-7FBD-4792-A97E-D457886874A2@opencsw.org> <1335534378.6649.2.camel@latte> <4FA605DC.5020400@cs.ucla.edu> <4FA6B161.6050408@cs.ucla.edu> <4FA6EDA1.9060902@cs.ucla.edu> Message-ID: Paul Eggert writes: > Hmm, well, the latter point is dubious. SunOS 5.10 standard headers do not > mention int_fast*_t, except in stdint.h where C requires their definition. I'd consider that definition to be authoritative... > Presumably this is for the same reason that Gnulib avoids these types. > Arguably these types are not part of the SunOS 5.10 ABI, since nothing > in the SunOS library binary interface use them. C is still the system programming language, and int_fast*_t are part of the language. But maybe we're splitting hairs now. > The main reason is to keep gnulib simple and maintainable. > I'd rather not have to hand-maintain details about an > operating system that hasn't shipped since 2009. I thought the point of gnulib was to do those annoying things to keep the maintanance in one place. But I guess I'll have to leave to the judgment of the gnulib maintainers which systems are too old and obsolete to care about. > How does the nettle stuff work? Does it #define int_fast8_t etc > in config.h? It is defined in a generated file nettle-stdint.h. And the names of the types are all typedefs, no #defines. With AX_CREATE_STDINT_H, it's a file generated independently of config.h. The file starts with a section with a bunch of #defines set up by configure, and then the rest of the file uses various tests on these to decide what file to include and which types to define. Where the common case is to just include the system's . The file is then installed with the other nettle header files. > If so, there's a simple workaround in gnulib: > don't futz with int_fast8_t if it's already #defined. If one could get that to work, it might solve the problem for the case that the types are defined but not used. But if it implies that size of those types will depend on #include order, it seems very brittle to me. I think it would be better to let the gnulib application say explicitly whether or not it needs those types, and not define them at all in the common case that they aren't needed. I.e., #ifdef GL_WANT_INTX_FAST_T # define int_fast8_t gl_int_fast8_t #endif Not sure where one would define GL_WANT_INTX_FAST_T, if desired, though. Should be specified in some way in configure.ac, but it can't be put in config.h if the stdint.h types are needed for installed header files. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Tue May 8 10:55:51 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 08 May 2012 10:55:51 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <4FA8580B.4000301@cs.ucla.edu> (Paul Eggert's message of "Mon, 07 May 2012 16:17:31 -0700") References: <4FA82C2B.80905@cs.ucla.edu> <1928362.AFeediWS73@linuix> <4FA8580B.4000301@cs.ucla.edu> Message-ID: Paul Eggert writes: > Thanks. This seems to be a win regardless of whether it > fixes the nettle problems, Dago, can you test if it solves the SunOS-5.9 problem for you? I.e., apply Paul's patch to gnutls/gnulib, then *unapply* the nettle workaround I comitted yesterday, and see if you still get compilation errors related to nettle-stdint.h. I'm not sure how picky the compiler is about multiple but equivalent typedefs of the same name. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From nisse at lysator.liu.se Tue May 8 17:56:49 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 08 May 2012 17:56:49 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <4FA93648.1040202@cs.ucla.edu> (Paul Eggert's message of "Tue, 08 May 2012 08:05:44 -0700") References: <4FA82C2B.80905@cs.ucla.edu> <1928362.AFeediWS73@linuix> <4FA8580B.4000301@cs.ucla.edu> <4FA93648.1040202@cs.ucla.edu> Message-ID: Paul Eggert writes: > Such usage violates the C standard: even if one compiler > is not picky about it, your're likely to run into another that > does. So a better solution is needed. I've committed a workaround to nettle-types.h. It now does /* Pretend these types always exists. Nettle doesn't use them. */ #define _STDINT_HAVE_INT_FAST32_T 1 #include "nettle-stdint.h" which overrides the result of the configure checks used in the generated nettle-stdint.h. Maybe not the prettiest way to do it, but apparantly it solves the problem, independently of your fixes to gnulib. (But I'd still like to know if your fixes also solves this problem, independently of that Nettle workaround). > I'd rather have an approach where the code simply does "#include > " and assumes that int_fast8_t works. I don't understand the > overall situation, though, so I don't know what exactly to suggest. I think when you build gnutls on a system lacking stdint.h, a typical compilation unit will contain /* From gnulib's stdint.h */ typedef unsigned int gl_uint32_t; #define uint32_t gl_uint32_t ... /* From nettle/nettle-stdint.h */ typedef unsigned int uint32_t; where the final line is macro-expanded to typedef unsigned int gl_uint32_t; >From what I've seen, this has been no problem. And I hope nettle-stdint.h and gnulib/stdint.h always do these types in the same way. There is some freedom in how to define these types; on a system where for example int and long are the same size, typedef unsigned long uint32_t; typedef unsigned int uint32_t; would be a two distinct but correct definitions, and I imagine the compiler would complain. And to consider the problematic int_fast*_t types, until a day or two ago, you got /* From gnulib's stdint.h */ typedef long gl_int_fast8_t; #define int_fast8_t gl_int_fast8_t ... /* From nettle/nettle-stdint.h */ typedef char int_fast8_t; which no reasonable compiler will accept. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From dam at opencsw.org Tue May 8 21:35:43 2012 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 8 May 2012 21:35:43 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: References: <4FA82C2B.80905@cs.ucla.edu> <1928362.AFeediWS73@linuix> <4FA8580B.4000301@cs.ucla.edu> Message-ID: Hi Niels, Am 08.05.2012 um 10:55 schrieb Niels M?ller: > Paul Eggert writes: >> Thanks. This seems to be a win regardless of whether it >> fixes the nettle problems, > > Dago, can you test if it solves the SunOS-5.9 problem for you? I.e., > apply Paul's patch to gnutls/gnulib, then *unapply* the nettle > workaround I comitted yesterday, and see if you still get compilation > errors related to nettle-stdint.h. > > I'm not sure how picky the compiler is about multiple but equivalent > typedefs of the same name. If Paul or you could provide a bootstrapped tarball I'll be happy to test it as I am not really familiar with gnulib bootstrapping. Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From nisse at lysator.liu.se Tue May 8 22:05:43 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Tue, 08 May 2012 22:05:43 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: (Dagobert Michelsen's message of "Tue, 8 May 2012 21:35:43 +0200") References: <4FA82C2B.80905@cs.ucla.edu> <1928362.AFeediWS73@linuix> <4FA8580B.4000301@cs.ucla.edu> Message-ID: Dagobert Michelsen writes: > If Paul or you could provide a bootstrapped tarball I'll be happy to test it as > I am not really familiar with gnulib bootstrapping. I'm also not very familiar with gnulib. But I think it should work fine to just get the gnutls release and apply Paul's patch to the file gnutls-3.0.9/gl/stdint.in.h before running the configure script. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From wrm at hushmail.com Mon May 21 16:03:16 2012 From: wrm at hushmail.com (William Morris) Date: Mon, 21 May 2012 09:03:16 -0500 Subject: Configure problem on OS-X Message-ID: Hi. I obtained the latest nettle using git. Running 'configure' on OS-X (10.7.4) fails. config.status: error: cannot find input file: config.h.in Any ideas? Output from 'configure' given below. -- William Morris iMac:nettle $ ./configure checking build system type... x86_64-apple-darwin11.4.0 checking host system type... x86_64-apple-darwin11.4.0 checking for -R flag... none Searching for libraries checking /usr/local/lib/lib... not found checking /usr/local/lib... added checking /sw/local/lib... not found checking /sw/lib... not found checking /usr/gnu/lib... not found checking /opt/gnu/lib... not found checking /sw/gnu/lib... not found checking /usr/freeware/lib... not found checking /usr/pkg/lib... not found checking for gcc... gcc checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking whether make sets $(MAKE)... yes checking for ranlib... ranlib checking for nm... nm checking for objdump... no checking for a BSD-compatible install... /usr/bin/install -c checking for a thread-safe mkdir -p... ./install-sh -c -d checking whether ln -s works... yes configure: Compiler uses 64-bit ABI. To change, set CC. configure: WARNING: Don't know where to install 64-bit libraries on this system. configure: Libraries to be installed in ${exec_prefix}/lib. configure: Looking for assembler files in x86_64/. checking CCPIC... -fPIC checking if globals are prefixed by underscore... yes checking if we should use a .note.GNU-stack section... no checking for ELF-style .type,%function pseudo-ops... no checking for ELF-style .type,#function pseudo-ops... no checking if .align assembly directive is logarithmic... yes checking for m4... /usr/bin/m4 checking for an ANSI C-conforming const... yes checking for inline... inline checking how to run the C preprocessor... gcc -E checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for uid_t in sys/types.h... yes checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for size_t... yes checking whether time.h and sys/time.h may both be included... yes checking for long... yes checking size of long... 8 checking for uint64_t... yes checking alignment of uint64_t... 8 checking openssl/blowfish.h usability... yes checking openssl/blowfish.h presence... yes checking for openssl/blowfish.h... yes checking openssl/des.h usability... yes checking openssl/des.h presence... yes checking for openssl/des.h... yes checking openssl/cast.h usability... yes checking openssl/cast.h presence... yes checking for openssl/cast.h... yes checking openssl/aes.h usability... yes checking openssl/aes.h presence... yes checking for openssl/aes.h... yes checking for working alloca.h... yes checking for alloca... yes checking malloc.h usability... no checking malloc.h presence... no checking for malloc.h... no checking for strerror... yes checking whether byte ordering is bigendian... no checking for memxor... no checking for __attribute__... yes checking for stdint types... stdint.h (shortcircuit) make use of stdint.h in nettle-stdint.h (assuming C99 compatible system) checking for fcntl file locking... yes checking for __gmpz_getlimbn in -lgmp... no configure: WARNING: GNU MP not found, or not 3.1 or up, see http://gmplib.org/. Support for public key algorithms will be unavailable. checking for __gmpz_powm_sec... no checking for BF_ecb_encrypt in -lcrypto... yes checking for library containing clock_gettime... no configure: creating ./config.status config.status: creating config.make config.status: creating config.m4 config.status: creating Makefile config.status: creating tools/Makefile config.status: creating testsuite/Makefile config.status: creating examples/Makefile config.status: creating nettle.pc config.status: creating hogweed.pc config.status: error: cannot find input file: config.h.in iMac:nettle $ From nisse at lysator.liu.se Mon May 21 23:26:00 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Mon, 21 May 2012 23:26:00 +0200 Subject: Configure problem on OS-X In-Reply-To: (William Morris's message of "Mon, 21 May 2012 09:03:16 -0500") References: Message-ID: William Morris writes: > I obtained the latest nettle using git. > Running 'configure' on OS-X (10.7.4) fails. > > config.status: error: cannot find input file: config.h.in > > Any ideas? config.h.in is generated from configure.ac. You need to run the .bootstrap script after you check out the sources from git. Currently, that script just runs autoconf and autoheader, and it's the latter program which generates config.h.in. Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance. From dam at opencsw.org Tue May 22 22:53:50 2012 From: dam at opencsw.org (Dagobert Michelsen) Date: Tue, 22 May 2012 22:53:50 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <4FA98D52.6030304@cs.ucla.edu> References: <4FA82C2B.80905@cs.ucla.edu> <1928362.AFeediWS73@linuix> <4FA8580B.4000301@cs.ucla.edu> <4FA98D52.6030304@cs.ucla.edu> Message-ID: <7D268C49-8FEF-4CA7-879C-C8797A455A7E@opencsw.org> Hi, Am 08.05.2012 um 23:17 schrieb Paul Eggert: > On 05/08/2012 01:05 PM, Niels M?ller wrote: >> I think it should work fine to just get the gnutls release and apply >> Paul's patch to the file gnutls-3.0.9/gl/stdint.in.h before running the >> configure script. > > Yes, that's the idea. Thanks. Sorry for the delay, I got distracted. I just tested the latest stdint.in.h from gnulib master and gnutls 3.0.19 passes this stage now. However, I didn't change anything on the nettle includes and now get lots of warnings. Probably the gl_* definitions should not belong to the installed nettle headers? CC pkcs12_bag.lo "/opt/csw/include/nettle/nettle-stdint.h", line 237: warning: typedef redeclared: gl_int_fast8_t "/opt/csw/include/nettle/nettle-stdint.h", line 238: warning: typedef redeclared: gl_int_fast16_t "/opt/csw/include/nettle/nettle-stdint.h", line 239: warning: typedef redeclared: gl_int_fast32_t "/opt/csw/include/nettle/nettle-stdint.h", line 241: warning: typedef redeclared: int64_t "/opt/csw/include/nettle/nettle-stdint.h", line 244: warning: typedef redeclared: gl_uint_fast8_t "/opt/csw/include/nettle/nettle-stdint.h", line 245: warning: typedef redeclared: gl_uint_fast16_t "/opt/csw/include/nettle/nettle-stdint.h", line 246: warning: typedef redeclared: gl_uint_fast32_t "/opt/csw/include/nettle/nettle-stdint.h", line 248: warning: typedef redeclared: uint64_t Best regards -- Dago -- "You don't become great by trying to be great, you become great by wanting to do something, and then doing it so hard that you become great in the process." - xkcd #896 From nisse at lysator.liu.se Wed May 23 10:37:05 2012 From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=) Date: Wed, 23 May 2012 10:37:05 +0200 Subject: Problem with int types persists on nettle 2.4 and gnutls 3.0.19 on Solaris 9 Sparc In-Reply-To: <7D268C49-8FEF-4CA7-879C-C8797A455A7E@opencsw.org> (Dagobert Michelsen's message of "Tue, 22 May 2012 22:53:50 +0200") References: <4FA82C2B.80905@cs.ucla.edu> <1928362.AFeediWS73@linuix> <4FA8580B.4000301@cs.ucla.edu> <4FA98D52.6030304@cs.ucla.edu> <7D268C49-8FEF-4CA7-879C-C8797A455A7E@opencsw.org> Message-ID: Dagobert Michelsen writes: > Sorry for the delay, I got distracted. I just tested the latest stdint.in.h from gnulib > master and gnutls 3.0.19 passes this stage now. Thanks. > However, I didn't change anything on the nettle includes and now get > lots of warnings. Are these warnings harmless? If so, I think this is good enough. > Probably the gl_* definitions should not belong to the installed > nettle headers? They don't, they're defined in gnulib, and appear in the nettle-stdint.h warnings because of gnulib's preprocessor macros. In gnutls/gl/stdint.h: typedef signed char gl_int_fast8_t; ... #define int_fast8_t gl_int_fast8_t; In nettle-stdint.h (included later in the compilation unit): typedef int8_t int_fast8_t; > CC pkcs12_bag.lo > "/opt/csw/include/nettle/nettle-stdint.h", line 237: warning: typedef redeclared: gl_int_fast8_t > "/opt/csw/include/nettle/nettle-stdint.h", line 238: warning: typedef redeclared: gl_int_fast16_t > "/opt/csw/include/nettle/nettle-stdint.h", line 239: warning: typedef redeclared: gl_int_fast32_t > "/opt/csw/include/nettle/nettle-stdint.h", line 241: warning: typedef redeclared: int64_t > "/opt/csw/include/nettle/nettle-stdint.h", line 244: warning: typedef redeclared: gl_uint_fast8_t > "/opt/csw/include/nettle/nettle-stdint.h", line 245: warning: typedef redeclared: gl_uint_fast16_t > "/opt/csw/include/nettle/nettle-stdint.h", line 246: warning: typedef redeclared: gl_uint_fast32_t > "/opt/csw/include/nettle/nettle-stdint.h", line 248: warning: typedef redeclared: uint64_t I don't understand why you get "uint64_t" rather than "gl_uint64_t", it seems gnulib handles this type (and int64_t) differently from the rest. But if it works now, despite these warnings, I think we can consider this problem solved. Do you agree? (And then I also have the fix in nettle to avoid defining the *int_fast*_t types at all, which you have tested previously and which also solved the problem, right?) Regards, /Niels -- Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26. Internet email is subject to wholesale government surveillance.