From nisse at lysator.liu.se  Wed Jan  5 22:48:22 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 05 Jan 2011 22:48:22 +0100
Subject: howto use openssl generated rsa keys with nettle
In-Reply-To: <4D245F61.9000700@tut.fi> (Jyke Jokinen's message of "Wed, 05 Jan
	2011 14:09:05 +0200")
References: <4D245F61.9000700@tut.fi>
Message-ID: <nnsjx7xbrt.fsf@stalhein.lysator.liu.se>

Jyke Jokinen <jyke.jokinen at tut.fi> writes:

> In private key the fields are (openssl rsa -in key.pem -text):
> privateExponent:
> prime1:
> prime2:
> exponent1:
> exponent2:
> coefficient:
>
> Prime1&2 should be the factors p and q in rsa_private_key(?)
> What values should be used for a, b and c?

If n = p q, e is the public exponent and d is the private exponent, then
one should have

  a = d mod (p - 1)
  b = d mod (q - 1)
  c = q^{-1} (mod p)

I've tried to explain this in rsa.h, suggestions for improving the
documentation are welcome.

I'm pretty sure openssl uses the same values. Off the top of my head I
can't say for sure how to pair together nettle's names and openssl's,
but I think they're in the same order, i.e.,

  privateExponent = d
  prime1 = p
  prime2 = q
  exponent1 = a
  exponent2 = b
  coefficient = c

Anyway, a, b, and c are redundant and can be computed from p, q and d,
so you can check if the pairing above is correct.

For converting keys in pkcs#1 format, also have a look at the program
examples/pkcs1-conv.c, and the function rsa_keypair_from_der which it
uses.

Regards,
/Niels

PS. I've added your address to the non-member whitelist. I hope that
works, and that you can now post to the list. (Default posting policy is
members-only, as a desperate anti-spam measure).

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Fri Jan  7 22:11:45 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 07 Jan 2011 22:11:45 +0100
Subject: elliptic curve in nettle?
In-Reply-To: <4D114271.7060007@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Tue, 21 Dec 2010 19:12:33 -0500")
References: <4D0FDAAB.7060702@fifthhorseman.net>
	<nnpqsvzdfw.fsf@stalhein.lysator.liu.se>
	<4D114271.7060007@fifthhorseman.net>
Message-ID: <nnk4igxvu6.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

>> patents situation,
>
> Unfortunately, the patent system seems to be such that even if i were a
> patent lawyer (i am not, fortunately), i could make no iron-clad
> guarantees.  The best i can offer is a sort of suggestive inference:

I'm aware that there are no guarantees. I think a reasonable requirement
for implementing a new algorithm (ecc or otherwise) is that at least all
known patent holders license related patents on royalty-free (and
otherwise GPL compatible) terms.

Jeffrey Walton <noloader at gmail.com> writes:

> I think Daniel is on the right track by choosing standardized domain
> parameter support. When using standard parameters, you only have to
> choose your private key and publish the public key. In this case, RFC
> 5114 - Additional Diffie-Hellman Groups for Use with IETF Standards
> (et al), would also be of interest.

Noted.

> For those interested, Certicom, which holds many EC patents and is
> owned by RIM, lost a few "slam dunk" cases recently. The events caused
> paralysis in RIM's legal department to the point where the sales team
> has not inked a license in over a year. When I inquired about
> licensing over the summer, I was told to go to RSA Data Securities
> even though RSA is probably violating Certicom. The fellow who advised
> me worked for Certicom.

Intriguing story... as I've said, I haven't been following the area.

My conclusions for now are:

1. It makes sense to add support for certain elliptic curves or types of
   curves to nettle. I'm still not quite sure what the applications are,
   diffie-hellman key exchange have been mentioned, do the most
   important standards also use them for encryption and signatures
   (e.g., ElGamal style)? The implementation ought to include an
   ecc exponentiation primitive that can be used for various applications.

   (BTW, Nettle currently doesn't include any support for ElGamal using
   the usual modular group, is that something that would be useful? I
   try to give higher priority to algorithms that are in used in real
   protocols and applications, and lower priority to more academic
   constructions).
   
2. I will not have much time to spend on ecc in the near future. I'm
   happy to comment on, and integrate, well-written patches. As usual,
   test cases are a very important part of the implementation...

3. On the legal side, I'd like to have some clear evidence that the
   particular curves implemented are unlikely to lead to trouble with
   known patents, possibly with fsf legal staff or sflc in the loop. I'm
   not sure I know the area well enough to provide all needed input to
   legal staff, though, so I may need help with this part as well.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Fri Jan  7 22:43:39 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Fri, 07 Jan 2011 16:43:39 -0500
Subject: elliptic curve in nettle?
In-Reply-To: <nnk4igxvu6.fsf@stalhein.lysator.liu.se>
References: <4D0FDAAB.7060702@fifthhorseman.net>	<nnpqsvzdfw.fsf@stalhein.lysator.liu.se>	<4D114271.7060007@fifthhorseman.net>
	<nnk4igxvu6.fsf@stalhein.lysator.liu.se>
Message-ID: <4D27890B.4070404@fifthhorseman.net>

On 01/07/2011 04:11 PM, Niels M?ller wrote:
> 1. It makes sense to add support for certain elliptic curves or types of
>    curves to nettle. I'm still not quite sure what the applications are,
>    diffie-hellman key exchange have been mentioned, do the most
>    important standards also use them for encryption and signatures
>    (e.g., ElGamal style)? The implementation ought to include an
>    ecc exponentiation primitive that can be used for various applications.
> 
>    (BTW, Nettle currently doesn't include any support for ElGamal using
>    the usual modular group, is that something that would be useful? I
>    try to give higher priority to algorithms that are in used in real
>    protocols and applications, and lower priority to more academic
>    constructions).

OpenSSH 5.7 (due out later this month) will add the use of Elliptic
Curve DH and DSA.  Interoperability with OpenSSH by ssh clients using
nettle would be an excellent real-world scenario.

ElGamal is still widely used for asymmetric OpenPGP encryption.  Try
scanning the public keyservers for people with ElGamal subkeys (i wish i
had some easy way to present statistics from them -- sorry i don't!)

so yes, both EC and ElGamal have very clear real-world (non-academic)
usefulness.

> 3. On the legal side, I'd like to have some clear evidence that the
>    particular curves implemented are unlikely to lead to trouble with
>    known patents, possibly with fsf legal staff or sflc in the loop. I'm
>    not sure I know the area well enough to provide all needed input to
>    legal staff, though, so I may need help with this part as well.

i'll point the SFLC lawyers at this thread.  hopefully they can get in
touch.

	--dkg


From simon at josefsson.org  Fri Jan  7 22:58:33 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Fri, 07 Jan 2011 22:58:33 +0100
Subject: elliptic curve in nettle?
In-Reply-To: <nnk4igxvu6.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Fri, 07 Jan 2011 22:11:45 +0100")
References: <4D0FDAAB.7060702@fifthhorseman.net>
	<nnpqsvzdfw.fsf@stalhein.lysator.liu.se>
	<4D114271.7060007@fifthhorseman.net>
	<nnk4igxvu6.fsf@stalhein.lysator.liu.se>
Message-ID: <87oc7scr5i.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> 3. On the legal side, I'd like to have some clear evidence that the
>    particular curves implemented are unlikely to lead to trouble with
>    known patents, possibly with fsf legal staff or sflc in the loop. I'm
>    not sure I know the area well enough to provide all needed input to
>    legal staff, though, so I may need help with this part as well.

I believe the SFLC is aware of the ECC situation, so you may want to
talk to them about it.  It is a messy area.

/Simon

From nisse at lysator.liu.se  Wed Feb  2 22:15:50 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 02 Feb 2011 22:15:50 +0100
Subject: GCM mode and GMAC
Message-ID: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>

Hi,

Nikos Mavrogiannopoulos have been looking into support for Galois
Counter Mode (GCM), see http://www.cryptobarn.com/papers/gcm-spec.pdf

My understanding of GCM is that the main point is a new MAC function
which allows efficient hardware implementation. As far as I see, there's
no clear advantage of using GCM instead of plain CTR mode combined with
the same MAC function (applied to the plaintext).

For Nettle, I think the first step ought to be to properly support the
MAC function, GMAC. The most fundamental difference to other MAC
functions is that it takes two input strings (besides the key). When
used as a plain MAC, the second input is empty, while when used with
GCM, the first input is auxillary data to be authenticated, and the
second input is the cryptotext.

Some questions:

  * Naming: Is "gmac" a good enough name? Or "ghash" (the name of the
    primitive which takes a key and two inputs, in the paper)? Or do we
    need something more verbose, like galois_mac or gmac128 or so?
    
  * Specification: It's not entirely clear to me how the spec is to be
    interpreted when one of the input strings is empty. The most
    reasonable interpretation would be that there should be zero blocks
    to process (n or m equal to zero). This requires some bending of the
    notation in equation (2), for example, with m = 0, n = 1, we should
    have

      X_0 = 0
      X_1 = C_1^* ? H
      X_2 = (X_1 + (0 || len(C))) ? H

    and with m = 1, n = 0,

      X_0 = 0
      X_1 = A_1^* ? H
      X_2 = (X_1 0 (len(A) || 0)) ? H

    Do you agree?
    
  * Interface: I think the basic use case with empty second input should
    be just like other MAC:s,

      struct gmac_ctx;
      
      /* Key size fixed to GMAC_KEY_SIZE == 16 */
      void
      gmac_set_key(struct gmac_ctx *ctx, const uint8_t *key);
      
      void
      gmac_update(struct gmac_ctx *ctx,
		  unsigned length, const uint8_t *data);
      
      void
      gmac_digest(struct gmac_ctx *ctx,
		  unsigned length, uint8_t *digest);

    The context struct and the set_key function is essential to be able
    to do any optimizations using key-dependant tables.
    
    But then we need a function to mark the end of the first input and
    the start of the second. Name for that one?

      void
      gmac_next(struct gmac_ctx *ctx);

    This will pad the current input to a block boundary, and switch to
    using a different length counter.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.


From simon at josefsson.org  Thu Feb  3 09:52:38 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Thu, 03 Feb 2011 09:52:38 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnfws6qgs9.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Wed, 02 Feb 2011 22:15:50 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
Message-ID: <87sjw5fqjt.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Hi,
>
> Nikos Mavrogiannopoulos have been looking into support for Galois
> Counter Mode (GCM), see http://www.cryptobarn.com/papers/gcm-spec.pdf

Hi!  NIST Has some other links for it:

http://csrc.nist.gov/groups/ST/toolkit/BCM/modes_development.html

> My understanding of GCM is that the main point is a new MAC function
> which allows efficient hardware implementation. As far as I see, there's
> no clear advantage of using GCM instead of plain CTR mode combined with
> the same MAC function (applied to the plaintext).

Shouldn't you MAC the ciphertext?  That's the proved secure approach.

>   * Naming: Is "gmac" a good enough name? Or "ghash" (the name of the
>     primitive which takes a key and two inputs, in the paper)? Or do we
>     need something more verbose, like galois_mac or gmac128 or so?

The name GMAC is well established:

http://en.wikipedia.org/wiki/Galois/Counter_Mode

>   * Specification: It's not entirely clear to me how the spec is to be
>     interpreted when one of the input strings is empty. The most
>     reasonable interpretation would be that there should be zero blocks
>     to process (n or m equal to zero). This requires some bending of the
>     notation in equation (2), for example, with m = 0, n = 1, we should
>     have

If this is a real problem with latest specification, it might make sense
to bring this up somewhere.

>   * Interface: I think the basic use case with empty second input should
>     be just like other MAC:s,
>
>       struct gmac_ctx;
>       
>       /* Key size fixed to GMAC_KEY_SIZE == 16 */
>       void
>       gmac_set_key(struct gmac_ctx *ctx, const uint8_t *key);
>       
>       void
>       gmac_update(struct gmac_ctx *ctx,
> 		  unsigned length, const uint8_t *data);
>       
>       void
>       gmac_digest(struct gmac_ctx *ctx,
> 		  unsigned length, uint8_t *digest);
>
>     The context struct and the set_key function is essential to be able
>     to do any optimizations using key-dependant tables.
>     
>     But then we need a function to mark the end of the first input and
>     the start of the second. Name for that one?
>
>       void
>       gmac_next(struct gmac_ctx *ctx);
>
>     This will pad the current input to a block boundary, and switch to
>     using a different length counter.

How about gmac_authenticate?

Further, I'm wondering if some other authenticating MACs cannot process
data in parallell, which would argue for an interface like this:

       struct gmac_ctx;
       
       /* Key size fixed to GMAC_KEY_SIZE == 16 */
       void
       gmac_set_key(struct gmac_ctx *ctx, const uint8_t *key);
       
       void
       gmac_update(struct gmac_ctx *ctx,
 		  unsigned length, const uint8_t *data);
       
       void
       gmac_digest(struct gmac_ctx *ctx,
 		  unsigned length, uint8_t *digest);

       void
       gmac_authenticate(struct gmac_ctx *ctx,
 		         unsigned length, const uint8_t *data);

Or something.

/Simon

From nisse at lysator.liu.se  Thu Feb  3 13:40:47 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 03 Feb 2011 13:40:47 +0100
Subject: GCM mode and GMAC
In-Reply-To: <87sjw5fqjt.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Thu, 03 Feb 2011 09:52:38 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<87sjw5fqjt.fsf@latte.josefsson.org>
Message-ID: <nn7hdhqoj4.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> http://csrc.nist.gov/groups/ST/toolkit/BCM/modes_development.html

I was just wondering what an "authoritative" reference would be.

> Shouldn't you MAC the ciphertext?  That's the proved secure approach.

I'm not familiar with those subtleties. Intuitively, it makes sense to me
to MAC the cleartext, since that's closest to the *meaning* of the
message. IIRC I picked up this (possibly outdated) advice from Applied
Cryptography years ago.

If you can trick the receiver to use the wrong key or iv for decryption,
then the receiver gets a garbled message, and if the MAC is applied to
cryptotext rather than cleartext, the message will still apear to be
authentic. So some care is needed when applying the MAC to ciphertext
only (and I'm talking about the general combination of encryption and
MAC, not the specific combination in GCM which I hope gets things
right).

E.g., in ssh the mac is done as

     mac = MAC(key, sequence_number || unencrypted_packet)

and replacing the unencrypted_packet by the corresponding ciphertext,
with no other changes, would likely cause some trouble.

>>   * Naming: Is "gmac" a good enough name? Or "ghash" (the name of the
>>     primitive which takes a key and two inputs, in the paper)? Or do we
>>     need something more verbose, like galois_mac or gmac128 or so?
>
> The name GMAC is well established:
>
> http://en.wikipedia.org/wiki/Galois/Counter_Mode

Ok. gmac it should be then. Or perhaps gmac128, in case anyone is using
64-bit gmac or planning for larger sizes?

> If this is a real problem with latest specification, it might make sense
> to bring this up somewhere.

I'd have to check the NIST version of the spec.

> Further, I'm wondering if some other authenticating MACs cannot process
> data in parallell, which would argue for an interface like this:
>
>        struct gmac_ctx;
>        
>        /* Key size fixed to GMAC_KEY_SIZE == 16 */
>        void
>        gmac_set_key(struct gmac_ctx *ctx, const uint8_t *key);
>        
>        void
>        gmac_update(struct gmac_ctx *ctx,
>  		  unsigned length, const uint8_t *data);
>        
>        void
>        gmac_digest(struct gmac_ctx *ctx,
>  		  unsigned length, uint8_t *digest);
>
>        void
>        gmac_authenticate(struct gmac_ctx *ctx,
>  		         unsigned length, const uint8_t *data);
>
> Or something.

I'm not sure I understand what you are referring to. At least for gmac,
I don't think one can mix the two inputs, one must complete one before
starting on the other. And I'd prefer that this restriction is clearly
expressed in the interface.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From noloader at gmail.com  Thu Feb  3 17:35:25 2011
From: noloader at gmail.com (Jeffrey Walton)
Date: Thu, 3 Feb 2011 11:35:25 -0500
Subject: GCM mode and GMAC
In-Reply-To: <nn7hdhqoj4.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<87sjw5fqjt.fsf@latte.josefsson.org>
	<nn7hdhqoj4.fsf@stalhein.lysator.liu.se>
Message-ID: <AANLkTinGY_JC8mVyiqRef0z96jC_gKbhBEYuzugfDaew@mail.gmail.com>

On Thu, Feb 3, 2011 at 7:40 AM, Niels M?ller <nisse at lysator.liu.se> wrote:
> Simon Josefsson <simon at josefsson.org> writes:
>
>> http://csrc.nist.gov/groups/ST/toolkit/BCM/modes_development.html
>
> I was just wondering what an "authoritative" reference would be.
NIST SP800-38D. See
http://csrc.nist.gov/groups/ST/toolkit/BCM/current_modes.html and
http://csrc.nist.gov/publications/nistpubs/800-38D/SP-800-38D.pdf.

[SNIP]

Jeff

From nisse at lysator.liu.se  Thu Feb  3 19:49:11 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 03 Feb 2011 19:49:11 +0100
Subject: GCM mode and GMAC
In-Reply-To: <87sjw5fqjt.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Thu, 03 Feb 2011 09:52:38 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<87sjw5fqjt.fsf@latte.josefsson.org>
Message-ID: <nnpqr9oswo.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> The name GMAC is well established:
>
> http://en.wikipedia.org/wiki/Galois/Counter_Mode

And if I understand the spec correctly, T = GMAC(K, M) is computed
roughly as follows

  H = E_K(0...0)
  T = GHASH_H(M || ...) XOR E_K(IV)

I.e, the MAC key K is converted to the "hash subkey H" using the
encryption function of some block cipher (typically AES), and this block
cipher is also used together with the IV to get a value XOR:ed to the
output of GHASH.

I imagine the final XOR is essential for the MAC security (to hide the
otherwise very regular algebraic structure of GHASH).

When writing the previous mail, I hadn't realized that also the MAC part
depends on the block cipher, and should be parametrized by the block
cipher used. This makes it less natural to view GMAC as an independent
algorithm.

Also, the need for an IV (never repeated with the same key) necessarily
makes the interface more complex than, e.g., the HMAC interface.

Just like for DSA, where would be some use for a deterministic variant
where the IV (or random number in the case of DSA) is determined as some
function of the message (and possibly also of the key, although the
dependence on the key clearly should be one-way).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Thu Feb  3 21:20:57 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 03 Feb 2011 21:20:57 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnfws6qgs9.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Wed, 02 Feb 2011 22:15:50 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
Message-ID: <nnlj1wq386.fsf@stalhein.lysator.liu.se>

Replying to myself:

>     The context struct and the set_key function is essential to be able
>     to do any optimizations using key-dependant tables.

It actually gets a bit complicated. I think we need several
context/state structs.

First, the key. The only way the key is directly used is for encrypting
things using the underlying block cipher. So it makes sense to represent
the key as

  const void *cipher_ctx;
  nettle_crypt_func *encrypt;

(except that for historical reasons, the first argument for
nettle_crypt_func is not const).

Next, we have tables derived from the key. These are needed for
otpimized implementations of ghash, and should be reused when several
messages are processed using the same key. E.g., this could be done as

  struct gcm128_ctx
  {
    const void *cipher_ctx;
    nettle_crypt_func *encrypt;

    uint8_t H[16]; /* Hash subkey */
    uint8_t M0[16][256]; /* Key-dependent table. */
  };

  void
  gcm128_ctx_init(struct gcm128_ctx *ctx,
  		  const void *cipher_ctx,
		  nettle_crypt_func *encrypt);

(It would be more consistent with the rest of nettle to not store those
pointers in gcm128_ctx, since its supposed to be ok to memcpy context
structs around. One could either pass them as arguments to all
functions, or inline an aes_ctx if aes is all we care about. I include
them here, to simplify function prototypes).

Functions to process complete gcm messages can take this ctx as
argument. The main ghash iteration would also take this context struct
as argument,

  /* Set state = (state XOR data) dot H */
  void
  ghash_update(const struct gcm128_ctx *ctx,
  	       uint8_t *state,
               const uint8_t *data);

For streaming operations, we also need a per-message state struct,
something like

  struct gcm128_msg
  {
    uint8_t Y[16];  /* counter */
    uint8_t J[16];  /* encryption of that */
    uint8_t J0[16]; /* first encrypted counter block to be used for constructing the digest. */
    uint8_t hash[16]; /* hashing state */

    unsigned index; /* Index when doing partial blocks */
    uint32_t auth_length;
    uint32_t msg_length;
  };

  void
  gcm128_msg_init(struct gcm128_msg *msg, const struct gcm128_ctx *ctx,
  		  uint32_t iv_length, const uint8_t *iv);

  /* Process auxillary auth data */
  void
  gcm128_msg_auth(struct gcm128_msg *msg, const struct gcm128_ctx *ctx,
  		  uint32_t length, const uint8_t *data);

  /* Only difference between encrypt and decrypt is if data is hashed
     before or after xoring with the key stream. */
  void
  gcm128_msg_encrypt(struct gcm128_msg *msg, const struct gcm128_ctx *ctx,
  		     uint32_t length, uint8_t *dst, const uint8_t *src);

  void
  gcm128_msg_decrypt(struct gcm128_msg *msg, const struct gcm128_ctx *ctx,
  		     uint32_t length, uint8_t *dst, const uint8_t *src);

  gcm128_msg_digest(struct gcm128_msg *msg, const struct gcm128_ctx *ctx,
  		    uint32_t length, uint8_t *dst);

Comments? It gets more complicated than almost anything currently in
Nettle.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Sun Feb  6 00:08:27 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 06 Feb 2011 00:08:27 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnfws6qgs9.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Wed, 02 Feb 2011 22:15:50 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
Message-ID: <nn8vxuoz9w.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> Nikos Mavrogiannopoulos have been looking into support for Galois
> Counter Mode (GCM), see http://www.cryptobarn.com/papers/gcm-spec.pdf

I've checked in a first version, based on Nikos' code. Tentative
interface as follows:

  #define GCM_BLOCK_SIZE 16
  #define GCM_IV_SIZE (GCM_BLOCK_SIZE - 4)
  
  #define GCM_TABLE_BITS 0
  
  struct gcm_ctx {
    /* Key-dependent state. */
    /* Hashing subkey */
    uint8_t h[GCM_BLOCK_SIZE];
  #if GCM_TABLE_BITS
    uint8_t h_table[1 << GCM_TABLE_BITS][GCM_BLOCK_SIZE];
  #endif
    /* Per-message state, depending on the iv */
    /* Original counter block */
    uint8_t iv[GCM_BLOCK_SIZE];
    /* Updated for each block. */
    uint8_t ctr[GCM_BLOCK_SIZE];
    /* Hashing state */
    uint8_t x[GCM_BLOCK_SIZE];
    uint64_t auth_size;
    uint64_t data_size;
  };
  
  /* FIXME: Should use const for the cipher context. Then needs const for
     nettle_crypt_func, which also rules out using that abstraction for
     arcfour. */
  void
  gcm_set_key(struct gcm_ctx *ctx,
  	    void *cipher, nettle_crypt_func *f);
  
  void
  gcm_set_iv(struct gcm_ctx *ctx, unsigned length, const uint8_t *iv);
  
  void
  gcm_auth(struct gcm_ctx *ctx, unsigned length, const uint8_t *data);
  
  void
  gcm_encrypt(struct gcm_ctx *ctx, void *cipher, nettle_crypt_func *f,
  	    unsigned length, uint8_t *dst, const uint8_t *src);
  
  void
  gcm_decrypt(struct gcm_ctx *ctx, void *cipher, nettle_crypt_func *f,
  	    unsigned length, uint8_t *dst, const uint8_t *src);
  
  void
  gcm_digest(struct gcm_ctx *ctx, void *cipher, nettle_crypt_func *f,
  	   unsigned length, uint8_t *digest);

Comments on both structure and naming are welcome.

> My understanding of GCM is that the main point is a new MAC function
> which allows efficient hardware implementation.

The unoptimized GF(2^128) multiply function really is awfully slow. On
x86_64, gmac takes 830 cycles/byte! We can compare to the sha functions,
where sha1, sha256 and sha512 take respectively 8, 18 and 12
cycles/byte, so the current code is two orders of magnitude slower than
hmac-sha1.

It remains to see how much table space and/or assembly hacking is needed
to get reasonable performance.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Sun Feb  6 00:30:56 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sun, 06 Feb 2011 00:30:56 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
Message-ID: <4D4DDDB0.9070908@gnutls.org>

On 02/06/2011 12:08 AM, Niels M?ller wrote:

> The unoptimized GF(2^128) multiply function really is awfully slow. On
> x86_64, gmac takes 830 cycles/byte! We can compare to the sha functions,
> where sha1, sha256 and sha512 take respectively 8, 18 and 12
> cycles/byte, so the current code is two orders of magnitude slower than
> hmac-sha1.
> It remains to see how much table space and/or assembly hacking is needed
> to get reasonable performance.

There is a special instruction for that on new intel and AMD CPUs...
http://software.intel.com/en-us/articles/intel-carry-less-multiplication-instruction-and-its-usage-for-computing-the-gcm-mode/
http://en.wikipedia.org/wiki/CLMUL_instruction_set

Unfortunately I don't have anything close to those cpus...

regards,
Nikos

From nisse at lysator.liu.se  Sun Feb  6 22:23:56 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 06 Feb 2011 22:23:56 +0100
Subject: GCM mode and GMAC
In-Reply-To: <4D4DDDB0.9070908@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sun, 06 Feb 2011 00:30:56 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4DDDB0.9070908@gnutls.org>
Message-ID: <nn4o8gq2kz.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> On 02/06/2011 12:08 AM, Niels M?ller wrote:
>
>> It remains to see how much table space and/or assembly hacking is needed
>> to get reasonable performance.
>
> There is a special instruction for that on new intel and AMD CPUs...
> http://software.intel.com/en-us/articles/intel-carry-less-multiplication-instruction-and-its-usage-for-computing-the-gcm-mode/
> http://en.wikipedia.org/wiki/CLMUL_instruction_set

Interesting. I haven't played with any such special instructions (even
if it ought to make a bit of difference also for aes).

Anyway, I've been hacking a bit on the C-implementation over the day,
and the galois hashing (gmac) is now 18 times(!) faster. Summary of
changes:

  Original unoptimized code:
  
           Algorithm        mode Mbyte/s cycles/byte cycles/block
                gmac        auth    1.49      829.83     13277.27
  
  Optimized rshift, rewritten to use word-sized operations:
  
           Algorithm        mode Mbyte/s cycles/byte cycles/block
                gmac        auth    4.62      268.14      4290.23
  
  Optimized gf_mul, rewritten to use separate byte and bit loops:
  
           Algorithm        mode Mbyte/s cycles/byte cycles/block
                gmac        auth    5.46      227.18      3634.90
  
  Moved reduction into shift function:
  
           Algorithm        mode Mbyte/s cycles/byte cycles/block
                gmac        auth    6.79      182.69      2923.02
  
  Introduced 4-bit tables:
  
           Algorithm        mode Mbyte/s cycles/byte cycles/block
                gmac        auth   27.14       45.68       730.82

Remaining tricks:

* Try 8-bit tables (which increases storage need a lot, the modest 4-bit
  tables need only 256 additional bytes per key, for gf_mul, and a
  32-byte constant table for gf_shift. Extending to 8 bits makes both
  tables 16 times larger).

* See if it makes sense to write any assembler for the hashing function.

* Various smaller microoptimizations, like a public memxor-variant for
  when areas are known to be word-aligned. Or inline that xor:ing.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Sun Feb  6 22:53:49 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sun, 06 Feb 2011 22:53:49 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
Message-ID: <4D4F186D.2090404@gnutls.org>

On 02/06/2011 12:08 AM, Niels M?ller wrote:

> void gcm_set_key(struct gcm_ctx *ctx, void *cipher, nettle_crypt_func
> *f);

I don't like the name of the function name. It doesn't reveal anything
about its purpose. There is no key to set there. I'd suggest the
original gcm_init.

Moreover by not allowing the setting the blocksize as option any
extension on that code to work with 64-bit ciphers, will require
an abi break, or a new gcm64 mode... (what if 256-bit ciphers are added
in the future?)

> void gcm_encrypt(struct gcm_ctx *ctx, void *cipher, nettle_crypt_func
> *f, unsigned length, uint8_t *dst, const uint8_t *src); void 
> gcm_digest(struct gcm_ctx *ctx, void *cipher, nettle_crypt_func *f, 
> unsigned length, uint8_t *digest);

As I already mentioned I prefer having the cipher and f, to context
to avoid supplying on individual calls. There is no advantage (that I
can see) on having on each function parameters, and it just delegates
the storage of those two pointers, to caller's structures instead. It's
no big deal but it is inconvenience.

regards,
Nikos

From nmav at gnutls.org  Sun Feb  6 22:54:46 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sun, 06 Feb 2011 22:54:46 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nn4o8gq2kz.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4DDDB0.9070908@gnutls.org>
	<nn4o8gq2kz.fsf@stalhein.lysator.liu.se>
Message-ID: <4D4F18A6.3020009@gnutls.org>

On 02/06/2011 10:23 PM, Niels M?ller wrote:
> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
> 
>> On 02/06/2011 12:08 AM, Niels M?ller wrote:
>>
>>> It remains to see how much table space and/or assembly hacking is needed
>>> to get reasonable performance.
>>
>> There is a special instruction for that on new intel and AMD CPUs...
>> http://software.intel.com/en-us/articles/intel-carry-less-multiplication-instruction-and-its-usage-for-computing-the-gcm-mode/
>> http://en.wikipedia.org/wiki/CLMUL_instruction_set
> 
> Interesting. I haven't played with any such special instructions (even
> if it ought to make a bit of difference also for aes).
> 
> Anyway, I've been hacking a bit on the C-implementation over the day,
> and the galois hashing (gmac) is now 18 times(!) faster. Summary of
> changes:

[...]
>   Introduced 4-bit tables:
>   
>            Algorithm        mode Mbyte/s cycles/byte cycles/block
>                 gmac        auth   27.14       45.68       730.82

That's pretty impressive!


From nisse at lysator.liu.se  Mon Feb  7 10:26:47 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 07 Feb 2011 10:26:47 +0100
Subject: GCM mode and GMAC
In-Reply-To: <4D4F186D.2090404@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sun, 06 Feb 2011 22:53:49 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4F186D.2090404@gnutls.org>
Message-ID: <nnzkq8nqjs.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> On 02/06/2011 12:08 AM, Niels M?ller wrote:
>
>> void gcm_set_key(struct gcm_ctx *ctx, void *cipher, nettle_crypt_func
>> *f);
>
> I don't like the name of the function name. It doesn't reveal anything
> about its purpose. There is no key to set there. I'd suggest the
> original gcm_init.

I see your point, and I think I'll change the name. If we introduce a
gcm_aes wrapper, that will have a _set_key method.

> Moreover by not allowing the setting the blocksize as option any
> extension on that code to work with 64-bit ciphers, will require
> an abi break, or a new gcm64 mode... (what if 256-bit ciphers are added
> in the future?)

I don't think it's useful to be that general here. Variants with
different block sizes will likely need a different context struct
anyway.

I could rename gcm to gcm128 in the interface if that's clearer.

> As I already mentioned I prefer having the cipher and f, to context
> to avoid supplying on individual calls. There is no advantage (that I
> can see) on having on each function parameters, and it just delegates
> the storage of those two pointers, to caller's structures instead. It's
> no big deal but it is inconvenience.

I haven't yet made up my mind on this, but let me explain the reason for
having these pointers as function arguments.

The idea is that context structs in nettle should be non-magic with no
pointers, so that it can be copied or relocated in memory at will. Say
we implement gcm_aes as

  struct gcm_aes_ctx {
    struct gcm_ctx gcm;
    struct aes_ctx aes;
  };

  void
  gcm_aes_encrypt(struct gcm_aes_ctx *ctx,
  		  unsigned length, uint8_t *dst, const uint8_t *src)
  {
    gcm_encrypt(&ctx->gcm, &ctx->aes, (nettle_crypt_func) aes_encrypt,
    	        length, dst, src);
  }

Then the context struct is still non-magic. We can call gcm_aes_set_key,
and then create multiple copies of gcm_aes_ctx (using plain memcpy)
which are independent. If we add some pointers to struct gcm_ctx (which
*does* increase the storage size of gcm_aes_ctx, although that's maybe
not a big deal), plain copying will leave pointers pointing to other
objects, and we'll have to introduce a function gcm_aes_copy.

And if you think copying here is the wrong thing (since we waste memory
with multiple identical copies of an aes_ctx representing the key), then
I think one should also split gcm_ctx into a key-dependent part (which
should be shared rather than copied, in particular if we go for larger
key-dependent tables) and a message-dependent part (which also shouldn't
be copied, instead multiple instances should be independently
initialized).

We can compare with the hmac code; there's no context struct for the
general construction, but hmac_sha1_ctx and other's defined using the
HMAC_CTX macro put both key-the dependent parts (.inner, .outer) and the
message dependent part (.state) in a single struct.

When thinking about it, maybe the right thing is to redesign the general
gcm-code to use a separate struct for the hash subkey, passed as
argument to the functions needing it.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Mon Feb  7 11:04:17 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 07 Feb 2011 11:04:17 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nn4o8gq2kz.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4DDDB0.9070908@gnutls.org>
	<nn4o8gq2kz.fsf@stalhein.lysator.liu.se>
Message-ID: <4D4FC3A1.20206@gnutls.org>

On 02/06/2011 10:23 PM, Niels M?ller wrote:

> Interesting. I haven't played with any such special instructions (even
> if it ought to make a bit of difference also for aes).
> Anyway, I've been hacking a bit on the C-implementation over the day,
> and the galois hashing (gmac) is now 18 times(!) faster. Summary of
> changes:

I've also done a comparison benchmark of AES-GCM (the 4-bit table one)
versus HMAC-SHAx+AES-CBC... AES-GCM in software is disappointing...

Checking AES-128-GCM (16kb payload)... Encrypted 97.67 Mb in 5.00 secs:
19.53 Mb/sec

Checking AES-128-CBC with SHA256 (16kb payload)... Encrypted and hashed
246.14 Mb in 5.00 secs: 49.23 Mb/sec

Checking AES-128-CBC with SHA1 (16kb payload)... Encrypted and hashed
354.16 Mb in 5.00 secs: 70.83 Mb/sec


regards,
Nikos

From nmav at gnutls.org  Mon Feb  7 11:35:15 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 07 Feb 2011 11:35:15 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnzkq8nqjs.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4F186D.2090404@gnutls.org>
	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se>
Message-ID: <4D4FCAE3.6030306@gnutls.org>

On 02/07/2011 10:26 AM, Niels M?ller wrote:

>> Moreover by not allowing the setting the blocksize as option any 
>> extension on that code to work with 64-bit ciphers, will require
>> an abi break, or a new gcm64 mode... (what if 256-bit ciphers are 
>> added in the future?)
> I don't think it's useful to be that general here. Variants with 
> different block sizes will likely need a different context struct 
> anyway.

Indeed...

> I could rename gcm to gcm128 in the interface if that's clearer.

I like the plain gcm...

>> As I already mentioned I prefer having the cipher and f, to context
>> to avoid supplying on individual calls. There is no advantage (that
>> I can see) on having on each function parameters, and it just
>> delegates the storage of those two pointers, to caller's structures
>> instead. It's no big deal but it is inconvenience.
> I haven't yet made up my mind on this, but let me explain the reason 
> for having these pointers as function arguments. The idea is that 
> context structs in nettle should be non-magic with no pointers, so 
> that it can be copied or relocated in memory at will. Say we 
> implement gcm_aes as
[...]
It makes sense, and although I've never used context structs like that
I could understand if someone did. I like to see context structures as
things that will take away my burden of maintaining several pointers and
data that relate to the operation.

Moreover different AEAD modes might have different requirements, but
it might be nice to have a consistent low level interface on them (if
possible of course).
If an AEAD mode doesn't require the encryption function at
the _digest operation, it would mean it would have different function
parameters. For me it would be best if it was consistent, even if
it is low-level... but it's your call...

[...]
> When thinking about it, maybe the right thing is to redesign the 
> general gcm-code to use a separate struct for the hash subkey,
> passed as argument to the functions needing it.

I would like less of the internals of gcm exposed to the user rather
than more. As a user of nettle I wouldn't even want to know that there
is a hash subkey on gcm.

regards,
Nikos

From nisse at lysator.liu.se  Mon Feb  7 12:21:42 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 07 Feb 2011 12:21:42 +0100
Subject: GCM mode and GMAC
In-Reply-To: <4D4FCAE3.6030306@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Mon, 07 Feb 2011 11:35:15 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4F186D.2090404@gnutls.org>
	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se> <4D4FCAE3.6030306@gnutls.org>
Message-ID: <nnvd0wnl89.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> I would like less of the internals of gcm exposed to the user rather
> than more. As a user of nettle I wouldn't even want to know that there
> is a hash subkey on gcm.

In any case we should probably have a gcm_aes interface (and whatever
other variants are relevant) that is easier to use than the lowest level
gcm interface.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Mon Feb  7 13:20:20 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 07 Feb 2011 13:20:20 +0100
Subject: GCM mode and GMAC
In-Reply-To: <4D4FC3A1.20206@gnutls.org> (Nikos Mavrogiannopoulos's message of
	"Mon, 07 Feb 2011 11:04:17 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4DDDB0.9070908@gnutls.org>
	<nn4o8gq2kz.fsf@stalhein.lysator.liu.se> <4D4FC3A1.20206@gnutls.org>
Message-ID: <nnr5bkniij.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> I've also done a comparison benchmark of AES-GCM (the 4-bit table one)
> versus HMAC-SHAx+AES-CBC... AES-GCM in software is disappointing...

Now I've tried 8-bit tables. Then I get into the same ballpark as md5
and the sha functions (benchmarking on intel x86_64):

         Algorithm        mode Mbyte/s cycles/byte cycles/block
               md5      update  174.20        7.12       455.48
              sha1      update  158.09        7.84       501.89
            sha256      update   68.36       18.14      1160.65
            sha512      update  104.99       11.81      1511.55
              gmac        auth   65.93       18.80       300.87

I think both sha512 and gmac benefit from 64-bit wide registers, while
md5, sha1 and sha256 does not. And I think there are still a couple of
microoptimizations left to do for gmac.

(I'm only benchmarking gmac; the encryption should be about the same as
AES in ECB or CTR mode, which is roughly 17 cycles/byte on the same
hardware).

Now the question is if it's a good tradeoff to expand the key to a 4 KB
table.

BTW, I hadn't noticed before that sha512 is faster per byte than sha256.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Mon Feb  7 13:55:21 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 07 Feb 2011 13:55:21 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnvd0wnl89.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4F186D.2090404@gnutls.org>	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se>
	<4D4FCAE3.6030306@gnutls.org>
	<nnvd0wnl89.fsf@stalhein.lysator.liu.se>
Message-ID: <4D4FEBB9.50909@gnutls.org>

On 02/07/2011 12:21 PM, Niels M?ller wrote:
>> I would like less of the internals of gcm exposed to the user
>> rather than more. As a user of nettle I wouldn't even want to know
>> that there is a hash subkey on gcm.
> 
> In any case we should probably have a gcm_aes interface (and
> whatever other variants are relevant) that is easier to use than the
> lowest level gcm interface.

Could be... Another thing. I've implicitly used gcm_set_iv() as a way to
reset the GCM mode. Unfortunately it is not enough. The auth_size and
data_size have to be set to zero as well. Do you think that should
be done in the set_iv function as well?

I've currently done that in gnutls, and with that change gnutls talks
GCM with others servers.

regards,
Nikos

From nmav at gnutls.org  Mon Feb  7 14:01:15 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 07 Feb 2011 14:01:15 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnr5bkniij.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4DDDB0.9070908@gnutls.org>	<nn4o8gq2kz.fsf@stalhein.lysator.liu.se>
	<4D4FC3A1.20206@gnutls.org>
	<nnr5bkniij.fsf@stalhein.lysator.liu.se>
Message-ID: <4D4FED1B.4020706@gnutls.org>

On 02/07/2011 01:20 PM, Niels M?ller wrote:
> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
> 
>> I've also done a comparison benchmark of AES-GCM (the 4-bit table one)
>> versus HMAC-SHAx+AES-CBC... AES-GCM in software is disappointing...
> Now I've tried 8-bit tables. Then I get into the same ballpark as md5
> and the sha functions (benchmarking on intel x86_64):
>          Algorithm        mode Mbyte/s cycles/byte cycles/block
>                md5      update  174.20        7.12       455.48
>               sha1      update  158.09        7.84       501.89
>             sha256      update   68.36       18.14      1160.65
>             sha512      update  104.99       11.81      1511.55
>               gmac        auth   65.93       18.80       300.87
> I think both sha512 and gmac benefit from 64-bit wide registers, while
> md5, sha1 and sha256 does not. And I think there are still a couple of
> microoptimizations left to do for gmac.
> (I'm only benchmarking gmac; the encryption should be about the same as
> AES in ECB or CTR mode, which is roughly 17 cycles/byte on the same
> hardware).
> Now the question is if it's a good tradeoff to expand the key to a 4 KB
> table.

4kb is not much on a desktop. There are constraint systems where this
might be a problem though. Libtomcrypt had a definition to cope with
this issue (e.g. LOW_FOOTPRINT or so).

On the other hand systems that might have an assembler-optimized
version, would need to share the same big state as well... I don't know.
That's why I like hiding that stuff :)

regards,
Nikos

From nisse at lysator.liu.se  Mon Feb  7 17:01:24 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 07 Feb 2011 17:01:24 +0100
Subject: GCM mode and GMAC
In-Reply-To: <4D4FEBB9.50909@gnutls.org> (Nikos Mavrogiannopoulos's message of
	"Mon, 07 Feb 2011 13:55:21 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4F186D.2090404@gnutls.org>
	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se> <4D4FCAE3.6030306@gnutls.org>
	<nnvd0wnl89.fsf@stalhein.lysator.liu.se> <4D4FEBB9.50909@gnutls.org>
Message-ID: <nnmxm7omuj.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> Could be... Another thing. I've implicitly used gcm_set_iv() as a way to
> reset the GCM mode. Unfortunately it is not enough.

It's intended to work, current gcm_set_iv in cvs does

  /* Reset the rest of the message-dependent state. */
  memset(ctx->x, 0, sizeof(ctx->x));
  ctx->auth_size = ctx->data_size = 0;

Is there something I'm missing?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Mon Feb  7 22:49:22 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 07 Feb 2011 22:49:22 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnmxm7omuj.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4F186D.2090404@gnutls.org>	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se>
	<4D4FCAE3.6030306@gnutls.org>	<nnvd0wnl89.fsf@stalhein.lysator.liu.se>
	<4D4FEBB9.50909@gnutls.org>
	<nnmxm7omuj.fsf@stalhein.lysator.liu.se>
Message-ID: <4D5068E2.9020903@gnutls.org>

On 02/07/2011 05:01 PM, Niels M?ller wrote:
> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
> 
>> Could be... Another thing. I've implicitly used gcm_set_iv() as a
>> way to reset the GCM mode. Unfortunately it is not enough.
> It's intended to work, current gcm_set_iv in cvs does /* Reset the
> rest of the message-dependent state. */ memset(ctx->x, 0,
> sizeof(ctx->x)); ctx->auth_size = ctx->data_size = 0; Is there
> something I'm missing?

No forget it. I was mistaken on the reason of the issue I had. The
current version is ok and inter-operable with.other TLS-GCM versions.

regards,
Nikos

From nmav at gnutls.org  Tue Feb  8 11:01:49 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 08 Feb 2011 11:01:49 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnzkq8nqjs.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4F186D.2090404@gnutls.org>
	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se>
Message-ID: <4D51148D.70203@gnutls.org>

On 02/07/2011 10:26 AM, Niels M?ller wrote:
> I haven't yet made up my mind on this, but let me explain the reason
> for having these pointers as function arguments. The idea is that
> context structs in nettle should be non-magic with no pointers, so
> that it can be copied or relocated in memory at will. Say we
> implement gcm_aes as

What I was wondering is how would you think of implementing the
cpu-specific optimizations in assembly? What I had in mind is that the
_init function would detect the particular instructions
present and set some function pointers to the structure that
will assist the operations such as gf_mul to select the proper
variant... However as it seems, this is not how it can be done
if function pointers are not stored...

regards,
Nikos

From nisse at lysator.liu.se  Tue Feb  8 12:11:25 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 08 Feb 2011 12:11:25 +0100
Subject: GCM mode and GMAC
In-Reply-To: <4D51148D.70203@gnutls.org> (Nikos Mavrogiannopoulos's message of
	"Tue, 08 Feb 2011 11:01:49 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4F186D.2090404@gnutls.org>
	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se> <4D51148D.70203@gnutls.org>
Message-ID: <nn7hdaok6a.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> What I was wondering is how would you think of implementing the
> cpu-specific optimizations in assembly?

If/when we do that, and if we want to be able to select which code to
use at runtime (rater than compile time), I think we should use global
pointers, one for each routine that can make use special instructions.
Either setup automagically at first use, or at library load time (using
the same mechanisms as C++ constructors).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Tue Feb  8 13:09:33 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 08 Feb 2011 13:09:33 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nn7hdaok6a.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4F186D.2090404@gnutls.org>	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se>
	<4D51148D.70203@gnutls.org>
	<nn7hdaok6a.fsf@stalhein.lysator.liu.se>
Message-ID: <4D51327D.2030203@gnutls.org>

On 02/08/2011 12:11 PM, Niels M?ller wrote:

>> What I was wondering is how would you think of implementing the 
>> cpu-specific optimizations in assembly?
> If/when we do that, and if we want to be able to select which code
> to use at runtime (rater than compile time),
The compile time will cause problems to distributions that ship a single
library across compatible architectures. Given that they will ship
the version without special instructions for compatibility, most systems
will not be affected by such optimizations.

> I think we should use global pointers, one for each routine that can
> make use special instructions. Either setup automagically at first
> use, or at library load time (using the same mechanisms as C++
> constructors).

I like the latter... An explicit global library initialization function
might also do.

regards,
Nikos

From nisse at lysator.liu.se  Tue Feb  8 13:26:43 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 08 Feb 2011 13:26:43 +0100
Subject: GCM mode and GMAC
In-Reply-To: <4D51327D.2030203@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Tue, 08 Feb 2011 13:09:33 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4F186D.2090404@gnutls.org>
	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se> <4D51148D.70203@gnutls.org>
	<nn7hdaok6a.fsf@stalhein.lysator.liu.se> <4D51327D.2030203@gnutls.org>
Message-ID: <nn39nyogos.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> On 02/08/2011 12:11 PM, Niels M?ller wrote:
>
>> I think we should use global pointers, one for each routine that can
>> make use special instructions. Either setup automagically at first
>> use, or at library load time (using the same mechanisms as C++
>> constructors).
>
> I like the latter... An explicit global library initialization function
> might also do.

GMP does the former. Not as pretty, but seems to work well, and maybe a
bit easier to do portably.

Regards
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Tue Feb  8 14:21:40 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 08 Feb 2011 14:21:40 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nn39nyogos.fsf@stalhein.lysator.liu.se>
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se>
	<4D4F186D.2090404@gnutls.org>	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se>
	<4D51148D.70203@gnutls.org>	<nn7hdaok6a.fsf@stalhein.lysator.liu.se>
	<4D51327D.2030203@gnutls.org>
	<nn39nyogos.fsf@stalhein.lysator.liu.se>
Message-ID: <4D514364.8080600@gnutls.org>

On 02/08/2011 01:26 PM, Niels M?ller wrote:
> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
> 
>> On 02/08/2011 12:11 PM, Niels M?ller wrote:
>>
>>> I think we should use global pointers, one for each routine that can
>>> make use special instructions. Either setup automagically at first
>>> use, or at library load time (using the same mechanisms as C++
>>> constructors).
>> I like the latter... An explicit global library initialization function
>> might also do.
> GMP does the former. Not as pretty, but seems to work well, and maybe a
> bit easier to do portably.

How does it deal with multi-threaded access?

From nisse at lysator.liu.se  Tue Feb  8 14:36:13 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 08 Feb 2011 14:36:13 +0100
Subject: GCM mode and GMAC
In-Reply-To: <4D514364.8080600@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Tue, 08 Feb 2011 14:21:40 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4F186D.2090404@gnutls.org>
	<nnzkq8nqjs.fsf@stalhein.lysator.liu.se> <4D51148D.70203@gnutls.org>
	<nn7hdaok6a.fsf@stalhein.lysator.liu.se> <4D51327D.2030203@gnutls.org>
	<nn39nyogos.fsf@stalhein.lysator.liu.se> <4D514364.8080600@gnutls.org>
Message-ID: <nny65qmywi.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> How does it deal with multi-threaded access?

The worst thing that can happen is that several threads examine the
cpuid flags, and then write identical pointers to the same locations at
approximately the same time.

And as far as I have understod, that is no problem, because, reading and
writing a pointer value is an atomic operations on the concerned
architectures.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Thu Feb 10 00:30:25 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Thu, 10 Feb 2011 00:30:25 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnfws6qgs9.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Wed, 02 Feb 2011 22:15:50 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
Message-ID: <874o8cu6pa.fsf@latte.josefsson.org>

I looked at recent GCM code and noticed this:

/* FIXME: Should use const for the cipher context. Then needs const for
   nettle_crypt_func, which also rules out using that abstraction for
   arcfour. */
void
gcm_set_key(struct gcm_key *key,
            void *cipher, nettle_crypt_func *f);

However GCM (like CCM) is only specified for block ciphers, and further,
only for 128-bit block ciphers.  Thus I wonder if avoiding use of const
just to let the abstraction support a stream cipher is wise?

/Simon

From nisse at lysator.liu.se  Thu Feb 10 07:27:45 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 10 Feb 2011 07:27:45 +0100
Subject: GCM mode and GMAC
In-Reply-To: <874o8cu6pa.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Thu, 10 Feb 2011 00:30:25 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<874o8cu6pa.fsf@latte.josefsson.org>
Message-ID: <nnpqr0mmji.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> /* FIXME: Should use const for the cipher context. Then needs const for
>    nettle_crypt_func, which also rules out using that abstraction for
>    arcfour. */
> 
> However GCM (like CCM) is only specified for block ciphers, and further,
> only for 128-bit block ciphers.  Thus I wonder if avoiding use of const
> just to let the abstraction support a stream cipher is wise?

This nettle_Crypt_func is not gcm-specific. It is used primarily for the
nettle_cipher class in nettle-meta.h:

  struct nettle_cipher
  {
    const char *name;
    
    unsigned context_size;
    
    /* Zero for stream ciphers */
    unsigned block_size;
  
    /* Suggested key size; other sizes are sometimes possible. */
    unsigned key_size;
  
    nettle_set_key_func *set_encrypt_key;
    nettle_set_key_func *set_decrypt_key;
  
    nettle_crypt_func *encrypt;
    nettle_crypt_func *decrypt;
  };

This currently is used to represent both block and stream ciphers,

  [...]
  extern const struct nettle_cipher nettle_aes256;
  
  extern const struct nettle_cipher nettle_arcfour128;
  
  extern const struct nettle_cipher nettle_camellia128;
  [...]

Currently, arcfour is the only supported stream cipher (they seem to be
out of fashion, are thare any other stream ciphers in use? A5 maybe?)

So the question is, should we decide that nettle_cipher is f?r block
ciphers only (where the encrypt and decrypt functions don't change any
state )? Fitting arcfour and block ciphers into the same
abstraction doesn't make much sense anyway, since they should be used
very differently. Then we can make the context argument const for
nettle_crypt_func, but we'd also have to delete

  extern const struct nettle_cipher nettle_arcfour128;

or replace it with something else, which is an incompatible interface
change. As long as it's the only supported stream cipher, it doesn't
make much sense to me create a new general stream cipher construction.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Thu Feb 10 12:02:19 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 10 Feb 2011 12:02:19 +0100
Subject: GCM updates Re: GCM mode and GMAC
In-Reply-To: <nnr5bkniij.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Mon, 07 Feb 2011 13:20:20 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<nn8vxuoz9w.fsf@stalhein.lysator.liu.se> <4D4DDDB0.9070908@gnutls.org>
	<nn4o8gq2kz.fsf@stalhein.lysator.liu.se> <4D4FC3A1.20206@gnutls.org>
	<nnr5bkniij.fsf@stalhein.lysator.liu.se>
Message-ID: <nnlj1om9tw.fsf_-_@stalhein.lysator.liu.se>

I've done some further updates.

* I've introduced a specialized function gcm_gf_add, used instead of
  memxor when blocks are aligned, and also avoiding looping overhead and
  (if it is inlined, which I think it should be) call overhead. Current
  performance on x86_64 is 28.5 cycles / byte with 4-bit tables (current
  default), and 8.5 cycles / byte with 8-bit tables. Close to a factor
  of two improvement.

* I've introduced a union gcm_block, which is used internally to ensure
  that the gf elements have the right alignment. Tested on sparc32 and
  sparc64 (big endian and pickier about alignment).

* I've split out the message-independent state to a separate struct
  gcm_key, which needs to be passed as argument to all gcm functions.

* I've added a struct gcm_aes_ctx and related functions. This is an
  all-in-one context, including all of the cipher context, the hashing
  subkey, and message state.

* I've added support for IV:s of arbitrary lengths, and added the rest
  of the testcases from http://www.cryptobarn.com/papers/gcm-spec.pdf

* I've simplified the configuration of internal multiplication routines
  a bit, and rewritten the table generation to use just shifts and adds
  (as suggested in http://www.cryptobarn.com/papers/gcm-spec.pdf), which
  means that when tables are used, there's no need to keep the bitwise
  multiplication function which doesn't use tables.

I think the code is stabilizing a bit now.

One naming question: Should gcm_aes_auth be renamed to gcm_aes_update,
for consistency with other hash and mac functions? I'm tempted to do
that.

Regards,
/Niels
  
-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Thu Feb 10 13:26:57 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Thu, 10 Feb 2011 13:26:57 +0100
Subject: Stream ciphers (was: Re: GCM mode and GMAC)
In-Reply-To: <nnpqr0mmji.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Thu, 10 Feb 2011 07:27:45 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<874o8cu6pa.fsf@latte.josefsson.org>
	<nnpqr0mmji.fsf@stalhein.lysator.liu.se>
Message-ID: <87ipwsnkha.fsf_-_@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Currently, arcfour is the only supported stream cipher (they seem to be
> out of fashion, are thare any other stream ciphers in use? A5 maybe?)

There are newer stream ciphers, mostly due to eSTREAM:

http://www.ecrypt.eu.org/stream/

I don't understand the rationale for stream ciphers today though.  The
traditional argument for stream ciphers was speed but you get 10GBps+
with nice modes like AES-GCM.  Further, you can build a secure key
stream generator from any secure block cipher (see for example [1]).

Maybe the argument today is cost of hardware, but for that to be
effective in the long run you have to beat Moore's law.

/Simon

[1] http://csrc.nist.gov/groups/ST/toolkit/BCM/documents/proposedmodes/kfb/kfb-spec.pdf

From simon at josefsson.org  Thu Feb 10 13:38:19 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Thu, 10 Feb 2011 13:38:19 +0100
Subject: GCM mode and GMAC
In-Reply-To: <nnpqr0mmji.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Thu, 10 Feb 2011 07:27:45 +0100")
References: <nnfws6qgs9.fsf@stalhein.lysator.liu.se>
	<874o8cu6pa.fsf@latte.josefsson.org>
	<nnpqr0mmji.fsf@stalhein.lysator.liu.se>
Message-ID: <87ei7gnjyc.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> So the question is, should we decide that nettle_cipher is f?r block
> ciphers only (where the encrypt and decrypt functions don't change any
> state )? Fitting arcfour and block ciphers into the same
> abstraction doesn't make much sense anyway, since they should be used
> very differently. Then we can make the context argument const for
> nettle_crypt_func, but we'd also have to delete
>
>   extern const struct nettle_cipher nettle_arcfour128;
>
> or replace it with something else, which is an incompatible interface
> change. As long as it's the only supported stream cipher, it doesn't
> make much sense to me create a new general stream cipher construction.

Breaking API compatibility is painful.  Other libraries like libgcrypt
also try to use the same interface for both stream ciphers and block
ciphers too.  However, in my experience this makes things difficult at a
higher level -- the distinction perculate up because stream ciphers
doesn't (for example) have a block length so you either have to say it
has a block length of 1 byte (or 1 bit if you support bit-lengths) which
can cause problems if you want to do MAC or other processing on a
block-by-block basis (MAC:ing each byte is not a good idea..).

So I would support making stream ciphers a different beast than block
ciphers as far as the API goes, unless the API change is too painful.

If we had used a object oriented language, there could be a super-class
"cipher" and two sub-classes "block cipher" and "stream cipher".  Then
some functions could take any "cipher" and some (like GCM) could take
any "block cipher".  Fortunately we aren't using OO here though. :-)

/Simon

From nisse at lysator.liu.se  Thu Feb 10 17:04:40 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 10 Feb 2011 17:04:40 +0100
Subject: LGPL blowfish
In-Reply-To: <871v5lzx67.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Mon, 13 Dec 2010 13:15:12 +0100")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
Message-ID: <nnhbcbnaef.fsf@stalhein.lysator.liu.se>

[ Resending, got list address wrong first time. ]

Simon Josefsson <simon at josefsson.org> writes:

> nisse at lysator.liu.se (Niels M?ller) writes:
>
>> Simon Josefsson <simon at josefsson.org> writes:
>>
>>> I'll look at serpent next.
>>
>> Great!
>
> Turns out it was a bit more complicated, will debug some more...

BTW, how far did you get with serpent?

Regards,
/Niels
-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Fri Feb 11 11:07:02 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 11 Feb 2011 11:07:02 +0100
Subject: Serpent
In-Reply-To: <87oc6j6o55.fsf_-_@latte.josefsson.org> (Simon Josefsson's
	message of "Fri, 11 Feb 2011 08:12:06 +0100")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
Message-ID: <nnwrl6lwah.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> nisse at lysator.liu.se (Niels M?ller) writes:
>
>> Maybe you can try adding some of the testvectors at
>> http://www.cs.technion.ac.il/~biham/Reports/Serpent/ to nettle and
>> libgcrypt, and see what happens? (On the nettle side, I'll try to give
>> that a reasonably high priority).
>
> As you noted privately, they fail in nettle but work in libgcrypt.

A while ago, the email masquerading setup was broken on the machine
where I write email. Because of this, the invalid adress
nettle-bugs at lysator.liu.se has appeared in headers of some of my emails,
and then copied into replies to those. The correct address is
nettle-bugs at lists.lysator.liu.se.

> According to Eli's page and the AES submission paper, there is Serpent-0
> and Serpent-1 and the paper discuss (page 22-23) that the key schedule
> has changed.

Do you think nettle's implementation is serpent-0, or is it just broken?
I'm puzzled, because I'm fairly sure I got the test vectors from
serpent's submission package (I could try to double check that), which
if I understand correctly ought to be serpent-1. I vaguely remember I
had some difficulty understanding the organization of the test data,
though.

And I'm sorry if you have wasted some time debugging fully correct key
scheduling.

> I updated the wikipedia article on this:
> http://en.wikipedia.org/wiki/Serpent_%28cipher%29

Hmm, I can't find any mention of serpent-0 there?

> Still, I'm not sure it is worth the time to fix Serpent in Nettle.  I
> suggest to remove it, that will save some code size as well.

It's clearly no use to keep the current broken implementation. But I
think it would be nice to have serpent, for a couple of reasons:

* It's generally good to have a couple of ciphers with aes-compatible
  key and block sizes. Besides aes/rijndael and serpent, there's
  twofish and camellia.

* I'm not sure what performance one can get out of serpent compared to
  aes, in particular on 64-bit processors. AES doesn't fit well with
  64-bit operations, camellia is better in that respect but includes
  some awkward operations (current 64-bit code could perhaps be improved
  abit using larger tables). 

* I'd prefer to not remove existing features.

If you have a half-done port of libgrypt's serpent code, maybe you or I
could finish it? 

I'll start by looking into the test vectors, I'd like to figure out
where nettle's came from, and I'd like to have a serpent-test.c
including correct test vectors, even if we end up removing all other
serpent code.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Fri Feb 11 11:39:04 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Fri, 11 Feb 2011 11:39:04 +0100
Subject: Serpent
In-Reply-To: <nnwrl6lwah.fsf@stalhein.lysator.liu.se>
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
Message-ID: <AANLkTinixUSwmyzHkGyEwoXhoCn950mLR1dhJn25PLB-@mail.gmail.com>

On Fri, Feb 11, 2011 at 11:07 AM, Niels M?ller <nisse at lysator.liu.se> wrote:

> I'll start by looking into the test vectors, I'd like to figure out
> where nettle's came from, and I'd like to have a serpent-test.c
> including correct test vectors, even if we end up removing all other
> serpent code.

I'am just repeating myself and might become annoying... but does it
really worth the time? Is there anyone using that algorithm, or anyone
planning to use it? As it stands now serpent is an academic cipher,
it is not used in any protocol.

regards,
Nikos

From nisse at lysator.liu.se  Fri Feb 11 11:43:34 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 11 Feb 2011 11:43:34 +0100
Subject: Serpent
In-Reply-To: <nnwrl6lwah.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Fri, 11 Feb 2011 11:07:02 +0100")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
Message-ID: <nnsjvulull.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> I'm puzzled, because I'm fairly sure I got the test vectors from
> serpent's submission package (I could try to double check that),

I have checked that now. I downloaded and unpacked the submission
package http://www.cl.cam.ac.uk/~rja14/Papers/serpent.tar.gz

Test vectors are in the floppy4 directory. In floppy4/ecb_vk.txt, the
first test vector is

  KEYSIZE=128
  
  PT=00000000000000000000000000000000
  
  I=1
  KEY=80000000000000000000000000000000
  CT=49afbfad9d5a34052cd8ffa5986bd2dd

and in floppy4/ecb_vt.txt the first text vector is

  KEYSIZE=128
  
  KEY=00000000000000000000000000000000
  
  I=1
  PT=80000000000000000000000000000000
  CT=10b5ffb720b8cb9002a1142b0ba2e94a

These are the first two testvectors in nettle's serpent-test.c.

On the other hand, the file

http://www.cs.technion.ac.il/~biham/Reports/Serpent/Serpent-128-128.verified.test-vectors

contains the testvector

Set 2, vector#  0:
                           key=00000000000000000000000000000000
                         plain=80000000000000000000000000000000
                        cipher=A3B35DE7C358DDD82644678C64B8BCBB
                     decrypted=80000000000000000000000000000000
            Iterated 100 times=1DDF9883B4663045753758E0B9B2C09B
           Iterated 1000 times=BE5AE44A1CF1BB86DD7A3B61CEEA01EC

Same inputs as in ecb_vt.txt, but different output.

What results do you get with libgcrypt for the above two test vectors?

I think I have to mail the authors...

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Fri Feb 11 12:11:41 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 11 Feb 2011 12:11:41 +0100
Subject: Serpent
In-Reply-To: <AANLkTinixUSwmyzHkGyEwoXhoCn950mLR1dhJn25PLB-@mail.gmail.com>
	(Nikos Mavrogiannopoulos's message of "Fri, 11 Feb 2011 11:39:04
	+0100")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
	<AANLkTinixUSwmyzHkGyEwoXhoCn950mLR1dhJn25PLB-@mail.gmail.com>
Message-ID: <nnk4h6ltaq.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <n.mavrogiannopoulos at gmail.com> writes:

> As it stands now serpent is an academic cipher,
> it is not used in any protocol.

It's specified as an optional algorithm in the ssh protocol (see RFC
4253 and RFC 4344).

That said, I think the main use for serpent or any other of the losing
aes finalists are as backups in case some serious problem is discovered
with aes.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Fri Feb 11 12:28:33 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Fri, 11 Feb 2011 12:28:33 +0100
Subject: Serpent
In-Reply-To: <nnk4h6ltaq.fsf@stalhein.lysator.liu.se>
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
	<AANLkTinixUSwmyzHkGyEwoXhoCn950mLR1dhJn25PLB-@mail.gmail.com>
	<nnk4h6ltaq.fsf@stalhein.lysator.liu.se>
Message-ID: <AANLkTimC=9wxFwWGnRkAiFs9fMQMCPTYVOn9Z+ODm=bP@mail.gmail.com>

On Fri, Feb 11, 2011 at 12:11 PM, Niels M?ller <nisse at lysator.liu.se> wrote:
>> As it stands now serpent is an academic cipher,
>> it is not used in any protocol.
> It's specified as an optional algorithm in the ssh protocol (see RFC
> 4253 and RFC 4344).

Ah, I didn't know that.

> That said, I think the main use for serpent or any other of the losing
> aes finalists are as backups in case some serious problem is discovered
> with aes.

Nowdays this is also camellia for that purpose...

regards,
Nikos

From dkg at fifthhorseman.net  Fri Feb 11 16:41:25 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Fri, 11 Feb 2011 10:41:25 -0500
Subject: Serpent
In-Reply-To: <AANLkTinixUSwmyzHkGyEwoXhoCn950mLR1dhJn25PLB-@mail.gmail.com>
References: <87lj4jvj9o.fsf@latte.josefsson.org>	<nntyj7rp85.fsf@stalhein.lysator.liu.se>	<nnsjynogry.fsf@stalhein.lysator.liu.se>	<nnoc9boglx.fsf@stalhein.lysator.liu.se>	<87ei9tu5fu.fsf@latte.josefsson.org>	<nnvd35iubw.fsf@stalhein.lysator.liu.se>	<871v5lzx67.fsf@latte.josefsson.org>	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>	<87lj1nhf3j.fsf@latte.josefsson.org>	<8739nveg2k.fsf@latte.josefsson.org>	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>	<87oc6j6o55.fsf_-_@latte.josefsson.org>	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
	<AANLkTinixUSwmyzHkGyEwoXhoCn950mLR1dhJn25PLB-@mail.gmail.com>
Message-ID: <4D5558A5.1060805@fifthhorseman.net>

On 02/11/2011 05:39 AM, Nikos Mavrogiannopoulos wrote:
> I'am just repeating myself and might become annoying... but does it
> really worth the time? Is there anyone using that algorithm, or anyone
> planning to use it? As it stands now serpent is an academic cipher,
> it is not used in any protocol.

I know people who use serpent with Linux's dm-crypt for encrypted block
devices.  It's not entirely academic.  having multiple implementations
of serpent would be a Good Thing.

	--dkg


From simon at josefsson.org  Fri Feb 11 19:51:47 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Fri, 11 Feb 2011 19:51:47 +0100
Subject: Serpent
In-Reply-To: <nnwrl6lwah.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Fri, 11 Feb 2011 11:07:02 +0100")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
Message-ID: <87sjvuz9oc.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

>> According to Eli's page and the AES submission paper, there is Serpent-0
>> and Serpent-1 and the paper discuss (page 22-23) that the key schedule
>> has changed.
>
> Do you think nettle's implementation is serpent-0, or is it just
> broken?

I don't know, I suspect it is serpent-0 but haven't confirmed it.

>> I updated the wikipedia article on this:
>> http://en.wikipedia.org/wiki/Serpent_%28cipher%29
>
> Hmm, I can't find any mention of serpent-0 there?

I see it, maybe there is some caching?

> If you have a half-done port of libgrypt's serpent code, maybe you or I
> could finish it? 

Sure!  I suspect everything except the key schedule is working in my
port.  Should I try to make it compatible with libgcrypt, and presumably
Serpent-1?  Thus modifying serpent-test.c?  I'll send a patch later.

/Simon

From nisse at lysator.liu.se  Fri Feb 11 20:08:23 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 11 Feb 2011 20:08:23 +0100
Subject: Serpent
In-Reply-To: <87sjvuz9oc.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Fri, 11 Feb 2011 19:51:47 +0100")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
	<87sjvuz9oc.fsf@latte.josefsson.org>
Message-ID: <nnfwrul788.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> Should I try to make it compatible with libgcrypt, and presumably
> Serpent-1?

It ought to be serpent-1, the algorithm proposed for AES. RFC 4344
references the algorithm like this:

   [SERPENT]   Anderson, R., Biham, E., and Knudsen, L., "Serpent: A
               proposal for the Advanced Encryption Standard", NIST
               AES Proposal, 1998.
                                 
> Thus modifying serpent-test.c?  I'll send a patch later.

I've already checked in a few new test cases in serpent-test.c (taken
from http://www.cs.technion.ac.il/~biham/Reports/Serpent/), but #if:ed
out for now.

I'd also like to hear what response we get from the serpent people
regarding conflicting test vectors, it seems so strange if the test
vectors in the aes submission were broken.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Fri Feb 11 00:19:25 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 11 Feb 2011 00:19:25 +0100
Subject: LGPL blowfish
In-Reply-To: <8739nveg2k.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Thu, 10 Feb 2011 22:26:59 +0100")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
Message-ID: <nnaai3mq9u.fsf@stalhein.lysator.liu.se>

[ Resend, the posting originally used an incorrect list address. /nisse ]

Simon Josefsson <simon at josefsson.org> writes:

> I wonder which of nettle or libgcrypt is correct -- and further, I
> really wonder if anyone cares at all about Serpent if a problem like
> this haven't been noticed before?

Interesting...

As far as I recall the nettle history (and also looking at the
comments), the nettle adaptation of serpent.c was done by Rafael
Sevilla, if I have touched that code I think it's trivial changes only.

And I think that I wrote serpent-test.c, based on the test vectors in the
serpent AES-competition package.

I don't remember if I have ever done any interoperability testing of
serpent with lsh (which uses nettle's implementation). Trying to connect
to some openssh servers, it seems they don't enable serpent by default.

Maybe you can try adding some of the testvectors at
http://www.cs.technion.ac.il/~biham/Reports/Serpent/ to nettle and
libgcrypt, and see what happens? (On the nettle side, I'll try to give
that a reasonably high priority).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Tue Feb 22 23:07:14 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Tue, 22 Feb 2011 17:07:14 -0500
Subject: nettle documentation tweaks?
Message-ID: <4D643392.9090100@fifthhorseman.net>

hi nettle folks--

i've been reading the latest nettle docs [0], and i wanted to say that
they are excellent.  Clear explanations, and useful contextual help
about how various crypto primitives can be used in real-world settings
as well.  Thanks for this!

I have two nit-picks that i'd like to share in hopes of making excellent
documentation even better.

 0) avoid replay attacks in pubkey authorization example.

Section 6.5 contains the following paragraph:

> Besides signing meaningful messages, digital signatures can be used for
> authorization. A server can be configured with a public key, such that
> any client that connects to the service is given a random nonce
> message. If the server gets a reply with a correct signature matching
> the nonce message and the configured public key, the client is granted
> access. So the configuration of the server can be understood as ?grant
> access to whoever knows the private key corresponding to this
> particular public key, and to no others?. 


This simple scheme is actually vulnerable to a replay attack: an
man-in-the-middle attacker can pose as the service to the user while
simultaneously connecting to the service directly.  The attacker gets a
nonce from the server, passes it to the user for signing, returns the
user's response to the service, and gains access.

One way to avoid this is to ensure that the "random nonce" to be signed
by the user is actually a negotiated value that neither party can fully
control.

I recognize that this documentation shouldn't go into deep detail here,
but i've seen real-world attempts to implement cryptosystems that are
subject to this kind of replay attack :(. i think it would be a good
idea to warn against it somehow.


 1) "bad things may happen", "a few subtle issues", etc. should have
outbound links

There are several places in the docs that mention potential security
concerns implementators should be aware of.  For example, the remark
about CBC providing "information leakage" could link to
http://www.kb.cert.org/vuls/id/958563

Again, i'm not pushing for major changes or lots of intrusive commentary
here (and i don't know how difficult it is to make outbound links in the
documentation system used by nettle) but it would make already good docs
even better.


Thanks again for this library!

Regards,

	--dkg

[0] http://www.lysator.liu.se/~nisse/nettle/nettle.html#Cipher-modes


From nisse at lysator.liu.se  Mon Feb 28 17:19:16 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 28 Feb 2011 17:19:16 +0100
Subject: nettle documentation tweaks?
In-Reply-To: <4D643392.9090100@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Tue, 22 Feb 2011 17:07:14 -0500")
References: <4D643392.9090100@fifthhorseman.net>
Message-ID: <nnk4gknncr.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> i've been reading the latest nettle docs [0], and i wanted to say that
> they are excellent.  Clear explanations, and useful contextual help
> about how various crypto primitives can be used in real-world settings
> as well. 

Thanks!

> Section 6.5 contains the following paragraph:
>
>> Besides signing meaningful messages, digital signatures can be used for
>> authorization. [...]

> This simple scheme is actually vulnerable to a replay attack: an
> man-in-the-middle attacker can pose as the service to the user while
> simultaneously connecting to the service directly.  The attacker gets a
> nonce from the server, passes it to the user for signing, returns the
> user's response to the service, and gains access.

> One way to avoid this is to ensure that the "random nonce" to be signed
> by the user is actually a negotiated value that neither party can fully
> control.

Right. A common and reasonable way to handle it is by negotiating a
session key using DH, use that key for message authentication of the
session (and why not also encryption, while we're at it). Then either
side can let the other authenticate itself by signing a nonce which is
not an independent random value, but computed in some way from the
session key.

I have to think a bit more on how to explain this in a concise way, any
suggestions? I think it may be appropriate to not recommend any protocol
which doesn't include some kind of authentication of the session. And
from this point of view, the session-key thing can be seen as an
optimization, where the alternative is to sign each message.

>  1) "bad things may happen", "a few subtle issues", etc. should have
> outbound links
>
> There are several places in the docs that mention potential security
> concerns implementators should be aware of.  For example, the remark
> about CBC providing "information leakage" could link to
> http://www.kb.cert.org/vuls/id/958563

I've added this link. Patches for other missing references are
appreciated...

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Thu Mar 17 06:23:54 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Thu, 17 Mar 2011 01:23:54 -0400
Subject: nettle perl bindings
Message-ID: <4D819AEA.4060609@fifthhorseman.net>

Hi Nettle Folks--

I'm building Perl bindings for libnettle.  I hope to claim the
Crypt::Nettle namespace.

The project is in its infancy, but i currently have coverage for all
hash functions and ciphers.

My next steps are adding bindings for Yarrow and RSA.

At the moment, my source code is available via git:

 git clone git://lair.fifthhorseman.net/~dkg/libcrypt-nettle-perl


You can test it with:

 cd libcrypt-nettle-perl
 perl Makefile.PL
 make
 make test


You can read the docs with:

 pod2text lib/Crypt/Nettle.pm
 pod2text lib/Crypt/Nettle/Hash.pm
 pod2text lib/Crypt/Nettle/Cipher.pm

I have not yet uploaded it to CPAN.  I'd be very happy to get feedback
from anyone interested.

Regards,

	--dkg


From nisse at lysator.liu.se  Thu Mar 17 08:56:12 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 17 Mar 2011 08:56:12 +0100
Subject: nettle perl bindings
In-Reply-To: <4D819AEA.4060609@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Thu, 17 Mar 2011 01:23:54 -0400")
References: <4D819AEA.4060609@fifthhorseman.net>
Message-ID: <nn4o72goyr.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> I'm building Perl bindings for libnettle.  I hope to claim the
> Crypt::Nettle namespace.

Nice!

I'm not very familiar with perl, but I have had a quick look at the
documentation.

> You can read the docs with:
>
>  pod2text lib/Crypt/Nettle.pm

A typo:

:     In the future, it should support asymmetric encrpytion and pseudo-random
                                                    ^^
:     number generation.

: COPYRIGHT AND LICENSE
:     Copyright (c) Daniel Kahn Gillmor Crypt::Nettle is free software, you
:     may redistribute it and/or modify it under the same terms as Perl
:     itself.

The GPL/LGPL license of the nettle library itself may apply to perl
programs using these bindings. I don't know if it's customary to
document this in a bit more detail?

>  pod2text lib/Crypt/Nettle/Hash.pm

:  hmac_data($algo, $data)

How do you provide the key? 

I'm not sure it's the right design to mix hash functions and macs (and
how will you deal with macs that are not based on the hmac construction)?

>  pod2text lib/Crypt/Nettle/Cipher.pm

Typo:

: ABSTRACT
:     Crypt::Nettle::Cipher provides an object interface to symmetric
:     encrpytion and decryption from the nettle C library. Each
        ^^

:  new($is_encrypt, $algo, $key, $mode, $iv)

You include arctwo algorithms twice in the algorithm list. Maybe you
should exclude serpent until the recently discovered interoperability
problems are sorted out?

How do you deal with algorithms with a large number of possible key
sizes? Maybe it would be better to view, e.g., aes and arcfour as just
two algorithm, and let the size of the given key imply the keysize?

The $is_encrypt flag to new seems a bit awkward. Maybe it would be
easier with

  my $ctx = new ($algo, $mode) /* Possibly with $mode defaulting to
                                  ecb?, and not allowed at all for stream ciphers. */

  $ctx->set_encrypt_key($key, $iv) /* $iv optional and required when applicable */
  $ctx->set_decrypt_key($key, $iv)

:  process($data)

I think the requirement that the length is a multiple of the block size
needs to be relaxed a bit. For CTR mode, one should allow a partial
block for the last call. And *maybe* for all calls (with an internal block
buffer to let CTR work like a stream cipher), even if that's not how
nettle's ctr mode support works.

Maybe you should think about how to add gcm support. Which is a bit more
complicated, with both per-key state and per-message state, and
additional inputs and outputs.

How do you query if a cipher is a block or a stream cipher?
block_size() returning 0?

Happy hacking,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Thu Mar 17 09:45:00 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Thu, 17 Mar 2011 04:45:00 -0400
Subject: nettle perl bindings
In-Reply-To: <nn4o72goyr.fsf@stalhein.lysator.liu.se>
References: <4D819AEA.4060609@fifthhorseman.net>
	<nn4o72goyr.fsf@stalhein.lysator.liu.se>
Message-ID: <4D81CA0C.5050406@fifthhorseman.net>

Thanks for the quick review Niels!


On 03/17/2011 03:56 AM, Niels M?ller wrote:
> A typo:


The typos and stupidly-broken hmac_data() convenience function have been
fixed and pushed.  I also added a unit test for hmac_data() within
t/03-hmac.t which would have caught the mistake in the first place.


> : COPYRIGHT AND LICENSE
> :     Copyright (c) Daniel Kahn Gillmor Crypt::Nettle is free software, you
> :     may redistribute it and/or modify it under the same terms as Perl
> :     itself.
> 
> The GPL/LGPL license of the nettle library itself may apply to perl
> programs using these bindings. I don't know if it's customary to
> document this in a bit more detail?

I welcome suggestions for improved text.  I agree that the intersections
of the various licenses can be a bit confusing.

> You include arctwo algorithms twice in the algorithm list. 

That's what i get for copying/pasting from the docs :P

Section 6.2.11 of http://www.lysator.liu.se/~nisse/nettle/nettle.html
actually lists them twice.

> Maybe you
> should exclude serpent until the recently discovered interoperability
> problems are sorted out?

I'd prefer to expose the functions exposed by the underlying library if
possible.  If there are incompatibilities, we should be catching them in
the test suite (though my test suite for ciphers only covers aes and
cast and camellia at the moment).

> How do you deal with algorithms with a large number of possible key
> sizes? Maybe it would be better to view, e.g., aes and arcfour as just
> two algorithm, and let the size of the given key imply the keysize?

hm, this is an interesting idea.  I'll have to think about how to
implement that, but i think it's doable.

> The $is_encrypt flag to new seems a bit awkward. Maybe it would be
> easier with
> 
>   my $ctx = new ($algo, $mode) /* Possibly with $mode defaulting to
>                                   ecb?, and not allowed at all for stream ciphers. */
> 
>   $ctx->set_encrypt_key($key, $iv) /* $iv optional and required when applicable */
>   $ctx->set_decrypt_key($key, $iv)
> 
> :  process($data)

My problem with this is that i then have to handle the case where the
user invokes process() without having remembered to set a key.  I agree
that $is_encrypt seems a little bit clumsy (and it's irrelevant for many
of the ciphers), but i think the fact that it's readable mitigates
things somewhat.

> I think the requirement that the length is a multiple of the block size
> needs to be relaxed a bit. For CTR mode, one should allow a partial
> block for the last call. And *maybe* for all calls (with an internal block
> buffer to let CTR work like a stream cipher), even if that's not how
> nettle's ctr mode support works.

I'm actually not enforcing any of these constraints in the perl code --
they'll just crop up if the user passes the wrong data down to the
library underneath.

> Maybe you should think about how to add gcm support. Which is a bit more
> complicated, with both per-key state and per-message state, and
> additional inputs and outputs.

interesting -- is GCM part of nettle itself, or do you think i should
implement it in the perl wrapper?  I didn't see any mention of GCM in
the online docs.

> How do you query if a cipher is a block or a stream cipher?
> block_size() returning 0?

yep, that's it at the moment.

Let me know what other feedback you have,

	--dkg


From simon at josefsson.org  Thu Mar 17 09:45:19 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Thu, 17 Mar 2011 09:45:19 +0100
Subject: nettle perl bindings
In-Reply-To: <4D819AEA.4060609@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Thu, 17 Mar 2011 01:23:54 -0400")
References: <4D819AEA.4060609@fifthhorseman.net>
Message-ID: <87tyf2f84g.fsf@latte.josefsson.org>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> Hi Nettle Folks--
>
> I'm building Perl bindings for libnettle.  I hope to claim the
> Crypt::Nettle namespace.
>
> The project is in its infancy, but i currently have coverage for all
> hash functions and ciphers.
>
> My next steps are adding bindings for Yarrow and RSA.

Don't forget to add RSA blinding, otherwise it may be vulnerable in the
real world.  I wish Nettle supported this natively, RSA is not generally
safe without it.

/Simon

From dkg at fifthhorseman.net  Thu Mar 17 10:01:52 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Thu, 17 Mar 2011 05:01:52 -0400
Subject: nettle perl bindings
In-Reply-To: <87tyf2f84g.fsf@latte.josefsson.org>
References: <4D819AEA.4060609@fifthhorseman.net>
	<87tyf2f84g.fsf@latte.josefsson.org>
Message-ID: <4D81CE00.7000007@fifthhorseman.net>

Hi Simon--

On 03/17/2011 04:45 AM, Simon Josefsson wrote:
> Don't forget to add RSA blinding, otherwise it may be vulnerable in the
> real world.  I wish Nettle supported this natively, RSA is not generally
> safe without it.

Thanks for this suggestion -- i'm not sure that the perl bindings are
the right place to do this, though.  Do other Nettle language bindings
handle RSA blinding?  I'd rather have the perl bindings stay fairly
close to the underlying C library.

My understanding is that RSA blinding is a countermeasure against timing
attacks, and that it introduces a new dependency on some sort of RNG
(though perhaps a weak one?) to parts of the process that wouldn't
otherwise need it.  I'd certainly prefer to have that handled within the
lower-level library if possible, though i wouldn't mind creating and
handing in a yarrow context for each of these operations.

	--dkg


From nisse at lysator.liu.se  Thu Mar 17 10:29:53 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 17 Mar 2011 10:29:53 +0100
Subject: nettle perl bindings
In-Reply-To: <4D81CA0C.5050406@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Thu, 17 Mar 2011 04:45:00 -0400")
References: <4D819AEA.4060609@fifthhorseman.net>
	<nn4o72goyr.fsf@stalhein.lysator.liu.se>
	<4D81CA0C.5050406@fifthhorseman.net>
Message-ID: <nnzkouf626.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> I welcome suggestions for improved text.  I agree that the intersections
> of the various licenses can be a bit confusing.

I guess it will be easier when we have moved to LGPL. Then it's going to
take some effort to write a perl program which use these bindings and
violate the licensing terms. (With the GPL, I'm actually not sure myself
under which circumstances the perl program would have to be GPL:ed).

> Section 6.2.11 of http://www.lysator.liu.se/~nisse/nettle/nettle.html
> actually lists them twice.

Ooops. Fixed now.

> My problem with this is that i then have to handle the case where the
> user invokes process() without having remembered to set a key.

Can't you just raise some error ? You'd need to have some flag to
remember if it's been initialized, but you need that anyway for the
is_encrypt method, right?

> I'm actually not enforcing any of these constraints in the perl code --
> they'll just crop up if the user passes the wrong data down to the
> library underneath.

That seems a bit dangerous. I thought the principle was that it
shouldn't be easy to write perl code which triggers some assertion
failure in some C routine.

> interesting -- is GCM part of nettle itself, or do you think i should
> implement it in the perl wrapper?  I didn't see any mention of GCM in
> the online docs.

It's in the CVS version of Nettle, but not yet in any release. Maybe
most of the discussion was private rather than on this list?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Thu Mar 17 10:35:50 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 17 Mar 2011 10:35:50 +0100
Subject: nettle perl bindings
In-Reply-To: <4D81CE00.7000007@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Thu, 17 Mar 2011 05:01:52 -0400")
References: <4D819AEA.4060609@fifthhorseman.net>
	<87tyf2f84g.fsf@latte.josefsson.org>
	<4D81CE00.7000007@fifthhorseman.net>
Message-ID: <nnvczif5s9.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> My understanding is that RSA blinding is a countermeasure against timing
> attacks, and that it introduces a new dependency on some sort of RNG
> (though perhaps a weak one?) to parts of the process that wouldn't
> otherwise need it.

I confess I don't remember the details of why blinding is desirable.
Does it improve hiding of the key, message, or both? 

Would it help to use a powm function which has data-independent timing?
There's a powm_sec in gmp which is supposed to do this (assuming
underlying arithmetic instructions have data independent timing), and
which is only slighly slower than the general version for sizes of
interest. But a few other functions are still missing to make it really
useful.

It would make sense to add an RSA interface which takes a randomness
source as input (for blinding), and a DSA interface which doesn't need a
randomness source (and instead uses something like the hash of the
message beeing signed as the "random" value needed, like it's done
putty).

But neither is currently a high priority for me.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Thu Mar 17 10:50:05 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Thu, 17 Mar 2011 05:50:05 -0400
Subject: nettle perl bindings
In-Reply-To: <nnzkouf626.fsf@stalhein.lysator.liu.se>
References: <4D819AEA.4060609@fifthhorseman.net>	<nn4o72goyr.fsf@stalhein.lysator.liu.se>	<4D81CA0C.5050406@fifthhorseman.net>
	<nnzkouf626.fsf@stalhein.lysator.liu.se>
Message-ID: <4D81D94D.2060600@fifthhorseman.net>

On 03/17/2011 05:29 AM, Niels M?ller wrote:
> Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:
>> My problem with this is that i then have to handle the case where the
>> user invokes process() without having remembered to set a key.
> 
> Can't you just raise some error ? 

Sure, but that introduces a different kind of ugliness -- i'm not sure
that trading off this ugliness for the other one is worthwhile, though
i'm willing to be convinced.

> You'd need to have some flag to
> remember if it's been initialized, but you need that anyway for the
> is_encrypt method, right?

i'm not sure i see the parallel.  is_encrypt() says whether the
Crypt::Nettle::Cipher object was initialized as an encrypting cipher or
a decrypting cipher.

>> I'm actually not enforcing any of these constraints in the perl code --
>> they'll just crop up if the user passes the wrong data down to the
>> library underneath.
> 
> That seems a bit dangerous. I thought the principle was that it
> shouldn't be easy to write perl code which triggers some assertion
> failure in some C routine.

Yes, that would be ideal.  I think the right way to go in the long term
(as noted in the BUGS section of Cipher.pm) is to add internal buffering
for process() calls so the user can pass arbitrarily-sized data to the
object.

The extra fiddly bits with this arrangement are:

 0) process_in_place() is still brittle: you won't be able to call it at
all if the internal buffer isn't on a clean block boundary, and you also
won't be able to call it with arbitrary-sized data.

 1) the data retrieved from process() isn't guaranteed to be the same
size as the data fed in.

 2) i'll need to introduce a finish() call that handles some sort of
padding and emits the final data.

It's a bunch more bookkeeping internally, but it does seem like it would
let the user treat the block ciphers as something approximating a stream
cipher without having to think much about it, which would be nice.

I think getting RSA working is higher priority for me at the moment, but
i'll certainly keep this suggestion on my plate.

> It's in the CVS version of Nettle, but not yet in any release. Maybe
> most of the discussion was private rather than on this list?

When a new version is released, i'll be happy to update the perl
bindings to enable access to the new features :)

Is the revision control for Nettle publicly visible?

	--dkg


From n.mavrogiannopoulos at gmail.com  Thu Mar 17 10:59:46 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Thu, 17 Mar 2011 10:59:46 +0100
Subject: nettle perl bindings
In-Reply-To: <nnvczif5s9.fsf@stalhein.lysator.liu.se>
References: <4D819AEA.4060609@fifthhorseman.net>
	<87tyf2f84g.fsf@latte.josefsson.org>
	<4D81CE00.7000007@fifthhorseman.net>
	<nnvczif5s9.fsf@stalhein.lysator.liu.se>
Message-ID: <AANLkTimT30Vx2GigTOCQAFx-eLHuk=GrB15hCN15=xS8@mail.gmail.com>

On Thu, Mar 17, 2011 at 10:35 AM, Niels M?ller <nisse at lysator.liu.se> wrote:
> Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:
>
>> My understanding is that RSA blinding is a countermeasure against timing
>> attacks, and that it introduces a new dependency on some sort of RNG
>> (though perhaps a weak one?) to parts of the process that wouldn't
>> otherwise need it.
> I confess I don't remember the details of why blinding is desirable.
> Does it improve hiding of the key, message, or both?

Actually RSA is has pretty much limited utility without blinding since
retrieving  the RSA private key from a web server has been shown
practical since 2003 and attacks were known since 1996 (Kocher).
gnutls implements blinding over nettle's functions. You might add a
warning on the documentation of nettle's functions.

The papers discussion the attacks:
* Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and
Other Systems by Kocher (1996)
* Remote timing attacks are practical  by  Boneh and Brumley
* Improving Brumley and Boneh Timing Attack on Unprotected SSL Implementations

regards,
Nikos

From nisse at lysator.liu.se  Thu Mar 17 11:27:28 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 17 Mar 2011 11:27:28 +0100
Subject: nettle perl bindings
In-Reply-To: <4D81D94D.2060600@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Thu, 17 Mar 2011 05:50:05 -0400")
References: <4D819AEA.4060609@fifthhorseman.net>
	<nn4o72goyr.fsf@stalhein.lysator.liu.se>
	<4D81CA0C.5050406@fifthhorseman.net>
	<nnzkouf626.fsf@stalhein.lysator.liu.se>
	<4D81D94D.2060600@fifthhorseman.net>
Message-ID: <nnr5a6f3e7.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

>> You'd need to have some flag to
>> remember if it's been initialized, but you need that anyway for the
>> is_encrypt method, right?
>
> i'm not sure i see the parallel.  is_encrypt() says whether the
> Crypt::Nettle::Cipher object was initialized as an encrypting cipher or
> a decrypting cipher.

I was just thinking of the internal book-keeping, which I imagine would be the
same (but with three possible states rather than two).

> Is the revision control for Nettle publicly visible?

Yes. A little bit awkward, in that you need to get the complete lsh
tree, but follow the instructions at http://www.lysator.liu.se/~nisse/nettle/
and it should work.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Thu Mar 17 22:19:13 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Thu, 17 Mar 2011 22:19:13 +0100
Subject: nettle perl bindings
In-Reply-To: <4D81CE00.7000007@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Thu, 17 Mar 2011 05:01:52 -0400")
References: <4D819AEA.4060609@fifthhorseman.net>
	<87tyf2f84g.fsf@latte.josefsson.org>
	<4D81CE00.7000007@fifthhorseman.net>
Message-ID: <87zkot78dq.fsf@latte.josefsson.org>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> Hi Simon--
>
> On 03/17/2011 04:45 AM, Simon Josefsson wrote:
>> Don't forget to add RSA blinding, otherwise it may be vulnerable in the
>> real world.  I wish Nettle supported this natively, RSA is not generally
>> safe without it.
>
> Thanks for this suggestion -- i'm not sure that the perl bindings are
> the right place to do this, though.  Do other Nettle language bindings
> handle RSA blinding?  I'd rather have the perl bindings stay fairly
> close to the underlying C library.

Yes -- I agree.

Btw, thanks for working on perl bindings, that sounds really useful.

nisse at lysator.liu.se (Niels M?ller) writes:

> It would make sense to add an RSA interface which takes a randomness
> source as input (for blinding), and a DSA interface which doesn't need a
> randomness source (and instead uses something like the hash of the
> message beeing signed as the "random" value needed, like it's done
> putty).

Yes, an interface like that seems like a simple and sufficient solution
to the problem.

/Simon

From dkg at fifthhorseman.net  Fri Mar 18 00:05:43 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Thu, 17 Mar 2011 19:05:43 -0400
Subject: nettle perl bindings
In-Reply-To: <4D81D94D.2060600@fifthhorseman.net>
References: <4D819AEA.4060609@fifthhorseman.net>	<nn4o72goyr.fsf@stalhein.lysator.liu.se>	<4D81CA0C.5050406@fifthhorseman.net>	<nnzkouf626.fsf@stalhein.lysator.liu.se>
	<4D81D94D.2060600@fifthhorseman.net>
Message-ID: <4D8293C7.2070002@fifthhorseman.net>

On 03/17/2011 05:50 AM, Daniel Kahn Gillmor wrote:
> I think getting RSA working is higher priority for me at the moment, 

And now it's done -- I just wrapped up Crypt::Nettle::Yarrow and
Crypt::Nettle::RSA!  I'm about ready to tag this thing as Crypt::Nettle
version 0.1.  I'll try to do a CPAN upload shortly if i can figure that out.

If anyone can run the test suite and report back, i'd appreciate it.

  git clone git://lair.fifthhorseman.net/~dkg/libcrypt-nettle-perl
  cd libcrypt-nettle-perl
  perl Makefile.PL
  make
  make test

Niels, this is clearly a derivative work of Nettle.  I'm happy to put
whatever free license you think is appropriate on it.  Is GPL-2+ OK?

Regards,

	--dkg


From nisse at lysator.liu.se  Fri Mar 18 06:55:26 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 18 Mar 2011 06:55:26 +0100
Subject: nettle perl bindings
In-Reply-To: <4D8293C7.2070002@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Thu, 17 Mar 2011 19:05:43 -0400")
References: <4D819AEA.4060609@fifthhorseman.net>
	<nn4o72goyr.fsf@stalhein.lysator.liu.se>
	<4D81CA0C.5050406@fifthhorseman.net>
	<nnzkouf626.fsf@stalhein.lysator.liu.se>
	<4D81D94D.2060600@fifthhorseman.net>
	<4D8293C7.2070002@fifthhorseman.net>
Message-ID: <nnaagtezw1.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> Niels, this is clearly a derivative work of Nettle.  I'm happy to put
> whatever free license you think is appropriate on it.  Is GPL-2+ OK?

If by "GPL-2+" you mean GPL version 2 or (user's option) any later version,
that's fine with me. 

As you know, changing the Nettle license to (some version of) the LPGL
is planned. When that happens, it would make some sense for the perl
bindings to follow, even if sticking to the GPL is still a perfectly
acceptable option.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Fri Mar 18 14:28:20 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Fri, 18 Mar 2011 09:28:20 -0400
Subject: nettle perl bindings
In-Reply-To: <nnaagtezw1.fsf@stalhein.lysator.liu.se>
References: <4D819AEA.4060609@fifthhorseman.net>	<nn4o72goyr.fsf@stalhein.lysator.liu.se>	<4D81CA0C.5050406@fifthhorseman.net>	<nnzkouf626.fsf@stalhein.lysator.liu.se>	<4D81D94D.2060600@fifthhorseman.net>	<4D8293C7.2070002@fifthhorseman.net>
	<nnaagtezw1.fsf@stalhein.lysator.liu.se>
Message-ID: <4D835DF4.4030701@fifthhorseman.net>

On 03/18/2011 01:55 AM, Niels M?ller wrote:
> Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:
> 
>> Niels, this is clearly a derivative work of Nettle.  I'm happy to put
>> whatever free license you think is appropriate on it.  Is GPL-2+ OK?
> 
> If by "GPL-2+" you mean GPL version 2 or (user's option) any later version,
> that's fine with me. 

yes, this is what i mean.

> As you know, changing the Nettle license to (some version of) the LPGL
> is planned. When that happens, it would make some sense for the perl
> bindings to follow, even if sticking to the GPL is still a perfectly
> acceptable option.

Sure, I'd be happy to follow the licensing changes.

	--dkg


From dkg at fifthhorseman.net  Mon Mar 21 07:24:22 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 21 Mar 2011 02:24:22 -0400
Subject: typo in manual for arctwo
Message-ID: <4D86EF16.70702@fifthhorseman.net>

Hi Nettle folks--

There's a typo in the Nettle manual about the ARCTWO block size.  the
attached patch (against 2.1) should fix it.

Of course, it might already be fixed on the development trunk.  Is there
a public revision control system for nettle someplace i can look at?

	--dkg

PS i sent this message earlier PGP/MIME-signed, but it was rejected by
the mailing list with "After content filtering, the message was empty"
-- could the content filtering be relaxed a little bit to allow
multipart/signed messages?

The message MIME structure looked like this:

???multipart/signed
 ???multipart/mixed
 ???text/plain
 ???text/x-diff attachment
 ??application/pgp-signature attachment

I then tried re-sending it without the cryptographic signature, which
produced a message like this:

???multipart/mixed
 ??text/plain
 ??text/x-diff attachment

This second message was rejected with the message:

"The message's content type was not explicitly allowed"


Both of these seem like pretty reasonable message structures for posts
to a cryptographic software development mailing list; it'd be nice if
they could go through :)

Here is the patch inline:


arctwo-doc-fix.patch

--- nettle.texinfo.orig	2011-03-16 19:08:50.000000000 -0400
+++ nettle.texinfo	2011-03-16 19:09:26.000000000 -0400
@@ -871,7 +871,7 @@
 @end deftp

 @defvr Constant ARCTWO_BLOCK_SIZE
-The AES block-size, 8
+The ARCTWO block-size, 8
 @end defvr

 @defvr Constant ARCTWO_MIN_KEY_SIZE

From dkg at fifthhorseman.net  Mon Mar 21 07:28:45 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 21 Mar 2011 02:28:45 -0400
Subject: typo in manual for arctwo
In-Reply-To: <4D86EF16.70702@fifthhorseman.net>
References: <4D86EF16.70702@fifthhorseman.net>
Message-ID: <4D86F01D.9020605@fifthhorseman.net>

On 03/21/2011 02:24 AM, Daniel Kahn Gillmor wrote:
> Of course, it might already be fixed on the development trunk.  Is there
> a public revision control system for nettle someplace i can look at?

Gah, this question was already answered by niels several days ago,
though it was after i sent this patch originally, and this message was
caught up in fighting with the mailing list configuration.

It looks like this fix has not yet been made on the development trunk,
if my reading of the CVS repo is accurate.

	--dkg


From nisse at lysator.liu.se  Mon Mar 21 14:41:09 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 21 Mar 2011 14:41:09 +0100
Subject: List configuration (Was: Re: typo in manual for arctwo)
In-Reply-To: <4D86EF16.70702@fifthhorseman.net> (Daniel Kahn Gillmor's message
	of "Mon, 21 Mar 2011 02:24:22 -0400")
References: <4D86EF16.70702@fifthhorseman.net>
Message-ID: <nnpqpkbngq.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> PS i sent this message earlier PGP/MIME-signed, but it was rejected by
> the mailing list with "After content filtering, the message was empty"
> -- could the content filtering be relaxed a little bit to allow
> multipart/signed messages?

I agree this rejection was silly. I'm not so good at mailman
configuration. The "pass_mime_types" option was set to

  text/plain
  multipart/signed

I'm now changing it to

  text
  multipart
  application/pgp-signature

Do you think that is good enough? Other suggestions?

Or is it better to completely disable this content-type filtering?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Mon Mar 21 14:43:36 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 21 Mar 2011 14:43:36 +0100
Subject: typo in manual for arctwo
In-Reply-To: <4D86EF16.70702@fifthhorseman.net> (Daniel Kahn Gillmor's message
	of "Mon, 21 Mar 2011 02:24:22 -0400")
References: <4D86EF16.70702@fifthhorseman.net>
Message-ID: <nnlj08bncn.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> arctwo-doc-fix.patch
>
> --- nettle.texinfo.orig	2011-03-16 19:08:50.000000000 -0400
> +++ nettle.texinfo	2011-03-16 19:09:26.000000000 -0400
> @@ -871,7 +871,7 @@
>  @end deftp
>
>  @defvr Constant ARCTWO_BLOCK_SIZE
> -The AES block-size, 8
> +The ARCTWO block-size, 8
>  @end defvr
>
>  @defvr Constant ARCTWO_MIN_KEY_SIZE

Checked in now. Thanks for spotting it.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Mon Mar 21 16:35:03 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 21 Mar 2011 11:35:03 -0400
Subject: making algorithms in nettle dynamically enumerable
Message-ID: <4D877027.6040207@fifthhorseman.net>

Hi Nettle folks--

In the course of writing Crypt::Nettle, it occurs to me that i'm going
to need to update the list of available ciphers and digests for each new
version of nettle that comes out.  This also means that newer versions
of the perl bindings won't work by default against older versions of
libnettle, which seems like it might make the perl bindings less useful
for people who are stuck (for whatever reason) with an older version of
libnettle.

The attached patch would present an interface to dynamically enumerate
the available ciphers and digest algorithms (and armor methods)
supported by the current version of libnettle.  bindings built against
this meta-interface wouldn't need to be closely tied to any particular
version of nettle, and so could be both forward- and backward-compatible.

I'd be happy with any other interface that would give me the same
effect, of course, and i'm not convinced that this is the best way to do
it -- it just seemed the simplest in the current implementation.

It would also be nice to make a similar interface to genericize cipher
modes (ECB, CBC, CTR, and GCM), if that's possible; i'm not sure what
the best approach is for doing that.

Feedback welcome, of course.

	--dkg


-------------- next part --------------
A non-text attachment was scrubbed...
Name: meta-lists.patch
Type: text/x-diff
Size: 3801 bytes
Desc: not available
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110321/15388116/attachment.patch 

From dkg at fifthhorseman.net  Mon Mar 21 17:12:47 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 21 Mar 2011 12:12:47 -0400
Subject: List configuration (Was: Re: typo in manual for arctwo)
In-Reply-To: <nnpqpkbngq.fsf@stalhein.lysator.liu.se>
References: <4D86EF16.70702@fifthhorseman.net>
	<nnpqpkbngq.fsf@stalhein.lysator.liu.se>
Message-ID: <4D8778FF.4060708@fifthhorseman.net>

On 03/21/2011 09:41 AM, Niels M?ller wrote:
> I agree this rejection was silly. I'm not so good at mailman
> configuration. The "pass_mime_types" option was set to
> 
>   text/plain
>   multipart/signed
> 
> I'm now changing it to
> 
>   text
>   multipart
>   application/pgp-signature
> 
> Do you think that is good enough? Other suggestions?

I'm no mailman guru either, but these settings changes seem to have let
my recent message "making algorithms in nettle dynamically enumerable"
go through fine, while signed.  So thanks!

> Or is it better to completely disable this content-type filtering?

I've never used content-type filtering -- i'd lean toward turning it off
unless there is a well-understood problem that it fixes (e.g. if a
spammer is targetting the list with a bunch of garbage).  But i'm happy
with it as it currently stands :)

Thanks for the fix,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110321/7b0b7d15/attachment.pgp 

From simon at josefsson.org  Mon Mar 21 19:23:46 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Mon, 21 Mar 2011 19:23:46 +0100
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <4D877027.6040207@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Mon, 21 Mar 2011 11:35:03 -0400")
References: <4D877027.6040207@fifthhorseman.net>
Message-ID: <878vw8tjrh.fsf@latte.josefsson.org>

One comment is that it is more difficult to export variables portably
than functions (especially Windows), so sometimes it is preferred to
export a function that returns a static array than exporting the static
array itself.  However, I think nettle already relies on exporting
variables so then this point is moot.  I support the patch otherwise.

/Simon

From nisse at lysator.liu.se  Mon Mar 21 20:04:58 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 21 Mar 2011 20:04:58 +0100
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <4D877027.6040207@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Mon, 21 Mar 2011 11:35:03 -0400")
References: <4D877027.6040207@fifthhorseman.net>
Message-ID: <nnhbawb8h1.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> This also means that newer versions of the perl bindings won't work by
> default against older versions of libnettle,

I don't know how the build system for the perl bindings is setup, but
using something like autoconf should make it easier if you really want
to support multiple nettle versions, with or without public key support.

> The attached patch would present an interface to dynamically enumerate
> the available ciphers and digest algorithms (and armor methods)
> supported by the current version of libnettle. 

The need for this hasn't occured to me, but I see the point. What do
others think?

> bindings built against
> this meta-interface wouldn't need to be closely tied to any particular
> version of nettle, and so could be both forward- and backward-compatible.

But some other parts of the nettle interfaces have not been as stable
between releases, so it doesn't solve the problem completely.

> I'd be happy with any other interface that would give me the same
> effect, of course, and i'm not convinced that this is the best way to do
> it -- it just seemed the simplest in the current implementation.

Right, it's about as simple as one can get, and I like that. I think one
level of indirection like you do is desirable, for linking reasons. 

One may want to think about fancy hardware accelerators (currently not
supported at all), which would require some runtime check on what's
available.

> It would also be nice to make a similar interface to genericize cipher
> modes (ECB, CBC, CTR, and GCM), if that's possible; i'm not sure what
> the best approach is for doing that.

I'm not sure this is as useful. The modes have different properties and
generally don't work well as substitutes for each other. And at least,
the list is expected to be expanded less frequently than the list of
hashes or block ciphers.

Some comments on the implementation:

> +const struct nettle_armor *nettle_armors[] = {
> +  &nettle_base64,
> +  &nettle_base16,
> +  NULL
> +};

The typing should be something like

  const struct nettle_armor * const nettle_armors[]

to say that both the list itself and the pointed to structures are
const.

Also, I think there should be one compilation unit and object file per
list, so that it's, for example, possible to statically link with the
list of all hash algorithms without also dragging in all ciphers.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Mon Mar 21 20:42:22 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 21 Mar 2011 15:42:22 -0400
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <nnhbawb8h1.fsf@stalhein.lysator.liu.se>
References: <4D877027.6040207@fifthhorseman.net>
	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>
Message-ID: <4D87AA1E.7080001@fifthhorseman.net>

On 03/21/2011 03:04 PM, Niels M?ller wrote:
> I don't know how the build system for the perl bindings is setup, but
> using something like autoconf should make it easier if you really want
> to support multiple nettle versions, with or without public key support.

that's an interesting suggestion -- i'm afraid i don't know enough about
integrating autoconf with perlxs to know how to go down that path
without a more research, but i'll consider it for the future.  It would
only allow backward-compatibility, though -- not forward-compatibility.
 That is: if you built Crypt::Nettle against libnettle version X, and
then upgraded to libnettle version Y, you'd have to also re-build
Crypt::Nettle.

That's not a horrible tradeoff, i guess.

> But some other parts of the nettle interfaces have not been as stable
> between releases, so it doesn't solve the problem completely.

yes, that's true -- i'm proposing this just for the ciphers and hashes
at the moment.

> One may want to think about fancy hardware accelerators (currently not
> supported at all), which would require some runtime check on what's
> available.

that's an interesting part i hadn't considered.  I'm not certain how to
represent such a change, but i like the idea that one could discover
available/supported hardware dynamically somehow.  Would supporting that
use case suggest a different API?

> I'm not sure this is as useful. The modes have different properties and
> generally don't work well as substitutes for each other. And at least,
> the list is expected to be expanded less frequently than the list of
> hashes or block ciphers.

OK.  Maybe this is a good target for the autoconf approach you mentioned
above.

> Some comments on the implementation:
> 
>> +const struct nettle_armor *nettle_armors[] = {
>> +  &nettle_base64,
>> +  &nettle_base16,
>> +  NULL
>> +};
> 
> The typing should be something like
> 
>   const struct nettle_armor * const nettle_armors[]
> 
> to say that both the list itself and the pointed to structures are
> const.

Yes, you are right, thanks!

> Also, I think there should be one compilation unit and object file per
> list, so that it's, for example, possible to statically link with the
> list of all hash algorithms without also dragging in all ciphers.

Is there any reason to use this approach for statically-linked tools?
Maybe for UI or hardware discoverability, i guess.  It certainly
wouldn't help in terms of software discoverability or forward/backward
compatibility in a statically-linked library.

Do you want me to submit a modified patch with these changes?

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110321/f3c3ec58/attachment.pgp 

From nisse at lysator.liu.se  Mon Mar 21 21:07:41 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 21 Mar 2011 21:07:41 +0100
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <4D87AA1E.7080001@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Mon, 21 Mar 2011 15:42:22 -0400")
References: <4D877027.6040207@fifthhorseman.net>
	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>
	<4D87AA1E.7080001@fifthhorseman.net>
Message-ID: <nn8vw8b5ki.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> On 03/21/2011 03:04 PM, Niels M?ller wrote:

>> One may want to think about fancy hardware accelerators (currently not
>> supported at all), which would require some runtime check on what's
>> available.
>
> that's an interesting part i hadn't considered.  I'm not certain how to
> represent such a change, but i like the idea that one could discover
> available/supported hardware dynamically somehow.  Would supporting that
> use case suggest a different API?

To add hardware acceleration, one would need to either detect hardware
and build a list automatically at loadtime, or have a function to build
and return the list. It can no longer be a compile-time constant.

But I think the API you suggested is good enough for now.

It's hard to guess what really interface is needed before the hardware
support exists, and I'd rather not define something overly complicated
just because it might possibly be needed later.

>> Also, I think there should be one compilation unit and object file per
>> list, so that it's, for example, possible to statically link with the
>> list of all hash algorithms without also dragging in all ciphers.

> Is there any reason to use this approach for statically-linked tools?

The general principle is that independent functionality should be in
separate object files. I don't think this case is an exception. To be a
little more concrete, say you want a tool that can compute a hash sum
using any algorithm supported by nettle, and you then want that tool on
some constrained system with statically linked binaries, or included in
a statically linked busybox executable.

> Do you want me to submit a modified patch with these changes?

That would be nice. Documentation would also be appreciated (it should
probably mention why des is not on the list, but I don't think it needs
to be terribly many words). Test cases are perhaps not essential for
this, but maybe you can think of some relevant test?

It may also be good to review the naming. Until now, the name attribute
in nettle_cipher and similar structs has been intended to be displayed
for informational purposes only. With your interface, users will be
invited to use it as an identifier to look up an algorithm. How do you
intend to use that name in the perl bindings?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Mon Mar 21 21:32:56 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 21 Mar 2011 16:32:56 -0400
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <nn8vw8b5ki.fsf@stalhein.lysator.liu.se>
References: <4D877027.6040207@fifthhorseman.net>	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>	<4D87AA1E.7080001@fifthhorseman.net>
	<nn8vw8b5ki.fsf@stalhein.lysator.liu.se>
Message-ID: <4D87B5F8.8040908@fifthhorseman.net>

On 03/21/2011 04:07 PM, Niels M?ller wrote:
> But I think the API you suggested is good enough for now.

ok.

> The general principle is that independent functionality should be in
> separate object files. I don't think this case is an exception. To be a
> little more concrete, say you want a tool that can compute a hash sum
> using any algorithm supported by nettle, and you then want that tool on
> some constrained system with statically linked binaries, or included in
> a statically linked busybox executable.

Gotcha, this is a useful way to think about things.  Thanks for your
explanation.

> That would be nice. Documentation would also be appreciated (it should
> probably mention why des is not on the list, but I don't think it needs
> to be terribly many words). Test cases are perhaps not essential for
> this, but maybe you can think of some relevant test?

i'll work on a revised patch with docs and a test if i can figure out a
test as well.

> It may also be good to review the naming. Until now, the name attribute
> in nettle_cipher and similar structs has been intended to be displayed
> for informational purposes only. With your interface, users will be
> invited to use it as an identifier to look up an algorithm. How do you
> intend to use that name in the perl bindings?

The perl bindings already expose an enumeration + lookup interface, but
i've implemented it myself outside of nettle.  I'll be happy to keep the
existing perl API but drop the Crypt::Nettle implementation of it in
favor of one in the main library, though.

fwiw, i have no objections to the current naming scheme.  I'll probably
do a case-insensitive match against the exported names.

Your earlier suggestion that the user should be able to say "aes" and
then we would select an algorithm based on the supplied keysize suggests
a particular simple naming convention, where the keysize is always a
strict suffix to the algorithm name.

That is, a library might use a function by comparing the requested
algorithm name as a strict prefix with the size of the keylength.

I suppose that selection function could be part of libnettle as well,
instead of making each binding report it.  something like:

const nettle_cipher*
nettle_select_cipher(const char* algoname, int keylength = 0);

This would not remove the need for the other API, though, since it
doesn't provide algorithm enumeration.

I can try to include such an implementation in the revised patch if
y'all think it would be useful.

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110321/dcc4dc64/attachment.pgp 

From nisse at lysator.liu.se  Mon Mar 21 21:50:13 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 21 Mar 2011 21:50:13 +0100
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <4D87B5F8.8040908@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Mon, 21 Mar 2011 16:32:56 -0400")
References: <4D877027.6040207@fifthhorseman.net>
	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>
	<4D87AA1E.7080001@fifthhorseman.net>
	<nn8vw8b5ki.fsf@stalhein.lysator.liu.se>
	<4D87B5F8.8040908@fifthhorseman.net>
Message-ID: <nn4o6wb3lm.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> That is, a library might use a function by comparing the requested
> algorithm name as a strict prefix with the size of the keylength.

If that works with all algorithms, that's good. I recall I changed or
was about to change these things for the CAST cipher a while back.
"arctwo_gutmann" may also be a corner case.

> const nettle_cipher*
> nettle_select_cipher(const char* algoname, int keylength = 0);

I think this function might fit better as an example in the
documentation. (And you can't do default arguments like that in C).

There are so many queries one might want to do. Beside this example,
"give me all variants of aes", or "give me all block ciphers with a
given block size", so I think it's a bit premature to decide which query
functions to add.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Tue Mar 22 22:01:01 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Tue, 22 Mar 2011 17:01:01 -0400
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <4D87B5F8.8040908@fifthhorseman.net>
References: <4D877027.6040207@fifthhorseman.net>	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>	<4D87AA1E.7080001@fifthhorseman.net>	<nn8vw8b5ki.fsf@stalhein.lysator.liu.se>
	<4D87B5F8.8040908@fifthhorseman.net>
Message-ID: <4D890E0D.8080000@fifthhorseman.net>

On 03/21/2011 04:32 PM, Daniel Kahn Gillmor wrote:
> i'll work on a revised patch with docs and a test if i can figure out a
> test as well.

Attached is a patch that breaks out separate compilation units,
const-ifies the arrays, and adjusts the docs and the test suite.

I did not see nettle_armor in the docs at all, so i didn't add any docs
about nettle_armors[].  Maybe that whole section needs to be written?

Any further comments?

	--dkg
-------------- next part --------------
A non-text attachment was scrubbed...
Name: make-algorithms-dynamically-enumerable.patch
Type: text/x-diff
Size: 11193 bytes
Desc: not available
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110322/8e2b10d4/attachment.patch 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110322/8e2b10d4/attachment.pgp 

From dkg at fifthhorseman.net  Tue Mar 22 22:01:19 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Tue, 22 Mar 2011 17:01:19 -0400
Subject: developing nettle with git
Message-ID: <4D890E1F.30208@fifthhorseman.net>

hi folks--

I'm managing my nettle work via git.  I've extracted the cvs history of
the upstream repo (nettle-only -- i'd rather not deal with all of lsh)
and imported it into git.

OF at all possible I plan to keep the "origin" branch up-to-date with
the upstream CVS.

Anyone who prefers to use git is welcome to use my repo as a jumping off
point so you don't have to hassle with git cvsimport yourself.

You can find it here:

 git clone git://lair.fifthhorseman.net/~dkg/nettle

I would be happy to have nettle sources tracked as a separate project
from lsh, just to make it easier for people to hack on it.

Niels, if you want to consider transitioning nettle sources out of the
the lsh repository and into something like git (or any other dvcs), i'd
be happy to help you do that.

Regards,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110322/cf12c674/attachment.pgp 

From nisse at lysator.liu.se  Wed Mar 23 11:12:23 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 23 Mar 2011 11:12:23 +0100
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <4D890E0D.8080000@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Tue, 22 Mar 2011 17:01:01 -0400")
References: <4D877027.6040207@fifthhorseman.net>
	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>
	<4D87AA1E.7080001@fifthhorseman.net>
	<nn8vw8b5ki.fsf@stalhein.lysator.liu.se>
	<4D87B5F8.8040908@fifthhorseman.net>
	<4D890E0D.8080000@fifthhorseman.net>
Message-ID: <nnlj0687so.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> Attached is a patch that breaks out separate compilation units,
> const-ifies the arrays, and adjusts the docs and the test suite.

Thanks. I'm going to commit this. I noticed some incorrect file names in
the file headers (which I'll fix), and there's no copyright line. I
guess I should simply add

  Copyright (C) 2011 Daniel Kahn Gillmor

at the top of the new files.

> I did not see nettle_armor in the docs at all, so i didn't add any docs
> about nettle_armors[].  Maybe that whole section needs to be written?

The reason these functionality is undocumented is most likely that I
wasn't confident that the interface is right, it's not been used very
much.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Wed Mar 23 11:25:38 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 23 Mar 2011 11:25:38 +0100
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <4D890E0D.8080000@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Tue, 22 Mar 2011 17:01:01 -0400")
References: <4D877027.6040207@fifthhorseman.net>
	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>
	<4D87AA1E.7080001@fifthhorseman.net>
	<nn8vw8b5ki.fsf@stalhein.lysator.liu.se>
	<4D87B5F8.8040908@fifthhorseman.net>
	<4D890E0D.8080000@fifthhorseman.net>
Message-ID: <nnhbau876l.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> Any further comments?

The following (in nettle.texinfo) looks a bit strange.

> + at deftypevrx {Constant Struct * Constant} {struct nettle_hash *} nettle_hashes[]

> + at deftypevrx {Constant Struct * const} {struct nettle_cipher *} nettle_ciphers[]

I'm no texinfo guru, but I think the first {...} is supposed to be just a category
(Function, Macro, Variable, etc), and the second {...} the type. I thik

  @deftypevrx {Constant Struct} {struct nettle_hash **} nettle_hashes

would be more correct (unless we want a new categories "Algorithm List"
which I think is a bit overkill), but I haven't yet checked the
generated output.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Wed Mar 23 13:07:20 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 23 Mar 2011 13:07:20 +0100
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <nnlj0687so.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Wed, 23 Mar 2011 11:12:23 +0100")
References: <4D877027.6040207@fifthhorseman.net>
	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>
	<4D87AA1E.7080001@fifthhorseman.net>
	<nn8vw8b5ki.fsf@stalhein.lysator.liu.se>
	<4D87B5F8.8040908@fifthhorseman.net>
	<4D890E0D.8080000@fifthhorseman.net>
	<nnlj0687so.fsf@stalhein.lysator.liu.se>
Message-ID: <nnd3li82h3.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> Thanks. I'm going to commit this.

Checked in now. 

I also wrote a program tools/nettle-hash to take advantage of the new
interface. Comments appreciated. 

Should output be more compatible with md5sum, sha1sum et al? But I think
I like to also include information on the algorithm used.

Should it have a -c (check) mode as well?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Wed Mar 23 16:56:20 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Wed, 23 Mar 2011 11:56:20 -0400
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <nnhbau876l.fsf@stalhein.lysator.liu.se>
References: <4D877027.6040207@fifthhorseman.net>	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>	<4D87AA1E.7080001@fifthhorseman.net>	<nn8vw8b5ki.fsf@stalhein.lysator.liu.se>	<4D87B5F8.8040908@fifthhorseman.net>	<4D890E0D.8080000@fifthhorseman.net>
	<nnhbau876l.fsf@stalhein.lysator.liu.se>
Message-ID: <4D8A1824.6060607@fifthhorseman.net>

On 03/23/2011 06:25 AM, Niels M?ller wrote:
> Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:
> 
>> Any further comments?
> 
> The following (in nettle.texinfo) looks a bit strange.
> 
>> + at deftypevrx {Constant Struct * Constant} {struct nettle_hash *} nettle_hashes[]
> 
>> + at deftypevrx {Constant Struct * const} {struct nettle_cipher *} nettle_ciphers[]
> 
> I'm no texinfo guru, but I think the first {...} is supposed to be just a category
> (Function, Macro, Variable, etc), and the second {...} the type. I thik
> 
>   @deftypevrx {Constant Struct} {struct nettle_hash **} nettle_hashes
> 
> would be more correct (unless we want a new categories "Algorithm List"
> which I think is a bit overkill), but I haven't yet checked the
> generated output.

Ah, ok.  I'm no texinfo guru either -- i just put in what made the pdf
output look reasonable.  Your changes make at least as much sense to me
as what i put in there.

Thanks for fixing it,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110323/106c09e5/attachment.pgp 

From dkg at fifthhorseman.net  Wed Mar 23 16:59:21 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Wed, 23 Mar 2011 11:59:21 -0400
Subject: making algorithms in nettle dynamically enumerable
In-Reply-To: <nnlj0687so.fsf@stalhein.lysator.liu.se>
References: <4D877027.6040207@fifthhorseman.net>	<nnhbawb8h1.fsf@stalhein.lysator.liu.se>	<4D87AA1E.7080001@fifthhorseman.net>	<nn8vw8b5ki.fsf@stalhein.lysator.liu.se>	<4D87B5F8.8040908@fifthhorseman.net>	<4D890E0D.8080000@fifthhorseman.net>
	<nnlj0687so.fsf@stalhein.lysator.liu.se>
Message-ID: <4D8A18D9.5070600@fifthhorseman.net>

On 03/23/2011 06:12 AM, Niels M?ller wrote:
> Thanks. I'm going to commit this. I noticed some incorrect file names in
> the file headers (which I'll fix), and there's no copyright line. I
> guess I should simply add
> 
>   Copyright (C) 2011 Daniel Kahn Gillmor
> 
> at the top of the new files.

This is fine, thanks.  sorry about the incorrect file names -- i hate
being caught copying and pasting! :P

>> I did not see nettle_armor in the docs at all, so i didn't add any docs
>> about nettle_armors[].  Maybe that whole section needs to be written?
> 
> The reason these functionality is undocumented is most likely that I
> wasn't confident that the interface is right, it's not been used very
> much.

That makes sense to me.  I looked at trying to make a
Crypt::Nettle::Armor in the perl bindings to take advantage of it, and
decided that (a) i wasn't exactly sure how i wanted to use it, and (b)
it probably wasn't worth it because perl has good native base64 and
base16 encoding/decoding anyway.

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110323/974189f2/attachment.pgp 

From hoyt6 at llnl.gov  Thu Mar 24 22:40:52 2011
From: hoyt6 at llnl.gov (Hoyt, David)
Date: Thu, 24 Mar 2011 14:40:52 -0700
Subject: developing nettle with git
In-Reply-To: <4D890E1F.30208@fifthhorseman.net>
References: <4D890E1F.30208@fifthhorseman.net>
Message-ID: <B288BDCCFC5B874BBC7D22117EDFB6F9F3786D7E69@NSPEXMBX-C.the-lab.llnl.gov>

> Niels, if you want to consider transitioning nettle sources out of the the lsh repository and into something like git

+1 to that.

From nisse at lysator.liu.se  Fri Mar 25 11:45:05 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 25 Mar 2011 11:45:05 +0100
Subject: developing nettle with git
In-Reply-To: <4D890E1F.30208@fifthhorseman.net> (Daniel Kahn Gillmor's message
	of "Tue, 22 Mar 2011 17:01:19 -0400")
References: <4D890E1F.30208@fifthhorseman.net>
Message-ID: <nnmxkj7a32.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> Niels, if you want to consider transitioning nettle sources out of the
> the lsh repository and into something like git (or any other dvcs), i'd
> be happy to help you do that.

I'm certainly considering it; it would be nice to get away from cvs. I
was involved in the conversion of the gmp repository to mercurial just a
few years ago. That was fairly easy, only serious obstacle was that we
discovered a rcs file where some early revision was corrupt. After
purging that broken revision with cvs admin, things worked well.

But to convert nettle and lsh, there are some issues:

1. I'm familiar with mercurial, but I hear that git is a better choice
   these days, so maybe I should go with git instead. And then I have
   some new stuff to learn.

2. Currently, several more-or-less independent projects are in the lsh
   cvs tree. Besides nettle, there's libspki, argp, sftp. So how should
   this be split up? As far as I understand, neither hg nor git have
   any well-working analogue to cvs modules, which would otherwise have
   been a natural choice.

   Also, I guess this splitting of the repository complicates migration
   a bit.

3. Then there's the shared files which are symlinked by the top-level
   lsh/.bootstrap script. I guess with a split it will be unavoidable to
   have multiple separately version controlled copies of those files.

I guess it's possible to start with migrating nettle from cvs, and leave
the rest of the tree for later. But I'm not sure that's a good way to do
it.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Fri Mar 25 20:28:32 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Fri, 25 Mar 2011 15:28:32 -0400
Subject: developing nettle with git
In-Reply-To: <nnmxkj7a32.fsf@stalhein.lysator.liu.se>
References: <4D890E1F.30208@fifthhorseman.net>
	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>
Message-ID: <4D8CECE0.6090307@fifthhorseman.net>

On 03/25/2011 06:45 AM, Niels M?ller wrote:
> 1. I'm familiar with mercurial, but I hear that git is a better choice
>    these days, so maybe I should go with git instead. And then I have
>    some new stuff to learn.

i find myself more effective with git than with mercurial, but i
acknowledge that i have spent more time thinking about git (and many
more other projects i collaborate on use git as well, so not spending
time with it isn't much of an option for me).

> 2. Currently, several more-or-less independent projects are in the lsh
>    cvs tree. Besides nettle, there's libspki, argp, sftp. So how should
>    this be split up? As far as I understand, neither hg nor git have
>    any well-working analogue to cvs modules, which would otherwise have
>    been a natural choice.

git does have submodule support -- so you can have one repository
reference another at a given commit.  I don't use it often, but i can
help you sort it out if you like.  "git help submodule" for some starter
documentation/reference.

OTOH, if these separate pieces are actually different libraries, i'm not
sure that using submodules really makes a lot of sense.  I think you
just want one git repository per project.  Is there a reason that lsh
would need to be tightly-coupled with any of these libraries instead of
using their exported API?

One git repo per project is a pretty reasonable approach.  And it's not
hard to serve several git repos from a single server.  If you want help
setting that up, i'd be happy to help out.

> 3. Then there's the shared files which are symlinked by the top-level
>    lsh/.bootstrap script. I guess with a split it will be unavoidable to
>    have multiple separately version controlled copies of those files.

i think you might be right here.  For my git development, i took static
copies of those files and placed them directly in my git repo (the
commit log comment starts with "transfer files from lsh".

How often do these files actually change?  Is it critical that they be
kept in sync?

> I guess it's possible to start with migrating nettle from cvs, and leave
> the rest of the tree for later. But I'm not sure that's a good way to do
> it.

I actually don't think that's an unreasonable approach.  If we just
migrate one component (nettle), we can make sure the migrated setup is
OK and that your workflow feels sensible before going through the work
of the other subprojects.  We can apply any new knowledge learned from
the nettle transition to the migration of the other subprojects.

Regards,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110325/b750d794/attachment.pgp 

From nisse at lysator.liu.se  Sat Mar 26 08:46:09 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 26 Mar 2011 08:46:09 +0100
Subject: developing nettle with git
In-Reply-To: <4D8CECE0.6090307@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Fri, 25 Mar 2011 15:28:32 -0400")
References: <4D890E1F.30208@fifthhorseman.net>
	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>
	<4D8CECE0.6090307@fifthhorseman.net>
Message-ID: <nn39ma729q.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> git does have submodule support -- so you can have one repository
> reference another at a given commit.

I don't have any first-hand experience with git submodules, but I've
heard some stories saying that it's not yet very usable.

> Is there a reason that lsh would need to be tightly-coupled with any
> of these libraries instead of using their exported API?

The main reason is that lsh is the main testcase. So when introducing
new features one of those those libraries, I usually first hack lsh to
use it, to iron out both interface and implementation bugs. Those
intermediate versions may never be in the API of any released library.

git submodules (assuming that they work well) would probably be the
right tool to keep these libraries in subdirectories of the lsh work
tree.

Ah, and then it's the release procedure. I'd still like to bundle the
libraries with lsh, and having the libraries in subdirectories makes
that easier. And as long as the versions bundled with nettle are
configured for static linking, I see no problem of bundling lsh with
unreleased versions of the libraries.

> How often do these files actually change?  Is it critical that they be
> kept in sync?

It's not critical, it's just for convenience. It's nice to have a single
version, which is used (and tested) with all the projects.

>> I guess it's possible to start with migrating nettle from cvs, and leave
>> the rest of the tree for later. But I'm not sure that's a good way to do
>> it.
>
> I actually don't think that's an unreasonable approach.  If we just
> migrate one component (nettle), we can make sure the migrated setup is
> OK and that your workflow feels sensible before going through the work
> of the other subprojects.

If we do that, I guess we'd need the following steps:

1. migrate the nettle cvs directory to a git repository.

2. cvs rm all those files from the cvs tree (as usual, we can't cvs rm
   the directories themselves).

3. At some later time, migrate the lsh cvs repository to git. Should the
   old nettle subdirectory be excluded somehow? Perhaps not, if I want
   to be able to check out old lsh versions from git.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Mon Mar 28 01:56:47 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Sun, 27 Mar 2011 19:56:47 -0400
Subject: developing nettle with git
In-Reply-To: <nn39ma729q.fsf@stalhein.lysator.liu.se>
References: <4D890E1F.30208@fifthhorseman.net>	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>	<4D8CECE0.6090307@fifthhorseman.net>
	<nn39ma729q.fsf@stalhein.lysator.liu.se>
Message-ID: <4D8FCEBF.7070104@fifthhorseman.net>

On 03/26/2011 03:46 AM, Niels M?ller wrote:
> The main reason is that lsh is the main testcase. So when introducing
> new features one of those those libraries, I usually first hack lsh to
> use it, to iron out both interface and implementation bugs. Those
> intermediate versions may never be in the API of any released library.
> 
> git submodules (assuming that they work well) would probably be the
> right tool to keep these libraries in subdirectories of the lsh work
> tree.

Yeah, you can definitely do that with git submodules.  I'd be happy to
help you iron out kinks in that workflow if you like.

> Ah, and then it's the release procedure. I'd still like to bundle the
> libraries with lsh, and having the libraries in subdirectories makes
> that easier. And as long as the versions bundled with nettle are
> configured for static linking, I see no problem of bundling lsh with
> unreleased versions of the libraries.

hm, are you talking about making binary releases?  or source releases?

> If we do that, I guess we'd need the following steps:
> 
> 1. migrate the nettle cvs directory to a git repository.

this is already done, if you don't mind using my git repo as the base:

 git://lair.fifthhorseman.net/~dkg/nettle

You can just clone it and then publish it yourself :)

if you want to do the repository conversion yourself, i understand that
too -- i'm just trying to help.

To do it yourself, you want to use git cvsimport (from the git-cvs
package in debian).

> 2. cvs rm all those files from the cvs tree (as usual, we can't cvs rm
>    the directories themselves).
> 
> 3. At some later time, migrate the lsh cvs repository to git. Should the
>    old nettle subdirectory be excluded somehow? Perhaps not, if I want
>    to be able to check out old lsh versions from git.

i don't think you'll need to exclude the old nettle subdirectory from
any future lsh cvs->git migration, especially if the directory has been
removed from the HEAD already.

hth,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110327/940e8c0d/attachment.pgp 

From nisse at lysator.liu.se  Mon Mar 28 09:17:35 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 28 Mar 2011 09:17:35 +0200
Subject: developing nettle with git
In-Reply-To: <4D8FCEBF.7070104@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Sun, 27 Mar 2011 19:56:47 -0400")
References: <4D890E1F.30208@fifthhorseman.net>
	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>
	<4D8CECE0.6090307@fifthhorseman.net>
	<nn39ma729q.fsf@stalhein.lysator.liu.se>
	<4D8FCEBF.7070104@fifthhorseman.net>
Message-ID: <nnd3lb67e8.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> hm, are you talking about making binary releases?  or source releases?

I'm talking about the source releases of lsh.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Mon Mar 28 10:15:33 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 28 Mar 2011 10:15:33 +0200
Subject: developing nettle with git
In-Reply-To: <4D8FCEBF.7070104@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Sun, 27 Mar 2011 19:56:47 -0400")
References: <4D890E1F.30208@fifthhorseman.net>
	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>
	<4D8CECE0.6090307@fifthhorseman.net>
	<nn39ma729q.fsf@stalhein.lysator.liu.se>
	<4D8FCEBF.7070104@fifthhorseman.net>
Message-ID: <nn4o6n64pm.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> To do it yourself, you want to use git cvsimport (from the git-cvs
> package in debian).

I think I'd prefer to do the conversion myself. I've had a first look at
the man page for git-cvsimport.

I take it I first need to scan the cvs log for all comitters and write a
cvs-authors file (I remember I had some trouble with character set
issues for the corresponding file when doing the hg migration for gmp.
Should the names here be in the $LC_CTYPE charset, or always utf8?).

Then, I'm not sure how to deal with branches, in particular, the -r and
-o options. I'm not used to git, and with hg I also haven't used any
"real" branches, I've just cloned the repository. I'm a bit scared by
"You should never do any work of your own on the branches that are
created by git cvsimport." in the man page.

I think I'd like to keep the possibility of doing cvs import more than
once.

What command line did you use? What branches did you end up with?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Mon Mar 28 15:07:45 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 28 Mar 2011 15:07:45 +0200
Subject: developing nettle with git
In-Reply-To: <4D8FCEBF.7070104@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Sun, 27 Mar 2011 19:56:47 -0400")
References: <4D890E1F.30208@fifthhorseman.net>
	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>
	<4D8CECE0.6090307@fifthhorseman.net>
	<nn39ma729q.fsf@stalhein.lysator.liu.se>
	<4D8FCEBF.7070104@fifthhorseman.net>
Message-ID: <nnzkof4cm6.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> Yeah, you can definitely do that with git submodules.

Hmm. I'm not sure that will work the way I like. I've been told that if
I have an lsh directory with nettle as a submodule, I can't easily
modify nettle and commit and push changes from that tree.

git-subtree may be more useful (see
https://github.com/apenwarr/git-subtree/blob/master/git-subtree.txt and
http://psionides.jogger.pl/2010/02/04/sharing-code-between-projects-with-git-subtree/)

Maybe it makes things simpler to stick to a single repository for
everything (like the current cvs repository)? Is there any compelling
reason why nettle must be a separate repository? (If I understand the
docs correctly, one could split off a nettle repository later, using
git-subtree split).

Regardss,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Mon Mar 28 19:03:08 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 28 Mar 2011 13:03:08 -0400
Subject: developing nettle with git
In-Reply-To: <nnd3lb67e8.fsf@stalhein.lysator.liu.se>
References: <4D890E1F.30208@fifthhorseman.net>	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>	<4D8CECE0.6090307@fifthhorseman.net>	<nn39ma729q.fsf@stalhein.lysator.liu.se>	<4D8FCEBF.7070104@fifthhorseman.net>
	<nnd3lb67e8.fsf@stalhein.lysator.liu.se>
Message-ID: <4D90BF4C.400@fifthhorseman.net>

On 03/28/2011 03:17 AM, Niels M?ller wrote:
> Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:
> 
>> hm, are you talking about making binary releases?  or source releases?
> 
> I'm talking about the source releases of lsh.

I'm a software developer, a sysadmin, and a contributor to a distro
(debian).  No matter which of those three hats i'm wearing, i *strongly*
prefer to have source releases broken out into separable components,
rather than one monolithic tarball.  I prefer this for system
maintenance, for security, for clarity of API, and for ease of modification.

I'm not sure what the argument for monolithic source release is.

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110328/b416be1b/attachment.pgp 

From dkg at fifthhorseman.net  Mon Mar 28 19:22:54 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 28 Mar 2011 13:22:54 -0400
Subject: developing nettle with git
In-Reply-To: <nn4o6n64pm.fsf@stalhein.lysator.liu.se>
References: <4D890E1F.30208@fifthhorseman.net>	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>	<4D8CECE0.6090307@fifthhorseman.net>	<nn39ma729q.fsf@stalhein.lysator.liu.se>	<4D8FCEBF.7070104@fifthhorseman.net>
	<nn4o6n64pm.fsf@stalhein.lysator.liu.se>
Message-ID: <4D90C3EE.4010403@fifthhorseman.net>

On 03/28/2011 04:15 AM, Niels M?ller wrote:
> I take it I first need to scan the cvs log for all comitters and write a
> cvs-authors file (I remember I had some trouble with character set
> issues for the corresponding file when doing the hg migration for gmp.
> Should the names here be in the $LC_CTYPE charset, or always utf8?).

i'd do everything in utf8 myself, but then my LC_CTYPE always uses UTF-8
anyway.  I don't know what the consequences are of doing things other ways.

> Then, I'm not sure how to deal with branches, in particular, the -r and
> -o options. I'm not used to git, and with hg I also haven't used any
> "real" branches, I've just cloned the repository. I'm a bit scared by
> "You should never do any work of your own on the branches that are
> created by git cvsimport." in the man page.
> 
> I think I'd like to keep the possibility of doing cvs import more than
> once.

Meaning you want to update the cvs tree concurrently with the git repo?

This is doable, but it means that your git repo will need to be
regularly rebased against the CVS tree.  I've been doing this with the
git repo i've published, but that's because the CVS tree is still in
use.  doing regular git cvsimports against an active CVS tree will make
it more difficult for other people to develop against the git tree,
which seems like it defeats one of the nice advantages of moving to a
modern dvcs.

> What command line did you use? What branches did you end up with?

I ran something like this:

cvs -d :pserver:anonymous at cvs.lysator.liu.se:/cvsroot/lsh \
   co -d nettle lsh/nettle
cd nettle
git cvsimport

I ended up with no extra branches, though.  Just a handful of tags.
What sort of branches do you want to end up with?

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110328/225442df/attachment.pgp 

From dkg at fifthhorseman.net  Mon Mar 28 19:47:29 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 28 Mar 2011 13:47:29 -0400
Subject: developing nettle with git
In-Reply-To: <nnzkof4cm6.fsf@stalhein.lysator.liu.se>
References: <4D890E1F.30208@fifthhorseman.net>	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>	<4D8CECE0.6090307@fifthhorseman.net>	<nn39ma729q.fsf@stalhein.lysator.liu.se>	<4D8FCEBF.7070104@fifthhorseman.net>
	<nnzkof4cm6.fsf@stalhein.lysator.liu.se>
Message-ID: <4D90C9B1.1090906@fifthhorseman.net>

On 03/28/2011 09:07 AM, Niels M?ller wrote:
> Maybe it makes things simpler to stick to a single repository for
> everything (like the current cvs repository)? Is there any compelling
> reason why nettle must be a separate repository? (If I understand the
> docs correctly, one could split off a nettle repository later, using
> git-subtree split).

I'd personally like to work on nettle without dealing with the entire
source of lsh and the other components.

I suspect the same might be true of people who want to hack on the other
components, or on lsh itself.

If you want to encourage external participation, separate repositories
make more sense to me.

How do other folks on this list feel about the tradeoffs here?

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110328/d7b8fea2/attachment.pgp 

From simon at josefsson.org  Mon Mar 28 19:54:33 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Mon, 28 Mar 2011 19:54:33 +0200
Subject: developing nettle with git
In-Reply-To: <4D90BF4C.400@fifthhorseman.net> (Daniel Kahn Gillmor's message
	of "Mon, 28 Mar 2011 13:03:08 -0400")
References: <4D890E1F.30208@fifthhorseman.net>
	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>
	<4D8CECE0.6090307@fifthhorseman.net>
	<nn39ma729q.fsf@stalhein.lysator.liu.se>
	<4D8FCEBF.7070104@fifthhorseman.net>
	<nnd3lb67e8.fsf@stalhein.lysator.liu.se>
	<4D90BF4C.400@fifthhorseman.net>
Message-ID: <87oc4v5dwm.fsf@latte.josefsson.org>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> On 03/28/2011 03:17 AM, Niels M?ller wrote:
>> Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:
>> 
>>> hm, are you talking about making binary releases?  or source releases?
>> 
>> I'm talking about the source releases of lsh.
>
> I'm a software developer, a sysadmin, and a contributor to a distro
> (debian).  No matter which of those three hats i'm wearing, i *strongly*
> prefer to have source releases broken out into separable components,
> rather than one monolithic tarball.  I prefer this for system
> maintenance, for security, for clarity of API, and for ease of modification.

+1

As Nettle is looking to become more widely, it seems like a good time to
make a separate repository for it.

Lsh could just import the latest stable Nettle release, couldn't it?

/Simon

From nisse at lysator.liu.se  Mon Mar 28 20:39:29 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 28 Mar 2011 20:39:29 +0200
Subject: developing nettle with git
In-Reply-To: <4D90C3EE.4010403@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Mon, 28 Mar 2011 13:22:54 -0400")
References: <4D890E1F.30208@fifthhorseman.net>
	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>
	<4D8CECE0.6090307@fifthhorseman.net>
	<nn39ma729q.fsf@stalhein.lysator.liu.se>
	<4D8FCEBF.7070104@fifthhorseman.net>
	<nn4o6n64pm.fsf@stalhein.lysator.liu.se>
	<4D90C3EE.4010403@fifthhorseman.net>
Message-ID: <nnei5r3x9a.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> On 03/28/2011 04:15 AM, Niels M?ller wrote:
>> I think I'd like to keep the possibility of doing cvs import more than
>> once.
>
> Meaning you want to update the cvs tree concurrently with the git repo?

Not in the longer term, but maybe during a transition period. But I
guess it's not essential.

> I ran something like this:
>
> cvs -d :pserver:anonymous at cvs.lysator.liu.se:/cvsroot/lsh \
>    co -d nettle lsh/nettle
> cd nettle
> git cvsimport

It all ended up on the "origin" branch then? (I'm still not familiar with
git conventions). 

> I ended up with no extra branches, though.  Just a handful of tags.
> What sort of branches do you want to end up with?

There are a few branches, in the lsh tree I think there's
lsh_1_2_BRANCH, lsh_1_4_2_BRANCH, lsh_2_0_4_BRANCH,
experimental_BRANCH_20050201. I branched and tagged the complete tree at
the time, but appearantly that was before the move from lsh/src/nettle
to lsh/nettle, so there are no tags in the current nettle subdirectory.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Tue Mar 29 06:24:42 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Tue, 29 Mar 2011 00:24:42 -0400
Subject: weak-key ciphers and arcfour
Message-ID: <4D915F0A.7070604@fifthhorseman.net>

hi nettle folks--

the nettle docs say:

> In Nettle, most key setup functions have no return value, but
> for ciphers with weak keys, the return value indicates whether or not
> the given key is weak. 

 [...]

> A problem is that the key
> setup of ARCFOUR is quite weak, you should never use keys with
> structure, keys that are ordinary passwords, or sequences of keys like
> ``secret:1'', ``secret:2'', ...

 [...]

> void arcfour_set_key (struct arcfour_ctx *ctx, unsigned length, const uint8_t *key)
> Initialize the cipher. The same function is used for both encryption and
> decryption. 

Put together, these three statements seem contradictory.

If arcfour has weak keys, shouldn't arcfour_set_key return an int
indicating whether the key is considered weak (like the key setup
functions for BLOWFISH, DES, and DES3 do)?

Is the problem that there is no clear way to determine if an arcfour key
is weak?

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110329/75873f29/attachment.pgp 

From nisse at lysator.liu.se  Tue Mar 29 07:19:22 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 29 Mar 2011 07:19:22 +0200
Subject: weak-key ciphers and arcfour
In-Reply-To: <4D915F0A.7070604@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Tue, 29 Mar 2011 00:24:42 -0400")
References: <4D915F0A.7070604@fifthhorseman.net>
Message-ID: <nn62r24i79.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

>> A problem is that the key
>> setup of ARCFOUR is quite weak, you should never use keys with
>> structure, keys that are ordinary passwords, or sequences of keys like
>> ``secret:1'', ``secret:2'', ...

The problem with arcfour is not that some particular keys are
unexpectedly weak, but that the key bits are not spread out very well
into the internal state (sorry if this description is a bit vague; my
understanding is also a bit vague...).

So there's unexpectedly high correlation between the first bytes of the
key and the first bytes of the generated key stream, and I think there's
also undesired correlation between key streams for close keys.

I think the recommendations in the manual (hash the key first, and
discard the initial bytes of the key stream) are still adequate.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Tue Mar 29 08:18:30 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Tue, 29 Mar 2011 02:18:30 -0400
Subject: struct nettle_cipher for weak-key ciphers
Message-ID: <4D9179B6.4060702@fifthhorseman.net>

hi nettle folks--

now that Crypt::Nettle seems effective and functional, i'm starting to
look at using it in other systems i'm working on.  Suddenly, i realize
i'm missing access to 3DES and BLOWFISH, which i find i actually want :/

I'm missing them because there is no struct nettle_cipher for these
algorithms (or for DES, for that matter, though i care less about DES).

I seem to have a few options:

 0) Crypt::Nettle could write unique interfaces to those ciphers and
expose them to the user of the perl module as (for example)
Crypt::Nettle::Cipher::3DES and Crypt::Nettle::Cipher::Blowfish .  this
breaks symmetry with the rest of the interface, though.

 1) Crypt::Nettle could create its own struct nettle_cipher objects for
these ciphers, wrapping the weak key checking in some code of that
belongs to the perl module

 2) I could propose that nettle to create struct nettle_cipher objects
for these ciphers directly.


I prefer (1) or (2) because they'll keep a simple interface for
Crypt::Nettle.  I'm not sure how to do (2) without breaking ABI in
nettle somehow (or losing the weak-key error checking).  But going with
option (1) seems likely to cause code duplication in any other
higher-level bindings that use the struct nettle_cipher objects to
present a normalized interface.

Any thoughts on how i should proceed?  I can certainly do (1)
independently of libnettle itself, but if there's a way to handle (2)
more cleanly than i've been able to imagine thus far, i'd be happy to
hear about it.

Or, am i barking up the wrong tree entirely?  I'm imagining (for
example) a user who has in their possession a symmetrically-encrypted
message that they happen to know the key for.  The cipher used was one
of the "weak-key" ciphers, and it's even possible that the key is in
fact a weak key.  The user should still be able to decrypt the message
using Crypt::Nettle (or any other binding).

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110329/b5c64916/attachment.pgp 

From nmav at gnutls.org  Tue Mar 29 09:59:29 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 29 Mar 2011 09:59:29 +0200
Subject: struct nettle_cipher for weak-key ciphers
In-Reply-To: <4D9179B6.4060702@fifthhorseman.net>
References: <4D9179B6.4060702@fifthhorseman.net>
Message-ID: <4D919161.2000204@gnutls.org>

On 03/29/2011 08:18 AM, Daniel Kahn Gillmor wrote:

> now that Crypt::Nettle seems effective and functional, i'm starting to
> look at using it in other systems i'm working on.  Suddenly, i realize
> i'm missing access to 3DES and BLOWFISH, which i find i actually want :/
> I'm missing them because there is no struct nettle_cipher for these
> algorithms (or for DES, for that matter, though i care less about DES).
> I seem to have a few options:
> 
>  0) Crypt::Nettle could write unique interfaces to those ciphers and
> expose them to the user of the perl module as (for example)
> Crypt::Nettle::Cipher::3DES and Crypt::Nettle::Cipher::Blowfish .  this
> breaks symmetry with the rest of the interface, though.
>  1) Crypt::Nettle could create its own struct nettle_cipher objects for
> these ciphers, wrapping the weak key checking in some code of that
> belongs to the perl module
>  2) I could propose that nettle to create struct nettle_cipher objects
> for these ciphers directly.
> I prefer (1) or (2) because they'll keep a simple interface for
> Crypt::Nettle.  I'm not sure how to do (2) without breaking ABI in
> nettle somehow (or losing the weak-key error checking). 

I'd also prefer (2), because it reduces work for nettle consumers.
For gnutls I didn't use nettle_cipher at all and created my own
wrappers, because nettle cipher works only with few ciphers.
At least for TLS, weak key checking is not that important due to
low probability of selecting one, to be of any practical concern.

regards,
Nikos

From nmav at gnutls.org  Tue Mar 29 10:03:10 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 29 Mar 2011 10:03:10 +0200
Subject: developing nettle with git
In-Reply-To: <4D90C9B1.1090906@fifthhorseman.net>
References: <4D890E1F.30208@fifthhorseman.net>	<nnmxkj7a32.fsf@stalhein.lysator.liu.se>	<4D8CECE0.6090307@fifthhorseman.net>	<nn39ma729q.fsf@stalhein.lysator.liu.se>	<4D8FCEBF.7070104@fifthhorseman.net>	<nnzkof4cm6.fsf@stalhein.lysator.liu.se>
	<4D90C9B1.1090906@fifthhorseman.net>
Message-ID: <4D91923E.80101@gnutls.org>

On 03/28/2011 07:47 PM, Daniel Kahn Gillmor wrote:
> On 03/28/2011 09:07 AM, Niels M?ller wrote:
>> Maybe it makes things simpler to stick to a single repository for
>> everything (like the current cvs repository)? Is there any compelling
>> reason why nettle must be a separate repository? (If I understand the
>> docs correctly, one could split off a nettle repository later, using
>> git-subtree split).
> I'd personally like to work on nettle without dealing with the entire
> source of lsh and the other components.

That is the case with me as well. Moreover requiring someone to
checkout a different project to access nettle, just gives the impression
that the library is not intended for general use, but is rather an
internal library of that project.

regards,
Nikos

From simon at josefsson.org  Tue Mar 29 10:11:00 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Tue, 29 Mar 2011 10:11:00 +0200
Subject: weak-key ciphers and arcfour
In-Reply-To: <nn62r24i79.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Tue, 29 Mar 2011 07:19:22 +0200")
References: <4D915F0A.7070604@fifthhorseman.net>
	<nn62r24i79.fsf@stalhein.lysator.liu.se>
Message-ID: <8739m62vor.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:
>
>>> A problem is that the key
>>> setup of ARCFOUR is quite weak, you should never use keys with
>>> structure, keys that are ordinary passwords, or sequences of keys like
>>> ``secret:1'', ``secret:2'', ...
>
> The problem with arcfour is not that some particular keys are
> unexpectedly weak, but that the key bits are not spread out very well
> into the internal state (sorry if this description is a bit vague; my
> understanding is also a bit vague...).

There are some keys which are even weaker, for example keys beginning
with 00 00 FD and 03 FD FC, see this paper:

http://impic.org/papers/WeakKeys-report.pdf

On the other hand, RC4 is broken so the function might as well always
return 1 to indicate that the key is weak since it is used with RC4. :-)

/Simon

From nisse at lysator.liu.se  Tue Mar 29 11:02:03 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 29 Mar 2011 11:02:03 +0200
Subject: struct nettle_cipher for weak-key ciphers
In-Reply-To: <4D9179B6.4060702@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Tue, 29 Mar 2011 02:18:30 -0400")
References: <4D9179B6.4060702@fifthhorseman.net>
Message-ID: <nnpqpa2tbo.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> I seem to have a few options:
>
>  0) Crypt::Nettle could write unique interfaces to those ciphers and
> expose them to the user of the perl module as (for example)
> Crypt::Nettle::Cipher::3DES and Crypt::Nettle::Cipher::Blowfish .  this
> breaks symmetry with the rest of the interface, though.
>
>  1) Crypt::Nettle could create its own struct nettle_cipher objects for
> these ciphers, wrapping the weak key checking in some code of that
> belongs to the perl module
>
>  2) I could propose that nettle to create struct nettle_cipher objects
> for these ciphers directly.

I recommend (1). The way this is done in the Pike bindings (the
implementation is maybe a bit too complicated for its own good), I use a
struct pike_cipher very similar to nettle_cipher,

  /* Calls Pike_error on errors */ 
  typedef void (*pike_nettle_set_key_func)(void *ctx,
  					 ptrdiff_t length, const char *key,
  					 /* Force means to use key even
                                           if it is weak */
  					 int force);
  
  struct pike_cipher
  {
    const char *name;
    
    unsigned context_size;
  
    unsigned block_size;
  
    /* Suggested key size; other sizes are sometimes possible. */
    unsigned key_size;
  
    pike_nettle_set_key_func set_encrypt_key;
    pike_nettle_set_key_func set_decrypt_key;
  
    nettle_crypt_func encrypt;
    nettle_crypt_func decrypt;
  };

Here, the pike_nettle_set_key_func differs from nettle_set_key_func is
two ways, related to error handling:

1. It checks if the key size is appropriate for the algorithm, and
   raises an exception if not (in contrast, a bad key size passed to the
   nettle set_key function would abort the process with an assertion
   failure).

2. The behaviour for weak keys. If the force argument is zero (for Pike
   calls, it's an optional argument and omitting it also means zero), a
   weak key results in an exception. If the force argument is non-zero,
   a weak key is not not considered an error.

In these bindings, unlike yours, each cipher like AES is a single class,
with multiple supported key sizes. So all ciphers need their own set_key
wrapper for proper error checking.

In your case, where you have one separate class per possible key size, I
think you could do something similar and still use the new enumeration
interface for the "normal" algorithms. 

If you're fine with either having weak keys always raise an exception or
always be accepted, you could write set_key wrappers for the affected
ciphers which do precisely that and which adhere to the
nettle_set_key_func interface (note that des_set_key and des3_set_key
don't have a key size argument so they need wrappers also for that
reason). If you want it to be configurable, things get a bit more
complicated and you may need your own struct perl_cipher to extend
struct nettle_cipher (you could still enumerate the available
nettle_cipher and convert each to a corresponding perl_cipher). Or you
could just define separate classes with and without weak key checking.

There will be a little code duplication. But there ought to be code
*somewhere* to implements the language-specific pieces of the interface,
such as exception based error handling, and new features, like, e.g.,
the optional force argument above.

> Or, am i barking up the wrong tree entirely?  I'm imagining (for
> example) a user who has in their possession a symmetrically-encrypted
> message that they happen to know the key for.  The cipher used was one
> of the "weak-key" ciphers, and it's even possible that the key is in
> fact a weak key.  The user should still be able to decrypt the message
> using Crypt::Nettle (or any other binding).

I agree that there are certainly cases where you don't want to treat
weak keys as errors. Even though I think it makes sense to have a
default behaviour which treats weak keys as errors.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Tue Mar 29 11:08:31 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 29 Mar 2011 11:08:31 +0200
Subject: struct nettle_cipher for weak-key ciphers
In-Reply-To: <4D919161.2000204@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Tue, 29 Mar 2011 09:59:29 +0200")
References: <4D9179B6.4060702@fifthhorseman.net> <4D919161.2000204@gnutls.org>
Message-ID: <nnlizy2t0w.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> At least for TLS, weak key checking is not that important due to
> low probability of selecting one, to be of any practical concern.

In lsh, I disconnect when a weak key is detected. 

The problem with relying on "low probability" is that unless you
generate the random key all by yourself, you need that probability to be
low also in the presence of any possible attacks on the key agreement
protocol. The analysis needed to rule out such attacks may cause some
headache, which you can avoid by simply refusing to use weak keys if
they ever occur.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Tue Mar 29 11:25:17 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 29 Mar 2011 11:25:17 +0200
Subject: weak-key ciphers and arcfour
In-Reply-To: <8739m62vor.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Tue, 29 Mar 2011 10:11:00 +0200")
References: <4D915F0A.7070604@fifthhorseman.net>
	<nn62r24i79.fsf@stalhein.lysator.liu.se>
	<8739m62vor.fsf@latte.josefsson.org>
Message-ID: <nnhbam2s8y.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> There are some keys which are even weaker, for example keys beginning
> with 00 00 FD and 03 FD FC, see this paper:
>
> http://impic.org/papers/WeakKeys-report.pdf

This still exploits statistical properties of the first few generated
bytes, right? So if you generate and discard the first 512 or 1024 bytes
or so of the keystream, the statistics for these keys shouldn't be much
different from any oter keys, right?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Tue Mar 29 11:58:12 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Tue, 29 Mar 2011 11:58:12 +0200
Subject: weak-key ciphers and arcfour
In-Reply-To: <nnhbam2s8y.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Tue, 29 Mar 2011 11:25:17 +0200")
References: <4D915F0A.7070604@fifthhorseman.net>
	<nn62r24i79.fsf@stalhein.lysator.liu.se>
	<8739m62vor.fsf@latte.josefsson.org>
	<nnhbam2s8y.fsf@stalhein.lysator.liu.se>
Message-ID: <87sju61c5n.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Simon Josefsson <simon at josefsson.org> writes:
>
>> There are some keys which are even weaker, for example keys beginning
>> with 00 00 FD and 03 FD FC, see this paper:
>>
>> http://impic.org/papers/WeakKeys-report.pdf
>
> This still exploits statistical properties of the first few generated
> bytes, right?

Yes, that's true.

> So if you generate and discard the first 512 or 1024 bytes or so of
> the keystream, the statistics for these keys shouldn't be much
> different from any oter keys, right?

Right.  I think 512 bytes is a bit on the low end these days,
conservative recommendations are now up 3072 bytes:

http://www.users.zetnet.co.uk/hopwood/crypto/scan/cs.html#RC4-drop

Note that the RC4 keystream can be distinguished from random with only
about a few GB's of stream output regardless of how many initial bytes
are dropped.  This suggests to me that there are attacks that will work
regardless of how much initial output you discard.

/Simon

From nmav at gnutls.org  Tue Mar 29 17:07:13 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 29 Mar 2011 17:07:13 +0200
Subject: struct nettle_cipher for weak-key ciphers
In-Reply-To: <nnlizy2t0w.fsf@stalhein.lysator.liu.se>
References: <4D9179B6.4060702@fifthhorseman.net> <4D919161.2000204@gnutls.org>
	<nnlizy2t0w.fsf@stalhein.lysator.liu.se>
Message-ID: <4D91F5A1.7020806@gnutls.org>

On 03/29/2011 11:08 AM, Niels M?ller wrote:
> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
> 
>> At least for TLS, weak key checking is not that important due to
>> low probability of selecting one, to be of any practical concern.
> In lsh, I disconnect when a weak key is detected. 
> The problem with relying on "low probability" is that unless you
> generate the random key all by yourself, you need that probability to be
> low also in the presence of any possible attacks on the key agreement
> protocol. The analysis needed to rule out such attacks may cause some
> headache, which you can avoid by simply refusing to use weak keys if
> they ever occur.

In TLS the generated keys do not only depend on the key exchange
but also on several bytes of randomness contributed by both
peers. Even a key exchange with a malicious party, would produce
random keys with a little more than 224 bits of randomness.

Moreover if you handle weak keys, you should include it into
the protocol, i.e. do you restart the key exchange once
a weak key is detected, or you just terminate the handshake?
TLS has no provisions for a re-handshake once a weak-key
is detected.

regards,
Nikos

From dkg at fifthhorseman.net  Tue Mar 29 17:47:41 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Tue, 29 Mar 2011 11:47:41 -0400
Subject: struct nettle_cipher for weak-key ciphers
In-Reply-To: <nnpqpa2tbo.fsf@stalhein.lysator.liu.se>
References: <4D9179B6.4060702@fifthhorseman.net>
	<nnpqpa2tbo.fsf@stalhein.lysator.liu.se>
Message-ID: <4D91FF1D.9080408@fifthhorseman.net>

On 03/29/2011 05:02 AM, Niels M?ller wrote:
> If you're fine with either having weak keys always raise an exception or
> always be accepted, you could write set_key wrappers for the affected
> ciphers which do precisely that and which adhere to the
> nettle_set_key_func interface (note that des_set_key and des3_set_key
> don't have a key size argument so they need wrappers also for that
> reason). If you want it to be configurable, things get a bit more
> complicated and you may need your own struct perl_cipher to extend
> struct nettle_cipher (you could still enumerate the available
> nettle_cipher and convert each to a corresponding perl_cipher). Or you
> could just define separate classes with and without weak key checking.

this is quite a bit of code duplication across bindings.  I'd rather
just expose the fact of a weak key to the caller directly (whether
through exceptions, return codes, or some other mechanism.

 -----

Here's a proposal for (2) which i'll name "2a";  I believe it does
involve an ABI+API bump to libnettle, but should allow for a reduction
in the amount of code for all bindings (which in turn might make the
creation of future bindings more likely, thereby getting the nettle
goodness out to more people).  I know i'd be more likely to maintain
additional bindings if they are smaller/simpler.

redefine nettle_set_key_func to return an int instead of a void:

typedef int nettle_set_key_func(void *ctx,
				unsigned length,
				const uint8_t *key);

For the ciphers which have no weak keys, create wrapper functions around
their set_key functions which always return 1, and use those wrapper
functions to populate the standard nettle_cipher objects.

Add a wrapper function around des_set_key and des3_set_key that includes
a key length argument; add corresponding nettle_cipher objects for des
and des3.

Add new nettle_cipher objects for the remaining weak-key ciphers (only
blowfish?) without the wrapping functions.

 -----

And here is "2b", a more involved proposal for (2) -- it's a bigger
ABI+API change, but the exposed API becomes more normalized:

Redefine nettle_set_key_func as in "2a"; and also change all the
*_set_key() functions in nettle to return an int directly.  ciphers with
no weak keys will naturally always return 1.

Change des_set_key() and des3_set_key() to take length arguments like
every other *_set_key() function.

Add new nettle_cipher objects for all missing ciphers.

 -----

I understand the natural reluctance to make an ABI bump, and i think
it's good to do so carefully (and i regret that i didn't make this
proposal before the recent ABI bump to get it all done together).  But i
think the tradeoff in terms of simplicity of new bindings is an overall
positive one.

In either proposal, bindings still retain the ability to report weak
keys using language-specific mechanisms/error handling.

I'd be happy to write a patch for either 2a or 2b, if there was a chance
that they would be accepted upstream.  Either one would make me happy
(and more willing to step up to writing python bindings, which i'd like
to have on my plate for the future).

Regards,

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110329/a45f4c7b/attachment.pgp 

From nisse at lysator.liu.se  Tue Mar 29 21:38:31 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 29 Mar 2011 21:38:31 +0200
Subject: struct nettle_cipher for weak-key ciphers
In-Reply-To: <4D91FF1D.9080408@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Tue, 29 Mar 2011 11:47:41 -0400")
References: <4D9179B6.4060702@fifthhorseman.net>
	<nnpqpa2tbo.fsf@stalhein.lysator.liu.se>
	<4D91FF1D.9080408@fifthhorseman.net>
Message-ID: <nn39m53efc.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> this is quite a bit of code duplication across bindings.

Could you be a bit more concrete? Which variant of wrappers and weak-key
interface are you thinking about?

Before getting into specifics, I'd like to point out that the structs
declared in nettle-meta.h are not intended as a fully general algorithm
framework (to do that, one could, e.g., implement an interface on the
same level of abstraction as libgcrypt's, on top of nettle).

Besides missing des and blowfish, it doesn't provide a nettle_cipher
struct for every possible key size for all algorithms, and there's also
no mechanism to query possible key sizes. To me, the lack of a struct
nettle_cipher for des and blowfish-128 is comparable to the lack of
struct nettle_cipher for cast, arcfour and blowfish with 80 bit key. And
to support them all in a reasonable way would require something quite
different from the current struct nettle_cipher.

nettle-meta.h is intended to provide the simplest algorithm abstraction
possible, to cover the simple cases. And also as inspiration for
extended frameworks, either more general or more application specific.

This works better for hash algorithm (more regular properties than for
the ciphers), and in the Pike bindings, I use nettle_hash as is, but I
don't use nettle_cipher.

> Here's a proposal for (2) which i'll name "2a";  I believe it does
> involve an ABI+API bump to libnettle, but should allow for a reduction
> in the amount of code for all bindings (which in turn might make the
> creation of future bindings more likely, thereby getting the nettle
> goodness out to more people).  I know i'd be more likely to maintain
> additional bindings if they are smaller/simpler.
>
> redefine nettle_set_key_func to return an int instead of a void:
>
> typedef int nettle_set_key_func(void *ctx,
> 				unsigned length,
> 				const uint8_t *key);
>
> For the ciphers which have no weak keys, create wrapper functions around
> their set_key functions which always return 1, and use those wrapper
> functions to populate the standard nettle_cipher objects.
>
> Add a wrapper function around des_set_key and des3_set_key that includes
> a key length argument; add corresponding nettle_cipher objects for des
> and des3.

I could consider this, but I'm not convinced that it really solves an
important problem. To me, a language-specific wrapper like, e.g.,

  void
  pike_blowfish_set_key(void *ctx,
  		        ptrdiff_t length, const char *key,
  		        int force)
  {
    if (length < BLOWFISH_MIN_KEY_SIZE || length > BLOWFISH_MAX_KEY_SIZE)
      Pike_error("BLOWFISH_Info: Bad keysize for BLOWFISH.\n");
    if (!blowfish_set_key(ctx, length, (const uint8_t *)key) && !force)
      Pike_error("BLOWFISH_Info: Key is weak.\n");
  }

seems more useful than a language agnostic wrapper

  int 
  aes_set_encrypt_key_wrapper (struct aes_ctx *ctx,
  			       unsigned length, const uint8_t *key)
  {
    aes_set_encrypt_key (ctx, length, key);
    return 1;
  }

which lacks adequate error handling for bad key sizes. And if we extend
nettle_cipher to include a description of valid key sizes, and/or
introduce additional error codes for the set_key functions to signal
different types of bad keys, then we get quite far from the current
minimalistic flavor of nettle.

> And here is "2b", a more involved proposal for (2) -- it's a bigger
> ABI+API change, but the exposed API becomes more normalized:

I'm not going to do this. The low level cipher interface is not intended
to normalize away important differences between ciphers.

In particular, 

* I don't like dummy return values (which are either unnecessarily
  checked, making code ugly, or get people into the habit if ignoring
  return values). "Can't fail" (except by abort()) is a very simplifying
  property of a function, and then it shouldn't return an error code.

* I also don't like dummy function arguments, used like

  int des_set_key (struct des_ctx *ctx, 
		   unsigned length, const uint8_t *key)
  {
    assert (length == DES_KEY_SIZE);
    ...
  }

  (except in an optional wrapper function, where the above would be the
  right thing).

Remember that the C interface is intended to be nice also for
applications that use just one or two algorithms. These applications
should not suffer from cruft intended to unify the interface with some
other algorithm which the applications couldn't care less about. An
important case would be applications that use only the newer algorithms
like aes or camellia. They shouldn't have to bother about a return value
introduced just because it's needed for des.

BTW: There's another easy alternative which we could call (3): Keep
nettle_set_key_func as is. Introduce wrappers for des and blowfish which
just ignore weak keys, and provide nettle_cipher structs using these
wrappers. Then you get des and blowfish sans weak key detection, using
the same interface as the other ciphers. I think this would fit
reasonably with the nettle design principles. Question is: Would anybody
find it useful? For a general language binding, I would expect that one
would want to have the possibility to detect weak keys.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Thu Mar 31 17:34:32 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Thu, 31 Mar 2011 11:34:32 -0400
Subject: struct nettle_cipher for weak-key ciphers
In-Reply-To: <nn39m53efc.fsf@stalhein.lysator.liu.se>
References: <4D9179B6.4060702@fifthhorseman.net>	<nnpqpa2tbo.fsf@stalhein.lysator.liu.se>	<4D91FF1D.9080408@fifthhorseman.net>
	<nn39m53efc.fsf@stalhein.lysator.liu.se>
Message-ID: <4D949F08.9060904@fifthhorseman.net>

On 03/29/2011 03:38 PM, Niels M?ller wrote:
> Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:
> 
>> this is quite a bit of code duplication across bindings.
> 
> Could you be a bit more concrete? Which variant of wrappers and weak-key
> interface are you thinking about?

sorry -- i meant that it seemed like it would be unnecessary code
duplication for me to create a perl_nettle_cipher struct that matched
your pike_nettle_cipher.

> Before getting into specifics, I'd like to point out that the structs
> declared in nettle-meta.h are not intended as a fully general algorithm
> framework (to do that, one could, e.g., implement an interface on the
> same level of abstraction as libgcrypt's, on top of nettle).

hm, ok, i see your point.  it sounds like nettle_cipher might be a
reasonable choice for symmetric encryption (where the tools can select
reasonable key sizes, etc), but *not* a reasonable choice for symmetric
decryption (where we have to cope with arbitrary algorithms and keys
foisted on us by the incoming ciphertext).

> This works better for hash algorithm (more regular properties than for
> the ciphers), and in the Pike bindings, I use nettle_hash as is, but I
> don't use nettle_cipher.

Perhaps what i'm wondering is: can we define an cipher abstraction that
exposes the relevant details in C, provides a framework that is suitable
for symmetric decryption, and doesn't violate the minimalistic flavor of
nettle?

I think having something like this in the canonical sources (instead of
implemented outside of nettle) would make it easier to write and
maintain language bindings.

Or, as you said about your Pike bindings: "the implementation is maybe a
bit too complicated for its own good" -- wouldn't it be better to have
the extra implementation complexity in only one place instead of
expecting every binding that uses it to duplicate it?

> I could consider this, but I'm not convinced that it really solves an
> important problem. To me, a language-specific wrapper like, e.g.,
> 
>   void
>   pike_blowfish_set_key(void *ctx,
>   		        ptrdiff_t length, const char *key,
>   		        int force)
>   {
>     if (length < BLOWFISH_MIN_KEY_SIZE || length > BLOWFISH_MAX_KEY_SIZE)
>       Pike_error("BLOWFISH_Info: Bad keysize for BLOWFISH.\n");
>     if (!blowfish_set_key(ctx, length, (const uint8_t *)key) && !force)
>       Pike_error("BLOWFISH_Info: Key is weak.\n");
>   }
> 
> seems more useful than a language agnostic wrapper
> 
>   int 
>   aes_set_encrypt_key_wrapper (struct aes_ctx *ctx,
>   			       unsigned length, const uint8_t *key)
>   {
>     aes_set_encrypt_key (ctx, length, key);
>     return 1;
>   }

i think you might be comparing apples and oranges here.  I agree that
language-specific error handling is useful.  I'd expect any bindings to
report errors in some sort of native form.  But that doesn't mean that
we shouldn't have a cipher-agnostic interface in C that is capable of
reporting all the standard classes of errors.  But it would be nice for
the authors of the bindings to have a standard way (in C) to get error
reports that is cipher-agnostic.  Otherwise, we'll end up with a bunch
of duplicate cipher-specific code in each binding.

Maybe it's useful to think through what possible errors could come up
from a *_set_key function, and come up with a C interface that would
cover them all in some sort of distinguishable fashion?

So far, i think the errors i've heard are:

 * bad key size

 * weak key

Anything else?  Do we want a way to report the range of acceptable key
sizes for a given cipher?

> which lacks adequate error handling for bad key sizes. And if we extend
> nettle_cipher to include a description of valid key sizes, and/or
> introduce additional error codes for the set_key functions to signal
> different types of bad keys, then we get quite far from the current
> minimalistic flavor of nettle.
> 
>> And here is "2b", a more involved proposal for (2) -- it's a bigger
>> ABI+API change, but the exposed API becomes more normalized:
> 
> I'm not going to do this. The low level cipher interface is not intended
> to normalize away important differences between ciphers.

That seems reasonable to me.  I'm happy to discard proposal 2b.

> BTW: There's another easy alternative which we could call (3): Keep
> nettle_set_key_func as is. Introduce wrappers for des and blowfish which
> just ignore weak keys, and provide nettle_cipher structs using these
> wrappers. Then you get des and blowfish sans weak key detection, using
> the same interface as the other ciphers. I think this would fit
> reasonably with the nettle design principles. Question is: Would anybody
> find it useful? For a general language binding, I would expect that one
> would want to have the possibility to detect weak keys.

Yes, i agree that general language bindings should allow the user to
detect weak keys.

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
Url : http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110331/c39d1e13/attachment.pgp 

From nisse at lysator.liu.se  Fri Apr  1 11:28:58 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 01 Apr 2011 11:28:58 +0200
Subject: struct nettle_cipher for weak-key ciphers
In-Reply-To: <4D949F08.9060904@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Thu, 31 Mar 2011 11:34:32 -0400")
References: <4D9179B6.4060702@fifthhorseman.net>
	<nnpqpa2tbo.fsf@stalhein.lysator.liu.se>
	<4D91FF1D.9080408@fifthhorseman.net>
	<nn39m53efc.fsf@stalhein.lysator.liu.se>
	<4D949F08.9060904@fifthhorseman.net>
Message-ID: <nnipuy1fs5.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> can we define an cipher abstraction that exposes the relevant details
> in C, provides a framework that is suitable for symmetric decryption,
> and doesn't violate the minimalistic flavor of nettle?

Feel free to try...

Some points to keep in mind:

1. I think the current nettle_cipher is useful for some applications,
   even if it's not suitable for language bindings which want to provide
   access to everything available.

2. Error checking at set_key

> So far, i think the errors i've heard are:
>
>  * bad key size
>
>  * weak key
>
> Anything else?

One could possibly add parity error for DES, but I doubt anyone really
cares about that. One *might* want to make some distinction between
"hard" errors, like bad key sizes, and "soft" errors or warnings like
weak keys, where the resulting cipher context still can be used, if
desired.

3. The set of supported key sizes.

This is the issue with most potential for making things hairy (and this
information is not provided by nettle's cipher specific interfaces
either; there's a define for a single "recommended" key size, and when
applicable there are defines for the minimum and maximum key size. But
the user has to look in the documentation to find out which key sizes in
this range are actually supported).

> Do we want a way to report the range of acceptable key sizes for a
> given cipher?

I think a general algorithm abstraction ought to provide that. And I
think the model for this more general abstraction should be that ciphers
are parametrized by key size, which is different from current
nettle_cipher (and the design of your perl bindings) which is unaware
that the two ciphers aes-128 and aes-256 are in fact related.

There are two types of queries one could support:

 a) The ability to ask if a particular key size is ok.

 b) The ability to get the complete set of available key sizes.


From nmav at gnutls.org  Sun Apr  3 01:42:10 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sun, 03 Apr 2011 01:42:10 +0200
Subject: different lib directories for gnutls and nettle
In-Reply-To: <437hbgql64.wl%bjg@gnu.org>
References: <437hbgql64.wl%bjg@gnu.org>
Message-ID: <4D97B452.9080100@gnutls.org>

On 03/31/2011 12:48 AM, Brian Gough wrote:
> Hi, now that GNU TLS is using nettle I notice there is a difference
> between the --libdir for the two packages on 64 bit.
> Nettle installs to $prefix/lib64/ and GNU TLS looks for it in
> $prefix/lib/ -- so it fails and the standard configure/install to a
> prefix doesn't work.

Indeed it seems nettle for some reason uses the /usr/lib64. Niels,
is there really a reason for that? As I understand from the FHS[0]
/usr/lib is the system library directory. /usr/lib64 and /usr/lib32
might not even be there.

regards,
Nikos

[0].
http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA

From nisse at lysator.liu.se  Tue Apr 12 11:27:42 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 12 Apr 2011 11:27:42 +0200
Subject: different lib directories for gnutls and nettle
In-Reply-To: <4D97B452.9080100@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sun, 03 Apr 2011 01:42:10 +0200")
References: <437hbgql64.wl%bjg@gnu.org> <4D97B452.9080100@gnutls.org>
Message-ID: <nnd3kru8dt.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> Indeed it seems nettle for some reason uses the /usr/lib64. Niels,
> is there really a reason for that? As I understand from the FHS[0]
> /usr/lib is the system library directory. /usr/lib64 and /usr/lib32
> might not even be there.

Sorry for the late reply. A bit down in the same document,
http://www.pathname.com/fhs/pub/fhs-2.3.html, it says

  /lib64 and /lib32 : 64/32-bit libraries (architecture dependent)
  
  The 64-bit architectures PPC64, s390x, sparc64 and AMD64 must place
  64-bit libraries in /lib64, and 32-bit (or 31-bit on s390) libraries in
  /lib.

If you believe this, then it's wrong to install 64-bit libraries in lib.
And I believed so when I wrote this part of the nettle configure.ac...

But then it turned out that at least debian on x86_64 ignores this part
of the FHS, and puts the 64-bit libraries in lib, and 32-bit libraries
in lib32. Which seems to also be the freebsd way of doing it.

In the debian case, lib64 is a symlink to lib, so it doesn't matter for
a 64-bit builds (they will be installed in lib64, which is a symlink to
lib, which is tye right place), while a build with ./configure CC="gcc
-m32" && make && make install will install the 32-bit library files in
the wrong place.

I'm not sure what the right thing is. Is there any distribution which
actuallly does what the FHS says? If so, I guess configure needs to
check what directories exists, and install in lib$ABI whenever that
directory exists, otherwise in lib. I haven't got around to fixing that.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Tue Apr 12 16:48:21 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 12 Apr 2011 16:48:21 +0200
Subject: different lib directories for gnutls and nettle
In-Reply-To: <43y63fppdz.wl%bjg@gnu.org> (Brian Gough's message of "Tue, 12
	Apr 2011 14:31:36 +0100")
References: <437hbgql64.wl%bjg@gnu.org> <4D97B452.9080100@gnutls.org>
	<nnd3kru8dt.fsf@stalhein.lysator.liu.se> <43y63fppdz.wl%bjg@gnu.org>
Message-ID: <nn8vvfttje.fsf@stalhein.lysator.liu.se>

Brian Gough <bjg at gnu.org> writes:

> I don't know what is the correct interpretation of the FHS, but by
> default all other autoconf'ed and GNU packages always use
> libdir=$prefix/lib (with the --libdir option allowing it to be
> overridden if someone wants to use lib64 or whatever).

And the default is broken for multi-abi systems, since it means that a
plain ./configure && make && make install may well overwrite a working
library in /usr/local/lib with a library for a different ABI. With
autoconf defaults, this will happen, e.g., if you compile on Solaris (or
on some gnu/linux siystem which actually obeys the FHS), using a
compiler which by default generates 64-bit code).

Now, most packages are not aware of what ABI they are being compiled
for, but nettle is. It has to figure it out, in order to select the
right assembly files.

> The distributions already have their own ways of using --libdir on
> autoconf'ed packages so you don't need to worry about it yourself.

I don't worry so much for distributors. To use a sane libdir default is
intended to help people who compile nettle from source themselves,
rather than installing their distribution's nettle package.

I'd be happy to have autoconf solve this problem for me, but currently
it doesn't. Library installation is a bit complex, with various
workarounds. Another issue not currently solved by autoconf (nor by
nettle's build system) is w*ndows dlls, which should usually be
installed in bin rather than lib. I've heard that automake (or maybe it
was libtool) uses the workaround to install dlls in $libdir/../bin.
Which is really really ugly, but which appears to work most of the time.
Autoconf simply doesn't solve everything.

Finally: If the current hack doesn't work, I'd appreciate a complete bug
report. What system is it? Which library directories exists (incluing
symlinks)? Which directory is expected to hold libraries for which ABI?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Thu Apr 14 10:25:48 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 14 Apr 2011 10:25:48 +0200
Subject: different lib directories for gnutls and nettle
In-Reply-To: <438vvdpt35.wl%bjg@gnu.org> (Brian Gough's message of "Thu, 14
	Apr 2011 01:36:14 +0100")
References: <437hbgql64.wl%bjg@gnu.org> <4D97B452.9080100@gnutls.org>
	<nnd3kru8dt.fsf@stalhein.lysator.liu.se> <43y63fppdz.wl%bjg@gnu.org>
	<nn8vvfttje.fsf@stalhein.lysator.liu.se> <438vvdpt35.wl%bjg@gnu.org>
Message-ID: <nnsjtls0hf.fsf@stalhein.lysator.liu.se>

Brian Gough <bjg at gnu.org> writes:

> Nettle should work better with other GNU packages if it follows the
> GNU standards, as the assumption of lib/ as a default is a common one
> in other configure scripts.  

I don't change the default lightly, but I still think it's the right
thing to do when

1. The user has not provided --libdir explicitly.

2. One is building on a multi-abi system, which is a case where the
   autoconf default doesn't even try to do the right thing.

3. The autoconf default is known to be wrong.

> For comparison the GMP library has ABI detection via configure but
> keeps the library directory as lib/ by default.

I'm reasonably familiar with the gmp configure script. It's true that it
currently always uses lib/ as the default, regardless of ABI, but it's
ABI related hacks can also cause other nasty surprises. So it's a good
example to exhibit the complexity of the problem, but it's not currently
solving it.

As for linux on x86_64, it appears that debian (and derivatives, I
imagine) put 64-bit libraries in lib/, while the distributions with
roots in redhat (at least rhel 5 and fedora 14) and gentoo follow the
fhs conventions and put 32-bit libraries in lib/. So to do the right
thing, configure has to distinguish between these two cases. What a mess
(but just the type of mess configure.ac is intended to help sort out).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Thu Apr 14 10:55:33 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Thu, 14 Apr 2011 10:55:33 +0200
Subject: different lib directories for gnutls and nettle
In-Reply-To: <nnsjtls0hf.fsf@stalhein.lysator.liu.se>
References: <437hbgql64.wl%bjg@gnu.org> <4D97B452.9080100@gnutls.org>
	<nnd3kru8dt.fsf@stalhein.lysator.liu.se>
	<43y63fppdz.wl%bjg@gnu.org>
	<nn8vvfttje.fsf@stalhein.lysator.liu.se>
	<438vvdpt35.wl%bjg@gnu.org>
	<nnsjtls0hf.fsf@stalhein.lysator.liu.se>
Message-ID: <BANLkTi=aKxk9PP68Xr3DGsaa6+C0UjOCSA@mail.gmail.com>

On Thu, Apr 14, 2011 at 10:25 AM, Niels M?ller <nisse at lysator.liu.se> wrote:
>> Nettle should work better with other GNU packages if it follows the
>> GNU standards, as the assumption of lib/ as a default is a common one
>> in other configure scripts.
> I don't change the default lightly, but I still think it's the right
> thing to do when
> 1. The user has not provided --libdir explicitly.
> 2. One is building on a multi-abi system, which is a case where the
> ? autoconf default doesn't even try to do the right thing.
> 3. The autoconf default is known to be wrong.

I don't really find this a serious problem because libdir can be
provided by the one performing compilation and fix any issues,
anyway.

As a matter of policy though, I think the FHS way of
storing in /usr/lib 32-bit binaries, even if the default system compiler
outputs 64-bit binaries, is quite absurd, and looks like a relic from
the era that binary only programs came with 32-bit intel code only.
Systems like debian correctly for me do not follow this approach
because it has no benefit for the user of a multi-lib system and only
causes confusion, as programs in /usr/bin do not use libraries in
/usr/lib. What you call a multi-lib system
is actually a system with a native word size and a compatibility
mode with another (smaller) word size. Why one not using
the compatibility mode have an empty /usr/lib? This requirement
is intended only for the one distributing ancient 32-bit binaries and
I see no compelling reason for free software or open systems to
follow that by default.

regards,
Nikos

From nisse at lysator.liu.se  Fri Apr 15 21:13:29 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 15 Apr 2011 21:13:29 +0200
Subject: Serpent
In-Reply-To: <nnwrl6lwah.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Fri, 11 Feb 2011 11:07:02 +0100")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
Message-ID: <nny63bqqee.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> I'm puzzled, because I'm fairly sure I got the test vectors from
> serpent's submission package (I could try to double check that), which
> if I understand correctly ought to be serpent-1. I vaguely remember I
> had some difficulty understanding the organization of the test data,
> though.

The files defining these testvectors use an input and output format with
bytes in the reverse order. At the time (2001-06-17, almost ten years
ago), I thought this reversal was part of the cipher operation, rather
than a peculiarity with test programs and test vectors in the serpent
submission package, and incorrectly added byte reversal to the functions
in serpent.c.

I've now reverted this old change, which means that from now on serpent
in Nettle ought to agree with other implementations. And this naturally
also breaks compatibility for anyone using the serpent implementation of
all (or almost all) earlier versions of Nettle.

I've fixed the serpent-test.c to reverse inputs and outputs for the
testvectors copied from the serpent submission package, and added a few
additional test vectors (from
http://www.cs.technion.ac.il/~biham/Reports/Serpent/) using a sane byte
order.

Credits go to Eli Biham and Lars Knudsen for finally putting me on the
right track.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Fri Apr 15 21:50:34 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Fri, 15 Apr 2011 21:50:34 +0200
Subject: Serpent
In-Reply-To: <nny63bqqee.fsf@stalhein.lysator.liu.se>
References: <87lj4jvj9o.fsf@latte.josefsson.org>	<nntyj7rp85.fsf@stalhein.lysator.liu.se>	<nnsjynogry.fsf@stalhein.lysator.liu.se>	<nnoc9boglx.fsf@stalhein.lysator.liu.se>	<87ei9tu5fu.fsf@latte.josefsson.org>	<nnvd35iubw.fsf@stalhein.lysator.liu.se>	<871v5lzx67.fsf@latte.josefsson.org>	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>	<87lj1nhf3j.fsf@latte.josefsson.org>	<8739nveg2k.fsf@latte.josefsson.org>	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>	<87oc6j6o55.fsf_-_@latte.josefsson.org>	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
	<nny63bqqee.fsf@stalhein.lysator.liu.se>
Message-ID: <4DA8A18A.4030908@gnutls.org>

On 04/15/2011 09:13 PM, Niels M?ller wrote:

[...]
> I've now reverted this old change, which means that from now on serpent
> in Nettle ought to agree with other implementations. And this naturally
> also breaks compatibility for anyone using the serpent implementation of
> all (or almost all) earlier versions of Nettle.

Would this mean that nettle is now compatible with the libgcrypt version
of serpent? In that case would the LGPL libgcrypt code be used instead?

regards,
Nikos

From nisse at lysator.liu.se  Sat Apr 16 08:51:41 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 16 Apr 2011 08:51:41 +0200
Subject: Serpent
In-Reply-To: <4DA8A18A.4030908@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Fri, 15 Apr 2011 21:50:34 +0200")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
	<nny63bqqee.fsf@stalhein.lysator.liu.se> <4DA8A18A.4030908@gnutls.org>
Message-ID: <nntydyr8n6.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> Would this mean that nettle is now compatible with the libgcrypt version
> of serpent?

That's the idea.

> In that case would the LGPL libgcrypt code be used instead?

Simon said he was working on that. I don't know current status, so I
figured it was good to fix the old Nettle code too (it turned out to be
easy).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Sat Apr 16 10:39:19 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Sat, 16 Apr 2011 10:39:19 +0200
Subject: Serpent
In-Reply-To: <nntydyr8n6.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Sat, 16 Apr 2011 08:51:41 +0200")
References: <87lj4jvj9o.fsf@latte.josefsson.org>
	<nntyj7rp85.fsf@stalhein.lysator.liu.se>
	<nnsjynogry.fsf@stalhein.lysator.liu.se>
	<nnoc9boglx.fsf@stalhein.lysator.liu.se>
	<87ei9tu5fu.fsf@latte.josefsson.org>
	<nnvd35iubw.fsf@stalhein.lysator.liu.se>
	<871v5lzx67.fsf@latte.josefsson.org>
	<nnhbcbnaef.fsf@stalhein.lysator.liu.se>
	<87lj1nhf3j.fsf@latte.josefsson.org>
	<8739nveg2k.fsf@latte.josefsson.org>
	<nnaai3mq9u.fsf@stalhein.lysator.liu.se>
	<87oc6j6o55.fsf_-_@latte.josefsson.org>
	<nnwrl6lwah.fsf@stalhein.lysator.liu.se>
	<nny63bqqee.fsf@stalhein.lysator.liu.se> <4DA8A18A.4030908@gnutls.org>
	<nntydyr8n6.fsf@stalhein.lysator.liu.se>
Message-ID: <87k4eu3808.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
>
>> Would this mean that nettle is now compatible with the libgcrypt version
>> of serpent?
>
> That's the idea.
>
>> In that case would the LGPL libgcrypt code be used instead?
>
> Simon said he was working on that. I don't know current status, so I
> figured it was good to fix the old Nettle code too (it turned out to be
> easy).

I'll resume my work -- I recall that I got it to a stage where it built
and generated ciphertext, however it didn't match the test vectors that
were in Nettle back then.  Hopefully it will match now, and I can submit
the patch shortly.

/Simon

From nisse at lysator.liu.se  Fri May  6 14:10:23 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 06 May 2011 14:10:23 +0200
Subject: Fwd: Problem with nettle 2.1 and gnutls 2.12.3 on Solaris
In-Reply-To: <7C623346-4D33-43A3-849A-1CF016C0B6F4@baltic-online.de> (Dagobert
	Michelsen's message of "Thu, 5 May 2011 14:38:39 +0200")
References: <mailman.2520.1304598929.7710.nettle-bugs@lists.lysator.liu.se>
	<7C623346-4D33-43A3-849A-1CF016C0B6F4@baltic-online.de>
Message-ID: <nn4o58kp1s.fsf@stalhein.lysator.liu.se>

Dagobert Michelsen <dam at baltic-online.de> writes:

>> I am trying to compile gnutls 2.12.3 with libnettle 2.1 and get the
>> following errors. From the output I assume an incompatibility between
>> different gnulib inclusions.

For the record, nettle does not use gnulib. It uses AX_CREATE_STDINT_H,
iirc I copied that from autoconf-archive, long ago.

>> "/opt/csw/include/nettle/nettle-stdint.h", line 237: identifier redeclared: gl_int_fast8_t
>>        current : signed char
>>        previous: long : "./../gl/stdint.h", line 241

This is puzzling. On Solaris, nettle-stdint.h is expected to contain

  /* generated using gnu compiler gcc (GCC) 3.4.5 */
  #define _STDINT_HAVE_STDINT_H 1
  
  /* ................... shortcircuit part ........................... */
  
  #if defined HAVE_STDINT_H || defined _STDINT_HAVE_STDINT_H
  #include <stdint.h>
  #else
  ... lots of irrelevant stuff...
  #endif

And there seems to be a /usr/include/stdint.h on the system I look, so I
don't think it's a gcc-specific include file. What does your
nettle-stdin.h look like? If it is different, something went wrong at
configure time when nettle was installed..

And what is "./../gl/stdint.h", that's from gnulib? nettle-stdint.h does
not refer gl_int_fast8_t as far as I can see, so I guess there's a
#define int_fast8_t gl_int_fast8_t somewhere.

The failures seem to all be related to the various *_fast_*_t type,
where the spec leaves a lot of room for the implementation. So if
different packages defines these them selves, it's not very surprising
that they might have different ideas about what they should be. But in
the Solaris case, the right thing is most likely to use the definitions
from /usr/include/stdint.h, and *no* others. I haven't looked at gnulib,
but I *hope* it has some short-circuit code like in nettle-stdint.h, to
fall back to the system's stdint.h or inttypes.h if one of them exists
and works.

/Niels

PS. Sorry the nettle-bugs list is members only, if you want to reply and
are not on that list, please use my personal address instead of
nettle-bugs at lists.lysator.liu.se.

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Sat May 21 10:08:06 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sat, 21 May 2011 10:08:06 +0200
Subject: ECC
Message-ID: <4DD772E6.1010305@gnutls.org>

I'm working towards adding ECDH and ECDSA support to gnutls.
I've currently ported libtomcrypt to use gmp directly and nettle's
types. The code is under development, and can be found at:
http://git.savannah.gnu.org/gitweb/?p=gnutls.git;a=tree;f=lib/nettle;h=23d9eaa265f911825b09ee7caa4c20525fc9ff2f;hb=HEAD

The ECC functionality is confined to the ecc* files. It would be nice if
nettle included it or something similar.

btw. Is there any plan on an 2.2 release?

regards,
Nikos

From nmav at gnutls.org  Sat May 21 10:08:10 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sat, 21 May 2011 10:08:10 +0200
Subject: memory leak fix
Message-ID: <4DD772EA.9060509@gnutls.org>

The attached patch fixes a memory leak in DSA key generation.

regards,
Nikos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Fixed-memory-leak-in-DSA-key-generation.patch
Type: text/x-patch
Size: 670 bytes
Desc: not available
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110521/3bfa6ced/attachment.bin>

From nisse at lysator.liu.se  Sat May 21 12:52:17 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 21 May 2011 12:52:17 +0200
Subject: memory leak fix
In-Reply-To: <4DD772EA.9060509@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sat, 21 May 2011 10:08:10 +0200")
References: <4DD772EA.9060509@gnutls.org>
Message-ID: <nn7h9ke36m.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> The attached patch fixes a memory leak in DSA key generation.

Thanks, checked in now.

/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Sat May 21 13:04:36 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 21 May 2011 13:04:36 +0200
Subject: ECC
In-Reply-To: <4DD772E6.1010305@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sat, 21 May 2011 10:08:06 +0200")
References: <4DD772E6.1010305@gnutls.org>
Message-ID: <nn39k8e2m3.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> I'm working towards adding ECDH and ECDSA support to gnutls.

I don't have time at the moment to read the code. Could you summarize
which curves/algorithms/specifications you're implementing?

> The ECC functionality is confined to the ecc* files. It would be nice if
> nettle included it or something similar.

Contributions are welcome ;-) We'll see if I get any time to look into
it myself.

> btw. Is there any plan on an 2.2 release?

I guess I should put up a release TODO list on the web. Issues I'm aware
of and which I think should be completed before the next release are:

 * Fix the libdir defaults for linux.

 * Port libgrypt's LGPL:ed serpent implementation to nettle.

 * Overall license change to LGPL. Still not decided if it should be
   LGPLv2+, LGPLv3+, or dual license GPLv2+ and LGPLv3+ (in this
   notation, "+" means "or any later version, of the user's choice").

If it's going to take a long time to sort these things out, I could make
a relase sooner and postpone some or all of the above to a later
release. Main new feature since the previous release is the gcm support,
I think.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Sat May 21 13:43:02 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sat, 21 May 2011 13:43:02 +0200
Subject: ECC
In-Reply-To: <nn39k8e2m3.fsf@stalhein.lysator.liu.se>
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se>
Message-ID: <4DD7A546.8000808@gnutls.org>

On 05/21/2011 01:04 PM, Niels M?ller wrote:

>> I'm working towards adding ECDH and ECDSA support to gnutls.
> I don't have time at the moment to read the code. Could you
> summarize which curves/algorithms/specifications you're
> implementing?

For gnutls I care only about the SECG curves (SECP224R1, SECP256R1,
SECP384R1, SECP521R1) and with those I have tested the ECDH support.
The code was fixed for the curves y^2=x^3-3x+b. I tried to make it
apply to generic y^2=x^3+ax+b, but I haven't test that functionality
(nor plan to) because all of the above curves are of the a=-3 format.

libtomcrypt's code style is similar to nettle's thus the changes
required to port this to nettle would not be much, but always depends
on what you want to include there.

> * Port libgrypt's LGPL:ed serpent implementation to nettle. * Overall
> license change to LGPL. Still not decided if it should be LGPLv2+,
> LGPLv3+, or dual license GPLv2+ and LGPLv3+ (in this notation, "+"
> means "or any later version, of the user's choice").

May I suggest an intermediate release with GCM under LGPLv2+?
That would ensure that current applications using gnutls can use that
code.

regards,
Nikos

From nisse at lysator.liu.se  Mon May 30 10:36:50 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 30 May 2011 10:36:50 +0200
Subject: ECC
In-Reply-To: <4DD7A546.8000808@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sat, 21 May 2011 13:43:02 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
Message-ID: <nn8vtoa8kd.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> May I suggest an intermediate release with GCM under LGPLv2+?
> That would ensure that current applications using gnutls can use that
> code.

An all-LGPL release is not possible until the serpent code is replaced,
so any releases before that would use the current licensing terms (GPL
as a whole, certain parts under more permissive license).

So I'm not sure exactly what you are suggesting.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Mon May 30 18:39:49 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 30 May 2011 18:39:49 +0200
Subject: ECC
In-Reply-To: <nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
References: <4DD772E6.1010305@gnutls.org>	<nn39k8e2m3.fsf@stalhein.lysator.liu.se>
	<4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
Message-ID: <4DE3C855.9060906@gnutls.org>

On 05/30/2011 10:36 AM, Niels M?ller wrote:
> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
> 
>> May I suggest an intermediate release with GCM under LGPLv2+?
>> That would ensure that current applications using gnutls can use that
>> code.
> An all-LGPL release is not possible until the serpent code is replaced,
> so any releases before that would use the current licensing terms (GPL
> as a whole, certain parts under more permissive license).
> So I'm not sure exactly what you are suggesting.

I thought the GPL part applies only to the parts that are GPL and not to
the whole library. In any case would you suggest gnutls using nettle a
la lsh, i.e. include it as a static library? That way we can eliminate
licensing issues by removing the GPL parts.

regards,
Nikos

From nisse at lysator.liu.se  Mon May 30 20:07:59 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 30 May 2011 20:07:59 +0200
Subject: ECC
In-Reply-To: <4DE3C855.9060906@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Mon, 30 May 2011 18:39:49 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se> <4DE3C855.9060906@gnutls.org>
Message-ID: <nnzkm483k0.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> I thought the GPL part applies only to the parts that are GPL and not to
> the whole library. 

The manual says the following:

  Nettle is distributed under the GNU General Public License (GPL) (see
  the file COPYING for details). However, most of the individual files
  are dual licensed under less restrictive licenses like the GNU Lesser
  General Public License (LGPL), or are in the public domain. This means
  that if you don't use the parts of nettle that are GPL-only, you have
  the option to use the Nettle library just as if it were licensed under
  the LGPL. To find the current status of particular files, you have to
  read the copyright notices at the top of the files.

Things will get simpler once the serpent code is replaced.

> In any case would you suggest gnutls using nettle a la lsh, i.e.
> include it as a static library? That way we can eliminate licensing
> issues by removing the GPL parts.

My understanding is that as long as gnutls doesn't use the serpent code
(the only remaining GPL-only feature) and has no references to it, it's
ok no matter if you use static or dynamic linking. You can pretend that
the serpent code was never built and included into the library.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Mon May 30 22:47:16 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Mon, 30 May 2011 22:47:16 +0200
Subject: serpent again (was: Re: ECC)
In-Reply-To: <nn8vtoa8kd.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Mon, 30 May 2011 10:36:50 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
Message-ID: <87sjrvhq5n.fsf_-_@latte.josefsson.org>

I looked at reviving my serpent port from libgcrypt.  As a first step in
testing it, it would help if Nettle had the same test vector as
libgcrypt.  Please consider this patch.

/Simon

2011-05-30  Simon Josefsson  <simon at josefsson.org>

	* testsuite/serpent-test.c (test_main): Add test vectors from
	libgcrypt.

--- serpent-test.c.~1.3.~	2011-05-20 23:02:54.000000000 +0200
+++ serpent-test.c	2011-05-30 22:46:50.837183923 +0200
@@ -25,6 +25,26 @@
 int
 test_main(void)
 {
+  /* From libgcrypt */
+  test_cipher(&nettle_serpent128,
+	      HL("0000000000000000 0000000000000000"),
+	      HL("D29D576FCEA3A3A7 ED9099F29273D78E"),
+	      H("B2288B968AE8B086 48D1CE9606FD992D"));
+  test_cipher(&nettle_serpent192,
+	      HL("0000000000000000 0000000000000000 0000000000000000"),
+	      HL("D29D576FCEABA3A7 ED9899F2927BD78E"),
+	      H("130E353E1037C224 05E8FAEFB2C3C3E9"));
+  test_cipher(&nettle_serpent256,
+	      HL("0000000000000000 0000000000000000"
+		 "0000000000000000 0000000000000000"),
+	      HL("D095576FCEA3E3A7 ED98D9F29073D78E"),
+	      H("B90EE5862DE69168 F2BDD5125B45472B"));
+  test_cipher(&nettle_serpent256,
+	      HL("0000000000000000 0000000000000000"
+		 "0000000000000000 0000000000000000"),
+	      HL("0000000001000000 0200000003000000"),
+	      H("2061A42782BD52EC 691EC383B03BA77C"));
+
   /* The first test for each key size from the ecb_vk.txt and ecb_vt.txt
    * files in the serpent package. */
 
 

From nisse at lysator.liu.se  Mon May 30 23:02:31 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 30 May 2011 23:02:31 +0200
Subject: serpent again
In-Reply-To: <87sjrvhq5n.fsf_-_@latte.josefsson.org> (Simon Josefsson's
	message of "Mon, 30 May 2011 22:47:16 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
Message-ID: <nnaae39a1k.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> I looked at reviving my serpent port from libgcrypt.  As a first step in
> testing it, it would help if Nettle had the same test vector as
> libgcrypt.  Please consider this patch.

Thanks, checked in now. And the current (i.e., to-be-replaced) code
passes these tests.

/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Mon May 30 23:09:38 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Mon, 30 May 2011 23:09:38 +0200
Subject: serpent again
In-Reply-To: <87sjrvhq5n.fsf_-_@latte.josefsson.org> (Simon Josefsson's
	message of "Mon, 30 May 2011 22:47:16 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
Message-ID: <87oc2jhp4d.fsf@latte.josefsson.org>

Fixing a small bug in my 2010-12-07 port of serpent.c from libgcrypt to
nettle was all that was required to make it work.  Please consider this
work, applied as follows:

* Add ChangeLog entry as below.
* Remove serpent_sboxes.h.
* Apply patch to Makefile.in as below.
* Add attached serpent.c in place of the current code.

I have not compared the new file to the old one in Nettle, I'll leave
that to you.

/Simon

2011-05-30  Simon Josefsson  <simon at josefsson.org>

	* serpent.c: Rewrite, based on libgcrypt code.  License changed
	from GPL to LGPL.
	* serpent_sboxes.h: Removed.
	* Makefile.in: Drop serpent_sboxes.h.

Index: Makefile.in
===================================================================
RCS file: /cvsroot/lsh/lsh/nettle/Makefile.in,v
retrieving revision 1.32
diff -u -p -r1.32 Makefile.in
--- Makefile.in 23 Mar 2011 10:34:46 -0000      1.32
+++ Makefile.in 30 May 2011 21:07:01 -0000
@@ -128,7 +128,7 @@ DISTFILES = $(SOURCES) $(HEADERS) .boots
        README AUTHORS COPYING COPYING.LIB INSTALL NEWS TODO ChangeLog \
        memxor.c $(des_headers) descore.README \
        aes-internal.h camellia-internal.h cast128_sboxes.h desinfo.h desCode.h \
-       serpent_sboxes.h nettle-internal.h nettle-write.h prime-list.h \
+       nettle-internal.h nettle-write.h prime-list.h \
        asm.m4 \
        nettle.texinfo nettle.info nettle.html nettle.pdf sha-example.c
 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: serpent.c
Type: text/x-csrc
Size: 23779 bytes
Desc: not available
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110530/8ccc6db9/attachment.c>

From simon at josefsson.org  Mon May 30 23:16:57 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Mon, 30 May 2011 23:16:57 +0200
Subject: serpent again
In-Reply-To: <87oc2jhp4d.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Mon, 30 May 2011 23:09:38 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
Message-ID: <87ipsrhos6.fsf@latte.josefsson.org>

Btw, I have tested it on a core i7 (little endien) and a PPC machine
(big endian) and it worked.

(However, the 'gcm' self test crashed on PPC.)

/Simon

From nisse at lysator.liu.se  Mon May 30 23:29:45 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 30 May 2011 23:29:45 +0200
Subject: serpent again
In-Reply-To: <87oc2jhp4d.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Mon, 30 May 2011 23:09:38 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
Message-ID: <nn62or98s6.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> Fixing a small bug in my 2010-12-07 port of serpent.c from libgcrypt to
> nettle was all that was required to make it work.  Please consider this
> work, applied as follows:

Great!

I'll try to get this integrated reasonably soon. Have you compared the
performance of the old and new code?

Some minor things (which I think I can take care of myself):

> /* Serpent works on 128 bit blocks.  */
> typedef uint32_t serpent_block_t[4];
>
> /* Serpent key, provided by the user.  If the original key is shorter
>    than 256 bits, it is padded.  */
> typedef uint32_t serpent_key_t[8];

I dislike array typedefs.

> #define byte_swap_32(x) \
>   (0 \
>    | (((x) & 0xff000000) >> 24) | (((x) & 0x00ff0000) >>  8) \
>    | (((x) & 0x0000ff00) <<  8) | (((x) & 0x000000ff) << 24))

This and the endian test where it is used should be replaced by using
LE_READ_UINT32.

> /* Convert the user provided key KEY of KEY_LENGTH bytes into the
>    internally used format.  */
> static void
> serpent_key_prepare (const uint8_t * key, unsigned int key_length,
> 		     serpent_key_t key_prepared)

This function seems to assume that key is aligned on a four-byte
boundary, and that key_length is a multiple of four. The nettle
interface specifies no alignment requirement on the key. And the old
serpent code is supposed to support any key size (although unfortunately
I don't have any testcases for sizes other than 16, 24 and 32 bytes).

After this code is in, I'd like to try to do serpent with two blocks at
a time in parallel, for machines with native 64-bit registers (and change
at least the ctr code to do a couple of blocks at a time). I think that
might be about as fast as aes or camellia.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Mon May 30 23:47:53 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Mon, 30 May 2011 23:47:53 +0200
Subject: gcm crash on ppc (was: Re: serpent again)
In-Reply-To: <87ipsrhos6.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Mon, 30 May 2011 23:16:57 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<87ipsrhos6.fsf@latte.josefsson.org>
Message-ID: <87d3izhncm.fsf_-_@latte.josefsson.org>

Simon Josefsson <simon at josefsson.org> writes:

> (However, the 'gcm' self test crashed on PPC.)

Here is a backtrace.

espresso:~/nettle-2.2/testsuite jas$ gdb ./gcm-test
GNU gdb 6.3.50-20050815 (Apple version gdb-696) (Sat Oct 20 18:20:28 GMT 2007)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...
warning: --arch option not supported in this gdb.
Reading symbols for shared libraries .. done

(gdb) b main
Breakpoint 1 at 0x3ddc: file testutils.c, line 149.
(gdb) r
Starting program: /Users/jas/nettle-2.2/testsuite/gcm-test 
Reading symbols for shared libraries . done

Breakpoint 1, main (argc=1, argv=0xbffffb34) at testutils.c:149
149	  if (argc > 1)
(gdb) c
Continuing.

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00000000
0x00000000 in ?? ()
(gdb) bt
#0  0x00000000 in ?? ()
Cannot access memory at address 0x0
Cannot access memory at address 0x0
Cannot access memory at address 0x0
Cannot access memory at address 0x0
Cannot access memory at address 0x0
#1  0x000049f0 in test_aead (aead=Cannot access memory at address 0x0
Cannot access memory at address 0x0
Cannot access memory at address 0x0
Cannot access memory at address 0x0
Cannot access memory at address 0x0
Cannot access memory at address 0x0
0x80f0, key_length=16, key=0x3000f0 "", auth_length=0, authtext=0x300100 "", length=0, cleartext=0x300110 "", ciphertext=0x300120 "", iv_length=12, iv=0x300130 "", digest=0x300140 "X????~0a6\035W??EZ") at testutils.c:384
Cannot access memory at address 0x0
#2  0x000029ec in test_main () at gcm-test.c:14
Cannot access memory at address 0x0
Cannot access memory at address 0x0
Cannot access memory at address 0x0
#3  0x00003e30 in main (argc=1, argv=0xbffffb34) at testutils.c:161
(gdb) 

What is fishy is that nettle_gcm_aes128 doesn't seem to have been
initialized properly:

espresso:~/nettle-2.2/testsuite jas$ gdb ./gcm-test
GNU gdb 6.3.50-20050815 (Apple version gdb-696) (Sat Oct 20 18:20:28 GMT 2007)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...
warning: --arch option not supported in this gdb.
Reading symbols for shared libraries .. done

(gdb) p nettle_gcm_aes128
$1 = {
  name = 0x0, 
  context_size = 0, 
  block_size = 0, 
  key_size = 0, 
  set_key = 0, 
  set_iv = 0, 
  update = 0, 
  encrypt = 0, 
  decrypt = 0, 
  digest = 0
}
(gdb) 

Even stranger is that it is defined properly in other tests:

espresso:~/nettle-2.2/testsuite jas$ gdb ./blowfish-test
GNU gdb 6.3.50-20050815 (Apple version gdb-696) (Sat Oct 20 18:20:28 GMT 2007)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "powerpc-apple-darwin"...
warning: --arch option not supported in this gdb.
Reading symbols for shared libraries .. done

(gdb) p nettle_gcm_aes128
$1 = {
  name = 0xfabc "gcm-aes128", 
  context_size = 4404, 
  block_size = 16, 
  key_size = 16, 
  set_key = 0xc308 <nettle_gcm_aes_set_key>, 
  set_iv = 0xc37c <nettle_gcm_aes_set_iv>, 
  update = 0xc3d4 <nettle_gcm_aes_update>, 
  encrypt = 0xc42c <nettle_gcm_aes_encrypt>, 
  decrypt = 0xc4a8 <nettle_gcm_aes_decrypt>, 
  digest = 0xc524 <nettle_gcm_aes_digest>
}
(gdb) 

Possibly something could be broken on this machine.

/Simon


From simon at josefsson.org  Mon May 30 23:54:54 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Mon, 30 May 2011 23:54:54 +0200
Subject: serpent again
In-Reply-To: <nn62or98s6.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Mon, 30 May 2011 23:29:45 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<nn62or98s6.fsf@stalhein.lysator.liu.se>
Message-ID: <877h97hn0x.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Simon Josefsson <simon at josefsson.org> writes:
>
>> Fixing a small bug in my 2010-12-07 port of serpent.c from libgcrypt to
>> nettle was all that was required to make it work.  Please consider this
>> work, applied as follows:
>
> Great!
>
> I'll try to get this integrated reasonably soon. Have you compared the
> performance of the old and new code?

Old:
jas at latte:~/src/lsh/nettle/examples$ ./nettle-benchmark serpent
sha1_compress: 1139.20 cycles

benchmark call overhead: 0.002529 us
         Algorithm        mode Mbyte/s

        serpent256 ECB encrypt   28.69
        serpent256 ECB decrypt   30.94
        serpent256 CBC encrypt   26.99
        serpent256 CBC decrypt   30.82
jas at latte:~/src/lsh/nettle/examples$ 

New:
jas at latte:~/src/lsh/nettle/examples$ ./nettle-benchmark serpent
sha1_compress: 1140.00 cycles

benchmark call overhead: 0.002425 us
         Algorithm        mode Mbyte/s

        serpent256 ECB encrypt   14.26
        serpent256 ECB decrypt   16.34
        serpent256 CBC encrypt   13.93
        serpent256 CBC decrypt   15.93
jas at latte:~/src/lsh/nettle/examples$ 

> Some minor things (which I think I can take care of myself):
>
>> /* Serpent works on 128 bit blocks.  */
>> typedef uint32_t serpent_block_t[4];
>>
>> /* Serpent key, provided by the user.  If the original key is shorter
>>    than 256 bits, it is padded.  */
>> typedef uint32_t serpent_key_t[8];
>
> I dislike array typedefs.
>
>> #define byte_swap_32(x) \
>>   (0 \
>>    | (((x) & 0xff000000) >> 24) | (((x) & 0x00ff0000) >>  8) \
>>    | (((x) & 0x0000ff00) <<  8) | (((x) & 0x000000ff) << 24))
>
> This and the endian test where it is used should be replaced by using
> LE_READ_UINT32.

Sounds good.

>> /* Convert the user provided key KEY of KEY_LENGTH bytes into the
>>    internally used format.  */
>> static void
>> serpent_key_prepare (const uint8_t * key, unsigned int key_length,
>> 		     serpent_key_t key_prepared)
>
> This function seems to assume that key is aligned on a four-byte
> boundary, and that key_length is a multiple of four. The nettle
> interface specifies no alignment requirement on the key. And the old
> serpent code is supposed to support any key size (although unfortunately
> I don't have any testcases for sizes other than 16, 24 and 32 bytes).

Maybe throw an error for non-16/24/32 key sizes?  I'm not sure how
useful it is to support that.

> After this code is in, I'd like to try to do serpent with two blocks at
> a time in parallel, for machines with native 64-bit registers (and change
> at least the ctr code to do a couple of blocks at a time). I think that
> might be about as fast as aes or camellia.

Did the old code do that?  In any case, it looks like the performance of
serpent has a long way to go to be comparable to aes or camellia:

jas at latte:~/src/lsh/nettle/examples$ ./nettle-benchmark camellia
sha1_compress: 1139.20 cycles

benchmark call overhead: 0.002421 us
         Algorithm        mode Mbyte/s

       camellia128 ECB encrypt  146.18
       camellia128 ECB decrypt  146.37
       camellia128 CBC encrypt  127.87
       camellia128 CBC decrypt  142.89

       camellia192 ECB encrypt  110.03
       camellia192 ECB decrypt  109.84
       camellia192 CBC encrypt   98.50
       camellia192 CBC decrypt  108.31

       camellia256 ECB encrypt  109.72
       camellia256 ECB decrypt  110.03
       camellia256 CBC encrypt   98.46
       camellia256 CBC decrypt  108.29
jas at latte:~/src/lsh/nettle/examples$ ./nettle-benchmark aes
sha1_compress: 1149.60 cycles

benchmark call overhead: 0.002420 us
         Algorithm        mode Mbyte/s

            aes128 ECB encrypt  181.84
            aes128 ECB decrypt  181.58
            aes128 CBC encrypt  149.30
            aes128 CBC decrypt  178.05

            aes192 ECB encrypt  153.40
            aes192 ECB decrypt  155.35
            aes192 CBC encrypt  132.48
            aes192 CBC decrypt  153.33

            aes256 ECB encrypt  135.98
            aes256 ECB decrypt  135.14
            aes256 CBC encrypt  119.54
            aes256 CBC decrypt  134.13

    openssl aes128 ECB encrypt  203.50
    openssl aes128 ECB decrypt  190.98

    openssl aes192 ECB encrypt  171.97
    openssl aes192 ECB decrypt  162.42

    openssl aes256 ECB encrypt  149.75
    openssl aes256 ECB decrypt  142.56
jas at latte:~/src/lsh/nettle/examples$ 

/Simon

From nisse at lysator.liu.se  Tue May 31 10:06:49 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 31 May 2011 10:06:49 +0200
Subject: serpent again
In-Reply-To: <877h97hn0x.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Mon, 30 May 2011 23:54:54 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<nn62or98s6.fsf@stalhein.lysator.liu.se>
	<877h97hn0x.fsf@latte.josefsson.org>
Message-ID: <nn1uzf8fae.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> nisse at lysator.liu.se (Niels M?ller) writes:
>
>> I'll try to get this integrated reasonably soon. Have you compared the
>> performance of the old and new code?
>
> Old:
>         serpent256 ECB encrypt   28.69
>         serpent256 ECB decrypt   30.94
>         serpent256 CBC encrypt   26.99
>         serpent256 CBC decrypt   30.82

> New:
>         serpent256 ECB encrypt   14.26
>         serpent256 ECB decrypt   16.34
>         serpent256 CBC encrypt   13.93
>         serpent256 CBC decrypt   15.93

Then there's clearly some room for optimization.

> Maybe throw an error for non-16/24/32 key sizes?  I'm not sure how
> useful it is to support that.

Not terribly useful, I guess, but since it's well defined by the serpent
spec, I think it should be supported.

>> After this code is in, I'd like to try to do serpent with two blocks at
>> a time in parallel, for machines with native 64-bit registers (and change
>> at least the ctr code to do a couple of blocks at a time). I think that
>> might be about as fast as aes or camellia.
>
> Did the old code do that?

No, I didn't want to do that work with the old code.

> In any case, it looks like the performance of serpent has a long way
> to go to be comparable to aes or camellia:

On my Intel SU4100 laptop (64 bit), I get (still with the old serpent
code):

            aes256 ECB encrypt   54.72
            aes256 ECB decrypt   54.36

       camellia256 ECB encrypt   43.10
       camellia256 ECB decrypt   43.09

        serpent256 ECB encrypt   22.47
        serpent256 ECB decrypt   26.89

I would expect that the two-block-in-parallel trick can almost double
serpent performance (for ecb, ctr, cbc-decrypt, but not cbc-encrypt).
And then all three algorithms are definitely in the same ballpark.

(And of these algorithms, only aes uses handwritten x86_64 assembly
code).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Tue May 31 10:20:01 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Tue, 31 May 2011 10:20:01 +0200
Subject: serpent again
In-Reply-To: <nn1uzf8fae.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Tue, 31 May 2011 10:06:49 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<nn62or98s6.fsf@stalhein.lysator.liu.se>
	<877h97hn0x.fsf@latte.josefsson.org>
	<nn1uzf8fae.fsf@stalhein.lysator.liu.se>
Message-ID: <8739jvjn7y.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

>> Maybe throw an error for non-16/24/32 key sizes?  I'm not sure how
>> useful it is to support that.
>
> Not terribly useful, I guess, but since it's well defined by the serpent
> spec, I think it should be supported.

Which spec?  According to wikipedia

https://secure.wikimedia.org/wikipedia/en/wiki/Serpent_%28cipher%29

this is the official serpent homepage:

http://www.cl.cam.ac.uk/~rja14/serpent.html

and the link for "algorithm specification" is

http://www.cl.cam.ac.uk/~rja14/Papers/serpent.pdf

which says that 

   The user key length is variable, but for the purposes of this
   submission we fix it at 128, 192 or 256 bits

That is a bit vague, either it is variable or it isn't.

The Serpent-0 specification says

   The user key can be of any length between 64 and 256 bits.[5]

   [5] We define key lengths to be multiples of 32 bits for ease of
   implementation, and ...

Thus in any case, it seems, the key size is not completely variable and
always a multiple of 32 bits (which resolves the key alignment issue).

My impression is that Serpent is a bit confusingly specified, and it is
unclear which specification is actually the right one.  The test vector
confusion reinforced my impression here.

/Simon

From nisse at lysator.liu.se  Tue May 31 10:43:03 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 31 May 2011 10:43:03 +0200
Subject: serpent again
In-Reply-To: <8739jvjn7y.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Tue, 31 May 2011 10:20:01 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<nn62or98s6.fsf@stalhein.lysator.liu.se>
	<877h97hn0x.fsf@latte.josefsson.org>
	<nn1uzf8fae.fsf@stalhein.lysator.liu.se>
	<8739jvjn7y.fsf@latte.josefsson.org>
Message-ID: <nnsjrv6z1k.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> and the link for "algorithm specification" is
>
> http://www.cl.cam.ac.uk/~rja14/Papers/serpent.pdf

I think this is the authoritative spec, and same document which was
included in the submission package (floppy5/serpent.pdf).

>    The user key length is variable, but for the purposes of this
>    submission we fix it at 128, 192 or 256 bits
>
> That is a bit vague, either it is variable or it isn't.

I agree, this is not as definitive as I remembered it. Anyway, it
defines how short keys of any bitsize <= 256 bits should be padded, in
case anybody ever wants to use it, and gives 40-bit keys as an example.

> The Serpent-0 specification says
>
>    The user key can be of any length between 64 and 256 bits.[5]
>
>    [5] We define key lengths to be multiples of 32 bits for ease of
>    implementation, and ...

I don't think this is relevant.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Tue May 31 23:31:44 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 31 May 2011 23:31:44 +0200
Subject: serpent again
In-Reply-To: <nn62or98s6.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Mon, 30 May 2011 23:29:45 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<nn62or98s6.fsf@stalhein.lysator.liu.se>
Message-ID: <nn4o4a7e0v.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> I'll try to get this integrated reasonably soon. Have you compared the
> performance of the old and new code?

I've checked in the new code now, and I've fixed serpent_key_prepare to
use LE_READ_UINT32 (with no alignment requirements) and with support for
arbitrary key sizes up to 32 bytes.

This has been really straight forward.

On my development laptop, I get about the same performance for the new
and the old code (x86_64, gcc-4.4.5).

I'd like to get rid of the serpent_block_t typedef (I think the use of an
array may mess up register allcoation with some compilers, and that
might be the source of the large performance difference on Simon's
machine), and it's really silly to have a BLOCK_COPY in the round
function.

So I think I'll want to spend some time on reorganizing the code (and
there's also some purely stylistic things, like using uppercase for all
macros) before adding anything really new, like the two-block trick.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Wed Jun  8 11:18:46 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 08 Jun 2011 11:18:46 +0200
Subject: patch to fix build of nettle on gcc-2.95 architectures
In-Reply-To: <40b6-4def3400-7-6b8b4580@32337325> (Sebastian Reitenbach's
	message of "Wed, 08 Jun 2011 10:35:04 +0200")
References: <40b6-4def3400-7-6b8b4580@32337325>
Message-ID: <nn7h8whea1.fsf@stalhein.lysator.liu.se>

"Sebastian Reitenbach" <sebastia at l00-bugdead-prods.de> writes:

> I found nettle not compiling on openbsd sparc architecture which uses
> gcc-2.95 as default compiler. Attached little patch against nettle-2.1
> fixes the compilation problem for me.

Thanks, checked in now. Nettle is not supposed to rely on c99 features.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Wed Jun  8 11:24:44 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Wed, 8 Jun 2011 11:24:44 +0200
Subject: patch to fix build of nettle on gcc-2.95 architectures
In-Reply-To: <nn7h8whea1.fsf@stalhein.lysator.liu.se>
References: <40b6-4def3400-7-6b8b4580@32337325>
	<nn7h8whea1.fsf@stalhein.lysator.liu.se>
Message-ID: <BANLkTin1c47T05UG2rwmbj+7mJw91JcueA@mail.gmail.com>

On Wed, Jun 8, 2011 at 11:18 AM, Niels M?ller <nisse at lysator.liu.se> wrote:

>> I found nettle not compiling on openbsd sparc architecture which uses
>> gcc-2.95 as default compiler. Attached little patch against nettle-2.1
>> fixes the compilation problem for me.
> Thanks, checked in now. Nettle is not supposed to rely on c99 features.

These things make you feel really old. I still remember the time where C99
was a new thing, but now it's already 12 years old.

regards,
Nikos

From simon at josefsson.org  Wed Jun  8 11:46:06 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Wed, 08 Jun 2011 11:46:06 +0200
Subject: release?
In-Reply-To: <nn8vtoa8kd.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Mon, 30 May 2011 10:36:50 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
Message-ID: <87mxhsbqqp.fsf_-_@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
>
>> May I suggest an intermediate release with GCM under LGPLv2+?
>> That would ensure that current applications using gnutls can use that
>> code.
>
> An all-LGPL release is not possible until the serpent code is replaced,
> so any releases before that would use the current licensing terms (GPL
> as a whole, certain parts under more permissive license).

The serpent code is replaced, any other issues that we are waiting for?

A new Nettle release would be nice to make GnuTLS with Nettle easier to
install.  I guess anyone can prepare a snapshot, but an official release
is better.

/Simon

From nisse at lysator.liu.se  Wed Jun  8 13:43:30 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 08 Jun 2011 13:43:30 +0200
Subject: release?
In-Reply-To: <87mxhsbqqp.fsf_-_@latte.josefsson.org> (Simon Josefsson's
	message of "Wed, 08 Jun 2011 11:46:06 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87mxhsbqqp.fsf_-_@latte.josefsson.org>
Message-ID: <nn39jkh7kt.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> The serpent code is replaced, any other issues that we are waiting for?

 * Fix the libdir defaults for linux x86_64 (I think the following
   should work in most cases for for both debian and fhs conventions:
   When building for $ABI (32 or 64), check if /usr/lib$ABI exists. If
   so, install there, otherwise in $prefix/lib. Possibly bail out with
   an error if neither /usr/lib32 nor /usr/lib64 exists).

 * Decide which flavor of LGPL licensing should be used. Options:
   LGPLv2+, LGPLv3+ or dual license LGPLv3+ and GPLv2+.

 * Update NEWS and documentation (regarding serpent and licensing).

Comments appreciated. Anything else?

In the mean time, I'm also hacking further on the serpent code. The
double-block trick in C gave a modest improvement. I think it should
work great with sse2 assembly to do four blocks at a time. Primarily for
x86_64, but maybe also for plain x86 (but then maybe one can's use sse2
unconditionally). But that doesn't have to be completed before release.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Wed Jun  8 13:55:48 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Wed, 08 Jun 2011 13:55:48 +0200
Subject: release?
In-Reply-To: <nn39jkh7kt.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Wed, 08 Jun 2011 13:43:30 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87mxhsbqqp.fsf_-_@latte.josefsson.org>
	<nn39jkh7kt.fsf@stalhein.lysator.liu.se>
Message-ID: <87aadsa663.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Simon Josefsson <simon at josefsson.org> writes:
>
>> The serpent code is replaced, any other issues that we are waiting for?
>
>  * Fix the libdir defaults for linux x86_64 (I think the following
>    should work in most cases for for both debian and fhs conventions:
>    When building for $ABI (32 or 64), check if /usr/lib$ABI exists. If
>    so, install there, otherwise in $prefix/lib. Possibly bail out with
>    an error if neither /usr/lib32 nor /usr/lib64 exists).

I don't understand why this is a Nettle problem, since I don't recall
having to do anything like that for any of the libraries I maintain.
Could it be because I use automake and libtool?  If automake would solve
this issue, how about using it in Nettle?

>  * Decide which flavor of LGPL licensing should be used. Options:
>    LGPLv2+, LGPLv3+ or dual license LGPLv3+ and GPLv2+.

Is there a reason to change from the current LGPLv2+ before the next
release?

>  * Update NEWS and documentation (regarding serpent and licensing).

I can help with that, if you want.  I did a diff between 2.1 and the
current code, and it is a fairly small set of changes.

> Comments appreciated. Anything else?

Not that I can think of.

Using UTF-8 encoding for the files instead of ISO-8859-1 could be one.

> In the mean time, I'm also hacking further on the serpent code. The
> double-block trick in C gave a modest improvement. I think it should
> work great with sse2 assembly to do four blocks at a time. Primarily for
> x86_64, but maybe also for plain x86 (but then maybe one can's use sse2
> unconditionally). But that doesn't have to be completed before release.

Right.

/Simon

From nisse at lysator.liu.se  Wed Jun  8 17:49:12 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 08 Jun 2011 17:49:12 +0200
Subject: release?
In-Reply-To: <87aadsa663.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Wed, 08 Jun 2011 13:55:48 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87mxhsbqqp.fsf_-_@latte.josefsson.org>
	<nn39jkh7kt.fsf@stalhein.lysator.liu.se>
	<87aadsa663.fsf@latte.josefsson.org>
Message-ID: <nny61cfhmv.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> I don't understand why this is a Nettle problem, since I don't recall
> having to do anything like that for any of the libraries I maintain.
> Could it be because I use automake and libtool?

Sure, it's not a very nettle-specific problem. Maybe current automake
and/or libtool does the right thing, but I doubt it.

I think I've tried to explain what the problem is a couple of times
before, but appearantly I failed at that. I'll try one more time:
Please try out the the default behaviour of

  ./configure && make && make install

with your libraries (or any random library using autoconf and default
GNU installation conventions). I would expect it to break badly on,
e.g., a x86_64 redhat-based system, with an installed gcc which defaults
to 64-bit objects (-m64 enabled by default). Then make install will
install in /usr/local/lib, destroying any previous 32-bit installation
of the same library. Correct install location on this system, which
follows the linux fhs, is /usr/local/lib64.

Similar problems would occur on the (a bit less likely) combinations of

 64-bit Solaris (sparc or x86_64), if you have installed a gcc to
 default to 64-bit.

 freebsd or debian gnu/linux on x86_64, if for some reason you have
 installed a compiler which defaults to 32-bit.
 
I want the default behavior to not break things, that's all. And it's no
big deal. The bug in the current configure logic should be easy to fix
(I've probably spent about as much time on this email as it will take to
fix the problem).

> If automake would solve this issue, how about using it in Nettle?

No thanks. But I'd be happy to reuse any decent autoconf macros solving
the problem, from automake or elsewhere. If there are any.

>>  * Decide which flavor of LGPL licensing should be used. Options:
>>    LGPLv2+, LGPLv3+ or dual license LGPLv3+ and GPLv2+.
>
> Is there a reason to change from the current LGPLv2+ before the next
> release?

Maybe not. Moving to LGPLv2+ is definitely the easiest and smallest
change.

>>  * Update NEWS and documentation (regarding serpent and licensing).
>
> I can help with that, if you want.  I did a diff between 2.1 and the
> current code, and it is a fairly small set of changes.

That would be appreciated. I started on the NEWS entry a while back, I
just checked in my current version. I don't remember off the top of my
head what the last item, "Interface cleanup", refers to. (I usually
review the ChangeLog rather than actual diffs when updating the NEWS
file).

> Using UTF-8 encoding for the files instead of ISO-8859-1 could be one.

That should be done some time, but I don't think I'll bother about it
now. E.g., the move away from cvs seems more important, and that will
also have to wait until after release.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Sat Jun 11 07:28:29 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Sat, 11 Jun 2011 01:28:29 -0400
Subject: rsa keygen replicability
Message-ID: <4DF2FCFD.7060700@fifthhorseman.net>

Hi Nettle folks--

I'm wondering if there's an expectation of repeatable RSA key generation
across different versions of libnettle.

In particular, assuming a statically-seeded PRNG (e.g. seeding with a
fixed number of bytes from /dev/zero), should I be able to call
rsa_generate_keypair() and have it return a predictable result, even
across versions of libnettle?

Obviously, replicability is *not* what you'd want for production; but
for testing, it can be a very handy feature.

Right now, i'm trying with a statically-seeded yarrow256_ctx PRNG
against version 2.0 of nettle, and seeing a different key returned from
rsa_generate_keypair() that i see with version 2.1 of nettle.

Should this variation be expected? has the use of random numbers (or
other parts of the algorithm) changed significantly for
rsa_generate_keypair() between releases?

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110611/e9aa017a/attachment.pgp>

From nisse at lysator.liu.se  Mon Jun 13 14:02:35 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 13 Jun 2011 14:02:35 +0200
Subject: rsa keygen replicability
In-Reply-To: <4DF2FCFD.7060700@fifthhorseman.net> (Daniel Kahn Gillmor's
	message of "Sat, 11 Jun 2011 01:28:29 -0400")
References: <4DF2FCFD.7060700@fifthhorseman.net>
Message-ID: <nnd3ii9bxg.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> I'm wondering if there's an expectation of repeatable RSA key generation
> across different versions of libnettle.

I wouldn't expect it. On the other hand, I think most releases don't
change this.

> Right now, i'm trying with a statically-seeded yarrow256_ctx

When one really wants repeatability, one can use the (otherwise pretty
useless) lfib_knuth generator rather than yarrow256.

> has the use of random numbers (or other parts of the algorithm)
> changed significantly for rsa_generate_keypair() between releases?

The generation method was replaced in nettle-2.1. From the NEWS file.

	* Key generation for RSA and DSA changed to use Maurer's
	  algorithm to generate provably prime numbers (as usual, the
	  mathematical proof does not guaranteee that the
	  implementation is bug free).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From dkg at fifthhorseman.net  Mon Jun 13 16:43:32 2011
From: dkg at fifthhorseman.net (Daniel Kahn Gillmor)
Date: Mon, 13 Jun 2011 10:43:32 -0400
Subject: rsa keygen replicability
In-Reply-To: <nnd3ii9bxg.fsf@stalhein.lysator.liu.se>
References: <4DF2FCFD.7060700@fifthhorseman.net>
	<nnd3ii9bxg.fsf@stalhein.lysator.liu.se>
Message-ID: <4DF62214.60600@fifthhorseman.net>

On 06/13/2011 08:02 AM, Niels M?ller wrote:
> When one really wants repeatability, one can use the (otherwise pretty
> useless) lfib_knuth generator rather than yarrow256.

I've actually found that yarrow256 itself *is* repeatable across nettle
versions (well, 2.0 ? 2.1 at least); it was just the RSA key generation
that was not repeatable.

> The generation method was replaced in nettle-2.1. From the NEWS file.
> 
> 	* Key generation for RSA and DSA changed to use Maurer's
> 	  algorithm to generate provably prime numbers (as usual, the
> 	  mathematical proof does not guaranteee that the
> 	  implementation is bug free).

ah, thanks; this explains why.  Thanks for the pointer.

	--dkg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 1030 bytes
Desc: OpenPGP digital signature
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110613/e1bf3586/attachment.pgp>

From nisse at lysator.liu.se  Mon Jun 13 16:57:21 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 13 Jun 2011 16:57:21 +0200
Subject: rsa keygen replicability
In-Reply-To: <4DF62214.60600@fifthhorseman.net> (Daniel Kahn Gillmor's message
	of "Mon, 13 Jun 2011 10:43:32 -0400")
References: <4DF2FCFD.7060700@fifthhorseman.net>
	<nnd3ii9bxg.fsf@stalhein.lysator.liu.se>
	<4DF62214.60600@fifthhorseman.net>
Message-ID: <nn8vt5aiem.fsf@stalhein.lysator.liu.se>

Daniel Kahn Gillmor <dkg at fifthhorseman.net> writes:

> I've actually found that yarrow256 itself *is* repeatable across nettle
> versions (well, 2.0 ? 2.1 at least);

That's intended (but not promised). Makes regression testing easier.

There have been one incompatible change to yarrow in recent years,
related to seedfile management. (See ChangeLog for 2008-10-24 -- 2008-10-31).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Wed Jun 15 22:49:04 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 15 Jun 2011 22:49:04 +0200
Subject: release?
In-Reply-To: <nn39jkh7kt.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Wed, 08 Jun 2011 13:43:30 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87mxhsbqqp.fsf_-_@latte.josefsson.org>
	<nn39jkh7kt.fsf@stalhein.lysator.liu.se>
Message-ID: <nn4o3q95xb.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

>  * Fix the libdir defaults for linux x86_64 (I think the following
>    should work in most cases for for both debian and fhs conventions:
>    When building for $ABI (32 or 64), check if /usr/lib$ABI exists. If
>    so, install there, otherwise in $prefix/lib. Possibly bail out with
>    an error if neither /usr/lib32 nor /usr/lib64 exists).

I've checked in a fix for this now, and I also wrote up a web page with
the release TODO list (organization copied from corresponding pages for
recent GMP releases). See
http://www.lysator.liu.se/~nisse/nettle/plan.html

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Thu Jun 30 21:21:46 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 30 Jun 2011 21:21:46 +0200
Subject: serpent on x86_64, using sse2 instructions
Message-ID: <nnhb77nn05.fsf@stalhein.lysator.liu.se>

I've now tried to write some sse2 assembly code for serpent, to do four
blocks at a time in parallell (which helps for ecb and ctr mode, and cbc
decrypt, but not at all for cbc encrypt). First time I've tried to use
these x86 instructions for anything.

I'm currently under the belief that all existing x86_64 processors have
the needed instructions, so there are no configure time or run time
tests of the corresponding cpuid flags.

Speed is then close to aes128 (aes128 is slightly faster for encypt, and
serpent slightly faster for decrypt), and a bit faster than other aes
variants as well as camellia.

I imagine that with processors with whatever sse extension is needed to
get the 256-bit ymm registers, one can get almost twice the performance
for serpent, compared to the current using only the 128-bit xmm
registers.

And in the other direction, the code could easily be ported to 32-bit
x86 with sse2, with just a small penalty from the fewer registers.

Benchmark results on my laptop (intel SU4100):

         Algorithm        mode Mbyte/s cycles/byte cycles/block

            aes128 ECB encrypt   73.47       16.87       269.98
            aes128 ECB decrypt   71.73       17.28       276.53
            aes128 CBC encrypt   61.99       20.00       320.01
            aes128 CBC decrypt   70.52       17.58       281.29

            aes192 ECB encrypt   63.23       19.61       313.72
            aes192 ECB decrypt   61.86       20.04       320.69
            aes192 CBC encrypt   54.30       22.83       365.31
            aes192 CBC decrypt   61.04       20.31       324.97

            aes256 ECB encrypt   54.35       22.81       364.97
            aes256 ECB decrypt   54.28       22.84       365.42
            aes256 CBC encrypt   48.52       25.55       408.80
            aes256 CBC decrypt   53.56       23.15       370.34

       camellia128 ECB encrypt   57.53       21.55       344.80
       camellia128 ECB decrypt   57.52       21.55       344.86
       camellia128 CBC encrypt   51.99       23.84       381.51
       camellia128 CBC decrypt   56.72       21.86       349.74

       camellia192 ECB encrypt   43.05       28.80       460.72
       camellia192 ECB decrypt   43.01       28.82       461.16
       camellia192 CBC encrypt   39.94       31.04       496.68
       camellia192 CBC decrypt   42.53       29.15       466.40

       camellia256 ECB encrypt   43.03       28.81       461.00
       camellia256 ECB decrypt   43.02       28.82       461.14
       camellia256 CBC encrypt   39.90       31.07       497.11
       camellia256 CBC decrypt   42.61       29.10       465.54

        serpent256 ECB encrypt   68.88       18.00       287.98
        serpent256 ECB decrypt   79.81       15.53       248.54
        serpent256 CBC encrypt   23.20       53.43       854.86
        serpent256 CBC decrypt   78.25       15.84       253.50

It's conceivable that one can use simd instructions to get some speedup
also for aes and camellia, but it's not going to be almost trivial like
it is for serpent. (And for aes, I don't yet have any code using the
special instructions available in newer x86_64 processors).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.


From nisse at lysator.liu.se  Thu Jun 30 21:31:54 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 30 Jun 2011 21:31:54 +0200
Subject: gcm crash on ppc
In-Reply-To: <87d3izhncm.fsf_-_@latte.josefsson.org> (Simon Josefsson's
	message of "Mon, 30 May 2011 23:47:53 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<87ipsrhos6.fsf@latte.josefsson.org>
	<87d3izhncm.fsf_-_@latte.josefsson.org>
Message-ID: <nnd3hvnmj9.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> What is fishy is that nettle_gcm_aes128 doesn't seem to have been
> initialized properly:

Did you solve this problem? To me it seems like it could be

* setup of the shared-library magic, for this particular object file,
  somehow broken.

* bugs in the macos port of gcc (I don't know if you use apples "xcode"
  or some other gcc port, and I don't really know the difference
  either). According to the gmp people, there have been *lots* of
  problems with apple's linker and maybe also with other related tools.

* some other random hardware or software problem.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Thu Jun 30 23:54:51 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Fri, 1 Jul 2011 00:54:51 +0300
Subject: serpent on x86_64, using sse2 instructions
In-Reply-To: <nnhb77nn05.fsf@stalhein.lysator.liu.se>
References: <nnhb77nn05.fsf@stalhein.lysator.liu.se>
Message-ID: <BANLkTi=20dgzVx-n4bPwqWV0n7+OjxaZGg@mail.gmail.com>

On Thu, Jun 30, 2011 at 10:21 PM, Niels M?ller <nisse at lysator.liu.se> wrote:

> It's conceivable that one can use simd instructions to get some speedup
> also for aes and camellia, but it's not going to be almost trivial like
> it is for serpent. (And for aes, I don't yet have any code using the
> special instructions available in newer x86_64 processors).

In gnutls we use Andy Polyakov's code. It can be found at:

(AES-NI) http://cvs.openssl.org/dir?d=openssl/crypto/aes/asm

(AES-GCM) http://cvs.openssl.org/dir?d=openssl/crypto/modes/asm

Its license is LGPL compatible.

regards,
Nikos

From nmav at gnutls.org  Fri Jul  1 00:37:02 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Fri, 1 Jul 2011 01:37:02 +0300
Subject: Fwd: Problem compiling gnutls 2.12.7 on Solaris 9
In-Reply-To: <E429008A-9E83-4C4E-BBCA-547F3A6F427D@opencsw.org>
References: <E429008A-9E83-4C4E-BBCA-547F3A6F427D@opencsw.org>
Message-ID: <BANLkTi=StO5Vukn6OpomjTHf_=_giBqBvg@mail.gmail.com>

Hello Niels,
 nettle-stdint.h conflicts with gnulib's stdint.h on solaris. Is there
any real reason for nettle to define the uint_t types in the exported
API? Even if it wasn't for gnulib I would expect most applications
to define those types if not available, thus a conflict would anyway occur.


---------- Forwarded message ----------
From: Dagobert Michelsen <dam at opencsw.org>
Date: Wed, Jun 29, 2011 at 5:10 PM
Subject: Problem compiling gnutls 2.12.7 on Solaris 9
To: gnutls-devel at gnu.org


Hi,

I am trying to compile gnutls 2.12.7 with libnettle 2.1 on Solaris 9 Sparc with
Sun Studio 12 and get the following error:

gmake[6]: Entering directory
`/home/dam/mgar/pkg/gnutls/trunk/work/solaris9-sparc/build-isa-sparcv8/gnutls-2.12.7/lib/nettle'
?CC ? ? pk.lo
"/opt/csw/include/nettle/nettle-stdint.h", line 237: identifier
redeclared: gl_int_fast8_t
? ? ? ?current : signed char
? ? ? ?previous: long : "./../gl/stdint.h", line 241
"/opt/csw/include/nettle/nettle-stdint.h", line 238: warning:
modification of typedef with "int" ignored
"/opt/csw/include/nettle/nettle-stdint.h", line 238: identifier
redeclared: gl_int_fast16_t
? ? ? ?current : int
? ? ? ?previous: long : "./../gl/stdint.h", line 243
"/opt/csw/include/nettle/nettle-stdint.h", line 239: warning:
modification of typedef with "int" ignored
"/opt/csw/include/nettle/nettle-stdint.h", line 239: identifier
redeclared: gl_int_fast32_t
? ? ? ?current : int
? ? ? ?previous: long : "./../gl/stdint.h", line 245
"/opt/csw/include/nettle/nettle-stdint.h", line 241: warning: typedef
redeclared: int64_t
"/opt/csw/include/nettle/nettle-stdint.h", line 244: identifier
redeclared: gl_uint_fast8_t
? ? ? ?current : unsigned char
? ? ? ?previous: unsigned long : "./../gl/stdint.h", line 242
"/opt/csw/include/nettle/nettle-stdint.h", line 245: identifier
redeclared: gl_uint_fast16_t
? ? ? ?current : unsigned int
? ? ? ?previous: unsigned long : "./../gl/stdint.h", line 244
"/opt/csw/include/nettle/nettle-stdint.h", line 246: identifier
redeclared: gl_uint_fast32_t
? ? ? ?current : unsigned int
? ? ? ?previous: unsigned long : "./../gl/stdint.h", line 246
"/opt/csw/include/nettle/nettle-stdint.h", line 248: warning: typedef
redeclared: uint64_t
cc: acomp failed for pk.c

It looks like a gnulib incompatibility.


Best regards

?-- Dago


--
"You don't become great by trying to be great, you become great by
wanting to do something,
and then doing it so hard that you become great in the process." - xkcd #896


_______________________________________________
Gnutls-devel mailing list
Gnutls-devel at gnu.org
https://lists.gnu.org/mailman/listinfo/gnutls-devel

From nisse at lysator.liu.se  Fri Jul  1 17:57:55 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 01 Jul 2011 17:57:55 +0200
Subject: Fwd: Problem compiling gnutls 2.12.7 on Solaris 9
In-Reply-To: <BANLkTi=StO5Vukn6OpomjTHf_=_giBqBvg@mail.gmail.com> (Nikos
	Mavrogiannopoulos's message of "Fri, 1 Jul 2011 01:37:02 +0300")
References: <E429008A-9E83-4C4E-BBCA-547F3A6F427D@opencsw.org>
	<BANLkTi=StO5Vukn6OpomjTHf_=_giBqBvg@mail.gmail.com>
Message-ID: <nn8vsingcc.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

>  nettle-stdint.h conflicts with gnulib's stdint.h on solaris.

The same or a very similar issue was reported a while ago. On Solaris, I
expect nettle-stdint.h to short circuit, and just include the system's
stdint.h (or maybe inttypes.h).

What I'd like to know is if nettle's configure script sets up
nettle-stdint.h to include the right system include file, as I expect it
to:

* If no, that's clearly a bug in nettle's configure.ac or in the
  AX_CREATE_STDINT_H macro. To track it down, I would need to look at
  the installed nettle-stdint.h and at the config.log when nettle was
  built.

* If yes, then gnulib's definitions don't really conflict with
  *nettle*'s definitions, but with Solaris' definitions.

I'm also not sure if Sun's compiler and gcc agree on what, e.g.
uint_fast32_t should be (they ought to agree though; these types should
be nailed down by the relevant ABI specification, and then also gnulib
should stick to that spec).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Sat Jul  2 00:11:57 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 02 Jul 2011 00:11:57 +0200
Subject: release?
In-Reply-To: <nn4o3q95xb.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Wed, 15 Jun 2011 22:49:04 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87mxhsbqqp.fsf_-_@latte.josefsson.org>
	<nn39jkh7kt.fsf@stalhein.lysator.liu.se>
	<nn4o3q95xb.fsf@stalhein.lysator.liu.se>
Message-ID: <nn4o35odle.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> See http://www.lysator.liu.se/~nisse/nettle/plan.html

I think everything on this list is done now. I also checked in the new
serpent sse2 code, and I've looked through my older changes for memxor and
cbs decrypt (the latter can avoid a memcpy by using memxor3 with
overlapping operands), and I think they're ok.

So I think the 2.2 release is near. But I'm travelling with somewhat
limited hacking time and spotty connectivity.

I'm checking http://www.lysator.liu.se/~nisse/xeno-lsh/latest.html
(which also builds nettle). Any additional last minute testing of the
current cvs tree is appreciated.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From simon at josefsson.org  Sat Jul  2 09:22:32 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Sat, 02 Jul 2011 09:22:32 +0200
Subject: release?
In-Reply-To: <nn4o35odle.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Sat, 02 Jul 2011 00:11:57 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87mxhsbqqp.fsf_-_@latte.josefsson.org>
	<nn39jkh7kt.fsf@stalhein.lysator.liu.se>
	<nn4o3q95xb.fsf@stalhein.lysator.liu.se>
	<nn4o35odle.fsf@stalhein.lysator.liu.se>
Message-ID: <87wrg12ll3.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> So I think the 2.2 release is near.

Great!

> Any additional last minute testing of the
> current cvs tree is appreciated.

It builds and passes self checks on debian squeeze x86 and a Solaris
2.10 box (without gmp though) I have access to.

Clang-analyze on the code did not raise any errors in the core library,
but some other minor issues in other code:

Bug Group
Bug Type
File
Line

API
Argument with 'nonnull' attribute passed null
tools/sexp-conv.c
274

Dead store
Dead assignment
examples/io.c
88

Dead store
Dead assignment
examples/io.c
90

Logic error
Dereference of null pointer
tools/nettle-hash.c
204

Logic error
Dereference of null pointer
tools/nettle-hash.c
205

Logic error
Dereference of null pointer
examples/io.c
123

/Simon

From simon at josefsson.org  Sat Jul  2 09:30:52 2011
From: simon at josefsson.org (Simon Josefsson)
Date: Sat, 02 Jul 2011 09:30:52 +0200
Subject: gcm crash on ppc
In-Reply-To: <nnd3hvnmj9.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Thu, 30 Jun 2011 21:31:54 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<87ipsrhos6.fsf@latte.josefsson.org>
	<87d3izhncm.fsf_-_@latte.josefsson.org>
	<nnd3hvnmj9.fsf@stalhein.lysator.liu.se>
Message-ID: <87pqlt2l77.fsf@latte.josefsson.org>

nisse at lysator.liu.se (Niels M?ller) writes:

> Simon Josefsson <simon at josefsson.org> writes:
>
>> What is fishy is that nettle_gcm_aes128 doesn't seem to have been
>> initialized properly:
>
> Did you solve this problem? To me it seems like it could be
>
> * setup of the shared-library magic, for this particular object file,
>   somehow broken.
>
> * bugs in the macos port of gcc (I don't know if you use apples "xcode"
>   or some other gcc port, and I don't really know the difference
>   either). According to the gmp people, there have been *lots* of
>   problems with apple's linker and maybe also with other related tools.
>
> * some other random hardware or software problem.

The problem still occurs, but I'd be inclined to write it off as a
compiler bug because the system is running pretty old software (even if
it is latest published from Apple for this hardware).

espresso:~ jas$ gcc --version
powerpc-apple-darwin8-gcc-4.0.1 (GCC) 4.0.1 (Apple Computer, Inc. build 5370)
Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

espresso:~ jas$ 

/Simon

From nisse at lysator.liu.se  Thu Jul  7 22:36:06 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 07 Jul 2011 22:36:06 +0200
Subject: release?
In-Reply-To: <87wrg12ll3.fsf@latte.josefsson.org> (Simon Josefsson's message
	of "Sat, 02 Jul 2011 09:22:32 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87mxhsbqqp.fsf_-_@latte.josefsson.org>
	<nn39jkh7kt.fsf@stalhein.lysator.liu.se>
	<nn4o3q95xb.fsf@stalhein.lysator.liu.se>
	<nn4o35odle.fsf@stalhein.lysator.liu.se>
	<87wrg12ll3.fsf@latte.josefsson.org>
Message-ID: <nnliw93k21.fsf@stalhein.lysator.liu.se>

Simon Josefsson <simon at josefsson.org> writes:

> Clang-analyze on the code did not raise any errors in the core library,
> but some other minor issues in other code:

Thanks, I've reviewed the code in questions:

> API
> Argument with 'nonnull' attribute passed null
> tools/sexp-conv.c
> 274

This is a call to strtol.

	    int width = strtol(optarg, &end , 0);

optarg shouldn't be NULL, but that's not really nettle's responsibility
(-w has a required argument in the getopt_long call). Warning seems to
be a false positive.

> Dead store
> Dead assignment
> examples/io.c
> 88

> Dead store
> Dead assignment
> examples/io.c
> 90

This is a minor real bug.

> Logic error
> Dereference of null pointer
> tools/nettle-hash.c
> 204

> Logic error
> Dereference of null pointer
> tools/nettle-hash.c
> 205

This is
    
    length = alg->digest_size;
  else if (length > alg->digest_size)

and there's an if (!alg) die(...) earlier, where (at least for gcc) die
is declared as __attribute((__noreturn__)). Warning seems to be a false
positive.

But there's a related copy&paste error, since werror has the same
declaration and it *does* return,

> Logic error
> Dereference of null pointer
> examples/io.c
> 123

  buffer[done] = '\0';

buffer is return value from a realloc call with a NULL check and return.
The logic using feof and ferror looks a bit suspicious, but as long as
feof is false just after fopen, things should be ok (and feof should not
be set until one attempts to read).

Thanks,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Mon Jul 11 16:34:54 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 11 Jul 2011 16:34:54 +0200
Subject: ANNOUNCE: Nettle-2.2
Message-ID: <nn8vs43my9.fsf@stalhein.lysator.liu.se>

I'm happy to announce a new version of GNU Nettle, a low-level
cryptographic library. Unlike earlier releases, nettle is now licenced
under the LGPL, rather then the GPL. The nettle home page can be found
at http://www.lysator.liu.se/~nisse/nettle/

NEWS for the 2.2 release

	Licensing change:
     	
	* Relicensed as LGPL v2.1 or later (user's option).

	* Replaced blowfish and serpent implementation. New code is
          based on the LGPLed code in libgcrypt.

	New features:

	* Support for Galois/Counter Mode (GCM).

	* New interface for enumerating (most) available algorithms,
	  contributed by Daniel Kahn Gillmor.

	* New tool nettle-hash. Can generate hash digests using any
	  supported hash function, with output compatible with md5sum
	  and friends from GNU coreutils. Checking (like md5sum -c)
	  not yet implemented.

	Bug fixes:

	* The old serpent code had a byte order bug (introduced by
	  yours truly about ten years ago). New serpent implementation
	  does not interoperate with earlier versions of nettle.

	* Fixed ABI-dependent libdir default for Linux-based systems
	  which do not follow the Linux File Hierarchy Standard, e.g.,
	  Debian GNU/Linux.

	Optimizations:
	
	* x86_64 implemention of serpent.

	* x86_64 implemention of camellia.

	* Optimized memxor using word rather than byte operations.
          Both generic C and x86_64 assembler.

	* Eliminated a memcpy for in-place CBC decrypt.
	
	Miscellaneous:

	* In command line tools, no longer support -? for requesting
          help, since using it without shell quoting is a dangerous
          habit. Use long option --help instead.

	The shared library names are libnettle.so.4.1 and
	libhogweed.so.2.1, with sonames libnettle.so.4 and
	libhogweed.so.2.

Available at

  http://www.lysator.liu.se/~nisse/archive/nettle-2.2.tar.gz
  ftp://ftp.gnu.org/gnu/nettle/nettle-2.2.tar.gz

Happy hacking,
/Niels M?ller

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.


From nisse at lysator.liu.se  Sun Jul 17 17:06:43 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 17 Jul 2011 17:06:43 +0200
Subject: gcm crash on ppc
In-Reply-To: <nnd3hvnmj9.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Thu, 30 Jun 2011 21:31:54 +0200")
References: <4DD772E6.1010305@gnutls.org>
	<nn39k8e2m3.fsf@stalhein.lysator.liu.se> <4DD7A546.8000808@gnutls.org>
	<nn8vtoa8kd.fsf@stalhein.lysator.liu.se>
	<87sjrvhq5n.fsf_-_@latte.josefsson.org>
	<87oc2jhp4d.fsf@latte.josefsson.org>
	<87ipsrhos6.fsf@latte.josefsson.org>
	<87d3izhncm.fsf_-_@latte.josefsson.org>
	<nnd3hvnmj9.fsf@stalhein.lysator.liu.se>
Message-ID: <nnvcv10wvw.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> Did you solve this problem? To me it seems like it could be
>
> * setup of the shared-library magic, for this particular object file,
>   somehow broken.
>
> * bugs in the macos port of gcc (I don't know if you use apples "xcode"
>   or some other gcc port, and I don't really know the difference
>   either). According to the gmp people, there have been *lots* of
>   problems with apple's linker and maybe also with other related tools.
>
> * some other random hardware or software problem.

Several people have reported a bug of the first type: in
nettle-internal.h, the const structs nettle_gcm_* ought to be declared
extern. I guess I will have to issue a bugfix release soon. I.e., 

-const struct nettle_aead nettle_gcm_aes128;
+extern const struct nettle_aead nettle_gcm_aes128;

etc.

Fix checked in now. Simon: It would be nice if you could test if this
also solves the ppc problem.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Wed Jul 20 23:07:16 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 20 Jul 2011 23:07:16 +0200
Subject: pkg-config
Message-ID: <nnwrfczmnv.fsf@stalhein.lysator.liu.se>

I just checked in nettle.pc.in and hogweed.pc.in. configure then
generates nettle.pc and hogweed.pc, for use wiith pkg-config.

I don't have any previous experience with pkg-config, so comments are
much appreciated.

Not yet any make rules to install them. I don't know if installation
should be unconditional, or if one should first check, somehow, if
pkg-config is installed. (And then hogweed.pc should be installed only if the
hogweed library is built and installed).

Apparantly, at least one user wants pkg-config support, see
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=634294

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.


From nmav at gnutls.org  Sun Aug 14 09:51:22 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sun, 14 Aug 2011 09:51:22 +0200
Subject: ECC support
Message-ID: <4E477E7A.4050105@gnutls.org>

Hello,
 This patch adds ECC support using the GnuTLS libtomcrypt adaptation.

regards,
Nikos


btw. I had issues building the cvs version. The compilation was
failing:
nettle-hash.o: In function `list_algorithms':
/home/nmav/cvs/lsh/nettle/tools/nettle-hash.c:49: undefined reference to
`nettle_hashes'

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Added-ECC.patch
Type: text/x-patch
Size: 59589 bytes
Desc: not available
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110814/5996b376/attachment.bin>

From nisse at lysator.liu.se  Sun Aug 14 19:43:50 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 14 Aug 2011 19:43:50 +0200
Subject: ECC support
In-Reply-To: <4E477E7A.4050105@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sun, 14 Aug 2011 09:51:22 +0200")
References: <4E477E7A.4050105@gnutls.org>
Message-ID: <nnwrefnb1l.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

>  This patch adds ECC support using the GnuTLS libtomcrypt adaptation.

Nice! Needs some more work to really fit with nettle. Some initial
comments based on the header file:

> +/* assume curve is y^2 = x^3 - 3x + b
> + * instead of the generic y^2 = x^3 + ax + b
> + *
> + * (XXX: the generic case has been tested only
> + *  with the SECG curves.)
> + */

Maybe the naming in the itnerface should reflect that it's a special
case.

> +/* ---- ECC Routines ---- */
> +/* size of our temp buffers for exported keys */
> +#define ECC_BUF_SIZE 512
> +
> +/* max private key size */
> +#define ECC_MAXSIZE  66

Where do these maximums come from?

> +/** Structure defines a NIST GF(p) curve */
> +typedef struct {
> +   /** The size of the curve in octets */
> +   int size;
> +
> +   /** name of curve */
> +   const char *name; 
> +   
> +   /** curve's OID */
> +   const char *oid;
> +
> +   /** The prime that defines the field the curve is in (encoded in hex) */
> +   const char *prime;
> +
> +   /** The fields A param (hex) */
> +   const char *A;
> +
> +   /** The fields B param (hex) */
> +   const char *B;
> +
> +   /** The order of the curve (hex) */
> +   const char *order;
> +  
> +   /** The x co-ordinate of the base point on the curve (hex) */
> +   const char *Gx;
> + 
> +   /** The y co-ordinate of the base point on the curve (hex) */
> +   const char *Gy;
> +} ecc_set_type;

Avoid using typedef for plain structs.

I'm not sure I like using hex strings for the constants. Depends a bit
on usage, of course. E.g., it's a bit difficult to define a constant
mpz_t.

> +/** An ECC key */
> +typedef struct {
> +    /** Type of key, PK_PRIVATE or PK_PUBLIC */
> +    int type;
> +
> +    mpz_t prime;
> +    mpz_t order;
> +    mpz_t A;
> +    mpz_t Gx;
> +    mpz_t Gy;
> +
> +    /** The public key */
> +    ecc_point pubkey;
> +
> +    /** The private key */
> +    mpz_t k;
> +} ecc_key;

I'd follow nettle's DSA interface with separate structs for the public
and private parameters, and eliminate the PK_PRIVATE and PK_PUBLIC
constants. I think it would also make sense to move out the curve
parameters to it's own struct (typed as mpz_t, then, unlike ecc_set_tyep
above).

> +/* Key generation */
> +int ecc_make_key(void *random_ctx, nettle_random_func random, ecc_key *key, const ecc_set_type *dp);
> +int ecc_make_key_ex(void *random_ctx, nettle_random_func random, ecc_key *key, mpz_t prime, mpz_t order, mpz_t A, mpz_t Gx, mpz_t Gy);
> +void ecc_free(ecc_key *key);

I haven't figured out exactly what these do, but naming should most
likele be _init and _clear, for consistency with the rest of nettle and
with gmp.

> +/* EC-Diffie-Hellman */
> +int  ecc_shared_secret(ecc_key *private_key, ecc_key *public_key, 
> +                       unsigned char *out, unsigned long *outlen);

Haven't looked at this; for diffie-hellman over the normal ring one
would just use gmp's powm function. I think the corresponding ecc
function should also be public (maybe it already is?).

> +/* ECDSA */
> +int ecc_sign_hash(const unsigned char *in,  unsigned long inlen, 
> +                        struct dsa_signature *signature,
> +                        void *random_ctx, nettle_random_func random, ecc_key *key);
> +
> +int  ecc_verify_hash(struct dsa_signature * signature,
> +                     const unsigned char *hash, unsigned long hashlen, 
> +                     int *stat, ecc_key *key);

Do these correspond to the _sign_digest and _verify_digest functions for
dsa and rsa?

> +/* (Internal) low level functions */
> +ecc_point *ecc_new_point(void);
> +void       ecc_del_point(ecc_point *p);

It's more nettle style to let the caller allocate the structs.
ecc_point_init and ecc_point_clear would be more appropriate.

> +/* point ops (mp == montgomery digit) */
> +/* R = 2P */
> +int ecc_projective_dbl_point(ecc_point *P, ecc_point *R, mpz_t a,  mpz_t modulus);
> +
> +/* R = P + Q */
> +int ecc_projective_add_point(ecc_point *P, ecc_point *Q, ecc_point *R, mpz_t A, mpz_t modulus);

Is it customary jargon and notation to think about the the ecc group
operation as addition rather than multiplication? (Choice is arbitrary).

> +/* R = kG */
> +int ecc_mulmod(mpz_t k, ecc_point *G, ecc_point *R, mpz_t a, mpz_t modulus, int map);

I'd call the it ecc_scalar_mul or something like that, rather than
mulmod. Algebraically, any (cummutative?) group is a module over Z or
over Z/(group order) or something like that (sorry, I don't recall the
fine details since I studied algebra), and that's the type of
multiplication we're doing: Z x G -> G.

> +int mp_init_multi(mpz_t *a, ...);
> +void mp_clear_multi(mpz_t *a, ...);

Not sure I like these.

> +#define mp_isodd(a)                  (mpz_size(a) > 0 ? (mpz_getlimbn(a, 0) & 1 ? 1 : 0) : 0)

Just use mpz_odd_p.

> +#define MP_DIGIT_BIT (sizeof(mp_limb_t) * 8 - GMP_NAIL_BITS)

Same as GMP_NUMB_BITS.

/nisse

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Sun Aug 14 20:47:13 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sun, 14 Aug 2011 20:47:13 +0200
Subject: ECC support
In-Reply-To: <nnwrefnb1l.fsf@stalhein.lysator.liu.se>
References: <4E477E7A.4050105@gnutls.org>
	<nnwrefnb1l.fsf@stalhein.lysator.liu.se>
Message-ID: <4E481831.7080803@gnutls.org>

On 08/14/2011 07:43 PM, Niels M?ller wrote:

>> +/* assume curve is y^2 = x^3 - 3x + b
>> + * instead of the generic y^2 = x^3 + ax + b
>> + *
>> + * (XXX: the generic case has been tested only
>> + *  with the SECG curves.)
>> + */
> Maybe the naming in the itnerface should reflect that it's a special
> case.

You can use the generic code by keeping ecc_projective_add_point.c
instead of ecc_projective_add_point_3.c. I have not tested the
generic code though with other curves than the SECP that use
a = -3. The improvement from the special case is not that
significant.

>> +/* ---- ECC Routines ---- */
>> +/* size of our temp buffers for exported keys */
>> +#define ECC_BUF_SIZE 512
>> +/* max private key size */
>> +#define ECC_MAXSIZE  66
> Where do these maximums come from?

>From the sizes of the supported groups.

>> +/* Key generation */
>> +int ecc_make_key(void *random_ctx, nettle_random_func random, ecc_key *key, const ecc_set_type *dp);
>> +int ecc_make_key_ex(void *random_ctx, nettle_random_func random, ecc_key *key, mpz_t prime, mpz_t order, mpz_t A, mpz_t Gx, mpz_t Gy);
>> +void ecc_free(ecc_key *key);
> I haven't figured out exactly what these do, but naming should most
> likele be _init and _clear, for consistency with the rest of nettle and
> with gmp.

make_key is actually _init and _generate in one.

>> +/* EC-Diffie-Hellman */
>> +int  ecc_shared_secret(ecc_key *private_key, ecc_key *public_key, 
>> +                       unsigned char *out, unsigned long *outlen);
> Haven't looked at this; for diffie-hellman over the normal ring one
> would just use gmp's powm function. I think the corresponding ecc
> function should also be public (maybe it already is?).

Could be. This is a convenience function.

>> +/* ECDSA */
>> +int ecc_sign_hash(const unsigned char *in,  unsigned long inlen, 
>> +                        struct dsa_signature *signature,
>> +                        void *random_ctx, nettle_random_func random, ecc_key *key);
>> +
>> +int  ecc_verify_hash(struct dsa_signature * signature,
>> +                     const unsigned char *hash, unsigned long hashlen, 
>> +                     int *stat, ecc_key *key);
> Do these correspond to the _sign_digest and _verify_digest functions for
> dsa and rsa?

Indeed but they are not limited to a particular digest. Any hash can be
used.

>> +/* point ops (mp == montgomery digit) */
>> +/* R = 2P */
>> +int ecc_projective_dbl_point(ecc_point *P, ecc_point *R, mpz_t a,  mpz_t modulus);
>> +
>> +/* R = P + Q */
>> +int ecc_projective_add_point(ecc_point *P, ecc_point *Q, ecc_point *R, mpz_t A, mpz_t modulus);
> Is it customary jargon and notation to think about the the ecc group
> operation as addition rather than multiplication? (Choice is arbitrary).

I've never seen multiplication being used to describe this operation
(either in cryptography or pure mathematics).

>> +int mp_init_multi(mpz_t *a, ...);
>> +void mp_clear_multi(mpz_t *a, ...);
> Not sure I like these.

They simplify code utilizing multiple mpz_ts significantly.

regards,
Nikos

From nisse at lysator.liu.se  Sun Aug 14 22:06:02 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 14 Aug 2011 22:06:02 +0200
Subject: ECC support
In-Reply-To: <4E481831.7080803@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sun, 14 Aug 2011 20:47:13 +0200")
References: <4E477E7A.4050105@gnutls.org>
	<nnwrefnb1l.fsf@stalhein.lysator.liu.se> <4E481831.7080803@gnutls.org>
Message-ID: <nnobzrn4gl.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> Indeed but they are not limited to a particular digest. Any hash can be
> used.

And the hash algorithm is not encoded into the signature process
(copmare to rsa pkcs#1 signatures)? I have to read up on how these ecc
signatures are done. I think a comment said that it was analogous to
dsa, but dsa is tied quite hard to a particular hash function (and the
digest size should also match the size of the subgroup where the group
operations take place).

> I've never seen multiplication being used to describe this operation
> (either in cryptography or pure mathematics).

I think the group operation of an arbitrary group is usually written as
a multiplication in abstract algebra textbooks. E.g., Herstein's Topics
in Algebra. Maybe the reason for this tradition is that it is natural
when group elements are functions (in particular, permutations), and the
group operation is composition (but then Herstein has an unusual
convention for the argument order of compositions...).

Do I understand you correctly that the group operations is usually
written as an addition in the context of elliptic curves?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Mon Aug 15 01:23:21 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Mon, 15 Aug 2011 01:23:21 +0200
Subject: ECC support
In-Reply-To: <nnobzrn4gl.fsf@stalhein.lysator.liu.se>
References: <4E477E7A.4050105@gnutls.org>	<nnwrefnb1l.fsf@stalhein.lysator.liu.se>
	<4E481831.7080803@gnutls.org>
	<nnobzrn4gl.fsf@stalhein.lysator.liu.se>
Message-ID: <4E4858E9.9000803@gnutls.org>

On 08/14/2011 10:06 PM, Niels M?ller wrote:
> Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:
> 
>> Indeed but they are not limited to a particular digest. Any hash
>> can be used.
> And the hash algorithm is not encoded into the signature process 
> (copmare to rsa pkcs#1 signatures)? I have to read up on how these
> ecc signatures are done. I think a comment said that it was analogous
> to dsa, but dsa is tied quite hard to a particular hash function (and
> the digest size should also match the size of the subgroup where the
> group operations take place).

ECDSA is very similar to DSA. Specific hash functions are used for
specific curves, but it always depends on the profile. E.g. rfc5480
has a table with the allowed combinations and a table with the
recommended combinations.

Unfortunately DSA (and ECDSA) require a profile, or are practically
unimplementable.

>> I've never seen multiplication being used to describe this
>> operation (either in cryptography or pure mathematics).
> I think the group operation of an arbitrary group is usually written
> as a multiplication in abstract algebra textbooks. E.g., Herstein's
> Topics in Algebra. Maybe the reason for this tradition is that it is
> natural when group elements are functions (in particular,
> permutations), and the group operation is composition (but then
> Herstein has an unusual convention for the argument order of
> compositions...). Do I understand you correctly that the group
> operations is usually written as an addition in the context of
> elliptic curves?

You always can avoid the term addition by using the generic term group
operation or so. I've also seen the dot notation to describe operations
in a group, but I've rarely seen the actual term multiplication.
Note however that here you also have the "scalar multiplication", so
if you use this term, addition would be the appropriate for the
group operation.

regards,
Nikos

From nisse at lysator.liu.se  Mon Aug 15 07:01:23 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 15 Aug 2011 07:01:23 +0200
Subject: ECC support
In-Reply-To: <4E4858E9.9000803@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Mon, 15 Aug 2011 01:23:21 +0200")
References: <4E477E7A.4050105@gnutls.org>
	<nnwrefnb1l.fsf@stalhein.lysator.liu.se> <4E481831.7080803@gnutls.org>
	<nnobzrn4gl.fsf@stalhein.lysator.liu.se> <4E4858E9.9000803@gnutls.org>
Message-ID: <nnhb5jmfoc.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> I've also seen the dot notation to describe operations
> in a group, but I've rarely seen the actual term multiplication.

I'm referring to the notation, which somehow determines how how one
thinks about the operation.

> Note however that here you also have the "scalar multiplication", so
> if you use this term, addition would be the appropriate for the
> group operation.

Sure. If one chooses to use multiplicative notation for the group
operation, this operation would be called an exponentiation, not a
multiplication.

Regards,
/nisse

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From mcitadel at gmail.com  Tue Aug 23 04:51:16 2011
From: mcitadel at gmail.com (Andres Mejia)
Date: Mon, 22 Aug 2011 22:51:16 -0400
Subject: Few Questions About Nettle Project
Message-ID: <CALgfg4i_9Xftj4NkT_B8-O_dMG2d8WtMoLKrJN1-xZwuOGMO4Q@mail.gmail.com>

Hi,

Keep up the great work. I was looking for an LGPL-2.1 crypto library
that can be used as an alternative to libgcrypt. Reason being can best
be explained at [1].

So here are some questions I have. GNU Nettle is going to stick with
LGPL-2.1+ correct? I ask this since I'm currently working on
implementing full RAR archive support with libarchive, to ultimately
be used in XBMC Media Center. XBMC is GPL-2+ and we have no desire to
go to GPL-3.

Another question, the only thing I see missing in nettle that
libarchive can make use of is RIPEMD-160. In case I want to submit
patches, I should submit them here correct? Also, I suppose if I
wanted to submit patches, there's no issue in porting code from gcrypt
over to nettle correct? I think it would be easier (and faster) to
reuse the implementation in gcrypt than to implement the algorithm
from scratch.

-- 
Regards,
Andres Mejia

1. http://lists.debian.org/debian-devel/2011/04/msg01170.html

From nisse at lysator.liu.se  Tue Aug 23 06:49:39 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 23 Aug 2011 06:49:39 +0200
Subject: Few Questions About Nettle Project
In-Reply-To: <CALgfg4i_9Xftj4NkT_B8-O_dMG2d8WtMoLKrJN1-xZwuOGMO4Q@mail.gmail.com>
	(Andres Mejia's message of "Mon, 22 Aug 2011 22:51:16 -0400")
References: <CALgfg4i_9Xftj4NkT_B8-O_dMG2d8WtMoLKrJN1-xZwuOGMO4Q@mail.gmail.com>
Message-ID: <nnliukk9zw.fsf@stalhein.lysator.liu.se>

Andres Mejia <mcitadel at gmail.com> writes:

> So here are some questions I have. GNU Nettle is going to stick with
> LGPL-2.1+ correct?

At some point the license may be upgraded to LGPLv3. Or dual license
LGPLv3 and GPLv2+. The latter, more complex, alternative would be
helpful for you, right?

> Another question, the only thing I see missing in nettle that
> libarchive can make use of is RIPEMD-160. In case I want to submit
> patches, I should submit them here correct?

Right.

> Also, I suppose if I wanted to submit patches, there's no issue in
> porting code from gcrypt over to nettle correct?

Copying code from libgcrypt is fine. In general, to make it as easy as possible to
integrate patches, it's nice if they include:

1. An authoritative reference for the implemented algorithm.

2. Test cases.

3. Documentation.

4. And then both interface and implementation should try to follow
   nettle conventions.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From mbroz at redhat.com  Wed Aug 24 17:08:15 2011
From: mbroz at redhat.com (Milan Broz)
Date: Wed, 24 Aug 2011 17:08:15 +0200
Subject: Few Questions About Nettle Project
In-Reply-To: <CALgfg4i_9Xftj4NkT_B8-O_dMG2d8WtMoLKrJN1-xZwuOGMO4Q@mail.gmail.com>
References: <CALgfg4i_9Xftj4NkT_B8-O_dMG2d8WtMoLKrJN1-xZwuOGMO4Q@mail.gmail.com>
Message-ID: <4E5513DF.4020708@redhat.com>

On 08/23/2011 04:51 AM, Andres Mejia wrote:
> Another question, the only thing I see missing in nettle that
> libarchive can make use of is RIPEMD-160. In case I want to submit
> patches, I should submit them here correct? Also, I suppose if I
> wanted to submit patches, there's no issue in porting code from gcrypt
> over to nettle correct? I think it would be easier (and faster) to
> reuse the implementation in gcrypt than to implement the algorithm
> from scratch.

Btw cryptsetup uses ripemd160 if used in plain mode
(iow mode where encryption key is directly hashed password).
So despite cryptsetup has already optional Nettle crypto backend
support, is cannot provide backward compatibility without ripemd160.

So you have yet another user for these patches ;-)

Thanks,
Milan

From mcitadel at gmail.com  Thu Aug 25 16:14:46 2011
From: mcitadel at gmail.com (Andres Mejia)
Date: Thu, 25 Aug 2011 10:14:46 -0400
Subject: Few Questions About Nettle Project
In-Reply-To: <nnliukk9zw.fsf@stalhein.lysator.liu.se>
References: <CALgfg4i_9Xftj4NkT_B8-O_dMG2d8WtMoLKrJN1-xZwuOGMO4Q@mail.gmail.com>
	<nnliukk9zw.fsf@stalhein.lysator.liu.se>
Message-ID: <CALgfg4jZSqtFY+7CnOAWmJE7CaiMNU66rp5rG5TPTiWu5W2sKw@mail.gmail.com>

On Tue, Aug 23, 2011 at 12:49 AM, Niels M?ller <nisse at lysator.liu.se> wrote:
> Andres Mejia <mcitadel at gmail.com> writes:
>
>> So here are some questions I have. GNU Nettle is going to stick with
>> LGPL-2.1+ correct?
>
> At some point the license may be upgraded to LGPLv3. Or dual license
> LGPLv3 and GPLv2+. The latter, more complex, alternative would be
> helpful for you, right?

So long as XBMC can remain GPLv2, then it's ok.

However, I'm sure there are some projects out there that could benefit
from an LGPLv2.1 crypto library. I looked around and so far have found
one, libaacs which is itself LGPLv2.1. libaacs is used indirectly by
XBMC, through libbluray. Currently, libaacs uses libgcrypt for crypto,
but it would be nice if the number of crypto libraries in use are kept
to a minimum.

>> Another question, the only thing I see missing in nettle that
>> libarchive can make use of is RIPEMD-160. In case I want to submit
>> patches, I should submit them here correct?
>
> Right.
>
>> Also, I suppose if I wanted to submit patches, there's no issue in
>> porting code from gcrypt over to nettle correct?
>
> Copying code from libgcrypt is fine. In general, to make it as easy as possible to
> integrate patches, it's nice if they include:
>
> 1. An authoritative reference for the implemented algorithm.
>
> 2. Test cases.
>
> 3. Documentation.
>
> 4. And then both interface and implementation should try to follow
> ? nettle conventions.
>
> Regards,
> /Niels
>
> --
> Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
> Internet email is subject to wholesale government surveillance.
>



-- 
Regards,
Andres Mejia

From nisse at lysator.liu.se  Thu Aug 25 19:46:59 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Thu, 25 Aug 2011 19:46:59 +0200
Subject: Few Questions About Nettle Project
In-Reply-To: <CALgfg4jZSqtFY+7CnOAWmJE7CaiMNU66rp5rG5TPTiWu5W2sKw@mail.gmail.com>
	(Andres Mejia's message of "Thu, 25 Aug 2011 10:14:46 -0400")
References: <CALgfg4i_9Xftj4NkT_B8-O_dMG2d8WtMoLKrJN1-xZwuOGMO4Q@mail.gmail.com>
	<nnliukk9zw.fsf@stalhein.lysator.liu.se>
	<CALgfg4jZSqtFY+7CnOAWmJE7CaiMNU66rp5rG5TPTiWu5W2sKw@mail.gmail.com>
Message-ID: <nn8vqhjsdo.fsf@stalhein.lysator.liu.se>

Andres Mejia <mcitadel at gmail.com> writes:

> On Tue, Aug 23, 2011 at 12:49 AM, Niels M?ller <nisse at lysator.liu.se> wrote:

>> At some point the license may be upgraded to LGPLv3. Or dual license
>> LGPLv3 and GPLv2+. The latter, more complex, alternative would be
>> helpful for you, right?
>
> So long as XBMC can remain GPLv2, then it's ok.
>
> However, I'm sure there are some projects out there that could benefit
> from an LGPLv2.1 crypto library. I looked around and so far have found
> one, libaacs which is itself LGPLv2.1. libaacs is used indirectly by
> XBMC, through libbluray.

I have to look at this more closely before upgrading the license (which
I don't have any immediate plans of doing). I'm not sure, but I actually
don't see any obvious problem with linking LGPLv2 and LGPLv3 code
together.

The problem when linking GPLv2 and LGPLv3 code together is not that
LGPLv3 doesn't allow it (after all, LGPL, any version, allows linking
even with with proprietary code), but that GPLv2 requires that the
*entire* work (except system libraries) can be distributed under GPLv2,
with no additional restrictions, and some of the new conditions in
(L)GPLv3 (probably the ones related to drm or patents) are additional
restrictions.

And dual licensing as LGPLv3+ and GPLv2+ solves precisely that problem:
GPLv2 programs (including tivoized GPLv2 programs) can still use the
library under the GPLv2 conditions, while all other users of the
library, in particular, all proprietary applications, must abide by the
new conditions introduced in LGPLv3.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From mcitadel at gmail.com  Fri Aug 26 04:02:34 2011
From: mcitadel at gmail.com (Andres Mejia)
Date: Thu, 25 Aug 2011 22:02:34 -0400
Subject: Few Questions About Nettle Project
In-Reply-To: <nn8vqhjsdo.fsf@stalhein.lysator.liu.se>
References: <CALgfg4i_9Xftj4NkT_B8-O_dMG2d8WtMoLKrJN1-xZwuOGMO4Q@mail.gmail.com>
	<nnliukk9zw.fsf@stalhein.lysator.liu.se>
	<CALgfg4jZSqtFY+7CnOAWmJE7CaiMNU66rp5rG5TPTiWu5W2sKw@mail.gmail.com>
	<nn8vqhjsdo.fsf@stalhein.lysator.liu.se>
Message-ID: <CALgfg4gdmDofHQyUMy01S+-Mm6mpWi28zcsG-86791kf8SRMkg@mail.gmail.com>

On Thu, Aug 25, 2011 at 1:46 PM, Niels M?ller <nisse at lysator.liu.se> wrote:
> Andres Mejia <mcitadel at gmail.com> writes:
>
>> On Tue, Aug 23, 2011 at 12:49 AM, Niels M?ller <nisse at lysator.liu.se> wrote:
>
>>> At some point the license may be upgraded to LGPLv3. Or dual license
>>> LGPLv3 and GPLv2+. The latter, more complex, alternative would be
>>> helpful for you, right?
>>
>> So long as XBMC can remain GPLv2, then it's ok.
>>
>> However, I'm sure there are some projects out there that could benefit
>> from an LGPLv2.1 crypto library. I looked around and so far have found
>> one, libaacs which is itself LGPLv2.1. libaacs is used indirectly by
>> XBMC, through libbluray.

I just remembered another example of an LGPLv2.1 library that can make
use of nettle. It's called GPAC. It currently uses openssl for crypto.
I did at one time try to implement crypto support using libgcrypt,
until I read about the problems that arise when using libgcrypt.

> I have to look at this more closely before upgrading the license (which
> I don't have any immediate plans of doing). I'm not sure, but I actually
> don't see any obvious problem with linking LGPLv2 and LGPLv3 code
> together.
>
> The problem when linking GPLv2 and LGPLv3 code together is not that
> LGPLv3 doesn't allow it (after all, LGPL, any version, allows linking
> even with with proprietary code), but that GPLv2 requires that the
> *entire* work (except system libraries) can be distributed under GPLv2,
> with no additional restrictions, and some of the new conditions in
> (L)GPLv3 (probably the ones related to drm or patents) are additional
> restrictions.
>
> And dual licensing as LGPLv3+ and GPLv2+ solves precisely that problem:
> GPLv2 programs (including tivoized GPLv2 programs) can still use the
> library under the GPLv2 conditions, while all other users of the
> library, in particular, all proprietary applications, must abide by the
> new conditions introduced in LGPLv3.

This matrix here might help.
http://www.gnu.org/licenses/gpl-faq.html#AllCompatibility

So any LGPL program can use another LGPL library regardless of their
versions. However, GPLv2 programs that want to use an LGPLv3 library
will need to upgrade to GPLv3. This would be a problem for XBMC. Dual
licensing would be better for us.

And just to be clear, dual licensing as LGPLv3+ *or* GPLv2+ would be ok.

> Regards,
> /Niels
>
> --
> Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
> Internet email is subject to wholesale government surveillance.
>



-- 
Regards,
Andres Mejia

From mcitadel at gmail.com  Fri Aug 26 23:43:21 2011
From: mcitadel at gmail.com (Andres Mejia)
Date: Fri, 26 Aug 2011 17:43:21 -0400
Subject: RIPEMD-160 Implementation
Message-ID: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>

Here's a patch that will implement RIPEMD-160 for nettle. It's code
ported from libgcrypt. A reference to RIPEMD-160 is in the source.
Also, this patch includes testcases and I followed what looked to me
as nettle coding conventions.

There's no documentation update in this patch. I didn't look into how
documentation was generated in nettle.

Also, this patch is against the nettle-2.2 release. I couldn't get the
CVS snapshot of nettle building, particularly because of the
integration of LSH.

-- 
Regards,
Andres Mejia
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rmd160-implementation.patch
Type: text/x-diff
Size: 25663 bytes
Desc: not available
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110826/da79cecf/attachment.patch>

From nisse at lysator.liu.se  Sat Aug 27 21:05:09 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 27 Aug 2011 21:05:09 +0200
Subject: RIPEMD-160 Implementation
In-Reply-To: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
	(Andres Mejia's message of "Fri, 26 Aug 2011 17:43:21 -0400")
References: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
Message-ID: <nnr546iska.fsf@stalhein.lysator.liu.se>

Andres Mejia <mcitadel at gmail.com> writes:

> Here's a patch that will implement RIPEMD-160 for nettle. It's code
> ported from libgcrypt.

Thanks!

> There's no documentation update in this patch. I didn't look into how
> documentation was generated in nettle.

The documentation "master file" is nettle.texinfo. info, html and pdf
files are all generated from that.

> Also, this patch is against the nettle-2.2 release. I couldn't get the
> CVS snapshot of nettle building, particularly because of the
> integration of LSH.

Did you follow the instructions? If so, I'd be interested to hear how it
failed. Anyway, basing patches on the 2.2 release is fine too.

Some initial comments on the implementation added below. Mostly cosmetic. Let
me know if you plan to make an updated patch, or if I should address
this myself when I intergrate the code.

> +		 rmd160.c rmd160-compress.c rmd160-meta.c \

Naming: Is "rmd" a commonly used abbreviation? Otherwise, I think I'd
prefer to write out "ripemd" both in filenames and C symbols.

Are any other variants of ripemd in use? (According to wikipedia,
there's original ripemd, ripemd-128, ripend-160, ripemd-256 and
ripemd-320, but I have no idea which of them are in use today or are
likely to be used in the future).

> --- /dev/null
> +++ b/rmd160-compress.c
> @@ -0,0 +1,277 @@
> +/* rmd160-compress.c  -  RIPE-MD160 (Transform function)
> + * Copyright (C) 1998, 2001, 2002, 2003 Free Software Foundation, Inc.
> + *
> + * The nettle library is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published by
> + * the Free Software Foundation; either version 2.1 of the License, or (at your
> + * option) any later version.
> + *
> + * The nettle library is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
> + * or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Lesser General Public
> + * License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public License
> + * along with the nettle library; see the file COPYING.LIB.  If not, write to
> + * the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston,
> + * MA 02111-1307, USA.
> + */
> +
> +#include <string.h>
> +
> +#include "rmd.h"
> +
> +/****************
> + * Rotate the 32 bit unsigned integer X by N bits left/right
> + */
> +#if defined(__GNUC__) && defined(__i386__)
> +static inline uint32_t
> +rol(uint32_t x, int n)
> +{
> +  __asm__("roll %%cl,%0"
> +    :"=r" (x)
> +    :"0" (x),"c" (n));
> +  return x;
> +}
> +#else
> +#define rol(x,n) ( ((x) << (n)) | ((x) >> (32-(n))) )
> +#endif

I think current gcc may recognize (x << n) | (x >> (wordsize - n)) and
generate a rotate instruction (but I haven't tested). I don't usually
put very machine dependent things like this in the C source files; if
performance for this function is that important, one can write an
assembler implementation of the compression function.

> +/****************
> + * Transform the message X which consists of 16 32-bit-words
> + */
> +void
> +_nettle_rmd160_compress(uint32_t *state, const uint8_t *data)
> +{
> +  register uint32_t a,b,c,d,e;
> +  uint32_t aa,bb,cc,dd,ee,t;
> +#ifdef WORDS_BIGENDIAN
> +  uint32_t x[16];
> +  {
> +    int i;
> +    uint8_t *p2, *p1;
> +    for (i=0, p1=data, p2=(uint8_t*)x; i < 16; i++, p2 += 4 )
> +    {
> +      p2[3] = *p1++;
> +      p2[2] = *p1++;
> +      p2[1] = *p1++;
> +      p2[0] = *p1++;
> +    }
> +  }

I'd just use READ_UINT32. I'd expect that explicit shifting is faster
than repeated byte writes to memory.

I would also do the little endian special case (with memcpy) only if
memcpy really is measured to be faster (it may well be, I can't guess).

> --- /dev/null
> +++ b/rmd160-meta.c
> @@ -0,0 +1,32 @@
> +/* rmd160-meta.c */
> +
> +/* nettle, low-level cryptographics library
> + *
> + * Copyright (C) 2002 Niels M?ller

This copyright notice seems to be wrong (even if the contents of this
file is barely copyrightable).

> + * RIPEMD-160 is not patented, see (as of 25.10.97)

I'd prefer to write the date either in words, or using ISO-format
yyyy-mm-dd (1997-10-25).

> + *   http://www.esat.kuleuven.ac.be/~bosselae/ripemd160.html
> + * Note that the code uses Little Endian byteorder, which is good for
> + * 386 etc, but we must add some conversion when used on a big endian box.

Would it be possible to write efficient code using big-endian byteorder
for all intermediate values? My initial guess is no: even if we use
inverse byteorder for input, output, and constants, the rotate
operations will be a problem, and that will kill any performance gain
from omitted byteswapping of inputs and outputs.

> +/* The routine terminates the computation */
> +static void
> +rmd160_final(struct rmd160_ctx *ctx)
> +{
> +  uint32_t t, msb, lsb;
> +  uint8_t *p;
> +
> +  rmd160_update(ctx, 0, NULL); /* flush */;
> +
> +  t = ctx->nblocks;
> +  /* multiply by 64 to make a byte count */
> +  lsb = t << 6;
> +  msb = t >> 26;
> +  /* add the count */
> +  t = lsb;
> +  if( (lsb += ctx->index) < t )
> +    msb++;
> +  /* multiply by 8 to make a bit count */
> +  t = lsb;
> +  lsb <<= 3;
> +  msb <<= 3;
> +  msb |= t >> 29;

If it's really a 64-bit bit count, then I think the context struct needs
a larger counter. Also the logic for adding the counter and padding
could perhaps be borrowed from the corresponding sha1 or md5 code (but I
haven't read the ripemd160 spec, so maybe it's really doing something
different).

> +  /* append the 64 bit count */
> +  ctx->block[56] = lsb;
> +  ctx->block[57] = lsb >>  8;
> +  ctx->block[58] = lsb >> 16;
> +  ctx->block[59] = lsb >> 24;
> +  ctx->block[60] = msb;
> +  ctx->block[61] = msb >>  8;
> +  ctx->block[62] = msb >> 16;
> +  ctx->block[63] = msb >> 24;

Use WRITE_UINT32 (or one could have the input to the compression
function by an array of uint32_t rather than uint8_t, but that has the
drawback that an assembler implementation of the compression function
can no longer optimize the byteswapping of the input).

> +  _nettle_rmd160_compress(ctx->digest, ctx->block);
> +
> +  p = ctx->block;
> +#ifdef WORDS_BIGENDIAN
> +#define X(a) do { *p++ = ctx->digest[a]    ; *p++ = ctx->digest[a] >> 8; \
> +  *p++ = ctx->digest[a] >> 16; *p++ = ctx->digest[a] >> 24; } while(0)
> +#else /* little endian */
> +#define X(a) do { *(uint32_t*)p = ctx->digest[a] ; p += 4; } while(0)
> +#endif

This looks more obscure than it should be. I think it would be better to
have the _final function not do any byteswapping, but leave it to the
_digest function (which also knows how much data is needed, and which is
the function responsible for converting the internal state to a byte
sequence). And then write a new _nettle_write_le32 function (analogous
to write-be32.c:_nettle_write_be32) and share it with md5_digest.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From mcitadel at gmail.com  Sat Aug 27 22:05:48 2011
From: mcitadel at gmail.com (Andres Mejia)
Date: Sat, 27 Aug 2011 16:05:48 -0400
Subject: RIPEMD-160 Implementation
In-Reply-To: <nnr546iska.fsf@stalhein.lysator.liu.se>
References: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
	<nnr546iska.fsf@stalhein.lysator.liu.se>
Message-ID: <CALgfg4j1uKb+9yOY0AQzJ49wRocGdqkomaEdQV5yrU4E=GbRhA@mail.gmail.com>

On Sat, Aug 27, 2011 at 3:05 PM, Niels M?ller <nisse at lysator.liu.se> wrote:
> Andres Mejia <mcitadel at gmail.com> writes:
>
>> Here's a patch that will implement RIPEMD-160 for nettle. It's code
>> ported from libgcrypt.
>
> Thanks!
>
>> There's no documentation update in this patch. I didn't look into how
>> documentation was generated in nettle.
>
> The documentation "master file" is nettle.texinfo. info, html and pdf
> files are all generated from that.
>
>> Also, this patch is against the nettle-2.2 release. I couldn't get the
>> CVS snapshot of nettle building, particularly because of the
>> integration of LSH.
>
> Did you follow the instructions? If so, I'd be interested to hear how it
> failed. Anyway, basing patches on the 2.2 release is fine too.

It was when I tried to run autoreconf. I'll try again with the instructions.

> Some initial comments on the implementation added below. Mostly cosmetic. Let
> me know if you plan to make an updated patch, or if I should address
> this myself when I intergrate the code.
>
>> + ? ? ? ? ? ? ?rmd160.c rmd160-compress.c rmd160-meta.c \
>
> Naming: Is "rmd" a commonly used abbreviation? Otherwise, I think I'd
> prefer to write out "ripemd" both in filenames and C symbols.

Either is fine by me.

> Are any other variants of ripemd in use? (According to wikipedia,
> there's original ripemd, ripemd-128, ripend-160, ripemd-256 and
> ripemd-320, but I have no idea which of them are in use today or are
> likely to be used in the future).

RIPEMD-160 was the only implementation I found in libgcrypt. I'm not
aware if any of the other algorithms are in use.

>> --- /dev/null
>> +++ b/rmd160-compress.c
>> @@ -0,0 +1,277 @@
>> +/* rmd160-compress.c ?- ?RIPE-MD160 (Transform function)
>> + * Copyright (C) 1998, 2001, 2002, 2003 Free Software Foundation, Inc.
>> + *
>> + * The nettle library is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU Lesser General Public License as published by
>> + * the Free Software Foundation; either version 2.1 of the License, or (at your
>> + * option) any later version.
>> + *
>> + * The nettle library is distributed in the hope that it will be useful, but
>> + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
>> + * or FITNESS FOR A PARTICULAR PURPOSE. ?See the GNU Lesser General Public
>> + * License for more details.
>> + *
>> + * You should have received a copy of the GNU Lesser General Public License
>> + * along with the nettle library; see the file COPYING.LIB. ?If not, write to
>> + * the Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston,
>> + * MA 02111-1307, USA.
>> + */
>> +
>> +#include <string.h>
>> +
>> +#include "rmd.h"
>> +
>> +/****************
>> + * Rotate the 32 bit unsigned integer X by N bits left/right
>> + */
>> +#if defined(__GNUC__) && defined(__i386__)
>> +static inline uint32_t
>> +rol(uint32_t x, int n)
>> +{
>> + ?__asm__("roll %%cl,%0"
>> + ? ?:"=r" (x)
>> + ? ?:"0" (x),"c" (n));
>> + ?return x;
>> +}
>> +#else
>> +#define rol(x,n) ( ((x) << (n)) | ((x) >> (32-(n))) )
>> +#endif
>
> I think current gcc may recognize (x << n) | (x >> (wordsize - n)) and
> generate a rotate instruction (but I haven't tested). I don't usually
> put very machine dependent things like this in the C source files; if
> performance for this function is that important, one can write an
> assembler implementation of the compression function.
>
>> +/****************
>> + * Transform the message X which consists of 16 32-bit-words
>> + */
>> +void
>> +_nettle_rmd160_compress(uint32_t *state, const uint8_t *data)
>> +{
>> + ?register uint32_t a,b,c,d,e;
>> + ?uint32_t aa,bb,cc,dd,ee,t;
>> +#ifdef WORDS_BIGENDIAN
>> + ?uint32_t x[16];
>> + ?{
>> + ? ?int i;
>> + ? ?uint8_t *p2, *p1;
>> + ? ?for (i=0, p1=data, p2=(uint8_t*)x; i < 16; i++, p2 += 4 )
>> + ? ?{
>> + ? ? ?p2[3] = *p1++;
>> + ? ? ?p2[2] = *p1++;
>> + ? ? ?p2[1] = *p1++;
>> + ? ? ?p2[0] = *p1++;
>> + ? ?}
>> + ?}
>
> I'd just use READ_UINT32. I'd expect that explicit shifting is faster
> than repeated byte writes to memory.
>
> I would also do the little endian special case (with memcpy) only if
> memcpy really is measured to be faster (it may well be, I can't guess).
>
>> --- /dev/null
>> +++ b/rmd160-meta.c
>> @@ -0,0 +1,32 @@
>> +/* rmd160-meta.c */
>> +
>> +/* nettle, low-level cryptographics library
>> + *
>> + * Copyright (C) 2002 Niels M?ller
>
> This copyright notice seems to be wrong (even if the contents of this
> file is barely copyrightable).
>
>> + * RIPEMD-160 is not patented, see (as of 25.10.97)
>
> I'd prefer to write the date either in words, or using ISO-format
> yyyy-mm-dd (1997-10-25).
>
>> + * ? http://www.esat.kuleuven.ac.be/~bosselae/ripemd160.html
>> + * Note that the code uses Little Endian byteorder, which is good for
>> + * 386 etc, but we must add some conversion when used on a big endian box.
>
> Would it be possible to write efficient code using big-endian byteorder
> for all intermediate values? My initial guess is no: even if we use
> inverse byteorder for input, output, and constants, the rotate
> operations will be a problem, and that will kill any performance gain
> from omitted byteswapping of inputs and outputs.
>
>> +/* The routine terminates the computation */
>> +static void
>> +rmd160_final(struct rmd160_ctx *ctx)
>> +{
>> + ?uint32_t t, msb, lsb;
>> + ?uint8_t *p;
>> +
>> + ?rmd160_update(ctx, 0, NULL); /* flush */;
>> +
>> + ?t = ctx->nblocks;
>> + ?/* multiply by 64 to make a byte count */
>> + ?lsb = t << 6;
>> + ?msb = t >> 26;
>> + ?/* add the count */
>> + ?t = lsb;
>> + ?if( (lsb += ctx->index) < t )
>> + ? ?msb++;
>> + ?/* multiply by 8 to make a bit count */
>> + ?t = lsb;
>> + ?lsb <<= 3;
>> + ?msb <<= 3;
>> + ?msb |= t >> 29;
>
> If it's really a 64-bit bit count, then I think the context struct needs
> a larger counter. Also the logic for adding the counter and padding
> could perhaps be borrowed from the corresponding sha1 or md5 code (but I
> haven't read the ripemd160 spec, so maybe it's really doing something
> different).
>
>> + ?/* append the 64 bit count */
>> + ?ctx->block[56] = lsb;
>> + ?ctx->block[57] = lsb >> ?8;
>> + ?ctx->block[58] = lsb >> 16;
>> + ?ctx->block[59] = lsb >> 24;
>> + ?ctx->block[60] = msb;
>> + ?ctx->block[61] = msb >> ?8;
>> + ?ctx->block[62] = msb >> 16;
>> + ?ctx->block[63] = msb >> 24;
>
> Use WRITE_UINT32 (or one could have the input to the compression
> function by an array of uint32_t rather than uint8_t, but that has the
> drawback that an assembler implementation of the compression function
> can no longer optimize the byteswapping of the input).
>
>> + ?_nettle_rmd160_compress(ctx->digest, ctx->block);
>> +
>> + ?p = ctx->block;
>> +#ifdef WORDS_BIGENDIAN
>> +#define X(a) do { *p++ = ctx->digest[a] ? ?; *p++ = ctx->digest[a] >> 8; \
>> + ?*p++ = ctx->digest[a] >> 16; *p++ = ctx->digest[a] >> 24; } while(0)
>> +#else /* little endian */
>> +#define X(a) do { *(uint32_t*)p = ctx->digest[a] ; p += 4; } while(0)
>> +#endif
>
> This looks more obscure than it should be. I think it would be better to
> have the _final function not do any byteswapping, but leave it to the
> _digest function (which also knows how much data is needed, and which is
> the function responsible for converting the internal state to a byte
> sequence). And then write a new _nettle_write_le32 function (analogous
> to write-be32.c:_nettle_write_be32) and share it with md5_digest.

All this code is ported from libgcrypt. I'll leave it to you what
modifications you want to do with the code.

> Regards,
> /Niels
>
> --
> Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
> Internet email is subject to wholesale government surveillance.
>

I'm curious, why are the internal transform functions exposed to the public?

-- 
Regards,
Andres Mejia

From nmav at gnutls.org  Sat Aug 27 22:19:35 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sat, 27 Aug 2011 22:19:35 +0200
Subject: RIPEMD-160 Implementation
Message-ID: <4E595157.9000809@gnutls.org>

On 08/27/2011 09:05 PM, Niels M?ller wrote:

> Are any other variants of ripemd in use? (According to wikipedia,
> there's original ripemd, ripemd-128, ripend-160, ripemd-256 and
> ripemd-320, but I have no idea which of them are in use today or are
> likely to be used in the future).

ripemd and ripemd-128 are considered broken. I don't know if ripemd-x,
x>=256 have been used anywhere. Having the SHA-3 winner instead would be
a much better choice.

regards,
Nikos

From nisse at lysator.liu.se  Sat Aug 27 23:19:43 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 27 Aug 2011 23:19:43 +0200
Subject: RIPEMD-160 Implementation
In-Reply-To: <CALgfg4j1uKb+9yOY0AQzJ49wRocGdqkomaEdQV5yrU4E=GbRhA@mail.gmail.com>
	(Andres Mejia's message of "Sat, 27 Aug 2011 16:05:48 -0400")
References: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
	<nnr546iska.fsf@stalhein.lysator.liu.se>
	<CALgfg4j1uKb+9yOY0AQzJ49wRocGdqkomaEdQV5yrU4E=GbRhA@mail.gmail.com>
Message-ID: <nnmxeuimc0.fsf@stalhein.lysator.liu.se>

Andres Mejia <mcitadel at gmail.com> writes:

> I'm curious, why are the internal transform functions exposed to the public?

As long as the compression function is only implemented in C, it can be
declared static and put in the same file as the _update and _final
functions which use it.

But when the compression function is implemented in assembly for some
platforms (there's currently sha1 assembly for x86 and x86_64, and md5
assembly for x86), it's moved to a separate file, so that configure can
decide to link in an assembly file overriding the C version. And then
the function is exposed like any other global function (but with a name
starting with _nettle to indicate that it is internal).

One could play with ELF visibility properties, but I haven't tried that
and it's not a general solution.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Sat Aug 27 23:24:18 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 27 Aug 2011 23:24:18 +0200
Subject: RIPEMD-160 Implementation
In-Reply-To: <4E595157.9000809@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sat, 27 Aug 2011 22:19:35 +0200")
References: <4E595157.9000809@gnutls.org>
Message-ID: <nnippiim4d.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> ripemd and ripemd-128 are considered broken. I don't know if ripemd-x,
> x>=256 have been used anywhere. Having the SHA-3 winner instead would be
> a much better choice.

Thanks. The related naming question is: Should the header file be
nettle/ripend160.h (declaring the single algorithm ripemd160) or
nettle/ripemd.h (declaring all supported hash functions in the family)?

Current include file naming is not 100% consistent, with sha.h, md4.h
and md5.h, and luckily it doesn't have to. Question is, what makes the
most sense for ripemd?

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Sat Aug 27 23:33:09 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sat, 27 Aug 2011 23:33:09 +0200
Subject: RIPEMD-160 Implementation
In-Reply-To: <nnippiim4d.fsf@stalhein.lysator.liu.se>
References: <4E595157.9000809@gnutls.org>
	<nnippiim4d.fsf@stalhein.lysator.liu.se>
Message-ID: <4E596295.8030500@gnutls.org>

On 08/27/2011 11:24 PM, Niels M?ller wrote:
> Nikos Mavrogiannopoulos<nmav at gnutls.org>  writes:
>
>> ripemd and ripemd-128 are considered broken. I don't know if ripemd-x,
>> x>=256 have been used anywhere. Having the SHA-3 winner instead would be
>> a much better choice.
>
> Thanks. The related naming question is: Should the header file be
> nettle/ripend160.h (declaring the single algorithm ripemd160) or
> nettle/ripemd.h (declaring all supported hash functions in the family)?

I'd use ripemd160.h to avoid confusion with the broken ripemd.

regards,
Nikos

From nmav at gnutls.org  Sat Aug 27 23:35:35 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sat, 27 Aug 2011 23:35:35 +0200
Subject: make in freebsd
Message-ID: <4E596327.5080100@gnutls.org>

Hello,
  I tried to do a "./configure && make" in freebsd an got an error 
during make. If I use GNU make instead it works. I attach the error 
output from the make command and the generated Makefile.

regards,
Nikos

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: out.txt
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110827/6d024a77/attachment.txt>

From nisse at lysator.liu.se  Sun Aug 28 00:06:54 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 28 Aug 2011 00:06:54 +0200
Subject: make in freebsd
In-Reply-To: <4E596327.5080100@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sat, 27 Aug 2011 23:35:35 +0200")
References: <4E596327.5080100@gnutls.org>
Message-ID: <nnbovaik5d.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

>  I tried to do a "./configure && make" in freebsd an got an error
> during make. If I use GNU make instead it works. I attach the error
> output from the make command and the generated Makefile.

I think the problem is that bsd make doesn't allow multiple files to b
listed for a single include command. If you want to use bsd make, you
have to configure with --disable-dependency-tracking. ./configure --help
says

  --disable-dependency-tracking
                            Disable dependency tracking. Dependency tracking
			    doesn't work with BSD make

Any suggestions other places to document that problem?

The Makefile.in does

DEP_FILES = $(SOURCES:.c=.$(OBJEXT).d) $(SOURCES:.c=.p$(OBJEXT).d)
@DEP_INCLUDE@ $(DEP_FILES)

and "include" or "#" is substituted for @DEP_INCLUDE@, depending on
whether or not dependency tracking is enabled.

Regards,
/Niels


-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Sun Aug 28 00:14:43 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Sun, 28 Aug 2011 00:14:43 +0200
Subject: make in freebsd
In-Reply-To: <nnbovaik5d.fsf@stalhein.lysator.liu.se>
References: <4E596327.5080100@gnutls.org>
	<nnbovaik5d.fsf@stalhein.lysator.liu.se>
Message-ID: <4E596C53.7060000@gnutls.org>

On 08/28/2011 12:06 AM, Niels M?ller wrote:

>>   I tried to do a "./configure&&  make" in freebsd an got an error
>> during make. If I use GNU make instead it works. I attach the error
>> output from the make command and the generated Makefile.
> I think the problem is that bsd make doesn't allow multiple files to b
> listed for a single include command. If you want to use bsd make, you
> have to configure with --disable-dependency-tracking. ./configure --help
> says

Why don't you use automake for that? The automake generated makefiles
seem to be quite portable across different make programs.


regards,
Nikos

From nisse at lysator.liu.se  Sun Aug 28 08:28:13 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 28 Aug 2011 08:28:13 +0200
Subject: make in freebsd
In-Reply-To: <4E596C53.7060000@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Sun, 28 Aug 2011 00:14:43 +0200")
References: <4E596327.5080100@gnutls.org>
	<nnbovaik5d.fsf@stalhein.lysator.liu.se> <4E596C53.7060000@gnutls.org>
Message-ID: <nn7h5yhwxu.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> Why don't you use automake for that?

People seem to be asking that every other month...

Because I don't think it's important to be portable to all strange make
programs (anyone can install gnu make). And because automake causes all
sorts of nasty problems if you need to have unusual make rules to work
together with the automake generated rules. I did use automake in nettle
(and lsh) some years back. E.g., the "minor" change of behavior in
automake to stop using suffix rules and instead generate one rule for
each object file totally broke the way nettle sets up building of
assembly files.

I'm *not* going to switch back to automake, and I'm not going to use
libtool either.

/nisse

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Sun Aug 28 20:31:46 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 28 Aug 2011 20:31:46 +0200
Subject: RIPEMD-160 Implementation
In-Reply-To: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
	(Andres Mejia's message of "Fri, 26 Aug 2011 17:43:21 -0400")
References: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
Message-ID: <nn39glie0d.fsf@stalhein.lysator.liu.se>

Andres Mejia <mcitadel at gmail.com> writes:

> Here's a patch that will implement RIPEMD-160 for nettle.

Checked in now, with just some renaming to use "ripemd160" as the prefix
for file names and symbols. I'd like to address the things I listed in a
previous mail before release.

I also added it to examples/nettle-benchmark, to guide the further
modifications.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From mcitadel at gmail.com  Sun Aug 28 20:38:59 2011
From: mcitadel at gmail.com (Andres Mejia)
Date: Sun, 28 Aug 2011 14:38:59 -0400
Subject: RIPEMD-160 Implementation
In-Reply-To: <nnmxeuimc0.fsf@stalhein.lysator.liu.se>
References: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
	<nnr546iska.fsf@stalhein.lysator.liu.se>
	<CALgfg4j1uKb+9yOY0AQzJ49wRocGdqkomaEdQV5yrU4E=GbRhA@mail.gmail.com>
	<nnmxeuimc0.fsf@stalhein.lysator.liu.se>
Message-ID: <CALgfg4gtp5MjQtuBuL=t4A3HNLwVMyEFH1XPxLZKswAODC=OOQ@mail.gmail.com>

On Sat, Aug 27, 2011 at 5:19 PM, Niels M?ller <nisse at lysator.liu.se> wrote:
> Andres Mejia <mcitadel at gmail.com> writes:
>
>> I'm curious, why are the internal transform functions exposed to the public?
>
> As long as the compression function is only implemented in C, it can be
> declared static and put in the same file as the _update and _final
> functions which use it.
>
> But when the compression function is implemented in assembly for some
> platforms (there's currently sha1 assembly for x86 and x86_64, and md5
> assembly for x86), it's moved to a separate file, so that configure can
> decide to link in an assembly file overriding the C version. And then
> the function is exposed like any other global function (but with a name
> starting with _nettle to indicate that it is internal).
>
> One could play with ELF visibility properties, but I haven't tried that
> and it's not a general solution.

You could declare the prototype in a private header, rmd_private.h for example.

> Regards,
> /Niels
>
> --
> Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
> Internet email is subject to wholesale government surveillance.
>



-- 
Regards,
Andres Mejia

From nisse at lysator.liu.se  Sun Aug 28 20:41:17 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sun, 28 Aug 2011 20:41:17 +0200
Subject: RIPEMD-160 Implementation
In-Reply-To: <nnr546iska.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Sat, 27 Aug 2011 21:05:09 +0200")
References: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
	<nnr546iska.fsf@stalhein.lysator.liu.se>
Message-ID: <nny5ydgz02.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> I think current gcc may recognize (x << n) | (x >> (wordsize - n)) and
> generate a rotate instruction (but I haven't tested).

Now I have tested. gcc 4.4.5 seems to recognize it, even without
optimization.

Test program:

  #include <stdint.h>
  
  uint32_t rot32(uint32_t x, int k)
  {
    return (x << k) | (x >> (32-k));
  }
  
  uint32_t rot64(uint64_t x, int k)
  {
    return (x << k) | (x >> (64-k));
  }
    
Compile with gcc -S -O0, and examine the generated assembler.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Mon Aug 29 20:54:49 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 29 Aug 2011 20:54:49 +0200
Subject: RIPEMD-160 Implementation
In-Reply-To: <nn39glie0d.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Sun, 28 Aug 2011 20:31:46 +0200")
References: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
	<nn39glie0d.fsf@stalhein.lysator.liu.se>
Message-ID: <nnd3fogi9y.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

>  I'd like to address the things I listed in a previous mail before
> release.

Now I have went through not only ripemd160, but all the supported hash
functions. All are of the Merkle-Damg?rd type (except md2, which uses
more stupid padding). I've tried to reduce the code duplication (in the
source code) using some helper macros, and generally improve consistency
a bit.

ripemd160.c got reduced to 44 non-comment lines, with all the complexity
in ripemd160-compress.c, as it should.

Ah, and one other question: Do you want me to add your name to the
copyright headers of the various ripemd160.c (I don't know how much
editing you had to do when porting it from libgcrypt).

Documentation for the ripemd160 algorithm remains to do.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From mcitadel at gmail.com  Thu Sep  1 17:29:07 2011
From: mcitadel at gmail.com (Andres Mejia)
Date: Thu, 1 Sep 2011 11:29:07 -0400
Subject: RIPEMD-160 Implementation
In-Reply-To: <nnd3fogi9y.fsf@stalhein.lysator.liu.se>
References: <CALgfg4hH0=j13bFg1+9X-kJcj+NDARcQ7Ja6MYCkxX+tn0u--Q@mail.gmail.com>
	<nn39glie0d.fsf@stalhein.lysator.liu.se>
	<nnd3fogi9y.fsf@stalhein.lysator.liu.se>
Message-ID: <CALgfg4jBQoHphiEJ_VQSf_da6G3F5YPBhUTncZdB7rke7TPAsw@mail.gmail.com>

On Aug 29, 2011 2:54 PM, "Niels M?ller" <nisse at lysator.liu.se> wrote:
>
> nisse at lysator.liu.se (Niels M?ller) writes:
>
> >  I'd like to address the things I listed in a previous mail before
> > release.
>
> Now I have went through not only ripemd160, but all the supported hash
> functions. All are of the Merkle-Damg?rd type (except md2, which uses
> more stupid padding). I've tried to reduce the code duplication (in the
> source code) using some helper macros, and generally improve consistency
> a bit.
>
> ripemd160.c got reduced to 44 non-comment lines, with all the complexity
> in ripemd160-compress.c, as it should.
>
> Ah, and one other question: Do you want me to add your name to the
> copyright headers of the various ripemd160.c (I don't know how much
> editing you had to do when porting it from libgcrypt).

I don't think my name should be included in the copyright. I mainly stripped
out libgcrypt specific code like gcry_burn_stack() and spelled out some
typedefs (u32 -> uint32_t).

> Documentation for the ripemd160 algorithm remains to do.
>
> Regards,
> /Niels
>
> --
> Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
> Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Fri Sep  2 20:21:36 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Fri, 02 Sep 2011 20:21:36 +0200
Subject: ANNOUNCE: Nettle-2.3
Message-ID: <nnliu6erf3.fsf@stalhein.lysator.liu.se>

I'm happy to announce a new version of GNU Nettle, a low-level
cryptographics library. The Nettle home page can be found at
http://www.lysator.liu.se/~nisse/nettle/. 

NEWS for the 2.3 release

	* Support for the ripemd-160 hash function.

	* Generates and installs nettle.pc and hogweed.pc files, for
          use with pkg-config. Feedback appreciated. For projects
          using autoconf, the traditional non-pkg-config ways of
          detecting libraries, and setting LIBS and LDFLAGS, is still
          recommended.

	* Fixed a bug which made the testsuite fail in the GCM test on
	  certain platforms. Should not affect any documented features
	  of the library.

	* Reorganization of the code for the various Merkle-Damg?rd
	  hash functions. Some fields in the context structs for md4,
	  md5 and sha1 have been renamed, for consistency.
	  Applications should not peek inside these structs, and the
	  ABI is unchanged.
	  
	* In the manual, fixed mis-placed const in certain function
          prototypes.

	The library is intended to be binary compatible with
	nettle-2.2. The shared library names are libnettle.so.4.2 and
	libhogweed.so.2.1, with sonames still libnettle.so.4 and
	libhogweed.so.2.

Available at

  http://www.lysator.liu.se/~nisse/archive/nettle-2.3.tar.gz

and soon also at

  ftp://ftp.lysator.liu.se/pub/security/lsh/nettle-2.3.tar.gz
  ftp://ftp.gnu.org/gnu/nettle/nettle-2.3.tar.gz

Happy hacking,
/Niels M?ller

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.


From nisse at lysator.liu.se  Sat Sep  3 07:59:52 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 03 Sep 2011 07:59:52 +0200
Subject: ANNOUNCE: Nettle-2.3
In-Reply-To: <nnliu6erf3.fsf@stalhein.lysator.liu.se> ("Niels
	=?iso-8859-1?Q?M=F6ller=22's?=
	message of "Fri, 02 Sep 2011 20:21:36 +0200")
References: <nnliu6erf3.fsf@stalhein.lysator.liu.se>
Message-ID: <nnhb4udv3b.fsf@stalhein.lysator.liu.se>

nisse at lysator.liu.se (Niels M?ller) writes:

> 	* Support for the ripemd-160 hash function.

Appears there was a missing #include "config.h" in the
ripemd160-compress.c file in the release, breaking ripemd160 on all
big-endian systems. Sorry for the poor testing...

I'll have to issue a new release soon.

/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nisse at lysator.liu.se  Sat Sep  3 15:49:39 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 03 Sep 2011 15:49:39 +0200
Subject: ANNOUNCE: Nettle-2.4
Message-ID: <nn8vq5enws.fsf@stalhein.lysator.liu.se>

Due to a bug discovered in Nettle-2.3, released yesterday, I'm a bit
embarrassed to announce a new release of GNU Nettle.

Nettle is a low-level cryptographic library. Homepage at
http://www.lysator.liu.se/~nisse/nettle/.

NEWS for the 2.4 release

	This is a bugfix release only. It turned out ripemd160 in the
	2.3 release was broken on all big-endian systems, due to a
	missing include of config.h. nettle-2.4 fixes this.

	The library is intended to be binary compatible with
	nettle-2.2 and nettle-2.3. The shared library names are
	libnettle.so.4.3 and libhogweed.so.2.1, with sonames still
	libnettle.so.4 and libhogweed.so.2.
	
NEWS for the 2.3 release

	* Support for the ripemd-160 hash function.

	* Generates and installs nettle.pc and hogweed.pc files, for
          use with pkg-config. Feedback appreciated. For projects
          using autoconf, the traditional non-pkg-config ways of
          detecting libraries, and setting LIBS and LDFLAGS, is still
          recommended.

	* Fixed a bug which made the testsuite fail in the GCM test on
	  certain platforms. Should not affect any documented features
	  of the library.

	* Reorganization of the code for the various Merkle-Damg?rd
	  hash functions. Some fields in the context structs for md4,
	  md5 and sha1 have been renamed, for consistency.
	  Applications should not peek inside these structs, and the
	  ABI is unchanged.
	  
	* In the manual, fixed mis-placed const in certain function
          prototypes.

	The library is intended to be binary compatible with
	nettle-2.2. The shared library names are libnettle.so.4.2 and
	libhogweed.so.2.1, with sonames still libnettle.so.4 and
	libhogweed.so.2.

The latest version is available at

  http://www.lysator.liu.se/~nisse/archive/nettle-2.4.tar.gz
  ftp://ftp.lysator.liu.se/pub/security/lsh/nettle-2.4.tar.gz
  ftp://ftp.gnu.org/gnu/nettle/nettle-2.4.tar.gz

Happy hacking,
/Niels m?ller

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.


From n.mavrogiannopoulos at gmail.com  Mon Sep 12 13:22:45 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Mon, 12 Sep 2011 13:22:45 +0200
Subject: memxor
Message-ID: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>

Hello,
 I've run some tests with memxor on a x86-64 machine. My results are:
* C implementation (compiled with gcc 4.4):
        Xoring in chunks of 32768 bytes: done. 50.09 Gb in 5.00 secs:
10.02 Gb/sec
        Xoring (unaligned) in chunks of 32768 bytes: done. 39.90 Gb in
5.00 secs: 7.98 Gb/sec

* ASM implementation:
        Xoring in chunks of 32768 bytes: done. 38.32 Gb in 5.00 secs:
7.66 Gb/sec
        Xoring (unaligned) in chunks of 32768 bytes: done. 30.16 Gb in
5.00 secs: 6.03 Gb/sec

It seems that in x86-64 the ASM version is slower than the C one.
Moreover I noticed that the loop unrolling techniques used in the C
code have no visible performance benefit.

However, an SSE2 version of memxor (attached) increases performance by
30% or more in the same CPU.

* SSE2:
        Xoring in chunks of 32768 bytes: done. 69.94 Gb in 5.00 secs:
13.98 Gb/sec
        Xoring (unaligned) in chunks of 32768 bytes: done. 65.96 Gb in
5.00 secs: 13.19 Gb/sec

regards,
Nikos
-------------- next part --------------
A non-text attachment was scrubbed...
Name: memxor2.c
Type: text/x-csrc
Size: 3141 bytes
Desc: not available
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110912/685b598b/attachment.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: speed.c
Type: text/x-csrc
Size: 4428 bytes
Desc: not available
URL: <http://lists.lysator.liu.se/pipermail/nettle-bugs/attachments/20110912/685b598b/attachment-0001.c>

From nisse at lysator.liu.se  Mon Sep 12 13:59:21 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 12 Sep 2011 13:59:21 +0200
Subject: memxor
In-Reply-To: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	(Nikos Mavrogiannopoulos's message of "Mon, 12 Sep 2011 13:22:45
	+0200")
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
Message-ID: <nnmxea7yzq.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <n.mavrogiannopoulos at gmail.com> writes:

> It seems that in x86-64 the ASM version is slower than the C one.

Hmm, that's different from what I have seen. This is what I get,
benchmarked on a 1.3 GHz Intel SU4100, by running
examples/nettle-benchmark -f 1.3e9 memxor.

C-implementation (compiled with gcc-4.4.5):

         Algorithm        mode Mbyte/s cycles/byte cycles/block
            memxor     aligned 4885.87        0.25         2.03
            memxor   unaligned 2771.14        0.45         3.58
           memxor3     aligned 4569.70        0.27         2.17
           memxor3 unaligned01 2528.03        0.49         3.92
           memxor3 unaligned11 2603.92        0.48         3.81
           memxor3 unaligned12 1496.87        0.83         6.63

x86_64 assembly:

         Algorithm        mode Mbyte/s cycles/byte cycles/block
            memxor     aligned 4895.18        0.25         2.03
            memxor   unaligned 3284.10        0.38         3.02
           memxor3     aligned 4890.66        0.25         2.03
           memxor3 unaligned01 3409.62        0.36         2.91
           memxor3 unaligned11 2697.37        0.46         3.68
           memxor3 unaligned12 2030.06        0.61         4.89

So no difference for the aligned case. I think two cycles per 64-bit
word ("block" above means unsigned long) is the memory bandwidth. While
for the different unaligned cases, the assembly version is a bit faster,
shaving off 0.5-2 cycles per word.

> Moreover I noticed that the loop unrolling techniques used in the C
> code have no visible performance benefit.

That's what I have seen as well. I keep the small amount of manual
unrolling for the benefit of other machines and/or compilers (but I'm
not sure where it really matters).

> However, an SSE2 version of memxor (attached) increases performance by
> 30% or more in the same CPU.

I'll have a look at that.

> * SSE2:
>         Xoring in chunks of 32768 bytes: done. 69.94 Gb in 5.00 secs:
> 13.98 Gb/sec
>         Xoring (unaligned) in chunks of 32768 bytes: done. 65.96 Gb in
> 5.00 secs: 13.19 Gb/sec

I'm a bit puzzled by your results, I didn't expect any speedup with sse2
instructions for the aligned case. What machine are you benchmarking on?

I'm also not quite sure what's the right way to think about memory
bandwidth. nettle-benchmark uses blocks of 10 KByte and processes the
same block repeatedly, which means that it ought to fit in L1 (or at
least L2) cache.

Regards,
/nisse

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Mon Sep 12 14:43:20 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Mon, 12 Sep 2011 14:43:20 +0200
Subject: memxor
In-Reply-To: <nnmxea7yzq.fsf@stalhein.lysator.liu.se>
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
Message-ID: <CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>

On Mon, Sep 12, 2011 at 1:59 PM, Niels M?ller <nisse at lysator.liu.se> wrote:

(2nd reply on list)

>> It seems that in x86-64 the ASM version is slower than the C one.
> Hmm, that's different from what I have seen. This is what I get,
> benchmarked on a 1.3 GHz Intel SU4100, by running
> examples/nettle-benchmark -f 1.3e9 memxor.

The CPU reports itself as Intel(R) Xeon(R) CPU X5670  @ 2.93GHz (the
system has 24 such cpus). The output of nettle-benchmark on that
machine follows.

x86-64 assembly:
[nikos at koninck examples]$ ./nettle-benchmark -f 1.3e9 memxor
sha1_compress: 463.00 cycles

benchmark call overhead: 0.001871 us   2.43 cycles

        Algorithm        mode Mbyte/s cycles/byte cycles/block
           memxor     aligned 11887.63        0.10         0.83
           memxor   unaligned 11194.60        0.11         0.89
          memxor3     aligned 11863.08        0.10         0.84
          memxor3 unaligned01 11167.71        0.11         0.89
          memxor3 unaligned11 6479.55        0.19         1.53
          memxor3 unaligned12 10540.27        0.12         0.94

C-implementation (gcc-4.1.2):
[nikos at koninck nettle-2.4]$ examples/nettle-benchmark -f 1.3e9 memxor
sha1_compress: 463.30 cycles

benchmark call overhead: 0.001872 us   2.43 cycles

        Algorithm        mode Mbyte/s cycles/byte cycles/block
           memxor     aligned 11854.78        0.10         0.84
           memxor   unaligned 11186.80        0.11         0.89
          memxor3     aligned 11896.14        0.10         0.83
          memxor3 unaligned01 11169.13        0.11         0.89
          memxor3 unaligned11 6437.96        0.19         1.54
          memxor3 unaligned12 10485.28        0.12         0.95

I see no big difference between them. However the results we see from
my and your benchmark vary. How do you benchmark? What is ncalls in
time_function()?

My benchmark is simplistic, it counts speed, number of memxors in a
fixed amount of time.

>> Moreover I noticed that the loop unrolling techniques used in the C
>> code have no visible performance benefit.
> That's what I have seen as well. I keep the small amount of manual
> unrolling for the benefit of other machines and/or compilers (but I'm
> not sure where it really matters).

My personal preference would have been cleaner code.

> I'm also not quite sure what's the right way to think about memory
> bandwidth. nettle-benchmark uses blocks of 10 KByte and processes the
> same block repeatedly, which means that it ought to fit in L1 (or at
> least L2) cache.

I saw no differences in my benchmark when I decreased the buffer to 10k.

regards,
Nikos

From nisse at lysator.liu.se  Mon Sep 12 19:03:03 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Mon, 12 Sep 2011 19:03:03 +0200
Subject: memxor
In-Reply-To: <CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	(Nikos Mavrogiannopoulos's message of "Mon, 12 Sep 2011 14:43:20
	+0200")
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
Message-ID: <nnehzl8zi0.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <n.mavrogiannopoulos at gmail.com> writes:

> The CPU reports itself as Intel(R) Xeon(R) CPU X5670  @ 2.93GHz (the
> system has 24 such cpus). The output of nettle-benchmark on that
> machine follows.
>
> x86-64 assembly:
> [nikos at koninck examples]$ ./nettle-benchmark -f 1.3e9 memxor

To get the printed cycle numbers to make sense, you have to pass the
correct clock frequency to the -f option. -f 2.93e9 in your case.

> However the results we see from my and your benchmark vary.

Right, we'll have to figure out why. I'm puzzled.

> How do you benchmark? What is ncalls in time_function()?

time_function loops around the benchmarked function ncalls times, and
reads the clock before and after the loop. Qnd then, if the elapsed time
was too short, it increases ncalls and starts over.

> My benchmark is simplistic, it counts speed, number of memxors in a
> fixed amount of time.

I guess that should be good enough. I'm not so familiar with SIGALARM,
but I don't seen anything obviously wrong with it.

>> That's what I have seen as well. I keep the small amount of manual
>> unrolling for the benefit of other machines and/or compilers (but I'm
>> not sure where it really matters).
>
> My personal preference would have been cleaner code.

Well, for the unaligned case, the unrolling is also a natural way to
avoid moving values between s1 and s0, which I think is nice.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Tue Sep 13 10:14:05 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Tue, 13 Sep 2011 10:14:05 +0200
Subject: memxor
In-Reply-To: <nnehzl8zi0.fsf@stalhein.lysator.liu.se>
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>
Message-ID: <CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>

On Mon, Sep 12, 2011 at 7:03 PM, Niels M?ller <nisse at lysator.liu.se> wrote:

> To get the printed cycle numbers to make sense, you have to pass the
> correct clock frequency to the -f option. -f 2.93e9 in your case.
>> However the results we see from my and your benchmark vary.
> Right, we'll have to figure out why. I'm puzzled.

It seems --disable-assembler doesn't work for memxor. That is the
figures were so similar with nettle-benchmark because only assembler
was used. To be honest I've not understood how the assembler thing
works so I am not able to disable it even manually.

regards,
Nikos

From n.mavrogiannopoulos at gmail.com  Tue Sep 13 11:10:19 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Tue, 13 Sep 2011 11:10:19 +0200
Subject: memxor
In-Reply-To: <CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>
	<CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
Message-ID: <CAJU7zaK9z1-XTAe3LAZ-auSXVRK3UdFENt_Lqa3zLmhR6O=wKQ@mail.gmail.com>

On Tue, Sep 13, 2011 at 10:14 AM, Nikos Mavrogiannopoulos
<n.mavrogiannopoulos at gmail.com> wrote:

Corrected figures for nettle-benchmark. My previous issue seems to
have been because a new ./configure doesn't really undo the previous
settings. SSE is faster than the previous implementations (asm or C),
but ASM performs better than C in the unaligned case. I cannot figure
out why my benchmark shows otherwise (our unaligned test seem to be
pretty much identical). I include the overhead that you subtract, but
seems to be identical in both cases.

* ASM
benchmark call overhead: 0.001862 us   5.46 cycles

         Algorithm        mode Mbyte/s cycles/byte cycles/block
            memxor     aligned 11980.56        0.23         1.87
            memxor   unaligned 11269.30        0.25         1.98

* C implementation:
benchmark call overhead: 0.001875 us   5.49 cycles

         Algorithm        mode Mbyte/s cycles/byte cycles/block
            memxor     aligned 11777.25        0.24         1.90
            memxor   unaligned 7794.15        0.36         2.87

* SSE2
benchmark call overhead: 0.001868 us   5.47 cycles

         Algorithm        mode Mbyte/s cycles/byte cycles/block
            memxor     aligned 15961.09        0.18         1.40
            memxor   unaligned 15882.32        0.18         1.41


regards,
Nikos

From nisse at lysator.liu.se  Tue Sep 13 11:33:58 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 13 Sep 2011 11:33:58 +0200
Subject: memxor
In-Reply-To: <CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
	(Nikos Mavrogiannopoulos's message of "Tue, 13 Sep 2011 10:14:05
	+0200")
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>
	<CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
Message-ID: <nnwrdc7pmh.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <n.mavrogiannopoulos at gmail.com> writes:

> It seems --disable-assembler doesn't work for memxor.

Maybe you forgot to run make distclean? You should do that whenever you
change any interesting configure options.

I usually don't build in the source directory, and than I can have
multiple build directories configured with different options. Currently
I have build trees with plain ./configure, ./configure
--disable-assembler, ./configure --enable-shared, and ./configure
CC="gcc -m32" CXX="g++ -m32".

> To be honest I've not understood how the assembler thing works so I am
> not able to disable it even manually.

I'll try to explain. It's not very complicated.

./configure looks for certain files in the appropriate machine-specific
directory, and creates symlinks in the top-level nettle build directory.
Then if make needs to build foo.o, and it can find both foo.c and
foo.asm, by the order of the suffix list, it will use the foo.asm file.

The rule to build foo.o from foo.asm first runs m4 to produce foo.s, and
then invokes the compiler $(CC) -c ... on foo.s.

The symlinks are created by config.status (which is also run at the end
of ./configure), and deleted by make distclean. If you create or delete
any links by hand, make will follow.

The setup is inspired by gmp. If you look at what gmp does, there are
two main differences:

 1. There's hierarchy of machine specific directories for different
    flavors of the same architecture, and configure searches in multiple
    directories.

 2. It doesn't rely on make ordering the suffix list. Instead, it puts
    all C files in the "mpn/generic" subdirectory, and creates symlinks
    also for the portable C files.

And then gmp also has the possibility of creating a fat binary,
including optimized code for all different flavors of the given
architecture.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Tue Sep 13 12:39:54 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Tue, 13 Sep 2011 12:39:54 +0200
Subject: memxor
In-Reply-To: <nnwrdc7pmh.fsf@stalhein.lysator.liu.se>
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>
	<CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
	<nnwrdc7pmh.fsf@stalhein.lysator.liu.se>
Message-ID: <CAJU7za+-N9HN=y2+KSKgkmitNTXcPNGzmgbwYqep24-S-VeCUQ@mail.gmail.com>

On Tue, Sep 13, 2011 at 11:33 AM, Niels M?ller <nisse at lysator.liu.se> wrote:

>> It seems --disable-assembler doesn't work for memxor.
> Maybe you forgot to run make distclean? You should do that whenever you
> change any interesting configure options.

Indeed. I've now figured out. It would be also better if configure.ac
was self contained.  If I modify it and run autoconf I get:
configure.ac:366: warning: AC_LANG_CONFTEST: no AC_LANG_SOURCE call
detected in body
autoconf/lang.m4:198: AC_LANG_CONFTEST is expanded from...
autoconf/general.m4:2599: _AC_COMPILE_IFELSE is expanded from...
autoconf/general.m4:2609: AC_COMPILE_IFELSE is expanded from...
../../lib/m4sugar/m4sh.m4:610: AS_IF is expanded from...
autoconf/general.m4:2047: AC_CACHE_VAL is expanded from...
autoconf/general.m4:2060: AC_CACHE_CHECK is expanded from...
configure.ac:366: the top level

if I run .bootstrap from the lsh dir it succeeds but running configure
prints the warnings below and make fails.
checking malloc.h presence... no
configure: WARNING: malloc.h: accepted by the compiler, rejected by
the preprocessor!
configure: WARNING: malloc.h: proceeding with the compiler's result

Anyway, I attach an untested patch that shows how SSE2 or other cpu
version specific optimizations can be enabled at run-time.

regards,
Nikos

From nisse at lysator.liu.se  Tue Sep 13 13:17:34 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 13 Sep 2011 13:17:34 +0200
Subject: memxor
In-Reply-To: <CAJU7za+-N9HN=y2+KSKgkmitNTXcPNGzmgbwYqep24-S-VeCUQ@mail.gmail.com>
	(Nikos Mavrogiannopoulos's message of "Tue, 13 Sep 2011 12:39:54
	+0200")
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>
	<CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
	<nnwrdc7pmh.fsf@stalhein.lysator.liu.se>
	<CAJU7za+-N9HN=y2+KSKgkmitNTXcPNGzmgbwYqep24-S-VeCUQ@mail.gmail.com>
Message-ID: <nnsjo07ktt.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <n.mavrogiannopoulos at gmail.com> writes:

> if I run .bootstrap from the lsh dir it succeeds but running configure
> prints the warnings below and make fails.

Strange. I'm not sure what the warnings mean, but make definitely
shouldn't fail. I take it you have checked it out from cvs? Exactly what
did you do? I'd expect the following to work:

  cvs ... co lsh
  cd lsh
  ./.bootstrap
  cd nettle
  ./configure
  make
 
Then you shouldn't need to bother about the lsh directory again. You
have a symlink to the shared aclocal.m4 (and to some other shared
files).

> Anyway, I attach an untested patch that shows how SSE2 or other cpu
> version specific optimizations can be enabled at run-time.

Thanks. I'll have to think some more on how to organize this. Some
properties I'd like to have:

1. Don't require users to call any init function.

   One could define memxor to jump via a function pointer, and have an
   initial value for that pointer which jumps to the routine to set the
   pointer to the right function, and then use it. Overwriting the
   pointer should be atomic, so no locking needed even for multithreaded
   programs.

   Or for library formats that support that, hook in the initialization
   in the same way as C++ constructors for global data.

2. Have configure options like

   --enable-x86-sse2/--disable-x86-sse2

   which omits that wrapper function and its function pointer.

3. Avoid using gcc-specific things, including inline asm, in the C
   source files.

Other obvious uses for cpu detection in nettle:

  * The AES code could check for the special aes instructions.

  * The serpent code can use %xmm and %ymm registers, when present. On
    x86_64, as far as I'm aware all current implementations have sse2,
    but one could check for, and make use of, the 256-bit %ymm
    registers.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Tue Sep 13 14:41:08 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Tue, 13 Sep 2011 14:41:08 +0200
Subject: memxor
In-Reply-To: <CAJU7zaK0vqpG=ON7rYWM-oP-Ar9d6SWshyy=+up6btWHipQuOw@mail.gmail.com>
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>
	<CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
	<nnwrdc7pmh.fsf@stalhein.lysator.liu.se>
	<CAJU7za+-N9HN=y2+KSKgkmitNTXcPNGzmgbwYqep24-S-VeCUQ@mail.gmail.com>
	<nnsjo07ktt.fsf@stalhein.lysator.liu.se>
	<CAJU7zaK0vqpG=ON7rYWM-oP-Ar9d6SWshyy=+up6btWHipQuOw@mail.gmail.com>
Message-ID: <CAJU7zaLnkTkeq1Vo0bwh_rywGFWCCVfEmLQCmShXXiq+RCWg-w@mail.gmail.com>

On Tue, Sep 13, 2011 at 1:17 PM, Niels M?ller <nisse at lysator.liu.se> wrote:

> Then you shouldn't need to bother about the lsh directory again. You
> have a symlink to the shared aclocal.m4 (and to some other shared
> files).

Ok I figured out. I attach a working SSE2 detection.

> Thanks. I'll have to think some more on how to organize this. Some
> properties I'd like to have:
> 1. Don't require users to call any init function.
> ? One could define memxor to jump via a function pointer, and have an
> ? initial value for that pointer which jumps to the routine to set the
> ? pointer to the right function, and then use it. Overwriting the
> ? pointer should be atomic, so no locking needed even for multithreaded
> ? programs.

I don't think locking is an issue if you only call a function on
initialization. You can expect (and require) that a library isn't
going to be initialized by multiple threads. I don't know however of a
portable way to do initialization transparently without an explicit
function call.

> 3. Avoid using gcc-specific things, including inline asm, in the C
> ? source files.

The cpuid test would have then to be moved to an assembly file.

> Other obvious uses for cpu detection in nettle:
> ?* The AES code could check for the special aes instructions.

Indeed. Once a framework for overwriting functionality is set, those would
be not very hard to add. However setting such framework in nettle seems to
require substantial work as all exported functions need to be replaced by
function pointers thus breaking ABI. If this is done gradually (it has
to, as you
never know what you would be able to optimize in a new processor) it would
be worse, since every optimization added would break ABI.

Maybe it makes sense to have a libgcrypt-like high level interface and
optimizations
would be used only there. The existing C api remains an API to access
the C implementation.
This could also address the problem with optimized hash algorithms[0],
since most cpu-assisted
sha1 or sha256 implementations work on an output=hash(data,length)
basis and do not map
to the existing API.

[0]. http://www.mail-archive.com/openssl-dev at openssl.org/msg21787.html

> ?* The serpent code can use %xmm and %ymm registers, when present. On
> ? ?x86_64, as far as I'm aware all current implementations have sse2,
> ? ?but one could check for, and make use of, the 256-bit %ymm
> ? ?registers.

I wouldn't care of serpent optimizations much :)

regards,
Nikos

From nisse at lysator.liu.se  Tue Sep 13 15:56:58 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 13 Sep 2011 15:56:58 +0200
Subject: memxor
In-Reply-To: <CAJU7zaLnkTkeq1Vo0bwh_rywGFWCCVfEmLQCmShXXiq+RCWg-w@mail.gmail.com>
	(Nikos Mavrogiannopoulos's message of "Tue, 13 Sep 2011 14:41:08
	+0200")
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>
	<CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
	<nnwrdc7pmh.fsf@stalhein.lysator.liu.se>
	<CAJU7za+-N9HN=y2+KSKgkmitNTXcPNGzmgbwYqep24-S-VeCUQ@mail.gmail.com>
	<nnsjo07ktt.fsf@stalhein.lysator.liu.se>
	<CAJU7zaK0vqpG=ON7rYWM-oP-Ar9d6SWshyy=+up6btWHipQuOw@mail.gmail.com>
	<CAJU7zaLnkTkeq1Vo0bwh_rywGFWCCVfEmLQCmShXXiq+RCWg-w@mail.gmail.com>
Message-ID: <nnobyo7dg5.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <n.mavrogiannopoulos at gmail.com> writes:

> I don't know however of a portable way to do initialization
> transparently without an explicit function call.

It works for gmp. Which doesn't imply that we should do it in exactly
the same way, of course.

> The cpuid test would have then to be moved to an assembly file.

Right.

> Indeed. Once a framework for overwriting functionality is set, those would
> be not very hard to add. However setting such framework in nettle seems to
> require substantial work as all exported functions need to be replaced by
> function pointers thus breaking ABI.

I don't think the function pointers should be exported. If "fat" library
is enabled (default for x86), then the exported function should be

  void memxor (...)
  {
    (*memxor_p)(...);
  }

I think one should have the possibility to choose between fat and
non-fat builds, with the same ABI. There's going to be a small extra
call overhead in the fat case.

As long as all implementations can use the same ctx structs, there
should be no problem with the ABI. If we also want to support hardware
acelerators which are like black boxes, then some API and or ABI changes
may be necessary.

> I wouldn't care of serpent optimizations much :)

I'm not surprised ;-) But on processors which lack aes-instructions, but
which have 256-bit %ymm-registers, serpent can most likely be twice as
fast as aes if used in ctr mode (for the current code with 128-bit
%xmm-registers, serpent speed was somewhere between aes-128 and aes-192
last time I measured).

/nisse

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From n.mavrogiannopoulos at gmail.com  Tue Sep 13 18:46:51 2011
From: n.mavrogiannopoulos at gmail.com (Nikos Mavrogiannopoulos)
Date: Tue, 13 Sep 2011 18:46:51 +0200
Subject: memxor
In-Reply-To: <nnobyo7dg5.fsf@stalhein.lysator.liu.se>
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>	<CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>	<nnwrdc7pmh.fsf@stalhein.lysator.liu.se>	<CAJU7za+-N9HN=y2+KSKgkmitNTXcPNGzmgbwYqep24-S-VeCUQ@mail.gmail.com>	<nnsjo07ktt.fsf@stalhein.lysator.liu.se>	<CAJU7zaK0vqpG=ON7rYWM-oP-Ar9d6SWshyy=+up6btWHipQuOw@mail.gmail.com>	<CAJU7zaLnkTkeq1Vo0bwh_rywGFWCCVfEmLQCmShXXiq+RCWg-w@mail.gmail.com>
	<nnobyo7dg5.fsf@stalhein.lysator.liu.se>
Message-ID: <4E6F88FB.30805@gmail.com>

On 09/13/2011 03:56 PM, Niels M?ller wrote:

>> Indeed. Once a framework for overwriting functionality is set, those would
>> be not very hard to add. However setting such framework in nettle seems to
>> require substantial work as all exported functions need to be replaced by
>> function pointers thus breaking ABI.
> I don't think the function pointers should be exported. If "fat" library
> is enabled (default for x86), then the exported function should be
>
>    void memxor (...)
>    {
>      (*memxor_p)(...);
>    }
> I think one should have the possibility to choose between fat and
> non-fat builds, with the same ABI. There's going to be a small extra
> call overhead in the fat case.

Disabling the optional architectures would be possible but reducing the
call overhead in the "thin" case would require a lot of ifdefs. It would 
keep the external api intact, but the internals would look ugly.

> As long as all implementations can use the same ctx structs, there
> should be no problem with the ABI. If we also want to support hardware
> acelerators which are like black boxes, then some API and or ABI changes
> may be necessary.

This is not guaranteed. For example AES-NI and padlock require the
AES key to be aligned to 16-byte boundaries, something that the
current structures do not offer.

>> I wouldn't care of serpent optimizations much :)
> I'm not surprised ;-) But on processors which lack aes-instructions, but
> which have 256-bit %ymm-registers, serpent can most likely be twice as
> fast as aes if used in ctr mode (for the current code with 128-bit
> %xmm-registers, serpent speed was somewhere between aes-128 and aes-192
> last time I measured).

Why not use camellia as an alternative? It is newer design than serpent
and is pretty much standardized as the aes alternative.

In any case, I just noticed that for the x86-64 you don't really need to 
detect SSE2, it is just there by default. So maybe the SSE2 xor can just 
replace the x86-64 xor. For the plain x86 though this is not the case.

regards,
Nikos

PS. The ECC patch just got very low in my priority stack. If anyone else 
is interested into porting it to nettle, he would make me a favor.

From nisse at lysator.liu.se  Wed Sep 14 08:27:26 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 14 Sep 2011 08:27:26 +0200
Subject: memxor
In-Reply-To: <4E6F88FB.30805@gmail.com> (Nikos Mavrogiannopoulos's message of
	"Tue, 13 Sep 2011 18:46:51 +0200")
References: <CAJU7za+iBV8S9cETF5fgCrrDzGRNb0bE77or-hBeK4Xg8rSFWA@mail.gmail.com>
	<nnmxea7yzq.fsf@stalhein.lysator.liu.se>
	<CAJU7zaJFO23Wcwm0C9T+JhSZ47D7sOiOzSL39VdDcTYc5Ts=BQ@mail.gmail.com>
	<nnehzl8zi0.fsf@stalhein.lysator.liu.se>
	<CAJU7zaKdvwXRWTn=W=05iE+vqxWeABYw4bJjkzsY7qN1bRe-nQ@mail.gmail.com>
	<nnwrdc7pmh.fsf@stalhein.lysator.liu.se>
	<CAJU7za+-N9HN=y2+KSKgkmitNTXcPNGzmgbwYqep24-S-VeCUQ@mail.gmail.com>
	<nnsjo07ktt.fsf@stalhein.lysator.liu.se>
	<CAJU7zaK0vqpG=ON7rYWM-oP-Ar9d6SWshyy=+up6btWHipQuOw@mail.gmail.com>
	<CAJU7zaLnkTkeq1Vo0bwh_rywGFWCCVfEmLQCmShXXiq+RCWg-w@mail.gmail.com>
	<nnobyo7dg5.fsf@stalhein.lysator.liu.se> <4E6F88FB.30805@gmail.com>
Message-ID: <nnd3f37i5t.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <n.mavrogiannopoulos at gmail.com> writes:

>> I think one should have the possibility to choose between fat and
>> non-fat builds, with the same ABI. There's going to be a small extra
>> call overhead in the fat case.
>
> Disabling the optional architectures would be possible but reducing the
> call overhead in the "thin" case would require a lot of ifdefs. It
> would keep the external api intact, but the internals would look ugly.

Even if fat is default on x86, the non-fat case is important for other
architectures. I think the complexity will be manageable, and most of it
will be in the configure script and assembly code, not in the C files.

> This is not guaranteed. For example AES-NI and padlock require the
> AES key to be aligned to 16-byte boundaries, something that the
> current structures do not offer.

At least that's an ABI change which is harmless for other
implementations. A different question is how to portably tell the C
compiler that a certain structure must be 16-byte aligned.

> In any case, I just noticed that for the x86-64 you don't really need
> to detect SSE2, it is just there by default.

That's my understanding as well. You can test for it, but it's present
in all existing x86_64 cpus.

/nisse

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From coel at coelmay.com  Sat Sep 17 03:18:17 2011
From: coel at coelmay.com (Coel May)
Date: Sat, 17 Sep 2011 11:18:17 +1000
Subject: make issues
Message-ID: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com>

Hi,

In the process of attempting to build gnutls under os x I get an error saying libnettle is not found. Ok.

So I download nettle-2.4.tar.gz, extract and run ./configure. No problems.

Then I run make. the result is below.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
$ make
make all-here
/usr/bin/m4 ./asm.m4 machine.m4 config.m4 \
		aes-decrypt-internal.asm >aes-decrypt-internal.s
gcc -I.  -DHAVE_CONFIG_H -g -O2 -ggdb3 -Wno-pointer-sign -Wall -W   -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes   -Wpointer-arith -Wbad-function-cast -Wnested-externs -fPIC -MT aes-decrypt-internal.o -MD -MP -MF aes-decrypt-internal.o.d -fPIC -c aes-decrypt-internal.s
aes-decrypt-internal.asm:109:suffix or operands invalid for `push'
aes-decrypt-internal.asm:110:suffix or operands invalid for `push'
aes-decrypt-internal.asm:111:suffix or operands invalid for `push'
aes-decrypt-internal.asm:112:suffix or operands invalid for `push'
aes-decrypt-internal.asm:328:suffix or operands invalid for `pop'
aes-decrypt-internal.asm:329:suffix or operands invalid for `pop'
aes-decrypt-internal.asm:330:suffix or operands invalid for `pop'
aes-decrypt-internal.asm:331:suffix or operands invalid for `pop'
make[1]: *** [aes-decrypt-internal.o] Error 1
make: *** [all] Error 2

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Am I missing something? I must be, right?

Any help appreciated.

coel may
e: coel at coelmay.com
w: http://coelmay.com
t: twitter.com/coelmay


From nisse at lysator.liu.se  Sat Sep 17 06:51:18 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Sat, 17 Sep 2011 06:51:18 +0200
Subject: make issues
In-Reply-To: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com> (Coel May's
	message of "Sat, 17 Sep 2011 11:18:17 +1000")
References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com>
Message-ID: <nnty8b3h6h.fsf@stalhein.lysator.liu.se>

Coel May <coel at coelmay.com> writes:

> So I download nettle-2.4.tar.gz, extract and run ./configure. No problems.
> Then I run make. the result is below.
> [...]
> aes-decrypt-internal.asm:331:suffix or operands invalid for `pop'
> make[1]: *** [aes-decrypt-internal.o] Error 1
> make: *** [all] Error 2

You don't say which architecture you are running on, but I suspect you
have an x86_x64, and then nettle build tries to use x86 assemler files
(instead of the x86_64 files). The problem is that the config.guess
script shipped with nettle-2.4 is too old and doesn't recognize os x
on x86_64. Check what ./config.guess outputs if you just run it from the
shell.

You could try replacing config.guess with the latest version from
http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD,
and then run configure again.

I've had mixed reports, that worked fine for one user but failed for
another. As a workaround, you could also try to configure with
--disable-assembler.

In any either, if you're using the same build tree, you must first run
make distclean.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From coel at coelmay.com  Sat Sep 17 07:34:41 2011
From: coel at coelmay.com (Coel May)
Date: Sat, 17 Sep 2011 15:34:41 +1000
Subject: make issues
In-Reply-To: <nnty8b3h6h.fsf@stalhein.lysator.liu.se>
References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com>
	<nnty8b3h6h.fsf@stalhein.lysator.liu.se>
Message-ID: <E2AD709D-019C-4771-B65A-B9D10C0FA8B7@coelmay.com>

On 17/09/2011, at 2:51 PM, Niels M?ller wrote:

> You could try replacing config.guess with the latest version from
> http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD,
> and then run configure again.

Downloading the config.guess and replacing the original did the trick. And yes, x86_64 architecture.

Thanks for the help and quick response Niels

Regards
coel may
e: coel at coelmay.com
w: http://coelmay.com
t: twitter.com/coelmay


From derek at orange-pants.com  Mon Sep 26 19:11:24 2011
From: derek at orange-pants.com (Derek Downey)
Date: Mon, 26 Sep 2011 17:11:24 +0000 (UTC)
Subject: make issues
References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com>
	<nnty8b3h6h.fsf@stalhein.lysator.liu.se>
	<E2AD709D-019C-4771-B65A-B9D10C0FA8B7@coelmay.com>
Message-ID: <loom.20110926T191010-804@post.gmane.org>

Coel May <coel at ...> writes:

> Downloading the config.guess and replacing the original did the trick. 
> 


I can confirm this worked for me as well


From nettle-2011 at ryandesign.com  Sat Oct 15 03:10:22 2011
From: nettle-2011 at ryandesign.com (Ryan Schmidt)
Date: Fri, 14 Oct 2011 20:10:22 -0500
Subject: PATCH: Build proper Mac OS X dynamic libraries (dylibs)
Message-ID: <81BD25CF-0963-419E-A780-115ABAD89ECB@ryandesign.com>

Hello,

I've patched nettle in MacPorts to build proper Mac OS X dynamic libraries, and wanted to share the patch with you.


From nettle-2011 at ryandesign.com  Sat Oct 15 03:28:35 2011
From: nettle-2011 at ryandesign.com (Ryan Schmidt)
Date: Fri, 14 Oct 2011 20:28:35 -0500
Subject: PATCH: Build proper Mac OS X dynamic libraries (dylibs)
In-Reply-To: <81BD25CF-0963-419E-A780-115ABAD89ECB@ryandesign.com>
References: <81BD25CF-0963-419E-A780-115ABAD89ECB@ryandesign.com>
Message-ID: <52F6B123-B44A-49E8-BCC9-5A86D6E0B7E9@ryandesign.com>


On Oct 14, 2011, at 20:10, Ryan Schmidt wrote:

> I've patched nettle in MacPorts to build proper Mac OS X dynamic libraries, and wanted to share the patch with you.

Since your mailing list software appears to silently strip out attachments, I've posted it here:

http://www.ryandesign.com/tmp/nettle-configure.ac.diff




From nisse at lysator.liu.se  Tue Oct 18 20:32:28 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 18 Oct 2011 20:32:28 +0200
Subject: PATCH: Build proper Mac OS X dynamic libraries (dylibs)
In-Reply-To: <52F6B123-B44A-49E8-BCC9-5A86D6E0B7E9@ryandesign.com> (Ryan
	Schmidt's message of "Fri, 14 Oct 2011 20:28:35 -0500")
References: <81BD25CF-0963-419E-A780-115ABAD89ECB@ryandesign.com>
	<52F6B123-B44A-49E8-BCC9-5A86D6E0B7E9@ryandesign.com>
Message-ID: <nnk4826thf.fsf@stalhein.lysator.liu.se>

Ryan Schmidt <nettle-2011 at ryandesign.com> writes:

>> I've patched nettle in MacPorts to build proper Mac OS X dynamic
>> libraries, and wanted to share the patch with you.

Thanks. One question: You add $(LIBS) to LIBNETTLE_LIBS (and to
LIBHOGWEED_LIBS). Why, and what do you expect $(LIBS) to contain? It
usually contains -lgmp (if configure finds the gmp library), which
shouldn't be needed by the dynamic nettle library, and which is included
explicitly in LIBHOGWEED_LIBS.

> Since your mailing list software appears to silently strip out attachments,

Might depend on the content-type of the attachment. I'm not really
friends with mailman configuration. Sorry for the inconvenience. It's
also perfectly ok to post patches inline (unless perhaps if they're
really huge).

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nettle-2011 at ryandesign.com  Tue Oct 18 20:55:20 2011
From: nettle-2011 at ryandesign.com (Ryan Schmidt)
Date: Tue, 18 Oct 2011 13:55:20 -0500
Subject: PATCH: Build proper Mac OS X dynamic libraries (dylibs)
In-Reply-To: <nnk4826thf.fsf@stalhein.lysator.liu.se>
References: <81BD25CF-0963-419E-A780-115ABAD89ECB@ryandesign.com>
	<52F6B123-B44A-49E8-BCC9-5A86D6E0B7E9@ryandesign.com>
	<nnk4826thf.fsf@stalhein.lysator.liu.se>
Message-ID: <4B40B7FD-5D2A-4CB8-BA92-CB52F7783C80@ryandesign.com>


On Oct 18, 2011, at 13:32, Niels M?ller wrote:

> Ryan Schmidt writes:
> 
>>> I've patched nettle in MacPorts to build proper Mac OS X dynamic
>>> libraries, and wanted to share the patch with you.
> 
> Thanks. One question: You add $(LIBS) to LIBNETTLE_LIBS (and to
> LIBHOGWEED_LIBS). Why, and what do you expect $(LIBS) to contain? It
> usually contains -lgmp (if configure finds the gmp library), which
> shouldn't be needed by the dynamic nettle library, and which is included
> explicitly in LIBHOGWEED_LIBS.

I expected $(LIBS) to be empty, but I added it for consistency with the mingw32 and cygwin cases. But it's not in the solaris or default cases so I don't know if that was correct. I figured somebody might want to pass in additional libs in the LIBS variable some day. It is listed in "./configure --help" as an "influential environment variable" so I wanted its influence to be respected in the event that it is used.


>> Since your mailing list software appears to silently strip out attachments,
> 
> Might depend on the content-type of the attachment. I'm not really
> friends with mailman configuration. Sorry for the inconvenience. It's
> also perfectly ok to post patches inline (unless perhaps if they're
> really huge).

I've seen mail programs and mailing list software strip leading whitespace which pretty much destroys patches pasted inline so I try to refrain from doing that.





From nisse at lysator.liu.se  Tue Oct 18 21:28:47 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Tue, 18 Oct 2011 21:28:47 +0200
Subject: PATCH: Build proper Mac OS X dynamic libraries (dylibs)
In-Reply-To: <4B40B7FD-5D2A-4CB8-BA92-CB52F7783C80@ryandesign.com> (Ryan
	Schmidt's message of "Tue, 18 Oct 2011 13:55:20 -0500")
References: <81BD25CF-0963-419E-A780-115ABAD89ECB@ryandesign.com>
	<52F6B123-B44A-49E8-BCC9-5A86D6E0B7E9@ryandesign.com>
	<nnk4826thf.fsf@stalhein.lysator.liu.se>
	<4B40B7FD-5D2A-4CB8-BA92-CB52F7783C80@ryandesign.com>
Message-ID: <nn7h426qvk.fsf@stalhein.lysator.liu.se>

Ryan Schmidt <nettle-2011 at ryandesign.com> writes:

> I expected $(LIBS) to be empty, but I added it for consistency with
> the mingw32 and cygwin cases. But it's not in the solaris or default
> cases so I don't know if that was correct.

I see. The default (mainly linux and bsd) and solaris cases have more
solid testing. I think it's the mingw32 and cygwin cases which get it
wrong (if -l flags when building a dll have any effect at all there; I
don't know).

I've now applied the patch, deleted those occurences of $(LIBS), and
committed the result.

Thanks,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Tue Oct 25 19:06:36 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Tue, 25 Oct 2011 19:06:36 +0200
Subject: _nettle_write_be32
Message-ID: <4EA6EC9C.6090307@gnutls.org>

Hello,
 Would it be possible to export _nettle_write_be32? It is a useful
function and can be used on the accelerated versions of SHA in
architectures that support it (i.e. VIA cpus). I currently include a
copy of it, but would be nice if it was exported from the nettle API as
well.

btw. the MD_* macros were also proved to be very useful in that case.

regards,
Nikos

From nisse at lysator.liu.se  Wed Oct 26 10:16:21 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 26 Oct 2011 10:16:21 +0200
Subject: _nettle_write_be32
In-Reply-To: <4EA6EC9C.6090307@gnutls.org> (Nikos Mavrogiannopoulos's message
	of "Tue, 25 Oct 2011 19:06:36 +0200")
References: <4EA6EC9C.6090307@gnutls.org>
Message-ID: <nnd3dk5fsa.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

>  Would it be possible to export _nettle_write_be32?

This function is simple enough that it can be documented and made
public, if people find it useful.

> It is a useful function and can be used on the accelerated versions of
> SHA in architectures that support it (i.e. VIA cpus).

BTW, how does that acceleration work? Would it be possible/practical to
write a function matching the current _nettle_sha1_compress.

> btw. the MD_* macros were also proved to be very useful in that case.

These macros are really intended for internal use only. I'd prefer to
not export any interfaces like that. If functionality like this should
be exported at all, I'd be more comfortable with a function interface
for doing these things.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Wed Oct 26 10:30:45 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Wed, 26 Oct 2011 10:30:45 +0200
Subject: _nettle_write_be32
In-Reply-To: <nnd3dk5fsa.fsf@stalhein.lysator.liu.se>
References: <4EA6EC9C.6090307@gnutls.org>
	<nnd3dk5fsa.fsf@stalhein.lysator.liu.se>
Message-ID: <CAJU7za+YTQBzT-W1dfD-K52HNONb9=mZaWg0ad+P_y31ujYjnQ@mail.gmail.com>

On Wed, Oct 26, 2011 at 10:16 AM, Niels M?ller <nisse at lysator.liu.se> wrote:

>> It is a useful function and can be used on the accelerated versions of
>> SHA in architectures that support it (i.e. VIA cpus).
> BTW, how does that acceleration work? Would it be possible/practical to
> write a function matching the current _nettle_sha1_compress.

The new VIA CPUs (e.g. nano) provide the compression function, i.e.
the fourth argument in MD_UPDATE. Older models (C5,C7) provide a full
compression functionality. I.e. sha1(x) and you get the result, which
proved not to be entirely useful.

>> btw. the MD_* macros were also proved to be very useful in that case.
> These macros are really intended for internal use only. I'd prefer to
> not export any interfaces like that. If functionality like this should
> be exported at all, I'd be more comfortable with a function interface
> for doing these things.

Aren't they already exported in macros.h?

regards,
Nikos

From nisse at lysator.liu.se  Wed Oct 26 11:21:24 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 26 Oct 2011 11:21:24 +0200
Subject: _nettle_write_be32
In-Reply-To: <CAJU7za+YTQBzT-W1dfD-K52HNONb9=mZaWg0ad+P_y31ujYjnQ@mail.gmail.com>
	(Nikos Mavrogiannopoulos's message of "Wed, 26 Oct 2011 10:30:45
	+0200")
References: <4EA6EC9C.6090307@gnutls.org>
	<nnd3dk5fsa.fsf@stalhein.lysator.liu.se>
	<CAJU7za+YTQBzT-W1dfD-K52HNONb9=mZaWg0ad+P_y31ujYjnQ@mail.gmail.com>
Message-ID: <nn4nyw5crv.fsf@stalhein.lysator.liu.se>

Nikos Mavrogiannopoulos <nmav at gnutls.org> writes:

> Aren't they already exported in macros.h?

It seems you're right, in that macros.h is actually installed in
$includedir/nettle (unlike, e.g., nettle-internal.h). I must have
forgotten that when I put the MD_* definitions in that file... In any
case, they're undocumented, and I won't promise any backwards
compatibility in future versions.

Maybe the internal-use things in macros.h ought to be moved to some
uninstalled *-internal.h file. The *READ_* and *WRITE_* macros should
stay in macros.h, but everything else there is a bit questionable.

Regards,
/Niels

-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.

From nmav at gnutls.org  Wed Oct 26 13:27:20 2011
From: nmav at gnutls.org (Nikos Mavrogiannopoulos)
Date: Wed, 26 Oct 2011 13:27:20 +0200
Subject: _nettle_write_be32
In-Reply-To: <nn4nyw5crv.fsf@stalhein.lysator.liu.se>
References: <4EA6EC9C.6090307@gnutls.org>
	<nnd3dk5fsa.fsf@stalhein.lysator.liu.se>
	<CAJU7za+YTQBzT-W1dfD-K52HNONb9=mZaWg0ad+P_y31ujYjnQ@mail.gmail.com>
	<nn4nyw5crv.fsf@stalhein.lysator.liu.se>
Message-ID: <CAJU7zaLkch9Nqwy0uNSsuh5p4O76vn1M=azsxAr=Xov+egfKjg@mail.gmail.com>

On Wed, Oct 26, 2011 at 11:21 AM, Niels M?ller <nisse at lysator.liu.se> wrote:

>> Aren't they already exported in macros.h?
> It seems you're right, in that macros.h is actually installed in
> $includedir/nettle (unlike, e.g., nettle-internal.h). I must have
> forgotten that when I put the MD_* definitions in that file... In any
> case, they're undocumented, and I won't promise any backwards
> compatibility in future versions.

Since those are macros it wouldn't matter for binary compatibility so
it would be ok.  If you remove them I'll duplicate it in gnutls.

regards,
Nikos

From kenton.jacobsen at gmail.com  Wed Nov  2 04:55:07 2011
From: kenton.jacobsen at gmail.com (Kenton Jacobsen)
Date: Wed, 2 Nov 2011 03:55:07 +0000 (UTC)
Subject: make issues
References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com>
	<nnty8b3h6h.fsf@stalhein.lysator.liu.se>
Message-ID: <loom.20111102T045349-727@post.gmane.org>

Another success!


From anna at bit.ly  Mon Nov 14 23:34:44 2011
From: anna at bit.ly (Anna Smith)
Date: Mon, 14 Nov 2011 22:34:44 +0000 (UTC)
Subject: make issues
References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com>
	<nnty8b3h6h.fsf@stalhein.lysator.liu.se>
Message-ID: <loom.20111114T233148-168@post.gmane.org>

the link provided is broken :(  can you repost?

Niels M?ller <nisse at ...> writes:

> You could try replacing config.guess with the latest version from
> http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD,
> and then run configure again.




From coel at coelmay.com  Tue Nov 15 00:00:37 2011
From: coel at coelmay.com (Coel May)
Date: Tue, 15 Nov 2011 10:00:37 +1100
Subject: make issues
In-Reply-To: <loom.20111114T233148-168@post.gmane.org>
References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com>
	<nnty8b3h6h.fsf@stalhein.lysator.liu.se>
	<loom.20111114T233148-168@post.gmane.org>
Message-ID: <8A2043EC-16E6-4127-8CFF-7C73A5A30C86@coelmay.com>


it works ok for me.

try this one:
http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=master

or maybe try navigating from the config.git project root
http://git.savannah.gnu.org/gitweb/?p=config.git


On 15/11/2011, at 09:34 , Anna Smith wrote:

> the link provided is broken :(  can you repost?
> 
> Niels M?ller <nisse at ...> writes:
> 
>> You could try replacing config.guess with the latest version from
>> http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD,
>> and then run configure again.
> 
> 
> 
> _______________________________________________
> nettle-bugs mailing list
> nettle-bugs at lists.lysator.liu.se
> http://lists.lysator.liu.se/mailman/listinfo/nettle-bugs


From nisse at lysator.liu.se  Wed Nov 16 15:27:43 2011
From: nisse at lysator.liu.se (Niels =?iso-8859-1?Q?M=F6ller?=)
Date: Wed, 16 Nov 2011 15:27:43 +0100
Subject: make issues
In-Reply-To: <loom.20111114T233148-168@post.gmane.org> (Anna Smith's message
	of "Mon, 14 Nov 2011 22:34:44 +0000 (UTC)")
References: <1B081719-79E8-433F-AEDC-E4DEA2CCC12F@coelmay.com>
	<nnty8b3h6h.fsf@stalhein.lysator.liu.se>
	<loom.20111114T233148-168@post.gmane.org>
Message-ID: <nnsjlota5c.fsf@stalhein.lysator.liu.se>

Anna Smith <anna at bit.ly> writes:

> the link provided is broken :(  can you repost?

Works for me. Maybe you accidentally copied the "," at the end? That was
not intended as a part of the url. It should be just

  http://git.savannah.gnu.org/gitweb/?p=config.git;a=blob_plain;f=config.guess;hb=HEAD

Regards,
/Niels


-- 
Niels M?ller. PGP-encrypted email is preferred. Keyid C0B98E26.
Internet email is subject to wholesale government surveillance.