Blame on me. I should have gone English. The topic is how-to-replace-Crypto-with-Nettle.

- jhs or some pelix root can fix an account for you.

- I don't remember anything you wrote about a Pike Nettle interface, but I've not had any major problems with the current one. The current interface, but with Capitalized programs and a move from update/digest to feed/drain would satisfy me.

- I agree that not having Nettle in the source tree could be a good idea, if we post enough pointers to where to find it (e.g. putting releases on the Pike ftp). As long as changes can be countered by configure scripts I don't see that there is a problem with incompatible changes in Nettle.

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-11 22:30: Subject: nettle-1.7

---

Några saker:

• Fixa ett konto så jag kan checka in saker i pike-cvs:en (jag kör hellre med lösenord och lsh -G än med public-key autenticering).

• Ge mig lite feedback på hur gränssnittet borde se ut. Jag minns inte riktigt hur mycket jag har skrivit av mina funderingar.

• Hjälp till att hacka. Jag gillar pike, men använder det inte särskilt mycket nu förtiden, så min motivation är att det vore skoj, och ett sätt att testa att Nettles gränssnitt funkar, snarare än att det är något jag själv kommer att använda dagligen.

• Tjata lagom mycket.

En annan fråga är om man vill ha med Nettle i Pikes källkodsträd (om än inte i Pikes cvs), eller om man ska länka mot en nettle som är installerad på systemet. Det senare är nog egentligen sundare, nackdelen är att man fortfarande kan vänta sig en del inkompatibla ändringar mellan Nettle-versioner.

/ Niels Möller ()

I think it would be nice to have all buffer objects where you can push data several times and then then ask for (perhaps partial) result to have the same API (feed/drain/finish).

Do you mean that you would like to go from Crypto.foo()->feed(jox)->drain(); to Crypto.foo(jox); ?

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-11 23:00: Subject: nettle-1.7

---

...

• I don't remember anything you wrote about a Pike Nettle interface, but I've not had any major problems with the current one. The current interface, but with Capitalized programs and a move from update/digest to feed/drain would satisfy me.

update/digest is farly standard terminology for hash functions. Your wish is the first for feed/drain. feed/drain would make more sense for things liek a CBC object, I guess.

I think the most fundamental change I'd like to do (trivial renames) is that I'd like algorithms to be represented as objects, with a `()-method, instead of as plain programs.

/ Niels Möller ()

Sounds fine by me. Encryption program XYZ should be complimented with the encryption function xyz that does padding and encryption in one call. string aes(string key, string data)

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-11 23:17: Subject: nettle-1.7

---

The interface I imagine is something like

Crypto.AES()->set_encrypt_key(the_key)->crypt(data)

The basic AES object will only encrypt complete blocks. Then you can wrap CBC on top of that, but that would still probably handle complete blocks.

In order to handle arbitrary length input, one must use some padding convention. An object implementing padding could also do buffering. I'm not sure I see the point of feed/drain, the most natural interface to me would be

o->crypt(data) // encrypts or buffers more data. Returns as much // output as is available

o->finish() // Do final padding, return final output.

One could do feed/drain on top of that, but perhaps that's best done in pike rather than C.

For hash functions, the interface would be close to nettles,

o = Crypto.MD5(); o->update("foo"); // Returns self, so that o->update("bar"); // Crypto.MD5()->update("foo)->digest() works too. o->digest(); // Returns digest string

So, that's basically what I've been imagining, but I'm open to suggestions.

/ Niels Möller ()

There are no builtin padding routines in nettle. The reason is that padding is (i) simple, and (ii) quite application specific.

It would make sense to include some objects for popular padding methods in the pike module, though.

/ Niels Möller ()

Previous text:

2003-03-11 23:33: Subject: nettle-1.7

---

Does Nettle have support for various padding routines? I know there are quite a few available.

/ David Hedbor

Is there support for MD4 yet?

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-11 23:36: Subject: nettle-1.7

---

There are no builtin padding routines in nettle. The reason is that padding is (i) simple, and (ii) quite application specific.

It would make sense to include some objects for popular padding methods in the pike module, though.

/ Niels Möller ()

Yes. Marcus added it when you said Nettle was a superset of the Crypto module :-)

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-12 10:12: Subject: nettle-1.7

---

No. (It should be easy to add thogh. Did pike have an independent MD4 implementation?)

/ Niels Möller ()

Ok, now I have cvs access. Just have to build a stable pike from a srcdist before trying the cvs version...

For a start, I think it's best to create a directory "src/post_modules/Nettle", that builds a module "Nettle.so". This will be cmod-glue to nettle, with basic crypto objects.

Then friendlier objects and namespaces should be implemented in Pike, in Crypto.pmod.

Any objections to that plan?

/ Niels Möller ()

Previous text:

2003-03-11 23:17: Subject: nettle-1.7

---

The interface I imagine is something like

Crypto.AES()->set_encrypt_key(the_key)->crypt(data)

The basic AES object will only encrypt complete blocks. Then you can wrap CBC on top of that, but that would still probably handle complete blocks.

In order to handle arbitrary length input, one must use some padding convention. An object implementing padding could also do buffering. I'm not sure I see the point of feed/drain, the most natural interface to me would be

o->crypt(data) // encrypts or buffers more data. Returns as much // output as is available

o->finish() // Do final padding, return final output.

One could do feed/drain on top of that, but perhaps that's best done in pike rather than C.

For hash functions, the interface would be close to nettles,

o = Crypto.MD5(); o->update("foo"); // Returns self, so that o->update("bar"); // Crypto.MD5()->update("foo)->digest() works too. o->digest(); // Returns digest string

So, that's basically what I've been imagining, but I'm open to suggestions.

/ Niels Möller ()

I think _Nettle is better than Nettle, if it is not supposed to be used by normal programmers.

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-12 13:26: Subject: nettle-1.7

---

Ok, now I have cvs access. Just have to build a stable pike from a srcdist before trying the cvs version...

For a start, I think it's best to create a directory "src/post_modules/Nettle", that builds a module "Nettle.so". This will be cmod-glue to nettle, with basic crypto objects.

Then friendlier objects and namespaces should be implemented in Pike, in Crypto.pmod.

Any objections to that plan?

/ Niels Möller ()

I see no good reason to use _Nettle, when that API most likely will be both stable and useful.

/ Johan Sundström (folkskådare)

Previous text:

2003-03-12 15:22: Subject: nettle-1.7

---

Perhaps. The idea is that it should represent the C library in a pikish way, but with few extra features. So it could be used directly, if there ever are any obscure features in nettle that doesn't fit in Crypto.

As far as I see, module names like _Foo are used when the official name of a module is Foo, and Foo.pmod is written in Pike, and there also is an internal C module that is "owned" by Foo. Then _Foo is a good name for the internal module. But a while ago, someone argued that in this situation, both modules could be named Foo, and it would still work.

Either name is ok with me. What do others think?

/ Niels Möller ()

That's why I think Crypto.Nettle would be best. :)

/ Mirar

Previous text:

2003-03-12 17:44: Subject: nettle-1.7

---

OTOH, shouldn't the stable and useful API be called "Crypto"? Would they be sufficiently different to motivate _two_ official API:s?

/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)

That either name is ok?

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-13 02:54: Subject: nettle-1.7

---

I second that.

/ David Hedbor

Since you're not supposed to use _Nettle directly, giving it a name you don't like is a feature. :-) The thing you're supposed to used should be named Crypto and/or Crypto.Nettle or suchlike.

/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)

Previous text:

2003-03-13 06:02: Subject: nettle-1.7

---

Uh. Mainly that I don't like _Nettle. Perhaps Nettle or Crypto.Nettle. :)

/ David Hedbor

Now it's "Nettle".

Perhaps I should describe the current organization, for hash functions. Sorry it's a little hairy.

The Nettle library provides an abstraction struct nettle_hash, defined in nettle/nettle-meta.h. These are constant read-only structs that contains the name of the hash algorith, block size, digest size, and the size of the context struct. There's nettle_md5, nettle_sha1, etc. This is used to minimize the amount of glue code needed for each hash algorithm. The classes are:

PIKECLASS nettle_hash

This class has one instance variable, const struct nettle_hash *meta, and the methods name(), digest_size(), block_size().

The meta variable is initialized to NULL, it's up to a suitable mixin to initialize it.

PIKECLASS hash_instance

Inherits nettle_hash, and has one additional instance variable, void *ctx. This should point to a context struct for a hash algorithm, also initialized to NULL. This function has the familiar methods update and digest, and will raise errors if meta or ctx is NULL, or if input contains wide characters,

PIKECLASS nettle_md5

This is a mixin class, it has no superclass of its own. It's initializer uses get_storage to get the storage for an inherited nettle_hash class, supposedly inherited in parallel with this mixin. It checks that get_storage doesn't return NULL, and that the meta instance variable is not yet initialized. It then sets meta = &nettle_md5. That's all. I found out the hard way that the INIT method can't call Pike_error, so it will just leave meta alone on failure.

PIKECLASS md5_state

This is the state class for md5. It inherits nettle_md5 and hash_instance (and depends on these being initialized in the right order, hash_instance *first*. It has one new instance variable, a struct md5_ctx. The INIT method uses get_storage to get to the inherited hash_instance storage, and sets the ctx pointer to point to the struct md5_ctx.

That is, it sets up a pointer in one block in the storage area to point to another block (probably just a constant offset away from the pointer). That seems easier and cleaner that dependeing on the storage layout directly.

That's all, the md5_state object is equivalent to the current Crypto.md5 class. But then I'd like to put some code into Crypto.pmod as well, like

class md5_algorithm { inherit Nettle.nettle_md5; inherit Nettle.nettle_hash;

constant `() = Nettle.md5_state; };

constant MD5 = md5_algorithm();

Then one can do Crypto.MD5.digest_size() to get the digest size, Crypto.MD5()->update("foo")->digest(); one could also add a method Crypto.MD5.hash(string data) that hashes a string in one go.

For the hairyness factor, note that the order of the inherits is important here too. I'm assuming the INIT order is the same as for my cmod classes, but that should be properly defined somewhere.

For type checking purposes, the Nettle.md5_state class can be used. One can add some alias in Crypto too, of course, if you don't like either of

Nettle.md5_state foo; Crypto.MD5.`() foo; // ;-)

Perhaps this still doesn't quite justify the amount of small classes, but I believe that being able to get out the meta pointer from objects will be handy for example for the hmac code.

Block ciphers can be dealt with in mostly the same way, I think. Except that some ugly algorithms, like des, have weak keys, and need to raise exceptions in their set_key functions, which means that more information is needed than that which is provided by the the corresponding cipher meta abstraction in the nettle library.

The code is up for view at URL: http://pike.ida.liu.se/development/cvs/view.xml?module=Pike&file=7.5/src/post_modules/Nettle/hash.cmod&rev=1.1, even if the Nettle directory for some reason isn't listed at URL: http://pike.ida.liu.se/development/cvs/browse.xml?module=Pike&dir=/7.5/src/post_modules.

/ Niels Möller ()

Previous text:

2003-03-13 20:19: Subject: nettle-1.7

---

I thought the argument was that you MIGHT want to use it directly. Of course, name it _Crypto_Nettle.so (to keep with the Image module naming scheme) and then having most functionality in Crypto with the rest in Crypto.Nettle might be a better idea.

/ David Hedbor

You can try requesting any class names you like, but please be specific: Give the current name and the name you'd prefer for each class to be renamed. For instance, I don't think "MD5State" is particularly readable.

/ Niels Möller ()

Previous text:

2003-03-13 21:17: Subject: nettle-1.7

---

Could we request CamelCase class names?

(The Nettle directory is visible now, by the way. Sorry about the glitch.)

/ Johan Sundström (folkskådare)

That's the idea. Using capital letters for abbreviations and initial letters for each word, and lower case letters for the rest - and no _ characters.

If that is way too ugly, artistic freedom would probably allow for an occasional MD5_State, but my gut feeling is to avoid underscores in class names.

What feels most important to me personally is reserving all-lowercase (plus numbers and _) names for method/variable names, and occasional classes that mimic methods (Process.create_process, for example).

/ Johan Sundström (folkskådare)

Previous text:

2003-03-13 21:35: Subject: nettle-1.7

---

MD5State might not be perfectly readable but it does adhere to the pike class naming "standard".

/ David Hedbor

You've seen the refdoc/keywords.txt file?

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-18 14:38: Subject: nettle-1.7

---

Ok, now I renamed te classes in hash.cmod. I also tried to add some autodoc. I'd appreciate feedback on the latter, since the autodoc thing is new since the last time I hacked pike.

/ Niels Möller ()

Well, you can look at the other .txt-files there as well. They are not completely up to date, but probably quite helpful anyway.

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-18 15:15: Subject: nettle-1.7

---

I've found it now. It seems to explain the syntax, but not the stye.

/ Niels Möller ()

It will be interesting to compare performance between nettle and previous implementations.

/ Johan Sundström (folkskådare)

Previous text:

2003-03-18 17:42: Subject: nettle-1.7

---

And now I rewrote the AES glue using the same style, putting it into cipher.cmod.

/ Niels Möller ()

How compatible do you intend to keep it? If not very then it's probably best to move away the old module as-is to the compatibility tree and then start anew.

/ Martin Stjernholm, Roxen IS

Previous text:

2003-03-18 18:00: Subject: S: Demonstrera för fredlig lösning

---

For most functions, there shouldn't really be much difference, as the code is derived from the same roots.

AES/rijndael should be faster on x86 and sparc (due to some assembly code). RSA should probably be faster too (CRT optimization).

BTW, I wonder what to do about the old Crypto.pmod organization. It uses a subdirectory Crypto/, and a magic indexing operator in Crypto.pmod. I'm afraid I'm guilty to that construction, but it was a long time ago so I hope I can be forgiven ;-)

It would be better to have a directory Crypto.pmod, and perhaps a module.pmod too, just like other modules.

What's the best way to make that change, with no more CVS breakage than necessary?

/ Niels Möller ()

It was added to detect races where some code changes a socket to blocking code while other code (i.e. the callbacks) are still using it in nonblocking mode. Without that check, the result is typically a disastrous hang of the backend thread in that situation.

I can't think of a situation where one want to combine callbacks with blocking mode, because of the obvious risk to get a hanging backend. When and why is that used?

/ Martin Stjernholm, Roxen IS

Previous text:

2003-03-19 12:04: Subject: S: Demonstrera för fredlig lösning

---

Speaking of compatibility, can you please provide a compelling reason for the

if (!(::mode() & 0x400)) error ("Read callback called on blocking socket!\n");

in __stdio_read_callback()? Otherwise I'm going to remove it as it breaks compatibility.

/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)

I'd like more specific examples that motivates the use better.

There is no risk at all for deadlocks unless you also read using 'read' in a separate thread.

So you read using another thread. What's the callback for then? Do you read from that too, at the same time?

/ Martin Stjernholm, Roxen IS

Previous text:

2003-03-19 13:40: Subject: I/O callbacks in blocking mode

---

Lets take a few examples:

o Reading from the terminal o Reading from the network (i.e. AIDO) o Reading from anything else o Reading from another program via a pipe.

There is no risk at all for deadlocks unless you also read using 'read' in a separate thread.

/ Per Hedbor ()

Or if not unusable, at least higly dangerous. Using nonblocking read on a terminal causes the terminal to be closed if the program exits without setting the FD back to blocking mode.

/ Per Hedbor ()

Previous text:

2003-03-19 13:47: Subject: I/O callbacks in blocking mode

---

No, what I was saying was that the deadlock only occurs if you read from another thread. Why protect the user from that by making Readline unusable, as an example?

/ Per Hedbor ()

Yes. But now please explain how to read asynchronously from stdin without setting it in nonblocking mode with that 'sanity' check in place?

/ Per Hedbor ()

Previous text:

2003-03-19 13:53: Subject: I/O callbacks in blocking mode

---

You really mean stdin, not tty:s in general, right? At least I'm not aware of any problem opening /dev/tty and setting it in nonblocking mode.

Setting stdio fd:s into non-blocking mode is always more or less unfriendly to the environment.

/ Niels Möller ()

I don't think it's ugly to read in blocking mode using select. Even the examples in the manpage does it. Using stdin, actually.

/ Per Hedbor ()

Previous text:

2003-03-19 14:11: Subject: I/O callbacks in blocking mode

---

Well, you're right there's no *good* way to do that. Either

1. Set the fd to blocking mode anyway. Use atexit to restre it before

your program exit, and hope your shell will restore it for you if your process dies violently.

2. If you want to use the fd for things like readline, open /dev/tty

instead of stdin. That should avoid most of the problems with (1).

3. Create an extra socket pair, and spawn an extra shell or process

that copies stdin to the pipe (in blocking mode). You can set your end into non-blocking mode. Ugly, but seems to work. It's the function io.c:lsh_copy_file in lsh.

4. Do it your way, with select and read with the socket in blocking

mode. I consider that to be an ugly kludge, just like the other alternatives.

/ Niels Möller ()

Using Stdio.stdin()->read(10) will also *hang* the process until some data is availble on stdin. Even without read callbacks. What should we do, what should we do....

/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)

Previous text:

2003-03-19 15:38: Subject: I/O callbacks in blocking mode

---

The reason I think it is ugly is that it isn't robust. It will cause the process to *hang* if started in such a way that other some other process competes for the stdin data. If the program reads stdin and a socket (say an X interface), I want to be able to interact with it through the socket even if something fishy happened to its stdin.

To me, this is a corner where unix is simply braindamaged, and there's no way to deal with it that I particularly like.

/ Niels Möller ()

How can that happen?

/ Per Hedbor ()

Previous text:

2003-03-19 13:47: Subject: I/O callbacks in blocking mode

---

Sorry, I misread your last sentence. I wasn't talking about a deadlock within the same process, but that the backend thread hangs indefinitely on a connection from a client. That's just as bad.

/ Martin Stjernholm, Roxen IS

Yes, and the check was added to find that exact use.

/ Henrik Grubbström (Lysator)

Previous text:

2003-03-19 14:04: Subject: I/O callbacks in blocking mode

---

When I try to connect to Roxen WebServer 3.3.74-cvs running Pike 7.4.13 I get

Read callback called on blocking socket! /usr/local/pike/7.4.13/lib/modules/Stdio.pmod/module.pmod (version 1.163):677: Stdio.File("socket", "130.236.214.207 1522", 777 /* fd=20 */)->__stdio_read_callback() module.c:58()->Builtin()->Backend: module.c:58()->Builtin()->__backend->`()(3600.000000)

/ Martin Nilsson (har bott i google)

Well, it caused hangs on Solaris at some installations.

/ Henrik Grubbström (Lysator)

Previous text:

2003-03-19 14:10: Subject: I/O callbacks in blocking mode

---

I still totally fail to see why it would be dangerous. Which OS does not have a working select() or poll()?

/ Per Hedbor ()

Add a fd->set_enforce_nonblocking( 1 ) or something instead.

/ Per Hedbor ()

Previous text:

2003-03-19 14:14: Subject: I/O callbacks in blocking mode

---

I very much want to know _why_ that happened if no other thread was reading from the same socket, or the socket was added twice to the poll-set for some reason.

But please please please please please don't enforce it for the people who have quite a valid use for blocking asynchronous sockets. If you do it would force me to require a specially patched pike to run AIDO.

/ Per Hedbor ()

How would there be a "disastrous hang"? The read callback will only be called when there is something to read, in which case read(N,1) returns immediately. If you use read() to empty the data before the callback gets to it, then the read() in the callback will of course block until there's more data, but that's self-inflicted and rather what you expect would happen if you manually do read() on a blocking fd. If you want to "protect" the user from shooting himself in the foot, I'd rather suggest a check in the read() function that you don't have a read_callback at the same time as you are manually doing a read(). That would break fewer programs. I still think it would be unnecessary pampering though.

/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)

Previous text:

2003-03-19 13:29: Subject: I/O callbacks in blocking mode

---

It was added to detect races where some code changes a socket to blocking code while other code (i.e. the callbacks) are still using it in nonblocking mode. Without that check, the result is typically a disastrous hang of the backend thread in that situation.

I can't think of a situation where one want to combine callbacks with blocking mode, because of the obvious risk to get a hanging backend. When and why is that used?

/ Martin Stjernholm, Roxen IS

Readcallbacks with blocking sockets is not exactly error prone.

/ Per Hedbor ()

Previous text:

2003-03-19 15:47: Subject: I/O callbacks in blocking mode

---

...

How would there be a "disastrous hang"? /.../

It has been discussed later on in this thread. I suggest you read it through before responding.

...

If you want to "protect" the user from shooting himself in the foot, I'd rather suggest a check in the read() function that you don't have a read_callback at the same time as you are manually doing a read(). That would break fewer programs. I still think it would be unnecessary pampering though.

That's a good suggestion. I see it as a good measure to track down bugs more efficiently; in a high level language I don't see any real point in allowing error prone situations just because it would be "pampering" otherwise. That's rather one of the points with high level languages.

/ Martin Stjernholm, Roxen IS

I can always take an example from the metaserver communication in AIDO.

void read_callback( ... ) { while( mixed msg = protocol_parser->feed( data ) ) process_async_msg( msg ); }

mixed do_command( ... ) { fd->write( encode_command( command ) ); while( mixed msg = protocol_parser->feed( fd->read( 2048, 1 ) ) ) if( msg->type == REPLY ) return msg->payload; else process_async_msg( msg ); }

/ Per Hedbor ()

Previous text:

2003-03-19 15:52: Subject: I/O callbacks in blocking mode

---

No, you've showed that. But I think calling read() when there's a read callback installed is, or perhaps you can show legitimate situations for that too? (I don't count the read call in __stdio_read_callback now; that one ought to be done away with completely.)

/ Martin Stjernholm, Roxen IS

...

How would there be a "disastrous hang"? /.../

It has been discussed later on in this thread. I suggest you read it through before responding.

I have read the thread, but not seen any real answer to how it could happen on its own. There are numerous ways to _make_ it happen of course, for example the read callback i posted in 9904298. There may even be cases where the hang is _desired_ rather than "disastrous", it depends on the application.

That's a good suggestion. I see it as a good measure to track down bugs more efficiently; in a high level language I don't see any real point in allowing error prone situations just because it would be "pampering" otherwise. That's rather one of the points with high level languages.

You don't see a point in allowing useful constructs? Even when they were allowed by previous releases and used in lots of code?

/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)

Previous text:

2003-03-19 15:47: Subject: I/O callbacks in blocking mode

---

...

How would there be a "disastrous hang"? /.../

It has been discussed later on in this thread. I suggest you read it through before responding.

...

If you want to "protect" the user from shooting himself in the foot, I'd rather suggest a check in the read() function that you don't have a read_callback at the same time as you are manually doing a read(). That would break fewer programs. I still think it would be unnecessary pampering though.

That's a good suggestion. I see it as a good measure to track down bugs more efficiently; in a high level language I don't see any real point in allowing error prone situations just because it would be "pampering" otherwise. That's rather one of the points with high level languages.

/ Martin Stjernholm, Roxen IS

Fine.

The debug tools I'd like to use is rather one where you can set a flag on a thread which causes an error to be thrown whenever a blocking operation is attempted in it, and another that restricts the use of read and/or write on an fd to a single thread.

/ Martin Stjernholm, Roxen IS

Previous text:

2003-03-19 15:54: Subject: I/O callbacks in blocking mode

---

...

That's a good suggestion.

Please don't. :-)

The real code in my daemons is closer to this:

void read_callback( mixed ignore, string x ) { while( strlen( x ) ) { int len; if( strlen( x ) < 4 ) x += fd->read( 4-strlen(x) ); sscanf( x, "%4c%s", len, x ); if( strlen( x ) < len ) x += fd->read( len-strlen(x) ); if( mixed e = catch(handle_command( x[..len-1] )) ) { werror( describe_backtrace( e ) ); exit( 1 ); } x = x[len..]; } }

Please note that I am reading data in the read callback.

It seems somewhat pointless to have to add fd->set_read_callback( 0 ) at the beginning and fd->set_read_callback( read_callback ) at the end.

What I would really like, btw, is a read callback that does not do the actual read() call.

/ Per Hedbor ()

Well, if it could detect busywaiting and infinite loops too it'd be even better. It would be very useful for the backend thread in Roxen, which must not do anything that keeps it from returning to the backend quickly.

/ Martin Stjernholm, Roxen IS

Previous text:

2003-03-19 16:04: Subject: I/O callbacks in blocking mode

---

Hm? A type of thread that can only busy-wait, you mean? When is that useful? *curious*

/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)

Why would the backend hang because of a read callback? Isn't the read callback only called when there is data to be read, and can't hang?

/ Mirar

Previous text:

2003-03-19 13:29: Subject: I/O callbacks in blocking mode

---

It was added to detect races where some code changes a socket to blocking code while other code (i.e. the callbacks) are still using it in nonblocking mode. Without that check, the result is typically a disastrous hang of the backend thread in that situation.

I can't think of a situation where one want to combine callbacks with blocking mode, because of the obvious risk to get a hanging backend. When and why is that used?

/ Martin Stjernholm, Roxen IS

Well, I'm not sure exactly how much backwards compatibility is needed. But I'd really prefer to be able to be able to replace one or two classes at a time.

What is the compatibility tree, and how does it work?

Are there any testcases for any code that makes serious use of the crypto module, e.g. the SSL code?

/ Niels Möller ()

Previous text:

2003-03-19 11:26: Subject: S: Demonstrera för fredlig lösning

---

How compatible do you intend to keep it? If not very then it's probably best to move away the old module as-is to the compatibility tree and then start anew.

/ Martin Stjernholm, Roxen IS

Sounds fine to me. Someone with more permissions and clue than I should renamed the current Crypto.pmod file and create a directory Crypto.pmod. (At least I still think that Crypto, Crypto.MD5, etc are the right names for this stuff).

/ Niels Möller ()

Previous text:

2003-03-19 12:37: Subject: S: Demonstrera för fredlig lösning <-- Todays subject

---

The compatibility tree is modules that get activated if you use #pike <version>.

I would really like if the new class interface used SillyCaps instead of normal_caps, and we keep the old Crypto module for #pike 7.4 and below.

/ Per Hedbor ()

I have looked into it. Crypto.pmod is now a directory. All tests appears to be working.

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-19 13:02: Subject: S: Demonstrera för fredlig lösning <-- Todays subject

---

I can look into it.

/ Martin Nilsson (har bott i google)

Because [rendering excuse] now it takes less time to check in into those files.

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-19 19:06: Subject: S: Demonstrera för fredlig lösning <-- Todays subject

---

And why didn't you copy the ,v files?

/ Henrik Grubbström (Lysator)

AIDO uses it as well. Mainly through SSL, but it also uses Crypto.md5().

/ Per Hedbor ()

Previous text:

2003-03-19 12:53: Subject: S: Demonstrera för fredlig lösning

---

Primarily Roxen through SSL.pmod. There's also various other code in Roxen that use the hash algorithms, randomness generators etc. So it's not exactly an unused module that just sits around collecting dust.

/ Martin Stjernholm, Roxen IS

Substitution is something I did to solve things like ROT13 and Fantomenkrypto.

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-19 12:58: Subject: S: Demonstrera för fredlig lösning

---

Before starting to change stuff, I would *really* like to have a testsuite for that. I'm afraid the SSL code exercises a lot more code than the _Crypto and crypto testsuites.

By the way, what's Crypto.substitution used for? Is it intended to be interface compatible with any other classes?

/ Niels Möller ()

Well, they will all have their names changed.

/ Per Hedbor ()

Previous text:

2003-03-19 13:09: Subject: S: Demonstrera för fredlig lösning

---

The easiest way (which doesn't mean it's that easy) is probably to download a Roxen or Caudium, configure an https port and record the calls it makes to the Crypto module.

As for the hash and randomness classes, I hope there won't be any subtle changes in how they work.

/ Martin Stjernholm, Roxen IS

Those aren't subtle changes. What I mean is that I hope it'll be a simple matter of changing names without any noticeable risk to get other hard-to-track-down bugs.

/ Martin Stjernholm, Roxen IS

Previous text:

2003-03-19 13:20: Subject: S: Demonstrera för fredlig lösning

---

Well, they will all have their names changed.

/ Per Hedbor ()

I'm not sure that the compatibility glue can handle the Crypto module as is, but there is no need to use a magic module here, so rewriting it to a normal X.pmod/module.pmod solution shouldn't do any harm. We'll have to drop support for lobotomized crypto, but I think that's ok with everyone.

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-19 11:26: Subject: S: Demonstrera för fredlig lösning

---

How compatible do you intend to keep it? If not very then it's probably best to move away the old module as-is to the compatibility tree and then start anew.

/ Martin Stjernholm, Roxen IS

What's the reason for using the extra level of indirection by using `()? Ie, when is HashAlgorithm and MD5_Algorithm used directly?

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-20 17:58: Subject: nettle-1.7

---

Now I've done some initial hacking on Crypto.pmod/module.pmod.

I made it inherit Nettle, and added two classes, HashAlgorithm (an abstract class) and MD5_Algorithm. The latter is also instantiated as Crypto.MD5.

It has the methods

string name() int digest_size() int block_size() string hash(string) MD5_State `()()

(HashAlgorithm has the same methods, except that `() returns a HashState, and lacks an implementation).

Unfortunately, I couldn't use constant for either Crypto.MD5 or the `() method. Problem is that HashAlgorithm needs a prototype for `(), and it seems I can't override it in MD5_Algorithm using constant. Perhaps there's some trick that I've missed?

I wrapped the Nettle parts inside an #if constant(Nettle), is that enough to make sure that the module can be compiled if Nettle is unavailable?

/ Niels Möller ()

Isn't it better to create an MD5.pmod file then?

/ Martin Nilsson (har bott i google)

Previous text:

2003-03-20 18:23: Subject: nettle-1.7

---

The point is that I want Crypto.MD5 to be an object (and a singleton), not a class. So that you can use

Crypto.MD5.digest_size(); Crypto.MD5.hash("foo");

or

a = Crypto.MD5; ... a->name(); ... a->digest_size();

An instance of (a class inheriting) HashAlgorithm represents the algorithm itself, without any hashing state.

/ Niels Möller ()

You could have name as a constant and index it from the program instead.

/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)

Previous text:

2003-03-20 18:23: Subject: nettle-1.7

---

The point is that I want Crypto.MD5 to be an object (and a singleton), not a class. So that you can use

Crypto.MD5.digest_size(); Crypto.MD5.hash("foo");

or

a = Crypto.MD5; ... a->name(); ... a->digest_size();

An instance of (a class inheriting) HashAlgorithm represents the algorithm itself, without any hashing state.

/ Niels Möller ()

If you mean distributions with sourcetree, I don't agree that not having nettle in then is a good idea.

The Crypto/SSL parts of pike are rather central to it as it is.

Not always having hash/randomization/crypto functions available is not a good idea, as an example it would not be possible to write something like bittorrent in pike.

/ Per Hedbor ()

Previous text:

2003-03-11 22:44: Subject: nettle-1.7

---

Blame on me. I should have gone English. The topic is how-to-replace-Crypto-with-Nettle.

• jhs or some pelix root can fix an account for you.

• I don't remember anything you wrote about a Pike Nettle interface, but I've not had any major problems with the current one. The current interface, but with Capitalized programs and a move from update/digest to feed/drain would satisfy me.

• I agree that not having Nettle in the source tree could be a good idea, if we post enough pointers to where to find it (e.g. putting releases on the Pike ftp). As long as changes can be countered by configure scripts I don't see that there is a problem with incompatible changes in Nettle.

/ Martin Nilsson (har bott i google)

That would work, but it would be rather irritating for the one running configure. I really think that gmp and nettle should be included in the source distribution. Neither is all that large, and the system provided versions can be used if any are found.

/ Per Hedbor ()

Previous text:

2003-03-12 00:35: Subject: nettle-1.7

---

It is fine by me if configure fails if no Nettle is found on the system.

/ Martin Nilsson (har bott i google)

No. Either it should be included or it should not be included. If a security issue would arise with any of the libraries I'd like to be able to easilly know what packages that are voulnerable.

Yes - this is easilly found out on a unix system, but what about win*?

/ Peter Lundqvist (disjunkt)

Previous text:

2003-03-12 00:36: Subject: nettle-1.7

---

That would work, but it would be rather irritating for the one running configure. I really think that gmp and nettle should be included in the source distribution. Neither is all that large, and the system provided versions can be used if any are found.

/ Per Hedbor ()

Uhm - didn't think of that =)

/ Peter Lundqvist (disjunkt)

Previous text:

2003-03-12 00:44: Subject: nettle-1.7

---

? On windows we always use our own versions, of course. There is no alternative.

/ Per Hedbor ()

I'll attend to the first request as soon as you provide me with an ssh pubkey (or point to one at lysator or similar) and suggest a username. ("nisse"?)

As for the rest, we can probably all chip in a bit - Martin is not the only one itching for Nettle. :-)

/ Johan Sundström (folkskådare)

Previous text:

2003-03-11 22:30: Subject: nettle-1.7

---

Några saker:

• Fixa ett konto så jag kan checka in saker i pike-cvs:en (jag kör hellre med lösenord och lsh -G än med public-key autenticering).

• Ge mig lite feedback på hur gränssnittet borde se ut. Jag minns inte riktigt hur mycket jag har skrivit av mina funderingar.

• Hjälp till att hacka. Jag gillar pike, men använder det inte särskilt mycket nu förtiden, så min motivation är att det vore skoj, och ett sätt att testa att Nettles gränssnitt funkar, snarare än att det är något jag själv kommer att använda dagligen.

• Tjata lagom mycket.

En annan fråga är om man vill ha med Nettle i Pikes källkodsträd (om än inte i Pikes cvs), eller om man ska länka mot en nettle som är installerad på systemet. Det senare är nog egentligen sundare, nackdelen är att man fortfarande kan vänta sig en del inkompatibla ändringar mellan Nettle-versioner.

/ Niels Möller ()

I think it would be nice to have the option (--with-*) to use the nettle installed in the system if you wanted that.

Hopefully the API differences isn't bigger then that you could write configure tests for them?

/ Mirar

Previous text:

2003-03-11 22:30: Subject: nettle-1.7

---

Några saker:

• Fixa ett konto så jag kan checka in saker i pike-cvs:en (jag kör hellre med lösenord och lsh -G än med public-key autenticering).

• Ge mig lite feedback på hur gränssnittet borde se ut. Jag minns inte riktigt hur mycket jag har skrivit av mina funderingar.

• Hjälp till att hacka. Jag gillar pike, men använder det inte särskilt mycket nu förtiden, så min motivation är att det vore skoj, och ett sätt att testa att Nettles gränssnitt funkar, snarare än att det är något jag själv kommer att använda dagligen.

• Tjata lagom mycket.

En annan fråga är om man vill ha med Nettle i Pikes källkodsträd (om än inte i Pikes cvs), eller om man ska länka mot en nettle som är installerad på systemet. Det senare är nog egentligen sundare, nackdelen är att man fortfarande kan vänta sig en del inkompatibla ändringar mellan Nettle-versioner.

/ Niels Möller ()