The actual reason I beefed up Buffer a bit lately is *because* I need to do some protocol decoding of a byte stream.
Now I see IOBuffer arrive. In order to avoid code bloat, wouldn't it be a better idea to integrate the functionality of IOBuffer into Buffer and just keep one?
Sorry about the timing, I have had IOBuffer on the way for some time (I am still wondering where to put it, however, that has, believe it or not, been a blocker for me. Perhaps Stdio.Buffer? I will create a buffered stream that reads to and writes from said object, without creating pike strings)
Unfortunately it is not possible to make String.Buffer even close to as efficient as long as it uses a stringbuilder. And not using a stringbuilder slows some things down (sprintf comes to mind) and makes others more or less impossible (wide strings) without excessive code duplication.
The whole reason for IOBuffer is that it uses pointer arithmetics on guaranteed 8bit strings to be fast at both reading from the beginning and writing to the end at the same time (I am, by the way, considering converting it to be a circular buffer to avoid the one memmove it does at times), and it is also efficient at creating sub-buffers.
The fact that it is guaranteed to only contain 8bit characters helps a lot too.
Or is the performance difference so strikingly great that this sounds like a bad idea?
As things stands now, yes.
Things like subbuffers is unfortunately actually impossible when using a stringbuilder.
I guess I might outline the plan for IOBuffer some more (I actually did this during the last pike conference, but it has been a while. :))
o Add support for reading to and writing from file objects to it. Either add support in Stdio.File (also add System.Memory + String.Buffer?) or do it the other way around (that way lies madness, however, see also: Shuffler)
The main goal here is to do one copy less and avoid the pike string creation
o Add support for System.Memory & String.Buffer to add()
o Add support for reading other things than "binary holerith" and integers + line + word + json object + encode_value?
o Add support for "throw mode".
It is rather useful to be able to change what happens when you try to read data that is not in the buffer.
| void read_callback() | { | inbuffer->add( new_data ); | // Read next packet | while( Buffer packet = inbuffer->read_hbuffer(2) ) | { | packet->set_error_mode(Buffer.THROW_ERROR); | if( mixed e = catch(handle_packet( packet ) ) ) | if( e->buffer_error ) | protocol_error(); // illegal data in packet | else | throw(e); // the other code did something bad | } | }
The handle_packet function then just assumes the packet is well formed, and reads without checking if the read succeed (since it will, or an error will be thrown).
This code snipplet also demonstrates why the subbuffers are handy, read_hbuffer does not actually copy any data, the returned buffer just keeps a reference to the data in the original buffer.