Arne Goedeke wrote:
This sounds like something which could be connected to the fact that Stdio.File()->write(Stdio.Buffer()) does not release the interpreter lock. This makes it less useful for situations where write is not called from within one backend thread. This makes me think that in your situation using a temporary string would be better. I think this version is thread-safe, allows other threads to add to the buffer in parallel and avoids output_to:
string tmp = buffer->peek(SIZE); int bytes = fd->write(tmp);
if (bytes < 0) { // handle error } buffer->consume(bytes);
I understand the workaround, but having to do this violates the principle of least surprise (IMO). The Pike API should shield me from this.
It would be possible to make Stdio.File()->write(Stdio.Buffer()) release the interpreter lock at the additional cost of adding synchronization to Stdio.Buffer(). However, the main objective of this API (for me at least) was to make the common use-case more efficient in which you have one single backend doing all the writing on non-blocking sockets. In that situation I think it often makes more sense to have the IO thread complete one loop and release the interpreter lock when calling epoll. Other threads can then run while the backend thread is waiting for new IO events.
I agree with the reasoning, but isn't there some middle-ground solution which might work?
We're talking about merely allowing appending to the buffer while a write is in progress. That would mean the write "locks" (using Stdio.Buffer internals) the range of the buffer that it currently can grab and try to write. Other threads *are* allowed to add() to the end of the buffer, but nothing else. Releasing the interpreter lock for just doing that should not be necessary?