Picture this: ChiliMoon 2004, http.pike uses the Shuffler object. It works fine (after a few modifications to the do_log callback), except when I call up a CGI program.
cgi.pike uses a wrapper that uses a pair of pipes to pass the data along.
Now, doing an add_source for shuffler with the proper end of the pipe, will result in the pipe being read correctly (once, total amount read, no followup read that should have detected the end). After that, the content of that read() is lost.
Trying something different, I add three add_source() calls before, and three add_source() calls after the add_source() for the pipe. The three preceding and trailing add_source() calls have a fixed string content.
What happens now is: Shuffle[0x9c98b00]:add_source(XX,0,-1) --> 0x9081d48 <<-- const string Shuffle[0x9c98b00]:add_source(XX,0,-1) --> 0x9f64770 <<-- const string Shuffle[0x9c98b00]:add_source(XX,0,-1) --> 0x9c45038 <<-- const string Shuffle[0x9c98b00]:add_source(XX,0,-1) --> 0x9f5d690 <<-- pipe Shuffle[0x9c98b00]:add_source(XX,0,-1) --> 0x9c45068 <<-- const string Shuffle[0x9c98b00]:add_source(XX,0,-1) --> 0x9c45098 <<-- const string Shuffle[0x9c98b00]:add_source(XX,0,-1) --> 0x9c988c8 <<-- const string Shuffle[0x9c98b00]:set_done_callback(0x8d0141c) Shuffle[0x9c98b00]:start() Shuffle[0x9c98b00]:_set_callbacks() Shuffle[0x9c98b00]:__set_calllbacks(Pike)
...CGIWrapper is being done...
Shuffle[0x9c98b00]:write_callback() Shuffle[0x9c98b00]:_send_more(0) Shuffle[0x9c98b00]:_request(8192) from (nil) Shuffle[0x9c98b00]:__send_more_callback(8192) Shuffle[0x9c98b00]:__send_more_callback(): sending(48) Shuffle[0x9c98b00]:__send_more_callback(): sending(48): sent 48 Shuffle[0x9c98b00]:_give_back(8144) Shuffle[0x9c98b00]:write_callback() Shuffle[0x9c98b00]:_send_more(0) Shuffle[0x9c98b00]:_request(8192) from (nil) Shuffle[0x9c98b00]:__send_more_callback(8192) Shuffle[0x9c98b00]:__send_more_callback(): source done: 0x9081d48 Shuffle[0x9c98b00]:__send_more_callback(): sending(46) Shuffle[0x9c98b00]:__send_more_callback(): sending(46): sent 46 Shuffle[0x9c98b00]:_give_back(8146) Shuffle[0x9c98b00]:write_callback() Shuffle[0x9c98b00]:_send_more(0) Shuffle[0x9c98b00]:_request(8192) from (nil)
Without the patch attached below, the argument to __send_more_callback becomes -2 (from the leftovers.len), and the program crashes with a SEGV shortly after removing the callbacks. With the patch, the following happens:
Shuffle[0x9c98b00]:__send_more_callback(8192) Shuffle[0x9c98b00]:__send_more_callback(): source done: 0x9c45038 Shuffle[0x9c98b00]:__send_more_callback(): read pending Shuffle[0x9c98b00]:__remove_calllbacks(Pike) Shuffle[0x9c98b00]:_give_back(8192) Shuffle[0x9c98b00]:_set_callbacks() Shuffle[0x9c98b00]:__set_calllbacks(Pike)
However, at this point: - Data has been read from the CGIWrapper - But no trace of that appears in the output - I.e. in the output the six constant strings simply stick together.
Shuffle[0x9c98b00]:write_callback() Shuffle[0x9c98b00]:_send_more(0) Shuffle[0x9c98b00]:_request(8192) from (nil) Shuffle[0x9c98b00]:__send_more_callback(8192) Shuffle[0x9c98b00]:__send_more_callback(): source done: 0x9f5d690 Shuffle[0x9c98b00]:__send_more_callback(): sending(17) Shuffle[0x9c98b00]:__send_more_callback(): sending(17): sent 17 Shuffle[0x9c98b00]:_give_back(8175) Shuffle[0x9c98b00]:write_callback() Shuffle[0x9c98b00]:_send_more(0) Shuffle[0x9c98b00]:_request(8192) from (nil) Shuffle[0x9c98b00]:__send_more_callback(8192) Shuffle[0x9c98b00]:__send_more_callback(): source done: 0x9c45068 Shuffle[0x9c98b00]:__send_more_callback(): sending(16) Shuffle[0x9c98b00]:__send_more_callback(): sending(16): sent 16 Shuffle[0x9c98b00]:_give_back(8176) Shuffle[0x9c98b00]:write_callback() Shuffle[0x9c98b00]:_send_more(0) Shuffle[0x9c98b00]:_request(8192) from (nil) Shuffle[0x9c98b00]:__send_more_callback(8192) Shuffle[0x9c98b00]:__send_more_callback(): source done: 0x9c45098 Shuffle[0x9c98b00]:__send_more_callback(): sending(15) Shuffle[0x9c98b00]:__send_more_callback(): sending(15): sent 15 Shuffle[0x9c98b00]:_give_back(8177) Shuffle[0x9c98b00]:write_callback() Shuffle[0x9c98b00]:_send_more(0) Shuffle[0x9c98b00]:_request(8192) from (nil) Shuffle[0x9c98b00]:__send_more_callback(8192) Shuffle[0x9c98b00]:__send_more_callback(): source done: 0x9c988c8 Shuffle[0x9c98b00]:__send_more_callback(): no sources Shuffle[0x9c98b00]:_give_back(8192) Shuffle[0x9c98b00]:_all_done(0) Shuffle[0x9c98b00]:_all_done(0): Calling done callback: 0x8d0141c Shuffle[0x9c98b00]:sent_data() --> 189 Shuffle[0x9c98b00]:_remove_callbacks() Shuffle[0x9c98b00]:__remove_calllbacks(Pike) Shuffle[0x9c98b00]:exit()
Now, besides this patch, which prevents it from segfaulting, I'm a bit lost now. The shuffler innards are a maze of twists, turns and callbacks. I must be close, but the non-linear program flow is difficult to decipher.
If leftovers.len ever go below 0, something is seriously wrong elsewhere.
/ Per Hedbor ()
Previous text:
2004-06-20 23:27: Subject: Bilaga till: Drowning in the Shuffler (SEGV)
Index: Shuffler.cmod
RCS file: /pike/data/cvsroot/Pike/7.7/src/post_modules/Shuffler/Shuffler.cmod,v retrieving revision 1.29 diff -u -r1.29 Shuffler.cmod --- Shuffler.cmod 4 Apr 2004 15:03:37 -0000 1.29 +++ Shuffler.cmod 20 Jun 2004 21:15:29 -0000 @@ -26,7 +26,7 @@ #include "shuffler.h" #include "sources.h"
-#if 0 +#if 1 //srb #include <stdio.h> #define SHUFFLE_DEBUG1(fmt, arg1) fprintf(stderr,"Shuffle[%p]:" fmt, arg1) #define SHUFFLE_DEBUG2(fmt, arg1, arg2) fprintf(stderr,"Shuffle[%p]:" fmt, arg1, arg2) @@ -270,7 +270,7 @@ { int l = BLOCK; SHUFFLE_DEBUG2("_send_more(%d)\n", t, fd );
- if( t->leftovers.len )
- if( t->leftovers.len>0 ) l = t->leftovers.len; _request( t, l ); }
/ Brevbäraren
Per Hedbor () @ Pike (-) developers forum wrote:
If leftovers.len ever go below 0, something is seriously wrong elsewhere.
There is. The callbacks are seriously hosed, when pipes are involved.
Please review the attached patch and correct it (or tell me what I did wrong). I made an attempt at changing the callback setting to be conforming to Pike 7.6 and later. I know next to nothing about the callbacks, nor in the old, nor in the newer Pikes.
All I can say, is that it compiles and looked like a good idea. But it doesn't work. So, where did I go wrong?
Well, it looks good except for one problem: If I am not incorrect this no longer bypass the pike-level code when reading from 'pure' FD:s.
That was the reason for the check for t->fd, anyway.
/ Per Hedbor ()
Previous text:
2004-06-21 04:11: Subject: Bilaga till: Re: Bilaga till: Drowning in the Shuffler (SEGV)
Index: Shuffler.cmod
RCS file: /pike/data/cvsroot/Pike/7.7/src/post_modules/Shuffler/Shuffler.cmod,v retrieving revision 1.29 diff -u -r1.29 Shuffler.cmod --- Shuffler.cmod 4 Apr 2004 15:03:37 -0000 1.29 +++ Shuffler.cmod 21 Jun 2004 01:45:20 -0000 @@ -81,6 +81,8 @@
PIKECLASS Shuffle {
- CVAR struct fd_callback_box box; /* must be first? */
- PIKEVAR object shuffler; /*! @decl Shuffler shuffler; *! The @[Shuffler] that owns this @[Shuffle] object
@@ -102,52 +104,41 @@ CVAR struct object *this_obj; /* callbacks only have the struct */ CVAR struct object *file_obj; CVAR int callback;
CVAR int write_callback;
CVAR int fd; CVAR int sent; CVAR ShuffleState state;
CVAR struct data leftovers;
static void _send_more( int fd, struct Shuffle_struct *t );
- static void _send_more( struct Shuffle_struct *t ); static void __set_callbacks( struct Shuffle_struct *t ) {
- SHUFFLE_DEBUG2("__set_calllbacks(%s)\n", t,(t->fd>0?"C":"Pike") );
- if( t->fd >= 0 )
set_write_callback( t->fd, (void *)_send_more, t );- else if( t->file_obj && t->file_obj->prog )
- {
ref_push_object( t->this_obj );Pike_sp[-1].type = PIKE_T_FUNCTION;Pike_sp[-1].subtype = t->write_callback;safe_apply( t->file_obj, "set_write_callback", 1 );pop_stack();- }
- SHUFFLE_DEBUG2("__set_calllbacks(%s)\n", t,(t->box.fd>0?"C":"Pike") );
- if( t->box.fd >= 0 || t->file_obj && t->file_obj->prog )
set_fd_callback_events(&t->box, PIKE_BIT_FD_WRITE);
{ SHUFFLE_DEBUG1("EEP: No destination! Cannot set callbacks\n",t);
} }
static void __remove_callbacks( struct Shuffle_struct *t ) {
SHUFFLE_DEBUG2("__remove_calllbacks(%s)\n", t, (t->fd>=0?"C":"Pike") );
if( t->fd >= 0 )
set_write_callback( t->fd, 0,0 );else if( t->file_obj && t->file_obj->prog )
{
push_int(0);safe_apply( t->file_obj, "set_write_callback", 1 );pop_stack();}
- SHUFFLE_DEBUG2("__remove_calllbacks(%s)\n", t, (t->box.fd>=0?"C":"Pike") );
- if( t->box.fd >= 0 || t->file_obj && t->file_obj->prog )
set_fd_callback_events(&t->box, 0);
- { SHUFFLE_DEBUG1("EEP: No destination! Cannot remove callbacks\n",t);
- } }
- static int got_shuffler_event(struct fd_callback_box*box, int event) {
- struct Shuffle_struct*u = (struct Shuffle_struct*) box;
+#ifdef PIKE_DEBUG
- if (event != PIKE_FD_WRITE)
Pike_fatal ("Got unexpected event %d.\n", event);+#endif
- _send_more(u);
- }
- PIKEFUN void set_throttler( object t ) /*! @decl void set_throttler(Throttler t) *! Calling this function overrides the @[Shuffler] global throttler.
@@ -212,9 +203,8 @@ debug_malloc_touch(THIS->this_obj); THIS->callback = find_identifier("send_more_callback",Pike_fp->current_object->prog);
- THIS->write_callback =
find_identifier("write_callback",Pike_fp->current_object->prog);
INIT_FD_CALLBACK_BOX (&THIS->box, default_backend, 0,
-1, 0, got_shuffler_event);SHUFFLE_DEBUG1("init()\n", THIS ); }
@@ -222,7 +212,7 @@ EXIT { SHUFFLE_DEBUG1("exit()\n", THIS );
- THIS->fd = -1;
- THIS->box.fd = -1; if( THIS->file_obj ) { free_object( THIS->file_obj );
@@ -266,10 +256,10 @@ __send_more_callback( t, amount ); }
- static void _send_more( int fd, struct Shuffle_struct *t )
- static void _send_more(struct Shuffle_struct *t) { int l = BLOCK;
- SHUFFLE_DEBUG2("_send_more(%d)\n", t, fd );
- SHUFFLE_DEBUG2("_send_more(%d)\n", t, t->box.fd ); if( t->leftovers.len ) l = t->leftovers.len; _request( t, l );
@@ -454,10 +444,10 @@ SHUFFLE_DEBUG2("__send_more_callback(): sending(%d)\n", t, MINIMUM(amount,t->leftovers.len)); sent = -1;
- if( t->fd >= 0 )
- if( t->box.fd >= 0 ) { THREADS_ALLOW();
sent = fd_write( t->fd, t->leftovers.data+t->leftovers.off,
sent = fd_write( t->box.fd, t->leftovers.data+t->leftovers.off, MINIMUM(amount,t->leftovers.len) ); THREADS_DISALLOW();@@ -519,15 +509,6 @@ _give_back( THIS, amount ); }
- /*! @decl void write_callback( mixed|void x )
- */
- PIKEFUN void write_callback( mixed|void x )
- optflags OPT_SIDE_EFFECT;
- {
- SHUFFLE_DEBUG1("write_callback()\n", THIS );
- _send_more( 0,THIS );
- }
- /*! @decl void create(object fd, object shuffler, mixed throttler,@ *! mixed backend) */
@@ -553,20 +534,17 @@ }
if (find_identifier("query_fd", fd->prog) < 0)
THIS->fd = -1;
THIS->box.fd = -1;
THIS->fd = Pike_sp[-1].u.integer;
THIS->box.fd = Pike_sp[-1].u.integer; pop_stack();
- if( THIS->fd >= 0 )
- if( THIS->box.fd >= 0 ) {
set_nonblocking( THIS->fd, 1 );set_read_callback( THIS->fd, 0, 0 );set_write_callback( THIS->fd, 0, 0 );set_read_oob_callback( THIS->fd, 0, 0 );set_write_oob_callback( THIS->fd, 0, 0 );
set_nonblocking( THIS->box.fd, 1 );set_fd_callback_events(&THIS->box, 0);@@ -577,7 +555,7 @@ pop_stack(); }
- if( (THIS->fd >= 0) && (backend->type == PIKE_T_OBJECT) )
- if( (THIS->box.fd >= 0) && (backend->type == PIKE_T_OBJECT) ) { ref_push_object( THIS->file_obj ); safe_apply( backend->u.object, "add_file", 1 );
/ Brevbäraren
Per Hedbor () @ Pike (-) developers forum wrote:
Well, it looks good except for one problem: If I am not incorrect this no longer bypass the pike-level code when reading from 'pure' FD:s.
That was the reason for the check for t->fd, anyway.
So that means that: - The patch is ok as such, and needed anyway? - The patch is incomplete, and needs to be extended with ... to be able to read from pure FDs?
Or does it mean that the whole if check for t->fd can be scrapped because all things are equal now?
I'm confused, but this is my normal state :-).
The patch is probably OK as it is, and it's needed since it fixes crash-bugs.
However, it might now be somewhat slower to use the shuffler with 'pure' FD:s. I don't really know, though, since I have not checked the new backend code (the change in the backend code is most likely the reason for the crash)
/ Per Hedbor ()
Previous text:
2004-06-21 10:44: Subject: Re: Bilaga till: Re: Bilaga till: Drowning in the Shuffler (SEGV)
Per Hedbor () @ Pike (-) developers forum wrote:
Well, it looks good except for one problem: If I am not incorrect this no longer bypass the pike-level code when reading from 'pure' FD:s.
That was the reason for the check for t->fd, anyway.
So that means that:
- The patch is ok as such, and needed anyway?
- The patch is incomplete, and needs to be extended with ...
to be able to read from pure FDs?
Or does it mean that the whole if check for t->fd can be scrapped because all things are equal now?
I'm confused, but this is my normal state :-).
Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless).
"People who think they know everything are annoying to those of us who do."
/ Brevbäraren
Per Hedbor () @ Pike (-) developers forum wrote:
The patch is probably OK as it is, and it's needed since it fixes crash-bugs.
Ok.
However, it might now be somewhat slower to use the shuffler with 'pure' FD:s. I don't really know, though, since I have not checked the new backend code (the change in the backend code is most likely the reason for the crash)
Maybe. Martin thought it was the change in the way callbacks work.
However, the patch as such is incomplete. The shuffler doesn't work now (I have not checked it in yet). I'm out of depth here though. I already went out on a limb to create the current patch. But, now that it doesn't work fully, I don't know how to debug or fix it.
Could someone more familiar with callbacks check the changes I made, or someone (Grubba?) more familiar with the innards of the shuffler check why this breaks the shuffler?
someone (Grubba?) more familiar with the innards of the shuffler
How did grubba end up shuffler guru?
/ Martin Nilsson (räfsfiskal)
Previous text:
2004-06-21 12:04: Subject: Re: Drowning in the Shuffler (SEGV)
Per Hedbor () @ Pike (-) developers forum wrote:
The patch is probably OK as it is, and it's needed since it fixes crash-bugs.
Ok.
However, it might now be somewhat slower to use the shuffler with 'pure' FD:s. I don't really know, though, since I have not checked the new backend code (the change in the backend code is most likely the reason for the crash)
Maybe. Martin thought it was the change in the way callbacks work.
However, the patch as such is incomplete. The shuffler doesn't work now (I have not checked it in yet). I'm out of depth here though. I already went out on a limb to create the current patch. But, now that it doesn't work fully, I don't know how to debug or fix it.
Could someone more familiar with callbacks check the changes I made, or someone (Grubba?) more familiar with the innards of the shuffler check why this breaks the shuffler? -- Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless).
"People who think they know everything are annoying to those of us who do."
/ Brevbäraren
Martin Nilsson (r?fsfiskal) @ Pike (-) developers forum wrote:
someone (Grubba?) more familiar with the innards of the shuffler
How did grubba end up shuffler guru?
Sorry. I checked the last CVS update, glanced the wrong name from there.
Isn't grubba the overall super-guru when you have difficult problems and noone else to point at..?
/ Marcus Agehall (Scanian)
Previous text:
2004-06-21 12:08: Subject: Re: Drowning in the Shuffler (SEGV)
someone (Grubba?) more familiar with the innards of the shuffler
How did grubba end up shuffler guru?
/ Martin Nilsson (räfsfiskal)
I guess I am the shuffler guru, since I wrote it. However, I don't really know how the new backend works.
As soon as I take my vacation I will try to fix some performance problems in the shuffler, and perhaps also add a new non-pike-lock stealing mode of operation for pure FD:s (the latter would however not be all that portable, unless I copy large chunks of the backend code, which is most likely not a good idea)
Most likely the only thing you have to do to get the shuffler working is to disable the b_source_normal_file.c source by returning 0 at the very start of source_normal_file_make (eg, comment out line 88)
/ Per Hedbor ()
Previous text:
2004-06-21 12:04: Subject: Re: Drowning in the Shuffler (SEGV)
Per Hedbor () @ Pike (-) developers forum wrote:
The patch is probably OK as it is, and it's needed since it fixes crash-bugs.
Ok.
However, it might now be somewhat slower to use the shuffler with 'pure' FD:s. I don't really know, though, since I have not checked the new backend code (the change in the backend code is most likely the reason for the crash)
Maybe. Martin thought it was the change in the way callbacks work.
However, the patch as such is incomplete. The shuffler doesn't work now (I have not checked it in yet). I'm out of depth here though. I already went out on a limb to create the current patch. But, now that it doesn't work fully, I don't know how to debug or fix it.
Could someone more familiar with callbacks check the changes I made, or someone (Grubba?) more familiar with the innards of the shuffler check why this breaks the shuffler? -- Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless).
"People who think they know everything are annoying to those of us who do."
/ Brevbäraren
Per Hedbor () @ Pike (-) developers forum wrote:
I guess I am the shuffler guru, since I wrote it. However, I don't
Sorry, wrong attribution :-).
really know how the new backend works.
Who wrote that, and isn't it documented somewhere or even in a mail on this list, outlined? Or is it just UTSL?
problems in the shuffler, and perhaps also add a new non-pike-lock stealing mode of operation for pure FD:s (the latter would however not be all that portable, unless I copy large chunks of the backend code,
I'll settle for non-portable, but Posix, and then provide an alternate slower version for other poor souls.
Most likely the only thing you have to do to get the shuffler working is to disable the b_source_normal_file.c source by returning 0 at the very start of source_normal_file_make (eg, comment out line 88)
Ok, will try that. Watch this space.
Stephen R. van den Berg wrote:
Per Hedbor () @ Pike (-) developers forum wrote:
Most likely the only thing you have to do to get the shuffler working is to disable the b_source_normal_file.c source by returning 0 at the very start of source_normal_file_make (eg, comment out line 88)
This is what I get as a result: ADDSOURCE Stdio.File(0, 0, 777 /* fd=26 */) 0 -1 Shuffle[0x9df7148]:add_source(XX,0,-1) --> 0xa029cd0 Shuffle[0x9df7148]:add_source(XX,0,-1) --> 0x9e971d8 Shuffle[0x9df7148]:add_source(XX,0,-1) --> 0x9e97278 Shuffle[0x9df7148]:add_source(XX,0,-1) --> 0xa094818 Shuffle[0x9df7148]:add_source(XX,0,-1) --> 0x9df7110 Shuffle[0x9df7148]:add_source(XX,0,-1) --> 0x8e09748 Shuffle[0x9df7148]:add_source(XX,0,-1) --> 0x8e09778 : HITME LATER "" Shuffle[0x9df7148]:set_done_callback(0x8cf1e24) Shuffle[0x9df7148]:start() Shuffle[0x9df7148]:_set_callbacks() Shuffle[0x9df7148]:__set_calllbacks(Pike) : CGI: CGIScript::check_pid() Cannot call functions in destructed objects. Unknown program: destructed object->function(0) _static_modules.Builtin()->Backend: Pike.Backend(0)->`()(_static_modules.files() ->sendfile()) : CGI: Wrapper::write_callback() 13:58:09 : CGI: Wrapper::read_callback(0, "Status: 0\r\nContent-type: text/html
And then all Shuffle activities cease (the destructed object is key here, in case you were wondering :-).
The patch I used, in addition to the set presented earlier, is (temporarily, not intended to be used at home like this):
RCS file: /pike/data/cvsroot/Pike/7.7/src/post_modules/Shuffler/b_source_normal_file.c,v retrieving revision 1.12 diff -u -r1.12 b_source_normal_file.c --- pike/src/post_modules/Shuffler/b_source_normal_file.c 2 May 2004 00:35:16 -0000 1.12 +++ pike/src/post_modules/Shuffler/b_source_normal_file.c 21 Jun 2004 12:58:56 -0000 @@ -85,6 +85,8 @@ push_text("files.Fd_ref"); SAFE_APPLY_MASTER("resolv",1); Fd_ref_program = program_from_svalue(Pike_sp-1); + pop_stack(); + return 0; if (!Fd_ref_program) { pop_stack(); return 0;
Note, my original patch set has not been checked in yet (since it doesn't give me a working shuffler).
"New backend" is a bit of an exaggeration imo. Anyway, what's necessary to know about "boxes" is in backend.h. It's really not that complicated since the user doesn't have to bother with the refcounting intricacies besides setting ref_obj (that's the whole idea with them).
I looked briefly at the shuffler code but couldn't figure out the right place to put the box. Should be the same object that holds the fd, but the callbacks were used in some clever way so it wasn't that straighforward, as I remember.
Anyway, the old interface should still work, even for multiple backends right now (although I'd like to do away with the fd-to-backend map soon). Maybe turning on POLL_DEBUG in backend.cmod will show something interesting (logs a lot, though).
/ Martin Stjernholm, Roxen IS
Previous text:
2004-06-21 12:16: Subject: Re: Drowning in the Shuffler (SEGV)
I guess I am the shuffler guru, since I wrote it. However, I don't really know how the new backend works.
As soon as I take my vacation I will try to fix some performance problems in the shuffler, and perhaps also add a new non-pike-lock stealing mode of operation for pure FD:s (the latter would however not be all that portable, unless I copy large chunks of the backend code, which is most likely not a good idea)
Most likely the only thing you have to do to get the shuffler working is to disable the b_source_normal_file.c source by returning 0 at the very start of source_normal_file_make (eg, comment out line 88)
/ Per Hedbor ()
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
"New backend" is a bit of an exaggeration imo. Anyway, what's necessary to know about "boxes" is in backend.h. It's really not that complicated since the user doesn't have to bother with the refcounting intricacies besides setting ref_obj (that's the whole idea with them).
I didn't use ref_obj in the current patch set. After reading the docs, I'm not convinced that it should be used in the Shuffler. Correct choice?
I looked briefly at the shuffler code but couldn't figure out the right place to put the box. Should be the same object that holds the fd, but the callbacks were used in some clever way so it wasn't that straighforward, as I remember.
I put it in the same struct, and removed the fd there. So that should be about right.
What springs to mind, is, is using INIT_FD_CALLBACK_BOX in just the INIT section enough? Or should it be repeated at create time?
I didn't use ref_obj in the current patch set. After reading the docs, I'm not convinced that it should be used in the Shuffler. Correct choice?
Probably not. Why do you think so? If you set it to NULL there won't be any ref from the backend to the shuffler object when callbacks are in use. Typically the only case when ref_obj should be NULL is when it's statically allocated.
Btw, you probably can replace this_obj with box.ref_obj, provided this_obj is what it looks like.
I put it in the same struct, and removed the fd there. So that should be about right.
If you say so. Wasn't there some circumstances when other fd's could be passed to the callbacks or something like that?
What springs to mind, is, is using INIT_FD_CALLBACK_BOX in just the INIT section enough? Or should it be repeated at create time?
Yes and no. An already hooked box must not be hooked in again.
Regarding your question "must be first?": Yes, it looks like it must be first since you implicitly convert a pointer to the box to a pointer to Shuffle_struct in e.g. _send_more.
/ Martin Stjernholm, Roxen IS
Previous text:
2004-06-22 02:13: Subject: Re: Drowning in the Shuffler (SEGV)
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
"New backend" is a bit of an exaggeration imo. Anyway, what's necessary to know about "boxes" is in backend.h. It's really not that complicated since the user doesn't have to bother with the refcounting intricacies besides setting ref_obj (that's the whole idea with them).
I didn't use ref_obj in the current patch set. After reading the docs, I'm not convinced that it should be used in the Shuffler. Correct choice?
I looked briefly at the shuffler code but couldn't figure out the right place to put the box. Should be the same object that holds the fd, but the callbacks were used in some clever way so it wasn't that straighforward, as I remember.
I put it in the same struct, and removed the fd there. So that should be about right.
What springs to mind, is, is using INIT_FD_CALLBACK_BOX in just the INIT section enough? Or should it be repeated at create time? -- Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless).
How many weeks are there in a light year?
/ Brevbäraren
I retried converting the Shuffler to the new fd_boxes... (The fix I did stands, BTW, it's just that there is more damage than meets the eye).
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
Btw, you probably can replace this_obj with box.ref_obj, provided this_obj is what it looks like.
Ok, done.
I put it in the same struct, and removed the fd there. So that should be about right.
If you say so. Wasn't there some circumstances when other fd's could be passed to the callbacks or something like that?
No. Not that I noticed.
What springs to mind, is, is using INIT_FD_CALLBACK_BOX in just the INIT section enough? Or should it be repeated at create time?
Yes and no. An already hooked box must not be hooked in again.
Ok, understood it mostly. Know more about these boxes than I ever intended now :-).
Regarding your question "must be first?": Yes, it looks like it must be first since you implicitly convert a pointer to the box to a pointer to Shuffle_struct in e.g. _send_more.
Yes. Found that out, after adding the struct member. Anyway, it's not required anymore, since I use ref_obj now.
However... The following problem occurs:
ChiliMoon server, has a file descriptor on a kept alive connection to the client requesting a web page. This descriptor is already hooked on an FD_BOX from within http.pike.
Now, this descriptor is given (temporarily) to the shuffler, to write out any data it wants to write, then, the descriptor should be used by http.pike again to accept further requests on the kept-alive connection.
The problem obviously is, that the shuffler tries to setup an FD_BOX on that same FD. The FD_BOX code complains (rightfully so). How are we going to solve this? - The current Stdio code does not allow me to unhook the FD box temporarily, so fixing it from within http.pike is impossible. - The Shuffler has no way to get at the old handler and temporarily disable it, and/or chain to it.
So, should I extend the interface in backend.cmod to allow retrieving the old handler, so the shuffler can take evasive action? Or, should I extend the fd_box to include some kind of chain pointer to the previous fd_box, which are then all called in succession by the backend handler? Or, should the shuffler do something clever with the backend setting?
Please advise.
Stephen R. van den Berg wrote:
I retried converting the Shuffler to the new fd_boxes... (The fix I did stands, BTW, it's just that there is more damage than meets the eye).
Fixed it good now. I think. Per Hedbor, Martin Stjernholm, please verify what I came up with.
Regarding your question "must be first?": Yes, it looks like it must be first since you implicitly convert a pointer to the box to a pointer to Shuffle_struct in e.g. _send_more.
Yes. Found that out, after adding the struct member. Anyway, it's not required anymore, since I use ref_obj now.
Well, not quite. Had to resort to some good old fashioned offsetof() trick to make it work due to the hidden struct members. This trick can be used in the other source files as well, so that the box element doesn't need to be first anymore.
Now, this descriptor is given (temporarily) to the shuffler, to write out any data it wants to write, then, the descriptor should be used by http.pike again to accept further requests on the kept-alive connection.
The problem obviously is, that the shuffler tries to setup an FD_BOX on that same FD. The FD_BOX code complains (rightfully so).
So, should I extend the interface in backend.cmod to allow retrieving the old handler, so the shuffler can take evasive action?
This is what I ended up doing. Not sure if this is the cleanest solution, but it sure was the most non-intrusive change in the backend code.
Or, should I extend the fd_box to include some kind of chain pointer to the previous fd_box, which are then all called in succession by the backend handler?
I briefly tried this first. But after finding that there active and non-active lists of FD boxes, I decided to give up on this, too convoluted for a solution which will only be used in the Shuffler.
Or, should the shuffler do something clever with the backend setting?
I couldn't come up with anything sensible.
Please advise.
Change that into, please review; checking in proposed changes now (it seems to work now).
Stephen R. van den Berg wrote:
Stephen R. van den Berg wrote:
I retried converting the Shuffler to the new fd_boxes...
Fixed it good now. I think.
So I thought. I was wrong.
Per Hedbor, Martin Stjernholm, please verify what I came up with.
Well, I had to verify it myself, and as it turns out, I deleted too much. I understand it a bit more now. Just enough to cover my tracks. Fix fixed.
Please verify once more.
Regarding your question "must be first?": Yes, it looks like it must be first since you implicitly convert a pointer to the box to a pointer to Shuffle_struct in e.g. _send_more.
Yes. Found that out, after adding the struct member. Anyway, it's not required anymore, since I use ref_obj now.
You mean, there's nothing else in the Shuffle_struct you need there? Otherwise I'd like to point out that you can't reliably convert a struct object * to a pointer to your storage since it might be inherited multiple times.
- The current Stdio code does not allow me to unhook the FD box temporarily, so fixing it from within http.pike is impossible.
- The Shuffler has no way to get at the old handler and temporarily disable it, and/or chain to it.
Sorry for being a bit late in responding to this. I think it's an interface problem between the Fd class and Shuffler. The solution is to either take over the fd from the Fd object so that it temporarily becomes closed, or continue using the box directly in it.
In the first case, the Fd class should provide an release_fd() function which works like query_fd() but additionally deregisters the fd from itself (by using change_fd_for_box(&THIS->box,-1) to keep callback settings and backend pointers). The Fd object is then effectively closed during the time the shuffler uses the fd. Afterwards the shuffler should use another function, say Fd.take_fd, which does the reverse, after the shuffler has decommissioned its own box. Fd.open can already take an fd, but I don't think it does the right thing here since it sets FILE_NOT_OPENED too.
If you want the Fd object to continue working in parallell then you need to add an interface to it so that it can chain on to your callback, i.e. sort of a C level equivalent to the pike callback interface. It would presumably consist of an fd_box_callback, a pointer to your own storage, and the events bitfield. Another possibility is to simply use the pike level callbacks here too, but that might give problems with cyclic references.
In either case, there's no need to do anything at all in backend.cmod.
Your solution right now where you unhook the box in the Fd object under its feet is dangerous since the Fd object assumes that its box stays hooked. For instance, what happens if someone destructs it while the shuffler uses its fd? What happens if pike callbacks are registered? Are they silently ignored while the shuffler callbacks take precedence? If so, is that really right?
/ Martin Stjernholm, Roxen IS
Previous text:
2004-06-27 03:28: Subject: Two fd_boxes, same FD (Re: Drowning in the Shuffler (SEGV))
I retried converting the Shuffler to the new fd_boxes... (The fix I did stands, BTW, it's just that there is more damage than meets the eye).
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
Btw, you probably can replace this_obj with box.ref_obj, provided this_obj is what it looks like.
Ok, done.
I put it in the same struct, and removed the fd there. So that should be about right.
If you say so. Wasn't there some circumstances when other fd's could be passed to the callbacks or something like that?
No. Not that I noticed.
What springs to mind, is, is using INIT_FD_CALLBACK_BOX in just the INIT section enough? Or should it be repeated at create time?
Yes and no. An already hooked box must not be hooked in again.
Ok, understood it mostly. Know more about these boxes than I ever intended now :-).
Regarding your question "must be first?": Yes, it looks like it must be first since you implicitly convert a pointer to the box to a pointer to Shuffle_struct in e.g. _send_more.
Yes. Found that out, after adding the struct member. Anyway, it's not required anymore, since I use ref_obj now.
However... The following problem occurs:
ChiliMoon server, has a file descriptor on a kept alive connection to the client requesting a web page. This descriptor is already hooked on an FD_BOX from within http.pike.
Now, this descriptor is given (temporarily) to the shuffler, to write out any data it wants to write, then, the descriptor should be used by http.pike again to accept further requests on the kept-alive connection.
The problem obviously is, that the shuffler tries to setup an FD_BOX on that same FD. The FD_BOX code complains (rightfully so). How are we going to solve this?
- The current Stdio code does not allow me to unhook the FD box
temporarily, so fixing it from within http.pike is impossible.
- The Shuffler has no way to get at the old handler and temporarily
disable it, and/or chain to it.
So, should I extend the interface in backend.cmod to allow retrieving the old handler, so the shuffler can take evasive action? Or, should I extend the fd_box to include some kind of chain pointer to the previous fd_box, which are then all called in succession by the backend handler? Or, should the shuffler do something clever with the backend setting?
Please advise.
Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless).
Real programmers don't produce results, they return exit codes.
/ Brevbäraren
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
Regarding your question "must be first?": Yes, it looks like it must be first since you implicitly convert a pointer to the box to a pointer to Shuffle_struct in e.g. _send_more.
Yes. Found that out, after adding the struct member. Anyway, it's not required anymore, since I use ref_obj now.
You mean, there's nothing else in the Shuffle_struct you need there? Otherwise I'd like to point out that you can't reliably convert a struct object * to a pointer to your storage since it might be inherited multiple times.
Erm, found that out shortly after writing previous message, that's why I'm using yet another method in the "final" fix (offsetof).
- The current Stdio code does not allow me to unhook the FD box temporarily, so fixing it from within http.pike is impossible.
Sorry for being a bit late in responding to this. I think it's an interface problem between the Fd class and Shuffler. The solution is to either take over the fd from the Fd object so that it temporarily becomes closed, or continue using the box directly in it.
Ok, I'll pick method one, I think.
If you want the Fd object to continue working in parallell then you need to add an interface to it so that it can chain on to your
No need, the "specs" for the Shuffler say that it takes over the connection; would be a bit pointless if it didn't.
Your solution right now where you unhook the box in the Fd object under its feet is dangerous since the Fd object assumes that its box stays hooked. For instance, what happens if someone destructs it while the shuffler uses its fd? What happens if pike callbacks are registered? Are they silently ignored while the shuffler callbacks take precedence? If so, is that really right?
Well, I know the solution has flaws. But, in defense of the current solution, the application calling the Shuffler should know better than to try funny things with the FD while they have been passed on to the Shuffler.
/.../ the application calling the Shuffler should know better than to try funny things with the FD while they have been passed on to the Shuffler.
Certainly, but bugs do occur and simultaneous use of the same fd from different places is typically tricky to sort out (the oldest and most bothersome bugs with ssl connections were due to that). It's really valuable to get good errors when you do something wrong; it happens to the best of us.
/ Martin Stjernholm, Roxen IS
Previous text:
2004-07-01 17:18: Subject: Re: Two fd_boxes, same FD (Re: Drowning in the Shuffler (SEGV))
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
Regarding your question "must be first?": Yes, it looks like it must be first since you implicitly convert a pointer to the box to a pointer to Shuffle_struct in e.g. _send_more.
Yes. Found that out, after adding the struct member. Anyway, it's not required anymore, since I use ref_obj now.
You mean, there's nothing else in the Shuffle_struct you need there? Otherwise I'd like to point out that you can't reliably convert a struct object * to a pointer to your storage since it might be inherited multiple times.
Erm, found that out shortly after writing previous message, that's why I'm using yet another method in the "final" fix (offsetof).
- The current Stdio code does not allow me to unhook the FD box temporarily, so fixing it from within http.pike is impossible.
Sorry for being a bit late in responding to this. I think it's an interface problem between the Fd class and Shuffler. The solution is to either take over the fd from the Fd object so that it temporarily becomes closed, or continue using the box directly in it.
Ok, I'll pick method one, I think.
If you want the Fd object to continue working in parallell then you need to add an interface to it so that it can chain on to your
No need, the "specs" for the Shuffler say that it takes over the connection; would be a bit pointless if it didn't.
Your solution right now where you unhook the box in the Fd object under its feet is dangerous since the Fd object assumes that its box stays hooked. For instance, what happens if someone destructs it while the shuffler uses its fd? What happens if pike callbacks are registered? Are they silently ignored while the shuffler callbacks take precedence? If so, is that really right?
Well, I know the solution has flaws. But, in defense of the current solution, the application calling the Shuffler should know better than to try funny things with the FD while they have been passed on to the Shuffler. -- Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless).
"The difficult we do today; the impossible takes a little longer."
/ Brevbäraren
Stephen R. van den Berg wrote:
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
- The current Stdio code does not allow me to unhook the FD box temporarily, so fixing it from within http.pike is impossible.
Sorry for being a bit late in responding to this. I think it's an interface problem between the Fd class and Shuffler. The solution is to either take over the fd from the Fd object so that it temporarily becomes closed, or continue using the box directly in it.
Ok, I'll pick method one, I think.
Shuffler fixes, take four, are in. Per, mast, please review.
Also, I found an unrelated pop-bug in set_buffer, I believe (found it while trying to learn how the Pike stack works). Then again, maybe I don't understand what needs to be popped and pushed.
No, that was a real bug.
Btw, you forgot to check the argument in take_fd. I fixed that.
/ Martin Stjernholm, Roxen IS
Previous text:
2004-07-02 11:57: Subject: Re: Two fd_boxes, same FD (Re: Drowning in the Shuffler (SEGV))
Stephen R. van den Berg wrote:
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
- The current Stdio code does not allow me to unhook the FD box temporarily, so fixing it from within http.pike is impossible.
Sorry for being a bit late in responding to this. I think it's an interface problem between the Fd class and Shuffler. The solution is to either take over the fd from the Fd object so that it temporarily becomes closed, or continue using the box directly in it.
Ok, I'll pick method one, I think.
Shuffler fixes, take four, are in. Per, mast, please review.
Also, I found an unrelated pop-bug in set_buffer, I believe (found it while trying to learn how the Pike stack works). Then again, maybe I don't understand what needs to be popped and pushed. -- Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless). E-mails should be like a lady's skirt: Long enough to cover the subject, and short enough to be interesting.
/ Brevbäraren
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
No, that was a real bug.
Now, if everyone would proofread 40 lines every day, Pike would soon be bugfree :-).
Btw, you forgot to check the argument in take_fd. I fixed that.
Well, I had that first. Then I thought that this should be taken care of by the compiler and the prototypes. So, you're saying that the compiler doesn't use the prototypes?
Well, I had that first. Then I thought that this should be taken care of by the compiler and the prototypes. So, you're saying that the compiler doesn't use the prototypes?
Sure, but you can always anonymize a function:
function f=myobject->some_function; ... f(something);
here the type of the function isn't saved, and it becomes function(mixed...:mixed|void), so the compiler can't check the types.
If you call a pike function this way, there is no disaster. But if you call a C function, it has to check everything. A .cmod-made file does that (I think), but a C hardcore module has to do it itself (get_all_args is nice then).
/ Mirar
Previous text:
2004-07-02 20:42: Subject: Re: Two fd_boxes, same FD (Re: Drowning in the Shuffler (SEGV))
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
No, that was a real bug.
Now, if everyone would proofread 40 lines every day, Pike would soon be bugfree :-).
Btw, you forgot to check the argument in take_fd. I fixed that.
Well, I had that first. Then I thought that this should be taken care of by the compiler and the prototypes. So, you're saying that the compiler doesn't use the prototypes? -- Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless). E-mails should be like a lady's skirt: Long enough to cover the subject, and short enough to be interesting.
/ Brevbäraren
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
Anyway, the old interface should still work, even for multiple backends right now (although I'd like to do away with the
Following up on this hint, I decided to dig a little deeper. Found something. A C size_t type which was compared with > 0 and which stored -1 sometimes.
Checked in the fixes.
I noticed you used INT64. Are there precautions in place if it doesn't exist?
/ Martin Stjernholm, Roxen IS
Previous text:
2004-06-23 03:28: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
Anyway, the old interface should still work, even for multiple backends right now (although I'd like to do away with the
Following up on this hint, I decided to dig a little deeper. Found something. A C size_t type which was compared with > 0 and which stored -1 sometimes.
Checked in the fixes.
Sincerely, srb@cuci.nl Stephen R. van den Berg (AKA BuGless). Every successful person has had failures but repeated failure is no guarantee of eventual success.
/ Brevbäraren
Better would probably be to use ptrdiff_t.
/ Henrik Grubbström (Lysator)
Previous text:
2004-06-23 18:04: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
I noticed you used INT64. Are there precautions in place if it doesn't exist?
/ Martin Stjernholm, Roxen IS
Wouldn't ssize_t be better, given the description (size_t, which sometimes stores -1)?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2004-06-23 18:05: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
Better would probably be to use ptrdiff_t.
/ Henrik Grubbström (Lysator)
ptrdiff_t is defined by the ANSI C standard, while I believe ssize_t is from SVID3 or XPG4.
/ Henrik Grubbström (Lysator)
Previous text:
2004-06-23 18:17: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
Wouldn't ssize_t be better, given the description (size_t, which sometimes stores -1)?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
BTW: ptrdiff_t by definition has the same range as ssize_t.
/ Henrik Grubbström (Lysator)
Previous text:
2004-06-23 18:35: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
ptrdiff_t is defined by the ANSI C standard, while I believe ssize_t is from SVID3 or XPG4.
/ Henrik Grubbström (Lysator)
Huh? You mean that ISO defines ptrdiff_t as having the same range as ssize_t, without defining ssize_t? That sounds very odd. Maybe you meant the other way around?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2004-06-23 18:36: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
BTW: ptrdiff_t by definition has the same range as ssize_t.
/ Henrik Grubbström (Lysator)
No, I mean that the definition of ptrdiff_t (difference between two related pointers) means that it has to have the same range as ssize_t (maximum size of a memory block or -1).
/ Henrik Grubbström (Lysator)
Previous text:
2004-06-23 18:41: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
Huh? You mean that ISO defines ptrdiff_t as having the same range as ssize_t, without defining ssize_t? That sounds very odd. Maybe you meant the other way around?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Really? It should be ok to have a ptrdiff_t with a larger range, as far as I can see.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2004-06-23 19:12: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
No, I mean that the definition of ptrdiff_t (difference between two related pointers) means that it has to have the same range as ssize_t (maximum size of a memory block or -1).
/ Henrik Grubbström (Lysator)
In fact, ISO defines that prtdiff_t must be able to store the value -65535 (§7.18.3:2), whereas I don't think there is any such requirement for ssize_t.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2004-06-23 19:38: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
Really? It should be ok to have a ptrdiff_t with a larger range, as far as I can see.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
As far as I recall, C requires the signed integer types to behave like 2-complement values.
/ Henrik Grubbström (Lysator)
Previous text:
2004-06-23 19:38: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
Really? It should be ok to have a ptrdiff_t with a larger range, as far as I can see.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
So? What definition prevents you from having 17 bit ptrdiff_t and 18 bit ssize_t, or vice versa, if you have a 16 bit address range?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2004-06-23 19:47: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
As far as I recall, C requires the signed integer types to behave like 2-complement values.
/ Henrik Grubbström (Lysator)
You probably even _need_ 18 bits for ssize_t, since the largest object might be 65536 chars large, and 65536 can not be represented by a 2-complement 17 bit number. For ptrdiff_t there is no problem, since the largest difference between two valid pointers is 65535, not 65536.
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Previous text:
2004-06-23 19:49: Subject: Learned to swim (Re: Drowning in the Shuffler (SEGV))
So? What definition prevents you from having 17 bit ptrdiff_t and 18 bit ssize_t, or vice versa, if you have a 16 bit address range?
/ Marcus Comstedt (ACROSS) (Hail Ilpalazzo!)
Henrik Grubbstr?m (Lysator) @ Pike (-) developers forum wrote:
As far as I recall, C requires the signed integer types to behave like 2-complement values.
No. It can be ones-complement as well.
Henrik Grubbstr?m (Lysator) @ Pike (-) developers forum wrote:
As far as I recall, C requires the signed integer types to behave like 2-complement values.
Or even unsigned values with just the high bit set.
Martin Stjernholm, Roxen IS @ Pike developers forum wrote:
I noticed you used INT64. Are there precautions in place if it doesn't exist?
Well, I simply went UTSL... Further down in that file it says:
struct source *source_stream_make( struct svalue *s, INT64 start, INT64 len )
Then I see:
res->len = len; res->skip = start;
Ergo:
INT64 len, skip;
Is needed to fullfill this.
So, any precautions necessary must have been there already, I presume. Using ssize_t or ptrdiff_t is not a good idea in this case, since we're dealing with the sizes of datastreams (not even files). So the closest you're going to get is off_t (which is signed, I believe), but since it's a stream and not a file, not even that is good enough.
So, since INT64 goes in, and a stream can be rather large (larger than any existing file), I'd suggest to keep INT64 (unless the necessary precautions have not been made; but that's Per's department :-).
Following up on this hint, I decided to dig a little deeper. Found something. A C size_t type which was compared with > 0 and which stored -1 sometimes.
pike-devel@lists.lysator.liu.se
-
Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum -
Marcus Agehall (Scanian) ＠ Pike (-) developers forum
-
Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) ＠ Pike (-) developers forum
-
Martin Nilsson (r�fsfiskal) ＠ Pike (-) developers forum
-
Martin Stjernholm, Roxen IS ＠ Pike developers forum
-
Mirar ＠ Pike developers forum
-
Per Hedbor () ＠ Pike (-) developers forum
-
Stephen R. van den Berg