Hello Everybody,
Did I ask a stupid question when I said I have problems with accessing global class variables in a function during program shutdown? If I did, I'd also like to know, no offense taken :)
Coen Schalkwijk wrote:
Did I ask a stupid question when I said I have problems with accessing global class variables in a function during program shutdown? If I did, I'd also like to know, no offense taken :)
No, you did not (I think). The only thing I can say about it is that I observe the same (strange) effect that during Roxen shutdown, the logs are filled with these existentialist error messages which weren't there before (in older versions). Other than that, you'd need a better Pike/Roxen expert to give more insights.
Just my luck then :(
On 3/29/2010 12:20, Mirar @ Pike developers forum wrote:
Probably quite the opposite, noone has any good answer... :p
I remember spraying code back in the old days with
if (this_object()) my_function();
just to avoid that kind of behavoir. That's about it. :)
Thanks for the suggestion, problem is; for this to work I have to put this check around each statement that could possibly still be called when/on shutting down.This is, of course, not acceptable :)
On 3/29/2010 13:10, Mirar @ Pike developers forum wrote:
I remember spraying code back in the old days with
if (this_object()) my_function();
just to avoid that kind of behavoir. That's about it. :)
Are you using any custom modules?
The reason I ask, is that I rarely see these messages and if I do, it's usually due to some error in the code.
One potential source of these messages, are threads and possibly call outs that are not properly disposed of when the the modules receive the stop signal.
Hi Marcus,
I am indeed using custom modules. It is also within these modules where the error occurs. Also the code generating the problems is executed from within its own (queue-)thread.
When the (queue-)thread holding module is stopped, an end-identifier is placed in the queue in order to first finish all the left over tasks in the queue and then quit. I could just quit the entire queue-thread when the module is stopped. After adding the end-signal, it's still possible for a queue to receive new items, but they will never be executed, so that should not be the problem.
Next to that I've also noticed that the cause of the error is not always an error in the custom code: some problems have been solved by placing the error generating code 'closer' to the beginning of the close() call, so it seems to be more like a time related thingy.
It still is strange that a class function will still, erm, function, while the class variables are already gone...
Coen
On 3/29/2010 14:30, Marcus Agehall (nu med K-märkt fastighet) @ Pike (-) developers forum wrote:
Are you using any custom modules?
The reason I ask, is that I rarely see these messages and if I do, it's usually due to some error in the code.
One potential source of these messages, are threads and possibly call outs that are not properly disposed of when the the modules receive the stop signal.
What you need to do to avoid these errors, is to make sure your own worker thread stops before you return from stop() in your module, otherwise you have races that can break all sorts of things.
Now we get to the nitpicking part. Just out of curiousity:
Is this something you might expect when using threads (global vars gone but function still active) or is this a (Pike) problem?
On 3/29/2010 14:50, Marcus Agehall (nu med K-märkt fastighet) @ Pike (-) developers forum wrote:
What you need to do to avoid these errors, is to make sure your own worker thread stops before you return from stop() in your module, otherwise you have races that can break all sorts of things.
Not sure I understand your question.
It's a plain race condition between your worker thread and the rest of the system.
What happens is that your queue will continue to process data (even if it's just your quit-command) while the rest of the system assumes your module is all done and can be destructed. If module manages to destroy some variables in your module before your worker thread is done, chances are you'll see the error.
If you get this type of error, it means that something was still running while it shouldn't be. You should never expect to see this type of error if your threading is done correctly.
Hi Marcus,
I get the errors when a function is called from within my queue, which operates its own (single) thread. The function goes awry when it tries to call the instances' private (or public, or superclass) member variables. In my case it tries to up a counter (_cnt++). There is no external interferance with these variables.
What is strange in my opinion, but I could be totally wrong of course :), is that the function is correctly executed, as long as I do not call instance/class variables. It seems as if an class instance can be broken down while a function of that class is still active or that an instance is garbage collected in parts. What's the difference between a class function and class member when it comes down to garbage collecting?
In another case, I solved the problem by first copying, upon function entry, the required class member into a local member and all is fine.
Again, if I'm just asking something stoopid; just tell me :)
On 3/29/2010 15:25, Marcus Agehall (nu med K-märkt fastighet) @ Pike (-) developers forum wrote:
Not sure I understand your question.
It's a plain race condition between your worker thread and the rest of the system.
What happens is that your queue will continue to process data (even if it's just your quit-command) while the rest of the system assumes your module is all done and can be destructed. If module manages to destroy some variables in your module before your worker thread is done, chances are you'll see the error.
If you get this type of error, it means that something was still running while it shouldn't be. You should never expect to see this type of error if your threading is done correctly.
I get the errors when a function is called from within my queue, which operates its own (single) thread. The function goes awry when it tries to call the instances' private (or public, or superclass) member variables. In my case it tries to up a counter (_cnt++). There is no external interferance with these variables.
Now you lost me. I thought you got the error messages when you were shutting down Roxen? Are you saying you get the errors while your server is up and running too? If so, the situation is a bit different.
I get the error message when Roxen is shutting down.
My queues are provided with an end-signal upon a module->stop() call. Queues do not end immediately, only after they're empty. I do not get the error that the queue is calling a NULL or destructed object or function, but from within the queued & called function itself.
Forgive me if I'm only confusing you more, I'm just trying to provide you with as much info I can think of. Just in case something that causes my problems comes around.
Coen
On 3/29/2010 15:45, Marcus Agehall (nu med K-märkt fastighet) @ Pike (-) developers forum wrote:
Now you lost me. I thought you got the error messages when you were shutting down Roxen? Are you saying you get the errors while your
Ok, so then the situation is like this:
1) You run your module and the worker queue for your thread has X items.
2) You decide to stop the server and the stop() method in your module is called.
3) Your stop() method enqueues a stop-command to your worker thread as command X+1 and returns.
4) Roxen starts destroying your module and it's variables.
5) Your worker thread processes commands and tries to access variables that have already been destroyed.
The ONLY way around this, is to ensure that your worker thread is stopped when you return from stop() in your module. 9 out of 10 times, other solutions are just plain wrong and should be avioded.
Ah, I now see where I made a mistake; I took the class instance that holds the queued function as something standing on its own. But the instance is indeed a class member of the stopping/stopped module.
I also took it for 'granted' that a class instance could not be destroyed/collected as so long as it was 'active'. But that would mean you could never ever close any program at all. :) Silly me.
I understand now. Thank you for your patience.
Coen
On 3/29/2010 16:00, Marcus Agehall (nu med K-märkt fastighet) @ Pike (-) developers forum wrote:
Ok, so then the situation is like this:
- You run your module and the worker queue for your thread has X
items.
- You decide to stop the server and the stop() method in your module
is called.
- Your stop() method enqueues a stop-command to your worker thread as
command X+1 and returns.
Roxen starts destroying your module and it's variables.
Your worker thread processes commands and tries to access variables
that have already been destroyed.
The ONLY way around this, is to ensure that your worker thread is stopped when you return from stop() in your module. 9 out of 10 times, other solutions are just plain wrong and should be avioded.
Coen Schalkwijk wrote:
Is this something you might expect when using threads (global vars gone but function still active) or is this a (Pike) problem?
This has something to do with the way the Roxen modules operate, not with Pike(classes) in general.
Hi Stephen,
Ok. Could you (or anyone else?) tell me more about this, so I can take this into account? Does this mean that Roxen can control the or has its own garbage collector?
Coen
On 3/29/2010 15:31, Stephen R. van den Berg wrote:
Coen Schalkwijk wrote:
Is this something you might expect when using threads (global vars gone but function still active) or is this a (Pike) problem?
This has something to do with the way the Roxen modules operate, not with Pike(classes) in general.
Coen Schalkwijk wrote:
Does this mean that Roxen can control the or has its own garbage collector?
No.
Ok. Could you (or anyone else?) tell me more about this, so I can take this into account?
Picture this:
class A { object B; class C { f() { references B; } }; }
What is perfectly possible is that class A still exists, class C still exists, but that object B is destroyed.
Then once you reference B from within A.C.f() you have a problem. This has little to do with the garbage collector, but more with when what variables are destroyed (explicitly) in stop() functions of e.g. class A.
Hi Stephen,
Thank you for your explanation. From what I understand is that you presume an _active_ destruction of object B (in my case, just a simple int) but that does not happen (at least not by me :)) in any 'stop' function.
I could picture Roxen destructing all public class variables for example, and I currently get the problem with a public int, but I've also had problems with private class variables (a mapping(string:string) in this case)
Coen
On 3/29/2010 15:45, Stephen R. van den Berg wrote:
Coen Schalkwijk wrote:
Does this mean that Roxen can control the or has its own garbage collector?
No.
Ok. Could you (or anyone else?) tell me more about this, so I can take this into account?
Picture this:
class A { object B; class C { f() { references B; } }; }
What is perfectly possible is that class A still exists, class C still exists, but that object B is destroyed.
Then once you reference B from within A.C.f() you have a problem. This has little to do with the garbage collector, but more with when what variables are destroyed (explicitly) in stop() functions of e.g. class A.
Coen Schalkwijk wrote:
Ok. Could you (or anyone else?) tell me more about this, so I can take this into account?
Picture this:
[...]
Not quite, the problem is more likely to be something like this, which Marcus Agehall also has mentioned:
roxen_module.pike:
object B; f() { references B; }
void start() { Thread.thread_create(f); }
When Roxen loads the module, it will in essence do something like:
modules += ({ (program)"roxen_module.pike"() }); modules->start();
When time comes to shutdown Roxen will do something like:
modules->stop(); map(modules, destruct); // Other stuff... exit(0);
The problem here is that the thread started in start() will continue to run although its object has been destructed.
The fix is to implement a stop() in the module that makes sure that all threads belonging to the module have terminated before returning.
From the backtraces that you get, you should be able to identify from
where the problematic thread originated.
This has little to do with the garbage collector, but more with when what variables are destroyed (explicitly) in stop() functions of e.g. class A.
True.
Stephen.
Thank you too! Will entirely stop queue's before returning from stop!
On 3/29/2010 16:15, Henrik Grubbström (Lysator) @ Pike (-) developers forum wrote:
Coen Schalkwijk wrote:
Ok. Could you (or anyone else?) tell me more about this, so I can take this into account?
Picture this:
[...]
Not quite, the problem is more likely to be something like this, which Marcus Agehall also has mentioned:
roxen_module.pike:
object B; f() { references B; }
void start() { Thread.thread_create(f); }
When Roxen loads the module, it will in essence do something like:
modules += ({ (program)"roxen_module.pike"() }); modules->start();
When time comes to shutdown Roxen will do something like:
modules->stop(); map(modules, destruct); // Other stuff... exit(0);
The problem here is that the thread started in start() will continue to run although its object has been destructed.
The fix is to implement a stop() in the module that makes sure that all threads belonging to the module have terminated before returning.
From the backtraces that you get, you should be able to identify from where the problematic thread originated.
This has little to do with the garbage collector, but more with when what variables are destroyed (explicitly) in stop() functions of e.g. class A.
True.
Stephen.
pike-devel@lists.lysator.liu.se
-
Coen Schalkwijk -
Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum -
Marcus Agehall (nu med K-m�rkt fastighet) ＠ Pike (-) developers forum
-
Mirar ＠ Pike developers forum
-
Stephen R. van den Berg