I've developed the gc strategy a bit so that it also takes the time spent in the gc into account. There is now a low and a high garbage level. The gc tries to keep the low one as long as the total cpu time spent in the gc doesn't get too large, then it resorts to the higher one. I thought that could be useful since the gc otherwise can take a lot of time in a process that both has many nongarbage objects and generates a lot of garbage.
I've also added a function Pike.gc_parameters to make it possible to tune these parameters. The current defaults are:
Low garbage ratio: 20% of all allocated things is garbage. GC time ratio: 5% cpu time spent in the gc. High garbage ratio: 50% of all allocated things is garbage.
These are set on feeling only and hardly tested at all. It's therefore very appreciated if someone could do some tuning with a big and busy pike process. (The function Debug.gc_status is useful to get info about the gc performance.)
Speaking of Debug.gc_status, I'd like to move it to Pike.gc_status to bring it closer to the new Pike.gc_parameters. Thoughts about that?
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-12 17:20: Subject: GC improvement
I've developed the gc strategy a bit so that it also takes the time spent in the gc into account. There is now a low and a high garbage level. The gc tries to keep the low one as long as the total cpu time spent in the gc doesn't get too large, then it resorts to the higher one. I thought that could be useful since the gc otherwise can take a lot of time in a process that both has many nongarbage objects and generates a lot of garbage.
I've also added a function Pike.gc_parameters to make it possible to tune these parameters. The current defaults are:
Low garbage ratio: 20% of all allocated things is garbage. GC time ratio: 5% cpu time spent in the gc. High garbage ratio: 50% of all allocated things is garbage.
These are set on feeling only and hardly tested at all. It's therefore very appreciated if someone could do some tuning with a big and busy pike process. (The function Debug.gc_status is useful to get info about the gc performance.)
/ Martin Stjernholm, Roxen IS
Do you ever need gc_status in normal programs?
/ Martin Nilsson (Åskblod)
Previous text:
2003-01-12 17:22: Subject: GC improvement
Speaking of Debug.gc_status, I'd like to move it to Pike.gc_status to bring it closer to the new Pike.gc_parameters. Thoughts about that?
/ Martin Stjernholm, Roxen IS
I think so. You need it to tune the gc (now when it's tunable), and that's something one might want to do when profiling a normal program.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-12 17:24: Subject: GC improvement
Do you ever need gc_status in normal programs?
/ Martin Nilsson (Åskblod)
But then you are using it to debug/tune a normal program and not really as part of the program itself. Unless you let the program automatically alter gc_parameters with gc_status as input.
/ Martin Nilsson (Åskblod)
Previous text:
2003-01-12 17:29: Subject: GC improvement
I think so. You need it to tune the gc (now when it's tunable), and that's something one might want to do when profiling a normal program.
/ Martin Stjernholm, Roxen IS
Firstly I don't think the connection between debugging and tuning is a very strong one. Secondly I don't think it's far-fetched to consider a program that routinely at runtime monitors and tunes its own gc, either automatically or more or less manually. Do you suggest that Pike.gc_parameters is moved to Debug instead?
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-12 17:40: Subject: GC improvement
But then you are using it to debug/tune a normal program and not really as part of the program itself. Unless you let the program automatically alter gc_parameters with gc_status as input.
/ Martin Nilsson (Åskblod)
No, I don't really suggest anything. Just pointing out possible reasons for not moving it. Then, should the memory status functions be moved from Debug to Pike as well?
/ Martin Nilsson (Åskblod)
Previous text:
2003-01-12 17:44: Subject: GC improvement
Firstly I don't think the connection between debugging and tuning is a very strong one. Secondly I don't think it's far-fetched to consider a program that routinely at runtime monitors and tunes its own gc, either automatically or more or less manually. Do you suggest that Pike.gc_parameters is moved to Debug instead?
/ Martin Stjernholm, Roxen IS
I think so, since they are showing information about pike, not debug information per se.
/ Per Hedbor ()
Previous text:
2003-01-12 18:18: Subject: GC improvement
No, I don't really suggest anything. Just pointing out possible reasons for not moving it. Then, should the memory status functions be moved from Debug to Pike as well?
/ Martin Nilsson (Åskblod)
Yes, that'd be more consistent.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-12 18:18: Subject: GC improvement
No, I don't really suggest anything. Just pointing out possible reasons for not moving it. Then, should the memory status functions be moved from Debug to Pike as well?
/ Martin Nilsson (Åskblod)
I've also fixed some bugs in the old gc strategy:
o The decaying average calculation had a bug that caused the last reading (i.e. the last gc run) to be weighted very heavily. Thus a single "abnormal" gc run could cause the gc to schedule a fairly short or long gc interval the next time which might make it erratic.
o The aim for 20% garbage between gc runs was calculated on the wrong total amount of objects, so in effect the gc aimed for 16.7% instead. No big deal.
These bugs have been around at least since 0.6.
David, you've reported bad gc behavior. Could this explain your problems?
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-12 17:20: Subject: GC improvement
I've developed the gc strategy a bit so that it also takes the time spent in the gc into account. There is now a low and a high garbage level. The gc tries to keep the low one as long as the total cpu time spent in the gc doesn't get too large, then it resorts to the higher one. I thought that could be useful since the gc otherwise can take a lot of time in a process that both has many nongarbage objects and generates a lot of garbage.
I've also added a function Pike.gc_parameters to make it possible to tune these parameters. The current defaults are:
Low garbage ratio: 20% of all allocated things is garbage. GC time ratio: 5% cpu time spent in the gc. High garbage ratio: 50% of all allocated things is garbage.
These are set on feeling only and hardly tested at all. It's therefore very appreciated if someone could do some tuning with a big and busy pike process. (The function Debug.gc_status is useful to get info about the gc performance.)
/ Martin Stjernholm, Roxen IS
Well, symptom was that it was called very often. like within minutes of the previous run (this was in >1GB processes).
/ David Hedbor
Previous text:
2003-01-12 18:25: Subject: GC improvement
I've also fixed some bugs in the old gc strategy:
o The decaying average calculation had a bug that caused the last reading (i.e. the last gc run) to be weighted very heavily. Thus a single "abnormal" gc run could cause the gc to schedule a fairly short or long gc interval the next time which might make it erratic.
o The aim for 20% garbage between gc runs was calculated on the wrong total amount of objects, so in effect the gc aimed for 16.7% instead. No big deal.
These bugs have been around at least since 0.6.
David, you've reported bad gc behavior. Could this explain your problems?
/ Martin Stjernholm, Roxen IS
Then I don't think it has anything to do with this. After investigating the algorithm a bit, I don't see any other way so frequent runs can occur than through a large generation of garbage.
/ Martin Stjernholm, Roxen IS
Previous text:
2003-01-13 10:31: Subject: GC improvement
Well, symptom was that it was called very often. like within minutes of the previous run (this was in >1GB processes).
/ David Hedbor
pike-devel@lists.lysator.liu.se
-
David Hedbor ＠ Pike developers forum -
Martin Nilsson (�skblod) ＠ Pike (-) developers forum
-
Martin Stjernholm, Roxen IS ＠ Pike developers forum
-
Per Hedbor () ＠ Pike (-) developers forum
-
Peter Bortas ＠ Pike developers forum