Pike-devel March 2010

pike-devel@lists.lysator.liu.se

6 participants
15 discussions

GC fatal
by Stephen R. van den Berg 22 Apr '10

22 Apr '10

The following error has been occurring in all 7.8 Pike versions for the past 9 months (at least). I have a production machine running Roxen which crashes on an error like this roughly once a day. I'm willing to make custom changes to the production environment in Roxen and/or Pike to track this one down. Things that slow down the production environment for more than a factor of three are not acceptable. Any suggestions on how to tackle this problem? /data/src/gpike/src/gc.c:2519: GC fatal: Marked a thing without refs. **Block: 0xab493890 Type: object Refs: 0 **Got gc marker at 0xbcacbb0: flags=0x00000 refs=0 weak=0 xrefs=0 saved=-1 frame=(nil) **Program id: 66369 **The object is not on the object link list. **The object is destructed but program found from id. ******************* **Looking for references to 0xab493890: **Done looking for references to 0xab493890, found 0 refs. Pike was in GC stage 200 when this fatal occured. Backtrace at time of fatal: -:1: Buffer(16130 /* 23168 */)->`+=("' OR\n pdlnode.p30!='") /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/PXml.pike:302: RXML.PEnt(RXML.t_text(text/plain, xml-entity),RXML.CompositeTagSet(_user_tag+rxml_tag_set))->drain_output() /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/utils.pmod:197: RXML.utils->p_xml_entity_cb(RXML.PEnt(RXML.t_text(text/plain, xml-entity),RXML.CompositeTagSet(_user_tag+rxml_tag_set)),"&form.nrpropp30:mysql;") /usr/local/pike/7.8.409/lib/modules/Parser.pmod/_parser.so:1: RXML.PEnt(RXML.t_text(text/plain, xml-entity),RXML.CompositeTagSet(_user_tag+rxml_tag_set))->finish("SELECT COUNT(DISTINCT(pdloffer.id_pdloffer)) as aantal\n FROM pdloffer\n JOIN namedata ON (namedata.id = pdloffer.owner)\n JOIN pdlnode ON (pdlnode.id_pdlnode = pdloffer.id_pdlnode)\n JOIN pdltree ON "+[27495]) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/PXml.pike:389: RXML.PEnt(RXML.t_text(text/plain, xml-entity),RXML.CompositeTagSet(_user_tag+rxml_tag_set))->finish("SELECT COUNT(DISTINCT(pdloffer.id_pdloffer)) as aantal\n FROM pdloffer\n JOIN namedata ON (namedata.id = pdloffer.owner)\n JOIN pdlnode ON (pdlnode.id_pdlnode = pdloffer.id_pdlnode)\n JOIN pdltree ON "+[27495]) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:5491: RXML.PEnt(RXML.t_text(text/plain, xml-entity),RXML.CompositeTagSet(_user_tag+rxml_tag_set))->write_end("SELECT COUNT(DISTINCT(pdloffer.id_pdloffer)) as aantal\n FROM pdloffer\n JOIN namedata ON (namedata.id = pdloffer.owner)\n JOIN pdlnode ON (pdlnode.id_pdlnode = pdloffer.id_pdlnode)\n JOIN pdltree ON "+[27495]) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:6224: RXML.t_text(text/plain, xml-entity)->eval("SELECT COUNT(DISTINCT(pdloffer.id_pdloffer)) as aantal\n FROM pdloffer\n JOIN namedata ON (namedata.id = pdloffer.owner)\n JOIN "+[27566],Context(),RXML.CompositeTagSet(_user_tag+rxml_tag_set),UNDEFINED,UNDEFINED) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:350: TagSqlplugin(emit#sql)->eval_args(mapping[2],0,0,({"scope","filter","do-once","rowinfo","remainderinfo","skiprows","filter-exclude","sort","maxrows","source"})) modules/tags/rxmltags.pike:5985: Frame(emit)->do_enter(RequestID(/info/zoekresultaatmap.rxml?mode=rentpdriver&_id_pdlnode=449&geolat=52.4685437&geolng=4.6699539&id=810468&plaats=Beverwijk)) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:4520: Frame(emit)->_eval((a)0=Context(),RXML.PXml(RXML.t_any(any, xml),RXML.CompositeTagSet(_user_tag+rxml_tag_set)),RXML.t_any(any, xml)) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:459: TagEmit(emit)->_p_xml_handle_tag((a)1=RXML.PXml(RXML.t_any(any, xml),RXML.CompositeTagSet(_user_tag+rxml_tag_set)),mapping[2],"\n\n\n \n <set variable=\"var.numberresults\" value=\"&_.aantal;\" />\n \n") /usr/local/pike/7.8.409/lib/modules/Parser.pmod/_parser.so:1: RXML.PXml(RXML.t_any(any, xml),RXML.CompositeTagSet(_user_tag+rxml_tag_set))->finish("\n<emit query=\"SELECT COUNT(DISTINCT(pdloffer.id_pdloffer)) as aantal\n FROM pdloffer\n JOIN namedata ON (namedata.id = pdloffer.owner)\n JOIN pdlnode ON (pdlnode.id_pdlnode = pdloffer.id_pdlnode)\n JO"+[27598]) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/PXml.pike:389: RXML.PXml(RXML.t_any(any, xml),RXML.CompositeTagSet(_user_tag+rxml_tag_set))->finish("\n<emit query=\"SELECT COUNT(DISTINCT(pdloffer.id_pdloffer)) as aantal\n FROM pdloffer\n JOIN namedata ON (namedata.id = pdloffer.owner)\n JOIN pdlnode ON (pdlnode.id_pdlnode = pdloffer.id_pdlnode)\n JO"+[27598]) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:3819: Frame(eval)->_exec_array((a)0,RXML.RenewablePCode(RXML.t_any(any, xml),RXML.CompositeTagSet(_user_tag+emit+rxml_tag_set)),({"\n<emit query=\"SELECT COUNT(DISTINCT(pdloffer.id_pdloffer)) as aantal\n FROM pdloffer\n JOIN "+[27704]}),"do_return") /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:4825: Frame(eval)->_eval(@0,@2=RXML.RenewablePCode(RXML.t_any(any, xml),RXML.CompositeTagSet(_user_tag+emit+rxml_tag_set)),(a)3=RXML.t_any(any, xml)) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:8835: RXML.RenewablePCode(RXML.t_any(any, xml),RXML.CompositeTagSet(_user_tag+emit+rxml_tag_set))->_eval(@0,0) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:9206: RXML.RenewablePCode(RXML.t_any(any, xml),RXML.CompositeTagSet(_user_tag+emit+rxml_tag_set))->_eval(@0,0) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:3865: UserTag.Frame(gettotalfornodemode)->_exec_array((a)0,RXML.PCode(RXML.t_any(any, xml),RXMLTagSet(entreeding.com,rxml_tag_set)),({@2}),"do_return") /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:4825: UserTag.Frame(gettotalfornodemode)->_eval(@0,@4=RXML.PCode(RXML.t_any(any, xml),RXMLTagSet(entreeding.com,rxml_tag_set)),@3) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:8835: RXML.PCode(RXML.t_any(any, xml),RXMLTagSet(entreeding.com,rxml_tag_set))->_eval(@0,0) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:4775: Frame(set)->_eval((a)0,RXML.PCode(RXML.t_html(text/html, xml),RXMLTagSet(entreeding.com,rxml_tag_set)),RXML.t_html(text/html, xml)) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:8835: RXML.PCode(RXML.t_html(text/html, xml),RXMLTagSet(entreeding.com,rxml_tag_set))->_eval(@0,0) /usr/roxen/server-4.5.0/etc/modules/RXML.pmod/module.pmod:8243: RXML.PCode(RXML.t_html(text/html, xml),RXMLTagSet(entreeding.com,rxml_tag_set))->eval(@0,UNDEFINED) modules/tags/rxmlparse.pike:171: RoxenModule(entreeding.com/rxmlparse#0)->handle_file_extension(Stdio.File("/home/entreeding/info/zoekresultaatmap.rxml", "r", 666 /* fd=39 */),"rxml",,,0) base_server/configuration.pike:2357: Configuration(entreeding.com)->low_get_file(@5,UNDEFINED) base_server/configuration.pike:2491: Configuration(entreeding.com)->get_file(@5,UNDEFINED,UNDEFINED) base_server/configuration.pike:2466: Configuration(entreeding.com)->handle_request(@5,UNDEFINED) protocols/http.pike:2823: RequestID(/info/zoekresultaatmap.rxml?mode=rentpdriver&_id_pdlnode=449&geolat=52.4685437&geolng=4.6699539&id=810468&plaats=Beverwijk)->handle_request() base_server/roxen.pike:849: roxen->handler_thread(23) -- Sincerely, Stephen R. van den Berg. I'm never wrong. I once thought I was wrong, turns out, I was mistaken.

4 39

Problem with "Cannot access global variables in destructed object"
by Coen Schalkwijk 29 Mar '10

29 Mar '10

Hello Everybody, Did I ask a stupid question when I said I have problems with accessing global class variables in a function during program shutdown? If I did, I'd also like to know, no offense taken :) -- Met vriendelijke groet / With kind regards / mit besten Grüßen, Coen Schalkwijk Software Engineer coen.schalkwijk(a)rtl.nl <mailto:coen.schalkwijk@rtl.nl> coen(a)rtlinteractief.nl <mailto:coen@rtlinteractief.nl> +31 (0)35 671 8915

5 21

CVS repository conversion
by Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) ＠ Pike (-) developers forum 27 Mar '10

27 Mar '10

I'm currently in the progress of comparing grubba's git conversion of the Pike repository to my svn conversion, which should improve the quality of both. I've encountered a systematic difference where some kind of policy decision on what is "right" is needed though. The issue is with tags. In CVS, each file is tagged separately. If a file does not have a particular tag, you will not get that file when you check out the tag. There are in fact quite a few files in Pike which do not have certain tags even though they were current on trunk when the tag was made. This is typically the case with modules which were initially developed outside the main tree (and imported using the CVS "module" concept, which is not understood by cvs rtag). Examples include: lib/modules/SSL.pmod/ src/post_modules/COM/ src/post_modules/SDL/ src/post_modules/Shuffler/ src/post_modules/_Image_SVG/ lib/modules/Audio.pmod lib/modules/Tools.pmod/Legal.pmod Then, there is also a few tags (so far I have encountered v7_1_7, v7_1_20, and v7_7_28) where large parts of the tree (including src/ in its entirery) do not have the tag, suggesting that the tagging operation was interrupted, and the resulting broken tag was just left there instead of being (re)moved. So, there are two principle approaches here: Should the tags in the converted repository accurately reflect the tags in the CVS repository, so that you only get those files you would have gotten out of CVS, or do we take the revisionist approach of including also such files which do not have the tag but which were present on head when the tag was created, and therefore "ought" to have been tagged?

6 12

[Pike/Roxen] Problems with destructing objects during Roxen shutdown...
by Coen Schalkwijk 15 Mar '10

15 Mar '10

Hi All, [Something went wrong in sending this msg, might be you get this msg twice, sorry for that] I get a lot of 'Cannot access global variables in destructed object' errors when rebooting Roxen. It looks as though a function call is completed even after the object is destructed. I only get the error when setting (private) class members of an object. The rest of the functions seems to execute without problem. Is there any way to prevent this? Like either not execute the function if the object is (going to be) destructed or not destruct an instance until the function has completed? Thanks in advance! Coen RTL Nederland

1 0

Re: chunked-encoding
by Mirar ＠ Pike developers forum 15 Mar '10

15 Mar '10

Aha, I see.

1 0

Re: chunked-encoding
by Arne Gödeke 14 Mar '10

14 Mar '10

On 03/14/2010 10:45 PM, Mirar @ Pike developers forum wrote: > If Server.Request doesn't support chunked-encoding I'm sure it > wouldn't be bad if it was added. :) > > (Although I personally never really saw the need for it since TCP > allows you to send any chunks you want already.) Yes thats true. The whole thing is kindof pointless in the big picture. However, in browser applications we are pretty much stuck with javascript which cannot live without it. And using chunked-encoding one actually gets something like a TCP connection (at least one way from server to client).

1 0

chunked-encoding
by Arne Gödeke 14 Mar '10

14 Mar '10

In the yakitychat code we are currently abusing the Protocols.HTTP.Server.Request objects to do comet-like streaming using chunked-encoding. Right now we are just taking over the Stdio.File object but it would be nicer if the Request api had something like a send() function to send chunked encoded data. I was thinking about merging those portions of the into the Protocols.HTTP.Server.Request code. Is there anyone who already did this or has there been a discussion about the api? arne

2 1

matrix blerg
by Mirar ＠ Pike developers forum 14 Mar '10

14 Mar '10

I got tired of A*B == A*B*X for Math.Matrix objects. Could someone apply this (slightly kludgy, but working) patch? Index: math_matrix.c =================================================================== RCS file: /pike/data/cvsroot/Pike/7.8/src/modules/Math/math_matrix.c,v retrieving revision 1.40 diff -u -r1.40 math_matrix.c --- math_matrix.c 18 Dec 2007 23:24:49 -0000 1.40 +++ math_matrix.c 14 Mar 2010 12:51:10 -0000 @@ -16,6 +16,7 @@ #include "operators.h" #include "builtin_functions.h" #include "module_support.h" +#include "operators.h" #include "math_module.h" Index: matrix_code.h =================================================================== RCS file: /pike/data/cvsroot/Pike/7.8/src/modules/Math/matrix_code.h,v retrieving revision 1.21 diff -u -r1.21 matrix_code.h --- matrix_code.h 17 Sep 2008 12:05:13 -0000 1.21 +++ matrix_code.h 14 Mar 2010 12:51:10 -0000 @@ -493,12 +493,24 @@ { struct matrixX(_storage) *mx=NULL; struct matrixX(_storage) *dmx; - int n; + int n,i; FTYPE *s1,*s2,*d; if (args<1) SIMPLE_TOO_FEW_ARGS_ERROR("`+",1); + if (args>1) /* one add per argument */ + { + ref_push_object(THISOBJ); + for (i=0; i<args; i++) + { + push_svalue(Pike_sp-args+i); + f_add(2); + } + stack_pop_n_elems_keep_top(args); + return; + } + if (Pike_sp[-1].type!=T_OBJECT || !((mx=(struct matrixX(_storage)*) get_storage(Pike_sp[-1].u.object,XmatrixY(math_,_program))))) @@ -508,8 +520,6 @@ math_error("`+",Pike_sp-args,args,0, "Cannot add matrices of different size.\n"); - pop_n_elems(args-1); /* shouldn't be needed */ - dmx=matrixX(_push_new_)(mx->xsize,mx->ysize); s1=THIS->m; @@ -528,11 +538,23 @@ { struct matrixX(_storage) *mx=NULL; struct matrixX(_storage) *dmx; - int n; + int n,i; FTYPE *s1,*s2=NULL,*d; if (args) { + if (args>1) /* one subtract per argument */ + { + ref_push_object(THISOBJ); + for (i=0; i<args; i++) + { + push_svalue(Pike_sp-args+i); + f_minus(2); + } + stack_pop_n_elems_keep_top(args); + return; + } + if (Pike_sp[-1].type!=T_OBJECT || !((mx=(struct matrixX(_storage)*) get_storage(Pike_sp[-1].u.object,XmatrixY(math_,_program))))) @@ -543,8 +565,6 @@ math_error("`-",Pike_sp-args,args,0, "Cannot add matrices of different size.\n"); - pop_n_elems(args-1); /* shouldn't be needed */ - s2=mx->m; } @@ -634,7 +654,17 @@ if (args<1) SIMPLE_TOO_FEW_ARGS_ERROR("`*",1); - pop_n_elems(args-1); /* shouldn't be needed */ + if (args>1) /* one multiply per argument */ + { + ref_push_object(THISOBJ); + for (i=0; i<args; i++) + { + push_svalue(Pike_sp-args+i); + f_multiply(2); + } + stack_pop_n_elems_keep_top(args); + return; + } if (Pike_sp[-1].type==T_INT) {

1 1

pcvs2git.pike
by Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum 14 Mar '10

14 Mar '10

I just tried converting Roxen 2.4: Stage Memory use Commits --------------------------------------- Import 339 MB ~17000 Raking 359 MB ~17000 Verify 361 MB ~17000 Merging 662 MB 11886 Graphing 803 MB 10897 Generate 803 MB 10897 I believe that the memory use can be reduced by using more custom datatypes (currently there are a lot of mappings generated in the merging and graphing stages). Another way to reduce the memory use is to partition the graphs in the time axis.

5 58

MIME.StringRange vs. String.Range
by Stephen R. van den Berg 11 Mar '10

11 Mar '10

The MIME.StringRange object seems to be useful in more cases than just in the MIME module. Any objections if we move it into the String module and then name it Range (i.e. String.Range) instead? -- Stephen. Time is nature's way of making sure everything doesn't happen at once.

5 7

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel March 2010