Pike-devel

pike-devel@lists.lysator.liu.se

2 participants
3046 discussions

AES pike 7.4 example
by cschalkwijk 10 Feb '09

10 Feb '09

Hi, I need an example for using the AES encryption in pike 7.4. I can't get it to work. Thanks a bunch! -- Met vriendelijke groet / With kind regards / mit besten Grüßen, Coen Schalkwijk Software Engineer coen.schalkwijk(a)rtl.nl <mailto:coen.schalkwijk@rtl.nl> coen(a)rtlinteractief.nl <mailto:coen@rtlinteractief.nl> +31 (0)35 671 8915 __________________________________________________________ Deze e-mail en de inhoud is vertrouwelijk en uitsluitend bestemd voor de geadresseerde(n). Indien u niet de geadresseerde bent van deze e-mail verzoeken wij u dit direct door te geven aan de verzender door middel van een reply e-mail en de ontvangen e-mail uit uw systemen te verwijderen. Als u geen geadresseerde bent, is het niet toegestaan om kennis te nemen van de inhoud, deze te kopieren, te verspreiden, bekend te maken aan derden noch anderszins te gebruiken. The information contained in this e-mail is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Please notify us immediately if you have received it in error by reply e-mail and then delete this message from your system. __________________________________________________________

2 5

Re: Stdio.exist
by Mirar ＠ Pike developers forum 02 Feb '09

02 Feb '09

(if file_stat finds symlinks, that is.)

7 12

Confusing error message
by Stephen R. van den Berg 02 Feb '09

02 Feb '09

This looks fine: > string a=({"12","34","56"}); Compiler Error: 1: Bad type in assignment. Compiler Error: 1: Expected: string. Compiler Error: 1: Got : array(string(0..255)). This is confusing: > (string)({"12","34","56"}); cast: Item 2 is not an integer: "56" HilfeInput:1: HilfeInput()->___HilfeWrapper() Any ideas on how this last one can be improved? -- Sincerely, Stephen R. van den Berg. E-mails should be like a lady's skirt: Long enough to cover the subject, and short enough to be interesting.

6 6

Stdio.exist
by Martin Stjernholm, Roxen IS ＠ Pike developers forum 01 Feb '09

01 Feb '09

I just noticed that Stdio.exist returns 0 for dangling symlinks. I think that's unfortunate. What do you say, consider it a bug and fix it (with compat goo for #pike 7.6), or document it as intended behavior?

5 17

RFC: New Parser class for parsing structured datafiles (non-XML)
by Stephen R. van den Berg 28 Jan '09

28 Jan '09

I've revamped some datafileparsing routines I've been using for years, and made them more robust/generic in the process. I'd like to include this class into mainstream Pike (7.8 or 7.9) eventually, as Parser.Structure (for lack of a better name) perhaps. The sample records here are: a. Some bank statements in CSV format. b. Some bank statements in MT940 format. Any comments with regard to the provided interface, the datastructure description or the usefulness, or perhaps the superfluousness because this can already be done by Parser.LR and I'm just being silly for not reusing that? With regard to documentation, that will be improved, I know it's a bit scarce at the moment ;-). Sample input: ---------------------cut here------------------------- 1234563,"20081128","IC",00000,8689839,"KN: ARNL-1034758710-","",19.90,"A","M","BETREFT:25399280 ACCESS INTERNET BV","EUR" 2348165,"20081128","DV",00000,0,"DEBETRENTE GIROKWARTAALKREDIET","",10.04,"A","M","PER. 21/10-20/11 (1,15% PER MND)","EUR" 2341864,"20081128","DV",00000,0,"DEBETRENTE 1,39% PER MAAND,","",0.35,"A","M","PERIODE 21/10 - 20/11","EUR" 0000 01PSTBNL21XXXX00001 0000 01PSTBNL21XXXX00001 940 00 :20:MPBZ :25:0003890413 :28C:000 :60F:C071205EUR2085,93 :61:071206C11,90NGT NONREF :86:0001239968 Hr X Colijn LANDGRAAF Hr X Colijn LANDGRAAF 1700.163221231474 TRANSACTIEDATUM: 06-12-2007 :61:071206D109,10NIC 004952732340412 :86:0002243510 KN: 004952739440412 049-8223343 BETREFT FACTUUR D.D. 28-11-2007 INCL. 17,42 BTW KPN Telefonie :61:071206D34,95NDV 4700134832333684 :86:0998234454 KN: 4702162821344484 4700124821134444 R.2003-13224 K.122368 PEDESCO/KLONSSEN.ORG :62F:C081119EUR289,33 :86:D000004C000009D9961,90C60315,92 -XXX ---------------------cut here------------------------- Sample output (corresponds with the above input): ---------------------cut here------------------------- Found record: ([ /* 1 element */ "csv": ([ /* 1 element */ "gtz": ({ /* 3 elements */ ([ /* 11 elements */ "afbij": "A", "amount": "19.90", "bankno": "8689839", "mutatie": "M", "mutatiesoort": "IC", "mybankno": "1234563", "name": "KN: ARNL-1034758710-", "reference": "BETREFT:25399280 ACCESS INTERNET BV", "transferdate": "20081128", "valutacode": "EUR", "volgnummer": "00000" ]), ([ /* 11 elements */ "afbij": "A", "amount": "10.04", "bankno": "0", "mutatie": "M", "mutatiesoort": "DV", "mybankno": "2348165", "name": "DEBETRENTE GIROKWARTAALKREDIET", "reference": "PER. 21/10-20/11 (1,15% PER MND)", "transferdate": "20081128", "valutacode": "EUR", "volgnummer": "00000" ]), ([ /* 11 elements */ "afbij": "A", "amount": "0.35", "bankno": "0", "mutatie": "M", "mutatiesoort": "DV", "mybankno": "2341864", "name": "DEBETRENTE 1,39% PER MAAND,", "reference": "PERIODE 21/10 - 20/11", "transferdate": "20081128", "valutacode": "EUR", "volgnummer": "00000" ]) }) ]) ]) Found record: ([ /* 1 element */ "mt940": ([ /* 11 elements */ "accountno": "0003890413", "closingbalance": ([ /* 4 elements */ "amount": "289,33", "creditdebit": "C", "currency": "EUR", "date": "081119" ]), "creditamount": "60315,92", "creditentries": "000009", "debitamount": "9961,90", "debitentries": "000004", "messageheader1": ({ /* 2 elements */ ([ /* 2 elements */ "exportaddress": "PSTBNL21XXXX", "exportnumber": "00001" ]), ([ /* 2 elements */ "exportaddress": "PSTBNL21XXXX", "exportnumber": "00001" ]) }), "messagetrailer": ([ /* 2 elements */ "end": "XXX", "start": "-" ]), "openingbalance": ([ /* 4 elements */ "amount": "2085,93", "creditdebit": "C", "currency": "EUR", "date": "071205" ]), "reference": "MPBZ", "statements": ({ /* 3 elements */ ([ /* 6 elements */ "amount": "11,90", "creditdebit": "C", "description": "0001239968 Hr X Colijn LANDGRAAF\n" "Hr X Colijn LANDGRAAF 1700.163221231474\n" "TRANSACTIEDATUM: 06-12-2007", "paymentreference": "NONREF", "transactiontype": "GT ", "valuedate": "071206" ]), ([ /* 6 elements */ "amount": "109,10", "creditdebit": "D", "description": "0002243510 KN: 004952739440412\n" "049-8223343 BETREFT FACTUUR D.D. 28-11-2007\n" "INCL. 17,42 BTW KPN Telefonie", "paymentreference": "004952732340412", "transactiontype": "IC ", "valuedate": "071206" ]), ([ /* 6 elements */ "amount": "34,95", "creditdebit": "D", "description": "0998234454 KN: 4702162821344484\n" "4700124821134444 R.2003-13224 K.122368\n" "PEDESCO/KLONSSEN.ORG", "paymentreference": "4700134832333684", "transactiontype": "DV ", "valuedate": "071206" ]) }) ]) ]) ---------------------cut here------------------------- Sample code to actually produce the output: object rf=parsestruct(allformats,Stdio.stdin,1); { mapping m; while(m=rf->fetch()) write("Found record: %O\n",m); } Sample format description needed for parsing the above input: array|mapping allformats= ([ "csv": ({ ({"gtz", ({"mybankno",(<",">)}), ({"transferdate",(<",">)}), ({"mutatiesoort",(<",">)}), ({"volgnummer",(<",">)}), ({"bankno",(<",">)}), ({"name",(<",">)}), ({"kostenplaats",(<",">),(["drop":1])}), ({"amount",(<",">)}), ({"afbij",(<",">)}), ({"mutatie",(<",">)}), ({"reference",(<",">)}), ({"valutacode",(<",">)}), }), }), "mt940": ({ ({"messageheader1", ({"exporttime","0000",(["drop":1])}), ({"CS1"," ",(["drop":1])}), ({"exportday","01",(["drop":1])}), ({"exportaddress",12}), ({"exportnumber",5,(["regex":"[0-9]+"])}), }), ({"messageheader3",(["fold":1]), ({"messagetype","940",(["drop":1])}), ({"CS1"," ",(["drop":1])}), ({"messagepriority","00",(["drop":1])}), }), ({"TRN",(["fold":1]), ({"tag",":20:",(["drop":1])}), ({"reference","GTZPB|MPBZ|INGEB"}), }), ({"accountid",(["fold":1]), ({"tag",":25:",(["drop":1])}), ({"accountno",10}), }), ({"statementno",(["fold":1]), ({"tag",":28C:",(["drop":1])}), ({"settlementno",0,(["drop":1])}), }), ({"openingbalance", ({"tag",":60F:",(["drop":1])}), ({"creditdebit",1}), ({"date",6}), ({"currency","EUR"}), ({"amount",0,(["regex":"[0-9]+,[0-9][0-9]"])}), }), (["statements": ({ ({"statementline",(["fold":1]), ({"tag",":61:",(["drop":1])}), ({"valuedate",6}), ({"creditdebit",1}), ({"amount","[0-9]+,[0-9][0-9]"}), ({"CS1","N",(["drop":1])}), ({"transactiontype",3}), // 3 for Postbank, 4 for ING ({"paymentreference",0}), }), ({"informationtoaccountowner",(["fold":1]), ({"tag",":86:",(["drop":1])}), ({"description","[^\n]+(\n[^:][^\n]*)*"}), }), }), ]), ({"closingbalance",(["mandatory":1]), ({"tag",":62[FM]:",(["drop":1])}), ({"creditdebit",1}), ({"date",6}), ({"currency","EUR"}), ({"amount",0,(["regex":"[0-9]+,[0-9][0-9]"])}), }), ({"informationtoaccountowner",(["fold":1]), ({"tag",":86:",(["drop":1])}), ({"debit","D",(["drop":1])}), ({"debitentries",6}), ({"credit","C",(["drop":1])}), ({"creditentries",6}), ({"debit","D",(["drop":1])}), ({"debitamount","[0-9]+,[0-9][0-9]"}), ({"credit","C",(["drop":1])}), ({"creditamount","[0-9]+,[0-9][0-9]"}), ({"accountname","(\n[^-:][^\n]*)*",(["drop":1])}), }), ({"messagetrailer",(["mandatory":1]), ({"start","-"}), ({"end","XXX"}), }), }), ]); Implementation of the parsestruct class which actually does the parsing according to the specs above: class parsestruct { protected Stdio.FILE in; protected mapping|array fms; protected array(string) alread=({}); protected int eol; protected int maxregexmatch=1024; protected Regexp simple=Regexp("^[^[\\](){}<>^$|+*?\\\\]+$"); protected Regexp emptyline=Regexp("^[ \t\v\r\x1a]*$"); protected mixed severity=1; protected int verb=0; protected int recordcount=0; // verbose is the number of characters to display per progressline protected void create(void|array|mapping formats,void|string|Stdio.File|Stdio.FILE input, void|int verbose) { fms=formats; verb=verbose==1?70:verbose; if(!input) input=""; if(stringp(input)) input=Stdio.FakeFile(input); if(!in->unread) (in=Stdio.FILE())->assign(input); } private string read(int n) { string s; s=in->read(n); alread+=({s}); if(sizeof(s)!=n) throw(severity); return s; } private string gets(int n) { string s; if(n) { s=read(n); if(!s||sizeof(s)!=n||has_value(s,"\n")) throw(severity); } else { s=in->gets(); if(!s) throw(severity); alread+=({s,"\n"}); if(has_suffix(s,"\r")) s=s[..<1]; eol=1; } return s; } protected string getcsvfield(multiset delims) { int leadspace=1,inquotes=0; string c; array word=({}); for(c=read(1);c;) { if(delims[c]) { if(!inquotes) return word*""; word+=({c}); } else switch(c) { case "\"":leadspace=0; if(!inquotes) inquotes=1; else if((c=read(1))=="\"") word+=({c}); else { inquotes=0; continue; } break; default:leadspace=0; case " ":case "\t": if(!leadspace) word+=({c}); break; case "\n": if(!inquotes) { eol=1; return word*""; } word+=({c}); case "\r":case "\x1a":; } c=read(1); } throw(severity); } // "fname", string|regex = regexp match // int|width = field width - 0: till end of line // multiset|delim = delimiters // mapping extra params // array = nested definition/subfields // mandatory = 1 // fold = 1 : makes the record fold its fields into the parent protected mapping getrecord(array fmt,int found) { mapping ret=([]),options; string recname=fmt[0]; if(mappingp(fmt[1])) { options=fmt[1];fmt=fmt[2..]; #if 0 // srb FIXME if(options->mandatory) severity=sprintf("Missing mandatory record %s",recname); #endif } else options=([]),fmt=fmt[1..]; eol=0; foreach(fmt;;array field) { string fname=field[0]; mixed m=field[1]; string|array|mapping value; if(!mappingp(m)) { if(arrayp(m)) { value=getrecord(m,found); m=([]); } else m=([(intp(m)?"width":(stringp(m)?"regex":"delim")):m]); if(sizeof(field)>2) m+=field[2]; } #if 0 // srb FIXME if(!found && m->mandatory) severity=sprintf("Missing mandatory field %s",recname); #endif if(eol) throw(severity); if(!zero_type(m->width)) value=gets(m->width); if(m->delim) { multiset delim=m->delim; if(sizeof(delim-(<",",";","\t"," ">))) { for(value=({});;) { string c; if(!(c=read(1))) throw(severity); if(delim[c]) break; switch(c) { case "\n": throw(severity); default: value+=({c}); case "\r":case "\x1a":; } } value*=""; } else value=getcsvfield(delim); } if(m->regex) { Regexp rgx; if(stringp(m->regex)) { if(!value && simple->match(m->regex)) { m->width=sizeof(m->regex); value=gets(m->width); } m->regex=Regexp("^("+m->regex+")"+(value?"$":"")); } rgx=m->regex; if(value) { if(!rgx->match(value)) throw(severity); } else { string buf=in->read(m->maxregexmatch || maxregexmatch); if(!buf || !(value=rgx->split(buf))) { alread+=({buf}); throw(severity); } in->unread(buf[sizeof(value=value[0])..]); value-="\r"; if(has_suffix(value,"\n")) value=value[..<1]; } } if(!m->drop) ret[fname]=value; } if(!eol && gets(0)!="") throw(severity); severity=1; recordcount++; if(verb) { array s=({recname,"::"}); foreach(ret;string name;string value) if(sizeof(value)) s+=({value,","}); werror("%d %.*s\r",recordcount,verb, replace(s[..<1]*"",({"\n"," "," "}),({""," "," "}))); } return options->fold?ret:([recname:ret]); } private void add2map(mapping res,string name,mixed entry) { mapping|array tm = res[name]; if(tm) { if(mappingp(tm)) tm=({tm,entry}); else tm+=({entry}); res[name]=tm; } else res[name]=entry; } parsestruct feed(string content) { in->unread(content); return this; } int skipemptylines() { string line; int eof=1; while((line=in->gets()) && emptyline->match(line)); if(line) eof=0,in->unread(line+"\n"); return eof; } mapping fetch(void|array|mapping formats) { mapping ret=([]); int skipempty=0; if(!formats) { if(skipemptylines()) return UNDEFINED; skipempty=1;formats=fms; } if(arrayp(formats)) { foreach(formats;;array|mapping fmt) if(arrayp(fmt)) { array alreadold=alread; for(int found=0;;found=1) { alread=({}); mixed err=catch { mapping rec=getrecord(fmt,found); foreach(rec;string name;mixed value) add2map(ret,name,value); alreadold+=alread; continue; }; severity=1; if(err!=1) throw(err); in->unread(alread*""); break; } alread=alreadold; } else if(fmt=fetch(fmt)) ret+=fmt; if(skipempty) skipemptylines(); } else for(int found=1;found;found=0) { foreach(formats;string name;array|mapping subfmt) for(;;) { mapping m=fetch(subfmt); if(m) { found=1;add2map(ret,name,m); continue; } break; } if(skipempty && skipemptylines()) break; } return sizeof(ret) && ret; } } -- Sincerely, Stephen R. van den Berg. "The difficult we do today; the impossible takes a little longer."

2 1

SHA 0 or 1 in pike 7.4
by cschalkwijk 26 Jan '09

26 Jan '09

There is a SHA (or sha) function provided by Pike 7.4 in the Crypto class. But is this SHA-0 or SHA-1? -- Met vriendelijke groet / With kind regards / mit besten Grüßen, Coen Schalkwijk Software Engineer coen.schalkwijk(a)rtl.nl <mailto:coen.schalkwijk@rtl.nl> coen(a)rtlinteractief.nl <mailto:coen@rtlinteractief.nl> +31 (0)35 671 8915 __________________________________________________________ Deze e-mail en de inhoud is vertrouwelijk en uitsluitend bestemd voor de geadresseerde(n). Indien u niet de geadresseerde bent van deze e-mail verzoeken wij u dit direct door te geven aan de verzender door middel van een reply e-mail en de ontvangen e-mail uit uw systemen te verwijderen. Als u geen geadresseerde bent, is het niet toegestaan om kennis te nemen van de inhoud, deze te kopieren, te verspreiden, bekend te maken aan derden noch anderszins te gebruiken. The information contained in this e-mail is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Please notify us immediately if you have received it in error by reply e-mail and then delete this message from your system. __________________________________________________________

3 2

Looking for LZMA (de)compression in Pike
by cschalkwijk 22 Jan '09

22 Jan '09

Has anyone already done this? Can I have it. Please? :) Seriously. I'm currently using the 7zip LZMA sdk in a dotNET project which will probably communicate to pike (/roxen) using LZMA compressed info. Now I'm looking for some help realizing LZMA (de)compression in pike. -- Met vriendelijke groet / With kind regards / mit besten Grüßen, Coen Schalkwijk Software Engineer coen.schalkwijk(a)rtl.nl <mailto:coen.schalkwijk@rtl.nl> coen(a)rtlinteractief.nl <mailto:coen@rtlinteractief.nl> +31 (0)35 671 8915 __________________________________________________________ Deze e-mail en de inhoud is vertrouwelijk en uitsluitend bestemd voor de geadresseerde(n). Indien u niet de geadresseerde bent van deze e-mail verzoeken wij u dit direct door te geven aan de verzender door middel van een reply e-mail en de ontvangen e-mail uit uw systemen te verwijderen. Als u geen geadresseerde bent, is het niet toegestaan om kennis te nemen van de inhoud, deze te kopieren, te verspreiden, bekend te maken aan derden noch anderszins te gebruiken. The information contained in this e-mail is confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Please notify us immediately if you have received it in error by reply e-mail and then delete this message from your system. __________________________________________________________

2 3

Multi-cpu design
by Martin Stjernholm, Roxen IS ＠ Pike developers forum 21 Jan '09

21 Jan '09

I've been working on a multi-cpu design for Pike during the last couple of months, and now I think it's time to get some criticism on it. It's not finished, and unfortunately it's a bit long already. I still hope the people around here can find the time to read it through and mull a bit on it, because this hack is going to be rather large. I've tried to explain things briefly so that no knowledge of the pike core internals should be required - most of it is hopefully understandable for anyone who has written some C module code. The goal is not only to make pike run reasonably on 4-8 cores. It's rather to completely avoid inherent hotspots in the interpreter, so that the level of parallellism only depends on the pike program. I.e. this should scale to 100+ cpu's. Very briefly, the approach is to divide all data into thread local things that need no locking, and an arbitrary number of "lock spaces". A "lock space" is a monitor lock that controls access to any amount of data - it is up to the programmer to divide the shared data into suitable pieces that are locked independently. Some structures (e.g. arrays and possibly mappings) are lock-free. Refcounting is selectively - or perhaps even completely - disabled to avoid hotspots. I'll attach the whole thing as a comment to this message. I've also checked it in as multi-cpu.txt in a new branch "multi-cpu" in srb's pikex repository (lyskom people see 16711234). Btw, merry Christmas! ;)

8 25

Pike 7.8 released
by Peter Bortas ＠ Pike developers forum 01 Jan '09

01 Jan '09

After an excruciatingly long development period the Pike developers are proud to present the first release of Pike 7.8. Download from https://pelix.ida.liu.se/download/pub/pike/latest-stable/ The changes compared to the beta are pretty uninteresting fixes here and there. The 7.9 tree will open up as soon as the relevant people gets back from vacations. This release was done in the last minutes of 2008 and celebrated with champange and fireworks! Happy New Year everyone! Changes since Pike 7.6: ---------------------------------------------------------------------- This is a high level list of changes between Pike 7.6 and Pike 7.8. General bug fixes, build fixes and optimizations are not mentioned here. For a complete list of changes, please consult the CVS changelog either directly or through Code Librarian. This high level changelog is also available here: https://pike.ida.liu.se/download/notes/7.8.xml New / improved language functionality ------------------------------------- o New syntax to index from the end in range operations. A "<" can be added before an index in the [..] operator to count from the end instead, beginning with 0 for the last element. This is convenient to e.g. chop off the last element in an array: a[..<1]. o New `[..] operator function. Range operations have been separated from the `[] operator function and are now handled by the new `[..] which provides greater control for how the range bounds are specified. For compatibility, if there is no `[..] then `[] is still called for range operations. The `[..] callback will get four arguments, start value, start type, end value and end type. The type arguments is any of Pike.INDEX_FROM_BEG, Pike.INDEX_FROM_END or Pike.OPEN_BOUND. Here are a few examples of what input arguments different calls would generate. [..] 0 OPEN_BOUND 0 OPEN_BOUND [1..] 1 INDEX_FROM_BEG 0 OPEN_BOUND [..2] 0 OPEN_BOUND 2 INDEX_FROM_BEG [3..4] 3 INDEX_FROM_BEG 4 INDEX_FROM_BEG [<5..] 5 INDEX_FROM_END 0 OPEN_BOUND [6..<7] 6 INDEX_FROM_BEG 7 INDEX_FROM_END o Generalized this and this_program. It is now possible to refer to inherits in objects. Example: class A { int a; void foo() { werror("A\n"); } } class B { inherit A; int b; void foo() { werror("B\n"); } A::this_program get_a() { return A::this; } } In the above B()->get_a() will return an object with two symbols, 'a' and 'foo', but B()->get_a()->foo() will still write "B\n". o Added support for getters and setters. It is now possible to simulate variables with functions. Example: class A { private int a; int `b() { return a; } // Getter for the symbol b. void `b=(int c) { a = c; } // Setter for the symbol b. int c() { return b; // Calls `b(). } } object a = A(); a->b = 17; // Calls `b=(17). werror("%d\n", a->b); // Calls `b(). o Casting to derived types. It is now possible to call a value of type type to perform the corresponding value cast. eg: typedef string MyString; return MyString(17); // Casts 17 to "17". o Unicode escapes. Pike now understands the common way to escape unicode chars, using \uxxxx and \Uxxxxxxxx escapes. These escapes works both in string and character literals and in the preprocessor. The latter means that unicode escapes can be used in identifiers, which is useful if they contain characters that can't be represented raw in the source code charset. o Stricter type checker for function calls. The type checker for function calls is now based on the concept of currification. This should provide for error messages that are more easily understood. It also is much better att typechecking function calls utilizing the splice (@) operator. The mechanisms used by the typechecker are also made available as Pike.get_first_arg_type(), Pike.low_check_call() and Pike.get_return_type(). o Stricter typing of strings. The string type may now have an optional value range. string(0..255) bytes; o Support for having multiple different active backend implementations. In Pike 7.6 and earlier, there could only be one active backend implementation at a time, which was the one selected for best performance with lots of files (cf Pike.Backend below). This led to problems when Pike attempted to run poll-device based backends on older versions of operating systems where poll devices aren't available, and with extra overhead when the backend was only used with a very small set of files. Basic backend implementations: - Pike.PollDeviceBackend This is a backend that is implemented based on keeping the state in the operating system kernel and thus eliminating the need to send the state in every system call. This is however not available on all operating systems. Currently supported are /dev/poll (Solaris, etc), epoll(2) (Linux) and kqueue(2) (FreeBSD, MacOS X). - Pike.PollBackend This is a backend that is implemented based on the poll(2) system call. This is typically available on all System V or Linux based systems. - Pike.SelectBackend This is a backend that is implmented based on the BSD select(2) system call. This backend is available on all operating systems supported by Pike. Derived backend implementations: - Pike.Backend This is the backend selected among the basic backend implementations, which is likely to have the best performance when there are lots of files in the backend. - Pike.SmallBackend This is the backend selected among the basic backend implementations, which is likely to have the best performance when there are very few files in the backend. Note that there is also a default backend object: - Pike.DefaultBackend This is the Pike.Backend object that is used for file callbacks and call outs if no other backend object has been specified. o cpp The preprocessor now supports macro expansion in the #include and #string directives. #include USER_SETTINGS o Destruct reason passed to lfun::destroy. lfun::destroy now receives an integer flag that tells why the object is being destructed, e.g. if it was an explicit call to destroy(), running out of references, or reaped by the garbage collector. These integers are defined in the new Object module as DESTRUCT_EXPLICIT, DESTRUCT_NO_REFS, DESTRUCT_GC and DESTRUCT_CLEANUP. o Improved support for mixin. The Pike compiler now supports mixin for symbols that have been declared protected. Mixin is the concept of overloading symbols via multiple inheritance. In previous versions of Pike the mixin paradigm was only supported for public symbols. For more information about mixin see eg http://en.wikipedia.org/wiki/Mixin . o Implicit and explicit create(). The compiler now supports defining classes with both an implicit and an explicit create(). o Warnings for unused private symbols. The compiler now checks that all symbols that have been declared private actually are used. class A (int i) { protected string j; protected void create(string j) { A::j = reverse(j); } } o Warnings for unused local variables. The compiler now checks that all local variables are used. o Unicode Case information and the Unicode module are updated to Unicode 5.1.0. o The keyword protected The modifier protected is now an alias for the modifier static. NOTE: In the next release static will be deprecated. o extern declared variables Variables can now be declared as 'extern'. This means that inheriting classes must have them. They can be used like normal variables in the base class. Example: class A { extern int a; int dummy() { return ++a; } } o __attribute__ and __deprecated__ It's now possible to set custom attributes on types, so that it is possible to make custom type checking. This is currently used to improve the argument checking for sprintf() and related functions, and for marking symbols as deprecated. eg: __deprecated__ mixed obsolete_function(); __deprecated__(mixed) obsolete_return_value(); mixed fun(__deprecated__(mixed) obsolete_arg); __deprecated__(mixed) obsolete_variable; The deprecated type flag using __deprecated__ is a convenience syntax to use instead of e.g. void f(void|__attribute__("deprecated",int) timeout) Other uses of __attribute__ in type declarations can be seen in e.g. the type for werror(): > typeof(werror); (1) Result: scope(0,function(string : int) | function(__attribute__("sprintf_format", string), __attribute__("sprintf_args", mixed) ... : int) | function(array(string), mixed ... : int)) o __func__ The symbol __func__ now evaluates to the name of the current function. Note that this name currently can differ from the declared name in case of local functions (i.e. lambdas). Note also that __func__ behaves like a literal string, so implicit string concatenation is supported. eg: error("Error in " __func__ ".\n"); o __DIR__ __DIR__ is a new preprocessor symbol that resolves to the directory that the current file is placed in. Similar to how __FILE__ points out the file the code is placed in. o #pragma {no_,}deprecation_warnings Warnings for use of deprecated symbols can be turned off for a segment of code with #pragma no_deprecation_warnings and turned on again with #pragma deprecation_warnings o Compatibility name spaces Older versions of a function can be reached through its version name space. For example the 7.4 version of the hash function can be called through 7.4::hash(). o Iterator API The iterator API method Iterator->next() is no longer optional. Extensions and New Functions ---------------------------- o exit() Exit now takes optional arguments to act as a werror in addition to exiting the process. exit(1, "Error while opening file %s: %s\n", path, strerror(errno())); o getenv()/putenv() getenv() and putenv() are now accessing and modifying the real environment. o get_dir() Calling get_dir() with no arguments will now return the directory listing for the current directory. o undefinedp()/destructedp() undefinedp() and destructedp() have been added as more readable alternatives to zero_type(). o limit() The new toplevel function limit(a, x, b) is a convenience function that works like min(max(a,x),b). o listxattr(), getxattr(), setxattr(), removexattr() The new xattr functions listxattr(), getxattr(), setxattr and removexattr() allows extended file attributes to be modified from within Pike. o sprintf() and sscanf() - sprintf() now attempts to lookup the name of the program when formatting types of objects and programs. - The new formatting directive %H can be used to format a string as a binary hollerith string. > sprintf("%2H", "Hello"); (1) Result: "\0\5Hello" - The new formatting directive %q can be used to format a atring as a quoted string, quoting all control character and non-8-bit characters as well as quote characters. Like %O but always on one line. > sprintf("%q", "abc \x00 \" \' \12345"); (1) Result: "\"abc \\0 \\\" ' \\u14e5\"" - Ranges in sscanf sets can now start with ^, even at the beginning of the set specifier. Example: %[^-^] matches a set with only ^ in it. To negate a set with - in it, the - character must now be the last character in the set specifier. o encode/decode value and programs --with-portable-bytecode is now the default. Pike programs that have been dumped on one architecture now can be decoded on another. o gethrtime, gethrvtime, gauge Added support for POSIX style timers using clock_gettime(3). Notably this fixes nice high resolution thread local cpu time and monotonic real time on reasonably modern Linux systems. There are new constants CPU_TIME_* and REAL_TIME_* in the System module to allow pike code to query various properties of the CPU and real time clocks in use. o ADT.BitBuffer Added read() method that reads whole bytes from the buffer and returns as a string. o ADT.Queue It is now possible to use sizeof() and values() on a Queue object to get the size of the queue and all elements in the queue as an array. o ADT.Stack Stack objects can now be cast to arrays to get all elements on the stack. o ADT.Struct - New Item class SByte, SWord and SLong represents a signed byte, word and longword respectively. - The Item classes int8, uint8, int16, uint16, int32, uint32, int64 and uint64 are aliases for already existing Item classes. - sizeof() now works for empty Struct objects. - sizeof() can be used on individual Item objects. - Struct objects can now be used as Items in other Structs. class Pair { inherit ADT.Struct; Item id = uint8(); Item val = uint64(); } class File { inherit ADT.Struct; Item magic = Chars(4); Item one = Pair(); Item two = Pair(); } o Array - New function combinations() returns all combinations of a specified length with elements from a given array. - Added push(), pop(), shift() and unshift() functions for Perl weirdos. o Calendar - Added new calendar Badi, used in the Baha'i religion. - Fixed bugs in discordian, where the year() was off by 1166 and now() was off 5 month per year. - Time objects now handle * and / with floats. A split() method has been added for more advanced splitting where the preferred time quanta can be given as an argument. - A new formatting method format_ext_time_short() has been added to Time objects. - Timezones are now read from /etc/localtime, if available. - Cleaned up year-day (yd) handling so that it never goes below 1. Instead be careful to use either year (y) or week-year (wy) depending on context. - Fixed inconsistent range handling in year() and years() that made them almost but not quite zero-based. Now they are one-based just like day()/days(), month()/months() etc. - Cleaned up conversion between weeks and years: E.g. if a week has days in two years then converting it to years will produce a range of both years. The attempt to always map a week to a single year is gone since it's inconsistent with how other overlapping time ranges are handled. If the user wants to convert a week to the year it "unambiguously" belongs to, (s)he can do Calendar.ISO.Year(week->year_no()). - Did away with the attempt to map leap days between February 24th and 29th before and after year 2000, since doing so breaks date arithmetic. - The last four changes above are not entirely compatible. Compatibility with 7.6 and older is retained with #pike 7.6. o CompilerEnvironment & CompilerEnvironment()->PikeCompiler The Pike compiler has been refactored to be more object-oriented and more transparent. It is now possible to customize the compiler by overloading functions in the above two classes. The compiler object used by Pike internally is available through the global constant DefaultCompilerEnvironment. o Debug The new function count_objects() will return the different kinds of objects existing within the Pike process. Useful when trying to pinpoint a leak of Pike objects. o Error The new function mkerror() will normalize any thrown value into a proper error object (or 0). o Filesystem Traversion has been extended with two new create arguments. The first one is a flag to supress errors and the other is a sorting function which will be applied to the entries in every directory. This callback can also be used to filter the directory entries. o Float The function isnan() can be used to check if a float is Not a Number. > Float.isnan(Math.inf/Math.inf); (1) Result: 1 o Gdbm Gdbm databases can be opened in synchronous mode or locking mode by adding "s" or "l" respectively in the mode parameter to the create method. o Geography.Position It is now possible to encode Position objects with encode_value(). o GLUE - The default event callback will now exit both on Exit and Escape events. - The texture, list and light id generators will now be created after we are sure that GL has been initialized, to prevent crashes on several systems. - BaseTexture and BaseDWIM now supports clamped textures. o Gmp Many small changes to fix Gmp build issues on different platforms. The most visible being that the Gmp module now is statically linked to work around Mac OS X 10.5 linker issues. o Gz - Added compress() and uncompress() functions as an simpler and more efficient but non-streaming interface. - Support for RLE And FIXED compression method, if supported by zlib. Give Gz.RLE or Gz.FIXED to the strategy argument of compress() or the deflate constructor. - Added support for configurable compression window size. This is the last argument to compress() and the constructor for inflate/deflate. o Image.Colortable - The new method greyp() can be used to query if the color in the color object is grey. - Partial support for serializing color objects. Dithering type and lookup mode is not currently saved. o Image.Dims Support for parsing out the dimensions of TIFF files has been added. o Image.FreeType - Added support for handling monochrome (bitmap) fonts. o Image.Image - Image object can now be serialized and deserialized with encode_value() and decode_value(). - It is possible to convert the colors in an image object to and from YUV (YCrCb) with the new rgb_to_yuv() and yuv_to_rgb() methods. o Image.Layer - It is now possible to get the raw bitmap data of a layer by casting the layer object to a string. o Image.PNG - Properly decode cHRM (chrome), sBIT (sbit), gAMA (gamma), pHYs (physical), oFFs (offset), tIME (time) chunks. - The compression level and strategy when compressing PNG images can be controlled by passing "zlevel" and "zstrategy" options to the encode() method. Available strategies are filtered, huffman, rle and fixed. Image.PNG.encode( img, ([ "zlevel":9, "zstrategy":Gz.RLE ]) ); o Image.TIFF Added support for little endian TIFF files. o Int Int.inf is an object that can be used as an infinitly large integer. o Java - If possible, Pike now uses libffi instead of creating our own trampolines. Adding libffi as a bundle is supported. - Added built-in support for SPARC64 trampolines. - The method signature fuzzy-matcher now only considers Pike strings to match formal parameters of type String, Object, Comparable, CharSequence or java.io.Serializable. - When the fuzzy-matcher can not decide on a single method signature, all candidates are now listed in the error message. o Locale.Charset - The character name normalizer now recognizes Unicode prefixes, e.g. "unicode-1-1-utf-7", though the Unicode version itself is ignored. - Added support for the following character sets GB18030/GBK (CP936) UTF-EBCDIC DIN-31624 (ISO-IR-38) ISO-5426:1980 (ISO-IR-53) ISO-6438:1996 (ISO-IR-39, ISO-IR-216) ISO-6937:2001 (ISO-IR-156) GSM 03.38 Mysql Latin 1 - Added typed encode and decode error objects, Locale.Charset.EncodeError and Locale.Charset.DecodeError, to make it possible to catch such errors in a better way. - ISO-IR non-spacers are now converted into combiners. o Math - Matrix multiplication was bugged and gave B*A instead of A*B, which is now fixed. - Matrix objects now have xsize() and ysize() methods to query their dimensions. - To ease your logarithmic needs log2() and logn() have been added. o MIME - Added remapping variants of the encode words functions with encode_word_text_remapped(), encode_word_quoted(), encode_words_quoted_remapped() and encode_words_quoted_labled_remapped(). - Added workaround for a bug in Microsoft Internet Explorer where it forgets to properly quote backslashes in the Content-Disposition field. - Fixed a bug that could occur when casting MIME.Message objects to strings. o Mysql - Two functions set_charset() and get_charset() have been added to set and query the connection charset. The create() function also takes a "charset" setting in the options mapping. - Improved Unicode support. The MySQL driver now supports (possibly wide) Unicode queries and text return values, handling the connection charset and all encoding/decoding internally. This is enabled by setting the charset to "unicode", or through the new functions set_unicode_encode_mode() and set_unicode_decode_mode(). See their documentation for further details. o Odbc The Odbc module has been updated to support the UnixODBC library, and several issues with Unicode and FreeTDS handling have been fixed. o Oracle - The module has been updated to work with Oracle 10. - An approximation of the number of rows in a result object can be queried from the new function num_rows(). o Parser.HTML - Allow string and array as argument to _set_*_callback(). Those variants work just like a function only returning the string or array. o Parser.Pike and Parser.C - Parser.Pike and Parser.C have been rewritten in C for increased performance. - The #string directives should be handled correctly now. o Parser.RCS - The RCS parser has been rewritten to be more robust with regards to broken RCS data. o Parser.XML.NSTree - Added add_child_before() and add_child_after() methods to the NSNode object. o Parser.XML.Simple - The autoconvert() function, used to autodetect the character encoding of an XML file and decode it, has been moved from being a method of the Parser.XML.Simple object to being a function in the Parser.XML module. o Parser.XML.SloppyDOM Yet another DOM-like module aimed at quickly and conveniently parse an xml snippet (or whole document) to access its parts. Nodes can be selected using a subset of XPath. Footnote: This module was previously part of Roxen WebServer. o Parser.XML.Tree The module code has been refactored and a second "simple" interface has been added. The (new) SimpleNode interface implements a node tree interface similar to the (old) Node interface, with two major differences: - The SimpleNodes do not have parent pointers, this means that they do not generate cyclic data structures (and thus less garbage, and no need for zap_tree()), but also that it is no longer possible to find the root of a tree from any node in the tree. - Some methods in SimpleNode return different values than the corresponding method in Node; notably SimpleNode()->add_child(), which returns the current node rather than the argument. The refactoring also added individual classes for all of the XML node types (both for Nodes and for SimpleNodes). This allows for stricter typing of code involving XML tree manipulation. Several new functions added to manipulate and insert nodes in the XML tree. The module now also has a much better knowledge of DTDs and DOCTYPEs. o Postgres - Extended the SQL query interface to include affected_rows() and streaming_query() as well as variable bindings. - Automatic binary or text transfer for queryarguments and resultrows. o Pike - A new function count_memory() has been added which can calculate the memory consumed by arbitrary data structures. Useful when implementing memory caches. - A new function get_runtime_info() has been added which returns information about current ABI, if automatic bignums are enabled, what bytecode method is used, the size of native floats and integers and the native byte order. - The constants INDEX_FROM_BEG, INDEX_FROM_END and OPEN_BOUND has been added for use with the `[..] operator API. - The functions low_check_call(), get_return_type() and get_first_arg_type() allows for inspection of attributes and return values of functions. o Pike.Backend Besides the new multiple backend implementations described earlier, backends now support two new callbacks: before_callback and after_callback are two new variables in the backend objects. They can be set to functions that gets called just before and after a backend is waiting for events. o Process - The new function spawn_pike() will spawn a Pike process similar to the current one, using the same binary file, master and module paths. - The new function run() is an easy interface that will run a process and return a mapping with all the outputs and exit code. - Process.popen is now able to run in nonblocking mode. If a second argument is provided a file object will be opened with that mode and return, enabling two way communication with the new process. - The system() function has been extended to be able to pass stdin, stdout and stderr arguments to the spawn() call it performs. o Protocols.DNS - Added support for NAPTR (RFC 3403) and SPF (RFC 4408) records. - The gethostbyname() function now returns IPv6 addresses too, if available. - Fixed bugs in IPv6 record parsing. o Protocols.Bittorrent - Support for gzipped and compact tracker responses. - Many performance and bug fixes, such as 30% faster hashing of files. o Protocols.HTTP - Added support for httpu and httpmu requests. - Queries will now throw an exception in case of an errno condition. - A new function, do_async_method(), has been added to allow access to low level asynchronous HTTP calls. - The http_encode_string() function now knows how to encode UCS-2 characters. o Protocols.HTTP.Server - If accept() fails on the open port, the Port object will continue trying, to support servers with high load. o Protocols.HTTP.Query - Added unicode_data() method that will return the payload decoded according to the charset described in the Content-Type header. - Many fixes for bugs in asynchronous mode. - A query will not silently downgrade to http from https anymore if there is no crypto support. - Fixes for keep alive. o Protocols.LDAP - Enabled support for paged queries. - Added more resilience to UTF-8 encode errors. Locale.Charset.DecodeError is thrown for UTF-8 decode exceptions. - Added a connection pool for connection reuse. It is used through get_connection() and return_connection(). - Added some schema handling and use it to fix the UTF-8 conversion to only affect attributes that actually are UTF-8 encoded. - Added client.read(), client.read_attr(), client.get_root_dse_attr(), client.get_basedn(), client.get_scope(), client.get_attr_type_descr(), get_constant_name(), and a bunch of constants for various object oids, attributes, well-known object guids and other things. - Rewrote the search filter parser to handle LDAPv3 extensible matches. It now also throw errors on all syntactic errors (using a new FilterError object), instead of sending a partial filter to the server. It is also possible to compile filters separately through make_filter(), and there is a very simple cache for compiled filters through get_cached_filter(). - Added ldap_encode_string(), ldap_decode_string(), encode_dn_value(), canonicalize_dn(), parse_ldap_url(), client.get_parsed_url(), and client.get_bound_dn(). - Added client.result.fetch_all(). - Added new flag field to client.search() to control various aspects of how results are returned: SEARCH_LOWER_ATTRS lowercases all attribute names. SEARCH_MULTIVAL_ARRAYS_ONLY uses arrays for attribute values only for attributes that might return more than one value. SEARCH_RETURN_DECODE_ERRORS may be used to avoid throwing exceptions on decode errors. - Added client.get_protocol_version(), client.get_supported_controls(), and the possibility to specify controls in client.search(). - Made the result iterator somewhat more convenient: next() now advances one step past the end so the next fetch() returns zero. - Added client.error_number(), client.error_string(), and client.server_error_string() to make it possible to query errors when no result object is returned. o Protocols.SNMP The readmsg() method in the protocol object now takes an optional timout argument. o Protocols.XMLRPC The new AsyncClient class implements an asynchronous XMLRPC client. o Regexp.PCRE.Widestring - Replace matches in a string, with support for backreferences, now possible from replace_positional(). > Regexp.PCRE.Plain("my name is ([a-zA-Z]+)")-> >> replace_positional("hello, my name is john.", >> "%[0]s is my name"); (1) Result: "hello, john is my name." - Regexp.PCRE.Widestring is now reported in the basic feature list (pike --features). o Sql - Bugfixes in listing Postgres fields. - If ENABLE_SPAWN_RSQLD is defined, rsqld will be spawned when needed to complete rsql queries. - Added streaming_query() method to Sql objects which enables larger result sets than available memory. - It is possible to iterate over the result object from big_query() queries directly in foreach. - Support UNDEFINED to designate NULL in emulated bindings. - Support for ODBC DSN files. Sql.Sql db = Sql.Sql("dsn://user:pass@host/database"); - Some native support for the TDS protocol, used by Sybase and Microsoft SQL server. Sql.Sql db = Sql.Sql("tds://user:pass@host/database"); - Support for the SQLite database added. A raw interface is available through the SQLite module. Sql.Sql db = Sql.Sql("sqlite://relative/path/to/file"); Sql.Sql db = Sql.Sql("sqlite:///absolute/path/to/file"); - Sql.pgsql. New driver for native PostgreSQL network protocol support. It implements a superset of the existing Postgres driver. Current features: no library dependencies (no libpq), native binding support, streaming support, NOTIFY/LISTEN support (fully eventdriven, no polling), binary support for integer, float and string datatypes through big_typed_query(), native support for 64-bit ints and doubles, COPY FROM/TO STDIN/STDOUT support, multiple simultaneous streaming queries on the same connection (i.e. multiple PostgreSQL-portal- support), automatic precompilation and caching of often-used long-compile-time-needing queries, extended columndescriptions, accurate error messages under all circumstances, SSL-support, SQL-injection protection since it will ignore everything after the first semicolon delimiting the first command in the query, integrated statistics, _reconnect callback for sessions that use temptables. Performance tuned, with the helperclass _PGsql.PGsql it currently is around 21% faster than the old libpq based Postgres driver for smaller queries; for large bytea blobs and large texts, it speeds up even more. Support for this driver is indicated by PostgresNative appearing in the featurelist, and since it has no library or OS dependencies, it will always be available. This driver serves URLs of the form: pgsql:// (plain) and pgsqls:// (SSL). In case the old Postgres driver is disabled, this driver takes over postgres:// transparently as well. o SSL It is now possible to set certificates to SSL connections. Example: SSL.sslfile ssl_connection(Stdio.File conn, string my_key, string my_certificate) { ctx->client_rsa = Standards.PKCS.RSA.parse_private_key(my_key); // client_certificates is an array holding arrays of certificate // chains. since we've got a self-signed cert, our cert array has // only 1 element. ctx->client_certificates += ({ ({ my_certificate }) }); return SSL.sslfile(conn, ctx, 1, 1); } o Standards.IIM Some bugfixes in parsing Photoshop headers and DOS EPS Binary Headers. o Standards.ISO639_2 Updated with the latest ISO639-2 languages. o Standards.URI - Updated to conform to RFC 3986. - Added methods get_query_variables(), set_query_variables(), add_query_variable() and add_query_variables() to give a better API to to handle query variables. - The new method get_http_query() returns the query part and get_http_path_query() returns both the path and the query, both coded according to RFC 1738. o Standards.UUID - Added support for UUID version 5; name based with SHA hash, which can be generated from the make_version5 function. - An UUID object can now be used as namespace in the second argument to make_version3 as well as the new make_version5. o Standards.XML.Wix - Updated to support a more recent version of the Wix tool chain. - Improved generation of 8.3-style filenames. - Added support for Shortcut and IniFile-nodes. - Added support for overriding the language and installer version. - Improved support for TTF-files. o Stdio - Stdio.cp can now work recursively in a directory tree. It will also keep the permissions of files it copies. - Added Stdio.recursive_mv which works on every OS and also when the destination isn't on the same filesystem as the source. - Added more symbolc default termcap/terminfo bindings to Stdio.Readline. - Improved support for Terminfo on NetBSD. - read_file(), read_bytes(), write_file() and append_file() will may now throw exceptions on uncommon errors such as when write_file is unable to write all its data. - Stdio.File->openat(), statat() and unlinkat() opens, stats and removes a file or directory relative to an open directory. - Stdio.FILE->unread() allows pushing back binary strings into the input stream, as opposed to ungets() which pushes back lines. - Stdio.UDP has had enable_multicast(), set_multicast_ttl(), add_membership() and drop_membership() added to make real multicast use possible. o String - The function int2size has been rewritten to fixpoint as well as using the more common abbreviation of "B" for byte. - String.secure marks a string as "secure" which currently only means that the memory is cleared before it is freed. o System - resolvepath() is now enabled on more OSes and falls back to realpath(3C) if resolvepath(2) doesn't exists. - On systems that support it, setproctitle() can be used to set the title of the running application. - Added support for POSIX style timers using clock_gettime(3) to allow for high resolution thread local cpu time and monotonic real time on reasonable modern Linux systems for gethrvtime() and gauge(). Added CPU_TIME_RESOLUTION, CPU_TIME_IMPLEMENTATION, REAL_TIME_IS_MONOTONIC, REAL_TIME_RESOLUTION and REAL_TIME_IMPLEMENTATION constants to tell the system capabilities. o Tools.Hilfe - Added support for tab-completion on modules, global and local symbols and operators. - Added support for file/directory completion within strings - Added doc command and F1 key to print documentation on an item if available (currently only works for modules and classes written in pike). o Tools.Standalone - "pike -x cgrep" now tries to parse Java code. - "pike -x features" now tests for several more features. o Web.Crawler - Bugfix to support robots.txt created on windows. - User Agent change to "Mozilla 4.0 (PikeCrawler)" o Web.RDF - Added add_statement() method which allows new relations to be added to an RDF set. New modules / classes / methods added ------------------------------------- o Fuse FUSE (Filesystem in USErspace) provides a simple interface for userspace programs to export a virtual filesystem to the Linux kernel (and some other OS:es). FUSE also aims to provide a secure method for non privileged users to create and mount their own filesystem implementations. This module implements the needed interfaces to make it possible to write a FUSE filesystem in Pike. o ADT.List A simple doubly linked list of values. ADT.List l = ADT.List(1, 2, 3); l->insert(-1, 0); l->append(4, 5); foreach(l; int index; int value) { werror(" %d: value: %d\n", index, value); } o ADT.Set ADT.Set implements a datatype for sets. These sets behave much like multisets, except that they are restricted to containing only one instance of each member value. From a performance viewpoint, it is probably more efficient for a Pike program to use mappings to serve as sets, rather than using an ADT.Set, so ADT.Set is mainly provided for the sake of completeness and code readability. o Arg The new argument parser module allows for Getopt style argument parsing, but with a much simpler and object oriented API. class Parser { inherit Arg.Options; Opt verbose = NoOpt("-v")|NoOpt("--verbose")|Env("VERBOSE"); Opt name = HasOpt("-f")|HasOpt("--file")|Default("out"); Opt debug = MaybeOpt("-d")|MaybeOpt("--debug"); } void main(int argc, array(string) argv) { Parser p = Parser(argv); werror("name: %O, verbose: %O, debug: %O\n", p->name, p->verbose, p->debug); } A more simplistic interface is also available for smaller hacks and programs. void main(int argc, array(string) argv) { mapping opts = Arg.parse(argv); argv = opts[Arg.REST]; } o GSSAPI Implements Pike access to GSS-API v2 as specified in RFC 2743. This API is used to authenticate users and servers, and optionally also to encrypt communication between them. The API is generic and can be used without any knowledge of the actual implementation of these security services, which is typically provided by the operating system. The most common implementation is Kerberos, which means that the main benefit of this API is to allow clients and servers to authenticate each other using Kerberos, thereby making single sign-on possible in a Kerberized environment. o GTK2 Wrapper for the GTK2 library. Not yet 100% completed, but usable. o Protocols.DNS_SD This module provides an interface to DNS Service Discovery. The functionality of DNS-SD is described at <http://www.dns-sd.org/>. Using the Proctocols.DNS_SD.Service class a Pike program can announce services, for example a web site or a database server, to computers on the local network. When registering a service you need to provide the service name. service type, domain and port number. You can also optionally specify a TXT record. The contents of the TXT record varies between different services; for example, a web server can announce a path to a web page, and a printer spooler is able to list printer features such as color support or two-sided printing. The service is registered on the network for as long as the instance of the Service class is valid. o Bittorrent.Tracker Bittorrent tracker with support for scraping and UDP extension. o Protocols.HTTP.Server.Proxy A simple HTTP proxy. o Standards.TLD Country domains and other TLDs according to IANA. Useful when parsing log information or providing live information about clients connecting to your server. o Tools.Shoot Several new tests have been added to benchmark and improve on various aspects of Pike. ReplaceParallel and ReplaceSerial measure the times it takes to replace multiple substrings of a string in a single call to replace and with subsequent calls. TagRemoveArraySscanf, TagRemoveDivide, TagRemoveLoop, TagRemoveParserHTML, TagRemovePCRE, TagRemoveSearch and TagRemoveSscanf measure different methods of completing the same task; to remove XML tags from a string. o Web.CGI Provides a CGI interface on the callee side. Retrieves information from environment variables and populates the variables in the Request object. Deprecations ------------ o The keyword nomask has been deprecated. It was functionally equivivalent with the keyword final. o Stdio.File->set_peek_file_before_read_callback() is deprecated. Incompatible changes -------------------- These incompatible changes can be solved by adding #pike 7.6 to your source file or starting Pike with -V7.6 unless otherwise noted. o main() environment The main() method will no longer be given the environment as a mapping as third argument. Use an explicit call to getenv() instead. o Array.transpose_old This function has been removed. o Calendar Changes made to fix inconsistensies has created som unavoidable incompatibilities. See the entry for Calendar in the functional changes section for details. o _Charset The parts of this internal module that were written in Pike have moved to Locale.Charset. o Crypto The old crypto functions from Pike 7.4 have been removed. These functions produced a warning when used in Pike 7.6. o Debug.describe_program The API for this debug function has changed. o Image.Image The functions select_colors(), map_closest(), map_fast() and map_fs() has been removed. Use Image.Colortable operations instead. o Parser.XML The XML parsers are now stricter in verifying the correctness of the XML. The function compat_allow_errors can be called in the create method of the Parser.XML.Simple and Parser.XML.Validating (with "7.6" as argument for 7.6 compatibility). Parser.XML.Tree can be created with PARSE_COMPAT_ALLOW_ERRORS_7_6 as flag. o Protocols.LDAP.client The "dn" field wasn't properly utf-8 decoded in 7.6 and earlier. If your application does it yourself, you need to use the compatibility version of this class. o spider.XML The spider module no longer contains the XML parser. The functions isbasechar(), iscombiningchar(), isdigit(), isextender(), isfirstnamechar(), ishexchar(), isidiographic(), isletter(), isnamechar() and isspace() have also been moved to the Parser module. o Sql.Sql Pike will no longer create a .column entry in SQL query responses if there is no table name. o Standards.UUID Functions new() and new_string() have been removed. Use make_version1(-1)->encode() and make_version1(-1)->str() instead. o Stdio The functions read_file(), read_bytes(), write_file() and append_file() now always throw errors on error conditions, to allow easier use as errno doesn't have to be checked. read_file() and read_bytes() still return 0 if the file does not exist. o The modules Mird, Perl and Ssleay have been removed. Note that these modules are not available via the backwards compatibility layer. C level module API ------------------ o Improved support for embedding. Several declarations and definitions (most notably the debug and runtime flags) have moved from main.h to pike_embed.h, in an attempt to add support for embedding. o Major compiler API cleanups. The Pike compiler is now executing in a pike function context (rather than in an efun context), and it is possible to customize some of its behaviour via inherit (rather than via handler objects). As a consequence the compiler is now much closer to being thread-safe. o The global variable next_timeout is no more. It has been replaced by a backend-specific variable. Added backend_lower_timeout() for accessing the new variable. This fixes issues GTK, GTK2 and sendfile had with the new backend implementation. NOTE! C-API incompatibility! NOTE! Changed the argument for backend callbacks! The argument is now a struct Backend_struct * when called at entry (was NULL). The argument is now NULL when called at exit (was 1). o Pike_fp->context Pike_fp->context is now a pointer to the current struct inherit rather than a copy of it. This allows for easier access to inherited symbols in C-code. o Inherit level argument added to several object handling functions. In order to implement subtyping of objects, an extra argument "inherit_level" has been added to many object indexing related functions. o .cmod: Voidable pointer types are no longer promoted to mixed. o Support for class symbols with storage in parent scope. Also added support for aliased symbols. o Machine code backend for PPC64 - Machine code generation is now supported for PowerPC in 64-bit ABI mode. o Objectiv-C embedding framwork - Experimental support for interfacing with Objective-C code has been added. NOTE! This API is experimental and is subject to change NOTE! without notice. o Added %c and %C to get_all_args to get char * without NUL characters (no 0 valued characters inside the string). %c: char * Only narrow (8 bit) strings without NUL. This is identical to %s. %C: char * or NULL Only narrow (8 bit) strings without NUL, or 0 Building and installing ----------------------- o Dynamic modules now become DLLs on Windows. This means the homegrown module loader is no longer used, but it also means some DLL limitations: - PMOD_EXPORT is now required to allow access to an identifier in the pike core. - DLLs have to be recompiled too if pike.exe is recompiled. The primary motivation for this change is to work with the new library tracking (so-called "side-by-side assemblies") in Visual C++ 2005 and later. o Added ABI selection. It's now possible to select whether to compile in 32bit or 64bit mode at configure time by using the --with-abi option. o MinGW builds. It's now possible to build Pike in MinGW on windows computers from source distributions. o Cmod precompiler. - The cmod precompiler (pike -x precompile) now supports declaring all autogenerated identifiers as static. NOTE! C-API incompatibility! NOTE! As a side effect of this change, the DECLARATIONS statement NOTE! now required in cmod files. New simplified method to write external C/C++ modules ----------------------------------------------------- It's now suggested that you do not use the fairly complex 'pike internal' style of external modules (configure.in with AC_MODULE_INIT etc). It's also no longer required that you have a configure script to use pike -x module. Instead simply locate the includefiles using 'pike -x cflags', and convert .cmod to .c files using 'pike -x precompile'. An example rather minimal 'pike -x module' compatible Makefile, without a configure script, using .cmod format files for a simple local module: | CC=gcc | CFLAGS := -O9 -fweb -shared -Wall $(CFLAGS) $(shell $(PIKE) -x cflags) -g | LD=$(CC) -shared -lGL | | all: Spike.so | | install: Spike.so | cp $< $(MODULE_INSTALL_DIR) | | Spike.so: Spike.o | $(LD) -o Spike.so Spike.o $(LDFLAGS) | | Spike.o: Spike.c | | Spike.c: Spike.cmod | $(PIKE) -x precompile $< > $@ It's usually OK not to use pike -x module at all, but it will pass on a few extra variables to your make (and configure script): PIKE: How to start the pike interpreter used running pike -x module MODULE_INSTALL_DIR: Where modules goes LOCAL_MODULE_PATH: Alternative (user local) module location Known OS-specific issues: * The new Windows installer does not install all the explorer bindings the old one did, neither does it set up start menu entries. * The new Windows installer does not precompile the installed modules. * The provided windows build could use a few more modules: Freetype, PCRE, SDL and GTK.

2 1

7.8 beta - installed precompile.pike still not quite right
by Bill Welliver 23 Dec '08

23 Dec '08

I'm not sure who made changes to precompile.pike, so I don't want to make this change myself on the chance there's a reason for it. I installed the 7.8 beta a few days ago and the include/pike/precompile.pike installed still doesn't compile, as it uses a relative path to the real Tools.Standalone.precompile that's only valid inside the source tree. It would seem that you can't use the same file for both purposes while still using a relative path. Any reason why the relative path couldn't be changed to "inherit Tools.Standalone.precompile"? As it stands now, most external cmod modules are broken because of it. Bill

4 6

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel