Pike-devel November 2016

pike-devel@lists.lysator.liu.se

9 participants
52 discussions

Inheriting from a mapping
by Stephen R. van den Berg 23 Nov '16

23 Nov '16

Given the attached sample code. It tries to profile the difference between accessing a native mapping and accessing it through a class which is declared inline as much as possible. When I run it, I find that the native mapping is about twice as fast and the inlined amapping class. What would it take to make an amapping class less than a factor 1.2 slower than the native mapping? Would being able to inherit from some kind of "_Builtin._mapping" class be of any use? I ran the tests on Pike 8.1. ---------------------------------cut here-------------------- #!/usr/local/bin/pike #define REPEAT 10000 class timeit { protected void create(string name, function(:void) fn) { int i,t1,t2; mixed e; t1 = gethrtime(); e = catch { for (i=REPEAT; i>0; i--) fn(); }; if (e) write("Skipped %s\n", name); else { t2 = gethrtime(); write("%f %s\n", (t2-t1)/(float)REPEAT, name); } } } class amapping { private mapping values; inline protected void create(mapping v) { values = v; } inline private mixed `[](string key) { return values[key]; } inline private mixed `->(string key) { return values[key]; } inline private mixed `[]=(string key, mixed v) { return values[key] = v; } inline private mixed `->=(string key, mixed v) { return values[key] = v; } inline private Iterator _get_iterator() { return get_iterator(values); } inline private array(string) _indices() { return indices(values); } private string _sprintf(int type, void|mapping flags) { string res = UNDEFINED; switch (type) { case 'O': res = sprintf("%O", values); int indent; if (flags && (indent = flags->indent)) res = replace(res, "\n", "\n" + " " * indent); break; } return res; } } int main(int argc, array(string) argv) { mapping m = ([ "abc":8, "acc":1, "adc":2, "aec":3, "afc":4, "agc":5, "ahc":6, "adc":2, "1dc":3, "3dc":2, "4dc":2, "5dc":8, "6dc":7, "7dc":2, "8dc":3, "rdc":2, ]); amapping am = amapping(m); timeit("mapping", lambda(){ int total = 0; foreach (indices(m);; string id) total += m[id]; return total; }); timeit("amapping", lambda(){ int total = 0; foreach (indices(m);; string id) total += am[id]; return total; }); return 0; } ---------------------------------cut here-------------------- -- Stephen.

2 2

Request for 8.1 refdoc on the Pike devel website alongside 8.0 refdoc
by Stephen R. van den Berg 23 Nov '16

23 Nov '16

I'd like to suggest that we make available a generated Refdoc for the development tree (8.1 currently) on the pike website somewhere too. Regenerated regularly through xenofarm perhaps. -- Stephen.

2 2

Re: Clean-room Engine.IO implementation committed to git 8.0/8.1
by Chris Angelico 23 Nov '16

23 Nov '16

On Wed, Nov 23, 2016 at 11:10 PM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: >>I agree, but using string(8bit) to mean "binary data" is something >>that's 100% backward compatible. > > It would not be backwards compatible, since that is not what > string(8bit) means today. By "binary data", I mean eight-bit strings of arbitrary bytes - like you'd read from a file or something. Currently, functions like Stdio.read_file simply return "string", but they'll effectively be returning string(8bit). >>Unicode text would always be referred >>to as string(21bit), even if it happens to contain nothing but Latin-1 >>characters. > > That doesn't really make sense. So you say that "R\xe4ksm\xf6rg\xe5s" > would have type string(21bit)? What type would "\U12345678" have? \U12345678 possibly should be an error, as it's not valid Unicode. Maybe the Pike string type can be used for other things, but they're not Unicode text - so you could use string(32bit) for those sorts of non-textual strings. (I don't know of any use cases, so I can't say beyond that.) My statement about Unicode text specifically excludes anything that isn't valid according to the Unicode standard. > What type would "Foo" have? How would you specify a UTF-8 encoded > literal? Now, these are questions that can't truly be answered with the current system. I would like the former to be string(7bit), and the latter would be either string(7bit) or string(8bit) depending on whether there are non-ASCII characters in it. But they're probably both just type 'string' at the moment. ChrisA

3 5

Re: Clean-room Engine.IO implementation committed to git 8.0/8.1
by Chris Angelico 23 Nov '16

23 Nov '16

On Thu, Nov 24, 2016 at 12:20 AM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: >>\U12345678 possibly should be an error, as it's not valid Unicode. > > It's valid Pike. Pike supports the full ISO/IEC 10646 31-bit range, > plus an equally large negative range. > >> so you could use string(32bit) for those sorts of >> non-textual strings. > > Not string(31bit)? > >>My statement about Unicode text specifically excludes >>anything that isn't valid according to the Unicode standard. > > Which makes it even worse since the set of valid characters change > with each release of the Unicode standard... AIUI the Unicode Consortium has declared that they will never define any characters beyond 0x10FFFF, as that would destroy UTF-16 as a valid encoding. And you're right that "valid according to the Unicode standard" is a bit too restrictive, but certainly "valid within the declaration of character range" should be safe. >>> What type would "Foo" have? How would you specify a UTF-8 encoded >>> literal? >> >>Now, these are questions that can't truly be answered with the current >>system. I would like the former to be string(7bit), > > Then you are contradicting yourself, since you claimed that Unicode > text would _always_ be referred to as string(21bit), and "Foo" is > definitely Unicode text (both 'F' and 'o' have been part of the > Unicode standard since the first version). string(7bit) would be implicitly upcastable to string(21bit), since ASCII text can be represented validly as either bytes (with the top bit clear) or as Unicode codepoint sequences. >>and the latter >>would be either string(7bit) or string(8bit) depending on whether >>there are non-ASCII characters in it. > > But how would the compiler know that the characters are UTF-8 encoded, > so that it does not assign a type of string(21bit) instead? Right, and that's something that can't be done in the current standard. Hence this entire proposal has to wait until some major changes can be done. In Python, it's done with a prefix - u"asdf" is a Unicode string, and b"asdf" is a byte string. It would need to be something similarly syntactic, once the two become actually different types. For today's Pikes, though, that's not possible, so the safest way is to simply keep track of it in the programmer's mind, without language support. ChrisA

3 3

My first CMOD
by Pontus Östlund 23 Nov '16

23 Nov '16

From time to time it so happens that I have to write vanilla CSS (you don't do that 2016 ;) and it gives me such anxiety you wouldn't believe it :D So I went about it and wrote a CMOD wrapper for libsass http://sass-lang.com/libsass <http://sass-lang.com/libsass>. Since I hardly know any C and very very little about the internals of Pike I would very much appreciate if someone would take a look at the module and give me feedback. And also, if you think it fits in the Pike repo, does it fit under Tools or should it be located somewhere else. It really doesn't matter to me personally. Any feedback is welcome. Regards ----------------------------- Pontus Östlund Developer • Roxen AB +46 70-662 81 69 www.roxen.com <http://www.roxen.com/> | twitter.com/roxen <https://twitter.com/roxen>

3 5

little endian read
by Lance Dillon 23 Nov '16

23 Nov '16

Is there an easy way to read a string from a file and read it in little endian format to convert to int? I have some old file formats that I want to convert, and header information is stored in little endian format. So far I have something like this for a WORD (2-bytes): Stdio.File fp;fp=Stdio.File(filename,"r");string str=fp->read(2);int res=array_sscanf(reverse(str),"%2c")[0]; Trying to find an easy way to make it variable, like 2 or 4 byte ints (there are some DWORDs in the file too). array_sscanf doesn't take parameters for the format, so I can't do "%*c",2 . Thanks..

5 12

Re: Clean-room Engine.IO implementation committed to git 8.0/8.1
by Chris Angelico 23 Nov '16

23 Nov '16

On Wed, Nov 23, 2016 at 10:00 PM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > If there are no character values >127, then the encoding step is a > no-op, so skipping it buys you nothing except making your code harder > to read. I envision a future time when we can usefully distinguish between string(21bit) and string(8bit), with string(7bit) being trivially transformed into either of the above. As such, calling utf8_to_string or string_to_utf8 would be an actual type conversion (albeit of a subtype, rather than actually requiring any major changes). ChrisA

3 2

Re: Clean-room Engine.IO implementation committed to git 8.0/8.1
by Chris Angelico 23 Nov '16

23 Nov '16

On Wed, Nov 23, 2016 at 10:30 PM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > I think you are conflagrating range with interpretation. Both a > Latin1 string and an UTF-8 encoded one are 8-bit strings (with a 0-255 > range). What would be useful is a datatype that declares that the > elements are not Unicode characters (as they are in the Latin1 string > case) but some raw binary encoding (as they are in the UTF-8 case), > optionally also specifying which encoding. This has been suggested > before (with "buffer" as a suggestion for the name of the new > datatype), but it has never been implemented due to the difficulty of > introducing such a datatype in a consistent way while still retaining > backward compatibility. > I agree, but using string(8bit) to mean "binary data" is something that's 100% backward compatible. Unicode text would always be referred to as string(21bit), even if it happens to contain nothing but Latin-1 characters. FWIW, I would support an actual division of data types, such that you cannot concatenate one onto the other. But having seen what happened with Python 2 -> Python 3, I would expect this to be a fairly significant backward compatibility break. It'd probably be something for Pike 9.0 or even 10.0. There would be an opportunity to learn from Python here, though, and maybe do things more smoothly. In any case, the first step is to *right now* think about binary data and textual data as different things, and distinguish them and convert them as appropriate. ChrisA

2 1

Clean-room Engine.IO implementation committed to git 8.0/8.1
by Stephen R. van den Berg 23 Nov '16

23 Nov '16

Please review, any comments are welcome. The docs still need improvement, working on that. Currently tackling Socket.IO. -- Stephen.

4 8

Patches from Debian
by Magnus Holmgren, Millnet/Lysator/Debian/Mensa ＠ Pike developers forum 23 Nov '16

23 Nov '16

Although the Autoconf manual says "Avoid running test programs if possible, because this prevents people from configuring your package for cross-compiling."

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel November 2016