Pike-devel June 2010

pike-devel@lists.lysator.liu.se

5 participants
20 discussions

Re: Debian packages
by Marc Dirix 30 Jul '10

30 Jul '10

> My plan for simplifying things while catering to various wishes is to > * remove pike7.8-full > * let pike7.8 Recommend all other packages. That way everything will > be installed if you run apt-get install pike7.8 unless you have > switched off automatic installation of recommended packages, but > it's still possible to opt out of individual packages. I have made a small set of patches which I'd like to be included in pike and debian packages in order to make the package emdebian (cross) compilable. I've attached the patches, but as described below there are still some caveats to them. Patches according to http://wiki.debian.org/EmdebianMakefile Patches are made against the current debian package (pike 7.8.352). 1) Patch to debian/rules. This patch can be included as is. The patch makes it possible to use a cross compiler for compiling pike. (cross compiling in debian you can do using "pkg-build -a arm -uc -us) 2) Patch to src/configure.in This patch can be included as is into pike, I think this was is a bug in configure.in because it originally tries to use the newly build pike instead of the pike installed on the host. 3) Patch to src/configure I think this patch should ultimately be done in src/configure.in. But I'm no M4 expert and haven't yet found which changes need to be made in src/configure.in 4) Patch to src/modules/Mysql/configure I think this patch was already included in src/modules/Mysql/configure.in. Then there is the issue of bytecompiling. As I understand it pike can do without byte-compiling. However the debian packaging copies the byte-compiled files into the several packages. If byte-compiling fails (because the newly build pike can't be used for byte-compiling) the package manager fails to create a package. I've only come up with one possible sollution to this. Which is, push the byte-compiling of the code into a post-install script. This way the byte-compiling can be done on the target machine. But maybe there is a better sollution? It is not allowed to have diff between the debian and the emdebian packaging files. Best regards, Marc

6 31

Ideas for better performance
by Jonas Walld�n ＠ Pike developers forum 22 Jul '10

22 Jul '10

I'd like to start a discussion on performance issues where I feel we can do a lot better with reasonable effort. My test case was initially the XSLT module in our CMS but with tools such as Shark (an OS X sampling profiler) I've encountered some interesting results that I've now reproduced in isolation. A fundamental observation is of course that CPUs and memory subsystems have different performance properties today compared to when core parts of Pike were written. One particular detail is the frequent use of modulo (or rather int division) in hash tables which on x86_64 (if I recall correctly) has a latency in the 100+ cycle range. Examples of where this is used is mappings, identifier lookup, gc passes etc. A very naive replacement of %= in find_shared_string_identifier() (which typically scores high in an object-oriented test case like XSLT) improved benchmark runtime ~3-7% on my Core Duo laptop. My replacement was a very simple pointer shift/XOR mod 2^n. Next, enabling some additional Shark probes on misaligned memory writes together with the int div probe gave very weird results for code that I thought would be trivial. It turns out that this: void fn() { array arg = ({ 1, 2, 3 }); } ...copies arg recursively using a temporary mapping. Ouch! Consider a real-world example like the following: string quote_text(string txt, void|array from, void|array to) { from = from || ({ "<", ">", "&" }); to = to || ({ "<", ">", "&" }); return replace(txt, from, to); } ...which semantically uses the inlined arrays as constants if the caller doesn't pass from/to. In my mind this would just be a refcount bump but Grubba pointed out that arrays are not copy-on-write so that isn't currently possible. A similar test with mappings still passes copy_svalues_recursively_no_free so apparently the c-o-w property isn't taken advantage of there either. Adding c-o-w to arrays and using it in these situations would be really nice. I also found examples like the zeroing in real_allocate_array() where gcc (v4.2.1 in my case) generated quite lousy code. A rewrite to a pointer-based loop with item-chunked initialization was faster. Don't know if the compiler has to play safe with the repeated dereferencing of fields with unknown aliasing effects inside unions but I'll gladly help rewrite such code if people agree it's beneficial. Finally, anyone try compiling with Clang/llvm-gcc yet? I tested with llvm-gcc last year and aside from miscompilation in one place it was a tad slower overall compared to gcc. Maybe recent versions are better? Can we make a plan for action on these items or do we need the elusive 7.9 branch open first?

13 58

pike.ida.liu.se
by Peter Bortas ＠ Pike developers forum 29 Jun '10

29 Jun '10

Due to some misscommunication pelix and thus pike.ida.liu.se is offline. In all likelyhood it will remain that way until tomorrow morning.

1 0

Pike 7.4
by Martin Nilsson (Opera Mini - AFK!) ＠ Pike (-) developers forum 21 Jun '10

21 Jun '10

I'm trying to compile 7.4 and get some include problems. Anyone directly knows the issue? Outputting result files... Updating GTK/time_stamp Updating GTK/pgtk.c Total time spent... 1.6s make[5]: Entering directory `/home/nilsson/Pike/7.4/build/linux-2.6.24-27-gener\ ic-i686/post_modules/GTK' Makefile:274: warning: overriding commands for target `depend' Makefile:197: warning: ignoring old commands for target `depend' Compiling pgtk.c In file included from /home/nilsson/Pike/7.4/src/program.h:12, from /home/nilsson/Pike/7.4/src/post_modules/GTK/pgtk.h:13, from pgtk.c:2: /home/nilsson/Pike/7.4/src/global.h:58:21: error: machine.h: No such file or di\ rectory /home/nilsson/Pike/7.4/src/global.h:219:2: error: #error Unsupported type chose\ n for native pike integers. /home/nilsson/Pike/7.4/src/global.h:231:2: error: #error Unsupported type chose\ n for pike floats. (machine.h is in /home/nilsson/Pike/7.4/build/linux-2.6.24-27-generic-i686/)

2 2

Re: Crypto.CRC
by Marc Dirix 19 Jun '10

19 Jun '10

> Btw, it looks rather odd to me that you inherit the hash state class > into the base algorithm class. The hash state is supposed to contain > the context for a specific hashing while the algorithm class has the > static data (e.g. the polynomial). And the algorithm class (which is a > module and hence just a single instance) cannot return itself as > state, because then several hashings running in parallell would > clobber each other. I don't quite understand? In Martin Nilsson's email he stated it should look like below, so `() returns CRCState. Isn't this achieved by inheriting? What do you mean by Clobbering? Isn't by inheriting for every new instance of CRC16 also a new instance of CRCState created? The class CRCState is complete, except for the polynom. Should I create a CRCState object in the _CRC16 class and return it for `() ? class CRC { string name(); int digest_size(); int block_size(); CRCState `() string hash(string data); } class CRCState { string name(); int digest_size(); int block_size(); string hash(string data); CRCState update(string data); string digest(void|int length); }

2 1

Re: Crypto.CRC
by Marc Dirix 18 Jun '10

18 Jun '10

It doesn't seem to work that way. Attached is the patch to Crypto.pmod/module.pmod (this one is not complete just for testing purpose) If try to use it: >object a = Crypto.CRC16(); /usr/local/pike/7.8.447/lib/modules/Crypto.pmod/module.pmod:118:Illegal program pointer. Compiler Error: 1: Index 'CRC16' not present in module Crypto. Compiler Error: 1: Indexed module was: master()->joinnode(({ /* 1 element */ master()->dirnode("/usr/local/pike/7.8.447/lib/modules/Crypto.pmod":0) })). Compiler Error: 1: Attempt to call a non function value `() (function call). Compiler Error: 1: Expected: function. Compiler Error: 1: Got : zero. If I create a file "CRC16.pmod" like attached in a previous email. I can use CRC16(); > > protected class _MySubModule > { > ... > } > > _MySubModule MySubModule = _MySubModule(); >

2 1

Re: Crypto.CRC
by Marc Dirix 18 Jun '10

18 Jun '10

Ok. So if I see things right, these mentioned classes should go into "Crypto.pmod/module.pmod" and for each CRC type, I create a module, ie "Crypto.pmod/CRC16X.pmod" ? On 06/15/2010 09:40 PM, Martin Nilsson (Opera Mini - AFK!) @ Pike (-) developers forum wrote: > The hash API basically works looks like this > > class CRC > { > string name(); > int digest_size(); > int block_size(); > CRCState `() > string hash(string data); > } > > class CRCState > { > string name(); > int digest_size(); > int block_size(); > string hash(string data); > CRCState update(string data); > string digest(void|int length); > } -- Dipl.-Ing. Marc Dirix Electronics Design Schmeddingstraat 24 6361AZ Nuth Tel.: +31(45)7440744 E-Mail: marc(a)electronics-design.nl Internet: www.electronics-design.nl

2 1

Re: Crypto.CRC
by Marc Dirix 17 Jun '10

17 Jun '10

I hope this is somewhat in the right direction? What should "digest_size" and "block_size" return? Wouldn't it for the CRC function be better if "digest" and "hash" return the integer value, instead of a string? Best regards, Marc

1 0

Re: Crypto.CRC
by Martin Nilsson (Opera Mini - AFK!) ＠ Pike (-) developers forum 16 Jun '10

16 Jun '10

The hash API basically works looks like this class CRC { string name(); int digest_size(); int block_size(); CRCState `() string hash(string data); } class CRCState { string name(); int digest_size(); int block_size(); string hash(string data); CRCState update(string data); string digest(void|int length); }

1 0

Re: Crypto.CRC
by Marc Dirix 16 Jun '10

16 Jun '10

> > But if they are to be added to the Crypto module, they should be fixed > to follow the same API as the other hash functions though. I think > that'd be preferable regardless where in Pike they're added, as well. > Besides, the naming convention for functions in Pike is_like_this, and > notthisway. If there is a consensus about adding the module to Crypto I'd happily adapt the module to be more like other hash functions, and follow naming conventions better.

3 2

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel June 2010