Pike-devel November 2017

pike-devel@lists.lysator.liu.se

7 participants
46 discussions

Re: Proposed improvements to Hilfe
by Chris Angelico 23 Nov '17

23 Nov '17

On Thu, Nov 23, 2017 at 8:44 AM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: >>The other is the addition of a new help() built-in function, which >>gives human-readable information about whatever object it's given. >>Currently it shows a little information about functions and programs, >>and not much else; the intent is that it can be expanded at will, >>since it's purely human-readable. There is no backwards-compatibility >>guarantee about its specific output format. > > This sounds like it might overlap with the existing "doc" function? I'm not sure. help() is designed to work at runtime; doc doesn't seem to do much with functions and such, and I've never actually used it. In my help() code, I have a TODO about having a command for it. Maybe the doc command could call help() in situations where that makes sense? I've rebased the rosuav/hilfe-improvements branch onto current 8.1, so it's once again mergeable if this is of interest. The smaller change of suppressing quit/exit is merged into 8.1, leaving help() unmerged as yet. ChrisA

1 0

Faster Calls
by Arne Goedeke 23 Nov '17

23 Nov '17

I have pushed a new version of the faster calls branch. I readded support for profiling. I also fixed one more open bug. Of course there may more more. Performance-wise the situation did not change since the conference, but I have done some experimentation to improve it. One follow-up change will probably be a re-work of the function call API used from within interpreter. I also want to add more tail call optimizations where missing. Current comparison to 8.1 on my intel: interpreter/automap.pike#efun | 27.8 M 0.4 % | 41.1 M 0.6 % | interpreter/automap.pike#private | 17.3 M 0.7 % | 25.8 M 0.3 % | interpreter/automap.pike#public | 17.6 M 0.2 % | 26.1 M 0.1 % | interpreter/call.pike#efun |173.9 M 0.6 % |171.6 M 0.8 % | interpreter/call.pike#index | 15.7 M 0.4 % | 14.3 M 0.4 % | interpreter/call.pike#private | 20.9 M 0.5 % | 18.7 M 0.2 % | interpreter/call.pike#public | 21.1 M 0.2 % | 18.6 M 0.4 % | interpreter/call_array.pike#efun | 23.7 M 0.1 % | 34.1 M 0.3 % | interpreter/call_array.pike#private | 15.6 M 0.3 % | 14.1 M 0.1 % | interpreter/call_array.pike#public | 15.3 M 0.3 % | 14.0 M 0.1 % | interpreter/map.pike#efun | 41.8 M 0.4 % | 37.0 M 0.5 % | interpreter/map.pike#private | 15.7 M 0.5 % | 22.9 M 0.4 % | interpreter/map.pike#public | 16.0 M 0.1 % | 22.9 M 0.2 % | interpreter/recurse.pike#private | 27.9 M 0.4 % | 38.1 M 0.3 % | interpreter/recurse.pike#public | 27.7 M 0.2 % | 37.2 M 0.7 % | interpreter/tailcall.pike#private | 20.6 M 0.4 % | 31.4 M 0.4 % | interpreter/tailcall.pike#public | 21.0 M 0.2 % | 30.7 M 0.5 % | As you can see, the only slow-downs currently are in standard lfun calls. I believe this can be fixed using some refactoring of the corresponding opcodes. I believe that we can also come up with a simpler API which can more easily be called directly from machine code. I think the branch is now in a state in which it could be merged into 8.1. I expect that there are still some issues which will come up when running more complex code bases. I did write a test-suite which tries to cover all different function call types and I will integrate that into the standard testsuite soon. Unless anyone sees any issue with the code as it stands, I will use the decision reached during the conference and merge this within the next week or so. Arne

3 4

Re: Proposed improvements to Hilfe
by Chris Angelico 23 Nov '17

23 Nov '17

On Thu, Nov 23, 2017 at 8:44 AM, Henrik Grubbström (Lysator) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: >>Branch: rosuav/hilfe-improvements >> >>Two smallish improvements to Hilfe. One is suppressing "quit" and >>"exit", per the TODO at the top; fairly simple but I'd like someone >>else's eyeballing to make sure it's going about things the right way. > > Do you handle the case where the user ends the session with EOF (ie > usually ^D) correctly? > > /grubba Yes; the readline history-pop function takes an optional string argument of the string to pop, and will do nothing if the last history element is not that exact string. So if you key an EOF, or if you terminate Hilfe in some other way, the history pop will silently do nothing. ChrisA

1 0

"old" mails on the list
by Peter Bortas 23 Nov '17

23 Nov '17

Hello list subscribers, I finally took over responsibility for the LysKOM exporter, and ran it through the backlog. Which means you just got a bunch of new mails that have a send date of today but might have been written months and months ago in LysKOM. From now on the exporter will export new texts every few minutes. Regards, -- Peter Bortas, NSC

2 1

image loading
by Lance Dillon 23 Nov '17

23 Nov '17

So I'm working on a program to decode .pic files from old grasp .gl files. I have code to extract the parts of the .gl file into individual files. There is a script file, plus a bunch of images, either in .pic or .clp format. I'm only working on decoding the .pic files. It supports multiple bit depths, but I'm only doing the vga format, 1 plane with a 768 byte palette (256 each for b, g, r). I have the code to decode into bgri from the rle encoded file. I still have to work on converting it either to rgb or rgba, which might just be a simple map. Anyway, how would I convert this raw pixel data (8 bits for each color) into, say, an Image.Image so I can write as png? I'm looking at the reference documentation and I don't really see anything for importing raw pixel data. I might be able to do it using the gtk2 pixmap functions, but I was wondering if it can be done with the pike functions. Thanks

4 3

Irregularities in HMAC implementation of Pike 8.0 or 8.1?
by Stephen R. van den Berg 23 Nov '17

23 Nov '17

Pike v8.0 release 509 running Hilfe v3.5 (Incremental Pike Frontend) > Crypto.SHA256.HMAC("a")("b"); (1) Result: "\b\336""2\231""1\342\225h7v\252\232CR\233\320\262u(m\363\26\3\0\304\233\244\350A\203""0\23" > Crypto.SHA256.HMAC("a",32)("b"); (2) Result: "\av\242\236\370\215K@N\230/h\351\236\236\254v\320\370\270""2\17\231[\202xb+i\222\3 " Pike v8.1 release 11 running Hilfe v3.5 (Incremental Pike Frontend) > Crypto.SHA256.HMAC("a")("b"); (1) Result: "\b\336""2\231""1\342\225h7v\252\232CR\233\320\262u(m\363\26\3\0\304\233\244\350A\203""0\23" > Crypto.SHA256.HMAC("a",32)("b"); (2) Result: "\b\336""2\231""1\342\225h7v\252\232CR\233\320\262u(m\363\26\3\0\304\233\244\350A\203""0\23" They can't both be right. Any ideas? I tried to debug it, but got lost in the multitude of Nettle inheritance-spaghetti levels. -- Stephen.

2 1

Thread.ResourceCount
by Stephen R. van den Berg 23 Nov '17

23 Nov '17

A proposed new class, currently in 8.1. It allows for resource counting where the decrement is implicit by tieing it to the destruction of a key object that you got when incrementing. The implementation has already been battle-tested in pgsql in 8.1. Any suggestions related to functionality, interface and/or naming? As far as I could determine, there does not seem to be any established name for this kind of object. It's related to a semaphore, but it does things markedly different. Because it allows me to get rid of some convoluted code in pgsql, and because it automatically solves (if any) remaining races related to asynchronous destruction of objects; I'd like to use this class in 8.0 as well (in order to keep the pgsql driver in sync between 8.0 and 8.1). Any objections if I add this to 8.0? -- Stephen.

3 2

Crypto.hmac_abc
by Martin Nilsson (Coppermist) ＠ Pike (-) developers forum 23 Nov '17

23 Nov '17

Is there a point to the latest additions to the Crypto module besides putting hex-string output directly into crypto functions? So instead of previously writing String.hex2string(Crypto.MD5.HMAC(a)(b)); you can now do Crypto.hash_hmac(Crypto.MD5, 1)(a)(b);

2 1

Proposed improvements to Hilfe
by Chris Angelico 23 Nov '17

23 Nov '17

Branch: rosuav/hilfe-improvements Two smallish improvements to Hilfe. One is suppressing "quit" and "exit", per the TODO at the top; fairly simple but I'd like someone else's eyeballing to make sure it's going about things the right way. The other is the addition of a new help() built-in function, which gives human-readable information about whatever object it's given. Currently it shows a little information about functions and programs, and not much else; the intent is that it can be expanded at will, since it's purely human-readable. There is no backwards-compatibility guarantee about its specific output format. There's nothing particularly invasive about either of these changes, but I'd appreciate someone else's view on the code before it gets merged. Thanks! ChrisA

3 2

signals and fork does not mix well
by ceder (-) Per Cederqvist ＠ Pike (-) developers forum 23 Nov '17

23 Nov '17

This test program should print and endless row of "/" and "\" characters, but instead it often crashes like this: > \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\//\\//\/\/\/\\/\//\\//\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\\/Parent got SIGQUIT! > /home/cederp/pike-fun/forked-signals.pike:10: /main()->handler_quit() > /home/cederp/pike-fun/forked-signals.pike:33: /main()->arm_hup() > -:1: Pike.Backend(0)->`()(3600.0) > Child got SIGHUP! > /home/cederp/pike-fun/forked-signals.pike:26: /main()->handler_hup() > /home/cederp/pike-fun/forked-signals.pike:17: /main()->arm_quit() > -:1: Pike.Backend(0)->`()(3600.0) The cause: when Pike is compiled with NEED_SIGNAL_SAFE_FIFO the signal handler writes the signal number as a single byte to the write end of a pipe(2) set up by INIT_FIFO (in src/signal_handler.c). This pipe is set to close on exec -- but it doesn't close when the process forks. So, if a Pike program uses fork, and one of the processes receives a signal, it is undefined if it is handled by the parent or the child process. Both processes will read from the same pipe. I think this patch fixes the issue (comments welcome): --- cut here --- Invoke signal handlers in the correct process after fork When using fork() from Pike, the child needs to have its own pipe to send pending signals over. If it is shared with the parent, either the child or the parent may receive the signal. Close the pipe and create a new one in the child after a fork. Do the same for the process wait info. diff --git a/src/signal_handler.c b/src/signal_handler.c index dca9fe3..55b8a13 100644 --- a/src/signal_handler.c +++ b/src/signal_handler.c @@ -317,6 +317,7 @@ #define QUICK_CHECK_FIFO(pre,TYPE) ( PIKE_CONCAT(pre,_first) != PIKE_CONCAT(pre,_last) ) #define INIT_FIFO(pre,TYPE) +#define REINIT_FIFO(pre,TYPE) #else /* NEED_SIGNAL_SAFE_FIFO */ @@ -361,6 +362,12 @@ set_close_on_exec(PIKE_CONCAT(pre,_fd)[1], 1); \ }while(0) +#define REINIT_FIFO(pre,TYPE) do { \ + close(PIKE_CONCAT(pre,_fd)[0]); \ + close(PIKE_CONCAT(pre,_fd)[1]); \ + INIT_FIFO(pre,TYPE); \ +} while(0) + #endif /* else NEED_SIGNAL_SAFE_FIFO */ #ifndef SAFE_FIFO_DEBUG_END @@ -4373,6 +4380,10 @@ void Pike_f_fork(INT32 args) /* forked copy. there is now only one thread running, this one. */ num_threads=1; #endif + + REINIT_FIFO(sig, unsigned char); + REINIT_FIFO(wait,wait_data); + /* FIXME: Ought to clear pid_mapping here. */ call_callback(&fork_child_callback, 0); push_int(0); --- cut here --- Perhaps REINIT_FIFO should reset the counters when !NEED_SIGNAL_SAFE_FIFO? The test program: --- cut here --- #!/usr/bin/env pike int is_child = 0; // This code runs in the child and repeatedly sends SIGQUIT to itself. void handler_quit() { if (!is_child) error("Parent got SIGQUIT!\n"); werror("\\"); call_out(arm_quit, 0.1); } void arm_quit() { if (!kill(getpid(), signum("SIGQUIT"))) error("Failed to send QUIT: %s\n", strerror(errno())); } // This code runs in the parent and repeatedly sends SIGHUP to itself. void handler_hup() { if (is_child) error("Child got SIGHUP!\n"); werror("/"); call_out(arm_hup, 0.1); } void arm_hup() { if (!kill(getpid(), signum("SIGHUP"))) error("Failed to send HUP: %s\n", strerror(errno())); } // Setup code. int main() { // Set up the same signal handlers in both the parent and the child. signal(signum("SIGHUP"), handler_hup); signal(signum("SIGQUIT"), handler_quit); if (fork()) { // Start the parent. is_child = 0; call_out(arm_hup, 0.1); } else { // Start the child. is_child = 1; call_out(arm_quit, 0.1); } // Both the parent and the child runs in the pike backend. return -1; }

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel November 2017