Pike-devel November 2016

pike-devel@lists.lysator.liu.se

9 participants
55 discussions

Re: MIME.Message and RFC 2047 encodings
by Chris Angelico 22 Nov '16

22 Nov '16

On Wed, Nov 23, 2016 at 1:35 PM, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > IÃ'm sorry, but your code is incorrect, and it cannot be done > correctly either. As I explained, the encoding rules depend on the > grammar of the paricular header field. Use of the _text functions is > correct only for headers which grammar declare them to be "text". > Headers which instead use the "phrase" non-terminal in their grammar > (such as the "From:" and "To:" headers) need to be treated > differently. There is even an example in RFC1522 which is not > correctly decoded by your class. Looks like the LysKOM to mailing list gateway is finally up again - yay! Small oddity of encoding (unless that's a typo in that "I'm" above) but that's no biggie. Thank you thank you thank you to whomever is responsible for this. ChrisA

1 0

Lambdas, captured variables
by Martin Karlgren 22 Nov '16

22 Nov '16

Hi all, I’m sometimes annoyed by the trampoline garbage that results when referencing variables in the surrounding scope from a lambda function (“closure”). Also, all variables in the frame referenced by the lambda will be kept around, even if just a single one is actually used in the lambda. void foo() { string var1; mapping var2; string bar = “whatever"; function f = lambda(string arg) { write("%O, %O\n", bar, arg); }; … other code that prevents tail-call optimization ... return f; } int main(int argc, array argv) { function f = foo(); f(“gazonk”); f = 0; // The lambda and the variables referenced in the frame used to execute “foo” are now garbage. } A possible workaround is to cut the reference to the “foo” frame: function f = lambda(string var1, string var2) { return lambda(string arg) { write("%O, %O, %O\n”, var1, var2, arg); }; }(var1, var2); However, this is pretty verbose. Maybe it would make sense to introduce some kind of syntactic sugar, for example: lambda[var1, var2](string arg) { }; This could perhaps translate to the construct above, and I think this syntax would map pretty well to how explicitly captured variables work in other languages. Some languages allow renaming of captured variables – not sure if it’s necessary or not. "lambda[myvar1 = var1, myvar2 = var2]() {}" or something. Ideally, variables from the surrounding scope not present in the capture list should not be usable in the lambda, to avoid unintended trampolines. Obviously, this would follow the same pass-by-value/pass-by-reference semantics of the various Pike types as in function arguments, which I personally think is a good thing. (A special case would be "lambda[](string arg) { }” which would prevent referencing any variables from surrounding scopes at all.) What do you all think? /Marty

4 8

MIME.Message and RFC 2047 encodings
by Chris Angelico 22 Nov '16

22 Nov '16

Currently, Pike's MIME.Message parser doesn't handle non-ASCII headers with specified encodings: > MIME.Message("Hello, world!", (["Subject": "Hello, \U0001F310"])); (10) Result: Message(([ ])) > (string)_; (11) Result: "Subject: Hello, \U0001f310\r\n" "Content-Length: 13\r\n" "\r\n" "Hello, world!" Going the other way: > MIME.Message("Subject: =?UTF-8?B?SGVsbG8sIPCfjJA=?=\r\n\r\nHello, world!"); (13) Result: Message(([ ])) > _->headers; (14) Result: ([ /* 1 element */ "subject": "=?UTF-8?B?SGVsbG8sIPCfjJA=?=" ]) I'm currently working with IMAP and RFC[2]822 messages. I could either implement RFC 2047 parsing in my app, or enhance MIME.Message to return Unicode strings automatically. Would this functionality be welcomed in trunk? ChrisA

3 12

Re: Build of 7.8.866 getting stuck in a loop
by Chris Angelico 22 Nov '16

22 Nov '16

On Wed, Nov 23, 2016 at 9:15 AM, Marcus Agehall (nu med K-märkt fastighet och ny elcentral) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > Since we are pretty far into 8.0 now, I'd say it's time to move > forward. > > Most smaller projects would probably not even notice the switch to 8.0 > and the bigger ones, such as Roxen, have already done the switch. > > Others may disagree, but I would say it's perfectly fine to drop 7.8 > and only ship 8.0. I try to still support 7.8, partly because the 8.0 builds for Windows are still on old libraries (incl GTK). And the build process defeated me :( ChrisA

1 0

WebSocket, skips parse_variables at times, why?
by Stephen R. van den Berg 22 Nov '16

22 Nov '16

I'm currently implementing the Engine.IO (and possibly Socket.IO) protocol in Pike (bidirectional realtime communication with JavaScript clients). While doing so I noticed (to my dismay) that in WebSocket, I find the following code snippet: protected int parse_variables() { if (has_index(request_headers, "sec-websocket-key")) return 0; return ::parse_variables(); } I.e. parse_variables() is skipped on real websocket connections. What would be the reason why this is being skipped? I need to parse the variables on the URL of the soon-to-be-upgraded websocket connection; so it would be more convenient if it was handled automatically. -- Stephen.

4 6

Build of 7.8.866 getting stuck in a loop
by Magnus Holmgren, Millnet/Lysator/Debian/Mensa ＠ Pike developers forum 22 Nov '16

22 Nov '16

I think you used the wrong comment character, though... > ...peep.in:39:Unknown preprocessor directive NUMBER. > Cpp() failed

1 0

Build of 7.8.866 getting stuck in a loop
by Magnus Holmgren, Millnet/Lysator/Debian/Mensa ＠ Pike developers forum 22 Nov '16

22 Nov '16

PEEP at 468: nop => PEEP at 468: nop => PEEP at 470: mark global(2) => mark & global(2) PEEP at 471: assign local and pop(1) local(1) => assign local(1) PEEP at 471: push int(0) => push 0 PEEP at 471: push 0 index => int index(0) PEEP at 472: mark local(1) => mark & local(1) PEEP at 473: mark call builtin(23) => mark, call builtin(23) PEEP at 474: push 0 assign local and pop(5) => clear local(5) PEEP at 474: clear local(5) & local(5) => & local(5) clear local(5) PEEP at 475: local(5) clear string subtype index => local index(5) PEEP at 475: global(3) local index(5) => global[local](3,5) PEEP at 476: mark global(3) => mark & global(3) PEEP at 476: local(5) clear string subtype index => local index(5) PEEP at 477: mark local(5) => mark & local(5) PEEP at 477: local(5) clear string subtype index => local index(5) PEEP at 477: global(3) local index(5) => global[local](3,5) PEEP at 474: nop => PEEP at 474: nop => PEEP at 479: call builtin 1(24) pop => call builtin1 & pop(24) PEEP at 480: push 0 assign local and pop(5) => clear local(5) PEEP at 480: clear local(5) & local(5) => & local(5) clear local(5) PEEP at 481: mark local(5) => mark & local(5) PEEP at 481: local(3) local(5) => 2 locals(3,5) PEEP at 481: 2 locals(3,5) clear string subtype index => local[local](5,3) PEEP at 482: push int(0) => push 0 PEEP at 482: push 0 index => int index(0) PEEP at 483: local(4) local(5) => 2 locals(4,5) PEEP at 483: push int(1) => push 1 PEEP at 483: push 1 index => int index(1) PEEP at 480: nop => PEEP at 480: nop => PEEP at 487: push 0 assign local and pop(5) => clear local(5) PEEP at 487: clear local(5) & local(5) => & local(5) clear local(5) PEEP at 488: mark string(147) => mark & string(147) PEEP at 489: local(4) local(5) => 2 locals(4,5) PEEP at 489: 2 locals(4,5) clear string subtype index => local[local](5,4) PEEP at 487: nop => PEEP at 487: nop => PEEP at 490: push 0 assign local and pop(5) => clear local(5) PEEP at 490: clear local(5) & local(5) => & local(5) clear local(5) PEEP at 491: local(5) clear string subtype index => local index(5) PEEP at 491: global(5) local index(5) => global[local](5,5) PEEP at 492: assign local and pop(6) local(6) => assign local(6) PEEP at 492: push int(-2147483648) => push -int(-2147483648) PEEP at 492: push -int(-2147483648) => push int(-2147483648) PEEP at 492: push int(-2147483648) => push -int(-2147483648) PEEP at 492: push -int(-2147483648) => push int(-2147483648) ... ad infinitum. So an integer overflow, but why?

1 0

Build of 7.8.866 getting stuck in a loop
by Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum 22 Nov '16

22 Nov '16

>Here are two: >(gdb) bt >#0 0x000000000055d953 in advance () at ./src/peep.c:910 >#1 asm_opt () at ./src/peep.c:1081 >#2 assemble (store_linenumbers=store_linenumbers@entry=1) at ./src/peep.c:441 [...] >(gdb) cont >Continuing. >^C >Program received signal SIGINT, Interrupt. >0x000000000048a738 in low_make_buf_space (space=space@entry=0, buf=buf@entry=0x98f820 <instrbuf>) at ./src/dynamic_buffer.c:17 >17 ./src/dynamic_buffer.c: Filen eller katalogen finns inte. >(gdb) bt >#0 0x000000000048a738 in low_make_buf_space (space=space@entry=0, buf=buf@entry=0x98f820 <instrbuf>) at ./src/dynamic_buffer.c:17 >#1 0x00000000005571b4 in instr (offset=0) at ./src/peep.c:878 >#2 opcode (offset=0) at ./src/peep.c:884 >#3 low_asm_opt () at ./src/peep_engine.c:4560 >#4 0x000000000055d969 in asm_opt () at ./src/peep.c:1098 >#5 assemble (store_linenumbers=store_linenumbers@entry=1) at ./src/peep.c:441 [...] Looks like it's probably an infinite loop in the peep-hole optimizer. Recompile the pike binary configured --with-debug, and run the script by hand with -a3. Hopefully the looping peep-hole rules will be obvious then. /grubba

1 0

Build of 7.8.866 getting stuck in a loop
by Magnus Holmgren, Millnet/Lysator/Debian/Mensa ＠ Pike developers forum 22 Nov '16

22 Nov '16

Nobody has any idea why the 7.8 build hangs but not the 8.0 build? Is it time to abandon pike7.8 already? Actually I'm not sure if there's any good reason to keep multiple versions in parallel in the same manner as Python; it was just packaged that way when I took over.

3 2

Added permessage-deflate support to WebSocket
by Stephen R. van den Berg 22 Nov '16

22 Nov '16

Added it to both 8.0 and 8.1. It's in use in my Socket.IO implementation, and so far it passes all tests. Since testing it involves using a webbrowser as counterpart, my tests are sadly not comprehensive. Arne, could you check the code and see if it meets your criteria on the proposed visible interfaces? There is a little bit of trickery involved in the rsv bits. In order to know if to decompress, we need to know the RSV1 bit on the *first* frame part of the larger frameset. -- Stephen.

4 11

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel November 2016