Pike-devel October 2016

pike-devel@lists.lysator.liu.se

4 participants
12 discussions

Optimised map
by Martin Karlgren 09 Jan '17

09 Jan '17

Hi, Having realised the benefits of functional programming, I’ve been quite annoyed by the rumour of how expensive function calls are in Pike. I decided to look into f_map and could see how much seemingly unnecessary work it does when it calls apply_svalue once for each entry in the array – it should be possible to reuse the pike_frame if it’s about to be thrown away after the function call (if it has other refs on the other hand, it can’t be reused – it’s probably used as a scope in some other frame). I’ve pushed my optimised variant in marty/optimised_map – it seems to work quite well and provides a major speedup. In fact, it’s a bit faster than the corresponding foreach variant. I haven’t verified correctness in various corner cases, and some input on whether it’s correct to do the things init_frame_reuse_context does only once before multiple function calls would be nice too. The *_reuse_context stuff in interpret.c should be applicable wherever the same svalue is applied repeatedly with the same number of arguments (I haven’t looked for it outside of f_map really). What do you all think? Good idea or did I overlook something? Without optimisation: map: 1.660797 array index: 1.335115 array append: 1.17917 With optimisation: map: 0.877659 array index: 1.351158 array append: 1.189812 Test program: int main() { array base = allocate(10000000, 1); float gmap = gauge { array res = map (base, lambda(int i) { return i + 2; }); }; float garrayindex = gauge { array res = allocate(sizeof(base)); foreach (base; int idx; int i) { res[i] = i + 2; } }; float garrayappend = gauge { array res = ({}); foreach (base, int i) { res += ({ i + 2 }); } }; werror ("map: %O\n", gmap); werror ("array index: %O\n", garrayindex); werror ("array append: %O\n", garrayappend); } /Marty

4 12

Clean-room Engine.IO implementation committed to git 8.0/8.1
by Stephen R. van den Berg 23 Nov '16

23 Nov '16

Please review, any comments are welcome. The docs still need improvement, working on that. Currently tackling Socket.IO. -- Stephen.

4 8

Lambdas, captured variables
by Martin Karlgren 23 Nov '16

23 Nov '16

Hi all, I’m sometimes annoyed by the trampoline garbage that results when referencing variables in the surrounding scope from a lambda function (“closure”). Also, all variables in the frame referenced by the lambda will be kept around, even if just a single one is actually used in the lambda. void foo() { string var1; mapping var2; string bar = “whatever"; function f = lambda(string arg) { write("%O, %O\n", bar, arg); }; … other code that prevents tail-call optimization ... return f; } int main(int argc, array argv) { function f = foo(); f(“gazonk”); f = 0; // The lambda and the variables referenced in the frame used to execute “foo” are now garbage. } A possible workaround is to cut the reference to the “foo” frame: function f = lambda(string var1, string var2) { return lambda(string arg) { write("%O, %O, %O\n”, var1, var2, arg); }; }(var1, var2); However, this is pretty verbose. Maybe it would make sense to introduce some kind of syntactic sugar, for example: lambda[var1, var2](string arg) { }; This could perhaps translate to the construct above, and I think this syntax would map pretty well to how explicitly captured variables work in other languages. Some languages allow renaming of captured variables – not sure if it’s necessary or not. "lambda[myvar1 = var1, myvar2 = var2]() {}" or something. Ideally, variables from the surrounding scope not present in the capture list should not be usable in the lambda, to avoid unintended trampolines. Obviously, this would follow the same pass-by-value/pass-by-reference semantics of the various Pike types as in function arguments, which I personally think is a good thing. (A special case would be "lambda[](string arg) { }” which would prevent referencing any variables from surrounding scopes at all.) What do you all think? /Marty

4 8

MIME.Message and RFC 2047 encodings
by Chris Angelico 23 Nov '16

23 Nov '16

Currently, Pike's MIME.Message parser doesn't handle non-ASCII headers with specified encodings: > MIME.Message("Hello, world!", (["Subject": "Hello, \U0001F310"])); (10) Result: Message(([ ])) > (string)_; (11) Result: "Subject: Hello, \U0001f310\r\n" "Content-Length: 13\r\n" "\r\n" "Hello, world!" Going the other way: > MIME.Message("Subject: =?UTF-8?B?SGVsbG8sIPCfjJA=?=\r\n\r\nHello, world!"); (13) Result: Message(([ ])) > _->headers; (14) Result: ([ /* 1 element */ "subject": "=?UTF-8?B?SGVsbG8sIPCfjJA=?=" ]) I'm currently working with IMAP and RFC[2]822 messages. I could either implement RFC 2047 parsing in my app, or enhance MIME.Message to return Unicode strings automatically. Would this functionality be welcomed in trunk? ChrisA

3 12

WebSocket, skips parse_variables at times, why?
by Stephen R. van den Berg 23 Nov '16

23 Nov '16

I'm currently implementing the Engine.IO (and possibly Socket.IO) protocol in Pike (bidirectional realtime communication with JavaScript clients). While doing so I noticed (to my dismay) that in WebSocket, I find the following code snippet: protected int parse_variables() { if (has_index(request_headers, "sec-websocket-key")) return 0; return ::parse_variables(); } I.e. parse_variables() is skipped on real websocket connections. What would be the reason why this is being skipped? I need to parse the variables on the URL of the soon-to-be-upgraded websocket connection; so it would be more convenient if it was handled automatically. -- Stephen.

4 6

Added permessage-deflate support to WebSocket
by Stephen R. van den Berg 22 Nov '16

22 Nov '16

Added it to both 8.0 and 8.1. It's in use in my Socket.IO implementation, and so far it passes all tests. Since testing it involves using a webbrowser as counterpart, my tests are sadly not comprehensive. Arne, could you check the code and see if it meets your criteria on the proposed visible interfaces? There is a little bit of trickery involved in the rsv bits. In order to know if to decompress, we need to know the RSV1 bit on the *first* frame part of the larger frameset. -- Stephen.

4 11

nettle in fedora/centos does not include the secp192 and 224 curves
by Martin Bähr 22 Nov '16

22 Nov '16

hi, i just discovered that nettle in fedora/centos does not include the secp192 and 224 curves. this is not discovered during build, but instead just leads to a runtime error. (the installation succeeds despite the error) to make building pike work properly, the absence of these needs to be tested, so the code referencing them can be skipped. i verified that, removing the references to those curves makes pike run without error, though i am not sure if this should be a compile option or if automatic detection is better. greetings, martin.

3 2

Performance/memory usage difference?
by Stephen R. van den Berg 22 Nov '16

22 Nov '16

Given two solutions: 1.==================================================== class A { final int b; final C e; class C { final int D() { return b; } }; void create() { e = C(); }; }; or 2.: ================================================ class C { final array b; final array D() { return b; }; void create(array _b) { b = _b; }; }; class A { final array b; final C e; void create() { e = C(b); }; }; ==================================================== Assuming that there are (a lot) more other members and methods in both classes (which are accessed within the class, not cross-class), and further assuming that the A class is being instantiated a *lot*: What would the runtime impact of these two approaches considering CPU time and memory used? I would guess that solution 1 would be slightly more efficient, both memory and CPU-wise. Or am I overlooking something here where due to the nested stack frames of solution 1, the actual CPU or memory usage is higher than expected? -- Stephen.

4 3

Pike 8.0.294 beta
by Peter Bortas 22 Nov '16

22 Nov '16

Pike 8.0.294 beta/release candidate: https://pike.lysator.liu.se/pub/pike/beta/8.0.294/Pike-v8.0.294.tar.gz Other builds: https://pike.lysator.liu.se/pub/pike/beta/8.0.294/Pike-v8.0.294-Darwin-15.4… https://pike.lysator.liu.se/pub/pike/beta/8.0.294/Pike-v8.0.294-win32-oldli… -- Peter Bortas, NSC

2 1

Promises in Pike vs other languages
by Marcus Agehall (nu med K-m�rkt fastighet och ny elcentral) ＠ Pike (-) developers forum 09 Nov '16

09 Nov '16

I have spent some time looking at the new stuff in Concurrent.pmod that Grubba has added to 8.1. Mostly it is code relating to the concept of Promises and Futures which are very popular amongst JavaScript people these days. I have noted that there seems to be some rather striking differences between how JavaScript treats promises and how Pike does it. I'm not sure this is a good thing(tm) and therefore I'd like to bring this up before 8.1 stabilizes. In Pike, we have two methods that register callback functions, Future.on_success() and Future.on_failure() as opposed to JS where only one method, then(), is used to register both callbacks. I don't mind having two different methods, but for easier transition between the two languages (i.e. for people writing both client-side and server-side code) having the same API would probably help. A bigger problem imho is the difference in how promises are actually resolved. In JavaScript, the return value of then() is always a *new* promise which then allows for chaining. In Pike, we return the same promise object. This means that code like my_promise->on_success(foo)->on_success(bar) in Pike would result only in a call to bar() once my_promise is resolved whereas in JavaScript, foo() would be called and it's return value would be the input to bar in a new promise. The whole creation of promises is also somewhat awkward in Pike. In JS, you simply pass a callback that will do all the work and that will recieve a success callback and a failure callback as arguments and off you go. In Pike, I'm not sure what the best way to do it is. I really like the idea of having promises in Pike, but would it not be much better if we tried to implement an API more similar to what the JS folks are used to? At least I think so... Here are three links to good resources I've found regarding promises in JS. I'm sure there are a ton of others as well. https://promisesaplus.com/ http://www.mattgreer.org/articles/promises-in-wicked-detail/ http://bluebirdjs.com/docs/api-reference.html

4 10

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel October 2016