Pike-devel August 2020

pike-devel@lists.lysator.liu.se

4 participants
7 discussions

Pike Meet-up 2018-Q1
by Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum 20 Oct '21

20 Oct '21

Hi. At Pike Conference 2017 it was decided that we ought to hold a more informal meeting in Linköping ~ once every quarter. As the three month mark from the conference is coming up soon, it seems time to announce the first such meeting for 2018: Pike Meet-up 2018-Q1 Date: 2018-02-03 (Sat) Time: 10:00 - afternoon/evening Place: Roxen IS AB S:t Larsgatan 22A Linköping Welcome! /grubba

2 13

Removing AggregateState increases the memory footprint of a Promise substantially
by Stephen R. van den Berg 07 Sep '20

07 Sep '20

Looking at: commit 36a7c5f2c3b44f5556fc2a8e05051211b0e3d834 Author: Henrik Grubbstr<C3><B6>m (Grubba) <grubba(a)grubba.org> Date: Wed Aug 26 17:44:39 2020 +0200 Concurrent.Promise: Refactored the dependency handling. I find: - final int _materialised; - final AggregateState _astate; + protected array(mapping(int:mixed)) dependency_results = ({ ([]), ([]) }); + protected int(0..) num_dependencies; + protected int(1bit) started; + protected int(0..) min_failure_threshold; + protected int(-1..) max_failure_threshold; + protected function(mixed, mixed, mixed ... : mixed) fold_fun; + protected array(mixed) extra_fold_args; If you are running a system with a huge number of promises, it thus causes a substantially increased memory footprint. The reason I used the _astate and _materialised variables was because it allows the memory footprint to be substantially lower after the promise has been setup. In systems with lots of long-lived promises running around, the memory footprint of a single promise seems quite relevant. -- Stephen.

1 1

Information leakage in httpserver
by Chris Angelico 29 Aug '20

29 Aug '20

string file = "."+combine_path("/",request->not_query); file = Protocols.HTTP.uri_decode(file); Stdio.Stat s = file_stat( file ); If the not_query contains "..%2F", combine_path sees that as a perfectly normal directory, and then it gets URI decoded, and Pike will happily read files outside of the directory it's been set to. Is there a reason the URI decode is done after combine_path? ChrisA

2 1

Segmentation fault cause
by Stephen R. van den Berg 28 Aug '20

28 Aug '20

Running without these commits, it runs forever. Running with these commits results in segmentation faults every few minutes to hours (this is in heavy threaded/lock code: pgsql driver). I cannot offer a stacktrace yet. commit 4781e5e4fe034cc8b2acef468131d5dbc51ca21f Author: Henrik Grubbström (Grubba) <grubba(a)grubba.org> Date: Sun Jun 28 16:33:02 2020 +0200 Threads.Mutex: Added {try_,}shared_lock(). src/threads.c | 256 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 246 insertions(+), 10 deletions(-) commit 35fa44331e70809a86a97a91bf80e27a38a6f35b Author: Henrik Grubbström (Grubba) <grubba(a)grubba.org> Date: Sat Jun 27 13:58:46 2020 +0200 Thread.Mutex: Keep track of pending locks. Mutex locks are now taken in the same order as the calls to lock(). This prevents potential starvation. src/threads.c | 181 +++++++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 137 insertions(+), 44 deletions(-) commit d461910bac03f0fddf714dff8794e73988d07ea6 Author: Henrik Grubbström (Grubba) <grubba(a)grubba.org> Date: Fri Jun 26 12:34:34 2020 +0200 Thread.Mutex: The key pointer is now to the storage (and not the object). src/threads.c | 69 ++++++++++++++++++++++++++++++++++--------------------------- 1 file changed, 39 insertions(+), 30 deletions(-) commit 37406b33d5670026537d4a9aca4776a88a94380b Author: Henrik Grubbström (Grubba) <grubba(a)grubba.org> Date: Thu Jun 25 15:37:42 2020 +0200 Build: Fixed some warnings. src/post_modules/GSSAPI/gssapi.cmod | 1 + src/post_modules/Nettle/hogweed.cmod | 1 + src/post_modules/SDL/SDL.cmod | 2 +- src/post_modules/ZXID/zxid.cmod | 2 +- 4 files changed, 4 insertions(+), 2 deletions(-) commit ea9a8260156eca36ceb1f555fd1c04c9f1c71685 Author: Henrik Grubbström (Grubba) <grubba(a)grubba.org> Date: Wed Jun 24 18:16:52 2020 +0200 Thread.MutexKey: Some preparations for having different kinds of keys. src/threads.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) -- Stephen.

3 30

Pike, WebSockets, and SSL
by Chris Angelico 12 Aug '20

12 Aug '20

Closing an SSL websocket bombs out. SSL sockets can only be closed in both directions at once, but websockets send a FRAME_CLOSE and then close in the write direction only. Is it necessary to close in the write direction only, or can the websocket simply close both ends at once? The exception can't easily be dealt with, since it happens in the backend. Demo code available here. If you don't have a certificate, it'll generate a self-signed; the behaviour is the same. https://github.com/Rosuav/StilleBot/blob/master/ws_halfclose.pike ChrisA

2 4

Re: Type checking of exponentiation
by Chris Angelico 09 Aug '20

09 Aug '20

On Mon, Aug 10, 2020 at 1:02 AM Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > > 4^-2 is not an integer. It's 0.0625 (or 1/16, if expressed as a > fraction, which seems to be what 8.1 has chosen here). True. In the original, though, this was guarded such that this expression would only ever have a positive exponent. I couldn't figure out a way to force STR to be interpreted as int(2..) though. The original was more like this, but the effect is the same: calc("STR < 2 ? STR : 4 ** (STR - 2)"); In theory, static analysis should show that the exponent has to be 0 or greater, but if it can't figure that out, I'd be okay with telling it that (eg with a cast). Attempting to add the cast just produces a warning "Soft cast of zero to int(2..) is a noop" and the same exception. ChrisA

1 0

Type checking of exponentiation
by Chris Angelico 09 Aug '20

09 Aug '20

In order to implement an expression evaluator, I have a two-pass compilation that first uses a magic resolver to find external name references, and then compiles the actual code to get a function back. This cut-down version of it showcases a strange phenomenon: //Magic resolver. Any symbol at all can be resolved; it'll come through as 0, but the name //will be retained. Used in the precompilation stage to capture external references. multiset(string) symbols; mixed resolv(string symbol, string fn, object handler) {symbols[symbol] = 1;} void calc(string formula) { //Precompile to get a list of used symbols symbols = (<>); //Note: As of Pike 8.1, p must be retained or the compile() call will be optimized out. program p = compile("mixed _ = " + formula + ";", this); //Compile the formula calculator itself. function f1 = compile(sprintf( "int _(mapping data) {%{int %s = (int)data->%<s;\n%}return %s;}", (array)symbols,formula ))()->_; write("Successfully compiled %O\n", formula); } int main() { calc("STR - 2"); calc("4 ** STR"); calc("4 ** (STR + 2)"); calc("4 ** (STR - 2)"); } Everything works fine (using Pike from master branch, c5decfee5) except the very last one, which complains that soft-casting an mpq object to int isn't valid. Other variants of this issue exhibit themselves as saying that "void" is invalid in some context. Any idea where to look for this? ChrisA

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel August 2020