Pike-devel

pike-devel@lists.lysator.liu.se

2 participants
3054 discussions

Re: Slow promise awaiting when large amounts of data transmitted
by Chris Angelico 04 May '26

04 May '26

On Mon, 4 May 2026 at 18:16, Henrik Grubbström (Lysator) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > > Could the Promise retain the backtrace as an array of frames, rather > > than describing it immediately and retaining the string? That is also > > fast (since most of the time it doesn't need to display anything), but > > will still produce equivalent output. Or would that result in a > > reference loop? > > The reason for the backtrace being saved as a string is indeed to > avoid reference issues. One possibility could be to truncate the > backtrace to just a few frames. > I wonder if it would be reasonable to have a cut-down backtrace display that doesn't show the contents of mappings or arrays, so the function arguments would be shown as some_function(([...])) instead of the full details. That still would keep the most relevant parts of the traceback, and could retain strings and numbers, but would prevent unbounded growth. ChrisA

2 1

Re: Slow promise awaiting when large amounts of data transmitted
by Chris Angelico 04 May '26

04 May '26

On Mon, 4 May 2026 at 16:30, Henrik Grubbström (Lysator) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > > > Having a large blob of data returned from an await() results in all > > subsequent awaits to slow down. Example: > > > > __async__ void reconnect() { > > mapping commands = await(load_commands()); //Returns a big array > > //werror("This should be a one: %O\n", > > await(Concurrent.resolve(1))); //Returns just an integer > > foreach (commands; string uuid; mapping data) do_stuff_with(uuid, data); > > } > > Wow! Somebody (other than me) is already using __async__ and await()! I sure am! I love it and use it in all my projects now, pretty much. My main Twitch bot (Mustard Mine) switched over to that back when they were "continue functions" and without a moment's hesitation I moved to proper async/await. > Concurrent.Promise et al should be agnostic with respect to the size > of data that is passed. As you have noted though, due to it saving > the backtrace at time of creation, the size of the backtrace will > affect timing. > > > If do_stuff_with() involves multiple promises/awaits, each one has a > > notional backtrace involving the entire large array. The inclusion of > > the dummy resolve(1) changes this so that the earlier call in the > > backtrace is much smaller. > > Try disabling the orig_backtrace handling (eg initialize it to "" > instead of the backtrace) in Concurrent.pmod, and see if it solves > your issue. Yes, hacking that out does prevent the problem. > > I'm not sure what changed recently, but when I upgraded Pike, this > > started to be slow. However, Concurrent.pmod doesn't seem to have > > changed recently, so I'm not sure what the cause is. (It's also > > possible an unrelated change of my own triggered this.) > > One easy way to reduce the backtrace size is to wrap the large value > in an object when passing it as an argument. I could do that. I'd have to figure out how much of a nuisance that'd be API-wise, since I don't know how much of a problem it's going to be. Notably, this is only really a problem when there are a *lot* of promise backtraces being generated (the code I posted above is simplified, but there's several layers of iteration and calls - this is loading roughly a hundred channel's worth of data, then for each channel, going and setting things up). > > Promises currently seem to retain a text backtrace. I'm not sure if > > that's relevant or not, as it has always been there, but it may be > > connected. > > They have been there for quite a while, albeit they were not enabled > by default initially. They were enabled by default as they were so > useful when debugging problematic Promises. > Ah, interesting. I thought that change happened years ago though, so I didn't think that was the cause. Could the Promise retain the backtrace as an array of frames, rather than describing it immediately and retaining the string? That is also fast (since most of the time it doesn't need to display anything), but will still produce equivalent output. Or would that result in a reference loop? ChrisA

3 3

Slow promise awaiting when large amounts of data transmitted
by Chris Angelico 04 May '26

04 May '26

Having a large blob of data returned from an await() results in all subsequent awaits to slow down. Example: __async__ void reconnect() { mapping commands = await(load_commands()); //Returns a big array //werror("This should be a one: %O\n", await(Concurrent.resolve(1))); //Returns just an integer foreach (commands; string uuid; mapping data) do_stuff_with(uuid, data); } If do_stuff_with() involves multiple promises/awaits, each one has a notional backtrace involving the entire large array. The inclusion of the dummy resolve(1) changes this so that the earlier call in the backtrace is much smaller. I'm not sure what changed recently, but when I upgraded Pike, this started to be slow. However, Concurrent.pmod doesn't seem to have changed recently, so I'm not sure what the cause is. (It's also possible an unrelated change of my own triggered this.) Promises currently seem to retain a text backtrace. I'm not sure if that's relevant or not, as it has always been there, but it may be connected. Any ideas what's going on here? ChrisA

2 1

Re: PKCS#8 private key format
by Chris Angelico 27 Apr '26

27 Apr '26

On Tue, 28 Apr 2026 at 06:54, Henrik Grubbström (Lysator) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > > What would be the consequences of adding support for the PKCS#8 type > > to these two convenience functions? This would be a simple change in > > each of two places - see branch rosuav/pkcs8-key-parsing - but there > > may be other consequences that I'm not aware of. > > Looks good, merged in Pike 9.0 and later. > Oh sweet! Thanks. ChrisA

1 0

PKCS#8 private key format
by Chris Angelico 27 Apr '26

27 Apr '26

Using LetsEncrypt's certbot will by default generate a private key in PKCS#8 format, not the PKCS#1 "Traditional" format. This CAN be used with Pike, but the most obvious way to parse certificates and keys does not work with this. Example: object pem = Standards.PEM.Messages(Stdio.read_file("privkey.pem") + Stdio.read_file("fullchain.pem")); pem->get_private_keys(); //Empty array, no PK found string pk = pem->get_fragments((<"PRIVATE KEY">))[0]->body; //This does return a valid key though object ctx = SSL.Context(); //ctx->add_cert(pk, pem->get_certificates()); //Fails - unable to parse key ctx->add_cert(Standards.PKCS.parse_private_key(pk), pem->get_certificates()); //Succeeds! What would be the consequences of adding support for the PKCS#8 type to these two convenience functions? This would be a simple change in each of two places - see branch rosuav/pkcs8-key-parsing - but there may be other consequences that I'm not aware of. Anyone with more SSL knowledge than me able to weigh in? ChrisA

2 1

pike_frame->fun = FUNCTION_BUILTIN
by Arne Goedeke 16 Apr '26

16 Apr '26

In INIT, EXIT and some gc callbacks the fun in Pike_fp is set to FUNCTION_BUILTIN. there are some places in the code (e.g. ADT.List()->_get_iterator) where ->fun is used in INHERIT_FROM_INT et al, which segfaults (except with PIKE_DEBUG). I am not sure where to fix this. Should the *_FROM_INT macros check for FUNCTION_BUILTIN or is this better to be handled in the individual INIT/EXIT methods ? does anyone object to making ADT.Stack and ADT.Queue inherit ADT.List? arne

3 2

managing comport settings in linux
by Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum 09 Apr '26

09 Apr '26

Not a good idea; the design philosophy of Pike is that Pike-level code should never be able to cause memory corruption at C-level. With ioctls there's no way to know what data a specific ioctl wants without reading the documentation for that specific ioctl for that specific OS. ioctls and fcntls are a way for device handlers to hook in APIs that aren't standard across all filedescriptors. / Henrik Grubbström (Lysator) Previous text: >2004-08-17 13:32: >Subject: managing comport settings in linux >-------------------------------------------------------------------- >What would be nice, is a way to define ioctl methods in >Pike. Perhaps something like > >object i = Stdio.IOCTL(4242 /* request_id */ ); >i->add_int_parameter(); >i->add_string_parameter(); >i->set_return_type(STRING); > >string ret = i(fd, 1, "somestring"); > > >That ought to be possible, or..? > > / Marcus Agehall (PacketFront) >

4 3

Pike Meet-up 2024-Q2
by Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum 09 Apr '26

09 Apr '26

Hi. At Pike Conference 2017 it was decided that we ought to hold a more informal meeting in Linköping ~ once every quarter. As the 6 month mark from the latest conference is coming up soon, it seems time to announce the second such meeting for 2024: Pike Meet-up 2024-Q2 Date: 2024-05-11 (Sat) Time: 10:00 - afternoon/evening Place: Roxen IS AB Storgatan 46 Linköping Welcome! /grubba

1 5

gobject introspection
by Lance Dillon 24 Mar '26

24 Mar '26

Someone, I think rosuav (Chris Angelico), did a branch with gobject introspection, but I can't remember it, and none of the branches in git branch -ar seem like it is, at least the names don't seem like it: origin/rosuav/async-annotations origin/rosuav/deoptimize-continue-return origin/rosuav/enable-werror origin/rosuav/gtk2-drag-drop origin/rosuav/happy-eyeballs origin/rosuav/hilfe-improvements origin/rosuav/http-multi-connect origin/rosuav/inotify-config origin/rosuav/latex-markdown-renderer origin/rosuav/markdown-attribute-syntax origin/rosuav/markdown-parser-extensibility origin/rosuav/naglingcontrol origin/rosuav/pango_xy_to_char origin/rosuav/pgsql-ssl origin/rosuav/pre-listening-ports origin/rosuav/process-check-run origin/rosuav/sockopt origin/rosuav/support-gdk-quartz unless it is support-gdk-quartz. Would someone be able to point me to it? I want to do some testing... Thanks

2 1

Re: Playing music using pike and SDL.
by Martin Nilsson (lvl 60) ＠ Pike (-) developers forum 13 Mar '26

13 Mar '26

Were you not able to get any C code to play through SDL either? Does your SDL library have a mixer compiled in it (I think it is optional)?

5 4

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel