Pike-devel

pike-devel@lists.lysator.liu.se

2 participants
3049 discussions

Concurrent.Promise() and the executor
by Tobias S. Josefowitz 26 Oct '20

26 Oct '20

I have just noticed that that Concurrent.Promise()->create() just plainly calls the passed executor with success and failure as callbacks/arguments. It is my understanding that the whole purpose of the executor is so that any errors thrown (intentionally or inadvertently) can be caught and converted to a rejection of the promise. If not for that (which is handy), I cannot see any purpose or justification for the executor. I think this qualifies as a bug that should just be fixed. I have pushed a commit to that extent to master. I think this should also go onto 8.0. Objections? Agreement all around? However, this opens up another issue, if rejecting with the error caught results in double finalisation, the error thrown about that (i.e. out of Promise()->create() from the callers perspective) will only have very limited hints about the "original" error thrown (it is present in the backtrace as argument to [-2], e.g. failure(), as the original error object/array, but visibility in prints is severly limited). I wonder if doing something along the lines of diff --git a/lib/modules/Concurrent.pmod b/lib/modules/Concurrent.pmod index dbbd509334..01e1d19e98 100644 --- a/lib/modules/Concurrent.pmod +++ b/lib/modules/Concurrent.pmod @@ -776,15 +776,27 @@ class Promise if (executor) { mixed err = catch(executor(success, failure, @extra)); - // This unfortunately does hide the real error in case of - // double-finalization, i.e. - // fail("I reject!"); - // error("I rejected.\n"); - // in the executor will leave our caller with an error about the - // Promise already having been finalized, not giving anyone too much - // information about where the double-finalization occured... - if (err) - failure(err); + if (err) { + // If rejecting the promise fails due to double-finalization, we + // outright lie about the backtrace here so that it includes the + // position where the "original" error is double thrown, which + // probably makes most related debugging easier but maybe some very, + // very hard... + if (mixed err2 = catch(failure(err))) { + mixed bt; + + catch(bt = err[1]); + + catch { + if (arrayp(err2) && arrayp(bt) && sizeof(bt) + && Program.inherits(bt[-1], __builtin.backtrace_frame)) { + err2[1] = err[1] + err2[1][<2..]; + } + }; + + throw(err2); + } + } } } TL;DR mangling the backtrace to pretend control flow was: site of error() -> Promise()->create() -> failure() -> finalise() would do more good than harm. I cannot say, on one hand I think this would really help people running into this problem, but at the same time it admittedly is just all kinds of wrong.

1 1

Annotations, some documentation
by H William Welliver 23 Oct '20

23 Oct '20

Hello, Pike friends: I’ve put together a first draft of a document describing annotations as well as a simple use of them outside the Pike “runtime". I’d be very glad if folks would have a look and provide feedback. I’ve not included any material regarding the “proposals” I’ve made in the last few days. Please have a look and let’s have a conversation about where things are versus where they (might) want to be. http://wiki.gotpike.org/PikeDevel/Annotations As always, your comments and suggestions are heartily welcome! Bill

1 0

Re: Thinking about annotations
by H William Welliver 23 Oct '20

23 Oct '20

Sorry… I didn’t include the list… feel free to chime in! > On Oct 22, 2020, at 2:35 PM, william(a)welliver.org wrote: > > On 2020-10-22 05:45, Henrik Grubbstrï¿½m (Lysator) @ Pike (-) developers forum wrote: > >>> 1. Iâ€™ve noticed that annotations(something, 1) returns annotations >>> associated not only with each identifier, but also with the class >>> containing the identifier. For example: >> That seems strange; don't you mean >> annotations(a()->gazonk) >> and >> annotations(a()->gazonk, 1) >> respectively above? > > Yes, sorry, I was trying to minimize the amount of data I included in the email and didn't update the example code. > >>> I donâ€™t think I would have expected that behaviorâ€¦ I think Iâ€™d have >>> been more likely to expect some sort of result that described >>> annotations on the identifier of the present class versus >>> annotations on the same identifier from any inherited class. I was >>> just wondering if there was a particular use case where that might >>> be handy? >> The use case that I had in mind (as far as I recall) was using >> annotations for serialization metadata. > > Ok, I guess I need to think about that a little more; I'm not sure I completely understand how that would be used. I might omit discussion of that flag until I know what I'm talking about ;) > >>> 2. I added Program.annotations() in order to query the annotations >>> directly made on a program. Iâ€™m not sure that the additional flag >>> argument works in a reasonable way: if set, >>> Program.annotations(prog) displays the annotations for the entire >>> inherit tree, whereas without the flag, only the annotations defined >>> directly in the present program are displayed. That seems backward >>> to me, but I did it that way to keep the â€œnatureâ€ of the behavior >>> similar to annotations(), but it think that should be reversed in >>> order to limit surprise. Thoughts? >> That does seem a bit backward. > > I'll switch the behavior; I agree that the default no-flag behavior should show the roll-up, as that's usually what someone would expect to see. > >>> 3. Annotations currently must be constant expressions >>> ... >> Incorrect. All of the current annotations when used are objects. Eg: >> @Pike.Annotations.Implements(AbstractSimpleNode); >> A Pike-level class can be marked as creating constant objects using >> the "annotation" >> @constant; >> (which sets the PROGRAM_CONSTANT flag in the class). > > Perfect! I was thinking there might be a way to specify this, I just couldn't put my finger on it. > > >>> 4. I wonder if it would be possible to define a syntax (and possibly >>> add placeholder functionality) for being able to annotate function >>> arguments. >>> For example: >>> void foo(string a, @NotNull: string b) { â€¦ } >> NotNull seems to me to be a property of the type rather than the argument. >> I'm currently in the process of reworking the type checker, which >> would fix the issue that 0 is accepted everywhere. > > Yes, that's probably true and perhaps a bad example. Here's one that uses the web request mapping idea I used in the original email (and that has real life analogs in Java and C#): > > @Web.Routing("/search/results", Web.Methods.GET): > string my_request((a)Web.Request.Parameter("search_term"): string param1, > @Web.Request.Parameter("max_items"): int param2) { ... } > > I had been thinking that annotations on an argument might be somehow stored in the type, and that there would be a way to examine them, but I don't have a full proposal for how that would work (hence my suggestion to define a syntax and allow that to be valid as a first step. > > As far as a proposal for the syntax, would having something along the lines of > > annotation_list full_type optional_dot_dot_dot optional_identifier > > Be reasonable for this purpose? > > Bill >

1 0

8.0 release candidate schedule
by H William Welliver 22 Oct '20

22 Oct '20

Hi all- Just wanted to remind folks that might have bug fixes for 8.0: please try to have them committed/cherry-picked in the next week. I plan to begin the process of cutting a release candidate around Nov 1. Thanks! Bill

1 0

Thinking about annotations
by H William Welliver 22 Oct '20

22 Oct '20

I’ve been away from the keyboard for a few months and am slowly getting back to work and I thought I’d celebrate by taking a closer look at the new annotation support that Grubba has put together. My plan is to have a “chapter” of material for a future edition of the pike manual. What follows are a few comments as well some questions (possibly disguising a feature request or two). 1. I’ve noticed that annotations(something, 1) returns annotations associated not only with each identifier, but also with the class containing the identifier. For example: class a { @“foo”; @“bar”: void gazonk() {} } annotations(a()) returns: ({ (<“bar”>) }) While annotations(a(), 1) returns: ({ (<“foo”, “bar”>) }) I don’t think I would have expected that behavior… I think I’d have been more likely to expect some sort of result that described annotations on the identifier of the present class versus annotations on the same identifier from any inherited class. I was just wondering if there was a particular use case where that might be handy? 2. I added Program.annotations() in order to query the annotations directly made on a program. I’m not sure that the additional flag argument works in a reasonable way: if set, Program.annotations(prog) displays the annotations for the entire inherit tree, whereas without the flag, only the annotations defined directly in the present program are displayed. That seems backward to me, but I did it that way to keep the “nature” of the behavior similar to annotations(), but it think that should be reversed in order to limit surprise. Thoughts? 3. Annotations currently must be constant expressions, and that works well for the builtin annotations, but it prevents annotations that take arguments, which is a pretty common use case in other languages. For example: @Web.Routing(“/path/to/method”, Web.Method.GET): void my_method(object request) { … } I looked a little more closely and realized that the built-in annotations are all classes. I was wondering what the benefit of that is versus being objects, the benefit/need for the expression to be constant, and whether it would be feasible to allow for “annotations with arguments” as described above. 4. I wonder if it would be possible to define a syntax (and possibly add placeholder functionality) for being able to annotate function arguments. For example: void foo(string a, @NotNull: string b) { … } I realize there isn’t currently a framework in place for asking about the nature of an individual argument, but I can imagine the feature might be useful, is present in other languages that implement metadata and if we thought it were feasible, might be nice to define the syntax now. Any thoughts? Apologies for all of the questions, I don’t recall all of the discussion over the years about this, and am attempting to map my C#/Java experience with metadata onto pike. Thanks in advance! Bill

2 1

CONCURRENT_DEBUG default state - active or not?
by Chris Angelico 09 Oct '20

09 Oct '20

lib/modules/Concurrent.pmod starts out by defining CONCURRENT_DEBUG to the value 0. This counts as defined, for the purposes of storing backtraces and such. Is that meant to be active by default? At the moment, there are a number of werror lines that are quite noisy; not sure whether they're part of current debugging efforts or if they're meant to be able to be controlled in that way. I added #ifdef guards, but since the file defines it initially, they're still showing up. For normal code, it would probably be better to not have the noisy output, but maybe it's best to hang onto the backtraces? I believe this applies equally to master and 8.0. ChrisA

1 0

Re: Information leakage in httpserver
by Tobias S. Josefowitz 08 Oct '20

08 Oct '20

As you may have noticed I took the liberty of applying the proposed fix recently.

2 1

New 8.0 release?
by H William Welliver 08 Oct '20

08 Oct '20

I’m in the process of getting pike promoted to pkgsrc from pkgsrc-wip was thinking that it’s been a while since we’ve had an 8.0 release. Would anyone object if I started the process of cutting a new release? Perhaps in 30 days? If anyone has special words (instead of my workman-like prose) for something new they’d like added to the release notes, send them my way! Bill

2 1

Removing AggregateState increases the memory footprint of a Promise substantially
by Stephen R. van den Berg 07 Sep '20

07 Sep '20

Looking at: commit 36a7c5f2c3b44f5556fc2a8e05051211b0e3d834 Author: Henrik Grubbstr<C3><B6>m (Grubba) <grubba(a)grubba.org> Date: Wed Aug 26 17:44:39 2020 +0200 Concurrent.Promise: Refactored the dependency handling. I find: - final int _materialised; - final AggregateState _astate; + protected array(mapping(int:mixed)) dependency_results = ({ ([]), ([]) }); + protected int(0..) num_dependencies; + protected int(1bit) started; + protected int(0..) min_failure_threshold; + protected int(-1..) max_failure_threshold; + protected function(mixed, mixed, mixed ... : mixed) fold_fun; + protected array(mixed) extra_fold_args; If you are running a system with a huge number of promises, it thus causes a substantially increased memory footprint. The reason I used the _astate and _materialised variables was because it allows the memory footprint to be substantially lower after the promise has been setup. In systems with lots of long-lived promises running around, the memory footprint of a single promise seems quite relevant. -- Stephen.

1 1

Pikefarm hang
by Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum 01 Sep '20

01 Sep '20

Hi. For Pikefarm participants information: Pikefarm build 5559 of Pike master seems to hang spending 100% CPU during the testsuite, and may need to be killed by hand. Sorry for the inconvenience. /grubba

1 0

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel