Pike now (8.1) has a decent promise/future subsystem. It's always had great handling of multiple asynchronous operations (GUI, socket, time delay, etc) with the convenience of just returning -1 from main. Interested in people's opinions on whether it would be of value to introduce generators and (built on them) asynchronous functions.
I'm borrowing terminology heavily from Python and ECMAScript here, and both of them have drawn from prior art.
A *generator function* is a function that can be partially run, then set aside until resumed. Effectively, its stack frame can be encapsulated in an object for future use. It would look something like this:
int get_numbers() { write("Starting\n"); yield 1; write("Continuing\n"); yield 2; write("Finishing\n"); return 3; }
Calling get_numbers would prepare an execution context, but not actually begin running the function at all. The caller would get back a *generator object*. Asking this generator object to produce its next value would begin executing the function, triggering the call to write, and would stop at the first yield point, returning that value (1). Asking for another value resumes the function, triggering the second write, and returns the next yielded value. The third time you request a value, it would run the function to completion, and report that it returned (not yielded) 3.
(The types yielded don't have to match the type of the return value. I'm not sure if there's a good way to declare the yielded types, or if it's best to just assume 'mixed'.)
A generator object would be an Iterator, allowing you to loop over the yielded values easily:
foreach (get_numbers() ;; int num)
The 'yield' keyword would be an expression/operator, allowing you to send a value back into the generator function as the result of the yield expression.
The awesomeness of this becomes more apparent when combined with Concurrent.Future objects. An asynchronous function would be a generator which yields Futures until it is ready to return a value. Calling such a function automatically returns a ready-to-go Future, set up such that it is automatically transform_with'd the next yielded Future, or transform'd into the returned value. It would be used somewhat thus:
void show_channel_info(string name) { int id = yield get_channel_id(name); mapping data = yield query_channel(id); write("Status: %s\n", data->status); mapping stream = yield get_stream_info(id); if (stream) write("%s is online, %s\n", name, stream->uptime); else write("%s is offline.\n", name); }
This is WAY more readable than the equivalent with a bunch of lambda functions to capture the intermediate values.
Thoughts? Concerns? Questions?
ChrisA
Chris Angelico wrote:
Pike now (8.1) has a decent promise/future subsystem. It's always had great handling of multiple asynchronous operations (GUI, socket, time delay, etc) with the convenience of just returning -1 from main. Interested in people's opinions on whether it would be of value to introduce generators and (built on them) asynchronous functions.
void show_channel_info(string name) { int id = yield get_channel_id(name); mapping data = yield query_channel(id); write("Status: %s\n", data->status); mapping stream = yield get_stream_info(id); if (stream) write("%s is online, %s\n", name, stream->uptime); else write("%s is offline.\n", name);
This is WAY more readable than the equivalent with a bunch of lambda functions to capture the intermediate values.
Some "random" thoughts on this: - I've considered the readability aspect, and I fully agree that having async functions with await/yield functionality is a boost to readability; and thus it is desirable to have the ability to use it in Pike. - I have not thought through your generator description; a cursory reading seems to indicate that has some desirable features. - I see that in your example you use a "yield" keyword. Is there a reason to not use the JavaScript compatible "await" instead?
On Sun, Aug 11, 2019 at 8:37 AM Stephen R. van den Berg srb@cuci.nl wrote:
Chris Angelico wrote:
Pike now (8.1) has a decent promise/future subsystem. It's always had great handling of multiple asynchronous operations (GUI, socket, time delay, etc) with the convenience of just returning -1 from main. Interested in people's opinions on whether it would be of value to introduce generators and (built on them) asynchronous functions.
void show_channel_info(string name) { int id = yield get_channel_id(name); mapping data = yield query_channel(id); write("Status: %s\n", data->status); mapping stream = yield get_stream_info(id); if (stream) write("%s is online, %s\n", name, stream->uptime); else write("%s is offline.\n", name);
This is WAY more readable than the equivalent with a bunch of lambda functions to capture the intermediate values.
Some "random" thoughts on this:
- I've considered the readability aspect, and I fully agree that having async functions with await/yield functionality is a boost to readability; and thus it is desirable to have the ability to use it in Pike.
- I have not thought through your generator description; a cursory reading seems to indicate that has some desirable features.
- I see that in your example you use a "yield" keyword. Is there a reason to not use the JavaScript compatible "await" instead?
No particular reason. In describing generators, I used the generator keyword from both Python and JavaScript, which is "yield"; async functions in both languages use "await". For this extremely high level summary of the proposal, either keyword makes sense.
I'm not sure whether we'll actually need both forms; it might be possible to build async functions directly on generators, without the notion of "generator delegation" (in Python, that's spelled "yield from").
The essential features would be:
1) Resumable functions, or a way to encapsulate a stack frame to be resumed later. Has a lot of consequences eg tracebacks.
2) A way to tag a function as "this will return a Concurrent.Future that resolves with its final return value, and it can be set aside for resumption upon resolution of another Future".
The first one is logically spelled "yield", the second one "await". But they could basically just be the same thing.
ChrisA
Pike now (8.1) has a decent promise/future subsystem. It's always had great handling of multiple asynchronous operations (GUI, socket, time delay, etc) with the convenience of just returning -1 from main. Interested in people's opinions on whether it would be of value to introduce generators and (built on them) asynchronous functions.
I'm borrowing terminology heavily from Python and ECMAScript here, and both of them have drawn from prior art.
A *generator function* is a function that can be partially run, then set aside until resumed. Effectively, its stack frame can be encapsulated in an object for future use. It would look something like this:
int get_numbers() { write("Starting\n"); yield 1; write("Continuing\n"); yield 2; write("Finishing\n"); return 3; }
Calling get_numbers would prepare an execution context, but not actually begin running the function at all. The caller would get back a *generator object*. Asking this generator object to produce its next value would begin executing the function, triggering the call to write, and would stop at the first yield point, returning that value (1). Asking for another value resumes the function, triggering the second write, and returns the next yielded value. The third time you request a value, it would run the function to completion, and report that it returned (not yielded) 3.
(The types yielded don't have to match the type of the return value. I'm not sure if there's a good way to declare the yielded types, or if it's best to just assume 'mixed'.)
Not matching the declared type seems like a bad idea.
A generator object would be an Iterator, allowing you to loop over the yielded values easily:
This is a separate issue. Note that iterators have both indices and values.
foreach (get_numbers() ;; int num)
The 'yield' keyword would be an expression/operator, allowing you to send a value back into the generator function as the result of the yield expression.
This seems to be something entirely different??
Anyway, I have a proof of concept implementation of the first use:
--- generator_test.pike ---
continue int foo(int start, int end) { while (start <= end) continue return start++; }
int main(int argc, array(string) argv) { function(:int) bar = foo(4, 8);
int i; do { i = bar(); if (zero_type(i)) break; werror("%d\n", i); } while (1); }
---------------------------
$ ./pike generator_test.pike 4 5 6 7 8 0
Note that the ending 0 is due to the implicit return 0 at the end of Pike functions.
/grubba
pike-devel@lists.lysator.liu.se