Utilising the momentum, I whipped out a Concurrent.Promise interface for Sql.Connection in git Pike 8.1.
The included examples in the docs could still use some love. Other than that, I thought this was a reasonable attempt at a sane interface between promises and SQL.
Lame sample code:
int main() { db = Sql.Sql(DBNGURL);
Concurrent.Future q1 = db->promise_query("SELECT :bar::INT b", (["bar":3])); Concurrent.Future q2 = db->promise_query("SELECT :foo::INT f", (["foo":4]));
array all = ({q1, q2});
all->on_success(lambda (Sql.FutureResult okresp) { werror("#########################Ok result %O %O\n", okresp, okresp->data); }); all->on_failure(lambda (Sql.FutureResult okresp) { werror("##########################Failed result %O\n", okresp); }); return -1; }
Promises only work with the default backend running. I have tested this with pgsql only, but it should work with any database backend.
Any critisism, suggestions and/or improvements are welcome.
Just a comment: There was some discussion during the conference about the Promise interfaces. I am not the right person to summarize the specific issues, but the bottom line was that the current APIs probably need some more work. This has nothing to do with your new API since it is merely using Promises. Maybe we can re-boot this discussion here on the list?
Otherwise, I personally really like the Promises APIs and avoid pure callback-based APIs when I have the choice.
Arne
On 11/22/17 16:15, Stephen R. van den Berg wrote:
Utilising the momentum, I whipped out a Concurrent.Promise interface for Sql.Connection in git Pike 8.1.
The included examples in the docs could still use some love. Other than that, I thought this was a reasonable attempt at a sane interface between promises and SQL.
Lame sample code:
int main() { db = Sql.Sql(DBNGURL);
Concurrent.Future q1 = db->promise_query("SELECT :bar::INT b", (["bar":3])); Concurrent.Future q2 = db->promise_query("SELECT :foo::INT f", (["foo":4]));
array all = ({q1, q2});
all->on_success(lambda (Sql.FutureResult okresp) { werror("#########################Ok result %O %O\n", okresp, okresp->data); }); all->on_failure(lambda (Sql.FutureResult okresp) { werror("##########################Failed result %O\n", okresp); }); return -1; }
Promises only work with the default backend running. I have tested this with pgsql only, but it should work with any database backend.
Any critisism, suggestions and/or improvements are welcome.
Specifically for the case with multiple blocking dependencies I would like to see an API like
Concurrent.Promise all = Concurrent.Promise();
all->depend( db->promise_query("SELECT foo") ); all->depend( db->promise_query("SELECT bar") );
all->then(do_ok, do_fail);
Martin Nilsson (Coppermist) @ Pike (-) developers forum wrote:
Specifically for the case with multiple blocking dependencies I would like to see an API like
Concurrent.Promise all = Concurrent.Promise();
all->depend( db->promise_query("SELECT foo") ); all->depend( db->promise_query("SELECT bar") );
all->then(do_ok, do_fail);
Ask and your wishes might be granted ;-).
Just checked into Pike 8.1: depend(), fold() and fold_finish().
The last two have been done with testing yet. The code has been sped up, and will scale now (possibly memory neutral compared to the old, but speed-wise it now is O(1), instead of O(n^2)).
Stephen R. van den Berg wrote:
Stephen R. van den Berg wrote:
Just checked into Pike 8.1: depend(), fold() and fold_finish().
The last two have been done with testing yet.
I tested depend(). Still todo: test fold(), fold_finish() and add documentation.
On second thought, the better interface probably would be:
Concurrent.Promise p = Concurrent.Promise(); Concurrent.Future f;
f = p->depend( future1 ); f = f->depend( future2 ); f->on_success(...);
Wouldn't it?
Stephen R. van den Berg wrote:
On second thought, the better interface probably would be:
Concurrent.Promise p = Concurrent.Promise(); Concurrent.Future f;
f = p->depend( future1 ); f = f->depend( future2 ); f->on_success(...);
Wouldn't it?
I don't see what that gives you.
Also I would like to make this possible:
void depend(Future f); variant Future depend();
That allows you to very easily synchronize across multiple call back based APIs.
Promise p = Promise(); call_out(666, lambda(Future f) { f->success(1); }, p->depend()); call_out(123, lambda(Future f) { f->success(1); }, p->depend()); call_out(42, lambda(Future f) { f->success(1); }, p->depend()); p->on_success(...);
Martin Nilsson (Coppermist) @ Pike (-) developers forum wrote:
Stephen R. van den Berg wrote:
On second thought, the better interface probably would be:
Concurrent.Promise p = Concurrent.Promise(); Concurrent.Future f;
f = p->depend( future1 ); f = f->depend( future2 ); f->on_success(...);
Wouldn't it?
I don't see what that gives you.
It allows for the standard Promise interface which allows you to chain everything.
I.e. in practice, you expect the following to be possible:
Concurrent.Promise()->depend(future1)->depend(future2)->on_success(...);
Also I would like to make this possible:
Promise p = Promise(); call_out(666, lambda(Future f) { f->success(1); }, p->depend()); call_out(123, lambda(Future f) { f->success(1); }, p->depend()); call_out(42, lambda(Future f) { f->success(1); }, p->depend()); p->on_success(...);
My proposed change above easily supports this. I'll look into it now.
Stephen R. van den Berg wrote:
Martin Nilsson (Coppermist) @ Pike (-) developers forum wrote:
Also I would like to make this possible:
Promise p = Promise(); call_out(666, lambda(Future f) { f->success(1); }, p->depend()); call_out(123, lambda(Future f) { f->success(1); }, p->depend()); call_out(42, lambda(Future f) { f->success(1); }, p->depend()); p->on_success(...);
The changes are in. Including testsuite entries.
The interface is slightly different from what you proposed:
Promise p = Promise(); call_out(666, lambda(Promise q) { q->success(1); }, p->depend()); call_out(123, lambda(Promise q) { q->success(1); }, p->depend()); call_out(42, lambda(Promise q) { q->success(1); }, p->depend()); p->on_success(...);
But I think it fits your needs as is.
Stephen R. van den Berg wrote:
Stephen R. van den Berg wrote:
Martin Nilsson (Coppermist) @ Pike (-) developers forum wrote:
Promise p = Promise(); call_out(666, lambda(Future f) { f->success(1); }, p->depend()); call_out(123, lambda(Future f) { f->success(1); }, p->depend()); call_out(42, lambda(Future f) { f->success(1); }, p->depend()); p->on_success(...);
The changes are in. Including testsuite entries.
Promise p = Promise(); call_out(666, lambda(Promise q) { q->success(1); }, p->depend()); call_out(123, lambda(Promise q) { q->success(1); }, p->depend()); call_out(42, lambda(Promise q) { q->success(1); }, p->depend()); p->on_success(...);
Turns out, there was some fundamental flaw in the implementation. I built in another layer now to "materialise" a promise. Now it should work in all circumstances.
After some consideration, I think this might be cleaner:
int main() { db = Sql.Sql(DBNGURL);
Sql.Promise q1 = db->promise_query("SELECT :bar::INT b", (["bar":3])) ->discard_records(0)->min_records(1); Sql.Promise q2 = db->promise_query("SELECT :foo::INT f", (["foo":4])) ->max_records(12)->discard_records(2); Sql.Promise q3 = db->promise_query("SELECT 3");
array all = ({q1, q2, q3})->future();
all->on_success(lambda (Sql.FutureResult okresp) { werror("#########################Ok result %O %O\n", okresp, okresp->data); }); all->on_failure(lambda (Sql.FutureResult okresp) { werror("##########################Failed result %O\n", okresp); }); return -1; }
Notice that I now return a Sql.Promise object, which can be further chained with ->min_records(int min) // Fail when you get less ->max_records(int max) // Fail when you get more ->discard_records(int over) // Do not store more than "over" ->future() // See the future
As far as actual implementations are concerned: I can vouch for the pgsql driver that it starts running the queries in the background in parallel as soon as each of the promise_query()s have been issued.
The other drivers will probably result in the queries running blocking and one at a time. Then again, there is nothing keeping you from using different db connections for each promise.
As far as naming the method is concerned, one could argue if it's supposed to be query_promise() or promise_query(). Any preferences?
Stephen R. van den Berg wrote:
As far as naming the method is concerned, one could argue if it's supposed to be query_promise() or promise_query(). Any preferences?
After fixing some docs, I think that the current promise_query() is best.
On another note, presuming that everyone agrees that this interface is pretty neat to begin with, I'd probably want to revisit the HTTP.Promise interface. There might be more elegant solutions there that mimic the way Sql.Promise works.
pike-devel@lists.lysator.liu.se
-
Arne Goedeke -
Martin Nilsson (Coppermist) ＠ Pike (-) developers forum -
Stephen R. van den Berg