Dear Pike users and developers,
I wanted to ask whether there is any library implemented for Pike to support concurrency or if it planed to do so as an embedded feature or added package? I think the language has a great potential to be widely used if such enhancements are done. I am reading the book "Pike: an Introduction" and have not seen any concept about concurrency yet (I am not done with the book yet).
Thanks,
Danesh
On Sat, May 3, 2014 at 1:14 AM, Danesh Daroui danesh.daroui@gmail.com wrote:
I wanted to ask whether there is any library implemented for Pike to support concurrency or if it planed to do so as an embedded feature or added package? I think the language has a great potential to be widely used if such enhancements are done. I am reading the book "Pike: an Introduction" and have not seen any concept about concurrency yet (I am not done with the book yet).
Hi!
What exactly do you mean by concurrency? Pike comes with a Thread module, which works very nicely for I/O-bound operations; though if you can get your head around it, single-threaded callback handling works beautifully there.
Chris Angelico
Hi,
Thanks for your answer. I guess threads is something that I was looking for. By concurrency I meant being able to spawn multiple threads which will logically run in parallel. Moreover having the possibility to define mutex and semaphores which are apparently done in Pike (mutex is at least done) and also barrier and join threads, etc.
Regards,
Danesh
On Fri, 02 May 2014 17:21:26 +0200, Chris Angelico rosuav@gmail.com wrote:
On Sat, May 3, 2014 at 1:14 AM, Danesh Daroui danesh.daroui@gmail.com wrote:
I wanted to ask whether there is any library implemented for Pike to support concurrency or if it planed to do so as an embedded feature or added package? I think the language has a great potential to be widely used if such enhancements are done. I am reading the book "Pike: an Introduction" and have not seen any concept about concurrency yet (I am not done with the book yet).
Hi!
What exactly do you mean by concurrency? Pike comes with a Thread module, which works very nicely for I/O-bound operations; though if you can get your head around it, single-threaded callback handling works beautifully there.
Chris Angelico
On Sat, May 3, 2014 at 1:34 AM, Danesh Daroui danesh.daroui@gmail.com wrote:
Thanks for your answer. I guess threads is something that I was looking for. By concurrency I meant being able to spawn multiple threads which will logically run in parallel. Moreover having the possibility to define mutex and semaphores which are apparently done in Pike (mutex is at least done) and also barrier and join threads, etc.
Yep! You should be able to do all that.
Pike's own internals are guaranteed to be thread-safe, so you can happily work with all the refcounted objects from multiple threads. (This will damage "true concurrency", though, as accessing the same object from two threads means they'll lock against each other. In practice, this happens a lot, so you'll probably not be able to use multiple cores to maximum efficiency without some alternative ways of fiddling with things.) I run a MUD by the name of Minstrel Hall (minstrelhall.com) where every connected client has a separate thread; they all run very happily together. One can interact with another and they don't have to worry about treading on each other's toes.
For actual explicit locking between threads, the Thread module provides mutexes. They're a little odd to use at first, if you're used to a more classic mutex system; but it's very handy because you can't accidentally forget to unlock the semaphore (once the MutexKey goes out of scope, the Mutex is automatically unlocked). I've used it occasionally, but it's almost never necessary.
Operations like joining threads are provided as methods on the Thread.Thread object (eg wait()). I'm not sure where barriers would be needed, as I've never actually used them in any language, but I'm sure there'll be a way to implement it.
ChrisA
Hi Chris,
Thank you for your comprehensive answer. Its good to know that Pike supports such important features and I am actually very keen to discover how mutex works in Pike. However, I guess concurrency was not in the main focus when Pike was designed but it is still very good that it supports something like POSIX threads. Maybe it would be handy to add some new features, so loop parallelization or race condition detecting (like cilk add-on for C) would be possible in Pike too but it really depends on the class of applications that Pike is mostly used for.
Thanks,
D.
On Fri, May 2, 2014 at 5:52 PM, Chris Angelico rosuav@gmail.com wrote:
On Sat, May 3, 2014 at 1:34 AM, Danesh Daroui danesh.daroui@gmail.com wrote:
Thanks for your answer. I guess threads is something that I was looking
for.
By concurrency I meant being able to spawn multiple threads which will logically run in parallel. Moreover having the possibility to define mutex and semaphores which are apparently done in Pike (mutex is at least done) and also barrier and join threads, etc.
Yep! You should be able to do all that.
Pike's own internals are guaranteed to be thread-safe, so you can happily work with all the refcounted objects from multiple threads. (This will damage "true concurrency", though, as accessing the same object from two threads means they'll lock against each other. In practice, this happens a lot, so you'll probably not be able to use multiple cores to maximum efficiency without some alternative ways of fiddling with things.) I run a MUD by the name of Minstrel Hall (minstrelhall.com) where every connected client has a separate thread; they all run very happily together. One can interact with another and they don't have to worry about treading on each other's toes.
For actual explicit locking between threads, the Thread module provides mutexes. They're a little odd to use at first, if you're used to a more classic mutex system; but it's very handy because you can't accidentally forget to unlock the semaphore (once the MutexKey goes out of scope, the Mutex is automatically unlocked). I've used it occasionally, but it's almost never necessary.
Operations like joining threads are provided as methods on the Thread.Thread object (eg wait()). I'm not sure where barriers would be needed, as I've never actually used them in any language, but I'm sure there'll be a way to implement it.
ChrisA
Hi Danesh!
On Fri, May 2, 2014 at 5:34 PM, Danesh Daroui danesh.daroui@gmail.com wrote:
Hi,
Thanks for your answer. I guess threads is something that I was looking for. By concurrency I meant being able to spawn multiple threads which will logically run in parallel. Moreover having the possibility to define mutex and semaphores which are apparently done in Pike (mutex is at least done) and also barrier and join threads, etc.
Pike indeed has support for that, as you discovered. However, beware of the global interpreter lock. Pike code will never really run concurrently (i.e. more than one thread executing Pike code at the same time on more than one CPU/core). All concurrency you can get is on blocking I/O operations and possibly some heavier computations offered by the Pike "standard library".
Tobi
Hi Tobias,
Thanks for the info. I thought Pike would run all threads simultaneously when number of threads does not exceed number of available processors and otherwise using time sharing like other languages. I am still learning Pike and hence not familiar with I/O operations in Pike so maybe later will find out when to get benefit of concurrency when Pike is used.
Regards,
Danesh
On Fri, May 2, 2014 at 5:55 PM, Tobias S. Josefowitz <t.josefowitz@gmail.com
wrote:
Hi Danesh!
On Fri, May 2, 2014 at 5:34 PM, Danesh Daroui danesh.daroui@gmail.com wrote:
Hi,
Thanks for your answer. I guess threads is something that I was looking
for.
By concurrency I meant being able to spawn multiple threads which will logically run in parallel. Moreover having the possibility to define mutex and semaphores which are apparently done in Pike (mutex is at least done) and also barrier and join threads, etc.
Pike indeed has support for that, as you discovered. However, beware of the global interpreter lock. Pike code will never really run concurrently (i.e. more than one thread executing Pike code at the same time on more than one CPU/core). All concurrency you can get is on blocking I/O operations and possibly some heavier computations offered by the Pike "standard library".
Tobi
On Sat, May 3, 2014 at 6:07 AM, Danesh Daroui danesh.daroui@gmail.com wrote:
Thanks for the info. I thought Pike would run all threads simultaneously when number of threads does not exceed number of available processors and otherwise using time sharing like other languages. I am still learning Pike and hence not familiar with I/O operations in Pike so maybe later will find out when to get benefit of concurrency when Pike is used.
Here's an example:
void client(Stdio.FILE sock) { sock->write("Enter a number: "); int n=(int)sock->gets(); array(string) strs=allocate(n); for (int i=0;i<n;++i) { sock->write("Enter string #%d: ",i); strs[i]=sock->gets(); } sock->write("Bye!\n"); sock->close(); }
void main() { Stdio.Port mainsock=Stdio.Port(12345); while (object sock=mainsock->accept()) { object buffered=Stdio.FILE(); buffered->assign(sock); Thread.Thread(client,buffered); } }
Each client that connects (TELNET makes a viable client here) gets its own thread, and they effectively do run simultaneously. The threads spend most of their time waiting inside gets(), so the internal locking isn't a problem. As far as you're concerned, each one is running straight through its instance of client(), with its own local variables, without interfering with anything else.
To actually make use of multiple cores, though, there are some tricks available. Easiest one is to start separate processes, rather than threads. I haven't often needed to do this, but it's certainly possible.
ChrisA
Danesh Daroui wrote:
Thanks for the info. I thought Pike would run all threads simultaneously when number of threads does not exceed number of available processors and otherwise using time sharing like other languages.
Well, most popular interpreter-based languages have the same problem as Pike: they do not use more than one CPU core per process (e.g. Perl, Python, Ruby, Javascript).
Martin Sjternholm has played with some techniques which allow true concurrency even in an interpreted environment, but since he has been assimilated by Google (Martin, are you reading this? :-), no progress in that direction has been made (at least not for Pike).
On Mon, May 5, 2014 at 9:21 AM, Stephen R. van den Berg srb@cuci.nl wrote:
Danesh Daroui wrote:
Thanks for the info. I thought Pike would run all threads simultaneously when number of threads does not exceed number of available processors and otherwise using time sharing like other languages.
Well, most popular interpreter-based languages have the same problem as Pike: they do not use more than one CPU core per process (e.g. Perl, Python, Ruby, Javascript).
It's also worth noting that there are several Python interpreters, of which not all use a Global Interpreter Lock; CPython (the most popular one and the one most people think of as "python") does, and there've been periodic calls to rewrite things to not need it. So far, the impetus to remove such a lock has been relatively weak; the advantages of the GIL (simplicity and performance) majorly outweigh the advantages of no-GIL (ability to use multiple cores simultaneously) for most applications.
ChrisA
pike-devel@lists.lysator.liu.se
-
Chris Angelico -
Danesh Daroui -
Stephen R. van den Berg -
Tobias S. Josefowitz