We discussed this a bit during the Pike Conferance. These are my thoughts on it:
On Mon, Oct 17, 2016 at 10:07 PM, Stephen R. van den Berg srb@cuci.nl wrote:
Chris Angelico wrote:
// This leaves stdin and stdout and stderr unaltered Process.pipe.run("fgrep -e test").run("sort").run("wc");
If Pike were a shell language, this would make sense. But I would much
It would make sense, for any programming language, not only for shell languages.
The pipe-syntax seems interesting, though it probably has some corner cases where you could create objects where several parts of the pipe tries to run at the same time.
prefer this:
sizeof(Process.check_output("fgreb -e test") / "\n");
That's one of those places where we would benefit from an easier to use interface for Process.Process. Process.run sucks up memory for problems that doesn't really need it. It should preferably be streamed and/or auto-iterated in more of the manner of:
count( "\n", Process.check_output("fgreb -e test") );
where this in effect becomes a line or character iterator and only keeps at most one line in memory at the time.
Pike isn't primarily about invoking subprocesses; it has a rich set of text processing primitives built-in, so trying to make subprocess chaining smoother is usually a waste of effort.
If it would be a *lot* of effort, I'd agree. But I'm guessing that it actually is easier to implement than you might expect; and then it makes for a very clutterfree and straightforward way to start one or more (piped) processes.
The pipe-syntax and Chris's easy to understand convenience function seem orthogonal with Chris's API easier to grasp. So I'd be glad to see versions of both eventually go in. I'm not overly enthused with continuing the tradition I introduced with Process.run to just buffer everything in memory if it can be avoided though.
Chris: What you have seem generally useful, but it lies in a namespace that will get a bit busy if we implement all the special cases as we think of them. I have similar function not checked in that would confuse users if we both committed. No in the least because I find exceptions useless for most smaller scripts unless I just plan on not catching anything. Which means my scripts are full of very similar code but where the functions return 0 on failure, not throwing, but dumping the failure pretty-printed to the console. Something I've also been planning to Process for a while, but not come up with a set of functions that doesn't make it confusing for users to choose among all the stuff.
As I see it there are a few things that should happen in regards to external process spawning:
1. Can we come up with an almost as easy to use API as Process.run that plays better with memory and latency?
The stupidest version of that would be to have the call where you specify the number of bytes or lines to read and then hands back the result together with a function to call for the next portion. That's real stupid, but still preferable to playing with Process() directly and risk a lock-up because you didn't handle your pipes perfectly.
2. Make more convenience-APIs
I count both Stephens pipes and Chris easy-call APIs to this.
(1) have to be thought about before we do (2), because depending on the solution for (1) or we will end up with duplicate or triplicate APIs for the same thing. I promise to pour some real energy into thinking about this in a few weeks when I have time again, but meanwhile please jump in with what everyone's ideas are about what problems you are trying to solve and how you imagine it solved.
Regards,