Hi,
Having realised the benefits of functional programming, I’ve been quite annoyed by the rumour of how expensive function calls are in Pike. I decided to look into f_map and could see how much seemingly unnecessary work it does when it calls apply_svalue once for each entry in the array – it should be possible to reuse the pike_frame if it’s about to be thrown away after the function call (if it has other refs on the other hand, it can’t be reused – it’s probably used as a scope in some …
[View More]other frame).
I’ve pushed my optimised variant in marty/optimised_map – it seems to work quite well and provides a major speedup. In fact, it’s a bit faster than the corresponding foreach variant. I haven’t verified correctness in various corner cases, and some input on whether it’s correct to do the things init_frame_reuse_context does only once before multiple function calls would be nice too. The *_reuse_context stuff in interpret.c should be applicable wherever the same svalue is applied repeatedly with the same number of arguments (I haven’t looked for it outside of f_map really).
What do you all think? Good idea or did I overlook something?
Without optimisation:
map: 1.660797
array index: 1.335115
array append: 1.17917
With optimisation:
map: 0.877659
array index: 1.351158
array append: 1.189812
Test program:
int main()
{
array base = allocate(10000000, 1);
float gmap = gauge {
array res = map (base, lambda(int i) { return i + 2; });
};
float garrayindex = gauge {
array res = allocate(sizeof(base));
foreach (base; int idx; int i) {
res[i] = i + 2;
}
};
float garrayappend = gauge {
array res = ({});
foreach (base, int i) {
res += ({ i + 2 });
}
};
werror ("map: %O\n", gmap);
werror ("array index: %O\n", garrayindex);
werror ("array append: %O\n", garrayappend);
}
/Marty
[View Less]
On Tue, Mar 10, 2015 at 10:05 PM, Mirar @ Pike developers forum
<10353(a)lyskom.lysator.liu.se> wrote:
> Wait, I said REUSEPORT? What's the difference to REUSEADDR? Mysteries
> of TCP sockets...
Here's a decent explanation, I think:
http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and…
ChrisA
Hi guys!
As awesome a programming language Pike is, the official Pike site does not really reflect that - to say the least ;) The site today does not exacly shout that the language it self is constantly developed and rapidly progressing. I thought I might do something about that. And so I have.
Most things are starting to come together, so in a few weeks I feel we can go live with the new site. But before that happens I would love to have your thoughts and feedback on it. At the moment I have …
[View More]the site on my laptop which is not accessible from outsdie, but I’ve put toghether a page with some screenshots and some explainations which you can checkout.
I have also revamped the module refdoc to better align with the new site. And since the module refdoc is 100% static HTML pages I’ve put a sample of it online on Github.
Page with screenshots and som explainations of the new site:
http://poppa.github.io/pike-dumps/ <http://poppa.github.io/pike-dumps/>
Module refdoc:
http://poppa.github.io/pikedoc/ <http://poppa.github.io/pikedoc/>
So feel free to comment and give feedback. Oh, and since there are some really, really, really, really, really, really old content on the site we need to clean up some old stuff that’s no longer viable. I’ve already done some of that on my local copy, but I’ll put together a list of stuff we need to look into. But lets take that later...
Regards
-----------------------------
Pontus Östlund
Developer • Roxen AB
+46 70-662 81 69
www.roxen.com <http://www.roxen.com/> | twitter.com/roxen <https://twitter.com/roxen>
[View Less]
On Thu, Apr 28, 2016 at 1:40 AM, Henrik Grubbström (Lysator) @ Pike
(-) developers forum <10353(a)lyskom.lysator.liu.se> wrote:
>>Consider this usage:
>>
>>object stdout = Stdio.File();
>>Process.create_process(cmd, (["stdout": stdout->pipe()]);
>>
>>Is this at risk of failing, where stdout->pipe(Stdio.PROP_IPC) would succeed?
>
> Not currently, as the default type of pipe generated by pipe() is
> fd_pipe() (which maps to CreatePipe on …
[View More]WIN32 and is IPC capable).
>
>>Does that imply that PROP_IPC is effectively always implied when
>>creating a pipe?
>
> Not on WIN32.
All currently-supported platforms appear to have IPC capabilities in
the default pipe. Would it be worth changing the default from
PROP_NONBLOCK|PROP_BIDIRECTIONAL to
PROP_NONBLOCK|PROP_BIDIRECTIONAL|PROP_IPC? That wouldn't break any
current code, and would mean that simple calls to stdout->pipe() would
be guaranteed to work (instead of just happening to work).
ChrisA
[View Less]
What exactly does PROP_IPC (fd_INTERPROCESSABLE in the source) permit?
Consider this usage:
object stdout = Stdio.File();
Process.create_process(cmd, (["stdout": stdout->pipe()]);
Is this at risk of failing, where stdout->pipe(Stdio.PROP_IPC) would succeed?
I can't find anything that indicates exactly what "IPC" means, and
whether using one end of a pipe for a process's standard stream counts
as IPC. It seems like it ought to, but the Pike standard library isn't
consistent about …
[View More]requesting it (see Sql.pmod/rsql.pike:36,
Git.pmod/Export.pike:46, and githelper.pike:60 from the companion
repo).
In _Stdio/file.c, file_pipe(), checks are made for a pipe (if
PIPE_CAPABILITIES is defined), a socketpair, or a socketpair_ultra.
All three of these have fd_INTERPROCESSABLE set, so far as I can tell.
Does that imply that PROP_IPC is effectively always implied when
creating a pipe?
ChrisA
[View Less]
On Wed, Apr 6, 2016 at 10:25 AM, Peter Bortas @ Pike developers forum
<10353(a)lyskom.lysator.liu.se> wrote:
> Well, the build machine is an XP. I don't mind dropping anything
> before 7. But that means a whole new build environment. Not happening
> this week.
Fair enough. I'd guess that GTK 2.24.30 should run on XP, at least.
I'm just picking this up again after a flat-out week of musical
theatre, and am finding this:
http://gtk-win.sourceforge.net/home/index.php/Main/…
[View More]Downloads
The other option would be compiling GTK itself from source:
http://ftp.gnome.org/pub/gnome/sources/gtk+/2.24/
ISTR the last time I tried to build Pike on Windows was a bit of a
disaster. Sadly, there is nothing simple about cross-compilation. It'd
be really awesome if the Windows 10 Linux subsystem makes that easier.
ChrisA
[View Less]
Since Arne has been active on his faster_calls branch I switched the
experiment stream on Coverity Scan to have a look at it. These are the
new issues if found when going from marty/optimised_map to
arne/faster_calls. Without looking closer I suspect the four in Image
isn't related to the optimizations code, but the two in interpreter
probably are.
If you want to have a look at them follow the link at the bottom.
Gives a better overview than the condensed ASCII in this mail.
--
Peter Bortas
…
[View More]
---------- Forwarded message ----------
From: <scan-admin(a)coverity.com>
Date: Sun, Apr 17, 2016 at 6:48 PM
Subject: [Pike-automation] New Defects reported by Coverity Scan for
Pike-experiment
To: pike-automation(a)lists.lysator.liu.se
Hi,
Please find the latest report on new defect(s) introduced to
Pike-experiment found with Coverity Scan.
6 new defect(s) introduced to Pike-experiment found with Coverity Scan.
1 defect(s), reported by Coverity Scan earlier, were marked fixed in
the recent build analyzed by Coverity Scan.
New defect(s) Reported-by: Coverity Scan
Showing 6 of 6 defect(s)
** CID 1358595: Incorrect expression (UNUSED_VALUE)
/home/covscan/pike/pike-git/src/interpret.c: 2202 in unlink_previous_frame()
________________________________________________________________________________________________________
*** CID 1358595: Incorrect expression (UNUSED_VALUE)
/home/covscan/pike/pike-git/src/interpret.c: 2202 in unlink_previous_frame()
2196 */
2197 current->save_sp=prev->save_sp;
2198 current->save_mark_sp=prev->save_mark_sp;
2199 current->flags = prev->flags;
2200
2201 /* Unlink the top frame temporarily. */
>>> CID 1358595: Incorrect expression (UNUSED_VALUE)
>>> Assigning value from "prev" to "Pike_interpreter_pointer->frame_pointer" here, but that stored value is overwritten before it can be used.
2202 Pike_interpreter.frame_pointer=prev;
2203
2204 #ifdef PROFILING
2205 {
2206 /* We must update the profiling info of the previous frame
2207 * to account for that the current frame has gone away.
** CID 1358594: Control flow issues (MISSING_BREAK)
/home/covscan/pike/pike-git/src/interpret.c: 1956 in do_trace_frame_call()
________________________________________________________________________________________________________
*** CID 1358594: Control flow issues (MISSING_BREAK)
/home/covscan/pike/pike-git/src/interpret.c: 1956 in do_trace_frame_call()
1950 safe_describe_svalue(&obj_sval, 0, NULL);
1951 obj_name = complex_free_buf(&save_buf);
1952 PIKE_FN_START(function->name->size_shift == 0 ?
1953 function->name->str : "[widestring fn name]",
1954 obj_name.str);
1955 }
>>> CID 1358594: Control flow issues (MISSING_BREAK)
>>> The above case falls through to this one.
1956 case FRAME_CLONE:
1957 case FRAME_PARENT_CLONE:
1958 if (UNLIKELY(Pike_interpreter.trace_level)) {
1959 struct svalue tmp;
1960 struct program *p = frame->ptr;
1961 dynamic_buffer save_buf;
** CID 1358593: Null pointer dereferences (FORWARD_NULL)
/home/covscan/pike/pike-git/src/modules/Image/match.h: 202 in
image_match_phase()
________________________________________________________________________________________________________
*** CID 1358593: Null pointer dereferences (FORWARD_NULL)
/home/covscan/pike/pike-git/src/modules/Image/match.h: 202 in
image_match_phase()
196 DOUBLE_LOOP(0,1,1, *1, *1, *1)
197 else if (type==2)
198 DOUBLE_LOOP(0, haystack_certi[j],
needle_certi[ny*nxs+nx],.r,.g,.b)
199 else if (type==3)
200 DOUBLE_LOOP(AVOID_IS_TOO_BIG,1,1, *1, *1, *1)
201 else if (type==4)
>>> CID 1358593: Null pointer dereferences (FORWARD_NULL)
>>> Dereferencing null pointer "haystack_certi".
202 DOUBLE_LOOP(AVOID_IS_TOO_BIG, haystack_certi[j],
needle_certi[ny*nxs+nx],.r,.g,.b)
203
204 #undef NORMCODE
205 #undef AVOID_IS_TOO_BIG
206 #undef PIXEL_VALUE_DISTANCE
207 #undef DOUBLE_LOOP
** CID 1358592: Null pointer dereferences (FORWARD_NULL)
/home/covscan/pike/pike-git/src/modules/Image/match.h: 202 in
image_match_norm_corr()
________________________________________________________________________________________________________
*** CID 1358592: Null pointer dereferences (FORWARD_NULL)
/home/covscan/pike/pike-git/src/modules/Image/match.h: 202 in
image_match_norm_corr()
196 DOUBLE_LOOP(0,1,1, *1, *1, *1)
197 else if (type==2)
198 DOUBLE_LOOP(0, haystack_certi[j],
needle_certi[ny*nxs+nx],.r,.g,.b)
199 else if (type==3)
200 DOUBLE_LOOP(AVOID_IS_TOO_BIG,1,1, *1, *1, *1)
201 else if (type==4)
>>> CID 1358592: Null pointer dereferences (FORWARD_NULL)
>>> Dereferencing null pointer "haystack_certi".
202 DOUBLE_LOOP(AVOID_IS_TOO_BIG, haystack_certi[j],
needle_certi[ny*nxs+nx],.r,.g,.b)
203
204 #undef NORMCODE
205 #undef AVOID_IS_TOO_BIG
206 #undef PIXEL_VALUE_DISTANCE
207 #undef DOUBLE_LOOP
** CID 1358591: Null pointer dereferences (FORWARD_NULL)
/home/covscan/pike/pike-git/src/modules/Image/match.h: 202 in image_match_norm()
________________________________________________________________________________________________________
*** CID 1358591: Null pointer dereferences (FORWARD_NULL)
/home/covscan/pike/pike-git/src/modules/Image/match.h: 202 in image_match_norm()
196 DOUBLE_LOOP(0,1,1, *1, *1, *1)
197 else if (type==2)
198 DOUBLE_LOOP(0, haystack_certi[j],
needle_certi[ny*nxs+nx],.r,.g,.b)
199 else if (type==3)
200 DOUBLE_LOOP(AVOID_IS_TOO_BIG,1,1, *1, *1, *1)
201 else if (type==4)
>>> CID 1358591: Null pointer dereferences (FORWARD_NULL)
>>> Dereferencing null pointer "haystack_certi".
202 DOUBLE_LOOP(AVOID_IS_TOO_BIG, haystack_certi[j],
needle_certi[ny*nxs+nx],.r,.g,.b)
203
204 #undef NORMCODE
205 #undef AVOID_IS_TOO_BIG
206 #undef PIXEL_VALUE_DISTANCE
207 #undef DOUBLE_LOOP
** CID 1358590: Null pointer dereferences (FORWARD_NULL)
/home/covscan/pike/pike-git/src/modules/Image/match.h: 202 in image_match()
________________________________________________________________________________________________________
*** CID 1358590: Null pointer dereferences (FORWARD_NULL)
/home/covscan/pike/pike-git/src/modules/Image/match.h: 202 in image_match()
196 DOUBLE_LOOP(0,1,1, *1, *1, *1)
197 else if (type==2)
198 DOUBLE_LOOP(0, haystack_certi[j],
needle_certi[ny*nxs+nx],.r,.g,.b)
199 else if (type==3)
200 DOUBLE_LOOP(AVOID_IS_TOO_BIG,1,1, *1, *1, *1)
201 else if (type==4)
>>> CID 1358590: Null pointer dereferences (FORWARD_NULL)
>>> Dereferencing null pointer "haystack_certi".
202 DOUBLE_LOOP(AVOID_IS_TOO_BIG, haystack_certi[j],
needle_certi[ny*nxs+nx],.r,.g,.b)
203
204 #undef NORMCODE
205 #undef AVOID_IS_TOO_BIG
206 #undef PIXEL_VALUE_DISTANCE
207 #undef DOUBLE_LOOP
________________________________________________________________________________________________________
To view the defects in Coverity Scan visit,
https://scan.coverity.com/projects/pike-experiment?tab=overview
[View Less]
1
0
pike -x benchmark
by Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum
15 Apr '16
15 Apr '16
I noticed that starting the benchmark allocates an enormous amount of
memory. The reason seems to be Foreach{,2,3,4} benchmarks. Each of
these has the line
array const_array = enumerate(10000000);
on the top level. This means that there is an object occupying 160M
(10M * sizeof(svalue)) in each of these _programs_, so it is always
present when the programs are loaded, not just when the benchmarks are
run.
So in short, loading these four benchmarks (pike -x benchmark loads
all the …
[View More]benchmarks) allocates 640M ram. That's a bit steep for my
Raspberry Pi which only has 1G in total... (If I change the size of
the arrays to 3M elements then the benchmarks complete normally.)
Could perhaps each of these arrays be allocated before the respective
benchmark is run, and freed afterwards, instead of being allocated
when the benchmark is loaded and freed never?
[View Less]
On Wed, Apr 6, 2016 at 9:50 AM, Peter Bortas @ Pike developers forum
<10353(a)lyskom.lysator.liu.se> wrote:
>>I can't recommend this to my users while it ships GTK 2.12.11 :( Is
>>there anything I can do to help out there?
>
> Well. GTK+ used to be available as an all-in-one package for Windows
> with a readme file saying
>
> "This is a bundle containing the GTK+ stack and its dependencies for
> Windows. As such it is nothing more than a combination of the
…
[View More]> individual packages it consists of. For a list, see
> share/doc/gtk+-bundle-2.12.11-20080707/components.lst. Both run-time
> and developer packages are included."
>
> I _think_ that is discontinued. If you can figure out a place to get a
> working build environment for compiling with Visual Studio that would
> help a bit. If you can find a place to get something that works with
> VC9 it would help even more.
According to this page [1], the current versions require Vista or
later. Not sure whether that means GTK3 or GTK2 though. What Windows
versions is Pike aiming to support?
ChrisA
[1] http://www.gtk.org/download/windows.php
[View Less]
No final release today. The disk in my virtual Windows machine broke.
Fixing that took up time that should have been used for testing, so
you will only get some completely untested builds.
Pike 8.0.222 release candidate:
https://pike.lysator.liu.se/pub/pike/beta/8.0.222/Pike-v8.0.222.tar.gz
Windows build:
https://pike.lysator.liu.se/pub/pike/beta/8.0.222/Pike-v8.0.222-win32-oldli…
Changes since Pike 8.0.182 (release 2)
--------------------------------------------------------------…
[View More]--------
New features
------------
o Roxen.http_decode_string() now decodes UTF-16 surrogate pairs.
o Protocols.HTTP.Query now always sends a content-length header.
o Report feature flags for System.FSEvents and System.Inotify.
Bug fixes
---------
o The machine code generator on ia32 now aligns the stack
on a 16 byte boundary. This makes it possible to call
code that uses SSE2 instructions.
o Exceptions thrown in GTK signal handlers no longer cause
segfaults.
o The SSL client code now interprets the RFCs more leniently.
o Multiple fixes for the handling of accelerated
Filesystem.Monitor monitors, which could sometimes
lose notifications.
o Improved handling of response content-length in Protocols.HTTP.Query.
o Fixed case where unfinished programs could lose their parents.
o Fixed "Invalid service"-error from connect() on Fedora Core 4.
o Fixed next_result() in Sql.sql_array_result.
o Fixed sizeof() in Array.Iterator, Multiset.Iterator and
String.Iterator.
Building and installing
-----------------------
o Multiple fixes for building on Solaris.
o Improved scanning for Odbc header files.
o Fixed some issues with --with-abi flags not propagating
correctly to the modules.
--
Peter Bortas
[View Less]