Pike-devel

pike-devel@lists.lysator.liu.se

2 participants
3046 discussions

Re: new block allocator
by Tobias S. Josefowitz 22 Apr '12

22 Apr '12

On Mon, Apr 16, 2012 at 12:30 AM, Martin Stjernholm, Roxen IS @ Pike developers forum <10353(a)lyskom.lysator.liu.se> wrote: > Just to make it clear, you're talking about the linear lookup to find > the right block when structs are freed? Yes. >> No, std::allocator is a block allocator used by std::list, std::map >> and so on. Compared to glibc free/malloc, the std::allocator is quite >> fast. > > It would be interesting to see a comparison with plain old (glibc) > malloc, just to put it into perspective. There is updated graphs on https://github.com/arneg/GJAlloc/wiki/Performance-comparison. Turns out, in fact, I misattributed benefits to the std::allocator it does not have. >> Slight ones. For the CritBits, we need the allocator in the object >> storage (block allocators are used tree-locally). > > So the nodes for each critbit tree are allocated within its own blocks? > Sifting through the code just briefly I got the impression the nodes > were allocated from a single global pool, considering that > CB_NODE_ALLOC() doesn't take any context. Did I miss something? And my bad again. Currently the code does not do this, for comparability between the two block allocators as used by CritBits, I think. >> http://fortranland.com/compare_7.9_block_alloc.txt >> http://fortranland.com/compare_7.9_block_alloc_cleanup_on_exit.txt > > Hmm, do you have any theories to explain the variations seen in those > two? Currently, no explanation that would make you go "Ah ok, in that case, sure." is available. Especially not any based on facts :). > Otherwise it seems to me like there's still a lot of noise, and I > suspect none of those tests really stress BLOCK_ALLOC in the way where > your implementation would show its strength, i.e. freeing structs when > there's a large number allocated. > > I figured I could add a test for that, but I can't compile CritBit from > clean source on the arne/block_alloc branch: > > #### Making dynamic: post_modules/CritBit > make[5]: *** No rule to make target `/home/mast/Pike/devel/src/post_modules/CritBit/bitvector.h', needed by `inttree.o'. Stop. > make[4]: *** [all] Error 2 > make[3]: *** [CritBit] Error 1 That seems to be either an issue with dependencies, or you need to touch src/post_modules/CritBit/*.cmod, or both. In arne/block_alloc, bitvector.h was moved from src/post_modules/CritBit/ to src/ - any chance, your src/post_modules/CritBit/dependencies has size > 0? > And of course, a couple of CritBit benchmarks would be welcome - they > also ought to show off GJAlloc, right? ;) That can be arranged.

3 7

Re: new block allocator
by Tobias S. Josefowitz 22 Apr '12

22 Apr '12

On Tue, Apr 3, 2012 at 12:35 AM, Martin Stjernholm, Roxen IS @ Pike developers forum <10353(a)lyskom.lysator.liu.se> wrote: >> We pushed an experimental branch which uses a new block allocator we >> have been developing. /.../ > > I guess you already told us at the conference, but what was your > motivation for creating it? The linear lookup in the existing one... > > Is std::allocator the same thing as the (g)libc malloc in your graphs? No, std::allocator is a block allocator used by std::list, std::map and so on. Compared to glibc free/malloc, the std::allocator is quite fast. > I take it you already got it working as a replacement for the old > BLOCK_ALLOC, including that quirky heap walker that the old one > sports? Do you have any test where they are compared? We used pike -x benchmark (because results there jitter a lot we averaged ~160 runs, which seems to give useful precision). I will dig up some results from that. However, -x benchmark may not at all be what should be used for meaningful comparison, but unfortunately we did not come up with anything better yet. > >> /.../ Currently, GJAlloc is requested as a bundle by the CritBit >> module, which uses its API directly. > > Does it add any features? Slight ones. For the CritBits, we need the allocator in the object storage (block allocators are used tree-locally). Other than that, GJAlloc supports ba_free_all() and ba_destroy(). > >> All other modules use macro definitions which are compatible with the >> ones from block_allocator.h. > > You mean the old block_alloc.h? Yes, that we meant. best, tobi

4 6

Re: new block allocator
by Tobias S. Josefowitz 22 Apr '12

22 Apr '12

> I'd rather have it directly in the source than as a bundle, but I > guess it's up to you really. We do not want to keep anyone from working on it, of course. Just the idea of releasing this also "standalone" as a block allocator package seemed interesting, and still is. The current GJAlloc even includes some C++ code so that GJAlloc can be used instead of std::allocator and so on, where applicable. All of this (or at least the parts concerning Pike) certainly could be managed in the Pike tree and then from time to time made into a .tar.gz release of the allocator. However, what if we moved the allocator development git repository right next to the Pike repository, same ACLs as pike.git? We would also give out access to the github repository readily, of course, but that does not seem like a particularly smart process. In any case, suggestions on how to achieve a good process for both Pike and standalone releases are more than welcome.

3 2

Re: Pike 7.8 Beta
by Oleksii Prudkyi 19 Apr '12

19 Apr '12

18.04.12 21:35, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum написав(ла): > Built and verified ok on Ubuntu/ppc, Ubuntu/armhf, Debian/amd64 and > Solaris11/sparc. I am checked on OpenSuse i686 and look like it ok. and, two questions with building process: - is here exists easy way to "remap" system to use not-installed pike , i.e. some from ~/apps/Pike-v7.8.680 - is it possible to build pike without pike installed ? Best regards, Oleksii Prudkyi

2 1

mime type update
by Bill Welliver 15 Apr '12

15 Apr '12

Any objections to adding a few newish popular mime types to the MIME extension database? Specifically, I was thinking about the ODT types, but would be glad to add others if someone cares to send them my way. Bill

2 1

map/filter, mapping index/values
by Lance Dillon 12 Apr '12

12 Apr '12

I'm playing with map and filter and stuff, trying to do a functional way of removing an index from a map, like this: ({ ([ "name": "user1", "val": "test", "random": "other" )], ([ "name": "user2", "val": "another test", "random": "something else" ]), }) The array and mapping will be in that kind of format. name is unique among all the map entries, and all the keys exist with all the maps. I want to change it to something like this: ([ "user1": ([ "val": test", "random": "other" ]), "user2": ([ "val": "another test", "random": "something else" ]) ]) I can get the keys for the new map with array[*]["name"]. But removing that key from the maps and leaving the others is giving me a problem. Filter doesn't quite work because it only gives the value, and I'm not testing on the value, but the key. A foreach would work, but I'm wondering if I can do something like: mkmapping(a[*]["name"],map(a,lambda(mapping m) { filter(......something here If anybody has any ideas that would be great... thanks

2 1

7.8 release note assistance sought
by Bill Welliver 11 Apr '12

11 Apr '12

I've been working through the check-in comments and have about 3/4 of them distilled into meaningful release notes, however I'm getting to the point where I need a little assistance weeding out noise and making sense of what's left. I put together a list of the entries that I haven't added to the release notes yet, and I'd like to give everyone an opportunity to weigh in. The list is at the following url: http://wiki.gotpike.org/ReleaseToDo7-8-664 If you've got something to say about anything left, feel free to put together a sentence or two describing the change. You can either send them to me, or just add them to the bottom of the wiki page. Also, if you know the entry isn't worth writing about, feel free to remove it from the list. Otherwise, I won't put too much more effort into adding the rest of these. Best, Bill

1 0

Compiling Pike 7.9 crashes?
by Stephen R. van den Berg 10 Apr '12

10 Apr '12

#### Making dynamic: post_modules/Bz2 #### Making dynamic: post_modules/COM #### Making dynamic: post_modules/CritBit /var/src/roxen/rpike/build/linux-3.1.9-vs2.3.2.5-dirty-x86_64/precompile.sh precompile.pike --api=4 >"/var/src/roxen/rpike/src/post_modules/CritBit/inttree.c" "/var/src/roxen/rpike/src/post_modules/CritBit/inttree.cmod" || { rm "/var/src/roxen/rpike/src/post_modules/CritBit/inttree.c"; exit 1; } precompile: pike /var/src/roxen/rpike/bin/precompile.pike --api=4 /var/src/roxen/rpike/src/post_modules/CritBit/inttree.cmod (method=QQQQQQ) Unsupported API version: 4 (max: 3) precompile: /usr/bin/pike /var/src/roxen/rpike/bin/precompile.pike --api=4 /var/src/roxen/rpike/src/post_modules/CritBit/inttree.cmod (method=QQQQQQ) Unsupported API version: 4 (max: 3) precompile: /var/src/roxen/rpike/build/linux-3.1.9-vs2.3.2.5-dirty-x86_64/pike -DNOT_INSTALLED -DPRECOMPILED_SEARCH_MORE -m/var/src/roxen/rpike/build/linux-3.1.9-vs2.3.2.5-dirty-x86_64/master.pike /var/src/roxen/rpike/bin/precompile.pike --api=4 /var/src/roxen/rpike/src/post_modules/CritBit/inttree.cmod (method=Q) /var/src/roxen/rpike/build/linux-3.1.9-vs2.3.2.5-dirty-x86_64/precompile.sh: line 202: 6078 Segmentation fault $RUNPIKE $SCRIPT "$@" 1>&5 The "Unsupported API version" sounds suspicious. I was/am using Debian-testing Pike7.8 to bootstrap. Anything obvious I'm missing? -- Stephen. Being able to try has no purpose if failing is not an option.

3 6

Pike 7.8 Alpha
by H. William Welliver III 05 Apr '12

05 Apr '12

I've prepared a new 7.8 test release for consideration. Please feel free to download and test. http://pike.ida.liu.se/download/pub/pike/alpha/7.8.664/ The following known problems exist: - Change log is not complete (feel free to contribute) - A compilation problem exists under OSX 10.6/7: pike_threadlib includes mach*.h, which causes an error. Please feel free to get in touch with any comments or concerns. Bill bill(a)welliver.org

4 4

String handling optimisation
by Stephen R. van den Berg 05 Apr '12

05 Apr '12

Does anyone know how often in the code we actually depend on the fact that the same string will be at the same address in memory? Because I'm contemplating an optimisation which would involve making the string duplication avoidance opportunistic instead of mandatory. I.e. something along the lines of all strings shorter than stringmin will always be optimised to a single reference, and all strings above that *might* have more than one reference, but not necessarily do (i.e. they're not fully hashed all the time, to avoid the overhead of rehashing large strings repeatedly when juggling around lots of strings). All the places that depend on same string = same address would need to be patched. Also, to determine stringmin, some profiling of existing apps would be interesting. Is that statistic available for say Roxen, to know the distribution of string length and reference count in a running application? -- Stephen. Sex is like air. It's only a big deal if you can't get any.

7 13

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel