Pike-devel March 2012

pike-devel@lists.lysator.liu.se

4 participants
10 discussions

Sql.pgsql behavior in case of errors
by aldem-pikedev＠aldem.net 21 Aug '14

21 Aug '14

Hi everyone, I am preparing to use Postgres in my app, found the following strange behavior: ---snip--- Sql.Sql db = Sql.Sql("pgsql://localhost", "test", "test", "test"); db->query("BEGIN"); db->query("INSERT INTO requests (SvTrId) VALUES ('lala')"); catch(db->query("INSERT INTO requests (SvTrId) VALUES ('lala')")); db->query("INSERT INTO requests (SvTrId) VALUES ('juju')"); db->query("ROLLBACK"); ---snip--- Database is opened, row with 'lala' is inserted, second query (obviously) fails as SvTrId is the primary key, but row with 'juju' is inserted *and* appears in the database - i.e. the transaction is *committed*! Studying source code show, that once error is encounter, session is reset, i.e. running transaction is rolled back, everything is closed, temporary tables are deleted and session variables are reset as well, and after that, the next statement(s) will be committed immediately, as there is no active transaction. In my opinion, this is not exactly correct behavior when statement fails, as session still may be needed (especially temporary tables, open cursors etc). Im some cases, this is perfectly normal when SQL statement fails, especially in SERIALIZED transaction isolation level, and it would be more practical to leave the decision as of what to do to the application. Yes, this is true that it is impossible to continue executing statements in failed transaction, but temporary tables etc. may still be needed. Also, silent reconnection is not a good idea too - when there are many transactional statements are executed - it would be nice to have an option to modify this behavior. Best regards, /Alexander

5 6

Compiling Pike 7.9 crashes?
by Stephen R. van den Berg 11 Apr '12

11 Apr '12

#### Making dynamic: post_modules/Bz2 #### Making dynamic: post_modules/COM #### Making dynamic: post_modules/CritBit /var/src/roxen/rpike/build/linux-3.1.9-vs2.3.2.5-dirty-x86_64/precompile.sh precompile.pike --api=4 >"/var/src/roxen/rpike/src/post_modules/CritBit/inttree.c" "/var/src/roxen/rpike/src/post_modules/CritBit/inttree.cmod" || { rm "/var/src/roxen/rpike/src/post_modules/CritBit/inttree.c"; exit 1; } precompile: pike /var/src/roxen/rpike/bin/precompile.pike --api=4 /var/src/roxen/rpike/src/post_modules/CritBit/inttree.cmod (method=QQQQQQ) Unsupported API version: 4 (max: 3) precompile: /usr/bin/pike /var/src/roxen/rpike/bin/precompile.pike --api=4 /var/src/roxen/rpike/src/post_modules/CritBit/inttree.cmod (method=QQQQQQ) Unsupported API version: 4 (max: 3) precompile: /var/src/roxen/rpike/build/linux-3.1.9-vs2.3.2.5-dirty-x86_64/pike -DNOT_INSTALLED -DPRECOMPILED_SEARCH_MORE -m/var/src/roxen/rpike/build/linux-3.1.9-vs2.3.2.5-dirty-x86_64/master.pike /var/src/roxen/rpike/bin/precompile.pike --api=4 /var/src/roxen/rpike/src/post_modules/CritBit/inttree.cmod (method=Q) /var/src/roxen/rpike/build/linux-3.1.9-vs2.3.2.5-dirty-x86_64/precompile.sh: line 202: 6078 Segmentation fault $RUNPIKE $SCRIPT "$@" 1>&5 The "Unsupported API version" sounds suspicious. I was/am using Debian-testing Pike7.8 to bootstrap. Anything obvious I'm missing? -- Stephen. Being able to try has no purpose if failing is not an option.

3 6

Pike 7.8 Alpha
by H. William Welliver III 06 Apr '12

06 Apr '12

I've prepared a new 7.8 test release for consideration. Please feel free to download and test. http://pike.ida.liu.se/download/pub/pike/alpha/7.8.664/ The following known problems exist: - Change log is not complete (feel free to contribute) - A compilation problem exists under OSX 10.6/7: pike_threadlib includes mach*.h, which causes an error. Please feel free to get in touch with any comments or concerns. Bill bill(a)welliver.org

4 4

String handling optimisation
by Stephen R. van den Berg 06 Apr '12

06 Apr '12

Does anyone know how often in the code we actually depend on the fact that the same string will be at the same address in memory? Because I'm contemplating an optimisation which would involve making the string duplication avoidance opportunistic instead of mandatory. I.e. something along the lines of all strings shorter than stringmin will always be optimised to a single reference, and all strings above that *might* have more than one reference, but not necessarily do (i.e. they're not fully hashed all the time, to avoid the overhead of rehashing large strings repeatedly when juggling around lots of strings). All the places that depend on same string = same address would need to be patched. Also, to determine stringmin, some profiling of existing apps would be interesting. Is that statistic available for say Roxen, to know the distribution of string length and reference count in a running application? -- Stephen. Sex is like air. It's only a big deal if you can't get any.

7 13

pgsql fixes and other things
by Stephen R. van den Berg 04 Apr '12

04 Apr '12

I just pulled the latest pike version from git. It clearly has been a while, since I've actively participated. I apologise for my extended leave, it was due to some real-life issues that got in the way of things. That is not to say that I didn't stop using Pike or Roxen; quite the opposite, I just didn't have time to keep up on the mailinglists. I got snippets of updates through Marc though, so I wasn't completely ignorant. Anyway, the good news is: I'm back. I just glanced through the mailinglist archives (in my mailbox) and it is quite a bundle to skip through (not started on that yet). I first checked the state of the pgsql module. I see some minor updates/fixes. I have one small and one very important fix waiting here which should get in ASAP (which I have been hunting down for the past three years, but did not, until about 2 weeks ago, result in a SEGV; which finally made it possible to zoom in on it; which, in effect, was the final nudge that I needed to get back on track with the mailinglist here). Hmmm, I just glanced through http://pike.ida.liu.se/development/git/rw-git.xml Is it supposed to be automatic that when you "git commit" it will push it into the main repo? Do I still have access? If not, can I be reinstated? -- Stephen. Being able to try has no purpose if failing is not an option.

4 8

compilation handler for multi-tenant applications: update
by Bill Welliver 02 Apr '12

02 Apr '12

Hi all, A few months ago, started a conversation about what I'll call multi-tenant applications: that is, a single pike process running multiple applications (perhaps including copies of a given application) with isolated program and module spaces. The idea is to provide a similar capability to that provided by Java's ClassLoader API. That is, I'd like to be able to do the following: pike +-----------------|-----------------+ | | thread a1...an thread b1 ... bn compilation handler a compilation handler b loads classes for app1 instance 1 loads classes for app1 instance from location a,b from location a,c (perhaps) uses module foo.bar uses module foo.bar foo.bar != foo.bar though it would perhaps be acceptable if object_program(foo.bar) == object_program(foo.bar) applications would not be aware of each other (a condition of the multi-tenant contract) and thus object identity would not need to be maintained. I've been dabbling with the approach suggested at the Pike conference, which was to use a compilation handler to provide this functionality. I've come tantalizingly close, being able to use an overriden master to provide versions of compile_string() and friends that automatically select the desired compilation handler based on various criteria (such as threads in an application). While a bit clunky, it seems to allow me to control visibility of identifiers in a given application or thread, but there does seem to be a limitation that for me is fatal: the programs and module objects are cached by the master, and therefore any two applications are not truely isolated: they share a common set of modules (and join/dirnodes) and (though less problematic) precompiled programs.Because programs and modules seem to be indexed by filename, I've played around with adding a unique identifier in order to split the cache on a per-handler basis, but this hasn't worked either (programs seem to be loaded as desired, but modules are still problematic.) I've attempted to add storage of these caches in the compilation handler, but it results in extremely odd failures (for example, a given class will be cached as a zero value in the programs mapping, which means the program won't be found, even if it's on disk. As a brute force attempt to prove that the idea can work, I'm thinking about short circuiting the auto-reload functionality so that it always reloads a given class from disk. I'm not sure that this will actually prove beneficial, as modules would still be persistent. As always, any thoughts or suggestions would be welcome. Bill

4 8

Design advice sought: Backend implementation
by Bill Welliver 02 Apr '12

02 Apr '12

I'd like to solicit some design feedback for a new feature that I'd like to contribute. I apologize for the length of this message, but hopefully it contains all of the details needed to get the conversation started. First, some quick background: Over the past few years I've written a few modules[1] that provide access to a number of technologies that are either part of Darwin or one of Apple's Darwin-based OSes (OSX, iOS, etc). A common feature that keeps occurring is their reliance on an active run loop. Normally, this is either NSRunloop or CFRunLoop (they are distinct APIs, but they share a common set of event sources.) In the past, I've resorted to adding a persistent call-out to the default back end, which is similar to the approach taken by the GTK2 module. This is normally good enough for sitatuions where events are relatively infrequent, such as GUI-input work. However, it's not always suitable for higher interrupt volumes and it also has the side effect of presenting a continual, though low, load on the CPU. This isn't a huge problem, but it's unclean and if one were to increase the frequency of the poll-out to the CF/NS run loop, the load becomes higher, and thus harder to ignore. A Solution: After doing some research on better ways to approach the problem, I discovered that CF/NS run loops use kqueue at their core. As a result, a kqueue can be added to the run loop and polled for events. That's convenient, because pike uses kqueue on Darwin in its default backend, Pike.PollDeviceBackend. Swapping out the direct call to kevent() with one that calls the run loop results in "native" Pike I/O working the way it should, plus Darwin layered functionality working properly as well (and low CPU utilization, too). I think that this is a useful new feature to add to Pike, but I've got some concerns about how best to include it. Right now, I've created a branch called hww3/cfrunloop_backend[2] that includes the functionality and swaps out the existing implementation under Darwin. That is, of course, the simplest approach, but I think is probably not the best idea: Pros: - Uses kqueue() at its core, so the existing backend will fire events in the same way. - Allows a number of technologies to function normally when used from with Pike processes, without the use of hacks. Cons: - There is exactly one Run Loop per thread, which, if used universally, could present issues because this means behavior would differ from other Backends. - There is slightly more overhead per trip through the backend due to events being single-shot. This means that the kqueue has to be re-added to the runloop each loop. I don't think it's a major overhead, it still exists. Alternate Implementations: A second approach would be to provide an additional backend implementation, say Pike.CFBackend, that's available on Darwin based systems. I see one or two problems with this, though: - There would need to be a way to activiate this selectively at runtime, something like set_default_backend(Pike.CFBackend()), which probably doesn't exist right now, right? - Because 90% of the implementation would be shared with PollDeviceBackend, I think it would involve a fair amount of code duplication inside of backend.cmod, which I think would make maintenance more difficult. A third (and right now, I believe the best) approach would be to add one or two methods to Pike.Backend that could be implemented by PollDeviceBackend on Darwin and be used to activate this alternate mode. For example: int(0..1) has_client_mode() which would return true if there were a client optimization (I suggest client because CFRunLoop would be more useful for client apps than traditional server based apps, though that's certainly not an either/or situaiton) and: void enable_client_mode(int(0..1)) that would trigger calls to CFRunLoopRunInMode() instead of kevent() directly. Input Sought: I think either of the two alternative approaches would be just fine, assuming that the concerns could be addressed. I'm just not sure which would be best from a design and implementation standpoint (I sort of think the last option would be simplest for _me_ to get working, but if someone has desire to help tackle it, I'd be game to play along). The upshot of allowing the proposed backend changes for other folks using Pike are that: a) SDL and GL becomes usable out of the box on OSX b) I am prepared to contribute a few pieces of code, namely: - FSEvents module - Updated Filesystem.Monitor that uses FSEvents on Darwin and inotify on Linux to eliminate polling - DNS_SD querying support on OSX - ObjectiveC bridge (possibly) Hopefully we can have a bit of discussion about the options and approaches and come up with a solution that works without being terribly inconvenient. Please do drop me a note if anything about this proposal is unclear. Best, Bill [1] Public.ObjectiveC, Public.System.FSEvents, Public.IO.IOWarror, additions to DNS-SD; SDL and GL on OSX also share this requirement. [2] http://hg.welliver.org/pike/changesets, not yet pushed to the pike master repo.

5 6

abi and building with bundles
by Bill Welliver 28 Mar '12

28 Mar '12

I noticed a few problems that exist when building Pike using bundles. I'm not very familiar with how that all works, so I thought I'd report them here in the hope that someone more comfortable with making changes to this can take a closer look: 1. If you specify an ABI that isn't the "default" for a given system using --with-abi, the ABI specific compiler flags don't seem to be passed on to any subsequent bundles. Thus, you get all kinds of problems with mismatched abis when trying to link. A temporary workaround seems to be to define CFLAGS and pass them in for the whole build process. 2. A problem specific to Nettle seems to be that there's a small c++ test program that gets built as part of the Nettle bundle. When building for a non-default ABI, this test program can't link against the built library, causing the whole bundle to fail. A temporary workaround seems to be to define CXXFLAGS and pass these flags (such as -m32) as part of the overall build process. Bill

1 0

need permissions help on pelix
by H. William Welliver III 22 Mar '12

22 Mar '12

I'm trying to create some new release folders and unfortunately the permissions seem to be incorrect (I'm in the pike group): $ ls -l /pike/data/ftp/pub/pike/ total 20 -rw-r--r-- 1 nilsson pike 804 Sep 14 2004 README drwxrwsr-x 33 jhs pike 1024 Mar 19 22:12 all drwxr-sr-x 12 peter pike 512 Jan 3 2011 alpha drwxrwsr-x 14 peter pike 512 Apr 16 2011 beta drwxr-sr-x 2 jhs pike 512 Aug 16 2002 contrib drwxr-sr-x 2 jhs pike 512 Aug 2 2002 docs drwxr-sr-x 2 mast pike 1024 Mar 19 22:06 latest-stable drwxr-sr-x 2 jhs pike 512 May 12 2001 libs drwxr-sr-x 2 jhs pike 512 Aug 2 2002 man drwxr-sr-x 4 jhs pike 512 Aug 2 2002 patches Can someone help out with that? Thanks, Bill

2 1

Re: paging Peter Bortas
by Bill Welliver 15 Mar '12

15 Mar '12

I've been a bit behind on my email replies, hopefully this delay can be forgiven... > it shouldn't be any more dificult that making a caudium release. Well, that's a mess of its own that I've been dealing with :) > how about starting with an alpha build. > pick a version and throw it out so people can start testing. Yes, but how? It's more than just doing a "make export", as the bundles have to be included and all that, right? > if there is anything missing in this step of the process, we'll just > find out. > No doubt, but as my time is somewhat limited, I'd rather avoid spending an evening sorting it out if someone else has a set of notes written down. > apart from that i don't think there is all that much to it, other than > writing an announcement and uploading the tarball. (of course there may > be things, and it is probably those you are asking about, but even if > there are, i can't imagine anything that must be done before a version > can be released.) Best, Bill

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel March 2012