Pike-devel June 2017

pike-devel@lists.lysator.liu.se

5 participants
10 discussions

GTK/GDK on OSX
by H William Wellliver III 08 Jan '18

08 Jan '18

I’m running into some build problems on OSX. It seems that the GTK module assumes that if a system is NT that gdkwin.h is used and otherwise gdkx.h is used. Unfortunately, it seems that on some builds of GTK/GDK for OS X, neither of those is correct and that gdkquartz.h should be used. I’m guessing that some configure tests are needed to determine which of the 3 files is actually present. However a quick look at configure.in leaves me unsure how best to include that. Anyone with more expertise in the GTK module’s inner workings care to weigh in? Bill

4 9

Interactive debugging and live data inspection
by Stephen R. van den Berg 22 Nov '17

22 Nov '17

I just checked in a debugging aid I created to be able to do some live data inspection in running Pike programs (I used it to find the cause of a deadlock in pgsql). I'd appreciate feedback on the implementation and usability of Debug.Peek (in Pike 8.1). To get a rough impression of what it does, here's an excerpt out of the documentation: ------------------------------------------------ Class Debug.Peek Description Allows for interactive debugging and live data structure inspection in both single- and multi-threaded programs. Creates an independent background thread that every pollinterval will show a list of running threads. Optionally, a triggersignal can be specified which allows the dump to be triggered by signal. Example: In the program you'd like to inspect, insert the following one-liner: Debug.Peek("/tmp/test.pike"); Then start the program and keep it running. Next you create a /tmp/test.pike with the following content: void create() { werror("Only once per modification of test.pike\n"); } int main() { werror("This will run every iteration\n"); werror("By returning 1 here, we disable the stacktrace dumps\n"); return 0; } void destroy() { werror("destroy() runs just as often as create()\n"); } Whenever you edit /tmp/test.pike, it will automatically reload the file. ------------------------------------------------ -- Stephen.

3 4

Proposal: Create Stdio.Port from systemd-provided socket
by Chris Angelico 22 Nov '17

22 Nov '17

Branch: rosuav/systemd-sockets Currently, you can create a Stdio.Port("stdin") as a means of accepting a socket handed to you as FD 0. However, there's no way to accept a socket given as any other FD, making it impossible to use this for systemd's sockets. Example: $ cat /etc/systemd/system/listendemo.service [Unit] Description=Listen Demo Requires=listendemo.socket [Service] User=rosuav ExecStart=/usr/local/bin/pike /home/rosuav/listendemo.pike WorkingDirectory=/home/rosuav $ cat /etc/systemd/system/listendemo.socket [Socket] ListenStream=123 $ cat listendemo.pike object mainsock = Stdio.Port("systemd", acceptloop); void acceptloop() { while (object sock=mainsock->accept()) { write("Accepted sock: %O\n", sock); sock->write("Stub\n"); sock->close(); } } int main() { write("Main sock: %O\n", mainsock); return -1; } Note that the "User=rosuav" directive means that Pike is run as a non-root user. This spares the application the hassle of managing privileges securely, as it never gets root privileges (assuming, of course, that the only reason it needed root was to bind to a privileged port, which is true for a lot of programs). As with "stdin" mode, the second parameter is optional (blocking vs nonblocking). This won't break on non-systemd systems for two reasons: firstly, all it ever does is query environment variables, and secondly, it does this only if the application requests it explicitly, so you have to actually ask for it. Is this something that would be useful? Feel free to bikeshed the API - it's not in 8.1 yet, so anything can change. ChrisA

3 6

pgsql fixes and improvements are in 8.0 and 8.1
by Stephen R. van den Berg 27 Jun '17

27 Jun '17

I've been running a large regression-suite-test here locally on the pgsql driver. Out came some problems in the case of multiple statements running on the same connection in parallel raising one or more errors. The driver sometimes got out of sync with the database because they did not agree on which of the multiple running statements needed to be canceled early. These issues have been fixed. Regression tests now show zero problems. I'm running both 8.1: 66d6ffe9de148d15a4d52e25f23a66d81e98e0f8 and 8.0: 83a8fde6fd9d74a7f35361390fe4e3c1ad9d77ea on all my production systems as of now. Anyone using PostgreSQL with Pike is invited to test as well. -- Stephen.

1 0

Constants with program with parent dependency
by Arne Goedeke 27 Jun '17

27 Jun '17

Hi all, the following code does not work anymore in pike 8: constant fail = ({ A }); class A { array foo = fail; } As far as I can tell the reason was that previously unfinished programs were not considered to use their parent and so the reference of A in the array ended up being a program instead of the function. The commit which introduced this change in behavior was 86ce26dde736531829bbf1466c0d2504e8973466 My feeling is that the new behavior is actually correct, or is this a regression? Arne

1 0

Major problem with SQLite binding handling
by H. William Welliver III 22 Jun '17

22 Jun '17

I’ve noticed an odd problem a few years ago with using parameter bindings with SQLite and never really took the time to figure out the cause until now. I think it’ a pretty major problem that can be easily fixed, though would cause problems for anyone relying on the (imo) currently broken behavior. Basically, the problem is that when using parameter bindings with strings, the pike glue determines the type of the data based on the width of the string: if it’s a wide string, the parameter is bound as a text, but if it’s 8 bit, it’s bound as a blob. The problem is that text and blob values aren’t evaluated the same, so even though a text value and blob value may be identical byte-wise, they are not equal when doing comparisons and so queries that would seem to match actually don’t. This is compounded by the fact that SQLite has no restrictions on what may be stored in a field (type definitions on a table are merely default suggestions) and SQLite will happily store whatever you want (such as a blob value in a text field). For example, imagine a table m with an integer field id and a char field value: s->query(“INSERT INTO m VALUES(1, 'value a’)); s->query(“INSERT INTO m VALUES(2, :value)”, ([“:value”, “value a”])); s->query(“SELECT * FROM m where value=‘value a’”); The third query will return 1 record instead of the expected 2 because the field value in the second row is stored as a blob and thus does not match a text query. In order to retrieve both records, you must use LIKE or know to cast the value to text using “AS TEXT” in the query, which is obviously a lot less ideal in most cases. If all of your data is stored using the pike glue, and you’re always using parameters, it’s possible never to see the problem, but if not, this causes all kinds of problems because of the need for constantly casting values and possibly using blob literals, which are not human readable when you’re really trying to query for a non-wide string. My suggestion is to change the binding type for 8 bit string parameters to text in order to match the rest of the string handling. It would be good to have a datatype that caused a parameter to be bound as a blob, either a native byte array or some object that wraps an 8 bit string (Sql.BinaryString?) in order to cause the binding to act accordingly. Of course, this causes a compatibility problem for anyone who was using pike binding with bindings previously (is there anyone who us doing this and hasn’t run into this problem?), as any 8-bit strings that weren’t binary data would be stored as blobs, even if they were in a text typed field. These records would need to be re-stored with the proper text type, which could be done with a query to update the table. My sense is that this is the proper thing to do, as blob fields should be reserved for data that’s actually binary data (as opposed to text). Thoughts?

3 3

Pike 8.0.450 beta
by Peter Bortas 19 Jun '17

19 Jun '17

Onl popular demand, Pike 8.0.450 beta: https://pike.lysator.liu.se/pub/pike/beta/8.0.450/Pike-v8.0.450.tar.gz Other builds: https://pike.lysator.liu.se/pub/pike/beta/8.0.450/Pike-v8.0.450-Darwin-15.4… https://pike.lysator.liu.se/pub/pike/beta/8.0.450/Pike-v8.0.450-win32-oldli… This is not a release candidate, just something that includes Concurrent to play with until there is a real release. Changes since Pike 8.0.438 (release 9) ---------------------------------------------------------------------- New Features ------------ o Concurrent The Concurrent module simplify asynchronous code by synchronizing events in different ways. As an example the connection() function shown will repond with Concurrent.Promise object that at some point will represent either a connected socket or a failure. Concurrent.Promise connection(string host, int port) { Stdio.File con = Stdio.File(); Concurrent.Promise p = Concurrent.Promise(); if( !con->async_connect(host, port, lambda(int success) { if(success) p->success(con); else p->failure("Failed to connect to "+host+":"+port+"\n"); })) { p->failure("Failed to open socket.\n"); } return p; } The returned promise could the be used in various ways. // On success, call make_request(con, query). On failure call // werror(msg). connection(host, port) ->on_failure(werror) ->on_success(make_request, query); // On success, call make_request(con, query1) followed by // make_request(resp, query2), where resp is the return value from // make_reqest. connection(host, port) ->then(make_request, werror, query1) ->then(make_request, werror, query2); // Call bridge_ports(con1, con2) when both connections are // established. Concurrent.all(connection(host1, port1), connection(host2, port2)) ->then(bridge_ports, failure); // Call make_request(con) once either of the connections are // established. Concurrent.race(connection(host1, port1), connection(host2, port2)) ->then(make_requet, query); Bug fixes --------- o Search Fixed a race condition when updating the database. o Parser.HTML Fixed a condition where Pike would run out of stack space for large documents. o ADT.Heap Fixed heap corruption when the same object is pushed more than once. It will now be considered as calling adjust(). o Inotify - Addressed an issue where the backend might be stuck in pending indefinitely. - EventStreamMonitor now works with other backends. o Standards.BSON Fixed incorrect encoding/decoding of Binary data. o Stdio.Buffer Fixed a crash when attempting to create a rewind key on a buffer returned by read_buffer(). o mappings Fixed an off by one error in random(mapping) that randomly caused values of type PIKE_T_FREE from the freelist to be exposed to Pike.

1 0

Pike 8.0 release 9 RC1
by Peter Bortas 11 Jun '17

11 Jun '17

Pike 8.0.438 beta/release candidate: https://pike.lysator.liu.se/pub/pike/beta/8.0.438/Pike-v8.0.438.tar.gz Other builds: https://pike.lysator.liu.se/pub/pike/beta/8.0.438/Pike-v8.0.438-Darwin-15.4… https://pike.lysator.liu.se/pub/pike/beta/8.0.438/Pike-v8.0.438-win32-oldli… Changes since Pike 8.0.404 (release 8) ---------------------------------------------------------------------- New Features ------------ o Calendar Updated Calendar to use timezonedata from tzdata2017a. o GL & GLUT Prefer Quartz implementation on systems that have it (OS X). o GTK2 - Add -x objective-c for Darwin. - Support linking against other implementations than X11. o Protocols.HTTP Connecting to HTTP proxies over HTTPS is now supported. o Web.decode_jwk() Support decoding of symmetric (ie HMAC) keys. o Web.encode_jws() and Web.decode_jws() Added simplified API for dealing with JSON Web Signature values. Bug fixes --------- o Runtime - Variant dispatcher Improved string range checking. Fixes mismatches due to te lazy string range (ed string(8bit)) exceeding the typed string range (eg string(7bit)) for narrow strings. o mkpackage Support multiple sub-packages with the same name. Gnu-tar doesn't like being asked to extract the same file multiple times from the same tar-file. It claims that the second file doesn't exist in the archive. This can happen when a sub-package has been replaced with a fixed version. o smartlink Don't override MACOS_DEPLOYMENT_TARGET if it's been specified. This should prevent "compiled against version x but linking against version y" warnings. o Arg & ADT.Struct Survive use by old (ie 8.0.232) pike binaries. Roxen 6.0.130 uses a Pike 8.0.232 binary with current modules. o Backend.PollDeviceBackend Make set_signal_event_callback() optional. This fixes the warning "An expression of type function({ Backend = object(implements _static_modules.Builtin()->defaultBackendClass) } : void) cannot be assigned to a variable of type function({ Backend = object(implements _static_modules.Builtin()->Backend) } : void).". o bool Make sure that 0 is part of the range for the bool type. o Charset Fixed encoding errors for two characters in the GSM-03.38 charset. o Debug Fixed typo in mask in check_callback_chain(). Fixes infinite loop in circularity detection. o Gdbm Support building with gdbc 1.13. Fixes obscure compilation error when building with gdbm 1.13 due to a symbol clash with the symbol "fatal" used as an argument in the macro definition of gdbm_set_errno(). o Mapping Removed extraneous and conflicting prototype. Fixes linking issues on NT. o Protocols.HTTP - Fixed do_async_proxied_method() losing the timeout during TLS negotiation. - Now fails immediately if Query is unable to open a socket, instead of waiting 120 seconds for a time out. - Allow connection reuse also when the server is a hostname. o Thread.Mutex - Perform a thread yield on mutex onlock in an attempt to reduce thread starvation. o Sql.pgsql - Added an extra synchronisation condition, and increased parallelisation to resolve all detected race conditions (which in rare circumstances (many parallel queries on a single filedescriptor) could have resulted in a deadlock). -- Peter Bortas

1 0

Time for a release
by Peter Bortas 06 Jun '17

06 Jun '17

It's the first Tuesday in the month, and thus release time. The last couple of months the changes in stable did not motivate fixing missing CHANGES myself and pushing a release. And there are some checkins without changes, which violates stable policy. So review your stuff and check in CHANGES. -- Peter Bortas

1 0

Protocols.HTTP.Query dns cache
by H. William Welliver III 03 Jun '17

03 Jun '17

I was troubleshooting a problem earlier today and noticed that: a) Synchronous calls in Protocols.HTTP.Query don’t populate the DNS cache and b) The doesn’t seem to be any TTL for entries in either the cache Query or the one in Session. I didn’t remedy a) because I think doing so without having a way to expire records would cause undesirable behavior for hostnames (especially) with multiple addresses. Any cache probably ought to at least respect the TTL on the DNS record as an upper bound. Does anyone have any thoughts about this? Bill

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel June 2017