Pike-devel

pike-devel@lists.lysator.liu.se

1 participants
3041 discussions

Re: File descriptor usage in Protocols.HTTP.Server.SSLPort
by Chris Angelico 07 Apr '24

07 Apr '24

On Fri, 5 Apr 2024 at 23:59, Henrik Grubbström (Lysator) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > > Is there a better way to handle this than simply forcing garbage > > collection every request? > > The best way is to break the cycle before dropping the object on the > floor (or to not introduce it to begin with) or destruct the > problematic objects explicitly. There's no cycles in my code, and this phenomenon doesn't happen with a non-SSL connection, so I'm guessing it's something to do with encryption. > To identify the cycles you may want to log the cycles detected by the > gc; take a look at base_server/roxen.pike:reinstall_gc_callbacks(). Thanks, will investigate further. ChrisA

1 0

File descriptor usage in Protocols.HTTP.Server.SSLPort
by Chris Angelico 05 Apr '24

05 Apr '24

I've run into a strangeness with my TLS-based web server. It seems that, for every incoming request, three file descriptors are used (they all seem to be sockets), and they aren't immediately cleaned up. With keep-alive disabled, they CAN be disposed of, but it takes an explicit gc() call. With keep-alive active, they aren't garbage, so they stick around even with a gc() call. (Which is probably correct, but there might need to be a limit on how many get retained.) constant listen_addr = "::", listen_port = 9876; void http_handler(Protocols.HTTP.Server.Request req) { int before = sizeof(get_dir("/proc/self/fd")); int garbo = gc(); //Disabling this check results in FDs accumulating. int after = sizeof(get_dir("/proc/self/fd")); werror("Garbage %O, closed %d files, now %d open\n", garbo, before - after, after); //The "Connection: close" header is vital to the sockets becoming garbage. //Without it, they are retained pending a followup request, which doesn't change the //fundamental issue but does mean that a call to gc() doesn't clean them up. req->response_and_finish((["data": "OK", "extra_heads": (["Connection": "close"])])); //req->response_and_finish((["data": "OK"])); } int main() { //If you don't have a cert, the first request is slower b/c generating self-signed. //This ONLY happens with SSL connections. There are *three* file descriptors wasted //for every request. string cert = Stdio.read_file("certificate_local.pem"); string key = Stdio.read_file("privkey_local.pem"); array certs = cert && Standards.PEM.Messages(cert)->get_certificates(); string pk = key && Standards.PEM.simple_decode(key); Protocols.HTTP.Server.SSLPort(http_handler, listen_port, listen_addr, pk, certs); return -1; } What are the three FDs used for? Presumably one of them is the actual incoming socket, but the other two are less obvious. This phenomenon does not seem to happen with non-SSL ports, possibly relating to the simpler shutdown sequence for unencrypted sockets. This isn't usually a major problem, as the GC does get run eventually, but under certain types of workloads, it's easy to overload it and run out of FDs. This can be triggered by removing the gc() call from each request, and then any of these will spin up until the process FD limit is hit: // Pike: while (1) Protocols.HTTP.get_url_data("https://YOUR_SERVER.EXAMPLE:9876/"); # Python: import requests while 1: requests.get("https://YOUR_SERVER.EXAMPLE:9876/") : Bash: while wget -qO/dev/null https://YOUR_SERVER.EXAMPLE:9876/; do true; done // JavaScript in a web browser, if not blocked: while (1) await(fetch("https://YOUR_SERVER.EXAMPLE:9876/")); So I think it's probably not purely an issue with a client library misbehaving. Is there a better way to handle this than simply forcing garbage collection every request? ChrisA

2 1

Re: Pure Pike annotations?
by Chris Angelico 15 Feb '24

15 Feb '24

On Thu, 15 Feb 2024 at 23:39, Henrik Grubbström (Lysator) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > > I'm trying to make good use of annotations, but all the existing ones > > are in cmod files, and I'm trying to work out how best to write them > > in Pike. My goal is to have an annotation that can be parameterized, > > something like this: > > > > class Anno(string label) { > > inherit Pike.Annotation; > > @constant; > > } > > > > @Anno("foobar"): > > void some_func() { } > > The above is supposed to work. > > > But while this does result in the annotation itself showing up, the > > label on that annotation doesn't. > > Try having the Anno class in a separate file. Ah yes, now it works. Thanks. Didn't think to try that. As long as Anno itself is added as a constant, the parameterized versions work correctly. > > What's the intended way to create parameterized annotations? > > This seems to be a compiler bug. Thanks. Not a major issue since there's a workaround (and I'll usually have them in separate files anyway). ChrisA

2 2

Pure Pike annotations?
by Chris Angelico 15 Feb '24

15 Feb '24

(Side note: This mailing list seems very dead lately. Is the LysKOM gateway down?) I'm trying to make good use of annotations, but all the existing ones are in cmod files, and I'm trying to work out how best to write them in Pike. My goal is to have an annotation that can be parameterized, something like this: class Anno(string label) { inherit Pike.Annotation; @constant; } @Anno("foobar"): void some_func() { } But while this does result in the annotation itself showing up, the label on that annotation doesn't. What's the intended way to create parameterized annotations? ChrisA

2 1

Re: pike docker images
by Henrik Grubbstr�m (Lysator) ＠ Pike (-) developers forum 14 Feb '24

14 Feb '24

Hi. At Pike Conference 2017 it was decided that we ought to hold a more informal meeting in Linköping ~ once every quarter. As the 3 month mark from the latest conference is coming up soon, it seems time to announce the first such meeting for 2024: Pike Meet-up 2024-Q1 Date: 2024-02-17 (Sat) Time: 10:00 - afternoon/evening Place: Roxen IS AB S:t Larsgatan 22A Linköping Welcome! /grubba

3 4

Debian Pike packaging and GTK2
by Chris Angelico 04 Feb '24

04 Feb '24

The latest releases of Debian don't include a pike8.0-gtk package any more. Apparently it's because it depends on old libraries: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=885673 But all of those are optional dependencies. I just tested a fresh Debian 12 installation and was able to build Pike from source fairly easily. As long as libgtk2.0-dev is installed, the majority of the GTK2 module exists, and it's only GTK2.SourceView etc that are missing. How would we go about getting Pike GTK back into Debian? ChrisA

1 0

Re: Client certificate support in Sql.pgsql?
by william＠welliver.org 12 Jan '24

12 Jan '24

Hi Chris, the SSL module supports passing client certificates, and the Sql.pgsql module uses SSL.File, however it doesn't directly provide any hooks for configuring the client certs. I think it would be easy to add, basically you just need a way to pass a properly configured SSL.Context object to the SSL.File object in (I think the relevant place is in Sql.pgsql_util.pmod). Once that's done, the handshake process should see the client certs and then pass them when requested by the server. As a side note, I think anything that offers TLS support should provide a means for passing a Context, as that's where all of the configuration is done. I don't think that is currently the case, and might be a worthwhile side project. I'm a bit swamped here, otherwise I'd offer some tested code, but I think it be fairly straightforward to add a constructor variant to Sql.pgsql and the associated wiring below it. If you've got specific questions, I'm happy to try to answer them (I wrote the initial client cert support many, many years ago). Bill On 2023-12-03 19:51, Chris Angelico wrote: > Is it possible to use client SSL certificates for authentication with > the Sql.pgsql or (the deprecated) Sql.postgres driver? I've been > setting up a multihomed Pike program and would ideally like to be able > to have all nodes connect to the same database, using their SSL > certificates as proof of identity. In theory, this should work, but > I've had no success. Anyone done this and if so, how? > > The connection works in the command line psql interface, but not in > Pike. > > ChrisA

3 14

None
by None 08 Mar '22

08 Mar '22

| If the result is a BLOB then the sqlite3_column_bytes() routine | returns the number of bytes in that BLOB. No type conversions | occur. If the result is a string (or a number since a number can be | converted into a string) then sqlite3_column_bytes() converts the | value into a UTF-8 string and returns the number of bytes in the | resulting string. The value returned does not include the \000 | terminator at the end of the string. The sqlite3_column_bytes16() | routine converts the value into a UTF-16 encoding and returns the ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | number of bytes (not characters) in the resulting string. The \u0000 | terminator is not included in this count. As far as I can see, it does perform conversions internally, and they will most likely fail if the inserted data isn't properly encoded. >Regards, >/Al

1 0

None
by None 08 Mar '22

08 Mar '22

/* Macros for aggregating results built on the stack into an array, * while maintaining a bound on stack consumption. Use like this: * * check_stack(120); * BEGIN_AGGREGATE_ARRAY(estimated_size_of_final_array) { * for (...) { * ... stuff that produces a value on the stack ... * DO_AGGREGATE_ARRAY(120); * ... * } * } END_AGGREGATE_ARRAY; * * The array is left on top of the stack. */

1 0

None
by None 08 Mar '22

08 Mar '22

error prone to me :) Just to be sure: > Stdio.File("/tmp/foo", "r")->mode(); (2) Result: 4099 -- Bertrand LUPART Linkeo.com - R&D | http://www.linkeo.com/ +33 1 72 71 71 82 | 17, rue de la Banque - F75002 Paris Systèmes de gestion d'email entrant | Email management systems

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel