Pike-devel

pike-devel@lists.lysator.liu.se

3041 discussions

Hyperscan regexp engine
by Tomasz Jamroszczak 27 Jun '19

27 Jun '19

There's the https://github.com/intel/hyperscan regexp library created over 10 years by algorithm start-up https://branchfree.org/2019/02/28/paper-hyperscan-a-fast-multi-pattern-rege… then bought by Intel and further developed. The development did everything the right way and the lib boasts a lot of optimizations and speedup. Is there a plan to import the lib to Pike? Would it be easy and straight-forward to do so? Best Regards, Tomasz Jamroszczak -- Using Opera's mail client: http://www.opera.com/mail/

3 4

Shuffler.Shuffler, does anyone use it?
by Stephen R. van den Berg 26 Jun '19

26 Jun '19

I though Roxen might be using it, but maybe I remember it wrong. Anyone else? I ask because I'm changing Protocols.HTTP.Server.Request to use the Shuffler, and I'm close to done, but it appears that in the process I have had to fix some quite serious bugs in the current shuffler implementation which would/should have prevented it from working correctly in a lot of normal use cases already. I was then wondering if it is being used successfully anywhere already? -- Stephen.

5 17

set_read_callback()
by Stephen R. van den Berg 26 Jun '19

26 Jun '19

Does anyone know where it is defined in the source? I'm having issues in the Shuffler (which have been there for ages already) where a C-struct is being freed, but then after that, a callback is being called from pike which still references the (now freed) C-struct. The freeing of the C-struct needs to be delayed until all references to the callback function are gone (which probably implies that I need to turn the struct over to the garbage collector, which implies that it needs to be an svalue, I presume). If anyone has better ideas for this, I'm all ears. -- Stephen.

2 2

close callback on FakeFile
by Stephen R. van den Berg 25 Jun '19

25 Jun '19

FakeFile is supposed to emulate a regular file. This would mean that whenever you reach the end of the file/string you are reading, you are supposed to signal a close callback because eof has been reached. I.e. the close callback on FakeFile should therefore not need to wait for an actual close from the writer. Reaching the end of the string (=file) suffices. All this contrasts with FakePipe where this obviously not hold true and you have to wait for an explicit close from the writer side. -- Stephen.

1 0

MozJPEG encoding library
by Stephen R. van den Berg 19 Jun '19

19 Jun '19

I was thinking about writing glue to integrate the MozJPEG encoding library with Pike. I already figured it should live in src/post_modules/_Image_MozJPEG. The question remains what to do about the library itself? - The library itself is a JPEG encoding-only library. - It usually is not system provided (because it's encoding only). - The library itself is available at https://github.com/mozilla/mozjpeg - The library is about 1.5MB xz compressed and about 9MB uncompressed. The options are: a. Make it pluggable, so that if you manually provide the library source tree in the correct src/post_modules/_Image_MozJPEG/mozjpeg location it will automatically detect and compile it in. b. Make it fetch the library using git during the configure run, and then jump to option a. c. Include the library in the Pike source tar. d. Create a package for that library that I submit to all (Linux) distributions. (Not really an option, too much work). I'd think that option b) might be the most prudent one. What do you think? -- Stephen.

2 3

Longevity of a struct pike_string str pointer?
by Stephen R. van den Berg 18 Jun '19

18 Jun '19

I'm kneedeep in the Shuffler code again, slightly reworking it, fixing some bugs along the way, but also trying to make sure I don't introduce new ones. The following issue comes up: If I have a struct pike_string with a shared string assigned to it, and I take the value of the str member because I want to access the bytes, how long is this address guaranteed to stay valid? E.g. I see in file.c that we take the str->str value, then do THREADS_ALLOW() and subsequently call fd_write(). Which seems to imply that as long as the pike_string struct we took the address from is still alive, the address of the string it points will not change, even if other pike threads run in the meantime? -- Stephen.

2 1

SEGFAULT in Pike 8.1
by Stephen R. van den Berg 12 Jun '19

12 Jun '19

When I run the following in Pike 8.1, it SEGFAULTs: foo::bar = 1; Gets me: p.pike:4:No inherit or surrounding class foo. Segmentation fault -- Stephen.

2 15

Occasional hiccups during https negotiations with Google
by Stephen R. van den Berg 12 Jun '19

12 Jun '19

Not consistently, but once every 10 tries or so, I get the following backtrace which does not seem to originate in my own code: Can not modify the buffer right now, there are active subbuffers. -:1: _static_modules._Stdio()->Buffer(0 bytes, read=[..-1] data=[0..-1] free=[0.. 5300] allocated (read only))->add("\f\0\0n\3\0\35 \237{å\32B«=B/ûÓ\2\"DmøkFñ 3\35¤\3\37Ò\214+ÜÔ*\4\3\0F0D\2 =Í3]r¿\20CN\222²`ÞG\"Ð«·\23aÆ\r\237{q)Ùß\t\ 206w(\2 {\23$Ò\4vì\b\227ú\31n\24xÈ¢M\v8d F\1/z\211\vÈx¹ï\222") /usr/local/pike/8.1.13/lib/modules/SSL.pmod/Connection.pike:1051: SSL.ClientConnection(handshaking)->got_data("\26\3\3\0;\2\0\u00007\3\3\\ÿP¹\ 17êlU:b5ñwÖäO»àz£eæ(§DOWNGRD\1\0À+\0\0\17ÿ\1\0\1\0\0\v\0\2\1\0\0#\0\0\26\3\3 \r8\v\0\r4\0\r1\0\bË0\202\bÇ0\202\a¯ \3\2\1\2\2\209Î\222 tü½:O5Øj\21ö\rh0\r\ 6\t*\206H\206÷\r\1\1\v\5\u00000T1\v0\t\6\3U\4\6\23\2US1\u001e0\34\6\3"+[3389 ]+"\206w(\2 {\23$Ò\4vì\b\227ú\31n\24xÈ¢M\v8d F\1/z\211\vÈx¹ï\222\26\3\3\0\4\ 16\0\0\0") /usr/local/pike/8.1.13/lib/modules/SSL.pmod/File.pike:1985: SSL.File(Stdio.File("socket", "172.217.20.106:443", 777 /* fd=43 */), SSL.Cl ientConnection(handshaking))->ssl_read_callback(Stdio.File("socket", "172.21 7.20.106:443", 777 /* fd=43 */),"\26\3\3\0;\2\0\u00007\3\3\\ÿP¹\17êlU:b5ñwÖä O»àz£eæ(§DOWNGRD\1\0À+\0\0\17ÿ\1\0\1\0\0\v\0\2\1\0\0#\0\0\26\3\3\r8\v\0\r4\0 \r1\0\bË0\202\bÇ0\202\a¯ \3\2\1\2\2\209Î\222 tü½"+[3448]+"ú\31n\24xÈ¢M\v8d F \1/z\211\vÈx¹ï\222\26\3\3\0\4\16\0\0\0") /usr/local/pike/8.1.13/lib/modules/Stdio.pmod/module.pmod:1165: Stdio.File("socket", "172.217.20.106:443", 777 /* fd=43 */)->__stdio_read_ca llback() -:1: Pike.Backend(0)->`()(3600.0) Concurrent.Promise(pending,0): Promise broken. /usr/local/pike/8.1.13/lib/modules/Concurrent.pmod:1176: unknown function() -:1: _static_modules._Stdio()->Buffer(0 bytes, read=[..1279] data=[1280..1279] fr ee=[1280..0] subbuffer)->read_int8() /usr/local/pike/8.1.13/lib/modules/Standards.pmod/ASN1.pmod/Decode.pmod:68: Standards.ASN1.Decode.read_identifier(_static_modules._Stdio()->Buffer(0 byt es, read=[..1279] data=[1280..1279] free=[1280..0] subbuffer)) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/ASN1.pmod/Decode.pmod:100: Standards.ASN1.Decode.der_decode((a)0=_static_modules._Stdio()->Buffer(0 bytes , read=[..1279] data=[1280..1279] free=[1280..0] subbuffer),mapping[23]) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/ASN1.pmod/Decode.pmod:157: Standards.ASN1.Decode.der_decode(@0,@1=mapping[23]) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/ASN1.pmod/Decode.pmod:245: Standards.ASN1.Decode.secure_der_decode("0\202\5\0\202\20*.googleapis.com\20 2\24*.clients6.google.ae\202\24*.clients6.google.at\202\24*.clients6.google. be\202\24*.clients6.google.ca\202\24*.clients6.google.ch\202\24*.clients6"+[ 1094]+"is.com\202\26cloudendpointsapis.com\202\16googleapis.com",@1) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:703: Standards.X509.TBSCertificate(([ /* 5 elements */ "algorithm": Standards.ASN1.Types.Sequence(({ /* 2 elements */ Standards.ASN1.Types.Identifier(1.2.840.113549.1.1.11), Standards.ASN1.Types.Null() })), "issuer": ([ /* 3 elements */ "commonName": "Google Internet Authority G3", "countryName": "US", "organizationName": "Google Trust Services" ]), "serial": Gmp.mpz(76838582857112318721873235160560176488), "subject": ([ /* 5 elements */ "commonName": "*.googleapis.com", "countryName": "US", "localityName": "Mountain View", "organizationName": "Google LLC", "stateOrProvinceName": "California" ]), "version": 3 ]))->`raw_extensions=(Standards.ASN1.Types.Sequence(...)) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:938: Standards.X509.TBSCertificate(([ /* 5 elements */ "algorithm": Standards.ASN1.Types.Sequence(({ /* 2 elements */ Standards.ASN1.Types.Identifier(1.2.840.113549.1.1.11), Standards.ASN1.Types.Null() })), "issuer": ([ /* 3 elements */ "commonName": "Google Internet Authority G3", "countryName": "US", "organizationName": "Google Trust Services" ]), "serial": Gmp.mpz(76838582857112318721873235160560176488), "subject": ([ /* 5 elements */ "commonName": "*.googleapis.com", "countryName": "US", "localityName": "Mountain View", "organizationName": "Google LLC", "stateOrProvinceName": "California" ]), "version": 3 ]))->init(Standards.ASN1.Types.Sequence(...)) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:763: Standards.X509.TBSCertificate(([ /* 5 elements */ "algorithm": Standards.ASN1.Types.Sequence(({ /* 2 elements */ Standards.ASN1.Types.Identifier(1.2.840.113549.1.1.11), Standards.ASN1.Types.Null() })), "issuer": ([ /* 3 elements */ "commonName": "Google Internet Authority G3", "countryName": "US", "organizationName": "Google Trust Services" ]), "serial": Gmp.mpz(76838582857112318721873235160560176488), "subject": ([ /* 5 elements */ "commonName": "*.googleapis.com", "countryName": "US", "localityName": "Mountain View", "organizationName": "Google LLC", "stateOrProvinceName": "California" ]), "version": 3 ]))->create(@2=Standards.ASN1.Types.Sequence(...)) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:329: Standards.X509.TBSCertificate() /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:1448: Standards.X509.decode_certificate(Standards.PKCS.Signature.Signed(...)) /usr/local/pike/8.1.13/lib/modules/SSL.pmod/Connection.pike:283: SSL.ClientConnection(handshaking)->verify_certificate_chain(({"0\202\bÇ0\202 \a¯ \3\2\1\2\2\209Î\222 tü½:O5Øj\21ö\rh0\r\6\t*\206H\206÷\r\1\1\v\5\u00000T1 \v0\t\6\3U\4\6\23\2US1\u001e0\34\6\3U\4\n"+[2160]+"É\vèÍ|ÂPD\201\220.R#Z³\23 7iG(#ç","0\202\4\\0\202\3D \3\2\1\2\2\r\1ã©0\34ür\68?\232S\u001d0\r\6\t*\206 H\206÷\r\1\1\v\5\u00000L1 0\36\6\3U\4\v\23\27GlobalSign Roo"+[1029]+"\234@èÖ Usº<ß\220Ë\0[!\21gÂí2\36Þ"})) /usr/local/pike/8.1.13/lib/modules/SSL.pmod/Connection.pike:372: SSL.ClientConnection(handshaking)->handle_certificates(SSL.Buffer(0 bytes, r ead=[..3379] data=[3380..3379] free=[3380..3480] allocated)) /usr/local/pike/8.1.13/lib/modules/SSL.pmod/ClientConnection.pike:876: SSL.ClientConnection(handshaking)->handle_handshake(11,(a)3=SSL.Buffer(0 bytes , read=[..3379] data=[3380..3379] free=[3380..3480] allocated),_static_modul es._Stdio()->Buffer(3384 bytes, read=[..-1] data=[0..3383] free=[3384..0] su bbuffer)) /usr/local/pike/8.1.13/lib/modules/SSL.pmod/Connection.pike:1070: SSL.ClientConnection(handshaking)->got_data("\26\3\3\0;\2\0\u00007\3\3\\ÿP¹\ 17êlU:b5ñwÖäO»àz£eæ(§DOWNGRD\1\0À+\0\0\17ÿ\1\0\1\0\0\v\0\2\1\0\0#\0\0\26\3\3 \r8\v\0\r4\0\r1\0\bË0\202\bÇ0\202\a¯ \3\2\1\2\2\209Î\222 tü½:O5Øj\21ö\rh0\r\ 6\t*\206H\206÷\r\1\1\v\5\u00000T1\v0\t\6\3U\4\6\23\2US1\u001e0\34\6\3"+[3389 ]+"\206w(\2 {\23$Ò\4vì\b\227ú\31n\24xÈ¢M\v8d F\1/z\211\vÈx¹ï\222\26\3\3\0\4\ 16\0\0\0") /usr/local/pike/8.1.13/lib/modules/SSL.pmod/File.pike:1985: SSL.File(Stdio.File("socket", "172.217.20.106:443", 777 /* fd=43 */), SSL.Cl ientConnection(handshaking))->ssl_read_callback(Stdio.File("socket", "172.21 7.20.106:443", 777 /* fd=43 */),"\26\3\3\0;\2\0\u00007\3\3\\ÿP¹\17êlU:b5ñwÖä O»àz£eæ(§DOWNGRD\1\0À+\0\0\17ÿ\1\0\1\0\0\v\0\2\1\0\0#\0\0\26\3\3\r8\v\0\r4\0 \r1\0\bË0\202\bÇ0\202\a¯ \3\2\1\2\2\209Î\222 tü½"+[3448]+"ú\31n\24xÈ¢M\v8d F \1/z\211\vÈx¹ï\222\26\3\3\0\4\16\0\0\0") /usr/local/pike/8.1.13/lib/modules/Stdio.pmod/module.pmod:1165: Stdio.File("socket", "172.217.20.106:443", 777 /* fd=43 */)->__stdio_read_ca llback() -:1: Pike.Backend(0)->`()(3600.0) -- Stephen.

2 4

Scope runtime bug in Pike 8.0 and 8.1
by Stephen R. van den Berg 08 Jun '19

08 Jun '19

-------------------------------------------------------- class foo { #if 1 protected int bar; // Does not work (and is a bug) #else int bar; // Does work #endif this_program other; protected void create() { other = this; other->bar = 1; } } object x = foo(); -------------------------------------------------------- This gives a runtime error which seems incorrect to me. Flipping the #if causes it to run flawlessly. Is this a problem in Pike, or are my expectations too high? -- Stephen.

2 2

Proposal: named threads
by H. William Welliver III 29 May '19

29 May '19

I’ve been thinking about adding support for named threads and would like to get some feedback on my proposed solution. Named threads can be useful when, for example, debugging or printing a thread dump. There are varying levels of OS support for named threads, ranging from none at all (older versions of illumos, and Solaris before 11.3) to being able to name any thread at any time (glibc, some of the BSDs, etc). I’d like to shoot for a level of functionality that should be possible for all cases: - Ability to set a thread’s name at creation time - Ability to get a thread’s name any time - Add thread name to thread dump, if set A brief survey of the API provided by other languages suggests that most seem to accept a thread name following the function to be run. Since the existing Pike thread API accepts varargs after the function, there’s no way to determine when a thread name has been provided. With this in mind, I suggest the opposite: variant protected void create(string thread_name, function f, mixed … args); public string name(); Assuming `()() can have variants, a similar variant would be created in Thread.Thread as well. On systems that don’t support naming threads, a parameter would be added either to the pike thread object itself or to the underlying OS thread’s thread specific data. In this situation, the thread name would be available through pike, but not in gdb, etc (but it wouldn’t have been anyway, so no big loss there). Some questions: Each OS has its own limit on the length of the thread name. What should the behavior be? Fail if the length is too long, truncate it silently, or set the maximum length but store the full length somewhere so Pike code can see the full length? There seems to be some difference in behavior, with Solaris silently truncating and the others indicating errors. Being able to set thread names after creation seems like a possibly useful feature. Darwin permits setting the thread name only on the current thread. So, if there were a desire to be able to set the thread name outside thread creation, this would be a cross-platform limitation if we wanted to maintain consistency. Is that a desirable behavior, and is such a limitation warranted? Or, do we emulate the capability by insert code somewhere that indicates the name has been changed and when the current thread is about to run, change the name at that time? What else have I missed? As always, comments and suggestions are welcome! Bill

3 2

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel