Pike-devel June 2019

pike-devel@lists.lysator.liu.se

6 participants
18 discussions

Shuffler.Shuffler, does anyone use it?
by Stephen R. van den Berg 26 Jun '19

26 Jun '19

I though Roxen might be using it, but maybe I remember it wrong. Anyone else? I ask because I'm changing Protocols.HTTP.Server.Request to use the Shuffler, and I'm close to done, but it appears that in the process I have had to fix some quite serious bugs in the current shuffler implementation which would/should have prevented it from working correctly in a lot of normal use cases already. I was then wondering if it is being used successfully anywhere already? -- Stephen.

5 17

set_read_callback()
by Stephen R. van den Berg 26 Jun '19

26 Jun '19

Does anyone know where it is defined in the source? I'm having issues in the Shuffler (which have been there for ages already) where a C-struct is being freed, but then after that, a callback is being called from pike which still references the (now freed) C-struct. The freeing of the C-struct needs to be delayed until all references to the callback function are gone (which probably implies that I need to turn the struct over to the garbage collector, which implies that it needs to be an svalue, I presume). If anyone has better ideas for this, I'm all ears. -- Stephen.

2 2

close callback on FakeFile
by Stephen R. van den Berg 25 Jun '19

25 Jun '19

FakeFile is supposed to emulate a regular file. This would mean that whenever you reach the end of the file/string you are reading, you are supposed to signal a close callback because eof has been reached. I.e. the close callback on FakeFile should therefore not need to wait for an actual close from the writer. Reaching the end of the string (=file) suffices. All this contrasts with FakePipe where this obviously not hold true and you have to wait for an explicit close from the writer side. -- Stephen.

1 0

MozJPEG encoding library
by Stephen R. van den Berg 19 Jun '19

19 Jun '19

I was thinking about writing glue to integrate the MozJPEG encoding library with Pike. I already figured it should live in src/post_modules/_Image_MozJPEG. The question remains what to do about the library itself? - The library itself is a JPEG encoding-only library. - It usually is not system provided (because it's encoding only). - The library itself is available at https://github.com/mozilla/mozjpeg - The library is about 1.5MB xz compressed and about 9MB uncompressed. The options are: a. Make it pluggable, so that if you manually provide the library source tree in the correct src/post_modules/_Image_MozJPEG/mozjpeg location it will automatically detect and compile it in. b. Make it fetch the library using git during the configure run, and then jump to option a. c. Include the library in the Pike source tar. d. Create a package for that library that I submit to all (Linux) distributions. (Not really an option, too much work). I'd think that option b) might be the most prudent one. What do you think? -- Stephen.

2 3

Longevity of a struct pike_string str pointer?
by Stephen R. van den Berg 18 Jun '19

18 Jun '19

I'm kneedeep in the Shuffler code again, slightly reworking it, fixing some bugs along the way, but also trying to make sure I don't introduce new ones. The following issue comes up: If I have a struct pike_string with a shared string assigned to it, and I take the value of the str member because I want to access the bytes, how long is this address guaranteed to stay valid? E.g. I see in file.c that we take the str->str value, then do THREADS_ALLOW() and subsequently call fd_write(). Which seems to imply that as long as the pike_string struct we took the address from is still alive, the address of the string it points will not change, even if other pike threads run in the meantime? -- Stephen.

2 1

SEGFAULT in Pike 8.1
by Stephen R. van den Berg 12 Jun '19

12 Jun '19

When I run the following in Pike 8.1, it SEGFAULTs: foo::bar = 1; Gets me: p.pike:4:No inherit or surrounding class foo. Segmentation fault -- Stephen.

2 15

Occasional hiccups during https negotiations with Google
by Stephen R. van den Berg 12 Jun '19

12 Jun '19

Not consistently, but once every 10 tries or so, I get the following backtrace which does not seem to originate in my own code: Can not modify the buffer right now, there are active subbuffers. -:1: _static_modules._Stdio()->Buffer(0 bytes, read=[..-1] data=[0..-1] free=[0.. 5300] allocated (read only))->add("\f\0\0n\3\0\35 \237{å\32B«=B/ûÓ\2\"DmøkFñ 3\35¤\3\37Ò\214+ÜÔ*\4\3\0F0D\2 =Í3]r¿\20CN\222²`ÞG\"Ð«·\23aÆ\r\237{q)Ùß\t\ 206w(\2 {\23$Ò\4vì\b\227ú\31n\24xÈ¢M\v8d F\1/z\211\vÈx¹ï\222") /usr/local/pike/8.1.13/lib/modules/SSL.pmod/Connection.pike:1051: SSL.ClientConnection(handshaking)->got_data("\26\3\3\0;\2\0\u00007\3\3\\ÿP¹\ 17êlU:b5ñwÖäO»àz£eæ(§DOWNGRD\1\0À+\0\0\17ÿ\1\0\1\0\0\v\0\2\1\0\0#\0\0\26\3\3 \r8\v\0\r4\0\r1\0\bË0\202\bÇ0\202\a¯ \3\2\1\2\2\209Î\222 tü½:O5Øj\21ö\rh0\r\ 6\t*\206H\206÷\r\1\1\v\5\u00000T1\v0\t\6\3U\4\6\23\2US1\u001e0\34\6\3"+[3389 ]+"\206w(\2 {\23$Ò\4vì\b\227ú\31n\24xÈ¢M\v8d F\1/z\211\vÈx¹ï\222\26\3\3\0\4\ 16\0\0\0") /usr/local/pike/8.1.13/lib/modules/SSL.pmod/File.pike:1985: SSL.File(Stdio.File("socket", "172.217.20.106:443", 777 /* fd=43 */), SSL.Cl ientConnection(handshaking))->ssl_read_callback(Stdio.File("socket", "172.21 7.20.106:443", 777 /* fd=43 */),"\26\3\3\0;\2\0\u00007\3\3\\ÿP¹\17êlU:b5ñwÖä O»àz£eæ(§DOWNGRD\1\0À+\0\0\17ÿ\1\0\1\0\0\v\0\2\1\0\0#\0\0\26\3\3\r8\v\0\r4\0 \r1\0\bË0\202\bÇ0\202\a¯ \3\2\1\2\2\209Î\222 tü½"+[3448]+"ú\31n\24xÈ¢M\v8d F \1/z\211\vÈx¹ï\222\26\3\3\0\4\16\0\0\0") /usr/local/pike/8.1.13/lib/modules/Stdio.pmod/module.pmod:1165: Stdio.File("socket", "172.217.20.106:443", 777 /* fd=43 */)->__stdio_read_ca llback() -:1: Pike.Backend(0)->`()(3600.0) Concurrent.Promise(pending,0): Promise broken. /usr/local/pike/8.1.13/lib/modules/Concurrent.pmod:1176: unknown function() -:1: _static_modules._Stdio()->Buffer(0 bytes, read=[..1279] data=[1280..1279] fr ee=[1280..0] subbuffer)->read_int8() /usr/local/pike/8.1.13/lib/modules/Standards.pmod/ASN1.pmod/Decode.pmod:68: Standards.ASN1.Decode.read_identifier(_static_modules._Stdio()->Buffer(0 byt es, read=[..1279] data=[1280..1279] free=[1280..0] subbuffer)) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/ASN1.pmod/Decode.pmod:100: Standards.ASN1.Decode.der_decode((a)0=_static_modules._Stdio()->Buffer(0 bytes , read=[..1279] data=[1280..1279] free=[1280..0] subbuffer),mapping[23]) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/ASN1.pmod/Decode.pmod:157: Standards.ASN1.Decode.der_decode(@0,@1=mapping[23]) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/ASN1.pmod/Decode.pmod:245: Standards.ASN1.Decode.secure_der_decode("0\202\5\0\202\20*.googleapis.com\20 2\24*.clients6.google.ae\202\24*.clients6.google.at\202\24*.clients6.google. be\202\24*.clients6.google.ca\202\24*.clients6.google.ch\202\24*.clients6"+[ 1094]+"is.com\202\26cloudendpointsapis.com\202\16googleapis.com",@1) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:703: Standards.X509.TBSCertificate(([ /* 5 elements */ "algorithm": Standards.ASN1.Types.Sequence(({ /* 2 elements */ Standards.ASN1.Types.Identifier(1.2.840.113549.1.1.11), Standards.ASN1.Types.Null() })), "issuer": ([ /* 3 elements */ "commonName": "Google Internet Authority G3", "countryName": "US", "organizationName": "Google Trust Services" ]), "serial": Gmp.mpz(76838582857112318721873235160560176488), "subject": ([ /* 5 elements */ "commonName": "*.googleapis.com", "countryName": "US", "localityName": "Mountain View", "organizationName": "Google LLC", "stateOrProvinceName": "California" ]), "version": 3 ]))->`raw_extensions=(Standards.ASN1.Types.Sequence(...)) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:938: Standards.X509.TBSCertificate(([ /* 5 elements */ "algorithm": Standards.ASN1.Types.Sequence(({ /* 2 elements */ Standards.ASN1.Types.Identifier(1.2.840.113549.1.1.11), Standards.ASN1.Types.Null() })), "issuer": ([ /* 3 elements */ "commonName": "Google Internet Authority G3", "countryName": "US", "organizationName": "Google Trust Services" ]), "serial": Gmp.mpz(76838582857112318721873235160560176488), "subject": ([ /* 5 elements */ "commonName": "*.googleapis.com", "countryName": "US", "localityName": "Mountain View", "organizationName": "Google LLC", "stateOrProvinceName": "California" ]), "version": 3 ]))->init(Standards.ASN1.Types.Sequence(...)) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:763: Standards.X509.TBSCertificate(([ /* 5 elements */ "algorithm": Standards.ASN1.Types.Sequence(({ /* 2 elements */ Standards.ASN1.Types.Identifier(1.2.840.113549.1.1.11), Standards.ASN1.Types.Null() })), "issuer": ([ /* 3 elements */ "commonName": "Google Internet Authority G3", "countryName": "US", "organizationName": "Google Trust Services" ]), "serial": Gmp.mpz(76838582857112318721873235160560176488), "subject": ([ /* 5 elements */ "commonName": "*.googleapis.com", "countryName": "US", "localityName": "Mountain View", "organizationName": "Google LLC", "stateOrProvinceName": "California" ]), "version": 3 ]))->create(@2=Standards.ASN1.Types.Sequence(...)) /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:329: Standards.X509.TBSCertificate() /usr/local/pike/8.1.13/lib/modules/Standards.pmod/X509.pmod:1448: Standards.X509.decode_certificate(Standards.PKCS.Signature.Signed(...)) /usr/local/pike/8.1.13/lib/modules/SSL.pmod/Connection.pike:283: SSL.ClientConnection(handshaking)->verify_certificate_chain(({"0\202\bÇ0\202 \a¯ \3\2\1\2\2\209Î\222 tü½:O5Øj\21ö\rh0\r\6\t*\206H\206÷\r\1\1\v\5\u00000T1 \v0\t\6\3U\4\6\23\2US1\u001e0\34\6\3U\4\n"+[2160]+"É\vèÍ|ÂPD\201\220.R#Z³\23 7iG(#ç","0\202\4\\0\202\3D \3\2\1\2\2\r\1ã©0\34ür\68?\232S\u001d0\r\6\t*\206 H\206÷\r\1\1\v\5\u00000L1 0\36\6\3U\4\v\23\27GlobalSign Roo"+[1029]+"\234@èÖ Usº<ß\220Ë\0[!\21gÂí2\36Þ"})) /usr/local/pike/8.1.13/lib/modules/SSL.pmod/Connection.pike:372: SSL.ClientConnection(handshaking)->handle_certificates(SSL.Buffer(0 bytes, r ead=[..3379] data=[3380..3379] free=[3380..3480] allocated)) /usr/local/pike/8.1.13/lib/modules/SSL.pmod/ClientConnection.pike:876: SSL.ClientConnection(handshaking)->handle_handshake(11,(a)3=SSL.Buffer(0 bytes , read=[..3379] data=[3380..3379] free=[3380..3480] allocated),_static_modul es._Stdio()->Buffer(3384 bytes, read=[..-1] data=[0..3383] free=[3384..0] su bbuffer)) /usr/local/pike/8.1.13/lib/modules/SSL.pmod/Connection.pike:1070: SSL.ClientConnection(handshaking)->got_data("\26\3\3\0;\2\0\u00007\3\3\\ÿP¹\ 17êlU:b5ñwÖäO»àz£eæ(§DOWNGRD\1\0À+\0\0\17ÿ\1\0\1\0\0\v\0\2\1\0\0#\0\0\26\3\3 \r8\v\0\r4\0\r1\0\bË0\202\bÇ0\202\a¯ \3\2\1\2\2\209Î\222 tü½:O5Øj\21ö\rh0\r\ 6\t*\206H\206÷\r\1\1\v\5\u00000T1\v0\t\6\3U\4\6\23\2US1\u001e0\34\6\3"+[3389 ]+"\206w(\2 {\23$Ò\4vì\b\227ú\31n\24xÈ¢M\v8d F\1/z\211\vÈx¹ï\222\26\3\3\0\4\ 16\0\0\0") /usr/local/pike/8.1.13/lib/modules/SSL.pmod/File.pike:1985: SSL.File(Stdio.File("socket", "172.217.20.106:443", 777 /* fd=43 */), SSL.Cl ientConnection(handshaking))->ssl_read_callback(Stdio.File("socket", "172.21 7.20.106:443", 777 /* fd=43 */),"\26\3\3\0;\2\0\u00007\3\3\\ÿP¹\17êlU:b5ñwÖä O»àz£eæ(§DOWNGRD\1\0À+\0\0\17ÿ\1\0\1\0\0\v\0\2\1\0\0#\0\0\26\3\3\r8\v\0\r4\0 \r1\0\bË0\202\bÇ0\202\a¯ \3\2\1\2\2\209Î\222 tü½"+[3448]+"ú\31n\24xÈ¢M\v8d F \1/z\211\vÈx¹ï\222\26\3\3\0\4\16\0\0\0") /usr/local/pike/8.1.13/lib/modules/Stdio.pmod/module.pmod:1165: Stdio.File("socket", "172.217.20.106:443", 777 /* fd=43 */)->__stdio_read_ca llback() -:1: Pike.Backend(0)->`()(3600.0) -- Stephen.

2 4

Scope runtime bug in Pike 8.0 and 8.1
by Stephen R. van den Berg 08 Jun '19

08 Jun '19

-------------------------------------------------------- class foo { #if 1 protected int bar; // Does not work (and is a bug) #else int bar; // Does work #endif this_program other; protected void create() { other = this; other->bar = 1; } } object x = foo(); -------------------------------------------------------- This gives a runtime error which seems incorrect to me. Flipping the #if causes it to run flawlessly. Is this a problem in Pike, or are my expectations too high? -- Stephen.

2 2

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel June 2019