Pike-devel

pike-devel@lists.lysator.liu.se

1 participants
3045 discussions

Re: Pike Meet-up 2019-Q3
by Mateusz Krawczuk 19 Jul '19

19 Jul '19

Any chance to move it further towards the end of August? I'm away on 4th. 17th, or 24th maybe? pt., 19 lip 2019 o 14:22 Henrik Grubbström (Lysator) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> napisał(a): > > Hi. > > At Pike Conference 2017 it was decided that we ought to hold a > more informal meeting in Linköping ~ once every quarter. As the > nine month mark from the latest conference is coming up soon, > it seems time to announce the third such meeting for 2019: > > Pike Meet-up 2019-Q3 > Date: 2018-08-04 (Sun) > Time: 10:00 - afternoon/evening > Place: Roxen IS AB > S:t Larsgatan 22A > Linköping > > Note that this time the meeting is on a Sunday. > > Welcome! > > /grubba

2 1

Example how to use Protocols.HTTP2 ?
by Stephen R. van den Berg 03 Jul '19

03 Jul '19

Does anyone have any sample code that demonstrates how to use Protocols.HTTP2 ? -- Stephen.

2 1

Re: Hyperscan regexp engine
by Tomasz Jamroszczak 28 Jun '19

28 Jun '19

On Thu, 27 Jun 2019 17:39:01 +0200, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > You can still want things which do not exist. :-) But given the > attitude of the author I don't expect I'll be wanting either that or > hyperscan. If _you_ want something which is only of interest to > "network [companies] looking to scan 5,000 complex regexes in > streaming mode" (who does that?), feel free to integrate it, but keep > it on Monger please. OK, I see your standing. There are two clarifications needed though. First, "the author" of Hyperscan is not only Geoff Langdale, but quite a big team. Second, the library is useful not only for network security hardware. It's used by GitHub (https://github.blog/2018-10-17-behind-the-scenes-of-github-token-scanning/) and in general is just faster than current regexp engines' state of art. Top view of constraints and areas of improvements is described on https://www.hyperscan.io/2015/10/20/match-regular-expressions/. br, tj.

1 0

Re: Hyperscan regexp engine
by Tomasz Jamroszczak 27 Jun '19

27 Jun '19

On Thu, 27 Jun 2019 15:59:01 +0200, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: > So I interpret this as "ure3" (whatever that is) being the thing we > actually want, not this garbage fire? On https://branchfree.org/2019/02/28/paper-hyperscan-a-fast-multi-pattern-rege… the author in the commets writes on March 27, 2019: """ I would like “ure3” to be open. I also will need help. I designed Hyperscan but did not build it all by myself; [...] If I build a new regex matching system it will be a lot smaller and simpler, but it will still be a huge effort. """ And on April 15, 2019: """ if I get to it. """ So well, given the fact that the "ure3" doesn't exist, I doubt it is something we should or even could want. br, tj.

2 1

Re: Hyperscan regexp engine
by Tomasz Jamroszczak 27 Jun '19

27 Jun '19

On Thu, 27 Jun 2019 13:58:02 +0200, Marcus Comstedt (ACROSS) (Hail Ilpalazzo!) @ Pike (-) developers forum <10353(a)lyskom.lysator.liu.se> wrote: >> /paper-hyperscan-a-fast-multi-pattern-regex-matcher-for-modern-cpus/ > > For modern CPU:s? Looks more like they are targeting a certain 1970:s > architecture: > > cpuid(1, 0, &eax, &ebx, &ecx, &edx); > > Did you test it on RISC-V? From https://news.ycombinator.com/item?id=19270199: """ I suppose "Some Modern CPUs" was too long-winded a title? As I said, it doesn't take a genius to understand Intel's motivations. They bought the project, after all. Not being @ Intel anymore, I don't have access to the older stuff, and even if I did, the codebase has diverged significantly since. Without going on too much of a tirade - the experience of developing for all those platforms really sucked. Almost all the non-x86 platforms had significant bugs in their toolchains. One of the MIPS variants (particular architecture elided to spare the guilty) had bugs in their gcc intrinsics in a way that suggested that no-one had ever done any significant third-party dev on the platform. Big-endian was also a huge PITA. It was a ton of work to keep all those systems alive, and our machine rack looked like a zoo of dev boards and weirdo devices. In the "ure3" system I mention, I would make retargetability/portability to other systems a first-class goal. One way of achieving this is not having such a huge profusion of methods and complexity. Hyperscan is over-engineered for many use cases if you aren't a network company looking to scan 5,000 complex regexes in streaming mode at hopefully maximal performance. """ > Also: 114 kLOC for a regexp matcher. Srsly? For a 40x speed boost? Worth at least trying. BR, tj

2 2

Hyperscan regexp engine
by Tomasz Jamroszczak 27 Jun '19

27 Jun '19

There's the https://github.com/intel/hyperscan regexp library created over 10 years by algorithm start-up https://branchfree.org/2019/02/28/paper-hyperscan-a-fast-multi-pattern-rege… then bought by Intel and further developed. The development did everything the right way and the lib boasts a lot of optimizations and speedup. Is there a plan to import the lib to Pike? Would it be easy and straight-forward to do so? Best Regards, Tomasz Jamroszczak -- Using Opera's mail client: http://www.opera.com/mail/

3 4

Shuffler.Shuffler, does anyone use it?
by Stephen R. van den Berg 26 Jun '19

26 Jun '19

I though Roxen might be using it, but maybe I remember it wrong. Anyone else? I ask because I'm changing Protocols.HTTP.Server.Request to use the Shuffler, and I'm close to done, but it appears that in the process I have had to fix some quite serious bugs in the current shuffler implementation which would/should have prevented it from working correctly in a lot of normal use cases already. I was then wondering if it is being used successfully anywhere already? -- Stephen.

5 17

set_read_callback()
by Stephen R. van den Berg 26 Jun '19

26 Jun '19

Does anyone know where it is defined in the source? I'm having issues in the Shuffler (which have been there for ages already) where a C-struct is being freed, but then after that, a callback is being called from pike which still references the (now freed) C-struct. The freeing of the C-struct needs to be delayed until all references to the callback function are gone (which probably implies that I need to turn the struct over to the garbage collector, which implies that it needs to be an svalue, I presume). If anyone has better ideas for this, I'm all ears. -- Stephen.

2 2

close callback on FakeFile
by Stephen R. van den Berg 25 Jun '19

25 Jun '19

FakeFile is supposed to emulate a regular file. This would mean that whenever you reach the end of the file/string you are reading, you are supposed to signal a close callback because eof has been reached. I.e. the close callback on FakeFile should therefore not need to wait for an actual close from the writer. Reaching the end of the string (=file) suffices. All this contrasts with FakePipe where this obviously not hold true and you have to wait for an explicit close from the writer side. -- Stephen.

1 0

MozJPEG encoding library
by Stephen R. van den Berg 19 Jun '19

19 Jun '19

I was thinking about writing glue to integrate the MozJPEG encoding library with Pike. I already figured it should live in src/post_modules/_Image_MozJPEG. The question remains what to do about the library itself? - The library itself is a JPEG encoding-only library. - It usually is not system provided (because it's encoding only). - The library itself is available at https://github.com/mozilla/mozjpeg - The library is about 1.5MB xz compressed and about 9MB uncompressed. The options are: a. Make it pluggable, so that if you manually provide the library source tree in the correct src/post_modules/_Image_MozJPEG/mozjpeg location it will automatically detect and compile it in. b. Make it fetch the library using git during the configure run, and then jump to option a. c. Include the library in the Pike source tar. d. Create a package for that library that I submit to all (Linux) distributions. (Not really an option, too much work). I'd think that option b) might be the most prudent one. What do you think? -- Stephen.

2 3

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

Pike-devel