On Thu, Apr 4, 2019 at 11:11 PM Henrik Grubbström (Lysator) @ Pike (-)
developers forum <10353(a)lyskom.lysator.liu.se> wrote:
>
> > The crash is on this line:
> [...]
> > If pike is built with PIKE_DEBUG, you get this fatal instead:
> [...]
> > So the crash is in the compiler proper, not in the execution of the
> > program.
>
> Thanks for the report, test case and investigation.
>
> Fixed the code generator in Pike 8.0 and master.
>
Thanks!…
[View More] Confirming that it works with my original code too (from which
the test case was derived). Much appreciated!
ChrisA
[View Less]
Test case:
int score_image(string image, int r, int g, int b) {return 42;}
array(int) getpixel() {return ({1, 2, 3});}
int main(int argc, array(string) argv)
{
if (argc < 2) exit(0, "All good! Try again with -DBOOM for a segfault.\n");
array images = ({"foo", "bar", "quux"});
for (int x = 0; x < 10; ++x) //Without the loop, no boom
{
#ifdef BOOM
array pixel = getpixel();
array(int) scores = score_image(images[*], @pixel); //Segfault
#else
…
[View More] [int r, int g, int b] = getpixel();
array(int) scores = score_image(images[*], r, g, b); //But
avoiding @ works.
#endif
}
}
It's fine with either [*] or @pixel, and it's even fine with both, as
long as it's not in a loop. But put all three together, and the
interpreter segfaults before executing a single line of actual code,
which I suspect means an issue at the compilation stage. (Can be
triggered identically by casting "boom.pike" to program too.)
Tested with the latest master branch, and with 8.0.735.
No idea where to start looking for the cause, but it's a completely
consistent crash.
ChrisA
[View Less]
> Hey Grubba!
Hi.
> Thanks for a great and fruitful meetup. I've learned a lot of things there.
> I'm also really glad that we move the debugger development on. Wouldn't
> have done it without you.
[...]
> Also,would you be able to share the local variable lookup feature so I can
> move on with the debugger work? I remember on Saturday the feature was a
> little buggy. I hope it will be possible to you to fix it. Anyway, what you
> already did would still be useful to …
[View More]me for moving on with work on the
> client.
I'm in the process of cleaning up the code, and it will enter
Pike/master soon (tm).
> Thanks and hope to see you on the next meetup!
See you.
/grubba
[View Less]
Hi Mateusz.
>Regarding the annotations - I seem to have lost my notes from the meetup,
>and have some questions:
>I've managed to play around the @Implements annotation and it seems to work
>alright.
>
>Can you remind me what does the @constant annotation do?
@constant is a "fake" annotation that sets the PROGRAM_CONSTANT flag
on the class it is contained in. This flag indicates that creating
instances of the class is valid in constant expressions (like eg
in an annotation …
[View More]expression).
>I've tried running the annotations() buitin and pass both classes and class
>instances to it, but both ways the returned array is always empty.
You probably only have annotations on the class itself, and not any on
its members.
Note that annotations() works analogously to indices(), values()
and types().
To see annotations on a class (or inherit) you currently need to
have at least on non-protected member of the class, and specify
the recurse flag to annotations().
>Is it already possible to define your own annotations in Pike code? If yes,
>how to do it? Some examples would be great.
You may want to take a look at the annotation tests in src/testsuite.in.
An example adapted from the testsuite:
local class InheritedImplements {
// Extend the default Implements annotation.
inherit Pike.Annotations.Implements;
// Make it valid to clone this class in a constant expression context.
@constant;
// Make this attribute inherited.
@Pike.Annotations.Inherited;
}
class A {
// Everything that inherits A must implement A.
@InheritedImplements(A);
int foo() {}
}
class B {
inherit A;
int foo() { return 1; }
}
class C {
inherit A;
// This fails due to bad return type compared with A::foo().
float foo() { return 1.0; }
}
>Thanks in advance.
/grubba
[View Less]
I found two issues relating to extracting tar files where a file has
the setuid bit set:
- unsafe default
- setuid but lost
To demonstrate these issues, I created this simple C file:
--- cut here for setuid-demo.c ---
#include <stdio.h>
#include <unistd.h>
int main()
{
printf("running as id %d\n", (int)geteuid());
return 0;
}
--- cut here for setuid-demo.c ---
I compiled it, set the setuid bit, created a tar archive, and
inspected it:
$ gcc -Wall setuid-demo.c
$ …
[View More]chmod 4711 a.out
$ tar cf a.out.tar a.out
$ tar tvf a.out.tar
-rws--x--x cederp/cederp 8352 2019-02-04 11:42 a.out
Let's extract it using tar to see how it works:
$ rm a.out
$ tar xf a.out.tar
$ ls -l a.out
-rwx--x--x 1 cederp cederp 8352 feb 4 11:42 a.out*
Ok. We lost the setuid bit. But when we run as root we get it:
$ rm a.out
$ sudo tar xf a.out.tar
$ ls -l a.out
-rws--x--x 1 cederp cederp 8352 feb 4 11:42 a.out*
Now, let's do the same with Pike:
$ rm a.out
$ ~/rgit/pike/bin/pike
Pike v8.1 release 13 running Hilfe v3.5 (Incremental Pike Frontend)
> Filesystem.Tar("a.out.tar")->tar->extract("/", ".");
$ ls -l a.out
-rws--x--x 1 cederp cederp 8352 feb 4 11:42 a.out*
Looking good. But this is of course a flawed test. Let's see what
happens when we run this as root?
$ rm a.out
$ sudo ~/rgit/pike/bin/pike
Pike v8.1 release 13 running Hilfe v3.5 (Incremental Pike Frontend)
> Filesystem.Tar("a.out.tar")->tar->extract("/", ".");
$ ls -l a.out
-rws--x--x 1 root cederp 8352 feb 4 11:42 a.out*
Oh! I got a setuid-root file! This is the first problem: I don't
like that Filesystem.Tar by default extracts the setuid, setgid and
sticky bits, while it doesn't extract the user and group info.
Changing this would be backwards incompatible, though, so perhaps the
best thing you could do is document this.
But the issue is arguably in my code . I should have specified
EXTRACT_CHOWN if I wanted a root-extracted file to be owned by cederp.
Lets do that and see what happens:
$ rm a.out
$ sudo ~/rgit/pike/bin/pike
Pike v8.1 release 13 running Hilfe v3.5 (Incremental Pike Frontend)
> Filesystem.Tar("a.out.tar")->tar->extract(
"/", ".", 0, Filesystem.Tar.EXTRACT_CHOWN);
$ ls -l a.out
-rwx--x--x 1 cederp cederp 8352 feb 4 11:42 a.out*
The setuid bit is lost! This is because Filesystem.Tar first sets the
mode bits, and then sets the owner. Quoting chown(2):
When the owner or group of an executable file is changed
by an unprivileged user, the S_ISUID and S_ISGID mode
bits are cleared. POSIX does not specify whether this
also should happen when root does the chown(); the Linux
behavior depends on the kernel version, and since Linux
2.2.13, root is treated like other users.
It would work better to set the correct owner first, and then set the
mode bits.
[View Less]
Hey! Thank you very much for feedback.
I've applied most of the suggested changes. As for the lfun::create()
approach to object deserializing - I've been thinking about it, but it's
troublesome for some edge cases, like arrays and tuples.
czw., 17 sty 2019 o 18:56 Henrik Grubbström (Lysator) @ Pike (-) developers
forum <10353(a)lyskom.lysator.liu.se> napisał(a):
> Hi.
>
> >I came to the point where I spent so much time on providing Pike support
> >for QuickType, that I'…
[View More]d probably manage to manually write DAP and LSP
> pmods
> >by now. So I've decided to stop there and deliver it as-is. Despite the
> >generated code's likely poor performance, it's been thoroughly tested and
> >proven to work for most test cases both for JSON, and JSON Schema.
> >
> >I really, really wanted to reuse Standards.JSON API, or else I would wind
> >up rewriting the whole JSON serialization module. That's why I ended up
> >calling JSON.encode, which invokes the encoded object's encode_json
> method,
> >inside JSON.decode. It's necessary because Pike member names are not
> >necessary JSON's keys. For exmaple, 'else' is a perfectly valid key name
> in
> >JSON, while it's a reserved keyword in many programming languages.
> >QuickType has it's ways to handle such situations.
> >Another known blunder is that it tends to append member declarations with
> >'mixed' when rendering from JSON Schemas. It's likely related to how
> >QuickType handles Schema's optional properties.
> >Here you can play around with it: https://app.quicktype.io/#l=pike
>
> Hmm... From the JSON example:
>
> | #define TO_MIXED(x) Standards.JSON.decode(Standards.JSON.encode(x))
>
> Why do the above? Under normal circumstances it should be a (slow)
> no-op (at least when it is called from encode_json()).
>
> [...]
> | string encode_json() {
> | mapping(string:mixed) json = ([]);
> |
> | json["greeting"] = TO_MIXED(greeting);
> | json["instructions"] = TO_MIXED(instructions);
>
> More efficient would be to create the mapping inline:
>
> string encode_json() {
> mapping(string:mixed) json = ([
>
> "greeting": TO_MIXED(greeting),
> "instructions": TO_MIXED(instructions),
> ]);
>
> [...]
> | Welcome Welcome_from_JSON(mixed json) {
>
> Why is json declared mixed above?
>
> | Welcome retval = Welcome();
> |
> | retval.greeting = json["greeting"];
> | retval.instructions = json["instructions"];
> |
> | return retval;
> | }
>
> I'd prefer having an lfun::create() in the Welcome class,
> that knows how to initialize from the mapping.
>
> protected void create(mapping(string:mixed)|void json)
> {
> if (!json) return;
> greeting = json["greeting"];
> instructions = json["instructions"];
> }
> ...
> Welcome Welcome_from_JSON(mapping(string:mixed) json)
> {
> return Welcome(json);
> }
>
> /grubba
>
[View Less]
I'll be there!
sob., 19 sty 2019 o 14:12 Henrik Grubbström (Lysator) @ Pike (-) developers
forum <10353(a)lyskom.lysator.liu.se> napisał(a):
> Hi.
>
> At Pike Conference 2017 it was decided that we ought to hold a
> more informal meeting in Linköping ~ once every quarter. As the
> three month mark from the latest conference is coming up soon,
> it seems time to announce the first such meeting for 2019:
>
> Pike Meet-up 2019-Q1
> Date: 2018-02-…
[View More]02 (Sat)
> Time: 10:00 - afternoon/evening
> Place: Roxen IS AB
> S:t Larsgatan 22A
> Linköping
>
> Welcome!
>
> /grubba
>
[View Less]
I came to the point where I spent so much time on providing Pike support
for QuickType, that I'd probably manage to manually write DAP and LSP pmods
by now. So I've decided to stop there and deliver it as-is. Despite the
generated code's likely poor performance, it's been thoroughly tested and
proven to work for most test cases both for JSON, and JSON Schema.
I really, really wanted to reuse Standards.JSON API, or else I would wind
up rewriting the whole JSON serialization module. That's why I …
[View More]ended up
calling JSON.encode, which invokes the encoded object's encode_json method,
inside JSON.decode. It's necessary because Pike member names are not
necessary JSON's keys. For exmaple, 'else' is a perfectly valid key name in
JSON, while it's a reserved keyword in many programming languages.
QuickType has it's ways to handle such situations.
Another known blunder is that it tends to append member declarations with
'mixed' when rendering from JSON Schemas. It's likely related to how
QuickType handles Schema's optional properties.
Here you can play around with it: https://app.quicktype.io/#l=pike
If anyone's interested in improving this feature, hit me up or issue a pull
request straight to the QuickType repo. If you find any more bugs, do not
hesitate to open an issue there.
pt., 11 sty 2019 o 16:30 Tobias S. Josefowitz @ Pike developers forum <
10353(a)lyskom.lysator.liu.se> napisał(a):
> >Those don’t count ;)
>
> Well, yes, my point being these are why the return type even includes
> |object.
>
[View Less]
On Mon, Jan 14, 2019 at 10:35 AM Mateusz Krawczuk <mkrawczuk(a)opera.com> wrote:
>
> Wouldn't it be better to go straight to Pike-devel to preserve the wisdom for broader audience?
On the odd chance that pike-devel want late night musings in Swedish,
then sure. I'm going to veto anyone else doing that though, so I don't
think the chances of everyone agreeing to see my badly thought thru
rants are high.
Anyways, time to write something more coherent:
We need a better API for …
[View More]handling directory listing, especially for
coupling the listings with their metadata (stat struct). At the moment
this is the most compact way of listing files in a dir sorted by
creation time I could come up with:
string mydir = "/tmp";
array files = get_dir(mydir);
files = combine_path(mydir, files[*]);
sort(files, file_stat(files[*])->ctime);
And whenever automap feels like a good idea the API probably needs to
be looked at. This will also explode in your face if files have been
deleted between get_dir() and file_stat().
The question is what that API should be. We have a slightly extended
API available in Filesystem:
object fs = Filesystem.System("/tmp");
array files = fs->get_dir();
sort(files, fs->get_stats()->ctime);
While this is cute if the directory contents is completely static (see
Filesystem.Tar), not so much otherwise.
I have some ideas of how a new API should look, basically:
1. Add a function (possibly to Filesystem) that returns both the
listing and stats, so that TOCTOU error handling can be kept away from
the user.
2. Add a convenience function where you can request a already sorted
list based on any criteria you want.
3. bonus: more iterators
Regards,
--
Peter Bortas
[View Less]