You should absolutely positively not look at the Calendar to see how things are supposed to be.

The Calendar module has recursive dependencies that needs to be untangled before it can load.

(Time needs Timezone to be loaded, and Timezone needs Time to be loaded to be able to calculate the timezones for the calendar time. This is untangled by making a special instance of Time that is without timezones, namely the Calendar.ISO_UTC instance. Or so.)

Hmm... well, the hacky calendar way wasn't really easy (and maybe not even possible) in my situation, but managed to solve this with replacing the code

vars->exposed = .ExposedInterface();

with this: vars->exposed = ((program) "ExposedInterface.pike")();

solved it.. can't really see why it should make any difference though... ? (as with the inherit in my Proxy.pike, also needs to inherit from the string "ServerObject.pike" to work)...

I've run across this before, that I have to delay program resolvment until runtime, and not during compile time... is this a sign of me doing something wrong, or is this a shortcoming of the pike compiler?

//K

Kaos wrote:

Mirar @ Pike developers forum wrote:

...

You should absolutely positively not look at the Calendar to see how things are supposed to be.

The Calendar module has recursive dependencies that needs to be untangled before it can load.

(Time needs Timezone to be loaded, and Timezone needs Time to be loaded to be able to calculate the timezones for the calendar time. This is untangled by making a special instance of Time that is without timezones, namely the Calendar.ISO_UTC instance. Or so.)

I guess that it's the way it's not supposed to be.. but if I can get things to work by applying some master()->resolv during module "boot" (tagging on to the _module_value access to `[]) I'm going for it.. but I'll definitely bring up the discussion about what isn't working as expected, but as it is now I can do nothing since my app won't even run.. :(

//K

Martin Bähr wrote:

On Mon, Dec 20, 2004 at 02:27:28PM +0100, Kaos wrote:

...

replacing the code vars->exposed = .ExposedInterface(); with this: vars->exposed = ((program) "ExposedInterface.pike")(); solved it.. can't really see why it should make any difference though... ?

isn't the difference here that the first is evaluated at compile time, while the second is done at runtime?

greetings, martin. ps: i am half expecting to be wrong, so don't hesitate to correct me

Yes, and my statement was badly expressed. I know the latter is evaluated first at runtime, what I didn't (and still don't) understand is why it would make any difference since the program in question (ExposedInterface) has no outside refernces at all. So by not having it at all (or refed during runtime) works, but resolving it during compiletime results in unrecoverable error (for later referred modules...) doesn't quite make sense to me.. The problem still exists even if Exposed.. is left as an empty file!

I would like to have a crack at this, since it affects my work in a great deal, but feels it would take a lot of time and require extensive help from more pike-internals knowledgable people.

//K

Martin Stjernholm, Roxen IS @ Pike developers forum wrote:

Looks like cyclic resolver trouble. There are many (equally disgusting) ways to work around it. It's a bitch to fix. :(

Hmm.. are we up to it? (the fixing part) I would very much like to see it solved (I suppose I'm not alone ;) and wouldn't mind investing alot of effort into it, but won't be able to without support..

What I've seen this far is that it seems to be able to cope with the cyclic to some extent, but when more is added, it bails out..

If I understand the problem correctly, this is what happends:

compiling A... in A, find reference to B compile B in B, find reference to A already compiling A - error

This can't be the whole truth though, since this far it works fine for me, but toss in some more references between a few more modules, and it all comes down in pieces.. What seems to be the problem is when the compiler encounters static module references (A.moboo as opposed to A->fooboo).

Would it be possible to solve this by having the compiler first sweep through a program to add in all parts accessible by '.' and first after that compile its references, and finish up what's left?

I've tried to search in the archives of related discussions, but must have failed to use the right search terms..

//K

Martin Stjernholm, Roxen IS @ Pike developers forum wrote:

...

Hmm.. are we up to it? (the fixing part) I would very much like to see it solved (I suppose I'm not alone ;) and wouldn't mind investing alot of effort into it, but won't be able to without support..

I spent a couple of weeks on it, more or less full time. I gave up eventually. Was utterly defeated. It just laughed at my effort. The problem is built in at a very deep level and there's a lot of code involved.

You or anyone else is of course very (if not to say extremely) welcome to take a stab at it. I'd be happy to assist, but you must be prepared to dig through the resolver machinery at all levels, the program dumper, and the better part of the compiler. Maybe it just needs fresh eyes.

I have a fresh set of eyes, I'll eye through the code to see if I'm able to make out the big picture.. if so, then I wouldn't mind getting out my blade and have a stab at it ;)

...

If I understand the problem correctly, this is what happends:

compiling A... in A, find reference to B compile B in B, find reference to A already compiling A - error

This can't be the whole truth though, since this far it works fine for me, but toss in some more references between a few more modules, and it all comes down in pieces..

Yes. The reason it works most of the time are that it has been extended several times to cope with the most obvious cases, and later on with many not so obvious cases. But to really solve the problem I believe it's necessary to:

1. Make the resolver machinery fully two pass, so that the compilation passes can span several files. This means the resolver code in the master needs to handle compilation passes.

Doesn't sound too awfull, but then I'm not yet in a position to know what I'm talking about here.

2. Compile the individual classes in a file independently of each other. Right now the compiler goes through every file from the beginning to the end two times and that's it.

Ooh.. definitely sounds thougher.. Are the rules for how to compile a file set in language.yacc?

The program dumper makes it more interesting since it encodes stuff in a very different order compared to compilation from source. The way that modules really are instantiated classes also complicates matters.

The many partial fixes are problematic since they are fairly elaborate themselves and makes the code difficult to understand (mainly the placeholders and the supporter system in program.c). In my attempt I tried to do yet another add-on solution, but as said earlier that failed miserably. It might work better if one begins by completely removing all the previous solutions in place.

If the files involved gets compiled successfully, won't the dumper be able to produce working object files (or what the files it dumps is called)? At least this could be adressed separately from the compiler issue.(?)

Grubbas project to make a totally new compiler will of course solve this problem (along with many other, on top of being incredibly cool in just about every way). Unfortunately it still has a long way to go to be useful, though.

Exciting! would love to hear (know) more about it :)

...

I've tried to search in the archives of related discussions, but must have failed to use the right search terms..

Many discussions on this has taken place IRL. I guess most of those that are in writing are in the old swedish forum in InfoKOM (the internal lyskom system at Roxen IS).

Oh, I see.. by "internal" does it mean I can't access it from outside (non roxen people), or from liu's LysKOM? Tried a lyskom client, but came to the conclusion of preferring the mailing list method to comunicate.. ;)

//K

Are the rules for how to compile a file set in language.yacc?

What do you mean?

I'm sorry to say that you are.. ;)

The code that goes out hunting for symbol definitions is around the SAFE_APPLY_MASTER or safe_apply_handler in program.c. Mainly when it calls "resolv", but e.g. the "handle_inherit" and "get_default_module" calls are also interesting.

That can happen anytime during compilation, and it's not a problem for the compiler to be invoked recursively to compile other files.

The two passes are for lexing and parsing the file, construct the abstract syntax tree (or node tree as it's called in pike - see las.c), resolving types and inherits. You can say it's centered around the two calls to do_yyparse in program.c. Most of the code in program.c, language.yacc and las.c are about this.

The thing is that it's easy to construct cases using inherit where one class in a file has to be finished (i.e. gone through both passes) while another is only in the first pass.

There are also plenty of cases where incomplete types occur in the first pass which are corrected in the second pass, but in the meantime those incomplete types are propagated to other classes in pass two and won't get corrected there. This is hardly noticeable except that type checking isn't always as good as it should be.

Here I should be able to give you a bunch of clear cut examples of all this, but unfortunately I don't have any readily available. Some are disabled in the test suite, others have only been discussed on whiteboards, but many have never been properly extracted from the code they were discovered in since these problems tend to be hard to minimalize.

Ok. I'm not sure what is done in each pass, so an example would probably do well.

Basically, the first pass resolves identifiers to look up types and the second generates the code.

You can see an example on line 2265 in the test suite as a disabled test. What it tests is that it ought to be possible to have two modules in different files where each contains a subclass which inherits a subclass from the other module.

I understand this as: during a programs second pass the compiler generates messages when types are in conflict (e.g. wrong type for function argument) but might not notice if the type used or checked against has only gone through the first pass. Correct me when necessary.

No, that's not what I meant. It's been a while since I dug into this, so what follows might not be completely correct. Anyway I _think_ this part of the problem stems from the placeholder programs.

Placeholders are deployed when the compiler discovers a cyclic dependency, i.e. the program being resolved is currently being compiled in an outer context. In that case the placeholder, which is a completely bogus program or object, is returned instead. Any identifier is successfully looked up inside it, and that just yields the placeholder again. The theory is that it's gone in the second pass so that the lookup is done correctly then (which perhaps yields a lookup error if it turns out to not exist). But as you've seen you can get these meaningless messages about placeholder problems, so that theory doesn't work.

The types of the placeholders are of course incorrect, and that makes the type of the program they are used in incorrect in turn. Those incorrect types can sometimes escalate to the second pass in other programs; exactly how I don't recall right now.

I'm convinced that the placeholders are fundamentally wrong. A 100% correct solution shouldn't need them. A 100% correct solution should also always ensure that the type of a class/program is completely accurate after the first pass; the trick to sometimes correct it in the second pass doesn't always cut it.

You might be aware of the design paradigm in Pike that compilation never is dependent on the declaration order in the file. As I'm sure you've encountered at some point or another, that isn't true in practice (it shows especially when inherits are used in reverse order). Anyway, I don't think that problem is very severe, so it's not overly important that a solution makes Pike any better in that regard.

/.../ Any hints on which disabled tests in the testsuite? (will have a look to see if I find them).

Come to think of it, I doubt there's more than the one mentioned above.

Hello everybody,

Grubbas project to make a totally new compiler will of course solve this problem (along with many other, on top of being incredibly cool in just about every way). Unfortunately it still has a long way to go to be useful, though.

Why not creating a compiler that would generate .NET's CLR for that? To be honest I don't know exactly the technical implications of that but the description that some people made in this list of this for Pike code looks like a nice solution to me.

From what they said, it would provide a more stable and more easily maintained runtime with the advantage of being embeddable and compatible with .NET languages like Perl, Python, Cobol, Java,... and of course C# and VB.

/ David