If I remember correctly it was statically compiled into one, big binary. Most things in the testsuite did work though.
Yes, I think that's what I got the first time. I've not had time to play around with it further, and I get the impression that pntld was created because NT shared libraries don't do lazy resolution? I'm not sure what the problem is, as load_module does load the module and resolve all of the symbols. It just seems to not like one or more segments. I'm not sure yet whether that's coming from pntld (in which case the problem is in pntld), or whether it's from the mingw library stubs that the module is linked against (in which case the problem is in the nt dynamic loading stuff).
I've pretty much given up on using ld -shared, as there doesn't seem to be any way to force it to supress errors for unresolved symbols (even though there are command options to do just that, they don't seem to have any effect).
As always, suggestions are welcome!
Bill
On Tue, 4 Oct 2005, Martin Nilsson (lvl 60) @ Pike (-) developers forum wrote:
If I remember correctly it was statically compiled into one, big binary. Most things in the testsuite did work though.
Yes, I think that's what I got the first time. I've not had time to play around with it further, and I get the impression that pntld was created because NT shared libraries don't do lazy resolution? I'm not sure what
The main reason is that NT shared libraries (AFAIK) don't support linking against symbols in the loading binary.
the problem is, as load_module does load the module and resolve all of the symbols. It just seems to not like one or more segments. I'm not sure yet whether that's coming from pntld (in which case the problem is in pntld), or whether it's from the mingw library stubs that the module is linked against (in which case the problem is in the nt dynamic loading stuff).
You can try enabling DLDEBUG in dlopen.c for lots of debug.
I've pretty much given up on using ld -shared, as there doesn't seem to be any way to force it to supress errors for unresolved symbols (even though there are command options to do just that, they don't seem to have any effect).
Thus the reason for the existence of pntld.
As always, suggestions are welcome!
Bill
The main reason is that NT shared libraries (AFAIK) don't support linking against symbols in the loading binary.
Yes, that's the impression I've gotten.
You can try enabling DLDEBUG in dlopen.c for lots of debug.
I'll give that a try; maybe I'll find something useful...
Thus the reason for the existence of pntld.
well, it certainly makes it easier to debug the linker!
Bill
OK, I've had a quick look at the MS PECOFF format document, and it seems that a type 7 relocation (which is what the error was) is the target's 32-bit relative virtual address, as opposed to type 6, which is the target's 32-bit (presumably absolute) virtual address. So, it looks like some of this is coming from the libraries provided by MinGW, and are passed through to the .so file by pntld. That makes me think that the problem might be in dlopen.c, which I've not had a chance to look into.
I'm really over my head here, so anyone who understands this better is welcome to jump in...
Bill
On Tue, 4 Oct 2005, Bill Welliver wrote:
The main reason is that NT shared libraries (AFAIK) don't support linking against symbols in the loading binary.
Yes, that's the impression I've gotten.
You can try enabling DLDEBUG in dlopen.c for lots of debug.
I'll give that a try; maybe I'll find something useful...
Thus the reason for the existence of pntld.
well, it certainly makes it easier to debug the linker!
Bill
OK, there seem to be 2 problems, first is that unless I run pntld with the -a argument, no symbols make it into my shared library. I've not had a chance to mess with that yet, but I'd love to hear speculation if there is any.
The second problem is that of the unsupported relocation type; I've traced through the dlopen code, and it appears that support seems to have been commented out:
#if 0 case COFFReloc_I386_dir32nb: #ifdef DLDEBUG fprintf(stderr,"DL: reloc absolute nb: loc %p = %p\n", loc,ptr); #endif ((int32 *)loc)[0]+=((INT32)ptr) - global_imagebase; break; #endif
Anyone know the reason for that? I'm going to try reenabling it...
Bill
On Tue, 4 Oct 2005, Bill Welliver wrote:
OK, I've had a quick look at the MS PECOFF format document, and it seems that a type 7 relocation (which is what the error was) is the target's 32-bit relative virtual address, as opposed to type 6, which is the target's 32-bit (presumably absolute) virtual address. So, it looks like some of this is coming from the libraries provided by MinGW, and are passed through to the .so file by pntld. That makes me think that the problem might be in dlopen.c, which I've not had a chance to look into.
OK, there seem to be 2 problems, first is that unless I run pntld with the -a argument, no symbols make it into my shared library. I've not had a chance to mess with that yet, but I'd love to hear speculation if there is any.
Sounds like you aren't using PIKE_MODULE_INIT and PIKE_MODULE_EXIT.
Or more likely; you aren't compiling with with a cl-compatible compiler, and thus don't get __declspec(dllexport) on PIKE_MODULE_{IN,EX}IT. In that case you need to patch module.h.
The second problem is that of the unsupported relocation type; I've traced through the dlopen code, and it appears that support seems to have been commented out:
No idea. Looks like it was commented out by Marcus 2001-09-14.
Anyone know the reason for that? I'm going to try reenabling it...
Sounds like you aren't using PIKE_MODULE_INIT and PIKE_MODULE_EXIT.
Well, this problem happens with all of the builtin modules. I'm focusing on Gmp right now, and they seem to use it, from what I can tell.
Or more likely; you aren't compiling with with a cl-compatible compiler, and thus don't get __declspec(dllexport) on PIKE_MODULE_{IN,EX}IT. In that case you need to patch module.h.
I tried manually including the __declspec ahead of PIKE_MODULE_INIT, and that didn't seem to make any difference, can __declspec appear twice on a function? According to MinGW, that's the preferred way to export symbols, so I'm not sure what the problem is. I've torn into pntld to see if I can find something wrong, but so far don't see anything misbehaved, it's as though the right information never makes it into the object files.
Incidentally, the gcc shipped is supposed to do auto-export, so maybe that's part of the problem, bit it's also supposed to shut off if I use __declspec.
Bill (further and further down the rabbit hole...)
Or more likely; you aren't compiling with with a cl-compatible compiler, and thus don't get __declspec(dllexport) on PIKE_MODULE_{IN,EX}IT. In that case you need to patch module.h.
I'm beginning to think that while MinGW supports the __declspec(dllexport) construct, it doesn't generate the same output in the object file that vc does. I say this because I've done just about everything I can think of to force the declspec, and none of it makes any difference to pntld.
I'm pretty much at a loss as far as what to do next. Unless someone else has a great idea, I'm going to drop the idea of getting dynamic modules working with MinGW.
Bill
Perhaps make something very simple and compile with both compilers and compare the results?
it's so simple, it might actually work :)
.. goes looking for vc++ command line tools
On Thu, 6 Oct 2005, Martin Nilsson (lvl 60) @ Pike (-) developers forum wrote:
Perhaps make something very simple and compile with both compilers and compare the results?
Thanks, I thought I saw that somewhere. For the record, the download is called "Microsoft Visual C++ Toolkit 2003" and includes all of the tools you'd need to build Pike with the "native" Microsoft toolchain.
Now to the interesting part:
I compiled the following file:
#include <stdio.h>
__declspec(dllexport) test() { printf("test()\n"); return; }
using the following command lines:
gcc -c test.c -o test_gcc.o cl /c test.c /Fo test_cl.o
and linked them using pntld:
pntld -o test_gcc.so test_gcc.o pntld -o test_cl.so test_cl.o
And the results seem to indicate that pntld doesn't deal with the gcc generated output file (ie __declspec(dllexport) doesn't do the same thing).
So, I'm going to try to figure out what the difference is and come up with a work around.
The two outputs from pntld (with debug enabled) are available here, if anyone has an interest:
http://hww3.riverweb.com/mingw
Bill
On Thu, Oct 06, 2005 at 11:54:27AM -0400, Bill Welliver wrote:
Thanks, I thought I saw that somewhere. For the record, the download is called "Microsoft Visual C++ Toolkit 2003" and includes all of the tools you'd need to build Pike with the "native" Microsoft toolchain.
does that mean that microsoft now offers a free compiler for windows?
greetings, martin.
Guess the answer is yes; I'm not sure how long this has been the case, though. I'm guessing it was a no-brainer... most windows developers want the ide, too (and it's actually not a horrible one, either).
does that mean that microsoft now offers a free compiler for windows?
Plus the fact that optimizations are turned off...
They aren't. Unless they changed something just a few months ago or so.
However, they are turned off in the 'cheap' visual studio version. You can replace that compiler with the freely downloadble one, and get optimizations, though.
But you have to specify the commandline options manually.
Thanks, I thought I saw that somewhere. For the record, the download is called "Microsoft Visual C++ Toolkit 2003" and includes all of the tools you'd need to build Pike with the "native" Microsoft toolchain.
Now to the interesting part:
[...]
And the results seem to indicate that pntld doesn't deal with the gcc generated output file (ie __declspec(dllexport) doesn't do the same thing).
The two outputs from pntld (with debug enabled) are available here, if anyone has an interest:
The main difference that I can see is that the gcc compiled version lacks "/DEFAULTLIB:"LIBC" /DEFAULTLIB:"OLDNAMES" /EXPORT:_test " in the .drectve section.
pntld line ~1257 scans the above string for export directives.
Bill
pntld line ~1257 scans the above string for export directives.
Looks like pntld looks at the segment characteristics rather than the name to determine whether it's a directive. I've added code to also check for the name. It seems that gcc produces directives that say -export:symbol, when symbol isn't present, but _symbol is. I added a check to look for that case, and now I'm data in my so files, but I can't know for sure what's actually in them because nm doesn't seem to want to read them (or the ones from the official pike package). When I get to the end of compilation, pike aborts because of a segmentation fault, trying to read address zero.
I'm not really sure how to debug that problem, but here is a copy of the dlopen debug output:
http://hww3.riverweb.com/mingw/startup.txt
Since the command line tools for vc++ are freely available, does it make more sense to use these tools instead of the mingw compiler chain? is that even feasible?
Bill
I think using MSVC is the way to go. MinGW is just to avoid having a unix-computer running configure/make. It's not like gcc is generating superior code or anything...
i wonder how gcc compares to a msvc which does not optimize. or does that even matter for compiling a compiler?
greetings, martin.
The difference between type 6 and type 7 is that type 7 is not "base relative", as far as I can see. The code in dlopen.c looks like this:
#if 0 case COFFReloc_I386_dir32nb: #ifdef DLDEBUG fprintf(stderr,"DL: reloc absolute nb: loc %p = %p\n", loc,ptr); #endif ((int32 *)loc)[0]+=((INT32)ptr) - global_imagebase; break; #endif
I assume the reason for the #if 0 is that nobody has been able to verify if this implementation is correct, since the M$ compiler didn't generate this kind of relocation.
pike-devel@lists.lysator.liu.se