-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
(Sorry for this stupid question.)
I'm trying to get nettle to build in OSX but are having some problems. If i simply do the bootstrap-configure-make I get problems in the assembly phase:
<quote> /Applications/Xcode.app/Contents/Developer/usr/bin/make all-here gcc -I. -DHAVE_CONFIG_H -g -O2 -ggdb3 -Wall -W -Wmissing-prototypes - -Wmissing-declarations -Wstrict-prototypes -Wpointer-arith - -Wbad-function-cast -Wnested-externs -fPIC -MT gcm-hash8.o -MD -MP -MF gcm-hash8.o.d -fPIC -c gcm-hash8.s gcm-hash8.s:220:18: error: unexpected token in '.section' directive .section .rodata ^
gcm-hash8.s:224:1: error: unknown directive .hword 0x0000,0xc201,0x8403,0x4602,0x0807,0xca06,0x8c04,0x4e05 ^
gcm-hash8.s:225:1: error: unknown directive .hword 0x100e,0xd20f,0x940d,0x560c,0x1809,0xda08,0x9c0a,0x5e0b ^ .... </quote>
Googling I found some other people having problems: https://gist.github.com/morgant/1753095
Turning of assembly (--disable-assembler to configure) I get through the compilation, assembly, but linking fails (similarly the link above):
<quote> Undefined symbols for architecture x86_64: "__nettle_aes_decrypt", referenced from: _nettle_aes_decrypt in aes-decrypt.po _nettle_aes128_decrypt in aes-decrypt.po _nettle_aes192_decrypt in aes-decrypt.po _nettle_aes256_decrypt in aes-decrypt.po "__nettle_aes_encrypt", referenced from: _nettle_aes_encrypt in aes-encrypt.po _nettle_aes128_encrypt in aes-encrypt.po _nettle_aes192_encrypt in aes-encrypt.po _nettle_aes256_encrypt in aes-encrypt.po (maybe you meant: __nettle_aes_encrypt_table) ... </quote>
This is on OSX 10.9 with gcc being a wrapper around clang:
Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn) Target: x86_64-apple-darwin13.0.0 Thread model: posix
and as being: Apple Inc version cctools-846.2.4, GNU assembler version 1.38
and ld being: @(#)PROGRAM:ld PROJECT:ld64-224.1 configured to support archs: armv6 armv7 armv7s arm64 i386 x86_64 armv6m armv7m armv7em LTO support using: LLVM version 3.3svn, from Apple Clang 5.0 (build 500.2.79)
Anybody succeeding and building nettle on OSX and if so how. Being able to use the optimized assembler versions would also be good.
- -- Med vänlig hälsning, Yours
Joachim Strömbergson - Alltid i harmonisk svängning. ======================================================================== Joachim Strömbergson Secworks AB joachim@secworks.se ========================================================================
Joachim Strömbergson joachim@secworks.se writes:
I'm trying to get nettle to build in OSX but are having some problems. If i simply do the bootstrap-configure-make I get problems in the assembly phase:
I don't use osx myself, and I'm not very familiar with it.
gcc -I. -DHAVE_CONFIG_H -g -O2 -ggdb3 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wpointer-arith -Wbad-function-cast -Wnested-externs -fPIC -MT gcm-hash8.o -MD -MP -MF gcm-hash8.o.d -fPIC -c gcm-hash8.s gcm-hash8.s:220:18: error: unexpected token in '.section' directive .section .rodata
What does the assembler/linker expect? What do you get from a C file containing
static const uint16_t t[4] = {1,2,3,4};
?
and as being: Apple Inc version cctools-846.2.4, GNU assembler version 1.38
That sounds like an ancient version of gas. Searching for it turns up a fred fish CD from 1994. I most likely had it installed back in the days. See http://ftp.back2roots.org/back2roots/cds/fred_fish/freshfish_vol05_9407/gnu/...
But it must have been patched heavily by Apple if it supports x86_64...
I'd strongly suggest first trying to build with modern gcc and binutils, before trying to find workarounds for apple's tools.
/Niels
On Fri, 13 Dec 2013, Niels Möller wrote:
Joachim Strömbergson joachim@secworks.se writes:
I'm trying to get nettle to build in OSX but are having some problems. If i simply do the bootstrap-configure-make I get problems in the assembly phase:
I don't use osx myself, and I'm not very familiar with it.
Just for what it's worth - I've been using nettle on OS X and it has worked just fine up including the 2.7 release.
gcc -I. -DHAVE_CONFIG_H -g -O2 -ggdb3 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wpointer-arith -Wbad-function-cast -Wnested-externs -fPIC -MT gcm-hash8.o -MD -MP -MF gcm-hash8.o.d -fPIC -c gcm-hash8.s gcm-hash8.s:220:18: error: unexpected token in '.section' directive .section .rodata
What does the assembler/linker expect? What do you get from a C file containing
static const uint16_t t[4] = {1,2,3,4};
?
This seems to output something like this: .section __TEXT,__const
and as being: Apple Inc version cctools-846.2.4, GNU assembler version 1.38
That sounds like an ancient version of gas. Searching for it turns up a fred fish CD from 1994. I most likely had it installed back in the days. See http://ftp.back2roots.org/back2roots/cds/fred_fish/freshfish_vol05_9407/gnu/...
But it must have been patched heavily by Apple if it supports x86_64...
It's indeed an old version forked from binutils, heavily patched and maintained by Apple since, but it's diverged pretty much from most things that have been added since.
I'd strongly suggest first trying to build with modern gcc and binutils, before trying to find workarounds for apple's tools.
Well - this is the stock compiler toolchain that you get with the system - the vast majority of developers on OS X use that instead of building their own toolchains. As said before, all previous releases of nettle have worked just fine on OS X.
// Martin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
Martin Storsjö wrote:
Just for what it's worth - I've been using nettle on OS X and it has worked just fine up including the 2.7 release.
Thanks for responding! What happens if you clone the repo, do the bootstrap-configure-make dance with the current repo? And what version of OSX and Xcode, command line tools are you running?
It's indeed an old version forked from binutils, heavily patched and maintained by Apple since, but it's diverged pretty much from most things that have been added since.
Well - this is the stock compiler toolchain that you get with the system - the vast majority of developers on OS X use that instead of building their own toolchains. As said before, all previous releases of nettle have worked just fine on OS X.
In my experience, things normally do unless very Linux specific or GNU specific. Since gcc in Xcode is actually a wrapper around clang/llvw.
- -- Med vänlig hälsning, Yours
Joachim Strömbergson - Alltid i harmonisk svängning. ======================================================================== Joachim Strömbergson Secworks AB joachim@secworks.se ========================================================================
On Fri, 13 Dec 2013, Joachim Strömbergson wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
Martin Storsjö wrote:
Just for what it's worth - I've been using nettle on OS X and it has worked just fine up including the 2.7 release.
Thanks for responding! What happens if you clone the repo, do the bootstrap-configure-make dance with the current repo?
I do run into the same issues as you - sorry for not making that clearer.
And what version of OSX and Xcode, command line tools are you running?
I'm using 10.8.5, xcode 5.0 - dunno where I see the version of the command line tools.
It's indeed an old version forked from binutils, heavily patched and maintained by Apple since, but it's diverged pretty much from most things that have been added since.
Well - this is the stock compiler toolchain that you get with the system - the vast majority of developers on OS X use that instead of building their own toolchains. As said before, all previous releases of nettle have worked just fine on OS X.
In my experience, things normally do unless very Linux specific or GNU specific. Since gcc in Xcode is actually a wrapper around clang/llvw.
Well - hand-written assembly is one of the more problematic parts indeed. For x86 assembly, some other projects use nasm/yasm, but for non-x86, most projects end up with more elaborate tools for converting the gas source to something that the apple version of it can handle, such as https://github.com/libav/gas-preprocessor. But I'm pleasantly surprised by how well it has worked with nettle so far without requiring any external tools.
// Martin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
Martin Storsjö wrote:
I'm using 10.8.5, xcode 5.0 - dunno where I see the version of the command line tools.
Ah, on a tool by tool basis. Such as
#> gcc --version: Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn) Target: x86_64-apple-darwin13.0.0
- -- Med vänlig hälsning, Yours
Joachim Strömbergson - Alltid i harmonisk svängning. ======================================================================== Joachim Strömbergson Secworks AB joachim@secworks.se ========================================================================
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
Martin Storsjö wrote:
Well - hand-written assembly is one of the more problematic parts indeed. For x86 assembly, some other projects use nasm/yasm, but for non-x86, most projects end up with more elaborate tools for converting the gas source to something that the apple version of it can handle, such as https://github.com/libav/gas-preprocessor. But I'm pleasantly surprised by how well it has worked with nettle so far without requiring any external tools.
Just to check: Are you saying that normally/previously you have been able to build nettle on OSX, but not now?
And if so did you then disable usage of the asm versions in nettle?
- -- Med vänlig hälsning, Yours
Joachim Strömbergson - Alltid i harmonisk svängning. ======================================================================== Joachim Strömbergson Secworks AB joachim@secworks.se ========================================================================
On Fri, 13 Dec 2013, Joachim Strömbergson wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
Martin Storsjö wrote:
Well - hand-written assembly is one of the more problematic parts indeed. For x86 assembly, some other projects use nasm/yasm, but for non-x86, most projects end up with more elaborate tools for converting the gas source to something that the apple version of it can handle, such as https://github.com/libav/gas-preprocessor. But I'm pleasantly surprised by how well it has worked with nettle so far without requiring any external tools.
Just to check: Are you saying that normally/previously you have been able to build nettle on OSX, but not now?
Yes, previously I've been able to build nettle on OSX including all assembler optimizations.
And if so did you then disable usage of the asm versions in nettle?
With the current git master version it fails, but if I disable assembly using --disable-assembler, it works.
Do note that if you've previously tried (unsuccessfully) to build in the same directory and change the configuration, you'll need to do "make distclean" inbetween, otherwise the new configuration won't take effect properly. This sounds like the issue you're running into when trying to use --disable-assembler.
// Martin
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
Martin Storsjö wrote:
With the current git master version it fails, but if I disable assembly using --disable-assembler, it works.
Do note that if you've previously tried (unsuccessfully) to build in the same directory and change the configuration, you'll need to do "make distclean" inbetween, otherwise the new configuration won't take effect properly. This sounds like the issue you're running into when trying to use --disable-assembler.
Yes, yes, yes! Thanks. It now builds.
- -- Med vänlig hälsning, Yours
Joachim Strömbergson - Alltid i harmonisk svängning. ======================================================================== Joachim Strömbergson Secworks AB joachim@secworks.se ========================================================================
Martin Storsjö martin@martin.st writes:
static const uint16_t t[4] = {1,2,3,4};
?
This seems to output something like this: .section __TEXT,__const
I see. So we need to define a system-dependent RODATA macro in config.m4, expanding to the right directive.
Second problem was .hword, what pseudoop does Apple's assembler use for 16-bit constants? I think current GNU as has a couple of different names for the same thing, so if we can find a name accepted by both assemblers that's nice, otherwise it's another problem which requires a macro in config.m4.
As said before, all previous releases of nettle have worked just fine on OS X.
That's very nice, and in a large part that's thanks to your work.
Regards, /Niels
On Fri, 13 Dec 2013, Niels Möller wrote:
Martin Storsjö martin@martin.st writes:
static const uint16_t t[4] = {1,2,3,4};
?
This seems to output something like this: .section __TEXT,__const
I see. So we need to define a system-dependent RODATA macro in config.m4, expanding to the right directive.
Second problem was .hword, what pseudoop does Apple's assembler use for 16-bit constants? I think current GNU as has a couple of different names for the same thing, so if we can find a name accepted by both assemblers that's nice, otherwise it's another problem which requires a macro in config.m4.
Ah right, I missed this part. Your C code snippet seemed to use .short - the whole relevant part is:
.section __TEXT,__const .align 1 ## @t _t: .short 1 ## 0x1 .short 2 ## 0x2 .short 3 ## 0x3 .short 4 ## 0x4
// Martin
Martin Storsjö martin@martin.st writes:
Your C code snippet seemed to use .short
Ok, I've now changed gcm-hash8.asm to also use .short.
Remains the configure test for rodata. Not sure what's best, but perhaps easiest to just check of $host_os is darwin.
Regards, /Niels
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
Niels Möller wrote:
Martin Storsjö martin@martin.st writes:
Your C code snippet seemed to use .short
Ok, I've now changed gcm-hash8.asm to also use .short.
I just tested a clean clone and it fails on the gcm-hash8.s due do the final section is being set to .rodata in the asm file. I changed it to __TEXT,__const and then it builds. Patch below.
diff --git a/x86_64/gcm-hash8.asm b/x86_64/gcm-hash8.asm index 60f444d..d33dfea 100644 - --- a/x86_64/gcm-hash8.asm +++ b/x86_64/gcm-hash8.asm @@ -188,7 +188,7 @@ ALIGN(16) EPILOGUE(_nettle_gcm_hash8)
define(<W>, <0x$2$1>) - - .section .rodata + .section __TEXT,__const ALIGN(2) .Lshift_table: .short W(00,00),W(01,c2),W(03,84),W(02,46),W(07,08),W(06,ca),W(04,8c),W(05,4e)
- -- Med vänlig hälsning, Yours
Joachim Strömbergson - Alltid i harmonisk svängning. ======================================================================== Joachim Strömbergson Secworks AB joachim@secworks.se ========================================================================
Joachim Strömbergson joachim@secworks.se writes:
I just tested a clean clone and it fails on the gcm-hash8.s due do the final section is being set to .rodata in the asm file. I changed it to __TEXT,__const and then it builds. Patch below.
Good progress.
Below is an untested patch for the configure machinery. Can you try this?
BTW, is ".section __TEXT,__const" different from the text segment used for the executable code? Or would it work just sas well to have an definition of RODATA, and get the table in the same section as the code?
Regards, /Niels
diff --git a/config.m4.in b/config.m4.in index 4c6565f..da57e25 100644 --- a/config.m4.in +++ b/config.m4.in @@ -6,6 +6,7 @@ define(<TYPE_PROGBITS>, <@ASM_TYPE_PROGBITS@>)dnl define(<ALIGN_LOG>, <@ASM_ALIGN_LOG@>)dnl define(<ALIGNOF_UINT64_T>, <@ALIGNOF_UINT64_T@>)dnl define(<W64_ABI>, <@W64_ABI@>)dnl +define(<RODATA>, <@ASM_RODATA@>)dnl divert(1) @ASM_MARK_NOEXEC_STACK@ divert diff --git a/configure.ac b/configure.ac index a4deaae..acfc59c 100644 --- a/configure.ac +++ b/configure.ac @@ -332,10 +332,19 @@ EOF AC_MSG_WARN([No assembler files found.]) fi fi + case "$host_os" in + darwin*) + ASM_RODATA='.section __TEXT,__const' + ;; + *) + ASM_RODATA='.section .rodata' + ;; + esac fi
AC_SUBST([OPT_ASM_NETTLE_SOURCES]) AC_SUBST([OPT_ASM_HOGWEED_SOURCES]) +AC_SUBST([ASM_RODATA])
AH_VERBATIM([HAVE_NATIVE], [/* Define to 1 each of the following for which a native (ie. CPU specific) diff --git a/x86_64/gcm-hash8.asm b/x86_64/gcm-hash8.asm index 60f444d..e742cec 100644 --- a/x86_64/gcm-hash8.asm +++ b/x86_64/gcm-hash8.asm @@ -188,7 +188,7 @@ ALIGN(16) EPILOGUE(_nettle_gcm_hash8)
define(<W>, <0x$2$1>) - .section .rodata + RODATA ALIGN(2) .Lshift_table: .short W(00,00),W(01,c2),W(03,84),W(02,46),W(07,08),W(06,ca),W(04,8c),W(05,4e)
nisse@lysator.liu.se (Niels Möller) writes:
Below is an untested patch for the configure machinery. Can you try this?
I haven't seen any feedback on this, but I have now checked it in nonetheless.
It would be nice if you could confirm that it actually works. If you or Martin also have a real gcc and GNU binutils installed on macosx, it would be nice if you could test that it works also when building with the GNU tools.
Regards, /Niels
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Aloha!
Niels Möller wrote:
I haven't seen any feedback on this, but I have now checked it in nonetheless.
It would be nice if you could confirm that it actually works. If you or Martin also have a real gcc and GNU binutils installed on macosx, it would be nice if you could test that it works also when building with the GNU tools.
Sorry for the late response, Christmas coming up soon.
I just made a fresh clone of the repo and can confirm that it builds fine on OSX 10.9 with the latest Xcode and command line tools from Apple. Good work, thanks!
- -- Med vänlig hälsning, Yours
Joachim Strömbergson - Alltid i harmonisk svängning. ======================================================================== Joachim Strömbergson Secworks AB joachim@secworks.se ========================================================================
On Tue, 17 Dec 2013, Niels Möller wrote:
nisse@lysator.liu.se (Niels Möller) writes:
Below is an untested patch for the configure machinery. Can you try this?
I haven't seen any feedback on this, but I have now checked it in nonetheless.
It would be nice if you could confirm that it actually works. If you or Martin also have a real gcc and GNU binutils installed on macosx, it would be nice if you could test that it works also when building with the GNU tools.
Yes, now it works fine here as well - sorry for not reacting on testing the patch earlier.
I don't have a real gcc and GNU binutils available unfortunately so I can't test if detecting that works as it should.
// Martin
nettle-bugs@lists.lysator.liu.se
-
Joachim Strömbergson -
Martin Storsjö -
nisse＠lysator.liu.se