I tried building nettle 1.6 with --enabled-shared on Red Hat Linux 8.0 on x86.
The command to build the shared library doesn't get the soname or the filename correct:
gcc -shared -Wl,-soname=libnettle.so. aes_p.o [...] nettle-openssl_p.o -o libnettle.so
The attached patch fixes the problem.
The example programs are built like this:
[...] ../libnettle.a -lcrypto -lgmp
In order to test the shared libraries, that needs to be something like this:
[...] -L.. -lnettle -lcrypto -lgmp
That will work even if a shared library wasn't build. The example programs will need to be invoked with LD_LIBRARY_PATH set appropriately.
However, when I tested linking one of the example programs with the shared library, it failed:
gcc -I.. -g -O2 -ggdb3 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wstrict-prototypes -Wpointer-arith -Wbad-function-cast -Wnested-externs -o rsa-keygen rsa-keygen.o io.o -L.. -lnettle -lcrypto -lgmp ../libnettle.so: undefined reference to `memxor' collect2: ld returned 1 exit status
I haven't had time to look at this one any further...
Regards,