On Tue, Mar 31, 2020 at 9:41 AM Niels Möller nisse@lysator.liu.se wrote:

Nikos Mavrogiannopoulos n.mavrogiannopoulos@gmail.com writes:

...

On Mon, Mar 30, 2020 at 1:23 PM Niels Möller nisse@lysator.liu.se wrote:

...

The error is

1217 ./bootstrap: getting translations into po/.reference for gnutls... 1218 wget: /lib64/libhogweed.so.5: version `HOGWEED_INTERNAL_5_0' not found (required by /lib64/libgnutls.so.30) 1219 wget: /lib64/libnettle.so.7: version `NETTLE_INTERNAL_7_0' not found (required by /lib64/libgnutls.so.30)

I don't quite understand all details. This job buils and installs nettle, as

./configure --disable-documentation --prefix=/usr --libdir=/usr/lib64 && make -j4 && make install

That works well as long as binary compatibility is kept. As nettle breaks it, applications in the system will be unable to run.

But advertised binary compatibility between Nettle releases considers only the symbols related to the public api, not the symbols with *INTERNAL* as version. When gnutls refers to internal symbols, there's no binary compatibility at all between nettle versions.

I would strongly recommend that the default configuration of gnutls don't refer to internal nettle symbols. And any packaging of a version with a non-default configuration enabling use of nettle internals (e.g, to support more gost curves) needs to depend on a particular version of nettle, rather than on the soname, which only captures binary compatibility of the public ABI.

That is not possible as we are introducing features into gnutls in a faster pace than they are introduced in nettle (e.g., x448 or gost). Without a predictable cadence of nettle releases we have to duplicate and bundle a lot of code into gnutls because it is unknown when the next nettle release will be. The options we have considered were bundling nettle into gnutls or backporting selected features even when depending on nettle's ABI. The latter is the approach we currently take as it still keeps separation between the projects. The main problem lies however with the unpredictability and long delay of nettle's releases. If that could be addressed it would solve the dependency issue as well.

regards, Nikos

On 4/2/20 8:03 AM, Niels Möller wrote:

nisse@lysator.liu.se (Niels Möller) writes:

...

Nikos Mavrogiannopoulos n.mavrogiannopoulos@gmail.com writes:

...

We may want to install nettle somewhere separately and instruct gnutls to use it from there rather than the default locations.

That seems to be the right solution to the immediate problem.

Below patch seems to work. See https://gitlab.com/gnutls/nettle/-/jobs/495327357. Does it look right? Some questions:

1. Gnutls' configure uses pkg-config to check for existence and version of nettle, but it doesn't use flags provided by pkg-config. We therefore need to set *both* PKG_CONFIG_PATH and CPPFLAGS/LDFLAGS to use a Nettle library in a custom location. Is it supposed to work that way? I would expect most things to either rely exclusively on pkg-config, or use other means of configuration. Except that I don't know if pkg-config can or should be used to supply options like -Wl,-rpath,...

When installed to a custom location, you have to set PKG_CONFIG_PATH correctly. And the configure.ac code for nettle should be something like (taken from wget2): PKG_CHECK_MODULES([NETTLE], nettle, [ with_libnettle=yes LIBS="$NETTLE_LIBS $LIBS" CFLAGS="$NETTLE_CFLAGS $CFLAGS" AC_DEFINE([WITH_LIBNETTLE], [1], [Use libnettle]) ], [ with_libnettle=no AC_MSG_WARN(*** LIBNETTLE was not found. Fallback to libgcrypt for hashing and checksumming.) ])

*BUT* I had some issues with just setting PKG_CONFIG_PATH alone on Debian unstable a while ago. I had to adjust to something different.

This is from my current MinGW recipe: unset CC unset CXX

PREFIX=x86_64-w64-mingw32

export PATH="/usr/$PREFIX/bin:$PATH" export INSTALLDIR="$PWD/$PREFIX" export PKG_CONFIG_PATH=$INSTALLDIR/lib/pkgconfig:/usr/$PREFIX/lib/pkgconfig export PKG_CONFIG_LIBDIR="$INSTALLDIR/lib/pkgconfig" export PKG_CONFIG="/usr/bin/${PREFIX}-pkg-config" export CPPFLAGS="-I$INSTALLDIR/include" export LDFLAGS="-L$INSTALLDIR/lib"

cd nettle bash .bootstrap ./configure --build=x86_64-pc-linux-gnu --host=$PREFIX --enable-mini-gmp --enable-shared --disable-documentation --prefix=$INSTALLDIR make clean make -j$(nproc) make install

Hope that helps a bit.

2. What's the right way to add line breaks for the very long configure command in .gitlab-ci.yml?

Just breaking the line (linefeed) is enough, as long as the continuation line doesn't start with a dash '-'. What is also ok and used in many places is adding a '' plus linefeed.

Regards, Tim

On 4/3/20 10:29 AM, Jeffrey Walton wrote:

On Tue, Mar 31, 2020 at 7:42 AM Jeffrey Walton noloader@gmail.com wrote:

...

On Mon, Mar 30, 2020 at 7:23 AM Niels Möller nisse@lysator.liu.se wrote:

...

I committed a change to update nettle version numbers, which implies a new symbol version for internal symbols.

That seems to break the gnutls ci build, https://gitlab.com/gnutls/nettle/-/jobs/487360242

The error is

1217 ./bootstrap: getting translations into po/.reference for gnutls... 1218 wget: /lib64/libhogweed.so.5: version `HOGWEED_INTERNAL_5_0' not found (required by /lib64/libgnutls.so.30) 1219 wget: /lib64/libnettle.so.7: version `NETTLE_INTERNAL_7_0' not found (required by /lib64/libgnutls.so.30)

I don't quite understand all details. This job buils and installs nettle, as

./configure --disable-documentation --prefix=/usr --libdir=/usr/lib64 && make -j4 && make install

replacing any previously instaled version with the same soname. So that's a somewhat broader test than I had expected. That explains why linking of wget is affected.

The more worring part is that the installed gnutls library seems to depend on internal nettle symbols (it would have been nice if the error message named the offending symbols). Is that intended?

It's not necessarily wrong, but any packaged version of gnutls built that way *must* be marked to depend on exactly the same versions of nettle with which it was built. And the ci rule would need to use a different prefix and some other way to tell the gnutls build to use the newly built nettle library instead of the system version.

On my own debian machine I have libgnutls.so.30, but apparently an older version. According to

objdump -T /usr/lib/x86_64-linux-gnu/libgnutls.so.30

the version I have refers only to symbols with version NETTLE_6 and HOGWEED_4, i.e., nettle version older than 3.5. And no mention of NETTLE_INTERNAL or HOGWEED_INTERNAL.

It is not limited to Nettle.

Last week, /usr/lib/x86_64-linux-gnu/libgnutls.so.30 could not find a symbol in libidn2.so I had installed into /usr/local or /opt/local. The problem cascaded to the point I could not open a Terminal to fix things.

My problem appeared again. I'm going to have to boot into recovery mode, delete /usr/local, and reboot to fix it.

System libraries have no business going into my directories. I don't mess with /bin, /usr/bin, /lib and /usr/lib. The system should stay out of our directories.

IMO, you have that in your hand. Check /etc/ld.so.conf.d/* for /usr/local/lib. Prefix that file to be included as last one (or even remove it). Execute 'ldconfig' when done.

That *should* change your library search path as wanted.

Regards, Tim