Hi,
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
I intend to create a release-candidate tarball in a day or two for final testing.
Regards, /Niels
Hi,
вс, 5 апр. 2020 г. в 21:03, Niels Möller nisse@lysator.liu.se:
Hi,
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
GnuTLS project would like to ask you to bump libhogweed soname as a way to enforce recompilation because of the ecc-internal.h usage. Other than that it would be nice to get GOST VKO key derivation algorithm to supplement GOST digital signatures algorithm.
-- With best wishes Dmitry
Dmitry Baryshkov dbaryshkov@gmail.com writes:
GnuTLS project would like to ask you to bump libhogweed soname as a way to enforce recompilation because of the ecc-internal.h usage.
I guess that makes sense. It will inconvenience all other packages which don't access internals and which could benefit from ABI compatibility and unchanged soname, though.
But for this release, the changes are mainly new features and a bug fix for cfb8_decrypt. And, e.g., no major optimizations of existing functions. So probably fairly harmless to force recompile to be able to use a new version.
Any other opinions?
That said, I would wish a default gnutls build did *not* use internal functions, and had a configure option to enable use of internals, for users and packagers who understand the consequences. That would be analogous to the --enable-gmp-internals configure argument for MPFR.
Other than that it would be nice to get GOST VKO key derivation algorithm to supplement GOST digital signatures algorithm.
I don't want to delay the release for that. Ok?
Regards, /Niels
Hello,
вс, 5 апр. 2020 г. в 23:27, Niels Möller nisse@lysator.liu.se:
Dmitry Baryshkov dbaryshkov@gmail.com writes:
GnuTLS project would like to ask you to bump libhogweed soname as a way to enforce recompilation because of the ecc-internal.h usage.
I guess that makes sense. It will inconvenience all other packages which don't access internals and which could benefit from ABI compatibility and unchanged soname, though.
Thank you.
That said, I would wish a default gnutls build did *not* use internal functions, and had a configure option to enable use of internals, for users and packagers who understand the consequences. That would be analogous to the --enable-gmp-internals configure argument for MPFR.
Heh. We all would wish for a better world. I'd wish for feedback on the patches. VKO has been sent on February, 15th. Two merge requests on git.lysator.liu.se also do not any feedback except FUD.
Other than that it would be nice to get GOST VKO key derivation algorithm to supplement GOST digital signatures algorithm.
I don't want to delay the release for that. Ok?
That's fine.
-- With best wishes Dmitry
Dmitry Baryshkov dbaryshkov@gmail.com writes:
вс, 5 апр. 2020 г. в 23:27, Niels Möller nisse@lysator.liu.se:
Dmitry Baryshkov dbaryshkov@gmail.com writes:
GnuTLS project would like to ask you to bump libhogweed soname as a way to enforce recompilation because of the ecc-internal.h usage.
I guess that makes sense. It will inconvenience all other packages which don't access internals and which could benefit from ABI compatibility and unchanged soname, though.
Thank you.
Done. And it's sufficient to bump the hogweed soname, right? I updated the description in NEWS as follows:
The shared library names are libnettle.so.7.1 and libhogweed.so.6.0. The libnettle.so library is intended to be fully binary compatible with nettle-3.5.
The libhogweed.so library was also intended to be fully binary compatible with nettle-3.5. However, the major version and the soname are incremented anyway, to avoid upgrade problems with recent Gnutls versions that depend on Nettle internals outside of the advertised ABI.
Is that a fair description? I don't think there's any need to mention any gnutls version numbers here, do you agree?
Regards, /Niels
пн, 6 апр. 2020 г. в 20:46, Niels Möller nisse@lysator.liu.se:
Dmitry Baryshkov dbaryshkov@gmail.com writes:
вс, 5 апр. 2020 г. в 23:27, Niels Möller nisse@lysator.liu.se:
Dmitry Baryshkov dbaryshkov@gmail.com writes:
GnuTLS project would like to ask you to bump libhogweed soname as a way to enforce recompilation because of the ecc-internal.h usage.
I guess that makes sense. It will inconvenience all other packages which don't access internals and which could benefit from ABI compatibility and unchanged soname, though.
Thank you.
Done. And it's sufficient to bump the hogweed soname, right? I updated the description in NEWS as follows:
The shared library names are libnettle.so.7.1 and libhogweed.so.6.0. The libnettle.so library is intended to be fully binary compatible with nettle-3.5. The libhogweed.so library was also intended to be fully binary compatible with nettle-3.5. However, the major version and the soname are incremented anyway, to avoid upgrade problems with recent Gnutls versions that depend on Nettle internals outside of the advertised ABI.Is that a fair description? I don't think there's any need to mention any gnutls version numbers here, do you agree?
Yes, I think it is fine. Thank you!
Niels M??ller wrote:
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
I presume that the bcrypt support is intentionally left out? Or did you just forget to include it? I did not get any feedback on v3.1.
"Stephen R. van den Berg" srb@cuci.nl writes:
Niels Möller wrote:
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
I presume that the bcrypt support is intentionally left out?
Yes, I don't consider it an "easy" change. And it's not the only ongoing improvement that's postponed to after the release.
I did not get any feedback on v3.1.
I did send a short message, just a minute or two earlier. Sorry it has to take some time.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
I've created a tarball at
http://www.lysator.liu.se/~nisse/archive/nettle-3.6rc1.tar.gz http://www.lysator.liu.se/~nisse/archive/nettle-3.6rc1.tar.gz.sig
Only recent change is the updated libhogweed soname, requested by Dmitry. Both testing and review of the NEWS file highly appreciated. Please report test results, both positive and negative, to the list.
I intend to run basic ./configure && make && make check tests on a few of the machines available in the gcc farm and on some of the (private) gmp test systems, and make the release next week, maybe as early as Tuesday.
Regards, /Niels
Hello Niels,
nisse@lysator.liu.se (Niels Möller) writes:
nisse@lysator.liu.se (Niels Möller) writes:
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
I've created a tarball at
http://www.lysator.liu.se/~nisse/archive/nettle-3.6rc1.tar.gz http://www.lysator.liu.se/~nisse/archive/nettle-3.6rc1.tar.gz.sig
Only recent change is the updated libhogweed soname, requested by Dmitry. Both testing and review of the NEWS file highly appreciated. Please report test results, both positive and negative, to the list.
It builds fine on all arches running Fedora rawhide (with GCC 10): https://koji.fedoraproject.org/koji/taskinfo?taskID=43267967
I suggest a couple of minor improvements on the NEWS entry as attached.
Regards,
On 2020-04-05 Niels Möller nisse-SamgB31n2u5IcsJQ0EH25Q@public.gmane.org wrote:
Hi,
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
Hello,
I am not totally sure but think the non-lockstep soname bump of libhogweed and libnettle is quite painful:
According to objdump -R GnuTLS uses internal symbols of both libhogweed and libnettle (e.g. _nettle_mpn_set_base256_le@HOGWEED_INTERNAL_5_0 and _nettle_write_le64@NETTLE_INTERNAL_7_0). These nettle *internal* symbols seem to be incompatible in 3.6, at least they are versioned differently (@NETTLE_INTERNAL_7_1).
This makes to impossible install nettle-3.6 development files (libhogweed6/libnettle7/nettle-dev) while keeping gnutls runtime (libgnutls30, which requires libnettle7 [not 3.6] and libhogweed5) working. One can co-install libhogweed6 in addition to libhogweed5, but upgrading libnettle7 to 3.6 breaks gnutls runtime due to unresolved symbols.
Having non-working gnutls is quite a big issue, since e.g. apt depends on it.
cu Andreas
PS: On top of this the Debian nettle *packaging* currently is broken with respect to unsynced soname bumps of libhogweed/libnettle runtime packages, which is a different issue, probably a bug.
Andreas Metzler ametzler@bebt.de writes:
According to objdump -R GnuTLS uses internal symbols of both libhogweed and libnettle (e.g. _nettle_mpn_set_base256_le@HOGWEED_INTERNAL_5_0 and _nettle_write_le64@NETTLE_INTERNAL_7_0). These nettle *internal* symbols seem to be incompatible in 3.6, at least they are versioned differently (@NETTLE_INTERNAL_7_1).
Can you post a complete list of the references to internal symbols?
Having non-working gnutls is quite a big issue, since e.g. apt depends on it.
Indeed. What options do we have? The ones I see are
1. Simply bump the soname of libnettle too. I think that's ok in this case, since there are no major fixes or improvements to old features in this release. But I wouldn't want to make it a habit.
2. Fix GnuTLS to not refer to internal libnettle symbols. Mark the nettle-3.6 package including libnettle.so.7 as conflicting with the non-fixed GnuTLS package.
I don't know how practical option (2) is. But for the example of nettle_write_le64, it would be a simple improvement to just include a copy in GnuTLS if it needs it. The nettle implementation is not particularly clever, and it's not intended to be part of the ABI. E.g, one todo item is to replace it with a macro expanding to memcpy, on platforms that are natively little-endian, in which case the symbol would simply disappear from the library.
PS: On top of this the Debian nettle *packaging* currently is broken with respect to unsynced soname bumps of libhogweed/libnettle runtime packages, which is a different issue, probably a bug.
I think it's a bug. Independent of the current issues with GnuTLS, I think it's reasonable to occasionally make ABI-incompatible changes to libhogweed, without breaking the libnettle ABI at the same time. (The opposite is also possible, but less likely to happen). In which case only one of the sonames would be bumped. There's currently no testing of libhogweed from nettle-X linking at runtime with a libnettle.so built from nettle-(X+1) or nettle-(X-1). Seems a bit tricky to set up.
Regards, /Niels
On 2020-04-12 Niels Möller nisse@lysator.liu.se wrote:
Andreas Metzler ametzler@bebt.de writes:
According to objdump -R GnuTLS uses internal symbols of both libhogweed and libnettle (e.g. _nettle_mpn_set_base256_le@HOGWEED_INTERNAL_5_0 and _nettle_write_le64@NETTLE_INTERNAL_7_0). These nettle *internal* symbols seem to be incompatible in 3.6, at least they are versioned differently (@NETTLE_INTERNAL_7_1).
Can you post a complete list of the references to internal symbols?
Hello Niels,
I am attaching
objdump -R path/to/so | grep INTERNAL
for x86_64-linux-gnu gnutls builds against nettle 3.5 and 3.6 (the latter patched to bump nettle soname, too). I am not an expert in this area - I think this generates the correct list.
Having non-working gnutls is quite a big issue, since e.g. apt depends on it.
Indeed. What options do we have? The ones I see are
- Simply bump the soname of libnettle too. I think that's ok in this case, since there are no major fixes or improvements to old features in this release. But I wouldn't want to make it a habit.
I agree that this "less than optimal". :-(
- Fix GnuTLS to not refer to internal libnettle symbols. Mark the nettle-3.6 package including libnettle.so.7 as conflicting with the non-fixed GnuTLS package.
I don't know how practical option (2) is. But for the example of nettle_write_le64, it would be a simple improvement to just include a copy in GnuTLS if it needs it. The nettle implementation is not particularly clever, and it's not intended to be part of the ABI. E.g, one todo item is to replace it with a macro expanding to memcpy, on platforms that are natively little-endian, in which case the symbol would simply disappear from the library.
I do not know either how feasible this is but the current situation, where GnuTLS relies on functions not part of the API, that could break anytime *IMHO* is not feasible either. [1]
In the medium term nettle might start enforcing saner usage by not exporting *INTERNAL* in libraries shipped in "make install"
cu Andreas
[1] (The current trigger, changing LIBNETTLE_MINOR which in turn breaks the ABI could be undone, if the affected internal symbols were compatible. But this does not solve the deeper problem.)
Andreas Metzler ametzler@bebt.de writes:
In the medium term nettle might start enforcing saner usage by not exporting *INTERNAL* in libraries shipped in "make install"
They're now accessible at link-time, with intended usage being low-level tests, and experimental code. The intention is that they should *not* be declared in installed header files, to make accidental use unlikely.
00000000001d4668 R_X86_64_JUMP_SLOT _nettle_write_le64@NETTLE_INTERNAL_7_0 00000000001d4698 R_X86_64_JUMP_SLOT _nettle_write_le32@NETTLE_INTERNAL_7_0 00000000001d48f8 R_X86_64_JUMP_SLOT _nettle_poly1305_block@NETTLE_INTERNAL_7_0
These seem to be the only references to NETTLE_INTERNAL. The first two are fairly trivial functions. I guess it would be easy to copy them into gnutls, maybe even in a debian patch.
The last is non-trivial, and it seems it is declared in the installed header poly1305.h (which is a mistake; other internal declarations were moved to foo-internal.h files a while ago, and never installed). I don't know why GnuTLS needs it; if there's a reasonable use-case, maybe it should be documented and made public.
00000000001d4940 R_X86_64_JUMP_SLOT _nettle_gmp_free_limbs@HOGWEED_INTERNAL_5_0 00000000001d4a88 R_X86_64_JUMP_SLOT _nettle_ecc_mod_mul@HOGWEED_INTERNAL_5_0 00000000001d4d68 R_X86_64_JUMP_SLOT _nettle_cnd_copy@HOGWEED_INTERNAL_5_0 00000000001d4e90 R_X86_64_JUMP_SLOT _nettle_gmp_alloc_limbs@HOGWEED_INTERNAL_5_0 00000000001d5690 R_X86_64_JUMP_SLOT _nettle_mpz_limbs_copy@HOGWEED_INTERNAL_5_0 00000000001d5958 R_X86_64_JUMP_SLOT _nettle_ecc_mod_random@HOGWEED_INTERNAL_5_0 00000000001d5a40 R_X86_64_JUMP_SLOT _nettle_ecc_mod_add@HOGWEED_INTERNAL_5_0 00000000001d5b40 R_X86_64_JUMP_SLOT _nettle_mpn_get_base256_le@HOGWEED_INTERNAL_5_0
00000000001bf310 R_X86_64_JUMP_SLOT _nettle_gmp_alloc_limbs@HOGWEED_INTERNAL_6_0 00000000001bfb48 R_X86_64_JUMP_SLOT _nettle_mpn_get_base256_le@HOGWEED_INTERNAL_6_0 00000000001c0450 R_X86_64_JUMP_SLOT _nettle_gmp_free_limbs@HOGWEED_INTERNAL_6_0 00000000001c0680 R_X86_64_JUMP_SLOT _nettle_mpn_set_base256_le@HOGWEED_INTERNAL_6_0 00000000001c0bb0 R_X86_64_JUMP_SLOT _nettle_ecc_mod_mul@HOGWEED_INTERNAL_6_0
The HOGWEED_INTERNAL references are more expected, since GnuTLS wants to support more curves than are in Nettle, and hook into the implementation. And not visible in this list, GnuTLS also depends on the layout of the internal struct ecc_curve. As I've said before, I'd wish this usage was (i) controlled by a GnuTLS ./configure argument, and (ii) disabled by default.
Regards, /Niels
Hello,
вс, 12 апр. 2020 г. в 21:37, Niels Möller nisse@lysator.liu.se:
00000000001d48f8 R_X86_64_JUMP_SLOT _nettle_poly1305_block@NETTLE_INTERNAL_7_0
The last is non-trivial, and it seems it is declared in the installed header poly1305.h (which is a mistake; other internal declarations were moved to foo-internal.h files a while ago, and never installed). I don't know why GnuTLS needs it; if there's a reasonable use-case, maybe it should be documented and made public.
This one is an interesting topic. GnuTLS uses _poly_1305 as a part of chacha backport (so there should be no need to export it just for GnuTLS). However maybe it should be exported to counterpart public poly1305_set_key() and poly1305_digest().
Regarding the MR for GnuTLS. The code is ready. I'll submit it after you decide upon _poly1305_block() and gostdsa vko.
пн, 13 апр. 2020 г. в 18:11, Dmitry Baryshkov dbaryshkov@gmail.com:
Hello,
вс, 12 апр. 2020 г. в 21:37, Niels Möller nisse@lysator.liu.se:
00000000001d48f8 R_X86_64_JUMP_SLOT _nettle_poly1305_block@NETTLE_INTERNAL_7_0
The last is non-trivial, and it seems it is declared in the installed header poly1305.h (which is a mistake; other internal declarations were moved to foo-internal.h files a while ago, and never installed). I don't know why GnuTLS needs it; if there's a reasonable use-case, maybe it should be documented and made public.
This one is an interesting topic. GnuTLS uses _poly_1305 as a part of chacha backport (so there should be no need to export it just for GnuTLS). However maybe it should be exported to counterpart public poly1305_set_key() and poly1305_digest().
Regarding the MR for GnuTLS. The code is ready. I'll submit it after you decide upon _poly1305_block() and gostdsa vko.
MR is merged.
Hello,
вс, 12 апр. 2020 г. в 20:23, Andreas Metzler ametzler@bebt.de:
On 2020-04-12 Niels Möller nisse@lysator.liu.se wrote:
Andreas Metzler ametzler@bebt.de writes:
According to objdump -R GnuTLS uses internal symbols of both libhogweed and libnettle (e.g. _nettle_mpn_set_base256_le@HOGWEED_INTERNAL_5_0 and _nettle_write_le64@NETTLE_INTERNAL_7_0). These nettle *internal* symbols seem to be incompatible in 3.6, at least they are versioned differently (@NETTLE_INTERNAL_7_1).
Can you post a complete list of the references to internal symbols?
Hello Niels,
I am attaching
objdump -R path/to/so | grep INTERNAL
for x86_64-linux-gnu gnutls builds against nettle 3.5 and 3.6 (the latter patched to bump nettle soname, too). I am not an expert in this area - I think this generates the correct list.
It looks correct to me. If nobody takes a step, I'm going to look on this in the next few days.
Having non-working gnutls is quite a big issue, since e.g. apt depends on it.
Indeed. What options do we have? The ones I see are
- Simply bump the soname of libnettle too. I think that's ok in this case, since there are no major fixes or improvements to old features in this release. But I wouldn't want to make it a habit.
I agree that this "less than optimal". :-(
- Fix GnuTLS to not refer to internal libnettle symbols. Mark the nettle-3.6 package including libnettle.so.7 as conflicting with the non-fixed GnuTLS package.
I don't know how practical option (2) is. But for the example of nettle_write_le64, it would be a simple improvement to just include a copy in GnuTLS if it needs it. The nettle implementation is not particularly clever, and it's not intended to be part of the ABI. E.g, one todo item is to replace it with a macro expanding to memcpy, on platforms that are natively little-endian, in which case the symbol would simply disappear from the library.
I do not know either how feasible this is but the current situation, where GnuTLS relies on functions not part of the API, that could break anytime *IMHO* is not feasible either. [1]
In the medium term nettle might start enforcing saner usage by not exporting *INTERNAL* in libraries shipped in "make install"
cu Andreas
[1] (The current trigger, changing LIBNETTLE_MINOR which in turn breaks the ABI could be undone, if the affected internal symbols were compatible. But this does not solve the deeper problem.) _______________________________________________ nettle-bugs mailing list nettle-bugs@lists.lysator.liu.se http://lists.lysator.liu.se/mailman/listinfo/nettle-bugs
Dmitry Baryshkov dbaryshkov@gmail.com writes:
for x86_64-linux-gnu gnutls builds against nettle 3.5 and 3.6 (the latter patched to bump nettle soname, too). I am not an expert in this area - I think this generates the correct list.
It looks correct to me. If nobody takes a step, I'm going to look on this in the next few days.
Thanks.
Regards, /Niels
Hello,
пн, 13 апр. 2020 г. в 09:52, Niels Möller nisse@lysator.liu.se:
Dmitry Baryshkov dbaryshkov@gmail.com writes:
for x86_64-linux-gnu gnutls builds against nettle 3.5 and 3.6 (the latter patched to bump nettle soname, too). I am not an expert in this area - I think this generates the correct list.
It looks correct to me. If nobody takes a step, I'm going to look on this in the next few days.
I remember your answer about not delaying the release. Any chance of getting VKO patch in? That would simplify things a bit.
Dmitry Baryshkov dbaryshkov@gmail.com writes:
I remember your answer about not delaying the release.
We can wait a few days to understand the references to NETTLE_INTERNAL. But if there's no other easy fix, I'll just bump the libnettle soname too, which should solve the immediate problem, right? Then we can still release this week.
Any chance of getting VKO patch in? That would simplify things a bit.
I'm not sure I understand the implications. Getting it in would let GnuTLS drop some of the references to HOGWEED_INTERNAL, is that what you mean with simplify, or something different? But all of them? And it doesn't look like it would affect references to NETTLE_INTERNAL.
Regards, /Niels
Hello,
пн, 13 апр. 2020 г. в 13:24, Niels Möller nisse@lysator.liu.se:
Dmitry Baryshkov dbaryshkov@gmail.com writes:
I remember your answer about not delaying the release.
We can wait a few days to understand the references to NETTLE_INTERNAL. But if there's no other easy fix, I'll just bump the libnettle soname too, which should solve the immediate problem, right? Then we can still release this week.
MR against GnuTLS will be open in a few hours.
Any chance of getting VKO patch in? That would simplify things a bit.
I'm not sure I understand the implications. Getting it in would let GnuTLS drop some of the references to HOGWEED_INTERNAL, is that what you mean with simplify, or something different? But all of them? And it doesn't look like it would affect references to NETTLE_INTERNAL.
This would allow dropping all HOGWEED_INTERNAL dependencies for GnuTLS compiled against new Nettle library.
Dmitry Baryshkov dbaryshkov@gmail.com writes:
We can wait a few days to understand the references to NETTLE_INTERNAL. But if there's no other easy fix, I'll just bump the libnettle soname too, which should solve the immediate problem, right? Then we can still release this week.
MR against GnuTLS will be open in a few hours.
Excellent!
I'm not sure I understand the implications. Getting it in would let GnuTLS drop some of the references to HOGWEED_INTERNAL, is that what you mean with simplify, or something different? But all of them? And it doesn't look like it would affect references to NETTLE_INTERNAL.
This would allow dropping all HOGWEED_INTERNAL dependencies for GnuTLS compiled against new Nettle library.
Nice. My understanding was that GnuTLS also includes additional GOST curves, and depends in the internal struct ecc_curve layout. Is that wrong? I can have a renewed look at the patch, but my gut feeling is to not add new features at this point.
Regards, /Niels
пн, 13 апр. 2020 г. в 15:08, Niels Möller nisse@lysator.liu.se:
I'm not sure I understand the implications. Getting it in would let GnuTLS drop some of the references to HOGWEED_INTERNAL, is that what you mean with simplify, or something different? But all of them? And it doesn't look like it would affect references to NETTLE_INTERNAL.
This would allow dropping all HOGWEED_INTERNAL dependencies for GnuTLS compiled against new Nettle library.
Nice. My understanding was that GnuTLS also includes additional GOST curves, and depends in the internal struct ecc_curve layout. Is that wrong? I can have a renewed look at the patch, but my gut feeling is to not add new features at this point.
GnuTLS includes only those two curves that are present in Nettle. Daiki did a great job on importing curve448 code into nettle. I'm reusing his script to bundle ecc code necessary for GOST. Two things remaining are gostdsa-vko (depends on ecc internals, etc) and gostdsa-mask (uses gmp and depends only on ecc->q.m). I can rework gostdsa-mask to remove this dependency afterwards. Thus the only significant problem is gostdsa-vko.
-- With best wishes Dmitry
nisse@lysator.liu.se (Niels Möller) writes:
Andreas Metzler ametzler@bebt.de writes:
PS: On top of this the Debian nettle *packaging* currently is broken with respect to unsynced soname bumps of libhogweed/libnettle runtime packages, which is a different issue, probably a bug.
I think it's a bug. Independent of the current issues with GnuTLS, I think it's reasonable to occasionally make ABI-incompatible changes to libhogweed, without breaking the libnettle ABI at the same time. (The opposite is also possible, but less likely to happen). In which case only one of the sonames would be bumped. There's currently no testing of libhogweed from nettle-X linking at runtime with a libnettle.so built from nettle-(X+1) or nettle-(X-1). Seems a bit tricky to set up.
This made me realize that also references to NETTLE_INTERNAL symbols from libhogweed.so would be a problem. In my closest shared build,
objdump -R libhogweed.so |grep NETTLE_INTERNAL
returns empty, but we ought to have a testcase to verify that we don't regress. Any such reference would imply that libhogweed must link to an exactly matching version of libnettle.se
Regards, /Niels
nettle-benchmark freezes on Android, because clang optimized away the bench_nothing function. Attached the proposed patch.
On Sun, Apr 5, 2020 at 2:03 PM Niels Möller nisse@lysator.liu.se wrote:
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
I intend to create a release-candidate tarball in a day or two for final testing.
nettle-3.6rc1.tar.gz tested OK on an old PowerMac with OS X 10.5 and an Intel Mac Mini with OS X 10.12.6 with SIP.
Jeff
On Mon, Apr 13, 2020 at 7:10 AM Jeffrey Walton noloader@gmail.com wrote:
On Sun, Apr 5, 2020 at 2:03 PM Niels Möller nisse@lysator.liu.se wrote:
I've updated the NEWS file, and I don't think there are any easy changes pending. Please let me know ASAP if there's anything missing in NEWS, or if there are additional changes you think ought to be included before release.
I intend to create a release-candidate tarball in a day or two for final testing.
nettle-3.6rc1.tar.gz tested OK on an old PowerMac with OS X 10.5 and an Intel Mac Mini with OS X 10.12.6 with SIP.
My bad... The Mac-Mini failed. I accidentally applied the nettle-3.5 patch to the nettle-3.6 RC.
Maybe you can switch to static linking for the tests and be done these stupid path problems that have plagued Unix and Linux for decades.
PASS: gostdsa-keygen PASS: cxx dyld: Library not loaded: /Users/jwalton/tmp/ok_to_delete/lib/libnettle.7.dylib Referenced from: /Users/jwalton/Build-Scripts/nettle-3.6rc1/testsuite/../tools/sexp-conv Reason: image not found cmp: EOF on test1.out FAIL: sexp-conv dyld: Library not loaded: /Users/jwalton/tmp/ok_to_delete/lib/libhogweed.6.dylib Referenced from: /Users/jwalton/Build-Scripts/nettle-3.6rc1/testsuite/../tools/pkcs1-conv Reason: image not found ./pkcs1-conv-test: line 26: 2890 Abort trap: 6 $EMULATOR ../tools/pkcs1-conv > testkey.priv <<EOF -----BEGIN RSA PRIVATE KEY----- MIICXQIBAAKBgQC3792bBgQ/mc8aYOFaLEJES/JipmLAeVgznob/Vrzvdcx+bl6L 6gTphctU9ToOLC049dZYW3DJ53owUmbQgqB0vvLTjM9lGSEw4oXLrp7x/XVo/fZM UcRWq5H8Z0l6KANXHwcVcsjjqPBJ6WD/Is3o9rb58GU9GZcsMO2Zoh8z6wIDAQAB AoGABP+iwRS/xs6yPyBE34N2ZY6+zomBA4QIrpZvSr8bsVI9NW5gaWL5sTLunKdx ZXMz42li4tHRVdtRicCjhKUYIShH6J3ACKnBsCHwK6MgEyuDifxhmVt/b5xQNdOL bckwBXCL/XwkIkSgrvgUk/cXcvDXSdf7cRX+tgEHlbGjWGkCQQDaS9Xm3ZTIJ1CO /chlET2Cf/e5GzC79njzeg5oDyTG7qlXZudpZv5D6NatVoIDF4gfey6NKB7DNehT ff+v9wztAkEA17TN+cuFBuZX+KT3K7J1uavGqWOypDUy/h7PINODJLzoWAWnw94H NSu6/pXo1Q1WBMQa1jB1qxJaLpBp56iBNwJAUp6JIouSl/5pOvVKNxZDVXThaSml VD6AoIX9ldzFapVBelb0FqxoZ4NkXM50/n6VgnS4tawNmIx6lb8GWq8CMQJBAM5S lMofzyggX3jnYbycQFrOYYFYaWEDubi0A27koYYcYyj+j8+bqc1D/OLSxRg0X1jD st+5DnQJY9UyMPpyhNUCQQChMjCAamJP3xC7bOoza//k7E9kvx5IZcEsQWqok5BO PSVKy/gGBeN1Q7Rj+XoybQ/SqLpfgTYRI9UpbKmpkNuq -----END RSA PRIVATE KEY----- EOF
FAIL: pkcs1-conv dyld: Library not loaded: /Users/jwalton/tmp/ok_to_delete/lib/libnettle.7.dylib Referenced from: /Users/jwalton/Build-Scripts/nettle-3.6rc1/testsuite/../tools/nettle-pbkdf2 Reason: image not found cmp: EOF on test1.out FAIL: nettle-pbkdf2 PASS: symbols PASS: dlopen ===================== 3 of 107 tests failed ===================== make[1]: *** [check] Error 1 make: *** [check] Error 2
Jeffrey Walton noloader@gmail.com writes:
nettle-3.6rc1.tar.gz tested OK on an old PowerMac with OS X 10.5 and an Intel Mac Mini with OS X 10.12.6 with SIP.
Thanks for testing. Regarding the remaining DYLD_LIBRARY_PATH problem with tests on the the mac mini, I think we'll have to live with that for now. The proper solution is likely to move setting of these enviroment variables to the same place where $EMULATOR is expanded.
Regards, /Niels
On Mon, Apr 13, 2020 at 8:16 AM Niels Möller nisse@lysator.liu.se wrote:
Jeffrey Walton noloader@gmail.com writes:
nettle-3.6rc1.tar.gz tested OK on an old PowerMac with OS X 10.5 and an Intel Mac Mini with OS X 10.12.6 with SIP.
Thanks for testing. Regarding the remaining DYLD_LIBRARY_PATH problem with tests on the the mac mini, I think we'll have to live with that for now. The proper solution is likely to move setting of these enviroment variables to the same place where $EMULATOR is expanded.
The failure will also affect some of the BSDs. I know it affects NetBSD, too.
Why live with the failure when you have a patch that fixes it at https://github.com/noloader/Build-Scripts/blob/master/patch/nettle.patch?
dirname is Posix. It should be present just about everywhere.
Jeff
Jeffrey Walton noloader@gmail.com writes:
The failure will also affect some of the BSDs. I know it affects NetBSD, too.
I made a commit a while ago to always use an absolute name (based on autoconf's abs_top_builddir). See https://git.lysator.liu.se/nettle/nettle/-/commit/b3474802b81df6db83492adf25...
As far as I'm aware, that should fix BSDs and other systems disliking relative names in LD_LIRBARY_PATH. If you can verify the rc1 tarball on NetBSD (I only have FreeBSD nearby), that would be nice.
The remaining problem I'm aware of is the security "feature" to sometimes discard DYLD_LIBRARY_PATH completely. My understanding is that problem is limited to MacOS.
Regards, /Niels
On Mon, Apr 13, 2020 at 10:09 AM Niels Möller nisse@lysator.liu.se wrote:
Jeffrey Walton noloader@gmail.com writes:
The failure will also affect some of the BSDs. I know it affects NetBSD, too.
I made a commit a while ago to always use an absolute name (based on autoconf's abs_top_builddir). See https://git.lysator.liu.se/nettle/nettle/-/commit/b3474802b81df6db83492adf25...
As far as I'm aware, that should fix BSDs and other systems disliking relative names in LD_LIRBARY_PATH. If you can verify the rc1 tarball on NetBSD (I only have FreeBSD nearby), that would be nice.
I'll have to take your word for it. I upgraded to NetBSD 9, and now iConv is broken. I can't get beyond its install recipe. A broken iConv or Gettext means I can't build the necessary dependencies to test Nettle.
The remaining problem I'm aware of is the security "feature" to sometimes discard DYLD_LIBRARY_PATH completely. My understanding is that problem is limited to MacOS.
Actually a lot of software works with SIP on OS X. Nearly all software I use and test works as expected, which is about 120 packages. The problem seems to be limited to Git, GMP and Nettle. I think you want to get on the other side of the fence with all the software that works as expected.
Is there any reason you refuse to fix things? I'd be interested in knowing it.
You are also welcomed to an account on my Mac Mini. Send over your authorized_keys file.
Jeff
Jeffrey Walton noloader@gmail.com writes:
On Mon, Apr 13, 2020 at 10:09 AM Niels Möller nisse@lysator.liu.se wrote:
As far as I'm aware, that should fix BSDs and other systems disliking relative names in LD_LIRBARY_PATH. If you can verify the rc1 tarball on NetBSD (I only have FreeBSD nearby), that would be nice.
I'll have to take your word for it. I upgraded to NetBSD 9, and now iConv is broken. I can't get beyond its install recipe. A broken iConv or Gettext means I can't build the necessary dependencies to test Nettle.
Sounds a bit frustrating... Building the tarball shouldn't need more than a C compiler, GNU make and m4 (if you configure with --enable-mini-gmp, you can get reasonable test coverage without installing the GMP library).
Actually a lot of software works with SIP on OS X. Nearly all software I use and test works as expected, which is about 120 packages. The problem seems to be limited to Git, GMP and Nettle. I think you want to get on the other side of the fence with all the software that works as expected.
As I see it, it's a problem affecting a few of Nettle's test. Once installed, it should work fine. It might be a valid fix to just disable those tests.
I mean, DYLD_LIBRARY_PATH is useful mainly for developers, and it's not unreasonable to configure the system differently on a developer machine than on a vanilla server or desktop. On GNU/Linux, e.g, I'd recommend developers to enable traditional core dumps and use a liberal setting for sysctl kernel.yama.ptrace_scope.
Is there any reason you refuse to fix things?
Just that (i) it's going to take me a few hours to test and fix this properly, and I don't want to allocate those hours before the release, and (ii) I generally give lower priority to supporting proprietary systems.
You are also welcomed to an account on my Mac Mini. Send over your authorized_keys file.
Thanks for the offer, but by (ii), I don't want to spend my time working on mac os.
Regards, /Niels
On Mon, Apr 13, 2020 at 12:54 PM Niels Möller nisse@lysator.liu.se wrote:
Jeffrey Walton noloader@gmail.com writes:
On Mon, Apr 13, 2020 at 10:09 AM Niels Möller nisse@lysator.liu.se wrote:
As far as I'm aware, that should fix BSDs and other systems disliking relative names in LD_LIRBARY_PATH. If you can verify the rc1 tarball on NetBSD (I only have FreeBSD nearby), that would be nice.
I'll have to take your word for it. I upgraded to NetBSD 9, and now iConv is broken. I can't get beyond its install recipe. A broken iConv or Gettext means I can't build the necessary dependencies to test Nettle.
Sounds a bit frustrating... Building the tarball shouldn't need more than a C compiler, GNU make and m4 (if you configure with --enable-mini-gmp, you can get reasonable test coverage without installing the GMP library).
It is Nettle's OpenSSL dependency that is causing the trouble. OpenSSL depends on Unistring and Unbound, and they bring in the extra gear.
Actually a lot of software works with SIP on OS X. Nearly all software I use and test works as expected, which is about 120 packages. The problem seems to be limited to Git, GMP and Nettle. I think you want to get on the other side of the fence with all the software that works as expected.
As I see it, it's a problem affecting a few of Nettle's test. Once installed, it should work fine. It might be a valid fix to just disable those tests.
Then maybe you can drop the unneeded test?
It is important that folks can do a 'make && make check' and have things "just work". I know some organizations that needs stuff to work, or it requires a management sign-off complete with change control entries. It is an administrative nightmare.
I mean, DYLD_LIBRARY_PATH is useful mainly for developers, and it's not unreasonable to configure the system differently on a developer machine than on a vanilla server or desktop. On GNU/Linux, e.g, I'd recommend developers to enable traditional core dumps and use a liberal setting for sysctl kernel.yama.ptrace_scope.
Is there any reason you refuse to fix things?
Just that (i) it's going to take me a few hours to test and fix this properly, and I don't want to allocate those hours before the release, and (ii) I generally give lower priority to supporting proprietary systems.
I don't think copy/paste is going to take several hours. And a good CI pipeline should handle the testing for you. If you need hours of testing, then that probably points to a defect in the engineering process. The process has gaps where CI testing should occur.
Apple is mostly open sourced. You can find the source code at opensource.apple.com. I believe that's how distros like Hackintosh make their stuff work. They start with Apple sources and massage them.
Jeff
Jeffrey Walton noloader@gmail.com writes:
It is Nettle's OpenSSL dependency that is causing the trouble. OpenSSL depends on Unistring and Unbound, and they bring in the extra gear.
That's for benchmarking only. It's supposed to be autodetected and used only if available on the build system, but you can also run configure with --disable-openssl to explicitly disable it.
I don't think copy/paste is going to take several hours. And a good CI pipeline should handle the testing for you. If you need hours of testing, then that probably points to a defect in the engineering process. The process has gaps where CI testing should occur.
You asked "Is there any reason you refuse to fix things?" (which is kind of provocative). I tried to answer honestly. I refuse to argue this further.
Regards, /Niels
nettle-bugs@lists.lysator.liu.se
-
Andreas Metzler -
Daiki Ueno -
Dmitry Baryshkov -
Jeffrey Walton -
nisse＠lysator.liu.se -
Stephen R. van den Berg