Hi, I wonder if it would make sense to try to cut a release pretty soon (and without any arm64 changes)? Previous release was made end of April, and there's been quite a few improvements since then.
I wonder if it is possible to make a release in time for the upcoming debian release? https://lists.debian.org/debian-devel-announce/2020/11/msg00002.html Nettle-3.7 should be abi-compatible, and with unchanged soname, so I'm not sure if it would count as a "transition" in the debian world.
Regards, /Niels
On 2020-12-15 Niels Möller wrote:
Hi, I wonder if it would make sense to try to cut a release pretty soon (and without any arm64 changes)? Previous release was made end of April, and there's been quite a few improvements since then.
I wonder if it is possible to make a release in time for the upcoming debian release? https://lists.debian.org/debian-devel-announce/2020/11/msg00002.html Nettle-3.7 should be abi-compatible, and with unchanged soname, so I'm not sure if it would count as a "transition" in the debian world.
Hello,
it would not count as transition https://release.debian.org/bullseye/freeze_policy.html#transition
| When changes in a package cause the need for changes in other | packages, we call this a 'transition'. The most common example is | a library transition. Transitions require coordination between | maintainers and can take a long time to finish, especially when bugs | are discovered during the transition. Unfinished transitions can block | the migration of unrelated fixes in the packages involved in the | transition. From the start of the Transition and (build-)essentials | Freeze, it's no longer appropriate to start new transitions.
A "library transition" happens after a soname-bump, all reverse dependencies need to be rebuilt against the new version and if the API also changed incompatibly require source changes.
cu And- I am not hinting that delaying a release until after the start of the transition freeze would be welcome ;-)-reas
Andreas Metzler ametzler@bebt.de writes:
it would not count as transition https://release.debian.org/bullseye/freeze_policy.html#transition
Thanks, good to know.
cu And- I am not hinting that delaying a release until after the start of the transition freeze would be welcome ;-)-reas
Right, if we want to have a good chance to get into the release, we'd better be quick.
On a different thread,
Maamoun TK maamoun.tk@googlemail.com writes:
Isn't starting a new version with both powerpc64 and aarch64 changes is more reasonable? I'm not sure here, if there are a few commits before powerpc64 patches then it makes sense to wrap up the current version with powerpc64 code.
I think the powerpc64 code is in good shape now, and ready for release. Are you aware of anything that needs fixing?
We could aim to have another release in a couple of months, with optimizations for aarch64 and maybe others too. I don't want to try to add any new features before the release, if we aim to get done in time for debian freeze.
And there's also the bcrypt code, added half a year ago, which it would be nice to get into a released version.
One problem with the current state is that big-endian arm is most likely broken. I don't want to delay the release for that though, since I'm not able to fix it. If anyone is able to test and fix, soon would be a good time for it (but I'd consider a bug-fix release with just those fixes anytime later).
I'm appending my draft NEWS entries.
Regards, /Niels
NEWS for the Nettle 3.7 release
This release adds one new feature, the bcrypt password hashing function, and lots of optimizations. The release adds PowerPC64 assembly for a few algorithms, resulting in great speedups. Benchmarked on a Power9 machine, speedup was 13 times for AES256-CTR and AES256-GCM, and 3.5 times for Chacha.
New features:
* Support for bcrypt, contributed by Stephen R. van den Berg.
Optimizations:
* Much faster AES and GCM on PowerPC64 processors supporting the corresponding crypto extensions. Contributed by Mamone Tarsha.
* Speed of Chacha improved on PowerPC64, x86_64 and ARM Neon.
* Speed of Salsa20 improved on x86_64 and ARM Neon.
* Overhaul of some elliptic curve primitives, improving ECDSA signature speed.
Miscellaneous:
* Use a few more gmp-6.1 functions: mpn_cnd_add_n, mpn_cnd_sub_n, mpn_cnd_swap. Delete corresponding internal Nettle functions.
* Convert all assembly files to use the default m4 quote characters.
I think the powerpc64 code is in good shape now, and ready for release. Are you aware of anything that needs fixing?
No, all what I can think of are a couple of issues that make the powerpc64 code more stable (you can check their merge requests in the repo).
One problem with the current state is that big-endian arm is most likely broken. I don't want to delay the release for that though, since I'm not able to fix it. If anyone is able to test and fix, soon would be a good time for it (but I'd consider a bug-fix release with just those fixes anytime later).
I also delayed the big-endian support of GHASH algorithm for Aarch64 because I don't have access to big-endian device for testing. Do you think using cross-compile and qemu for the test would work? If so I will gladly contribute with those fixes.
regards, Mamone
Hi Niels and Maamoun,
On Fri, Dec 18, 2020 at 07:18:24PM +0200, Maamoun TK wrote:
One problem with the current state is that big-endian arm is most likely broken. I don't want to delay the release for that though, since I'm not able to fix it. If anyone is able to test and fix, soon would be a good time for it (but I'd consider a bug-fix release with just those fixes anytime later).
With the prospect of a new release on the horizon I finally gave myself a push and dug into chacha-3core yesterday. Porting over the basic IF_[LB]E mechanism from chacha-core-internal was easy and fixed up the first of the three interleaved blocks right away. For the other two I am still in the process of wrapping my head around how the interleaving works and how it would need some adjustment for BE.
Don't delay the release on my part because I guess I'm still the only joker running big-endian arm. ;)
I also delayed the big-endian support of GHASH algorithm for Aarch64 because I don't have access to big-endian device for testing. Do you think using cross-compile and qemu for the test would work? If so I will gladly contribute with those fixes.
qemu-user works nicely for aarch64_be. I used it to semi-natively compile a whole aarch64 userland. I could dust off pine64 board that is running that userland now for real-world testing if you like.
On 19/12/20 5:29 am, Niels Möller wrote:
Andreas Metzler writes:
it would not count as transition https://release.debian.org/bullseye/freeze_policy.html#transition
...
- Support for bcrypt, contributed by Stephen R. van den Berg.
I would have though this needs a soname bump. Otherwise software built to use bcrypt might try to link to the old version with same soname.
Amos
Amos Jeffries squid3@treenet.co.nz writes:
I would have though this needs a soname bump. Otherwise software built to use bcrypt might try to link to the old version with same soname.
My understanding is that one usually doesn't bump the soname when adding new functions.
I was trying to look at how it has been done in gmp, but there it's all done via automake/libtool, which makes me slightly confused. But I think the current soname libgmp10.so dates from gmp-5.0.0 released 10 years ago, and there's been quite a few new public gmp functions added since then, e.g., the various mnp_sec_* functions used in Nettle.
Then the soname only ensures that upgrading *the library* doesn't break installed and working applications. Upgrading an application (to a new version depending on new library functions) could break, and require library to be upgraded first. Preventing that kind of breakage has to be prevented by other means, e.g., dependencies in the packaging system.
Regards, /Niels
On Sat, Dec 19, 2020 at 4:44 AM Niels Möller nisse@lysator.liu.se wrote:
Amos Jeffries squid3@treenet.co.nz writes:
I would have though this needs a soname bump. Otherwise software built to use bcrypt might try to link to the old version with same soname.
My understanding is that one usually doesn't bump the soname when adding new functions.
Also see https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.....
Jeff
Jeffrey Walton noloader@gmail.com writes:
Also see https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.....
It's not entirely clear to me how libtool versions maps to soname, but from looking at GMP, I guess the number embedded in the soname is current - age. So for gmp-6.1.2, the libtool version is 13:2:3, current - age = 13 - 3 = 10, and soname is libgmp10.so.
Is that right? If so, the case
2. Programs using the previous version may use the new version as drop-in replacement, but programs using the new version may use APIs not present in the previous one. In other words, a program linking against the new version may fail with "unresolved symbols" if linking against the old version at runtime: set revision to 0, bump current and age.
would always leave soname unchanged, since the bump of both current and age cancel out when construction the soname.
The recent release of gmp-6.2.0 is of this form, and has libtool version 14:0:4, 14 - 4 = 10.
Am I getting the libtool covnentions right?
Regards, /Niels
On 2020-12-19 Niels Möller wrote:
Amos Jeffries squid3@treenet.co.nz writes:
I would have though this needs a soname bump. Otherwise software built to use bcrypt might try to link to the old version with same soname.
My understanding is that one usually doesn't bump the soname when adding new functions.
[...]
Then the soname only ensures that upgrading *the library* doesn't break installed and working applications. Upgrading an application (to a new version depending on new library functions) could break, and require library to be upgraded first. Preventing that kind of breakage has to be prevented by other means, e.g., dependencies in the packaging system.
Hello Niels,
that mirrors my understanding perfectly. Adding functions does not require a soname bump; removing functions or changing the number or type of arguments a function expects does[1].
cu Andreas
[1] With versioned symbols a library can provide both the old version of a symbol for applications built against earlier library versions and the newer interface for freshly linked apps to avoid breaking legacy apps and therefore avoid a soname bump in most cases. However this effort only pays off when a soname bump would bring loads of pain, i.e. glibc.
On Tue, Dec 15, 2020 at 10:47 AM Niels Möller nisse@lysator.liu.se wrote:
Hi, I wonder if it would make sense to try to cut a release pretty soon (and without any arm64 changes)? Previous release was made end of April, and there's been quite a few improvements since then.
It would be nice if cpu feature detection for fat builds made it into the next release. It is easier than arm64, and it is needed to avoid crashes at runtime.
Jeff
Jeffrey Walton noloader@gmail.com writes:
It would be nice if cpu feature detection for fat builds made it into the next release. It is easier than arm64, and it is needed to avoid crashes at runtime.
Can you be more specific on what the problem is? I'm aware of only one problem, and that's fat-arm.c relying on /proc/cpuinfo. Switching to using getauxval has been suggested. Blockers as I recall:
(i) it wasn't clear how to use getauxval to figure out if we're running on armv6 or older, and
(ii) it wasn't clear what's the right way to do it on android.
So this could surely be improved, but I'm not aware of any easy fix.
Regards, /Niels
On Sat, Dec 19, 2020 at 3:58 AM Niels Möller nisse@lysator.liu.se wrote:
Jeffrey Walton noloader@gmail.com writes:
It would be nice if cpu feature detection for fat builds made it into the next release. It is easier than arm64, and it is needed to avoid crashes at runtime.
Can you be more specific on what the problem is? I'm aware of only one problem, and that's fat-arm.c relying on /proc/cpuinfo. Switching to using getauxval has been suggested. Blockers as I recall:
https://lists.lysator.liu.se/pipermail/nettle-bugs/2020/008762.html
Jeff
Jeffrey Walton noloader@gmail.com writes:
https://lists.lysator.liu.se/pipermail/nettle-bugs/2020/008762.html
If you don't use --enable-fat, there will be no runtime detection. That's exactly as intended. Maybe your real problem is
"On other platforms, --enable-fat causes SHA to be built even when the compiler does not support SHA. To avoid the compile error I dropped --enable-fat."
Can you give some more details of that error? The C compiler shouldn't need to know anything about the sha or aes instructions, so your checks of compiler support doesn't seem that relevant. The *assembler* needs to recognize the instructions, and that could potentially be worked around by coding instructions as equivalent byte sequences instead.
Regards, /Niels
nisse@lysator.liu.se (Niels Möller) writes:
Hi, I wonder if it would make sense to try to cut a release pretty soon (and without any arm64 changes)? Previous release was made end of April, and there's been quite a few improvements since then.
I've pushed a couple of changes to increase version numbers, and add to the NEWS file.
One question: Is this a good time to make --enable-fat the default? It's been available for a couple of versions, I think using it is quite common, and I've not heard about any problems (except build failures if assembler doesn't recognize newish instructions, which at least isn't subtle).
Regards, /Niels
Since there are many variants of architectures, some are supported and others could be supported in the future, it becomes a little annoying for end-users to browse the configurable options and enable specific options to get maximum speed for corresponding algorithms so here --enable-fat comes in handy. I like the idea of making --enable-fat default to get over the case that the user didn't notice it or unaware of its advantages and disadvantages and therefore hesitant to choose it, so I think it's a good time to enable fat option by default if things are stable in assembly side, you got my vote.
regards, Mamone
On Sat, Dec 26, 2020 at 6:13 PM Niels Möller nisse@lysator.liu.se wrote:
nisse@lysator.liu.se (Niels Möller) writes:
Hi, I wonder if it would make sense to try to cut a release pretty soon (and without any arm64 changes)? Previous release was made end of April, and there's been quite a few improvements since then.
I've pushed a couple of changes to increase version numbers, and add to the NEWS file.
One question: Is this a good time to make --enable-fat the default? It's been available for a couple of versions, I think using it is quite common, and I've not heard about any problems (except build failures if assembler doesn't recognize newish instructions, which at least isn't subtle).
Regards, /Niels
-- Niels Möller. PGP-encrypted email is preferred. Keyid 368C6677. Internet email is subject to wholesale government surveillance. _______________________________________________ nettle-bugs mailing list nettle-bugs@lists.lysator.liu.se http://lists.lysator.liu.se/mailman/listinfo/nettle-bugs
I've made a release candidate tarball, see http://www.lysator.liu.se/~nisse/archive/nettle-3.7rc1.tar.gz
Intend to release in a day or two. I mostly trust the ci system, so I will do only a few tests on the tarball to try to catch any packaging mistakes. As usual, any additional testing highly appreciated.
Regards, /Niels
nettle-bugs@lists.lysator.liu.se
-
Amos Jeffries -
Andreas Metzler -
Jeffrey Walton -
Maamoun TK -
Michael Weiser -
nisse＠lysator.liu.se