Hello Niels,

On Tue, Feb 18, 2020 at 12:25:05PM +0100, Niels Möller wrote:

...

Let me know if there are any outstanding issues regarding armeb and I'll look into them.

To keep it in working shape, it would help a lot with additional tests in .gitlab-ci. I'm not really familiar with the cross setup used for arm, mips, and aarch64, Nikos helped with all that. But maybe armeb can be added in the same way?

That's exactly the rabbithole I went down and got lost in the last time: armeb is so niche that I'm not aware of any ready-to-install mainstream distribution for it. I am running Gentoo but that's not easy to maintain in a CI setting. I got a Debian-based qemu-user-binfmt_misc-based image working but that was very brittle as well because in the end it also needed to bootstrap the necessary components from source. So as it stands I cannot offer any improved CI for armeb that's maintainable with reasonable effort.

On the other hand: The armeb asm changes (for the most part) are so symmetric that any breakage would likely affect both armeb and armel.

Hi Niels,

On Tue, Feb 18, 2020 at 09:45:01PM +0100, Niels Möller wrote:

...

That's exactly the rabbithole I went down and got lost in the last time: armeb is so niche that I'm not aware of any ready-to-install mainstream distribution for it. I am running Gentoo but that's not easy to maintain in a CI setting. I got a Debian-based qemu-user-binfmt_misc-based image working but that was very brittle as well because in the end it also needed to bootstrap the necessary components from source.

I think there's been some effort to make debian easier to bootstrap for new architectures, but I don't know current state. Gnu guix people are also taking bootstrap very seriously.

I had been working with rebootstrap. But that's not really meant to be able to reliably bootstrap from source at any given point in time but instead to show up bootstrapping problems as a CI measure for the Debian distribution iteself. That means that it breaks on an almost daily basis and an armeb image built with it will need considerable attention to build again after a couple of weeks or months.

...

So as it stands I cannot offer any improved CI for armeb that's maintainable with reasonable effort.

Then you (or other volunteers with interest in armeb) needs to do manual tests of Nettle from time to time, to keep it in working shape.

I will again look into some sort of CI for armeb.

Hi Dmitry,

On Fri, Feb 21, 2020 at 07:02:54PM +0300, Dmitry Baryshkov wrote:

...

The asm for arm certainly needed adjustment to work on armeb. See https://lists.lysator.liu.se/pipermail/nettle-bugs/2018/007280.html.

What is the target hardware for armeb?

I am *using* it on Cubieboard 2's which are armv7. I don't think there's eveer been an actual hardware support statement.

I have tried building Nettle for armv5teb (thinking about NSLU2) and got failures because of `rev` instruction usage (present only on ARMv6/v7).

Huh, right. Most of the revs are in the v6 and neon subdirectories which should be fine because they are only activated for v6+. So we'd need to eliminate the revs in arm/memxor.asm and arm/memxor3.asm to support armv5teb.

Andrew, if you are testing it with qemu, there well might be emulation deficiencies.

I could dust off the old binfmt_misc setup for armv6veb and give that a whirl if it's of any help. I'm quite certain I initially worked around the asm problems on armeb by disabling asm when cross- and binfmt_misc-qemu-natively-compiling the userland for my Cubies. So I have likely never run the asm under qemu-user nor qemu-system.

Hi Niels,

On Sat, Feb 22, 2020 at 07:58:10AM +0100, Niels Möller wrote:

...

I am *using* it on Cubieboard 2's which are armv7. I don't think there's eveer been an actual hardware support statement.

Question is, what devices are out where, there armv5 support would be useful?

Well, arm devices are quite long-lived. I myself had my Marvel Kirkwood based NASes (which were armv5 but never ran BE) running way longer than I would have kept a similar x86 system around. These were 300Mhz and 128MiB of RAM, so totally underpowered, but they did their job. But with the current hype about aarch64 I'd expect the armv5 installation base to be shrinking.

...

...

I have tried building Nettle for armv5teb (thinking about NSLU2) and got failures because of `rev` instruction usage (present only on ARMv6/v7).

Lookingup NSLU2 on wikipedia, it seems it features a "Intel XScale cpu" featuring an "ARMv5TE ISA without the floating point instructions." I take it this can be run in BE mode?

The Intel XScale and IXP4xx are BE by default, IIRC. For them it's even a bit of a hike to run them LE since all their companion controllers and firmware blobs are BE as well.

...

Huh, right. Most of the revs are in the v6 and neon subdirectories which should be fine because they are only activated for v6+. So we'd need to eliminate the revs in arm/memxor.asm and arm/memxor3.asm to support armv5teb.

Options are:

1. Do nothing, let armbe assembly require armv6, and recommend --disable-assembly for armv5be.

asm was broken for *any* armeb platform before the changes. Now it's working fine for armv6+, but, ...

2. Eliminate use of rev in the armbe code.

... I've been looking at the revs and they now strike me as taking the easy way out anyway. They work around the implicit LE order in which some remaining bytes from a buffer are saved byte-wise. This should be doable without the revs by making the store work in BE fashion in BE mode. I plan to look into this but would certainly appreciate any help Dmitry and Андрей might be able to give here since it's not as if I've been doing nothing but arm asm the last couple of years - rather the opposite.

3. Somehow arrange in configure so that the memxor files in particular aren't used when targetting armv5be. If you want to test the effects before hacking configure.ac, I think it should be as easy as removing the memxor*.asm symlinks after running configure.

Nah, we can do better than that. :)

...

I could dust off the old binfmt_misc setup for armv6veb and give that a

That'd be armv7veb, BTW.

...

whirl if it's of any help. I'm quite certain I initially worked around the asm problems on armeb by disabling asm when cross- and binfmt_misc-qemu-natively-compiling the userland for my Cubies. So I have likely never run the asm under qemu-user nor qemu-system.

Is there some problem with configuring qemu to emulate armv6? I think that's what is used when testing non-BE ARM, but I'm not 100% sure about that.

No, qemu-armeb should be able to do armv5eb and armv6eb just fine. It's just that I have experience and existing configs for this armv7veb setup since that's what I'm running on my cubies. Once I've sorted that, I should be able to use it to compile to armv5 and armv6 to see what breaks at the compilation stage already. I could also run these armv5 and armv6 binaries through qemu-armeb to see what breaks at runtime. I should also be able to produce similar setups for armv5 and armv6 to uncover any remaining incompatibilities which are somehow masked on the more capable armv7veb setup.

As a ramble, since I wrote it up before re-reading and -interpreting your question: That's exactly what I mean with "cross- and binfmt_misc-qemu-natively-compiling" setup: With Gentoo I can quite easily create a system-integrated cross-compiler toolchain for armeb and the package manager can use it to cross-compile a rootfs containing a native armeb toolchain, nettle and all its dependencies. This can be taken just as far as cross-compiling nettle and then running the testsuite under qemu-armeb, the user emulation mode where qemu-armeb acts as interpreter for the foreign binary executable format via binfmt_misc and emulates the Linux syscalls for the userland programs. It can also go as far as chrooting into the rootfs with qemu-armb executing a whole armeb userland via binfmt_misc and then using the native toolchain installed therein to compile everything including nettle natively (from its point of view).

The latter is what current arm CI is doing IIRC but based on Debian's native support for arm combined with its capability of installing multiple architectures into a single rootfs at once (multillib).

Something similar to the Gentoo-approach should be doable with buildroot which I am currently looking into for armeb CI purposes: buildroot could generate a cross-compiler toolchain and rootfs for nettle CI. This would be much more maintainable than Debian's rebootstrap since armeb is an actual supported target platform of buildroot. A minor problem with it (and likely similar projects such as openwrt) is that they're focused on cross-compilation for an embedded system. So AFAI{K,CT,CS} they are not able to produce a native toolchain for the target platform which rules out qemu variant 2 above.

If anyone knows any other project which produces an armeb binary distribution or specializes in bootstrapping cross-compilers and rootfses that support armeb natively, I'd very much appreciate the pointer.

Hi Dmitry,

On Sun, Feb 23, 2020 at 07:08:54PM +0300, Dmitry Baryshkov wrote:

If I remember correctly, ARMv5 be was BE-32

Yep. So is the buildroot output I get for arm926 armeb. I just need the BE8 workaround on my BE8 armv7veb system to test the armv5 compilate.

On Sun, Feb 23, 2020 at 07:09:43PM +0300, Dmitry Baryshkov wrote:

I will check with fresh Yocto build later or tomorrow.

Thanks!

Could Yocto be used for CI then? Do they do any kind of binary releases for armeb? How long and voluminous is a build of an armeb system with and without a native toolchain?

Hello,

пн, 2 мар. 2020 г. в 21:44, Niels Möller nisse@lysator.liu.se:

I've been offline, skiiing, for a week, and I haven't yet caught up we email. http://www.lysator.liu.se/~nisse/misc/s%C3%A5nfj%C3%A4llet.jpg

Michael Weiser michael.weiser@gmx.de writes:

...

Hi Niels,

On Sat, Feb 22, 2020 at 07:58:10AM +0100, Niels Möller wrote:

...

Lookingup NSLU2 on wikipedia, it seems it features a "Intel XScale cpu" featuring an "ARMv5TE ISA without the floating point instructions." I take it this can be run in BE mode?

The Intel XScale and IXP4xx are BE by default, IIRC. For them it's even a bit of a hike to run them LE since all their companion controllers and firmware blobs are BE as well.

Ok, so than ARM BE may be more relevant for these older chips than for newer ones, which are almost always run in LE mode?

It has become mostly irrelevant after authors of SlugOS have invented a way to run LE binaries on NSLU2. Working in embedded hardware area I haven't seen an armeb project for ages.

...

...

2. Eliminate use of rev in the armbe code.

... I've been looking at the revs and they now strike me as taking the easy way out anyway. They work around the implicit LE order in which some remaining bytes from a buffer are saved byte-wise. This should be doable without the revs by making the store work in BE fashion in BE mode. I plan to look into this but would certainly appreciate any help Dmitry and Андрей might be able to give here since it's not as if I've been doing nothing but arm asm the last couple of years - rather the opposite.

I've only had a quick look at the patch later in the thread, but I agree that if we have a loop to store a byte at a time anyway, it should be straight forward to "just" reverse the shifting direction depending on byte order.

...

...

3. Somehow arrange in configure so that the memxor files in particular aren't used when targetting armv5be. If you want to test the effects before hacking configure.ac, I think it should be as easy as removing the memxor*.asm symlinks after running configure.

Nah, we can do better than that. :)

But relatedly, it looks like arm/v6/sha1-compress.asm and arm/v6/sha256-compress.asm need ARMv6 only on little-endian hosts. So it might be worthwhile to try to use these files on armv5be? I don't see an obvious great way to set it up though.

One way might be to add a arm/be/sha1-compress.asm file which simply includes the v6 file, and then add arm/be to the asm_path where appropriate.

I will take a look on fixing the code for armv5. Replacing rev with bit fiddling should be easy.