Isn't also TYPE_SUBTYPE wrong in the big endian case? It seems to me that on a 64bit big endian machine both subtype and type would always end up being 0.
Correct. Fixed last night.
like so?
#define TYPE_SUBTYPE(X,Y) (((Y)|((X)<<16)) << (sizeof(ptrdiff_t) - 32))
No, sizeof(ptrdiff_t) needs to be scaled by a factor 8...
The above version also requires casts to widen X and Y to not generate zeros anyway. Another related issue is that X and Y are often constants, and several 64-bit architectures do not have support for full 64-bit immediate constants, so I changed the field to always be 32-bit in the big-endian case.
/grubba