imported fix from CROSS upstream: endianness-aware csprng

[rtjk]

Fixes #1961.

The rejection sampling functions in CROSS rely on pointer casting, which caused the CSPRNG to generate different numbers on big-endian versus little-endian architectures, resulting in different KATs. This fix uses the compiler flag __BYTE_ORDER__ to detect endianness and swap the bytes when necessary. Thank you @bhess for pointing us in the right direction!

• Does this PR change the input/output behaviour of a cryptographic algorithm (i.e., does it change known answer test values)? (If so, a version bump will be required from x.y.z to x.(y+1).0.)
• Does this PR change the list of algorithms available -- either adding, removing, or renaming? Does this PR otherwise change an API? (If so, PRs in fully supported downstream projects dependent on these, i.e., oqs-provider will also need to be ready for review and merge by the time this is merged.)

[SWilson4]

Thanks for making the fix, @rtjk! I suggest that we hold merge for #1960 so that the s390x tests run on this PR. Perhaps you could rebase on @bhess's branch for that PR for now in case there are CI errors that need to be resolved (e.g., due to a gcc or clang version).

[rtjk]

@SWilson4 I rebased on branch bhe-reenable-travis and turned s390x back on for CROSS. It looks like Travis is running.

[rtjk]

Ouch! It still failed ... time to debug some more.

[SWilson4]

Ouch! It still failed ... time to debug some more.

FYI, looks like the s390x build uses gcc 9—does this older version support all the features you're using?

[rtjk]

It turns out there were a few more instances where type punning was used, causing an endianness issue. I've fixed these, so Travis CI should now pass. I also added two lines to copy_from_upstream.py to create template files if they aren’t already in place. Is this okay?

[baentsch]

I also added two lines to copy_from_upstream.py to create template files if they aren’t already in place. Is this okay?

Thanks for highlighting this @rtjk ! I don't think this is a good approach: Missing files in code generation imo are indicative of an incorrect code base and should lead to an error condition triggering a proper fix (manually adding the file(s) in question): Reason for my caution is that the code generation is so central to the proper operation of liboqs that I'm very wary of too much automation in this area.

Automated template file generation removed: Thanks @rtjk!

[baentsch]

Thanks for the fix @rtjk. I can't claim to have read all changes but defer to the tests passing as this code being OK: As with all core crypto code in liboqs the ultimate responsibility for correctness is with the upstream. Allow me to ask the question when/whether this code is going to flow upstream again (such as to get rid of the various "fix" statements in the version)?

[bhess]

Thanks for the update @rtjk!

[rtjk]

Allow me to ask the question when/whether this code is going to flow upstream again (such as to get rid of the various "fix" statements in the version)?

The plan is to release a new version of CROSS before moving on to NIST's second round (deadline January 17, 2025). This will consolidate all the changes we made since CROSS 1.2.

[SWilson4]

Thanks for the fix @rtjk!