Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor bevy_mikktspace to entirely safe rust #9050

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

LaylBongers
Copy link

Objective

4 years ago, with a heavy amount of hand-translation, I converted mikktspace.c to pure-rust so I could load GLTF files in-browser. I had intended this to be a quick and hacky conversion, to be gradually refactored over time, but to my surprise it's still in use in almost its original unsafe form! No more!

Fixes #7372

Solution

This pull requests finally converts the remaining unsafe code to safe rust, fixes rustc warnings, and most clippy warnings. I also re-added some comments that had gotten lost in the translation, and re-added flags.

The algorithm is still the same, the biggest changes made change it to use relative offsets into buffers instead of direct pointers.

@github-actions
Copy link
Contributor

github-actions bot commented Jul 5, 2023

Welcome, new contributor!

Please make sure you've read our contributing guide and we look forward to reviewing your pull request shortly ✨

@DJMcNab
Copy link
Member

DJMcNab commented Jul 5, 2023

A couple of results from my prior attempt (#4932) - this really needs proper testing. That is, comparisons that it gets exactly the same results as the original C library.
That should be both a wide range of known examples, and probably fuzzing (although the original C code does not have principled NaN handling, so we need to be careful there)
Certainly that we have the one test is good, but a full rewrite comes a much higher risk of disagreements here

I also don't particularly like that we own this code, although I also realise that there's not really any other potential steward.
Because of the testing requirements, it might make more sense for it to live in a different repo in this organisation, although working out the release process there is more awkward.

@Selene-Amanita Selene-Amanita added A-Rendering Drawing game state to the screen P-Unsound A bug that results in undefined compiler behavior labels Jul 5, 2023
@LaylBongers
Copy link
Author

I definitely agree with this, we need a test bench that validates the Rust port against references with non-trivial models generated using the original C library.

@LaylBongers
Copy link
Author

I have just now added a reference generator to the PR, including result files from said generator.
This is not yet used currently, but this will be the basis of new reference tests to make sure the Rust implementation of MikkTSpace gives identical results.

@LaylBongers
Copy link
Author

Regression tests validated against the original C are now included!

@LaylBongers
Copy link
Author

Squashed and cleaned, this should now be ready for review and merge!

@nicopap nicopap added the C-Code-Quality A section of code that is hard to understand or change label Jul 17, 2023
@mockersf
Copy link
Member

do you have an idea perf-wise how it compares to the unsafe version?

@LaylBongers
Copy link
Author

I have not done a benchmark check against either the unsafe translation, or the original C. I'd imagine it's not a huge difference given that the implementation hasn't actually changed at all. There will be additional checks as array accesses are bounds checked.

@LaylBongers LaylBongers requested a review from MinerSebas July 24, 2023 17:17
@hymm hymm added this to the 0.12 milestone Aug 10, 2023
fn set_tangent(
&mut self,
tangent: [f32; 3],
_bi_tangent: [f32; 3],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better (or more "bevy native" or w/e) to use glam Vec2/Vec3/Vec4s instead of [f32; 2/3/4] for the stuff in this trait? I'm not sure if it really matters, but the crate already relies on glam so it's not like that'd increase potential dependencies for non-bevy library users (who also aren't using glam).

Copy link
Contributor

@NthTensor NthTensor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really like to get this moving forward, and ideally land this early in the 0.15 cycle. Considering that it's tested to be byte equivalent to the reference implementation, and only adds costs on startup/asset-load I think it's pretty much ready.

The only hitch is that this introduces several large binary blobs for testing, which I personally am against including in the repo. Is there a way we could commit this without the blobs?

@janhohenheim janhohenheim added this to the 0.15 milestone Aug 23, 2024
@janhohenheim janhohenheim added the S-Needs-Review Needs reviewer attention (from anyone!) to move forward label Aug 23, 2024
@janhohenheim
Copy link
Member

janhohenheim commented Aug 23, 2024

Fully agree with @NthTensor. It would be a shame to let this accumulate more merge conflicts.
Maybe we could download the blobs on CI test runs and let the tests pass locally if the blobs were not found? @BD103
Added it to the 0.15 milestone for consideration; feel free to remove it again if you disagree.

@alice-i-cecile alice-i-cecile removed this from the 0.15 milestone Aug 23, 2024
@alice-i-cecile
Copy link
Member

I also want this, but we need to do some requirements gathering and collaboration. IMO a working group is likely to be the best way to get this into shape.

@janhohenheim
Copy link
Member

janhohenheim commented Aug 23, 2024

@alice-i-cecile hmm? I fail to see why we wouldn't just want to merge it. The version in this PR is

  • made by the person who wrote the original code
  • byte equivalent to the reference implementation (!)
  • contains regression tests
  • runs the same algorithm

This seems like a straight-up complete upgrade. Am I missing something?

@alice-i-cecile
Copy link
Member

Hmm, okay. I remember @superdump expressing reluctance in the past, but byte-identical behavior is very hard to argue with.

@janhohenheim janhohenheim added the D-Unsafe Touches with unsafe code in some way label Aug 23, 2024
@BD103
Copy link
Member

BD103 commented Aug 23, 2024

Maybe we could download the blobs on CI test runs and let the tests pass locally if the blobs were not found? @BD103

I'm not sure I understand what the blobs are used for in this circumstance. Does the generator C program generate all of the .bin files, and regression_test.rs checks that the Rust program matches the output of the C program?

@BenjaminBrienen BenjaminBrienen added S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Oct 31, 2024
@LaylBongers
Copy link
Author

That's correct, the tests check that the code generates byte-identical results. I figured that's the most appropriate test for this conversion, which should perform where possible identical floating point operations in identical orders, and thus produce byte-identical results.

@alice-i-cecile alice-i-cecile added D-Complex Quite challenging from either a design or technical perspective. Ask for help! X-Uncontroversial This work is generally agreed upon S-Needs-Review Needs reviewer attention (from anyone!) to move forward and removed S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels Nov 15, 2024
@alice-i-cecile alice-i-cecile added this to the 0.16 milestone Nov 15, 2024
@alice-i-cecile alice-i-cecile added the M-Needs-Release-Note Work that should be called out in the blog due to impact label Nov 15, 2024
@atlv24
Copy link
Contributor

atlv24 commented Nov 19, 2024

do we want to fuzz test this to check nan and infinity handling is identical with C? also, do we want to commit C into the bevy repo?

@BenjaminBrienen
Copy link
Contributor

I'm surprised that adding clang/cmake stuff is "uncontroversial".

@alice-i-cecile alice-i-cecile added X-Controversial There is active debate or serious implications around merging this PR and removed X-Uncontroversial This work is generally agreed upon labels Nov 19, 2024
@LaylBongers
Copy link
Author

This is mainly the reason I included the binary files in git tree rather than generating them at test time, so as to not include CMake into the actual build at any point. Those files are only there to generate the comparison files, and do not take part in the regular build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Rendering Drawing game state to the screen C-Code-Quality A section of code that is hard to understand or change D-Complex Quite challenging from either a design or technical perspective. Ask for help! D-Unsafe Touches with unsafe code in some way M-Needs-Release-Note Work that should be called out in the blog due to impact P-Unsound A bug that results in undefined compiler behavior S-Needs-Review Needs reviewer attention (from anyone!) to move forward X-Controversial There is active debate or serious implications around merging this PR
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Improve safety of bevy_mikktspace