You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Casting between different signed types can have unintuitive (incorrect?) results. Some examples of how to do this correctly would be welcomed.
Observed Behaviour
When casting negative Q1.14 values to Q3.12, the results are very unexpected. The fractional bit of the number seems to be within ~1 of the true value, but the integer part can be quite a bit off. What should be -1 ends up being 3. Or positive 2 instead of -2.
Expected Behaviour
Casting a fixed point number from one format to another should maintain a roughly equivalent value.
To cut a long story short, it's because internally the conversion is using unsigned shift operations (i.e. 'logical' shifting is being forced rather than 'arithmetic' shifting).
I can't remember whether this was intentional or not, but it quite possibly was because technically signed right shift is 'implementation defined' prior to C++20, whilst unsigned shifting is well defined.
The fix would be fairly simple, it would just be a matter of removing a few casts and consequently some unneeded local type aliases.
However, even with that the result won't be quite perfect because of the inherant loss of precision involved.
Brief Description
Casting between different signed types can have unintuitive (incorrect?) results. Some examples of how to do this correctly would be welcomed.
Observed Behaviour
When casting negative Q1.14 values to Q3.12, the results are very unexpected. The fractional bit of the number seems to be within ~1 of the true value, but the integer part can be quite a bit off. What should be -1 ends up being 3. Or positive 2 instead of -2.
Expected Behaviour
Casting a fixed point number from one format to another should maintain a roughly equivalent value.
Source Code
Output:
Extra Information
The text was updated successfully, but these errors were encountered: