Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when talking about float range in MathJSON docs #22

Open
somebody1234 opened this issue Dec 28, 2022 · 1 comment
Open

Error when talking about float range in MathJSON docs #22

somebody1234 opened this issue Dec 28, 2022 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@somebody1234
Copy link

https://github.com/cortex-js/cortex-js.github.io/blob/2cdfc0e0c6bcf84aa090f4cddd9d7367e19e5e81/math-json/index.html#L1913-L1914

<li>the number is in the range \([-(2^{53})+1, (2^{53})-1]\) so it fits in a
64-bit float (<strong>IEEE 754-2008</strong>, 52-bit, about 15 digits of precision).</li>

The example unescaped number given (-234.534e-46) is beyond -2^53+1 - which is around 1e16

@arnog arnog self-assigned this Dec 28, 2022
@arnog arnog added the bug Something isn't working label Dec 28, 2022
@arnog
Copy link
Member

arnog commented Dec 28, 2022

The description is indeed incorrect. The number has to be representable as a 64-bit float, within a range of -1.797693134862315E+308 to -2.225073858507201E-308, or from 2.225073858507201E-308 to 1.797693134862315E+308 and with a precision of 15 digits.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

2 participants