Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The 18 digit numbers generated by the snowflake algorithm will lose accuracy after reading #522

Closed
mokexinxin opened this issue Sep 14, 2023 · 2 comments

Comments

@mokexinxin
Copy link

mokexinxin commented Sep 14, 2023

Hello:
I want to use JSON to read the 18 digit numbers generated by the snowflake algorithm, but I encountered an accuracy loss issue after reading, as follows
{'a':738529527931269425}
But after I ran it, I showed the following results:
{"a":738529527931269376}

The running result shows incorrect numbers, missing is accuracy. If I want to fully obtain this number without converting it to a string, is there a solution?

Note: This issue has been encountered while running sjsonnet. Preliminary troubleshooting may be in this project, and we hope the author can help solve it.
Thank you very much!

@lihaoyi
Copy link
Member

lihaoyi commented Sep 14, 2023

duplicate of #520 (comment)

@lihaoyi lihaoyi closed this as completed Sep 14, 2023
@lihaoyi
Copy link
Member

lihaoyi commented Sep 14, 2023

Sjsonnet works with a Javascript data model. That means double precision floating point numbers. There really isn't anything we can do here: Even if uPickle is able to read the raw JSON number as a full un-truncated string, Sjsonnet does not, and

  1. It's unlikely we can use a different number representation for Sjsonnet, due to performance considerations: BigDecimals are orders of magnitude slower and more memory-intensive than Doubles, and Sjsonnet is somewhat performance sensitive (perf is it's raison d'etre for existing)

  2. databricks/sjsonnet is meant to be equivalent to google/jsonnet, and google/jsonnet also truncates long numbers. It truncates your example differently - which is its own issue I've opened as Rounding of large numbers inconsistent with google/jsonnet databricks/sjsonnet#186 - but it still truncates it, so your use case is not supported on any implementation of the Jsonnet language

You'll have to wrap your long numbers it in a string, or use a different configuration language that uses arbitrary precision numbers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants