Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mysql中float精度计算误差容忍度太低 #296

Open
hnwyllmm opened this issue Oct 18, 2023 · 1 comment
Open

mysql中float精度计算误差容忍度太低 #296

hnwyllmm opened this issue Oct 18, 2023 · 1 comment
Labels
enhancement New feature or request

Comments

@hnwyllmm
Copy link
Collaborator

Enhancement
MySQL中对float类型的数据计算总是出现让人无法接受的误差。
样例1：
比如在表中执行：
insert into t values(29.14);
select * from t where id < 29.14;
将会看到29.14这条数据。

样例2：
插入稍微大一点的数字，比如10365.23，MySQL中查询数据时会得到10365.2
@hnwyllmm hnwyllmm added the enhancement New feature or request label Oct 18, 2023
@hnwyllmm
Copy link
Collaborator Author

按照网上的建议，在MySQL中都使用 DECIMAL(15,2)这种或者类似的写法，可以在miniob中把float替换掉，写一个支持默认的decimal, 小数长度固定的浮点数

hnwyllmm pushed a commit that referenced this issue Nov 13, 2024
@nautaa
fix rollup (#478)
42925b0 
### What problem were solved in this pull request?

Problem: ref #296
double_to_str 函数中使用print %.2f 并不会将浮点数四舍五入，导致测试时与MySQL浮点数四舍五入后结果不同。

### What is changed and how it works?
先进行四舍五入，然后使用 %.2f 转换为字符串
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@hnwyllmm