Skip to content

Why does double type get rounded when adding in a static_map #603

Closed Answered by PointKernel
avithemad asked this question in Q&A
Discussion options

You must be logged in to vote

@avithemad Is it possible to get a minimal repro in https://godbolt.org/z/1GvxoKbP3?

The cudf groupby and aggregate operations also utilize cuco hash tables, but we haven't observed any of the rounding errors mentioned here. Since cuco hash tables store data exactly as provided, the issue is more likely with the aggregation. You might want to examine the following details more closely:

  double res = l_extendedprice[tid]*(1.0f - l_discount[tid]);

Since 1.0f is a float, you might want to use 1. or 1.0 to prevent narrow conversions. Also, could it be an issue with how the output is being printed, for example, by converting the value to an integer before printing?

Replies: 1 comment 1 reply

Comment options

You must be logged in to vote
1 reply
@avithemad
Comment options

Answer selected by avithemad
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants