Skip to content

Commit

Permalink
Fix fromRational bug (#338)
Browse files Browse the repository at this point in the history
Postgresql and Haskell differ in how they print numbers at the limits of
a double's precision. When these numbers have whole digits, we have less
precision to give to the decimal part, so we need to be less accurate
than 1e15.
  • Loading branch information
TeofilC authored Aug 7, 2024
1 parent b384871 commit d9a6244
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion tests/Main.hs
Original file line number Diff line number Diff line change
Expand Up @@ -625,7 +625,14 @@ testFromRational = databasePropertyTest "fromRational" \transaction -> do
pure $ fromRational rational
diff result (~=) double
where
a ~= b = abs (a - b) < 1e-15
wholeDigits x = fromIntegral $ length $ show $ round x
-- A Double gives us between 15-17 decimal digits of precision.
-- It's tempting to say that two numbers are equal if they differ by less than 1e15.
-- But this doesn't hold.
-- The precision is split between the whole numer part and the decimal part of the number.
-- For instance, a number between 10 and 99 only has around 13 digits of precision in its decimal part.
-- Postgres and Haskell show differing amounts of digits in these cases,
a ~= b = abs (a - b) < 10**(-15 + wholeDigits a)
infix 4 ~=


Expand Down

0 comments on commit d9a6244

Please sign in to comment.