Skip to content

Commit

Permalink
Fix fromRational bug
Browse files Browse the repository at this point in the history
Postgresql and Haskell differ in how they print numbers at the limits of
a double's precision. When these numbers have whole digits, we have less
precision to give to the decimal part, so we need to be less accurate
than 1e15.
  • Loading branch information
TeofilC committed Aug 7, 2024
1 parent b384871 commit 72c2a69
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion tests/Main.hs
Original file line number Diff line number Diff line change
Expand Up @@ -625,7 +625,14 @@ testFromRational = databasePropertyTest "fromRational" \transaction -> do
pure $ fromRational rational
diff result (~=) double
where
a ~= b = abs (a - b) < 1e-15
wholeDigits x = fromIntegral $ length $ show $ round x
-- A Double gives us between 15-17 decimal digits of precision.
-- It's tempting to say that two numbers are equal if they differ by less than 1e15.
-- But this doesn't hold.
-- The precision is split between the whole numer part and the decimal part of the number.
-- For instance, a number between 10 and 99 only has around 13 digits of precision in its decimal part.
-- Postgres and Haskell show differing amounts of digits in these cases,
a ~= b = abs (a - b) < 10**(-15 + wholeDigits a)
infix 4 ~=


Expand Down

0 comments on commit 72c2a69

Please sign in to comment.