From d9a624441743b74c3fc35fac5d9e096c07168ae8 Mon Sep 17 00:00:00 2001 From: Teo Camarasu Date: Wed, 7 Aug 2024 14:52:39 +0100 Subject: [PATCH] Fix fromRational bug (#338) Postgresql and Haskell differ in how they print numbers at the limits of a double's precision. When these numbers have whole digits, we have less precision to give to the decimal part, so we need to be less accurate than 1e15. --- tests/Main.hs | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/tests/Main.hs b/tests/Main.hs index 9020a840..8e2a4b33 100644 --- a/tests/Main.hs +++ b/tests/Main.hs @@ -625,7 +625,14 @@ testFromRational = databasePropertyTest "fromRational" \transaction -> do pure $ fromRational rational diff result (~=) double where - a ~= b = abs (a - b) < 1e-15 + wholeDigits x = fromIntegral $ length $ show $ round x + -- A Double gives us between 15-17 decimal digits of precision. + -- It's tempting to say that two numbers are equal if they differ by less than 1e15. + -- But this doesn't hold. + -- The precision is split between the whole numer part and the decimal part of the number. + -- For instance, a number between 10 and 99 only has around 13 digits of precision in its decimal part. + -- Postgres and Haskell show differing amounts of digits in these cases, + a ~= b = abs (a - b) < 10**(-15 + wholeDigits a) infix 4 ~=