Fix output precision limit for double values (issue #118) #119
Conversation
pgSphere used its own setting (set_sphere_output_precision function) to limit the output precision of double values, that could not be greater than 15 significant digits (DBL_DIG). It introduced some problems with dump/restore. PostgreSQL uses its own precision setting: extra_float_digits. The PostgreSQL setting allows to use more significant digits. This patch changes the default pgSphere output behaviour to utilize PostgreSQL extra_float_digits. Now, extra_float_digits is used by default, until set_sphere_output_precision is called. The old behaviour is kept for compatibility purposes. Once, set_sphere_output_precision is called, pgSphere starts to use the old behaviour (read, please, issue postgrespro#118 discussion). The patch introduces a new function - reset_sphere_output_precision. It is used to reset to the PostgreSQL behaviour after using set_sphere_output_precision.
|
This is fantastic, @vitcpp! I'm not sure I understand why there are two new |
|
Hi @esabol, once I've changed the default behaviour of the output by improving the precision, some tests show differences in numbers. It is why I've added two new test variant comparison files. But, it seems it may be a mistake. It would be better to just replace the old files. I will check. Thank you. BTW, I think to add some a new test for testing output precision with different settings. P.S. Test variant comparison files are used if the output may vary on different platforms: https://www.postgresql.org/docs/current/regress-variant.html P.P.S. I'm going to check it on 32 bit platform as well. |
|
The changes should definitely have tests. I was just confused as to how the new |
|
expected/epochprop.out differs for 10, 11 and for 12+ versions, because the default precision was 15 significant digits prior to PG12. It is why I've added one more variant comparison file. At the left - output for PG 10-11, at the right - output for PG 12+.
|
|
Why not put |
expected/output_precision.out - PG 10-11 expected/output_precision_1.out - PG 12+
It makes sense, thank you! It seems, set_sphere_output_precision(8) is set at the top of almost all tests. |
|
I've added one more commit to set extra_float_digits = 2 in epochprop and bounding_box_gist tests as proposed by @df7cb . It allows to remove expexted/bounding_box_gist_2.out but expected/epochprop_1.out is still required. The reason to create expected/epochprop_1.out is that the earlier versions output exactly 17 digits if extra_float_digits is 2, some of these digits may be meaningless. PG versions 12+ use a special function that outputs only meaningful digits even if extra_float-digits = 2, it is why in PG 12+ some numbers may contain less than 17 digits if the text representation if enough to define the double number. |
pgSphere used its own setting (set_sphere_output_precision function) to limit the output precision of double values, that could not be greater than 15 significant digits (DBL_DIG). It introduced some problems with dump/restore. PostgreSQL uses its own precision setting: extra_float_digits. The PostgreSQL setting allows to use more significant digits.
This patch changes the default pgSphere output behaviour to utilize PostgreSQL extra_float_digits. Now, extra_float_digits is used by default, until set_sphere_output_precision is called.
The old behaviour is kept for compatibility purposes. Once, set_sphere_output_precision is called, pgSphere starts to use the old behaviour (read, please, issue #118 discussion).
The patch introduces a new function - reset_sphere_output_precision. It is used to reset to the PostgreSQL behaviour after using set_sphere_output_precision.