Update code for compatibility with PHP8

[JimBacon]

Relates to BiologicalRecordsCentre/iRecord#1047

Also update check_php_version() in modules/indicia_setup/helpers/config_test.php and the installation docs at https://indicia-docs.readthedocs.io/en/latest/administrating/warehouse/warehouse-installation.html

[JimBacon]

Running PHP Compatibility checks for PHP >= 7.3 with
phpcs -p --standard=PHPCompatibility --runtime-set testVersion 7.3- --report=summary .
returns the following summary results.

-----------------------------------------------------------------------------------------
FILE                                                                    ERRORS  WARNINGS
-----------------------------------------------------------------------------------------
occurrence_comment_quick_reply_page.php                                     0       1
application/helpers/vague_date.php                                          0       1
application/libraries/MY_Database.php                                       0       1
application/libraries/WorkQueue.php                                         0       1
application/models/taxa_taxon_list.php                                      0       1
application/models/termlists_term.php                                       0       1
modules/indicia_svc_data/views/services/data/termlists_term/li.php          0       1
system/config/encryption.php                                                2       0
system/core/Kohana.php                                                      23      0
system/core/utf8.php                                                        0       1
system/core/utf8/ucwords.php                                                1       0
system/libraries/Database.php                                               0       1
system/libraries/Encrypt.php                                                15      0
system/libraries/Input.php                                                  0       2
system/libraries/ORM.php                                                    0       1
system/libraries/Validation.php                                             0       2
system/libraries/drivers/Cache/Sqlite.php                                   5       0
system/libraries/drivers/Database/Mssql.php                                 15      0
system/libraries/drivers/Database/Mysql.php                                 15      0
system/libraries/drivers/Database/Pdosqlite.php                             1       0
system/vendor/swift/Swift/Connection/Sendmail.php                           0       1
system/vendor/swift/Swift/Connection/SMTP.php                               0       2
system/vendor/swift/Swift/Iterator/MySQLResult.php                          5       0
system/vendor/swift/Swift/Message/Encoder.php                               0       4
system/vendor/swift/Swift/Plugin/MailSend.php                               0       1
vendor/maennchen/zipstream-php/test/BigintTest.php                          4       0

The errors are all in the Kohana system code (plus a test in a vendor supplied library by the look of it)

[JimBacon]

The errors in the system folder are given by
phpcs -pn --standard=PHPCompatibility --runtime-set testVersion 7.3- --report-width=100 system
which yields

FILE: /system/vendor/swift/Swift/Iterator/MySQLResult.php
----------------------------------------------------------------------------------------------------
FOUND 5 ERRORS AFFECTING 5 LINES
----------------------------------------------------------------------------------------------------
 50 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
    |       | instead
 76 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
    |       | instead
 92 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
    |       | instead
 93 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
    |       | instead
 94 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
    |       | instead
----------------------------------------------------------------------------------------------------


FILE: system/config/encryption.php
----------------------------------------------------------------------------------------------------
FOUND 2 ERRORS AFFECTING 2 LINES
----------------------------------------------------------------------------------------------------
 29 | ERROR | The constant "MCRYPT_MODE_NOFB" is deprecated since PHP 7.1 and removed since PHP 7.2
 30 | ERROR | The constant "MCRYPT_RIJNDAEL_128" is deprecated since PHP 7.1 and removed since PHP
    |       | 7.2
----------------------------------------------------------------------------------------------------


FILE: system/libraries/drivers/Database/Mssql.php
----------------------------------------------------------------------------------
FOUND 15 ERRORS AFFECTING 15 LINES
----------------------------------------------------------------------------------
  39 | ERROR | Extension 'mssql' is removed since PHP 7.0
  66 | ERROR | Extension 'mssql' is removed since PHP 7.0
  95 | ERROR | Extension 'mssql' is removed since PHP 7.0
 107 | ERROR | Extension 'mssql' is removed since PHP 7.0
 283 | ERROR | Extension 'mssql' is removed since PHP 7.0
 332 | ERROR | Extension 'mssql' is removed since PHP 7.0
 340 | ERROR | Extension 'mssql' is removed since PHP 7.0
 345 | ERROR | Extension 'mssql' is removed since PHP 7.0
 346 | ERROR | Extension 'mssql' is removed since PHP 7.0
 348 | ERROR | Extension 'mssql' is removed since PHP 7.0
 366 | ERROR | Extension 'mssql' is removed since PHP 7.0
 429 | ERROR | Extension 'mssql' is removed since PHP 7.0
 432 | ERROR | Extension 'mssql' is removed since PHP 7.0
 446 | ERROR | Extension 'mssql' is removed since PHP 7.0
 459 | ERROR | Extension 'mssql' is removed since PHP 7.0
----------------------------------------------------------------------------------


FILE: system/libraries/drivers/Database/Pdosqlite.php
--------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
--------------------------------------------------------------------------------------
 212 | ERROR | Extension 'sqlite' is removed since PHP 5.4
--------------------------------------------------------------------------------------


FILE: /system/libraries/drivers/Database/Mysql.php
----------------------------------------------------------------------------------------------------
FOUND 15 ERRORS AFFECTING 15 LINES
----------------------------------------------------------------------------------------------------
  41 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
  61 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
  87 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
  99 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 278 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 298 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 363 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 371 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 376 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 377 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 395 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 456 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 459 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 473 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
 483 | ERROR | Extension 'mysql_' is deprecated since PHP 5.5 and removed since PHP 7.0; Use mysqli
     |       | instead
----------------------------------------------------------------------------------------------------


FILE: system/libraries/drivers/Cache/Sqlite.php
--------------------------------------------------------------------------------
FOUND 5 ERRORS AFFECTING 5 LINES
--------------------------------------------------------------------------------
  26 | ERROR | Extension 'sqlite' is removed since PHP 5.4
  53 | ERROR | Extension 'sqlite' is removed since PHP 5.4
  60 | ERROR | Extension 'sqlite' is removed since PHP 5.4
  97 | ERROR | Extension 'sqlite' is removed since PHP 5.4
 102 | ERROR | Extension 'sqlite' is removed since PHP 5.4
--------------------------------------------------------------------------------


FILE: system/libraries/Encrypt.php
----------------------------------------------------------------------------------------------------
FOUND 15 ERRORS AFFECTING 10 LINES
----------------------------------------------------------------------------------------------------
  73 | ERROR | Function mcrypt_get_key_size() is deprecated since PHP 7.1 and removed since PHP
     |       | 7.2; Use OpenSSL instead
  73 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
  82 | ERROR | Function mcrypt_get_iv_size() is deprecated since PHP 7.1 and removed since PHP 7.2;
     |       | Use OpenSSL instead
  82 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 104 | ERROR | The constant "MCRYPT_RAND" is deprecated since PHP 7.1 and removed since PHP 7.2
 111 | ERROR | The constant "MCRYPT_DEV_URANDOM" is deprecated since PHP 7.1 and removed since PHP
     |       | 7.2
 116 | ERROR | The constant "MCRYPT_DEV_RANDOM" is deprecated since PHP 7.1 and removed since PHP
     |       | 7.2
 121 | ERROR | The constant "MCRYPT_RAND" is deprecated since PHP 7.1 and removed since PHP 7.2
 126 | ERROR | The constant "MCRYPT_RAND" is deprecated since PHP 7.1 and removed since PHP 7.2
 134 | ERROR | Function mcrypt_create_iv() is deprecated since PHP 7.1 and removed since PHP 7.2;
     |       | Use random_bytes() or OpenSSL instead
 134 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 137 | ERROR | Function mcrypt_encrypt() is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | OpenSSL instead
 137 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 161 | ERROR | Function mcrypt_decrypt() is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | OpenSSL instead
 161 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
----------------------------------------------------------------------------------------------------


FILE: system/core/Kohana.php
----------------------------------------------------------------------------------------------------
FOUND 23 ERRORS AFFECTING 7 LINES
----------------------------------------------------------------------------------------------------
 606 | ERROR | Function mcrypt_get_iv_size() is deprecated since PHP 7.1 and removed since PHP 7.2;
     |       | Use OpenSSL instead
 606 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 606 | ERROR | The constant "MCRYPT_RIJNDAEL_256" is deprecated since PHP 7.1 and removed since PHP
     |       | 7.2
 606 | ERROR | The constant "MCRYPT_MODE_ECB" is deprecated since PHP 7.1 and removed since PHP 7.2
 607 | ERROR | Function mcrypt_create_iv() is deprecated since PHP 7.1 and removed since PHP 7.2;
     |       | Use random_bytes() or OpenSSL instead
 607 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 607 | ERROR | The constant "MCRYPT_RAND" is deprecated since PHP 7.1 and removed since PHP 7.2
 609 | ERROR | Function mcrypt_decrypt() is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | OpenSSL instead
 609 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 609 | ERROR | The constant "MCRYPT_RIJNDAEL_256" is deprecated since PHP 7.1 and removed since PHP
     |       | 7.2
 609 | ERROR | The constant "MCRYPT_MODE_ECB" is deprecated since PHP 7.1 and removed since PHP 7.2
 664 | ERROR | Function mcrypt_get_iv_size() is deprecated since PHP 7.1 and removed since PHP 7.2;
     |       | Use OpenSSL instead
 664 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 664 | ERROR | The constant "MCRYPT_RIJNDAEL_256" is deprecated since PHP 7.1 and removed since PHP
     |       | 7.2
 664 | ERROR | The constant "MCRYPT_MODE_ECB" is deprecated since PHP 7.1 and removed since PHP 7.2
 665 | ERROR | Function mcrypt_create_iv() is deprecated since PHP 7.1 and removed since PHP 7.2;
     |       | Use random_bytes() or OpenSSL instead
 665 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 665 | ERROR | The constant "MCRYPT_RAND" is deprecated since PHP 7.1 and removed since PHP 7.2
 668 | ERROR | Function mcrypt_encrypt() is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | OpenSSL instead
 668 | ERROR | Extension 'mcrypt' is deprecated since PHP 7.1 and removed since PHP 7.2; Use
     |       | openssl (preferred) or pecl/mcrypt once available instead
 668 | ERROR | The constant "MCRYPT_RIJNDAEL_256" is deprecated since PHP 7.1 and removed since PHP
     |       | 7.2
 668 | ERROR | The constant "MCRYPT_MODE_ECB" is deprecated since PHP 7.1 and removed since PHP 7.2
 950 | ERROR | Since PHP 7.0, functions inspecting arguments, like debug_backtrace(), no longer
     |       | report the original value as passed to a parameter, but will instead provide the
     |       | current value. The parameter "$message" was changed on line 879.
----------------------------------------------------------------------------------------------------


FILE: system/core/utf8/ucwords.php
-------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-------------------------------------------------------------------------------------------------
 21 | ERROR | preg_replace() - /e modifier is deprecated since PHP 5.5 and removed since PHP 7.0
-------------------------------------------------------------------------------------------------

None of these are new deprecations suggesting that, if they haven't bitten us yet, then they probably won't do so now.

[JimBacon]

Current situation is that I am stuck on unit testing for PHP8 as we have a dependency on DbUnit which is obsolete.
See #375 (comment)

[johnvanbreda]

Although I don't like this as a solution, do we need a re-written set of tests that work in recent PHPUnit versions?

[JimBacon]

I have modified all the tests so that they are working with the latest version of PHPUnit.
That was necessary as we needed to use a version of PHPUnit compatible with PHP8.

The issue is with our modules/phpUnit/libraries/Indicia_DatabaseTestCase which inherits from PHPUnit\DbUnit\TestCase.
The original, https://github.com/sebastianbergmann/dbunit, is no longer maintained.
I have worked around this by employing a fork, https://github.com/misantron/dbunit.
It is not a fork that gives any assurance of ongoing maintenance.
Installed with Composer, it is limited to PHP7.

A possible solution is to include the source code of DbUnit in our repo and keep it compatible with changes to PHPUnit.

However, the justification for DbUnit being abandonned was

it is usually easier to implement a custom, project-specific solution for database fixture management than to use the generic solution provided by DbUnit. The effort to build such a custom solution is usually quite low, resulting in less than two hundred lines of code in a trait that is used by the database test case classes.

So that is the other possible solution.

[JimBacon]

I just pushed some work I did a while back so it doesn't get lost. This is working towards replacing DbUnit by creating my own, very simple, drop-in alternative for creating fixtures.

All the current tests which manipulate the database would need updating to follow this pattern and then DbUnit could be discarded. That would enable us to switch to PHP8 without losing our ability to run the tests.

[johnvanbreda]

@JimBacon it might be helpful to know what changes are required to existing tests - is it only when fixtures are used to predefine data? If so then this may only affect a handful of tests (other than whatever is required to update the main fixtures).

[JimBacon]

It is just about tests where we set up a database with pre-defined data. I haven't got a clear idea of how much work that is but manageable, I imagine. I don't feel like I have necessarily understood all the things that DbUnit does but my first attempt to replace it seemed to work okay.

To update the test

• I changed the parent class to the new one I've created (SimpleDatabaseTestCase)
• I created the fixture definition by merging two arrays, a core fixture used widely, and a local fixture specific to the test in hand.

The diff makes the changes look more significant that they were because there was a change of indent to the local fixture definition.

If you are developing any new tests, it would be great if you could move in this direction. Very happy for you to offer any improvements to what I have done. (Its all in the PHP8 branch at present but a merge with dev would probably not cause a problem.)

[JimBacon]

I just noticed that the master branch of https://github.com/misantron/dbunit does support PHP8 but it has not been released on packagist. By making use of this version, I have been able to run the unit tests with PHP8. That revealed some compatability issues which I fixed so all the tests pass.

I have merged the work in to the develop branch and set Travis to run the unit tests on both PHP7.3 and PHP8.0 henceforth.

Although the test coverage is limited, and there may still be a few compatibility issues lurking, I'm going to close this issue. It is probably a good idea to continue moving away from using DbUnit but it is no longer blocking us.