State Machines

Z-Push always had the concept of "state machines" (storage for the device states). Since Z-Push 2.3.0 the new SQL state machine is available. This new state machine is the recommended way of handling shared states between multiple Z-Push servers. For single server installation it does not provide a performance benefit.

File StateMachine

The file state machine saves the states into a local directory. It's available and configured by default, without any additional configuration or package to be installed.

It has only one configuration parameter in the main configuration file, called STATE_DIR.

The STATE_DIR needs to point to an existing directory where the configured webserver (or php processor) has full read and write permissions. The default location is /var/lib/z-push.

SQL StateMachine

Originally being develop and contributed by the Z-Push community. It requires PHP-PDO and should work with several implementations. MySQL was extensively tested, but other databases like Postgres should also work. The list of supported engines is here: http://php.net/manual/en/pdo.drivers.php.

Please provide feedback if you try other engines.

The SQL state machine has its own configuration file where the server, port and database need to be defined.

Switching the state machine - State migration

Z-Push provides a script to migrate existing file states to the database. The script is in the tools directory.

The migration is a delicate procedure. If it goes wrong your phones will not be able to find the states and will be re-synchronized. As this will happen for all devices in a short period of time, this could bring down your servers, as all devices will synchronize completely at the same time.

Please follow the steps below to migrate the states.

Installation

To use a state machine the package z-push-state-sql has to be installed.

Preparation

Change the configuration of the sql state machine, located in /etc/z-push/state-sql.conf.php

Set:

1. the engine (see available engines above)
2. the host - default: localhost
3. the port - default: 3306 for mysql
4. the database - default: zpush
5. the db username - the user needs permissions to create the database and tables. You can also create the database manually and not grant that permission to the user.
6. the db password for that user
7. options

The database server could be located on a remote system, in the DMZ or behind a firewall. It's not recommended that you connect to the database with root. You should use a specialized user for z-push.

Testing

After configuring, you can run the migration in a dry-run. This will also give you an indication how long the migration will take. Generally it's quite fast as the states are not very big.

1. Configure the sql state machine as described above
2. Run the migration script in /usr/share/z-push/tools/: ./migrate-filestates-to-db.php or php migrate-filestates-to-db.php

The migration will happen normally, but the migrated states will never be used. E.g. z-push-admin will not be able to work on these states.

Check if you see any warnings, also in the z-push log.

If you encounter any issue, retest setting the LOGLEVEL should be in LOGLEVEL_DEBUG in your configuration.

Attention

The use of the new states is triggered by switching the configs STATE_MACHINE to SQL. You should NOT attempt this on your production system (just editing the configuration) as it will cause race conditions (the phone did an action between the beginning of the migration and the change of the configuration file). Follow the steps below!

Reset the SQL states

If you tested, the reset needs to be done before the actual migration. You can also reset and re-run the migration script several times, but it's important that you terminate your tests by resetting the states database!

You can start over by dropping the database manually. For mysql you do:

1. # mysql -u root -p
2. Enter your password

3. show databases; you should see your database.

4. drop database zpush; deletes the database zpush

You could of course also use a different username (needs permission to drop the database).

Migrating production system

Make sure you have reset the states before you run the actual migration. To avoid race conditions we recommend stopping the webserver. This way, it's impossible that a device connects and changes the states while migrating.

If you have a z-push cluster (several servers sharing the same states) you need to stop all z-push instances before proceeding.

Migration:

1. Stop your webserver
   1. It must be avoided that during the migration states are changed. The easiest is to shortly stop the webserver or disable z-push in its configurations.
2. Run the migration script:

          ./migrate-filestates-to-db.php              or
          php migrate-filestates-to-db.php

3. Let the script terminate
   1. Look for errors or warnings, also in the z-push log!
   2. On errors, you can just restart the webserver/re-enable z-push and keep running on the old file states. Proceed as if you were testing.
4. Everything good?
5. Change your general z-push config, switching STATE_MACHINE to SQL
   1. Optional: run z-push-admin e.g. with a list action to ensure that you see your devices and users.
   2. You can verify the usage of the SQL statemachine looking in your z-push.log (at LOGLEVEL_DEBUG) looking for "SqlStateMachine() ..." loglines.
6. Start the webserver / re-enable z-push
7. Look for warnings and errors, especially related to the State machine. There should be no StateNotFoundExceptions.

StateNotFoundExceptions

Something went wrong. It's certainly wise to analyse what happened.

A StateNotFoundException means, that the states could not be found. The server will automatically issue a resync of the device. This will first work on folder level but quickly escalate so the entire device is re-synchronized. Potentially all devices that will connect now will get this error. You have an important and fast decision to make.

The immediate recommendation is to stop the webserver to prevent that more mobiles connect and will reset the states.

From there, you have two options:

1. You rollback your configuration to the file statemachine, so that all devices which have not yet connected will continue to synchronize with the old (still valid) states. All devices that already got the StateNotFoundException will probably get another StateNotFoundException (as the new states were not saved in the file statemachine while running), but this is certainly only a small amount. This depends on how long the new SQL statemachine was active. If it's less than 50 % of your devices, you should go back to the files. If it's more, you could choose option 2.
2. Keep the sql statemachine in place. The devices will resynchronize and receive fully new states. This is especially recommended if more than 50% of your users started resynchronizing anyway. The load will be the same (quite high probably).