Replication with Rails on Heroku

Configuration

First, install the gem as usual (see 'Install' section in the README).

If you have one or more read-only follower databases on a Heroku deployment, place this dynamic Octopus configuration in config/shards.yml to configure your followers as Octopus slaves for replication usage.

This configuration uses the environmental variables that Heroku sets up when you create your heroku-postgresql add-on databases to automatically set up any slaves that are present. This assumes that you desire all non-primary databases to be used as read-only slaves. If you don't, see the 'More Info and Options' section below for your options.

Your primary database will be configured as usual by the configuration that Heroku injects into database.yml.

How closely your followers (slaves) follow master is application specific, so the followers are not configured to automatically send all read queries to the followers by default. In Octopus lingo, this does not mark your slaves as 'fully replicated'.

Mark the appropriate AR models by setting replicated_model:

class StaticThing < ActiveRecord::Base
  replicated_model
end

This results in using your followers for read queries, and master for write queries on that model. This is appropriate for models that won't yield unexpected behavior when read queries come from a slave that may be a few seconds behind the master they follow.

That's everything required to get started. You can read more about Octopus to learn how to use the using methods in controllers, models and AR relations in a more granular fashion, if needed. If you do, read below about the Octopus.followers monkey-patch to ensure your code is ready for future scaling, which is necessary until using_group functionality is more robust.

Initializer (Recommended)

Place this initializer code in config/initializers/octopus.rb for:

• Convenient logging of the slaves configured at app initialization
• Use of Octopus.followers to retrieve configured followers
  • Example: StaticThing.using(Octopus.followers).all

More Info and Options

Logging

Octopus integrates with Rails logging and will prepend the SQL log line with the shard/slave queried:

Master:

DynamicThing Load (0.2ms)  SELECT "dynamic_things".* FROM "dynamic_things"

Follower(s):

[Shard: orange_follower]  StaticThing Load (0.3ms)  SELECT "static_things".* FROM "static_things" 

[Shard: pink_follower]  StaticThing Load (0.2ms)  SELECT "static_things".* FROM "static_things" 

There was a bug with this logging customization for the recommended Rails configuration format (used above), this is fixed as of issue/pull #166. Use the rocketmobile/octopus fork if this pull request isn't merged yet upon your reading.

Enabling/Disabling Followers as Slaves

If using the above configuration, you can choose the followers you want to have slave duties with environmental variables. Whitelist followers you want or blacklist the followers you don't want with SLAVE_ENABLED_FOLLOWERS and/or SLAVE_DISABLED_FOLLOWERS. Use a comma+space separated list of (case insensitive) follower colors:

heroku config:add SLAVE_ENABLED_FOLLOWERS=PINK, CRIMSON

Without these variables set (the default), all followers will be used as slaves. This may be undesirable for you. One example of usage is when adding an additional follower to a live application, where the above ENV vars are used to ensure the new follower is excluded until it is sufficiently caught up to master for duty.

Note: the blacklist config isn't currently written to work in development or test environments. The blacklist var is effectively ignored in non-Heroku environments.

Development and Test configuration

By default, 2 followers will be simulated (pointing at your development or test database) in development or test environments. The number of dev/test followers is easily changed with the following code in the configuration file:

Array.new(2){YAML::load_file(File.open("config/database.yml"))[Rails.env]}

Additional Params in DB URL

Heroku's dynamic configuration for your primary database allows you too add additional configuration params as URI params on the DB URL configuration variable (cross-reference: SO thread).

This same method of additional params via the _URL ENV key is supported for slave configuration with the above dynamic Octopus configuration.

Full Replication

If you understand the implications and what all application reads to go to your slaves, simply change false to true in the following line in config/shards.yml:

fully_replicated: false