-
Notifications
You must be signed in to change notification settings - Fork 505
Replication with Rails on Heroku
First, install the gem as usual (see 'Install' section in the README).
Next, place this dynamic Octopus configuration in config/shards.yml.
This configures your Heroku follower databases as Octopus slaves for read-only replication usage.
Finally, mark the appropriate AR models by using the replicated_model class method:
class StaticThing < ActiveRecord::Base
replicated_model
end
This will result in your followers being used for read queries, and master for write queries for that model. This is appropriate for models that won't yield unexpected behavior when read queries come from a slave that may be a few seconds behind the master they follow.
That's everything required to get started with replication reads on Heroku with Octopus.
You can read more about Octopus in the repo README to learn how the using methods allow you to use your read-only followers in a more granular fashion in controllers, models and Active Record queries, if needed. If you do, the Octopus.followers initializer monkey-patch can help ensure your code is ready for future horizontal DB scaling without any code changes.
Place this initializer code in config/initializers/octopus.rb for:
- Convenient logging of the slaves configured at app initialization
- Use of
Octopus.followersto retrieve configured followers- Example:
StaticThing.using(Octopus.followers).all - Necessary until
using_groupfunctionality is more robust.
- Example:
This configuration uses the environmental variables that Heroku sets up when you create your heroku-postgresql add-on databases to automatically set up any slaves that are present. This assumes that you desire all non-primary databases to be used as read-only slaves. If you don't, see the 'More Information and Options' section below for your options.
Your primary database will still be configured as usual by the configuration that Heroku injects into database.yml.
How closely your followers (slaves) follow master is application specific. As such, this recommended configuration does not configure your application to automatically send all read queries to your followers by default. In Octopus lingo, your slaves will not be marked as 'fully replicated' with this configuration, and you a required to set the appropriate models with replicated_model to have queries sent to your followers. See 'Full Replication' below if this isn't desired for your application.
Octopus integrates with Rails logging and will prepend the SQL log line with the shard/slave queried:
Master:
DynamicThing Load (0.2ms) SELECT "dynamic_things".* FROM "dynamic_things"
Follower(s):
[Shard: orange_follower] StaticThing Load (0.3ms) SELECT "static_things".* FROM "static_things"
[Shard: pink_follower] StaticThing Load (0.2ms) SELECT "static_things".* FROM "static_things"
There was a bug with this logging customization for the recommended Rails configuration format (used above), this is fixed as of issue/pull #166. Use the rocketmobile/octopus fork if this pull request isn't merged yet upon your reading.
If using the above configuration, you can choose the followers you want to have slave duties with environmental variables. Whitelist followers you want or blacklist the followers you don't want with SLAVE_ENABLED_FOLLOWERS and/or SLAVE_DISABLED_FOLLOWERS. Use a comma+space separated list of (case insensitive) follower colors:
heroku config:add SLAVE_ENABLED_FOLLOWERS=PINK, CRIMSON
Without these variables set (the default), all followers will be used as slaves. This may be undesirable for you. One example of usage is when adding an additional follower to a live application, where the above ENV vars are used to ensure the new follower is excluded until it is sufficiently caught up to master for duty.
Note: the blacklist config isn't currently written to work in development or test environments. The blacklist var is effectively ignored in non-Heroku environments.
By default, 2 followers will be simulated (pointing at your development or test database) in development or test environments. The number of dev/test followers is easily changed with the following code in the configuration file:
Array.new(2){YAML::load_file(File.open("config/database.yml"))[Rails.env]}
Heroku's dynamic configuration for your primary database allows you too add additional configuration params as URI params on the DB URL configuration variable (cross-reference: SO thread).
This same method of additional params via the _URL ENV key is supported for slave configuration with the above dynamic Octopus configuration.
If you understand the implications and what all application reads to go to your slaves, simply change false to true in the following line in config/shards.yml:
fully_replicated: false