So you want to use multiple slave databases with Rails? How does this syntax strike you:
A few months ago, there was quite a bit of noise in the Rails community about Rails’ inability to easily make use of multiple databases. Dr Nic quickly responded with a solution. Everyone rejoiced in the wonder of the community.
However, the solution lacks the syntactic sugar that Rails does so well. It also has the load balancing logic above the models in user code, rather than below them, where it seems to belong.
Enter Mysql Replication Adapter.
This library contains a Rails database adapter that helps your Rails app talk to a single write master and multiple read-only slaves in a very elegant way. The configuration is also more environment-oriented than connection-oriented, so it hides the concept of multiple connections from the programmer altogether. All you have to do is say that you want a #find, #find_by_sql, or aggregate function spread over slave databases and the adapter does it for you.
Similar to Dr Nic’s approach, you still have to explicitly say when you want your queries to be sent to a slave database. We opted for this approach over a silent automatic balancing option because it reduces your ability to accidently really bork something up by sending it to a database that is just slightly behind in replication.
Here at Rapleaf, we’re using this adapter in a lot of our projects, because it allows us to leverage the additional read bandwidth of our two slave databases.
Installation & Use
(This all assumes that you have Mysql master-slave replication set up in advance.)
First, install it via the gem:
Next, pop open environment.rb and add:
to the top before the initialize loop.
(Note: It *has* to go before the initializer loop, or it will load too late.)
Next, open up database.yml and configure it like so:
development:
adapter: mysql_replication
database: yourdb
username: root
password:
host: localhost
port: 3306
slaves:
- host: slave1
port: 3306
username: readonlyuser
password:
- host: slave2
port: 3306
username: readonlyuser
password:
Finally, to actually have Rails use one of the slave databases for a given find, use syntax like this:
That’s all there is to it. If you’ve specified slave databases, when you make the above call, a random slave will be chosen and the query executed against that database. If you try and do a write operation (like find_or_create_by_*), the adapter will throw an error, so don’t worry, you won’t be able to corrupt your database.
One nice feature is that if you’re logging queries, whether because you’re in development mode or otherwise, the log will tell you which slave database the query got sent to. This could be helpful for debugging purposes.
There are a few other ways to make use of slave databases, but I’ll leave that for another post. You can also check out the readme in the gem directory for more details if you’re interested.

18 Comments
Nice work guys. Personally, I’d like to see :use_slave => true be the default since it seems to me that if you have this set up that 9 times out of 10 you’d want to use the slaves for reads and override the that default for corner cases where you didn’t want that behavior. Oh and having all databases supported would be nice, but then I guess the name would have to be changed.
I’m just now taking a look at this gem after hearing about it awhile back. So far looks pretty slick. I do have a question though.
Why not modify the execute() method to look at the query itself and run it against @master if it’s INSERT/UPDATE/DELETE/ALTER/CREATE/DROP ?
For example, the insert()/update() methods call ensure_master() which does the check, but complains.
Is there a reason you couldn’t just switch to the @master at that point and keep on going?
I have a feeling I’m not seeing something, but if it’s possible, I think I’ll patch that as it would be handy in our situation to always go to the slaves and automatically go back to the master only for updates.
Thoughts?
@Philip: Automatically executing write queries against the master was the original idea. However, we opted against this approach because it complicates the semantics of using the adapter. You could unintentionally perform a write inside of what you believed was a read-only section, which could have complicated side effects. If the adapter silently did the write to the master, you’d never know that you had this confusing setup.
Instead, our approach is that you just have to be explicit when you’re using a slave database. It’s a little less magical, but we think it’s a decent tradeoff.
I have a question about the setup of the database.yml file.
Does the host have to say slave1 or is that just the name of the host for the first slave?
How do you set a different name of the database for the slave, do you just add another line database: slavedbname, after the slave decleration?
Thanks in advance
@Immad: It doesn’t need to be named “slave1″. You have it right about different database names, or any other option for that matter: just fill in any attribute you need changed under the slave’s configuration.
Bryan,
We’re looking to use multiple databases in the production environment while only having one running locally on the developer’s box. The example database.yml above specifies multiple databases on the same port in the development environment. Is this how the gem handles one database engine per development environment? Or do we actually have to have multiple database engines running on the developer’s box? Our development is being done on Windows boxes (per corporate standards).
Thanks,
Bill
@Bill: You don’t need to have multiple real database engines behind the scenes in development. In fact, you don’t even need to use MysqlReplicationAdapter in development if you don’t want to. You can use the regular Mysql adapter. As long as you require in mysql replication adapter, the regular mysql adapter gets the same extensions to allow :use_slave => true operation, though it doesn’t actually do anything. This was designed specifically to make development (and testing) easier.
Is it ready for Rails 2.0.2 yet?
Cool. Seems to work as advertised. Have been testing this for the last several days in an isolated production config for a site.
Question: what happens when a slave drops out, when there are 2 or more configured?
Will this establish another connection to the next (or any) available slave in the list?
I know — go read the source….
I managed to get it working with rails 2, but it’s a bit different- it runs as a plugin instead of a gem. Still works with rails 1, too, though you still need to hack it into environment.rb. I uploaded it at http://rubyforge.org/tracker/index.php?func=detail&aid=20068&group_id=4116&atid=15778 as a bug because I’m an idiot and forgot about the patches list…
I have problem with setting this adapter.
Currently I am developing in windowXP using rails2.0.2. After I add require line in my environment.rb and restart my server, I got All this message.
Exiting
c:/ruby/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:266:in `load_missing_constant’: uninitialized constant RAILS_CONNECTION_ADAPTERS (NameError)
from c:/ruby/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:453:in `const_missing’
from c:/ruby/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:465:in `const_missing’
from c:/ruby/lib/ruby/gems/1.8/gems/mysql_replication_adapter-0.4.0/lib/mysql_replication_adapter.rb:8
from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:32:in `gem_original_require’
from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:32:in `require’
from c:/ruby/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:496:in `require’
from c:/ruby/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:342:in `new_constants_in’
from c:/ruby/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:496:in `require’
… 29 levels…
from c:/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in `require’
from ./script/server:3
from -e:2:in `load’
from -e:2
Do you have any idea what’s wrong with my setting ?
I recently found this rewrite (as a plugin) that works with 2.0.2
http://rubyforge.org/tracker/index.php?func=detail&aid=20068&group_id=4116&atid=15778
any updates to the gem to get it working with Rails 2+?
thanks!
FYI, I took this gem and updated it so that it’s compatible with Rails 2.2.2 (as a plugin).
You can find it here:
http://github.com/findchris/mysql_replication_adapter/tree/master
-Chris
Nice write up…usually I never reply to these thing but this time I will,Thanks for the great info.
How do you set a different name of the database for the slave, do you just add another line database: slavedbname, after the slave decleration?
Thanks in advance
You got it. Each subsection should be an independent DB config section.
FYI: We recently made the switch to mysql_proxy, and that helped a lot to automatically direct reads/writes to the appropriate database.
However, race conditions can occur if master-to-slave replication takes too long and you SELECT a recently inserted value. So I wrote a Rails DB adapter to allowing overriding mysql_proxy by directing queries to named databases, as in: User.first(:use_db => :master)
The adapter, mysql_proxy_adapter, can be found here:
http://github.com/findchris/mysql_proxy_adapter/
-Chris
(@findchris)
8 Trackbacks
[...] The first excellent post by Bryan is now up about Mysql Replication Adapter. [...]
[...] fellows at Rapleaf, there should be no reason to say it takes 300 to scale a Rails app. Rapleaf just released a Mysql Replication Adapter that allows you to basically use multiple replication slave for reads and a single master writes. [...]
[...] MySQL Replicator Adaptor for Rails: http://blog.rapleaf.com/dev/?p=5 [...]
[...] a/r mysql slave (rapleaf): query read-only slaves in ActiveRecord [...]
[...] also from Rapleaf (Sorry forgot to take a picture) explained about their Mysql Replication Adapter gem, which enables you to fairly transparently use slave databases when you need to by patching [...]
[...] Mysql Replication Adapter (tags: ruby rails sysadmin database mysql cluster 247up) [...]
Replication Notes
Current Method Rails App (read and write to master) MySQL DB Replication Method (for failover only,…
[...] original mysql_replication_adapter was first made available by RapLeaf. It was a gem that was released in mid-2007 and hosted on [...]