Author Archives: chris

Application Deployment at Rapleaf

By chris | Published: May 25, 2010
One challenge we constantly face at Rapleaf is how to remain agile as we grow. In the early days, we would say “I’m going to deploy” to a handful of other engineers in the room. When we needed to move beyond that, we designed a deployment system with 3 key objectives: Remain agile. [...]
Posted in Operations | Tagged , , , , , , , , | Leave a comment

Person API v3

By chris | Published: January 4, 2010
We just updated our person API to allow queries by social network profile ID. The new style URLs are also more natural to write. If you want to query by email: http://api.rapleaf.com/v3/person/email/dummy@rapleaf.com If you want to query by the SHA1 hash of an email: http://api.rapleaf.com/v3/person/hash/sha1/1147e414eec8b785fb760f13f7890a767ffaef6e If you want to query by Myspace profile ID: http://api.rapleaf.com/v3/person/web/myspace/1234567 If you want to [...]
Posted in Miscellaneous | 5 Comments

Command-line auto completion for Hadoop DFS commands

By chris | Published: November 17, 2009
We like to keep things simple here at Rapleaf. One small tweak we made right after we installed hadoop was to alias 'hadoop dfs' to 'hdfs'. It rolls off the fingers nicely. We are also constantly typing 'hdfs -ls this' or 'hdfs -du that'. If we are not sure what this/that [...]
Posted in HDFS, Hadoop, bash | Tagged , , , , , , | 4 Comments

“Multi-threaded” MySQL Replication

By chris | Published: May 8, 2009
Several of our MySQL slave databases have excessive replication lag. They continue to fall further and further behind the master and will never catch up. One solution is to resync the master/slave occasionally. This is a manual and time-consuming process. Another solution is to partition the database. This requires code [...]
Posted in Mysql | 2 Comments

Making sure Ruby Daemons die

By chris | Published: February 15, 2008
We use the Daemons Ruby Gem for a variety of applications. It has served us well, but we found ourselves wrapping the “stop” command with a shell script that makes sure the process actually dies. This behavior is necessary for our deploy scripts which restart daemons. Thanks to the magic of Ruby, [...]
Posted in Daemons, Extensions, Ruby | 20 Comments

Timing out Rails from Mongrel

By chris | Published: November 15, 2007
While we’re on the timeout tip, I’d like to talk about a fail-safe timeout for mongrel rails. We’ve talked about making sure that net/http and memcache-client behave, but what about other slow actions? Since mongrel only processes one rails request at a time, other requests can start to pile up. In rare [...]
Posted in Extensions, Mongrel, Ruby | 2 Comments