Author Archives: ckline

Application Deployment at Rapleaf

By ckline | Published: May 25, 2010

One challenge we constantly face at Rapleaf is how to remain agile as we grow. In the early days, we would say “I’m going to deploy” to a handful of other engineers in the room. When we needed to move beyond that, we designed a deployment system with 3 key objectives: Remain agile. Engineers should [...]

Posted in Operations | Tagged , , , , , , , , | Leave a comment

Person API v3

By ckline | Published: January 4, 2010

Update: We have deprecated our v3 API in favor of our new Personalization API Please refer to the documents found there for more information on accessing our API. We just updated our person API to allow queries by social network profile ID. The new style URLs are also more natural to write. If you want [...]

Posted in Miscellaneous | 6 Comments

Command-line auto completion for Hadoop DFS commands

By ckline | Published: November 17, 2009

We like to keep things simple here at Rapleaf. One small tweak we made right after we installed hadoop was to alias ‘hadoop dfs’ to ‘hdfs’. It rolls off the fingers nicely. We are also constantly typing ‘hdfs -ls this’ or ‘hdfs -du that’. If we are not sure what this/that is, we type ‘hdfs [...]

Posted in bash, Hadoop, HDFS | Tagged , , , , , , | 16 Comments

“Multi-threaded” MySQL Replication

By ckline | Published: May 8, 2009

Several of our MySQL slave databases have excessive replication lag. They continue to fall further and further behind the master and will never catch up. One solution is to resync the master/slave occasionally. This is a manual and time-consuming process. Another solution is to partition the database. This requires code updates and is also time-consuming. [...]

Posted in Mysql | 2 Comments

Making sure Ruby Daemons die

By ckline | Published: February 15, 2008

We use the Daemons Ruby Gem for a variety of applications. It has served us well, but we found ourselves wrapping the “stop” command with a shell script that makes sure the process actually dies. This behavior is necessary for our deploy scripts which restart daemons. Thanks to the magic of Ruby, we were able [...]

Posted in Daemons, Extensions, Ruby | 20 Comments

Timing out Rails from Mongrel

By ckline | Published: November 15, 2007

While we’re on the timeout tip, I’d like to talk about a fail-safe timeout for mongrel rails. We’ve talked about making sure that net/http and memcache-client behave, but what about other slow actions? Since mongrel only processes one rails request at a time, other requests can start to pile up. In rare cases, those other [...]

Posted in Extensions, Mongrel, Ruby | 2 Comments

  • Rapleaf Is Hiring!

    We are looking for engineers who want to solve challenging problems.

    We have great people, do great work, and have great perks.

    Know someone who might be interested? Refer a friend and get $5,000 for successful hires.

    See our current openings at
    www.rapleaf.com/careers