Comments on: Ruby and HBase http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/ For engineers, by engineers. Thu, 09 Feb 2012 03:22:26 +0000 hourly 1 By: Greg http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-612 Greg Wed, 26 Aug 2009 19:49:40 +0000 http://blog.rapleaf.com/dev/?p=16#comment-612 I'm working on a project called BigRecord that's attempting to bridge the gap between Hbase and RoR. You can find it on github: http://github.com/openplaces/bigrecord/tree/master Bigrecord is an OM (not really relational anymore) with a similar API and codebase to ActiveRecord. The changes that needed to be done on ActiveRecord in order to work with Hbase were non-trivial, so it wasn't just a matter of creating a new adapter for AR. Bigrecord was being used internally at openplaces.org for nearly 2 years now, and when I joined the company, one of my tasks was to open source the tool. It's lacking any useful documentation, but for the most part, it works along with a spec suite. Something worth warning about is the stack required to get it all working. We don't use the thrift or rest interface of Hbase, but instead went with a JRuby/DRb approach. This is because we found that using JRuby/DRb to communicate natively via the Java API was the fastest method to communicate with Hbase. (Disclaimer: don't ask me to provide any real data proving this... I don't have it :P) Feel free to e-mail me greg.lu [at] gmail.com or find me on github: www.github.com/enell with any questions. I’m working on a project called BigRecord that’s attempting to bridge the gap between Hbase and RoR. You can find it on github: http://github.com/openplaces/bigrecord/tree/master

Bigrecord is an OM (not really relational anymore) with a similar API and codebase to ActiveRecord. The changes that needed to be done on ActiveRecord in order to work with Hbase were non-trivial, so it wasn’t just a matter of creating a new adapter for AR.

Bigrecord was being used internally at openplaces.org for nearly 2 years now, and when I joined the company, one of my tasks was to open source the tool. It’s lacking any useful documentation, but for the most part, it works along with a spec suite.

Something worth warning about is the stack required to get it all working. We don’t use the thrift or rest interface of Hbase, but instead went with a JRuby/DRb approach. This is because we found that using JRuby/DRb to communicate natively via the Java API was the fastest method to communicate with Hbase. (Disclaimer: don’t ask me to provide any real data proving this… I don’t have it :P )

Feel free to e-mail me greg.lu [at] gmail.com or find me on github: http://www.github.com/enell with any questions.

]]> By: Dude http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-602 Dude Thu, 23 Apr 2009 12:09:38 +0000 http://blog.rapleaf.com/dev/?p=16#comment-602 Wont the servlet serve as a bottleneck during the high write environment?..though it can be scaled using suitable hardware and app server but that would mean another additional headache to take care of.. Did you try jruby + hbase? Wont the servlet serve as a bottleneck during the high write environment?..though it can be scaled using suitable hardware and app server but that would mean another additional headache to take care of..
Did you try jruby + hbase?

]]> By: feedbackmine http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-592 feedbackmine Sun, 19 Apr 2009 06:02:50 +0000 http://blog.rapleaf.com/dev/?p=16#comment-592 This gem no longer works with latest habse server. For people who are need a working hbase ruby client, I recommend sishen's hbase-ruby: http://github.com/sishen/hbase-ruby/tree/master This gem no longer works with latest habse server.

For people who are need a working hbase ruby client, I recommend sishen’s hbase-ruby:
http://github.com/sishen/hbase-ruby/tree/master

]]> By: Paul http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-582 Paul Tue, 03 Feb 2009 16:35:58 +0000 http://blog.rapleaf.com/dev/?p=16#comment-582 The last update to this thread was over a year ago. Since rhino has been released, but is FAR from usable IMHO. Care to give us a status update on the ORM project? The last update to this thread was over a year ago. Since rhino has been released, but is FAR from usable IMHO. Care to give us a status update on the ORM project?

]]> By: Peter Dolan http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-572 Peter Dolan Sat, 05 Jan 2008 00:47:16 +0000 http://blog.rapleaf.com/dev/?p=16#comment-572 @bryan: That's understandable. It looks like Quinn Slack and I will be tackling the Ruby ORM mapping, which should be very useful for a new backend for reintegrate.us, which is effectively a large feedfetching system (exactly what a system like HBase is good at). Would it be possible for us all to get access to the ruby-hbase source via SVN or CSV? I looked around a bit but couldn't find any repository links. @bryan: That’s understandable. It looks like Quinn Slack and I will be tackling the Ruby ORM mapping, which should be very useful for a new backend for reintegrate.us, which is effectively a large feedfetching system (exactly what a system like HBase is good at).

Would it be possible for us all to get access to the ruby-hbase source via SVN or CSV? I looked around a bit but couldn’t find any repository links.

]]> By: Engineering Rapleaf - Ruby-HBase ORM Already? http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-562 Engineering Rapleaf - Ruby-HBase ORM Already? Fri, 04 Jan 2008 22:19:46 +0000 http://blog.rapleaf.com/dev/?p=16#comment-562 [...] only took a few weeks for someone to decide to start working on an ORM layer to go over the ruby-hbase gem. Here’s Quinn Slack’s announcment for Rhino. It looks pretty new, and I’m [...] [...] only took a few weeks for someone to decide to start working on an ORM layer to go over the ruby-hbase gem. Here’s Quinn Slack’s announcment for Rhino. It looks pretty new, and I’m [...]

]]> By: Introducing Rhino, a Ruby ORM for HBase | Quinn Slack http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-552 Introducing Rhino, a Ruby ORM for HBase | Quinn Slack Fri, 04 Jan 2008 21:20:55 +0000 http://blog.rapleaf.com/dev/?p=16#comment-552 [...] the Bigtable-like distributed storage system from Apache Hadoop, had no Ruby API. That changed when Bryan Duxbury released ruby-hbase, an interface to the REST API he wrote for HBase. But there’s no Ruby ORM for HBase [...] [...] the Bigtable-like distributed storage system from Apache Hadoop, had no Ruby API. That changed when Bryan Duxbury released ruby-hbase, an interface to the REST API he wrote for HBase. But there’s no Ruby ORM for HBase [...]

]]> By: bryan http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-542 bryan Fri, 04 Jan 2008 18:26:10 +0000 http://blog.rapleaf.com/dev/?p=16#comment-542 @Peter: We're definitely interested in a ORM-style wrapper on top of ruby-hbase, though it hasn't become a priority for us just yet. Lately I've been more focused on contributing to HBase itself, trying to make it faster and more stable. If you start to put something together, by all means let me know. @Peter: We’re definitely interested in a ORM-style wrapper on top of ruby-hbase, though it hasn’t become a priority for us just yet. Lately I’ve been more focused on contributing to HBase itself, trying to make it faster and more stable.

If you start to put something together, by all means let me know.

]]> By: Quinn Slack http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-532 Quinn Slack Fri, 04 Jan 2008 06:00:39 +0000 http://blog.rapleaf.com/dev/?p=16#comment-532 Thanks! Using the gem now. By the way, in your code example, it should be HBase instead of Hbase in "Hbase::HTable.new." Thanks! Using the gem now. By the way, in your code example, it should be HBase instead of Hbase in “Hbase::HTable.new.”

]]> By: Peter Dolan http://blog.rapleaf.com/dev/2007/12/24/ruby-and-hbase/#comment-522 Peter Dolan Thu, 03 Jan 2008 05:21:01 +0000 http://blog.rapleaf.com/dev/?p=16#comment-522 Very cool. Thanks for taking that on. I'm looking at switching from MySQL to HBase for www.reintegrate.us, have you made any progress on adapting ActiveRecord::Base to (perhaps) ActiveRecord::HBase? I'd be interested in helping you out on that project or taking it on if you don't feel you'll have time for it. Very cool. Thanks for taking that on. I’m looking at switching from MySQL to HBase for http://www.reintegrate.us, have you made any progress on adapting ActiveRecord::Base to (perhaps) ActiveRecord::HBase? I’d be interested in helping you out on that project or taking it on if you don’t feel you’ll have time for it.

]]>