Or JRuby for fun and profit

I’ve been developing software for some time now, and have recently found myself applying ideas from various esoteric areas of computer-science to every day tasks of building common-place applications (often these concepts are quite old, indeed some were thought of in 1958).

One powerful idea that I’ve been playing with recently is that of embedding a DSL (domain specific language) into your basic application framework - and writing most of the features of the application in that DSL.

This is simply an implementation of the concept of raising the level of abstraction. The point, of course, being that when writing code in a DSL implemented in such a fashion, one can express ideas in terms of high-level abstractions that represent actual concepts from the problem domain. In other words, it is like using a programming language that has primitives rooted in the domain.

A lot of people have been writing about this kind of software design - and most implement these ideas in a dynamic language of their choice. How does one go about doing the same in a language like Java? That is what this article is about. And I cheat in my answer. Consider the following design stack -

I propose that you only implement basic and absolutely required pieces of functionality in Java - the things that rely on, say, external systems that expose a Java interface, or some EJB-type resource, or some other reason that requires the use of Java. The functionality you develop here is then exposed through an interface to the layers above. You can also add APIs for other support services you might need.

The layer above is a bunch of JRuby code that behaves as a facade to the Java API underneath. This leaves you with a Ruby API to that underlying Java (and other whatever, really!) stuff - and makes it possible to code against that functionality in pure Ruby. The JRuby interpreter runs as part of the deployable and simply executes all that Ruby code transparently. As far as your Ruby code is concerned, it doesn’t even care that some of the calls are internally implemented in Java. Sweet!

We can stop here. At this point, we are in a position to write the rest of our application in a nice dynamic language like Ruby. For some people, a nice fluent interface in Ruby suffices - and keeps all developers happy. This is depicted on the top-left part of the diagram.

However, we can go one step further, and implement a DSL in Ruby that raises the level of abstraction even more - as referred to earlier. So on top of the Ruby layer, you’d implement a set of classes that allow you to write code in simple domain-like terms, which then get translated into a form executable by the Ruby interpreter. This is shown in the top right part of the diagram. Ultimately, potentially any one (developers, QA or business analysts) could express their intent in something that looked very much like English.

So where to write what?

How much to put in your Java layer depends on the situation - some people (like me) prefer to write as little as possible in such a static language. Others like the static typing and the associated tool support, and prefer to put more code here. When merely shooting for a little bit of dynamism through the DSL engine in the layers above, most of the code could be written in Java, and a fluent API in the dynamic language could be enough. When shooting for rapid feature turn-around and a lot more flexibility, most of the code could be in the DSL or in the external dynamic language.

The answer to this question really depends on things like the requirements, team structure, skill-sets, and other such situational factors.

OK, so where’s the code?

My intention with this post was to stay at a high level - and talk of how one could structure an application to make it possible to embed a scripting engine into it, and to give an overview of the possibilities this creates. In subsequent posts, I will talk about how actual DSLs can be created, tested, and also how a team might be structured around it.

I attended GoRuCo 2007 yesterday, and it was a fine conference - where I met a bunch of old friends I hadn’t seen in a while. Here’s what the agenda looked like - I enjoyed the talks on Adhearsion, JRuby, and BNLs. And the conference took place in Google’s offices in NYC, and that was neat.

It was cool how many people knew of ThoughtWorks, and were interested in talking about what it was like to work there. (Probably less than the number that were interested in Google in the same way, but hey…) We ended the conference by finishing up with the afterparty in the wee hours of the morning. It was fun! Amazingly, both Deepthi and I won swag at the event - I got this neat laptop bag, and Deepthi got Rails Recipes. Oh, and here are some pictures.

You might want to read some live-blogging from the event.

Everyone knows that global variables are bad. However, they are quite unavoidable. Java’s System.out is an example.

Globals, sometimes offer a certain kind of flexibility. They offer a way to tweak the behavior of the entire (or a subset of) the system. For instance, one can redirect System.out to a different stream (into a file, say) and capture all the messages a program spits out.

This kind of stuff works nicely. Except when someone else goes and changes the same variable to something else in another part of the code, and you’ve no idea where.

What is needed, is something like optional dynamic scope, so that the globals can be used when needed, but can be managed better. After all, from the above example, what seems to be needed is a way for a piece of code to say - for my purposes, and for all code that runs when I’m called, I want the value of this global(s) to be _something_, and when I return, these globals should be reset.

This can be done manually, by saving the existing value of a global before setting it to something else, and then resetting it back when the code block completes running. Perl and Common Lisp have had a mechanism to do this type of stuff for a long time, built into the language.

Here’s a hacky (and probably naive) way to implement this in Ruby, to illustrate how this might work -


module Let
def let(bindings, &block)
old_bindings = capture_existing_bindings_for(bindings)
block.call
rehydrate_old_bindings_with(old_bindings)
end
def capture_existing_bindings_for(bindings)
old_bindings = { }
bindings.each do |k, v|
old_bindings[k] = eval “@”+k.to_s
create_binding k, v
end
return old_bindings
end
def create_binding(var_name, value)
instance_eval “@#{var_name}=value”
end
def rehydrate_old_bindings_with(old_bindings)
old_bindings.each do |k, v|
create_binding k, v
end
end
end


And the way you would use this, would look something like this -


require ‘let_module’
include Let
@num_var = 1
@char_var = ‘a’
class Car
attr_reader :name
def initialize(name)
@name = name
end
end
@obj_var = Car.new(”hyundai”)
def do_something
puts “num_var is ” + @num_var.to_s + “, char_var is ‘” + @char_var.to_s + “‘, obj_var is ” + @obj_var.name
puts “returning”
end
do_something
puts “changing num_var to 2, char_var to b, obj_var to ‘kia’”
let :num_var => 2, :char_var => ‘b’, :obj_var => Car.new(”kia”) do
do_something
puts “changing num_var to 3, char_var to c, obj_var to ‘toyota’”
let :num_var => 3, :char_var => ‘c’, :obj_var => Car.new(”toyota”) do
do_something
end
do_something
end
do_something


And when run, would produce this -

num_var is 1, char_var is ‘a’, obj_var is hyundai
returning
changing num_var to 2, char_var to b, obj_var to ‘kia’
num_var is 2, char_var is ‘b’, obj_var is kia
returning
changing num_var to 3, char_var to c, obj_var to ‘toyota’
num_var is 3, char_var is ‘c’, obj_var is toyota
returning
num_var is 2, char_var is ‘b’, obj_var is kia
returning
num_var is 1, char_var is ‘a’, obj_var is hyundai
returning

This could be one way to control unruly global variables in Ruby.

I downloaded and installed CruiseControl.rb today - from the ThoughtWorks website. Well, I suppose ‘install’ is not quite the right word as it comprised entirely of running a couple of commands, and took all of 30 seconds.

It works beautifully, right out of the box! The example on the website didn’t have any reference to the situation where you connect to your subversion server over ssh, so when I went ahead and tried it, it succeeded and then gave me the following message -

IMPORTANT!!! - It looks like you are connecting to your repository with an svn+ssh connection. For cruise to build this project, you need to have set up authentication caching for ssh, see this article
http://subversion.tigris.org/faq.html#ssh-auth-cache

What a pleasant surprise! I like software that was designed with users in mind. The information above for setting up ssh authentication works like a charm, btw.

Emacs, the God of all things. I’ve been working my way through, over the past several years (after having been introduced to it by Ravi), all kinds of books and tutorials on lisp. Unfortunately, I’ve not been particularly good at following through and becoming a master, so I’m not really a guru or anything when it comes to Emacs skills. (I promise, it will change this year.)

Anyway, while working on Ruby, I was looking to find a good editor. Who isn’t, after all? I’ve used Eclipse with RDT and RadRails, but it wasn’t compelling (that might change with this stuff). After moving to the Mac, however, I was looking to buy TextMate - which I’ve heard a lot about. Before buying it, I thought that since it was “inspired” by Emacs, why not give it a shot? Ultimately, I got comfortable using Emacs itself, so I haven’t really given TextMate a real shot, despite owning a license (thanks MacHeist!).

So. This is my first in a series of posts about using Emacs for Ruby and Rails. The “cool” thing for this time is - inferior mode for Ruby. This provides a SLIME-like interface for Ruby, and allows you to work really interactively. What that means is, while typing in the editor, you can ask fragments of code to be interpreted, change only particular definitions quickly and rerun other code that uses them, pipe all that back and forth between windows, so on and so forth. And at any time, you can switch to the Ruby process (running with Emacs using irb) in use and explore the current environment.

Check it out, its very neat and it even comes with Ruby.

I tried getting rails up and running on my (fairly) new iMac a few days back, and although I installed mysql gem currectly, the code was unable to connect to the mysql server. In fact, it threw an error -

mysql.c:2015: error: ulong undeclared

It befuddled me for a bit, because when the mysql gem is installed (gem list shows it), it compiles the native extensions - and that had succeeded. Or so I thought. Long story short - there appears to be a problem when trying to compile this gem on Intel Macs - and here’s the fix.

1. Go to gems directory - it will be somewhere under your ruby directory. Edit mysql.c, and add “#define ulong unsigned long” to it.
2. Then do ruby extconf.rb install mysql — –with-mysql-dir=yourmysqldirectory/
3. make
4. make install

This fixed things, and now, mysql can be used satisfactorily.

That was nightmarish!

All I wanted was SQLite working with Rails. That’s all. And it was quick and easy - on my Ubuntu workstation at home, that is. And then, began 2 hours of gruesome agony - as I tried desperately to get the damn thing to work on my RHEL staging box.

I only have shell access to this machine - so no fancy graphical package manager. If I had a package manager, I would have selected libsqlite0-dev (the missing item) as well as libsqlite3-dev just to be safe, and would have saved myself the afore-mentioned two hour hair-pulling.

Here are the steps to get SQLite working with Rails -

* Download sqlite from http://www.sqlite.org/download.html. Compile and install the usual way. Or just get the compiled versions.
* Install libsqlite-dev (both libsqlite-dev3 AND libsqlite-dev0-> AAARGH!)
* Download, compile and install swig -> http://www.swig.org/download.html
* Install the sqlite3-ruby gem
* Downlad ruby-dbi and follow instructions on http://ruby-dbi.rubyforge.org/

Everything should work now! I tested this with Typo 4.0.3 and it works. Phew!