During our last big rails project, Bloomfire, we found ourselves integrating with all kinds of external services. Because of this we had a diverse set of environment dependent configuration variables. A consistent pattern started to arise where we would extract our configuration variables into a YAML file and then wrap the configuration using a small class wrapper. This eventually gave rise to Environment Configurable, a library that makes environment dependent configuration easy in rails.

Version 1.0 of the Google Collections library was officially released December 30, 2009. I have been using the library for the past 6 months or so on a variety of Java projects, with great success. Today I gave a brown-bag talk to share with some interested Atoms.

I have included below the sample code I used as presentation material, in case others are interested.

Later this month, Shawn Anderson will be going back to Cali, his previous home, to make back-to-back presentations at two different conferences: LA RubyConf and SCALE (the second time in two years). For both, he is presenting on developing 2D games using Ruby and open source tools (including a library that he wrote).

One of the great things about the JRuby project is that it’s easy to run Ruby programs without installing Ruby. In fact, you don’t even need to install JRuby. All you need is a JVM runtime and jruby-complete.

Rationale

Check out this other post for a discussion of my reasons for locking down your JRuby runtime. In summary, embedding jruby-complete gives you complete control of your Ruby runtime. That’s a good thing. The downside is that discovering and executing commands through jruby-complete can be a pain. The rest of this post describes how to ameliorate the pain.

Running jruby-complete

java -jar jruby-complete-1.4.0.jar

java -jar jruby-complete-1.4.0.jar -e "puts 'Hello'"

java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -e "puts 'Hello'"

java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -e 'load "META-INF/jruby.home/bin/jirb"'

1
2
3
4

irb(main):001:0> puts "Hello"
Hello
=> nil
irb(main):002:0> %

1
2
3
4
5
6
7
8
9
10
11
12
13
14

JRUBY_COMPLETE = "jruby-complete-1.4.0.jar"
JRUBY = "java -Xmx500m -Xss1024k -jar #{JRUBY_COMPLETE}"

namespace :jruby do
  desc "Run JRuby help"
  task :help do
    sh %+#{JRUBY} --help+
  end

  desc "Run any command with JRuby"
  task :run do
    sh %+#{JRUBY} -e '#{ENV["cmd"]}'+
  end
end

Now I can type rake jruby:run cmd='puts "Hello"'. Shell escaping is becoming a real annoyance at this point. Thankfully, I’m usually not using jruby-complete to run silly little commands. By the time I’ve introduced a Rakefile I’ve got a real application with tasks oriented around testing and running it, so it’s rare that I’m using a task like jruby:run very often.

1
2
3

task :run do
  sh "#{JRUBY} lib/application_bootstrap.rb"
end

1
2
3
4

fletcher-git/github/jruby-complete-example(master) rake run
(in /Users/fletcher/git/github/jruby-complete-example)
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar lib/application_bootstrap.rb
Hello from application_bootstrap

1
2
3
4
5
6
7

  namespace :spec do
    desc "Run RSpec against a specific file"
    task :run do
      raise "You need to specify a spec with spec=" if not ENV["spec"]
      sh %+#{JRUBY} -S spec -f specdoc #{ENV["spec"]}+
    end
  end

1
2
3
4
5

describe "John Galt" do
  it "does not tolerate logical fallacies" do
    "A".should == "A"
  end
end

1
2
3
4
5
6
7
8
9
10

fletcher-git/github/jruby-complete-example(master) rake spec:run spec=spec/unit/objectivism_spec.rb
(in /Users/fletcher/git/github/jruby-complete-example)
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -S spec -f specdoc spec/unit/objectivism_spec.rb

John Galt
- does not tolerate logical fallacies

Finished in 0.123 seconds

1 example, 0 failures

1
2
3
4
5
6
7

fletcher-git/github/jruby-complete-example(master) which sp
sp () {
        rake spec:run spec=$@
}
fletcher-git/github/jruby-complete-example(master) sp spec/unit/objectivism_spec.rb
(in /Users/fletcher/git/github/jruby-complete-example)
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -S spec -f specdoc spec/unit/objectivism_spec.rb

Alright, so now that my application has been built up, I might want to start compiling the .rb files into .class files. Here comes jrubyc:

1
2
3
4
5
6
7
8
9
10
11
12

require "rake/clean"

namespace :jruby do
  output_directory = "classes"
  directory output_directory
  CLEAN.include output_directory

  desc "Compile Ruby files in lib"
  task :compile => output_directory do
    sh %+#{JRUBY} -S jrubyc -p com/atomicobject -t #{output_directory} lib+
  end
end

1
2
3
4
5
6
7
8
9
10
11

fletcher-git/github/jruby-complete-example(master) rake jruby:compile
(in /Users/fletcher/git/github/jruby-complete-example)
mkdir -p classes
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -S jrubyc -p com/atomicobject -t classes lib
Compiling all in '/Users/fletcher/git/github/jruby-complete-example/lib'...
Compiling lib/application_bootstrap.rb to class com/atomicobject/lib/application_bootstrap

fletcher-git/github/jruby-complete-example(master) rake jruby:run cmd='require "classes/com/atomicobject/lib/application_bootstrap"'
(in /Users/fletcher/git/github/jruby-complete-example)
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -e 'require "classes/com/atomicobject/lib/application_bootstrap"'
Hello from application_bootstrap

1. enabling antialiasing in Apple’s JVM via a Java property.
2. tweaking Substance’s widget behavior via a Java property.
3. enabling Yourkit Java Profiler via the -agentlib parameter.
4. including libraries and directories in the JVM’s classpath via the -cp parameter.

Since these parameters need to be passed before the -jar parameter, a more sophisticated method for setting up the JRuby command is needed than the constant I’ve used. A method like that is specific for your application and beyond the scope of this post, but is not be difficult to create.

Conclusion

There are an uncountable number of good things about JRuby and jruby-complete is one of them. A little help from scripts and your shell means you can build and run your application with a controlled Ruby runtime.

Additional resources

• The Rakefile, jruby-complete, and other files used in this post are available in this GitHub project.
• JarJar Links is an ant library that is useful for combining multiple jar files together.
• I wrote a post a while ago about using JarJar to combine jruby-complete and other application dependences into a single file.
• The AGI Production Simulator is built using the jruby-complete commands described in this post as well as the above jar-rolling technique.

1. Replicating the true jruby behavior is way, way beyond the scope of this post. Check out the jruby script if you really care. Most of the time the JVM heap and stack sizes are the most important things to worry about. ↩

Edit 2/6/2010: Reduced -e ‘load…’ parameters to -S

Is your Ruby C Extension lonely? FFI can help

Lonely? Many Ruby programmers have found the need to make calls on C libraries. Sometimes this is for writing performance-critical code and sometimes it is to use an existing library written in C. Ruby extensions are created by writing a small amount of glue code in C. The downside is that you need to compile your C code wrapper. This requires you to either release versions of the compiled code for all platforms or force your users to have a development environment. Furthermore, the extension is only guaranteed to run on the standard Matz Ruby Implementation (MRI). This means that your shiny new extension cannot necessarily be used by other Ruby VMs such as JRuby or MacRuby.

Make friends. If you want your C code to play nicely with the other VMs out there, then FFI (foreign function interface) is the answer. FFI is built-in to JRuby and MacRuby; MRI requires a compiled gem for FFI access. After a night of experimentation, I was able to get C code from the Chipmunk Physics library running in Ruby via FFI (chipmunk-ffi). Just tell FFI what function to attach, what arguments it takes, what the return type is, and it will take care of the rest. FFI is a great tool that any Ruby developer should have in their tool belt and consider against writing a C extension.

C code:

void cpInitChipmunk(void);

To call it from Ruby:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

# sample from chipmunk-ffi

require 'ffi'
module CP
  extend FFI::Library
  # load a C library
  ffi_lib 'chipmunk'
 
  # look up a function
  attach_function :cpInitChipmunk, [], :void

  # call the new function
  cpInitChipmunk
end

More Info:

why use FFI

projects using FFI

presentations on FFI

I love our morning stand-up meetings: short, low-stress, and effective. Besides agenda and news, we often share technical topics with the group.

Here’s a quick list of some recent projects we’ve noticed or discussed:

• Rango – a new web application framework in Ruby, influenced by the success of the Python project Django. There’s a nice overview in PDF form.
• Atomic Object’s github account was started, hosting a few of our open-source projects.
• jesus is a web interface to God.
• jaml is an HTML template library for JavaScript, influenced by Haml.
• Lawnchair is a JavaScript client-side document store.

• bundler – easily bundle gems within an application. This tech has spread like wildfire through the office – I think it is something we’ve been wanting at Atomic for a long, long time now.
• git-deploy – deploy new code with a single “git push” command. I’ve forked this project and made some modifications for to suit our project.
• cancan – “Simple authorization for Rails”
• gritter – jQuery code that gives growl-like notifications. It works well and has been well received by most of our users. (That said, I personally cannot stand growl and think notifications like this should die in a fire.)
• culerity – celerity hooks for cucumber. We’re definitely not the first project to use it, but Drew and I have made extensive modifications to improve culerity’s process management.

One of the nice things about kicking off a new application is that it’s the best time to integrate new, exciting technologies like these. Discovering and integrating them has helped make our development process better than ever.

I’ve just re-planted our VMWareFB OpenStep display driver to Github, and I’ve updated our existing web page accordingly. Precompiled, compressed configs are now available in the downloads section of the new Github project page.

Special thanks to Andreas Grabher and other folks over at www.nextcomputers.org for solving the chronic problem of running at resolutions higher than 1024×768. Andreas has provided an updated source tree and config file, so after 8 years we’re moving from 1.0.0 to 1.1.0.

It’s been almost 8 years since Bill Bereza first wrote this SVGA driver for our customers over at Valley City Linen and we’ve hardly tinkered with it since. Lots of people have found and used it over the years, and are still using it today. It’s a pleasant surprise that it continues to work reasonably well for people out there in NeXT land.

Now that the code is out where people can freely update and re-release it, I hope to pull in a few interested members of the NeXT/OpenStep community and let them take it from here. I know a few people have privately hacked or updated the driver to work better for their own installations; I’d love to have them fold those updates back into the public source.

(Thanks to Marissa Christy for the cool logo sketches!)

I have been working on a DataMapper adapter for a RESTful JSON web service. I put together a gem that includes the adapter implementation, as well as a few DataMapper::Resource classes that can be used as a starting point. I did not want to install my gem’s dependencies globally while developing it, so I ended up trying a couple of different things to manage my development environment.

The first thing I tried was rip. I had not used rip previously, but it looked like a good fit for gem development since you don’t need to distribute the dependencies with the gem. I set up a rip environment for my gem development, and at first it worked very well. I could use it when I was working on the gem or switch to an empty environment when working on something else. After running into some issues with my DataMapper adapter and a newer version of dm-core, I needed to reinstall several dependencies. Upon trying to re-install dm-serializer I was confronted with the following:

1
2
3
4
5
6

]rip install dm-serializer
undefined method `strip' for nil:NilClass
undefined method `strip' for nil:NilClass
undefined method `strip' for nil:NilClass
rip: install failed
-> undefined method `strip' for nil:NilClass

Google didn’t turn up anything, and I really didn’t have time to look into the problem. I think rip is a great project, but it was not going to solve my immediate problem without some debugging, so I moved on. Next I turned to Bundler.

I converted my deps.rip to the following Gemfile:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

source "http://gemcutter.org"
source "http://gems.github.com"

bin_path "vendor/gems/bin"

only :test do
  gem "rspec"
  gem "mocha"

  gem "dm-core", "0.10.1"
  gem "dm-serializer", "0.10.1"
  gem "dm-validations", "0.10.1"
  gem "json", "1.2.0"
end

disable_system_gems

I only need the runtime dependencies when running the tests, so everything goes in the only :test section. And I don’t want any bin scripts from my dependencies to end up as part of my gem, so I configure them to go in vendor/gems/bin. I also don’t want any of the dependencies to be included in the packaged gem, so I exclude them from the Jeweler configuration in the Rakefile:

1
2
3
4

Jeweler::Tasks.new do |gemspec|
  ...
  gemspec.files.exclude "vendor"
end

Finally the dependencies are loaded in the spec_helper.rb:

1
2

require File.expand_path(File.join(File.dirname(__FILE__), '..', 'vendor', 'gems', 'environment'))
Bundler.require_env :test

After making these changes I ran “gem bundle” and was able to continue development without needing to globally install any of my new gem’s dependencies. So far my experience with Bundler on this project and others has been excellent.

Recently, a colleague and I were experimenting with SWFUpload, a popular flash based file upload tool. We quickly realized that SWFUpload cannot forward cookies; and since our rails application utilizes cookie based sessions we were unable to upload a file. After a little bit of research we stumbled across a blost post that offered a solution using rack middleware.

In short, it has you add the required cookies as form parameters and then insert a rack middleware object before the rails session store. This object is responsible for extracting the form parameters and then inserting them into the “HTTP_COOKIE” header before rails is invoked. After following the instructions on the blog post we were able to successfully upload a file using SWFUpload.

This particular problem inspired me to write a lightweight rack middleware library called Cookie Monster. It solves the very problem that was described above and is wrapped up in an easy to use gem (or rails plugin).

 gem install rack-cookie-monster

or

script/plugin install -f git://github.com/dewind/rack-cookie-monster.git

Configuring it for rails is easy:

1
2
3
4
5
6
7
8
9

# In rails initializer
  Rack::CookieMonster.configure do |c|
    c.eat :_session_id
    c.eat :user_credentials
    c.share_with /^(Adobe|Shockwave) Flash/
    c.share_with "Burt"
  end

  Rack::CookieMonster.configure_for_rails

Last weekend I attended JRubyConf 2009. Overall I enjoyed the opening JRubyConf State of the Union, the follow-up JRuby on Rails, and the concluding JRuby Core Team Panel sessions the best. Despite browsing the JRuby mailing list as part of my daily routine, JRuby has a rapid development cycle; these sessions helped me get a clearer view of the JRuby team's current development goals.

I am curious when and how JRuby will take advantage of the invokedynamic and method handle technologies coming with the next Java platform release. After reading John Rose's paper Bytecodes meet Combinators: invokedynamic on the JVM, it sounds like language implementors will need to make a significant development investment if they want to fully utilize the new features. Charles clearly has a strong grasp on the new features. He's already tried out and given feedback on them, so I'm sure he'll be on the forefront of an implementation changeover. Still, I wonder what kind of timeframe we're looking at to go from the simulated method dispatching we have now to a fully fleshed out invokedynamic-backed implementation.

Demetrius Nunes recently posted about his Swinger library to the JRuby mailing list. Swinger uses Cucumber and Jemmy to provide a test harness for automating Swing tests.

I responded to Demetrius letting him know how great it is to hear others are interested in automating Swing testing. We’ve been using a combination of FEST and Ruby helpers specific to our own Swing applications. As Demetrius notes, Swing tests can be frustratingly brittle, but they are still well worth the investment. Swinger looks like it is both extensible and can help alleviate some Swing testing pain.

You can check out my response here and the entire discussion here.

This style is great for testing because it clearly establishes intent for each test. This point is invaluable when another developer either inherits your code, or you revisit it at some point in the future. The main problem with this style is the need to create stubs for all the calls in the describe's before block, or resort to using stub_everything for all your mock objects. These are both fine solutions for simple code, but as soon as the logic encounters a condition you may be setting yourself up for potential testing bugs, and difficulty when you add new logic.

One way to avoid this problem is to leverage the power of rspec, and use nested describe blocks for each condition statement. This helps limit the before block for each describe to only define the stubs for that condition block. This is helpful because it prevents the tester from accidentally defining a stubbed expectation for something that should not happen. It also creates a nice placeholder for adding logic into the code in the future.

The following example shows how nesting rspec describes works. It is important to note that the branch that contains the most logic should be encapsulated in the sub-describe.

Example Source Code:

1
2
3
4
5
6
7
8
9
10
11
12
13

class Example 
  def run(director, runner)
    runner.prepare
    if director.go?
      runner.run
      if runner.complete?
        director.stop
      end
      runner.stop
    end
    runner.finish
  end
end

Example Test Code with nesting describes:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

describe Example do
  before do
    @target = Example.new
  end
  
  describe '#run' do
    before do
      @director = mock
      @runner = mock
      
      @runner.stubs(:prepare)
      @director.stubs(:go?).returns false
      @runner.stubs(:finish)
    end
    
    it "should prepare the runner" do
      @runner.expects(:prepare)
      @target.run @director, @runner
    end
    
    it "should check the director" do
      @director.expects(:go?).returns false
      @target.run @director, @runner
    end
    
    describe 'director returns true for go' do
      before do
        @director.stubs(:go?).returns true # redefine
        @runner.stubs(:run) 
        @runner.stubs(:complete?).returns false
        @runner.stubs(:stop)
      end
      
      it "should run the runner" do
        @runner.expects(:run)
        @target.run @director, @runner
      end
      
      it "should check if the runner is complete" do
        @runner.expects(:complete?).returns false
        @target.run @director, @runner
      end
      
      describe 'runner returns true for complete' do
        before do
          @runner.stubs(:complete?).returns true # redefine
        end
        
        it "should stop the director" do
          @director.expects(:stop)
          @target.run @director, @runner
        end
      end
      
      it "should stop the runner" do
        @runner.expects(:stop)
        @target.run @director, @runner
      end
    end
    
    it "should finish the runner" do
      @runner.expects(:finish)
      @target.run @director, @runner
    end
    
  end
end

I realize the test code gets a bit long, but it is very easy to understand and extend. Also, nesting on conditional breaks creates an easy framework to follow when writing test code.

You can check it out on iTunes.

There were some interesting experiences with iPhone development that we'll cover in future blog post. One useful tip to know is that the UIImage class will try to manage its own memory if you construct it using the initWithContentsOfFile: method. This means it will unload and reload the image from the file as needed, which can kill performance if you're dealing with a lot of offscreen images that need to be shown on screen quickly. Using initWithData: using an NSData will let you manage the memory yourself.