I always have a shell open. Most of my daily project navigation and file-system work is done in Bash. If I can do something with a GUI or a terminal, I’ll prefer the terminal. I’m a polyglot programmer who often uses Windows, Linux, and Mac OS X all in the same day. There’s a tool I need whenever I work on the command-line: Working Directory.

Here’s a scenario:

Your project uses Java and Maven. That means the first useful bits of your code are buried somewhere like this:

/Users/capt.picard/work/FooFoo/src/main/java/org/foo

So you start out a new shell in your home directory and navigate all the way down there:

1
2
3
4

/Users/capt.picard$ cd work
/Users/capt.picard/work$ cd FooFoo
/Users/capt.picard/work/FooFoo$ cd src/main/java/org/foo
/Users/capt.picard/work/FooFoo/src/main/java/org/foo$

Whew! Maybe not a big deal once, but I hope you don’t have to do this every day! Fortunately you have command-line completion, cutting down on all that typing:

1
2

/Users/capt.picard$ cd wo[TAB]/Foo[TAB]/s[TAB]/m[TAB]/j[TAB][TAB][TAB]
/Users/capt.picard/work/FooFoo/src/main/java/org/foo$

Wait, what? That’s still a lot of typing! This is much quicker for users of Working Directory:

1
2

/Users/capt.picard$ wd
/Users/capt.picard/work/FooFoo/src/main/java/org/foo$

That’s it! Let’s see how that magic works.

Whenever you start a new project or workspace, just tell Working Directory (or just “wd”) that you want to make a new scheme:

/Users/capt.picard$ wdscheme foofoo

This creates a set of 10 slots associated with that scheme and copies in the contents of your previous scheme. (If you don’t need a clone of the previous scheme, you can clear those out with wdc.) Now you simply navigate to your project root, or wherever you most commonly need to start when you work on that project, using regular cd and all the tab-completions you can tolerate. Once there, you can store that directory in the default slot like this:

/Users/capt.picard/work/FooFoo/src/main/java/org/foo$ wds

If you then look at the list of slot assignments with wdl, you’ll see that you’ve stored the current directory in the 0th slot:

1
2
3
4
5
6
7
8
9
10
11

/Users/capt.picard$ wdl
0 /Users/capt.picard/work/FooFoo/src/main/java/org/foo
1
2
3
4
5
6
7
8
9

The first slot is special—you can access it without specifying a number at all. Just type wd to navigate to that slot and wds to set it to the current directory. The remaining slots require their number to retrieve or store. If you want to store the directory that holds your project tests in slot 3:

1
2
3
4
5
6
7
8
9
10
11
12
13

/Users/capt.picard$ cd work/FooFoo/src/test/java
/Users/capt.picard/work/FooFoo/src/test/java$ wds3
/Users/capt.picard/work/FooFoo/src/test/java$ wdl
0 /Users/capt.picard/work/FooFoo/src/main/java/org/foo
1
2
3 /Users/capt.picard/work/FooFoo/src/test/java
4
5
6
7
8
9

Now you can quickly navigate between source and test on the command-line:

1
2
3
4

/Users/capt.picard$ wd
/Users/capt.picard/work/FooFoo/src/main/java/org/foo$ wd3
/Users/capt.picard/work/FooFoo/src/test/java$ wd
/Users/capt.picard/work/FooFoo/src/main/java/org/foo$

You can have as many schemes as you want, but each scheme only stores 10 directories. This allows for maximum accessibility on the command-line with single-word aliases for going to or storing into each slot.

Working Directory has many other features that you may start to use over time:

• wdenv stores the current scheme into similarily-named environment variables for easy use on the command-line for any purpose, like cp.
• Storing directories in slots doesn’t require that you be in the directory. Just give a directory as a parameter to wds[0-9] and it will store it instead of the current directory.
• wd is cross-platform, written in Bash and Perl. I’ve used it heavily over the years on Windows+cygwin, Linux, OS X, and Solaris.
• wd provides a bash-completion function for its schemes. wdscheme [TAB][TAB] will list of all of your existing schemes.
• Since wd stores schemes and slots on disk, all of your terminals will share the same scheme, so there’s no need to set the scheme every session. Storing a directory from one terminal makes it available in any other terminal.
• You can also temporarily override the working scheme in a particular shell by just setting the value of the environment variable WDSCHEME. If this variable is set, it’s used instead of the current scheme on disk.

Working Directory has been available for some years and gets constant daily use by its small community. Most people who’ve started using it never stop! Lately you can fetch the source from github or download the latest stable release directly from the project’s Downloads area.

For the last several weeks I have been creating a tool that accepts a set of configuration files and builds a number of charts based on information in those configuration files. For a few reasons I chose to use Ruby and the WIN32OLE support in Excel for staging the data and creating the charts. There may be better Windows solutions to creating charts, but Excel isn’t bad at it and the OLE automation in Ruby makes it fairly easy to drive Excel.

Over the last few months I have helped a number of other projects in the office setup bundler. I’ve been using bundler with my new Rails application for the last six months and have earned a reputation as the bundler master within the office.

The fact of the matter is that, although I’m happy to help, the bundler website is so well done that my help was not strictly necessary. I’m a big fan of the large text, code snippets, succinct descriptions, and piecewise instructions. I’d say it is nothing short of negligent for someone using bundler to not read through the site at least once.

On a related note, aside from bundler, the git-deploy project has been the other piece of really awesome technology we’ve used on this latest Rails application. git-deploy makes great use of git and Passenger to make for simple and extremely fast deployments. It has also been trivial for me to add some of my own extensions to it. Some of my favorite moments have been when I’ve deployed a new version of the application to our staging system, while our tester was working on it, and have her not even notice. Big thanks to Mislav Marohnić for creating git-deploy.

• Some of the Problems Bundler Solves
• Ruby Require Order Problems
• Named Gem Environments and Bundler
• The How and Why of Bundler Groups

One of the things I find myself yearning for in a lot of programming languages is a powerful pattern matching system. I wrote one for ruby, but ruby's syntax just wasn't flexible enough to make something as elegant as I'd like. When I started using clojure, it seemed like a great little project for getting to know clojure's macro facilities as well as clojure itself. I set out to build a kickass pattern matching library for clojure that fit in with the language's way of doing things and was at least as expressive than any other pattern matching facility I've used.

The outcome of this effort, is matchure (github, clojars), a pattern matching library for clojure featuring

• equality checks,
• sequence destructuring,
• map destructuring,
• regexp matches,
• variable binding,
• "instance of" checking,
• arbitrary clojure expressions,
• and boolean operators (and, or, not).

All of which compile down to high performance clojure.

Introduction

Clojure, like most lisps, has a built-in facility which gets you part of the way there: destructuring in most variable-binding contexts. For example,

1
2
3

(let [a 1]
  a) 
; returns 1

binds 1 to the variable a. You can just as easily grab the first value out of a sequence:

1
2
3

(let [[fst & rst] (list 1 2 3)]
  [fst rst])
; returns [1 (2 3)]

This facility is even more powerful. You can also destructure maps as well. There's just one problem: let doesn't work well as a way of testing values.

1
2
3

(let [[fst & rst] (list)]
  [fst rst])
; returns [nil nil]

This is, in part, why I wrote matchure. Matchure provides a pattern matching facility that makes it really easy to perform complex tests against values and bind parts of them to variables.

Syntax and Examples

Like any lisp, clojure is homoiconic - clojure code is represented by clojure data structures. Unlike other lisps, however, clojure has a rich set of data structures with literal representations. '(1 2 3) is a linked list, [1 2 3] is a vector, with fast random access, and {:a 1, :b 2} is a hash map. Furthermore, these syntaxes have special places within clojure itself. Lists are clojure function or macro calls and vectors are used anywhere variables are bound or sequences are destructured. Aside from being a literal representation in code, the hash syntax is used for destructuring maps in let bindings.

Matchure takes this rich set of literal representations and uses it as a natural way to match patterns. At the moment, there are three main forms, if-match, when-match, and cond-match. They work similarly to the clojure built-ins, but match values.

At the most basic level, matchure can be used to test for equality.

1
2
3
4
5

(if-match [nil nil] true) ;=> true
(if-match [1 1] true) ;=> true
(if-match ["asdf" "asdf"] true) ;=> true
(let [s "asdf"]
  (if-match ["asdf" s] true)) ;=> true

Regular expression literals test for a match

(if-match [#"hello" "hello world"] true) ;=> true

and fully qualified class names test for instance? relationships.

1
2

(if-match [java.lang.String "foo"] true) ;=> true
(if-match [java.lang.Comparable "foo"] true) ;=> true

Matchure supports _ and ? as wildcards, so both of these match anything

1
2

(if-match [_ "foo"] true) ;=> true
(if-match [? "foo"] true) ;=> true

_ is idiomatic in clojure when you want to ignore a value. In matchure, ? has special meaning. ? can be thought of as "the thing this part of the pattern is matching against". As such, ? always matches successfully. It is also used in binding variables. ?foo matches successfully and stores the matched value in the variable foo.

(if-match [?foo "bar"] foo) ;=> "bar"

just like regular clojure, list literals represent clojure code. Here, too, the special meaning of ? comes into play. You can perform arbitrary tests this way using ? to represent the matched against value:

(if-match [(odd? ?) 1] true) ;=> true

Just like with let, you can destructure sequences

(if-match [[?fst & ?rst] [1 2 3]] [fst rst]) ;=> [1 (2 3)]

and maps

(if-match [{:foo (even? ?)} {:foo 2}] true) ;=> true

Finally, unlike any other pattern matching facility I've seen, matchure has support for boolean operators.

1
2

(if-match [[(and ?fst (even? ?)) & ?rst] [2 3 4]] [fst rst]) ;=> [2 (3 4)]
(if-match [[(and ?fst (even? ?)) & ?rst] [1 2 3]] [fst rst] :failed-match) ;=>:failed-match

or and not are also supported.

You can get the code from github, or use it in your clojure project by grabbing it from clojars.

Designing with film:

In these experiments we designed and invented spaces, objects, movements and audiovisual techniques that map and visualise the interactive phenomena of RFID. Many of the visual/cinematic concepts for Nearness and Immaterials were invented by exploring and experimenting with film.

Rather than investing time in creating complex software and hardware prototypes, the interactive experience can be quickly explored inside film compositing applications. These experiments have shown us that there is great value in having tools that offer efficient prototyping of interactions at an experiential level, that don’t need to rely on complex electronics or physical design. There is also value in working within a medium that is not tied to a specific location or a unique demonstrator, and that is editable, reproducible and transmissible allowing it to be shared freely and widely amongst a research group and across the internet.

The sample videos, in my opinion, aren’t particularly compelling. That said, the essential idea itself has a good deal of merit—use digital video and post production tools to mock up concepts, interactions, and experiences.

Post production video tools are powerful. Design experiments in video could provide rich, meaningful feedback and move far beyond 2D, static mock ups. Creating the video itself is an exercise in working through the essential design challenges. Because it’s video there is opportunity to confront issues like transitions and movement of elements that is difficult to do with static mock ups. Using this approach could apply equally well to user interface development and physical product design.

This week one of our customers asked us to create a small, single-purpose tool to help support one of our JRuby desktop applications. The requirement was simple: make it easy to input minimum, maximum, and beta values into a beta distribution function for a few thousand samples and visualize the results.

After a few hours of work we produced this:

1. We brought in JFreeChart, a powerful, yet easy-to-use charting library for Java.
2. We used the beta distribution functionality from the Bayesian Logic (BLOG) Inference Engine project.
3. We had prior experience with these two libraries from our JRuby projects.
4. The NetBeans visual Swing editor made it trivial to get some simple widgets on the screen.
5. As always, JarJar Links allows us to distribute this application as a single jar file.

Factor #1, JFreeChart, is far and away the most significant. Our experiences with JFreeChart are all positive: getting data into a nice looking chart has always been straightforward. The API is relatively easy to work with in that it strikes a good balance between configurability and getting a chart on the screen with minimal hassle.

The next time you’re looking to visualize data in your JRuby or Java application, don’t overlook JFreeChart.

During our last big rails project, Bloomfire, we found ourselves integrating with all kinds of external services. Because of this we had a diverse set of environment dependent configuration variables. A consistent pattern started to arise where we would extract our configuration variables into a YAML file and then wrap the configuration using a small class wrapper. This eventually gave rise to Environment Configurable, a library that makes environment dependent configuration easy in rails.

Many times during the sales process we are asked to give a cost estimation based on our early understanding of an application’s core features. In order to responsibly and efficiently estimate, we conduct a 50/90 range analysis. We use this technique because we want to establish a responsible middle ground between our optimistic and pessimistic estimates. Our estimate is appropriately buffered for the risk we see in the defined tasks.

This spreadsheet is the tool that we use to conduct our analysis. Tasks can be estimated in any time duration that you choose (i.e. hours, days, weeks). At the start of a large project we generally estimate tasks in days or weeks.

Version 1.0 of the Google Collections library was officially released December 30, 2009. I have been using the library for the past 6 months or so on a variety of Java projects, with great success. Today I gave a brown-bag talk to share with some interested Atoms.

I have included below the sample code I used as presentation material, in case others are interested.

Why in the world would you want to embed JRuby into your application instead of relying on a regular Ruby or JRuby installation? I can think of three reasons.

Note: this is a sister post to my description of the arguments needed to run JRuby via jruby-complete. Here I discuss the rationale for using JRuby like this.

What’s the upside?

You can bundle your Ruby runtime dependence into the application.

If you don’t know what I’m talking about here then you should probably stop reading right now. An old post by Err the blog summarizes the “vendor everything” principle quite well:

Quickly: fix it! Tell everyone to install the gem locally. Install the gem on your staging server. Carefully install the gem on your production server. Phew. Everyone’s got the same version, right? Right. Well, maybe. (At least the build works.)

1. I don’t want to ask my users to install anything outside of my application. I definitely don’t want to count on the end user having something installed. Depending on end users having a compatible version of Java is frustrating enough, let alone a JRuby installation. Depending on the JRuby installation can be annoying for developers as well. Take culerity as an example. You could spend a bunch of time screwing around with getting your system-wide JRuby installation just right. Or you could embed JRuby directly into culerity and be done with it.
2. I don’t need to worry about version incompatibilities between the various runtimes. Incompatibilities can mean features I depend on are unavailable, subtly different, or outright broken. An interesting example of “subtly different” came up in an application I recently developed. At one point the sort implementation in JRuby changed from a stable sort to an unstable sort. As far as I know, Ruby doesn’t guarantee sort stability one way or another, so the change was ok from a Ruby perspective. Not so much for me. It turns out my application assumed a stable sort. Had I been dependent on the user’s Ruby installation, my application may have been using either the stable or unstable sort implementation, which was a variance it could not tolerate. (I’m glad I had automated integration tests that revealed this problem to me. I eventually concluded that the application’s behavior was fine with either the stable or unstable sort.)
3. Using JarJar and a few tricks means you can distribute your Ruby application as a single jar file.

Your application may need to run in a constrained environment where you can’t install much, but Java is available.

A couple of years ago I started development on a new application for monitoring PeopleSoft services. Installing Ruby on the servers I was deploying to was not an option. Java was already there to support PeopleSoft. Thank goodness JRuby 1.0 had just been released.

Managing one file is easier than managing many files.

Copying around one jar file is easier than copying a directory with hundreds of files in it. Yeah, manipulating a directory isn’t always a pain, but the times where manipulating a big directory is a pain can be a really big pain.

What’s the downside?

Your application gets bigger.

jruby-complete is currently (as of version 1.4.0) sitting at around 11.5 megabytes. Do I care? Nope. This can be annoying but it generally doesn’t bother me. Especially when my application is already measured in tens of megabytes.

Figuring out and typing the correct commands to run your application isn’t trivial.

This part sucks. Running commands through jruby-complete is tedious when you don’t have any help from something like Rake tasks. The JVM boot time sucks as well. For these reasons I don’t recommend using jruby-complete for everything: when running small, one-off scripts, there’s just too much overhead. Thankfully, when I do depend on jruby-complete, I’m already working on an application where I’m using Rake to automate all kinds of things. So running lengthly java commands isn’t a big deal because my Rake support is already there.

Thankfully, this companion post is chock full of tips for alleviating the pain from those nasty commands.

One of the great things about the JRuby project is that it’s easy to run Ruby programs without installing Ruby. In fact, you don’t even need to install JRuby. All you need is a JVM runtime and jruby-complete.

Rationale

Check out this other post for a discussion of my reasons for locking down your JRuby runtime. In summary, embedding jruby-complete gives you complete control of your Ruby runtime. That’s a good thing. The downside is that discovering and executing commands through jruby-complete can be a pain. The rest of this post describes how to ameliorate the pain.

Running jruby-complete

java -jar jruby-complete-1.4.0.jar

java -jar jruby-complete-1.4.0.jar -e "puts 'Hello'"

java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -e "puts 'Hello'"

java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -e 'load "META-INF/jruby.home/bin/jirb"'

1
2
3
4

irb(main):001:0> puts "Hello"
Hello
=> nil
irb(main):002:0> %

1
2
3
4
5
6
7
8
9
10
11
12
13
14

JRUBY_COMPLETE = "jruby-complete-1.4.0.jar"
JRUBY = "java -Xmx500m -Xss1024k -jar #{JRUBY_COMPLETE}"

namespace :jruby do
  desc "Run JRuby help"
  task :help do
    sh %+#{JRUBY} --help+
  end

  desc "Run any command with JRuby"
  task :run do
    sh %+#{JRUBY} -e '#{ENV["cmd"]}'+
  end
end

Now I can type rake jruby:run cmd='puts "Hello"'. Shell escaping is becoming a real annoyance at this point. Thankfully, I’m usually not using jruby-complete to run silly little commands. By the time I’ve introduced a Rakefile I’ve got a real application with tasks oriented around testing and running it, so it’s rare that I’m using a task like jruby:run very often.

1
2
3

task :run do
  sh "#{JRUBY} lib/application_bootstrap.rb"
end

1
2
3
4

fletcher-git/github/jruby-complete-example(master) rake run
(in /Users/fletcher/git/github/jruby-complete-example)
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar lib/application_bootstrap.rb
Hello from application_bootstrap

1
2
3
4
5
6
7

  namespace :spec do
    desc "Run RSpec against a specific file"
    task :run do
      raise "You need to specify a spec with spec=" if not ENV["spec"]
      sh %+#{JRUBY} -S spec -f specdoc #{ENV["spec"]}+
    end
  end

1
2
3
4
5

describe "John Galt" do
  it "does not tolerate logical fallacies" do
    "A".should == "A"
  end
end

1
2
3
4
5
6
7
8
9
10

fletcher-git/github/jruby-complete-example(master) rake spec:run spec=spec/unit/objectivism_spec.rb
(in /Users/fletcher/git/github/jruby-complete-example)
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -S spec -f specdoc spec/unit/objectivism_spec.rb

John Galt
- does not tolerate logical fallacies

Finished in 0.123 seconds

1 example, 0 failures

1
2
3
4
5
6
7

fletcher-git/github/jruby-complete-example(master) which sp
sp () {
        rake spec:run spec=$@
}
fletcher-git/github/jruby-complete-example(master) sp spec/unit/objectivism_spec.rb
(in /Users/fletcher/git/github/jruby-complete-example)
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -S spec -f specdoc spec/unit/objectivism_spec.rb

Alright, so now that my application has been built up, I might want to start compiling the .rb files into .class files. Here comes jrubyc:

1
2
3
4
5
6
7
8
9
10
11
12

require "rake/clean"

namespace :jruby do
  output_directory = "classes"
  directory output_directory
  CLEAN.include output_directory

  desc "Compile Ruby files in lib"
  task :compile => output_directory do
    sh %+#{JRUBY} -S jrubyc -p com/atomicobject -t #{output_directory} lib+
  end
end

1
2
3
4
5
6
7
8
9
10
11

fletcher-git/github/jruby-complete-example(master) rake jruby:compile
(in /Users/fletcher/git/github/jruby-complete-example)
mkdir -p classes
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -S jrubyc -p com/atomicobject -t classes lib
Compiling all in '/Users/fletcher/git/github/jruby-complete-example/lib'...
Compiling lib/application_bootstrap.rb to class com/atomicobject/lib/application_bootstrap

fletcher-git/github/jruby-complete-example(master) rake jruby:run cmd='require "classes/com/atomicobject/lib/application_bootstrap"'
(in /Users/fletcher/git/github/jruby-complete-example)
java -Xmx500m -Xss1024k -jar jruby-complete-1.4.0.jar -e 'require "classes/com/atomicobject/lib/application_bootstrap"'
Hello from application_bootstrap

1. enabling antialiasing in Apple’s JVM via a Java property.
2. tweaking Substance’s widget behavior via a Java property.
3. enabling Yourkit Java Profiler via the -agentlib parameter.
4. including libraries and directories in the JVM’s classpath via the -cp parameter.

Since these parameters need to be passed before the -jar parameter, a more sophisticated method for setting up the JRuby command is needed than the constant I’ve used. A method like that is specific for your application and beyond the scope of this post, but is not be difficult to create.

Conclusion

There are an uncountable number of good things about JRuby and jruby-complete is one of them. A little help from scripts and your shell means you can build and run your application with a controlled Ruby runtime.

Additional resources

• The Rakefile, jruby-complete, and other files used in this post are available in this GitHub project.
• JarJar Links is an ant library that is useful for combining multiple jar files together.
• I wrote a post a while ago about using JarJar to combine jruby-complete and other application dependences into a single file.
• The AGI Production Simulator is built using the jruby-complete commands described in this post as well as the above jar-rolling technique.

1. Replicating the true jruby behavior is way, way beyond the scope of this post. Check out the jruby script if you really care. Most of the time the JVM heap and stack sizes are the most important things to worry about. ↩

Edit 2/6/2010: Reduced -e ‘load…’ parameters to -S

Atomic Object has been using Yammer internally for about a month. Yammer is enterprise level micro-blogging. Essentially, it is a private Twitter service.

Yammer has proven to be a great way to share information within our organization. Unlike Twitter, the service is restricted to our organization so the team can freely discuss internal projects, company tactics, and technology information. If any of the commentary is interesting to the outside world it is then broadcasted through the company twitter feed.

Our favorite part about Yammer is that it has provided a channel for our off-site Atoms to connect with the rest of the office. It has helped the team stay connected that much better, and is quickly replacing company wide emails.

Eric Sink, of Source Gear was January’s SoftwareGR speaker. Twenty five or so people turned out for the meeting. Eric chose to speak on the topic that he’s passionate about: version control. This is a deeply nerdy interest and his wife is tired of the subject, evidently. Version control in a nutshell is undo & history of change.

I love our morning stand-up meetings: short, low-stress, and effective. Besides agenda and news, we often share technical topics with the group.

Here’s a quick list of some recent projects we’ve noticed or discussed:

• Rango – a new web application framework in Ruby, influenced by the success of the Python project Django. There’s a nice overview in PDF form.
• Atomic Object’s github account was started, hosting a few of our open-source projects.
• jesus is a web interface to God.
• jaml is an HTML template library for JavaScript, influenced by Haml.
• Lawnchair is a JavaScript client-side document store.

• bundler – easily bundle gems within an application. This tech has spread like wildfire through the office – I think it is something we’ve been wanting at Atomic for a long, long time now.
• git-deploy – deploy new code with a single “git push” command. I’ve forked this project and made some modifications for to suit our project.
• cancan – “Simple authorization for Rails”
• gritter – jQuery code that gives growl-like notifications. It works well and has been well received by most of our users. (That said, I personally cannot stand growl and think notifications like this should die in a fire.)
• culerity – celerity hooks for cucumber. We’re definitely not the first project to use it, but Drew and I have made extensive modifications to improve culerity’s process management.

One of the nice things about kicking off a new application is that it’s the best time to integrate new, exciting technologies like these. Discovering and integrating them has helped make our development process better than ever.