Message-Oriented Programming

Mail Slot

I just finished a project that used a Kinect’s computer vision tools to gather information about people nearby, then communicate with other hardware to adjust the environment. (Vagueness due to NDA.) The Kinect’s SDK limited me to C#, C++, or Visual Basic, but the other device only had a USB HID interface (a generic human input device). Unfortunately, C# doesn’t have any native support for USB HIDs, and none of the several libraries I tried seemed ready for prime time. After digging deeper and deeper into their implementations and attempting to fix things, I decided to try another approach.

There wasn’t any reason my entire system had to be in C#, though. As long as the system agreed on a protocol, I could just use IPC (sockets) and build the controller side with whatever language had the right libraries. Python has a great Windows USB HID library, so I wrapped that in about a dozen lines of socket code, and it was good to go.

Later, the hardware vendor sent me a DLL that provided a higher-level C interface than raw byte buffers, so I replaced them with calls into the DLL. (Also from Python, via ctypes.) The C# code didn’t need to know or care about this, since its outgoing messages stayed the same. Breaking my system into small, autonomous components made me focus on what they needed to communicate to each other, and once that was established, I was free to change everything else.

While this was fairly small project, the clean breaks would have been an even bigger win as the project grew. If the hardware controller had known too much about the computer vision code (or vice versa), a short-term hack to meet a deliverable could have infected the entire system with its details. A couple months later, it might have even started to look like legacy code. (Eek!) Moving data from one process to another, or across languages, seemed to add just enough friction between the pieces that the overall system naturally resisted excessive coupling. Also, the channels between the different parts of the system were excellent interfaces for testing — I didn’t need mocks, because I could already test each piece in isolation.

People often list this modularity as a benefit of object-oriented programming, but few OOP languages have counter-pressures against passing around unnecessary context. As Joe Armstrong put it in Coders at Work, “You wanted a banana, but what you got was a gorilla holding the banana and the entire jungle.” Other parts of the system can grow to expect this baggage, making it harder to swap out the module down the line.

More subtly, passing around objects at all couples the program to the object system — the rules for looking up the objects’ data and behavior is an implicit global context. If the modules communicate by passing around self-contained textual data, or a simple structured format like JSON, then they impose fewer assumptions on each other. Different parts can be written using different platforms or languages, allowing access to far more libraries. (They can also be on physically separate computers.)

Designing around the messages, rather than the objects, brought my whole system into focus. The self-imposed requirement that my system communicate via C# objects turned out to be a liability, and once I let them go it was much easier to meet my actual goals.

  • […] Hacker News This entry was posted in Uncategorized by admin. Bookmark the […]

  • Chris says:

    LOL. I was going to point out that Alan Kay said that the messages was the more important part of OO programming. Then I followed the messages link.

  • Bretto says:

    Nice article, but talking about system architecture, I would mention the Dependency Injection pattern, as for me it was the key ( the missing link ) to develop elegant and flexible large system.

    • Scott Vokes Scott Vokes says:

      That couples everything to the same object system, though. An alternative is to have a group of elegant and flexible small systems communicating.

  • What you describe is very similar to Objective C with its message passing paradigm.

    • Justin DeWind Justin DeWind says:

      It is true that Objective-C uses a message passing system (insomuch as messages are resolved at runtime). However, with the invention of Automatic Reference Counting the compiler has “evolved” to become far stricter. So much so, that it will produce a compiler error if it can’t resolve the message.

  • Jeff Kotula says:

    You’re right that OO designs that pass around objects tie both the caller and callee to the object system, but that is really the point of choosing a paradigm to write a program in. Shared conventions and techniques lead to faster more flexible development. In your case the key was to loosely-couple two processing units, and that was a good call. But that doesn’t indicate a weakness in OO.

    I think it is poor practice to pass around binary serialized objects as .Net seems to encourage one to do. Inter-process communication should almost always be loosely-coupled in the manner you describe. But again, that isn’t a problem with OO, but with its application to a specific problem. OO does support modularity within an executable or SDK, but that doesn’t make it the only technique that supports modularity.

    • Scott Vokes Scott Vokes says:

      The key is to think about how different parts should communicate, rather than passing everything around just because an object (or serialization) system makes it convenient.

  • […] Message-Oriented Programming Filed under: Clips Leave a comment Comments (0) Trackbacks (0) ( subscribe to comments on this post ) […]

  • Bob Bane says:

    I did something similar around 1996 when a group I worked in at NASA built a distributed satellite data capture and processing system. All the modules messaged each other with a simple socket-based protocol and S- expressions:

    (add (new-algorithm (algorithm “veggie index”) (arguments …) …))

    This was a research project with a planner/scheduler in Common Lisp, front-end data capture and processing in Java, back-end data processing in C, and GUI control and visualization in tcl/tk, so the S-expressions were a great common tongue. The C modules did their parsing with a thin wrapper around SIOD, so we had the whole interpreter available if needed.

    • Scott Vokes Scott Vokes says:

      Cool! I hadn’t heard of anybody using SIOD in production, nice to see it’s been to space. :)

  • Isaac Gouy says:

    fyi “Message Oriented Programming” Journal of Object Technology 3(5):7-12,May-June 2004.

  • […] Message-Oriented Programming I am seeing this a lot more recently as more programs are becoming event-based and focused on the flow of data and interactions. […]

  • […] – Interesting article. I recommend you read it if you like to see programming from different […]

  • […] Message-Oriented Programming: People often list this modularity as a benefit of object-oriented programming, but few OOP […]

  • […] Coincidentally, some of Rich’s keynote reminded me of Scott Vokes’ recent spin post, Message Oriented Programming. […]

  • Comments are closed.