We're hiring!

We're actively seeking designers and developers for all three of our locations.

Development

Keeping a Development Journal

journalingLife gets crazy, and things slip our minds. A great discovery, a bug, or a general direction we were taking our code can get lost between development sessions.

In my personal projects, where time between development sessions can be days or weeks, I have been keeping a development journal. As much as it takes an additional couple minutes, it allows me to express intent and task my future self with necessary work.

Read more on Keeping a Development Journal…

Also posted in Development Practices | Leave a comment

Interacting with Git: Cryptic RPC Errors, HTTPS, and SSH

PC_Load_Letter_590

In my experience, working with a Git repository hosted by Gitlab, Github, or Gitorious has generally been issue free and enjoyable. Recently, however, three members of my team ran across the same cryptic RPC error when trying to push changes to a remote repository on Gitlab:

error: RPC failed; result=22, HTTP code = 411

In two cases, developers were trying to push new framework libraries (they were moderately sized, 2-9 MB). In the third case, a designer was trying to push a large batch of image assets. In all cases, the problem was caused by using the HTTPS protocol with a server configuration that disallowed individual files larger than 1 MB.

After some basic investigation (thanks Stack Overflow!), we found that using the SSH protocol with Git solved the problem. This type of issue could trip up a new user of Git, so I am going to use this post to briefly describe the problem and summarize the pros/cons of using HTTPS vs. SSH protocols to talk to remote repositories.

Read more on Interacting with Git: Cryptic RPC Errors, HTTPS, and SSH…

Also posted in Developer Tools | Tagged , , , | Leave a comment

Turning Asynchronous Networking Inside-Out

Developing asynchronous networking applications is an interesting problem with unique challenges and no shortage of solutions for, particularly in Python. But the way you’ve traditionally had to write your code to make it work well could leave you with a codebase that, while solid, could be challenging to read or follow.

I’ll take you through a quick introduction to what asynchronous networking is, show you a callback-based asynchronous implementation using the venerable and excellent Twisted, and then finally show you how Python’s new asyncio library uses coroutines to solve the problem in a whole new way.

Read more on Turning Asynchronous Networking Inside-Out…

Also posted in Development Techniques | Tagged , , | Leave a comment

Sharpen Your Git Saw – Aliases, Selective Staging, & Interactive Rebasing

Git is a powerful tool that we love as developers. It’s also complicated. I consider the bare essentials of Git, the minimum set of features to be familiar with before we can be productive, to be all of this:

  • local interaction: status, add, remove, commit, reset, checkout
  • branch management: checkout, merge
  • remote interaction: clone, fetch, push, pull

Once we’ve learned these basics well, we move onto advanced features and tricks. Read more on Sharpen Your Git Saw – Aliases, Selective Staging, & Interactive Rebasing…

Also posted in Developer Tools | Tagged , , | Leave a comment

Try All the Easy Fixes First

easy-fixes

Even after 10 years, I find I occasionally need a reminder to check all the obvious things when something goes wrong.

I just spent about a day debugging an issue with tests hanging, but only on one machine under continuous integration. The other machines were all fine. I assumed one of my tests was just broken: maybe I was polling for an update in the UI forever or something like that.

I definitely took the long way around trying to debug things. I analyzed which tests were running and how far each was getting by adding print statements that I could view in the CI output. I searched out all the tests that triggered events, just in case my mock event handler was doing Something Strange. Read more on Try All the Easy Fixes First…

Also posted in Development Techniques | Leave a comment

Property-Based Testing – Testing Assumptions You Don’t Know You’re Making

property-based-testing

Finding good test input can be tricky. Even with loads of unit tests, bugs still get through.

Consider a wear-leveling algorithm for flash memory — it takes a series of write operations and spreads them over the flash, because individual blocks can only be written and erased so many times before they wear out. If any write sequence leads to writes concentrating in specific blocks, something isn’t working.

Read more on Property-Based Testing – Testing Assumptions You Don’t Know You’re Making…

Also posted in Development Practices, Uncategorized | Tagged , | Leave a comment

Visualizing Garbage Collection Algorithms

Most developers take automatic garbage collection for granted. It’s just another amazing feature provided by our language run-times to make our jobs easier.

But if you try to peek inside a modern garbage collector, it’s very difficult to see how they actually work. There are thousands of implementation details that will confuse you unless you already have a good understanding of what it’s trying to do and how they can go fantastically wrong.

I’ve built a toy with five different garbage collection algorithms. Small animations were created from the run-time behavior. You can find larger animations and the code to create them at github.com/kenfox/gc-viz. It surprised me how much a simple animation reveals about these important algorithms.

Cleanup At The End: aka No GC

garbage-collection-NO_GCThe simplest possible way of cleaning up garbage is to just wait until a task is done and dispose of everything at once. This is a surprisingly useful technique, especially if you have a way of breaking up a task into pieces. The Apache web server, for example, creates a small pool of memory per request and throws the entire pool away when the request completes.

The small animation to the right represents a running program. The entire image represents the program’s memory. Memory starts out colored black, which means it isn’t used. Areas that flash bright green or yellow are memory reads or writes. The color decays over time so you can see how memory was used, but also see current activity. If you watch carefully, you can see patterns emerge where the program begins to ignore some memory. Those areas have become garbage — they are not used and not reachable by the program. Everything else that isn’t garbage is “live”. Read more on Visualizing Garbage Collection Algorithms…

Also posted in Development Techniques | Tagged , , , | 18 Comments

Plays Well with Others – Lessons in Reusable Tooling

I’ve used many languages for my scripting needs, but my favorite — believe it or not — is probably Bash. Bash may not have any types, and scripting with Bash may be fraught with pitfalls, but sometimes the problem at hand is solved most succinctly and elegantly with small focused programs that compose well.

Bash’s composition comes in the form of piping one program’s output into the next program. No program needs to know where its input comes from, just like Lego blocks don’t care what block they stack on. This is an incredibly useful approach. For instance, take a look at how easy it is process the contents of the clipboard:

pbpaste | base64 | pbcopy

We just base64-encoded whatever was in the clipboard (at least, on Mac OS X; Linux has similar programs under different names, and on Windows you may need to write your own versions of pbpaste and pbcopy). You can use pbpaste and pbcopy and the rest of the Bash toolbox to do whatever processing of clipboard text you have in mind, and thanks to pipes, you can do it very concisely. Read more on Plays Well with Others – Lessons in Reusable Tooling…

Also posted in Development Techniques, Growing as Makers | Tagged | Leave a comment

Git-SVN Gotcha with Empty Directories

This short post is intended to serve as a warning about a potential gotcha with git-svn, and how to prevent it.

An Anecdote

First, a sort of “postmortem” of my run-in with this issue:

I was working to migrate an old SVN repository full of documents to Git. We had decided that we didn’t need to maintain a complete history going forward, that we would just take what was currently there and put it in a new Git repository. We would keep the old SVN repository around for reference in case we ever did need to go back through that older history. We wanted to preserve the old history in SVN, but make a clean break from it for a fresh start with a new Git repo.

I used SVN to check out a fresh copy of the repo, removed .svn, turned the directory into a Git repo, and pushed it out to the new remote. All good there. Read more on Git-SVN Gotcha with Empty Directories…

Also posted in Developer Tools | Tagged , | Leave a comment

Floating Point Numbers & Currency Rounding Errors

A reminder to those who learned this lesson long ago, and a warning to those that have not learned it for themselves: Don’t use floating point numbers for money.

When doing any kind of calculation with currency, accuracy is extremely important. And floating point numbers (floats and doubles) don’t have an accurate enough representation to prevent rounding errors from accumulating when doing arithmetic with monetary values.

An excellent StackOverflow response that explains the issue states:

Representing money as a double or float will probably look good at first as the software rounds off the tiny errors, but as you perform more additions, subtractions, multiplications and divisions on inexact numbers, you’ll lose more and more precision as the errors add up. This makes floats and doubles inadequate for dealing with money, where perfect accuracy for multiples of base 10 powers is required.

Ways it Can Go Wrong

When it comes to dealing with money in a computer program, a developer needs to stay ever-vigilant. Just because you, or a previous developer, made the right call by using a DECIMAL column when creating the database schema, doesn’t mean that accuracy can’t be lost before the result of some calculation is displayed to an end user. It’s all too easy to slip up and lose the necessary accuracy without even realizing it. Read more on Floating Point Numbers & Currency Rounding Errors…

Also posted in Development Techniques | Tagged , | Leave a comment