Estimating software development efforts is hard. We’ve used a lot of different strategies, but the bottom line is that it’s hard to come up with a good estimate very often.
There are a few warning signs I have learned to look for that frequently indicate that my estimates might be off: Unfamiliar technology, vague requirements, or complicated or regulated business domains are common and typically very visible red flags that you’re probably already looking for. Today I want to focus on a warning sign that I’ve frequently seen ignored or underestimated: external services or APIs.
If you are building an application that is heavily dependent on another firm’s API, you should probably triple your estimate.
This might not be such a risk if we’re talking about a widely-used and well-tested public API like Google Maps or Paypal’s APIs. But if you are the first or sole consumer of an API or some data that is outside of your organization, you can very easily run into trouble. I’ve found it’s often easy to say, “Well, it’s just another SOAP service. I’ll just add Savon2 to my Gemfile and we’ll be done.” And there’s frequently pressure to estimate it like that (e.g. so no one has to take a high estimate to their boss.), but that’s a very risky thing to do.
h2. The Pitfalls
Here are a few places that I’ve frequently been tripped up:
h3. 1. Unreliable Connectivity
You might be saying to yourself, “Sometimes the service works well.” But so many things can go wrong even at the very basic level of just connecting:
* Maybe you’re not on the VPN, so you can’t actually try it.
* Maybe you can’t setup the VPN because their operations guy is on vacation.
* Or maybe your IP address changed through the project (because you wanted to change hosting providers or something) and then you can’t use their API for a week while you wait for them to update their configuration.
* Or there’s a problem because your SSL certificate is self-signed (even though it’s just the development environment).
h3. 2. Documentation & Stability
Popular applications have good documentation. Applications you control might not — but you can always read the source or look at a header file. But if it isn’t your API, you probably can’t even look at their code at all. If something changes in an incompatible way and they didn’t warn you, you’re stuck and can’t move forward on that part of your application until the their developer helps you fix it.
h3. 3. Testability
Testing external APIs is nontrivial: all of the nice tools (for example, staging your database and cleaning it between tests) won’t help you once you get outside of your application.
You really have two good options here: You can test against the real service, or you can build a fake one.
* If you use the real service, you need some sort of test environment there so you don’t pollute your data. You’ll also slow your tests down by an order of magnitude. And you’re going to need network connectivity to run your tests, which will frequently mean intermittent failures as things time out.
* Alternately, if you mock the service, your tests will be fast and reliable — these sort of tests are very useful for making sure your software implements a complex transaction as specified — but they won’t detect a breaking change in the API.
h3. 4. Single-Threaded Development
Most applications of a decent size can have at least some work done in parallel. I can work on screen X while you work on screen Y. If you plan ahead early, you can add people (“Brooks’ Law”:http://en.wikipedia.org/wiki/Brooks’s_law is only about adding people to an already-late project. Adding people early is frequently useful.) and get things done faster than just a single person.
That’s almost never possible when integrating a third party API. You may be able to get some leverage out of multiple people when *actually working*, but half of your calendar time can easily be be taken up in email exchanges or phone calls debugging a problem from both sides. It may not swallow that many actual hours of your direct attention, but even a relatively quick 2-3 hour turnaround on questions can eat up your timeline very quickly.
h2. Mitigating Outside API Risks
If you go into the project with your eyes open, aware of the risks, it’s possible to mitigate them. Here are a few ways to make sure:
h3. 1. Start Early
Since so much of your calendar is likely to be taken up in back-and-forth emails, you should get the ball rolling as quickly as possible. Even if you have nothing written, it’s trivial to throw together a test web service client with a tool like Savon. This can give you some assurance that you can connect, that you can make calls, and that the data coming back matches your documentation. If everything works, you can continue development with confidence; but if it doesn’t, you’ll still have time to address the problem.
h3. 2. Make a Fake Service
As we did with the Art Prize API and “have done on other projects”:https://spin.atomicobject.com/2013/10/01/benefits-fake-apis/, it is often very useful to have a fake service. You don’t need to implement everything, but if you have at least a moderately stable API, you can build a fake one that will let you keep working even when you’re offline or they’re down.
Having something like this is very useful — even if it returns the same hardcoded data every time you call it — because it means your development never has to be stuck waiting on something you can’t control.
h3. 3. Get in the Room
Finally, the best thing you can do is get in the same room as the people on the other side of the service. If you can manage it, it’s frequently possible to perform a week’s worth of debugging in an afternoon just by being in the same room and having complete logs from both sides of the transaction visible to everyone at the same time.
h2. Plan for It
There is no debating the value of using an API to integrate external services, but it presents many risks. If you plan your project and schedule with these dangers in mind, you can manage the risk and make your project more successful.