A Guide to Evaluating Closed- and Open-Source Libraries

Picking the right libraries for your project is important. The wrong ones can create long-term headaches and introduce technical debt that will eventually be very expensive to correct. When making your choice, there are a few important things to think about, and a few common things to avoid.

Considerations for Closed-Source Libraries

Libraries are either open-source or proprietary, and there are different considerations for both. Proprietary libraries usually have the backing of an entity with a monetary incentive to maintain it, but because the source code is closed, the community can’t help out with development. This means that development might be slow or that it won’t change to adopt popular features–situations that can weaken the cash flow potential. An example might be a library refusing to integrate with some other system in a market that the backing entity might have an interest in entering.

Closed-source libraries are rare in some ecosystems and common in others. When evaluating them, it’s important to consider the reputation of the product, the reputation of the company providing it, and the price, if applicable. If there’s a support forum available, check how busy it is and how responsive the company is to support requests. Try to gauge whether or not people seem generally happy or generally frustrated with their experience. If the forum is overwhelmed with unresolved questions, it’s probably a bad sign.

Read everything you can find about people’s experiences using the library. If it’s considered the standard library for some functional area in your ecosystem, it’s probably worth using as long as it seems like it’s easy to use and well maintained.

Considerations for Open-Source Libraries

Open-source libraries have no formal backing, but they can be maintained by a large community with an interest in making the best technical decisions rather than the best business decisions. At the same time, a library can quickly fall out of favor and wind up being completely abandoned.

When evaluating open-source libraries, I rely on the following tips:

1. Always consider the license.

With open-source libraries, the first thing to look at is the license. Adding a library to a project always has licensing implications. You want a permissive license that will allow you to modify it and continue to use it if it’s abandoned someday, and you want to be able to use it in a commercial, closed-source product if that’s the goal.

There’s been a ton written on the differences between the different licenses and what you need for which types of projects, so make sure you review some of that material before deciding. If you’re dealing with a non-permissive license, make sure the stakeholders understand the implications of that.

2. Use the standard.

If there’s a library so popular that it’s considered the standard by most people, you should probably just use it. Picking a popular library makes it easier to find answers to questions you might have or help for issues you might encounter. It may also be easier for future team members to onboard to a project with a library they already know. They won’t need to ask, “Why didn’t you use X?” when they see your DIY implementation.

A popular library will probably also have long-term support from the community even if the author decides to stop maintaining it, or it might be forked and maintained by a subset of the community if an ideological split occurs in the community. These splits can be painful and tumultuous, but they are often resolved. The library is often in a better place after the resolution.

3. Look at documentation.

It’s good if the library is well-documented. If there’s documentation aimed at users and potential contributors, it’s even better. This is a sign that the authors are open to receiving outside help, and they are conscientious enough to try and smooth the path.

Good documentation should include an overview of the functionality, examples of usage, a complete API reference, and information about architecture and how to run the specs. Look through the specs because they will help you understand how the code is structured and help you determine whether or not the code will be easy to modify later if needed.

Take a look at how the code is tested. Are the specs currently passing on the main branch? Do you see any major gaps in test coverage? Lapses here can indicate that the developer doesn’t value specs, or if they do, they might just not have the time or energy to devote to keeping them passing. Either way, it’s a bad sign.

4. Look through the issues and pull requests.

If the project is on GitHub, Bitbucket or another repository hosting site, look through the open and closed issues and pull requests. Notice how long they’ve been opened and how responsive the author is to them. If there are a lot of open issues or pull requests without any comments by the author, it’s a big red flag. It might mean the author has lost interest in maintaining the library, or other things have gotten in the way.

Also, try to read the tone of the authors or maintainers. If they seem grateful for the help and are willing to answer questions for the people hacking away on it, it indicates a healthy dynamic in that particular community. If they’re combative, dismissive, or get defensive when questioned, that indicates an unhealthy dynamic that you might run into if you ever need to dig into the internals of the library.

5. Don’t look at the number of stars.

Some people joke around about picking libraries based on the number of stars they have on GitHub. The number of stars is a good gauge for overall popularity, but popularity alone shouldn’t determine whether or not you pick a library. There are repos with tons of stars that are an absolute nightmare internally. It’s worth taking note of their popularity, but it shouldn’t be a determining factor in your decision.

Take a look at the number of forks, however. A high number of forks relative to other indicators of popularity might mean that the maintainers are so unresponsive that a lot of people who use the library end up forking it to address common issues.

It’s not uncommon to see an open pull request that addresses a bug ignored by a maintainer, but merged in on dozens of other forks. If you have to use a library in this state, you should also make your own fork instead of using someone else’s. If the maintainer ever gets around to merging that bug fix, you can always point your code back at the main repo instead of your fork.

What to Do When There Are No Good Options

Sometimes you evaluate a bunch of libraries, and they are all in bad shape for some reason or another. Your options then are to roll your own or pick one of them anyway with the intention of fixing them up for your needs.

This is a much bigger commitment than simply using a library, so take a deeper look into the code and try to gauge its quality. Is it understandable and written in a way that makes sense to you? Is it a domain where you have some experience, or is it completely outside of your normal expertise? Are there any parts of the stack where you have no experience? The more familiar you are with the pieces that are already there, the easier it will be to service or extend it.

If you end up fixing bugs or adding features to a library like this, open pull requests against the main repo. You will be helping out the maintainers who are probably overwhelmed with work, and you’ll be helping out countless others who will end up using the code you’ve written.

Do your due diligence before picking out libraries, and you will save yourself a lot of headaches. Avoid things that are poorly maintained unless you want to end up maintaining them, because you just might.