Article summary
This is the third in a three-part series about strategies for synchronizing data between systems during a software replacement project. You can read about one-way integration and two-way integration if you want to catch up.
As a quick recap, I’m using the following shorthand as a way to talk about scope in this series:
Viable Releasable Increment (VRI) – The set of features that could be released together in order to satisfy the needs of users and the selected data integration strategy.
No Integration: Two Approaches
It’s nice when you can get away with no integration. It removes one more thing that the development team has to keep in mind. We’ve done this successfully when boundaries existed (e.g., sports seasons with defined beginnings and endings), restrictions prevented integration, or it worked for our customer. But even in some of those situations where we were successful, I think a solution that involved integration could have provided value more quickly and helped reduce risk.
I view “no integration” as a last option to consider if there is a hard blocker or other options clearly don’t make sense. In large projects, the risks simply get too high to take on a big-bang release.
1. Separate Systems, Separate Data
With this solution, you don’t import any data to the new system, and you don’t load any into the existing system. Simply switch over to the new system and start using it.
Benefits
- It’s cheap because there’s no synchronization tooling to build.
Risks
- It could create some additional work if data is eventually needed in a system where it doesn’t exist.
- The VRI size could be very large, and in my experience, the larger the release is, the larger the risk of something going wrong.
- If both systems are in use, users may create extra work if they create data in the wrong system and then need it in the other.
When to use it
- When there is truly no need to have the same data in both systems.
- When thereis a way to segment data in a way that allows it to work (e.g., Basecamp 1, 2, and 3 are separate products, and a project exists in only one of the systems).
2. Duplicate Maintenance
With this method, you allow editing in both systems and don’t attempt to synchronize the data. Users will make updates in both places and keep them in sync.
Benefits
- This is very cheap since there’s no integration code to write.
Risks
- Data could easily get out of sync if the duplicate maintenance isn’t done consistently.
- Users could be unhappy with the additional work.
When to use it
- When there is a very limited dataset and a limited set of highly-trusted users managing it.
- When the risks posed by out-of-sync data are low.
Mix & Match
Remember, you don’t have to apply the same strategy across an entire project or product. Use what makes sense to meet the team’s goals for a particular area of the application. For example, on a recent project, we used a combination of one-way batch, one-way event-driven, and two-way event-driven integrations to enable production use of the application while we continued work.
In summary, each of the three categories of approaches can be used effectively when replacing software systems:
- Loading data in one direction can be a good approach to feed read-only features of the new system, or when syncing data from the new to the existing system can keep the existing system up-to-date as a fallback. It’s typically less costly to build integration in only one direction.
- Despite its cost, two-way integration is the ultimate solution to enable use of an existing and new system in parallel. It allows for fast value delivery and risk reduction by keeping the systems in sync.
- It’s nice when you can get away with no integration, but I view it as a last resort when replacing complex systems.
The strategies I listed are some of the options I’ve considered over the years, but there are a lot of gray areas between those options that could also work. And I’m sure there are other creative solutions that aren’t on my list.
This is part one of a three-part series on Strategies for Data Synchronization for Rewrite Projects: