Rewriting your organization’s current software is a huge undertaking. It can be intimidating and cause uncertainty, especially for those who are more averse to change.
A pilot is a helpful exercise during any software development process, but especially so during a rewrite where the scope is very large. A pilot allows you to test the software within a group or location without fully rolling it out to all users.
A pilot can help you:
- Mitigate risk
- Test features
- Identify areas of improvement
- Improve buy-in within the organization
If you decide to run a pilot, here are some tips and preparation guidelines to help the pilot run smoothly.
1. Set Clear Expectations and Goals
This is one of the most important items to discuss before a pilot. Everyone should be in alignment on which features are being tested. If the expectation is that features X, Y, and Z are being tested, but only features X and Y are ready, the pilot could be deemed a failure just because of a miscommunication.
It’s also important to discuss how you will measure success and evaluate the results. I try to lead with the expectation that we will find bugs and there will be things that do not work as expected. The important part is outlining a plan for how to deal with these issues as they come up. These could include:
- A troubleshooting guide
- Contact info for someone who can help work through an issue
- Contingency plans, such as when to fall back to the legacy system
Lastly, determine the duration of the pilot, and identify the people who will be participating in it.
2. Make a Timeline for Each Week Leading Up to the Pilot
Your timeline should cover activities and tasks for at least 30 days leading up to the pilot. These items could include:
- Reviewing goals/expectations and features
- Building training material
- Setting up users
- Adding data to the system
- Testing connectivity and other upfront setup
After making your initial plan, add a few days of padding to the beginning and end of it. You will always need more time than you think you do. Also, I’ve found it helpful to set up a daily standup meeting with pilot participants so you can have a planned touchpoint with them during the day.
If you are traveling to help with a pilot, it’s a good idea to send a team member out a couple of days in advance to do some setup…which leads me to Step #3.
3. Do Upfront Setup before the Pilot Begins
The day of the pilot will be stressful. Anything you can do in advance to make sure the system is ready will make people feel more at ease. Adding information such as users, customers, or other pertinent data allows you to resolve issues such as “I can’t log in” earlier, so you don’t have to start the day troubleshooting tedious admin issues.
4. Provide and Review Training Material
Opinions differ on training material and format. In theory, if a system is designed well, using it should be self-explanatory. In practice, especially for a new system, you always need a place where people can go to reference instructions, tips, and troubleshoot. Make the documentation specific to user roles, and encourage pilot participants to read the material prior to the start so they will be acclimated to the system.
5. Don’t Forget the “Small” Things
It’s essential to designate a note taker throughout the pilot and to set up a central place to collect the notes. I’ve been in a lot of situations where someone thinks someone else is taking notes, and you end up with people trying to remember things after the fact or piecing together information they half-heard. There will be a lot to manage and facilitate that day, so having a designated note taker will give you a better record as you assess the pilot.
If you’re using any sort of hardware, locate the nearest Best Buy or electronics store in case something needs replacing. This might sound trivial, but if there isn’t a store within 30 miles, you might need to make other backup plans or purchase backups in advance.
Hopefully, these items will help you plan your pilot–an event that can be very stressful, but also exciting! Now it’s time to run your pilot and evaluate the results.