In 2016, Scientific Technologies Corporation (STC) was awarded funding from the Centers for Disease Control and Prevention (CDC) to redesign its Vaccine Order Management System. As part of this year-long project, STC partnered with JEBCommerce to plan, coordinate, and conduct multiple rounds of usability testing during the development cycle. Read the case study.
We recently interviewed STC’s product manager, Matt Halloran, about the experience of incorporating usability as part of the process and how it impacted the final product and the company’s thinking about usability.
Tine Reese, JEBCommerce: This was an exciting project to be a part of because our usability work spanned many months and included field visits, observational interviews, prototype creation, and user testing. Had STC or the CDC done usability testing before?
Matt Halloran, STC: No, we had never done usability testing. It was something we had talked about at STC, but we’d never had a precipitating event to spur it. When we received the funding from the CDC to redesign our inventory management application, they explicitly required usability testing as part of the project. At that time, the CDC hadn’t done usability testing either and didn’t truly understand the impact it would have on this project or the overall process.
TR: In the end, what did the CDC think of usability testing?
MH: The CDC was ecstatic. They had never gone to medical providers and observed them interacting with the Vaccine Order Management System (VOMS). Previously, they just talked to state administrators, doing the equivalent of requirements gathering for functionality. Everyone talked about what they thought were the best solutions, but that didn’t always produce products that fit users’ needs. You can only do that through usability testing.
TR: Does STC have a new level of appreciation for what user testing can bring to a project?
MH: Absolutely! Now, anytime we’re creating new workflows, user testing is part of our process. We create early prototypes and do rapid user testing to see where the problems are. It has to be a part of the process because otherwise we won’t succeed in making a good product—we’ll make something that works, but we won’t make something that people enjoy using.
TR: Thinking big picture: since this was a publicly-funded project, what’s the benefit to the public of having an improved Vaccine Inventory Management System?
MH: This application tracks vaccination encounters and the inventory used to administer those vaccinations. It is a volunteer participation program; there is no direct benefit to the providers (users) other than access to a dataset for the system.
So, how does improved usability benefit the greater community? If the application is difficult to use, the providers will stop participating. If they don’t enter data and track inventory in the system, our knowledge for who’s covered and not covered in a particular disease state goes down—our accuracy goes down. If you follow that down its path, a system that’s easier to use translates to more usage, translates to more informed healthcare providers and translates, eventually, to a healthier population. When I go to my doctor and my doctor knows my complete history, she’s able to make better decisions about my health. While we may have focused on inventory management, it has downstream effects on providers’ willingness to use the system.
The ultimate goal is to have improved population health outcomes. When we make our software easier to use we’re empowering providers to make better decisions and protect the population. That’s why the CDC is interested.
TR: What are STC’s future plans for including usability in projects? Has it changed your process?
MH: The work we did with JEBCommerce highlighted that we don’t have to write a single bit of code to essentially create the user experience and vet it out. We can have all of that done before we bring on a single developer. This exposed our team to the idea that we don’t even need to bring in a developer until the very end. At that point, the developer has a very clear line of sight on what they need to deliver because the structure, screens, and workflow have been tested and we know it’s going to work; now they just need to build it.
TR: How did including usability affect your project timeline?
MH: It didn’t make the total project timeline any longer—the time we gained from a shorter development phase allowed us to include user testing without extending the project. But I think the benefits of the process outweigh any time it might add because now I have a product I’m confident about. When I deploy it I’m not nervous that this is the first time our end-users have seen it. How are they going to use it? I already know. That bit of anxiety is gone.
TR: So, the champagne comes out earlier?
MH: Yes. Now when we deploy and release, I know that the product isn’t going to flop. There were times in the past where I’ve crossed my fingers and hoped for the best because we didn’t do our homework up front.
TR: Our collaboration with your team was critical to the success of this project. VOMS is a complex application that required us to understand how vaccines are ordered, transferred, distributed, and accounted for in states and medical offices throughout the U.S. We couldn’t have done that without working closely with you.
MH: You guys were great about being a sounding board for me. I’m so intimately involved in how the application works, it’s sometimes hard to see the changes that would improve the experience for users. It helps that you aren’t industry experts in our domain. When something wasn’t obvious to you, we knew it wouldn’t be obvious to new users, and you helped identify those gaps that I couldn’t see.
You would always ask, “Does it have to work that way? Why not approach it like this?” It takes a lot of maturity for a product team, developers, and domain experts to let go and say, “Just because I know how it works doesn’t mean it has to work that way.” Getting people away from that is so valuable.
The Result
STC has redesigned the Vaccine Order Management System, leveraging the insights gained from usability testing. Notable improvements to the software include:
- Consistent and intuitive interface
- Efficient workflows that match users’ expectations
- Reduced number user errors and improved error feedback
- Improved user confidence that tasks have been successfully completed
Take a look at the new and improved Vaccine Order Management System!
