Costco Travel Search Widget
Costco Travel's search widget is the primary entry point into the website and a key driver for the company's revenue.
But searches for Costco Travel's new products were lower than expected, and so the UX team came to the rescue.
Context
In many ways, Costco Travel's search widget is the "face" of the company that lets our members know what kinds of products and deals they can take advantage of to get a great value on their vacation.
The search widget wasn't meeting business expectations. Costco Travel had recently added new product lines - "Hotel Only" and "Dynamic Packages" (Hotel + Flight; Hotel + Flight + Car; Hotel + Car) - to complement the curated packages that were the "bread and butter" for the company. Searches for the new products were lower than expected.
My role
As Manager of UX, I led a highly-scrutinized effort to redesign the search widget for Costco Travel's website.
- I defined the roadmap with product management and engineering
- I staffed the effort with the right amount of UX support
- I guided the UX team with prioritization and timely feedback
- Overall, I ensured the search widget evolved to meet our strategic needs.
Objectives
- Increase the number of searches performed from Costco Travel's search widget in general.
- Increase awareness of Costco Travel's new product offerings.
- Avoid any disruptions to "Vacation Package" revenue.
Results
As a result of the search widget design improvements, searches increased by over 8%.
The increase in search behavior has contributed to an increase in bookings of over 5% compared to pre-pandemic levels.
Laying the right foundation with research
The UX team's cross-functional partners in product and engineering wanted to move quickly. They wanted to boost sales of the new products and didn't want to take time for user research. But I knew that this was the first time that the search widget had been scrutinized by a proper UX team. If the team was going to design an effective solution, we needed to better understand the problem we were trying to solve. I ensured that we followed the right product development process, right from the get-go, by bringing in our user researcher to conduct usability studies and competitive research to get us started on the right path.
Our research phase proved to be invaluable. In head-to-head matchups, users only preferred our search widget between 10-20% of the time vs competitors like Expedia, Booking.com, Google Travel, and others. Here are some of the reasons:
Defining problems
Stakeholders were all over the board in terms of what they expected to come out of this project. Besides refreshing the search widget itself, some expected a full home page redesign, others expected a new search results page. They similarly were not aligned on how we would measure success.
I drove a series of workshops to get our stakeholders aligned on what we were trying to accomplish. We defined and agreed on what the business problems were using business metrics, as well as the user problems with the foundational research that we had conducted.
We also agreed that we would use percentage of home page sessions that resulted in each vertical search as our primary success metric, and we set up our analytics to collect this information.
Output of problem definition workshops
Ideation, accountability, and design reviews
As we moved into the ideation phase, I needed to make sure that our designers and researcher were collaborating effectively to drive the right solutions. I also needed them to be accountable for their work without micro-managing the project.
I set up weekly project syncs, design reviews and prep time with the designers and researcher where they would formally present their ideas to each other and discuss how well their designs were solving the problems we had uncovered. We were very tightly aligned on the direction of the project and what we would be presenting to stakeholders each week.
Spin off Vacation Packages from widget
Single field search widget
Verticals on left sidebar
Introducing AB testing
Since the culture at Costco Travel up to this point was to simply pick the design that stakeholders liked best, it was important to establish more rigorous ways to approach decision making, so I introduced the idea of A/B testing.
We decided to center our first A/B test around improving the styles on the search widget. We made the background white and softened the corners on the input fields. Heuristically, we knew that these changes should lead to increased engagement. It was a great opportunity to show how changes in a design can yield impressive results.
I led stakeholders through the process of setting up an A/B test correctly. We chose our dependent and independent variables, established our hypothesis, and leveraged Google Optimize to run the test. I created an A/B testing worksheet to guide our discussions, not just for the initial tests, but for all A/B tests going forward. This worksheet and process has now been used dozens of times to make decisions about the direction of the website.
The first A/B test was a tremendous success, with searches improving by nearly 6% across the board. Stakeholders were thrilled.
Mockup screen with style changes for the first Costco Travel A/B test
A/B test worksheet
A/B test results in Google Optimize
Incremental improvements
Costco Travel stakeholders were reluctant to give up old ways of making design decisions, so we had to move slowly and prove that every change we were making was valuable.
We went through several rounds of identifying a very specific problem from our foundational research, designing a solution for that problem, running qualitative studies to make sure that it was the right solution, then doing an A/B test to show that it was the right solution at scale. Even though this process took far longer than it might have at an organization with more UX maturity, it was the right thing to do in our situation so that we could win over stakeholders and have them trust us more implicitly further down the line.
After we made our first successful improvement, we continued to iterate on the search widget, introducing changes to the ordering of the vertical tabs, reducing inputs to fit onto a single line, and making changes to the field and button labels.
This was definitely a case of "slow but steady wins the race", and we earned the implicit trust of stakeholders in our methods many times over.
Added pill boxes for new products
Put new products in first position
Single line search
The impact of the Costco Travel Search Widget redesign
After making many of our planned changes to the search widget, we increased the number of searches that Costco Travel's search widget yields by over 8%. This improvement has had downstream effects on bookings, which are currently beating pre-pandemic levels by over 5%. Costco Travel is posting record-setting sales and revenue, due at least in part to the changes my team designed and tested.
There is still more work to do on the search widget. My team is continuously finding opportunities to remove friction and get users into the search and booking workflow.
Lessons learned
Level up stakeholders in UX.
Stakeholders may have varying levels of UX maturity, and it's important to use the many research tools that we have at our disposal to win them over. I took the time that each stakeholder needed to bring them along for the ride, which has paid off in big ways ever since.
Mixed methods matter.
In this project, some of our stakeholders were more convinced by showing user highlight reels, while others were more persuaded by the A/B test results. Being able to customize my messaging using both qualitative and quantitative research helped me to gain the buy-in I needed to move the project forward.
Take time for research. Always.
Each user study that we did on each iteration of the search widget led to more insights on how we could improve the product. This process led us in directions that we could not have anticipated at the beginning of the process, and helped us to build something truly useful and engaging for our users.