As of April 2024, about 5 million active Amazon users asked Rufus ~5.25 million times, seeking assistance with their shopping missions, and around 25% of the traffic comprise of broad queries that lack specific details, making it challenging for Rufus to provide helpful answers. This suggested that Rufus's approach at the time of just providing top category recommendations based on world knowledge might not be effective, as indicated by low click-through rates, customer dissatisfaction, and the need for more conversational understanding to clarify customer requirements.
With this in mind, I designed a 3-6 month UX vision for a new modular component & pattern for Clarifying Questions that can be adopted into horizontal experiences from broad to narrow queries, and potentially even be folded into search result pages on the base layer of the Amazon app.
I worked with my product manager to focus the scope of our experimentation on gifting. The reason was that we had high conviction that clarifying questions can add value in helping customers narrow down their search scope for gifts. Also, we had confidence that this use case can be accurately identified through existing signals and avoid false positives.
Here are a few explorations that I designed with these priorities in mind:
Make it shoppable.
Make it actionable.
Make cognitive load low.
I explored a new tappable suggestion pills to lower interaction costs for customers answering clarifying questions.
Initially, there was feedback from leadership that it seems too heavy-handed to ask customers to interact with tappable suggestions, but I proactively created a plan to test the new interaction pattern in two separate UXRs, and both times participants responded exceptionally well to the pills and stated that the suggestions “act as constraints that expedite their shopping experience by producing a concise and helpful [set of product recommendations] associated with their specific needs.”
Selected explorations that showcase a few ideas:
Multi-question quizzes
Multi-select options
Animation to help with cognitive load
Here's what launched! I actually used my feature to figure out what to buy my nephew for his birthday. 🥳
Dialed MVP experience to 20k users on June 7th, dialing up to 100% Rufus beta users on June 14th. The experiment showed that clarifying questions increased overall engagement with Rufus.
There was a 5.3% higher next-turn rate. (7.6% in the treatment group vs. 2.3% in the control group). Higher click through rate than the control group.
Team was encouraged by positive signals to continue improving clarifying questions with tappable suggestions.