A frequently asked question in product management interviews is the improve a product question. For example, the question may go like this: “Pick any product you like and tell me how you would improve it?” If there is a very popular product on the market, the interviewer may assume you know about it and ask: “How would you improve product X?”
The example question we will work on is: How would you improve Google Allo?
The key to answering this question is to think about which business metrics need improvement, what the users needs are, and what product changes you would implement to meet those needs in order to improve the business metrics. Below is a suggestion on how to structure your answer.
State how you will proceed
Explain to the interviewer how you will tackle this question. This tells your interviewer that you are thinking in a structured way. For example, “First I will describe what I think the product does and what can be improved. Then, I will describe who the users are, what their needs are, and how the product can be modified to meet the most important needs in order to improve its success in the market.”
Summarize what the product does and which problems it solves.
Business metrics to improve
Explain which metrics you think the product has problems with and why. For example:
- Customer base — is it too low? If so, then why? Or if they are doing well, perhaps you can make this metric even better.
- Revenue — does the business need to increase the number of paid users, or the amount paid users spend?
- Retention — are customer churning and not returning?
- Conversion — does the business need to convert more visitors into paid users?
- User engagement — are customers using the product?
Use cases and solutions
- Justify which metric or metrics you think are the most important to improve.
- Describe the types of users and use cases that the product is failing to support or could support in order to improve the business metric.
- Prioritize users and use cases based on criteria, such as revenue, simplicity, cost, and frequency of use.
- Brainstorm solutions for the prioritized use cases.
Evaluation of solutions
Evaluate tradeoffs between solutions using criteria, such as cost, complexity, competition, and revenue.
Which metrics would you use to validate your solutions? If for example your solution is to improve retention, than some example of metrics are:
- How many days after sign up do users come back?
- How many times a day, a week, a month do they use the product?
- How does it compare to past usage?
- Explain which solution you would recommend.
- Recap how the solution solves the problem and why it is the best.
State How You Would Proceed
INTERVIEWEE: I would like to start by talking about the issues Allo currently has, then move on to users and use cases, and end with ideas on how to improve Allo in order to meet those needs.
INTERVIEWER: Sounds good, go ahead.
INTERVIEWEE: Allo is a messaging app that uses AI technology to help provide answers to simple questions like what’s the weather today or what is my next appointment? It does this through a chatbot that the user can address by typing @google in the app.
The main issue with Allo is its very small customer base. Even though it had about 5 millions downloads during the launch, people are not using Allo. In my opinion the main reasons for that are:
- Allo lacks the basic features of a messaging app, like attaching a document and starting a phone call from the app. As soon as people see that they cannot do these functions they are turned off.
- The lack of a voice interface for users to get assistance while driving or multitasking is a problem. One can ask a question through voice, but the answer is always text. So for someone on the go, the expectation is that if you asked through voice, the answer should also be through voice. The app should be smart enough to understand that you are not typing for a reason. This leads to frustration.
- Users are confused about the difference between Allo and the three other Google communication apps, namely, Hangout, Duo, and Messenger. Due to this confusion, they end up not using any.
- Users are frustrated by the frequency the Google assistant chatbot is unable to answer or provide an accurate answer. The Google assistant chatbot basically spits out search results as opposed to providing a single answer when it is required. For example, if you ask the question, what is the best Thai restaurant around here? Allo lists all the Thai restaurants nearby with their rankings, but it doesn’t pick one for you. Or if you ask the question, what is my next appointment? Allo only looks at your default calendar in the Google Calendar app, thus if you have events created in multiple calendars it misses them. Furthermore, nothing will happen unless you have the Google Calendar app installed. Allo does not inform the user of that prerequisite.
To compete as a messaging app, I think these four issues need to be improved first. Adding the basic features I mentioned, should not be a problem. Facebook Messenger and Whatsapp already enable attaching a document and starting a call. I think Google is capable of adding a voice interface too. Google Home already has this feature, so it’s a matter of integrating the technology into Allo. Amazon and Apple have Alexa and Siri voice assistants, and as people start using home assistants more and more, they will expect a voice interface on messaging apps too.
With respect to the confusion in the market about Google’s four communication apps, I would consolidate the features of all these apps into one single messaging app. The trend in messaging apps is to have all types of communication mechanisms available from the app, like text, video and voice, because simplicity matters to people. We are seeing this trend with Facebook Messenger and WhatsApp. Furthermore, in China, where Tencent’s Wechat is the leader in messaging apps, people are using WeChat as their portal for everything, not only to communicate but for eCommerce and for accessing services.
The issue with the Google chatbot inaccuracies or inability to respond has to do with the AI technology underneath it. The technology is still in its infancy and needs more development to improve its accuracy. For now, the Google assistant chatbot is more of a search interface than an assistant. Its assistant role is in providing answers to questions related to other Google apps like the Google Calendar and Gmail, but even that is difficult. For example, if you asked Allo: who sent me email just now? The response is a list of emails rather than the last email. It is a matter of time for the technology to improve.
Use Cases and Solutions
I believe that if people were able to do other frequent tasks through a messaging app such as querying for specific information from a business or service, making commercial transactions, ordering or reserving a service, ask questions to a doctor, etc, the app will become an indispensable tool to users and businesses. For example, users may have questions like, when is my car insurance due? How much is my mortgage? Or the user might need help with tasks such as buying a gift or flowers for a friend’s birthday, or calling Lyft for a ride, or remembering the name of your favorite Chardonnay?
The information to answer these questions will most likely reside in databases of the many businesses and services people interact with, and opening Allo to third party chatbots would enable these use cases to be supported.
In the last month, Google announced the Google Actions SDK, which will enable third party developers to build chatbots for Allo. This is the right move, as it will increase the usefulness of Allo as an assistant to users.
Now, I would like to go over users and possible use case scenarios for chatbots, that if supported could improve usage for Allo and ultimately increase its customer base.
I see four different types of users of these chatbots:
Millennials have very distinct and different needs than the rest of the population for messaging. As the success of Snapchat revealed, millennials want a self-disappearing feature because it keeps their messages private and inaccessible to authority figures. They also like to communicate using rich media rather than text, because it’s fun and more expressive. So providing animated gifs, emojis, videos, drawing and editing tools is essential. Snapchat dominates that market, and it would be extremely hard to compete for this segment to start with. So for the moment, I don’t think targeting this segment first would be a good way to compete in the messaging app market.
In the adults segment Facebook Messenger, Whatsapp and Apple iMessenger compete in this market. People in this market are more willing to switch, therefore, it is easier to enter. There are use case scenarios where, I see chatbots providing greater utility to this segment:
- Answering questions that would otherwise require doing multiple searches and reading. For example, if a user asked: What is the best Thai restaurant around me? The chatbot should provide one single answer, not a list of restaurants, and a summary of the restaurant rating. It should also help the user with making a reservation.
- Assisting the user with tasks, such as playing music, reading a book aloud, reading the news aloud, and providing directions from point A to point B, not just opening Google Maps when asking the user for directions.
- Assisting with services, such as helping the user hail a ride with Uber or Lyft, order take out, make an appointment with your hairdresser, reserve a room at a hotel, or rent a car.
- Assisting with buying products, such as buying groceries from your preferred grocery store, ordering movie or theater tickets, or buying an airline ticket.
- Assisting with health services, such as answering health questions, diagnosing health issues, making an appointment with a doctor, or asking a question to a health insurance provider.
Based on these use cases, I think businesses and service providers would be very interested in developing chatbots to sell their products and services to users, market to users, provide customer service, and strengthen customer relationships.
As we can see, there are multiple use cases for third party chatbots to be developed. And, I can foresee hundreds of chatbots competing for customers. The question is, how will users be able to find which chatbot to use? In the beginning, I think a category list as well as a search feature should be included within Allo, so users can find a chatbot that will help them with whatever they want to do. In addition, just like with apps today, a chatbot category should be created within the Google Play Store so people have another way to find and install chatbots within Allo.
I mentioned various solutions: add all the basic features (attaching documents and making phone calls) to Allo so it is at parity with other messaging apps; add a voice interface; consolidate all the communication apps: Hangout, Duo, Messenger, and Allo into one; and open the platform to third party chatbots, which Allo has already started to do.
I think all of these features are important and I would put them in the product’s roadmap, but I would prioritize their timeline based on how critical they are to the user. Document attachment and the ability to make phone calls from a messaging app are basic features that any messaging app needs to have, so I would implement these first. As a second milestone, I would include a voice interface, since a voice interface is a must for users on the go or multitasking. And as a third milestone, I would create a category and search feature to find chatbots within Allo, especially in the beginning when there are just a few chatbots available. In parallel, I would create a new chatbot category in the Google Play store because chatbots are going to explode and users need a better way of finding them.
While Allo is being redesigned, I would eliminate the other two apps: Duo and Messenger, and only keep Allo as the de facto messaging app of Google. I would also integrate Duo’s video feature into Allo, which is becoming a basic feature of a messaging app. Google has been marketing Hangout as an enterprise video conferencing application, and I think that is the right approach. I would also start a marketing campaign to make the differentiation between Allo and Hangout very clear.
In order to evaluate whether these changes are successful, I would use metrics that will measure whether they are having an effect on the customer base.
To measure the utility of basic features:
- Number of users that use the added basic features (attach document and start a call) per day, per week, and per month. This would indicate whether these features are indeed ubiquitous.
- Number of documents being attached per user per day, week and month.
- Number of phone calls per user per day, per week and per month.
- Number of video calls per user per day, per week and per month.
- Number of messages sent per user per day, per week, and per month.
To measure voice interface utility:
- Percentage of all messages per user in which the voice interface was used. Is the percentage increasing or decreasing or flat?
- Percentage of voice requests that were not understood by the chatbot? And Why? If people are not using the voice interface, we need to know why in order to learn how to improve it.
- What kinds of questions is the voice interface used for? If only for directions, this would indicate the user is moving and thus the voice interface is indeed useful to them.
To measure usage:
- Number of daily-active users and monthly-active users.
To measure a chatbot’s utility:
- Which activities are chatbots frequently used for? Are they used for search, tasks, recommendations, buying stuff, or services? If the most active chatbots are being used for services or eCommerce, this would be a sign that people are using the app to do things that they would have otherwise done manually, indicating that Allo is becoming more useful.
In summary, Allo’s basic problem is with its customer base. The main reasons are: lack of basic features, lack of voice interface, limitations of Google assistant chatbot, and confusing product offerings. I proposed solving the first three issues but prioritizing them based on how critical they are. I would start by adding the document attachment and start a call features to Allo because they are basic features of any messaging app. Next, include a voice interface, and lastly open the platform to third party chatbots, which will enable users to get answers to more queries and assist them with frequent tasks. I suggested implementing a category and search feature within Allo, and adding a new chatbot category to the Google Play store, so users can find chatbots easily.
Finally, I recommended consolidating Google’s four communication apps into Allo and Hangout and run a marketing campaign to differentiate them clearly in the eyes of the consumer.