Goals for Instagram Hashtags (Part 1)
This article has two parts. Part 1 is about how to set goals and metrics for the Instagram Hashtag. And part 2 is about how to diagnose a problem with the feature. This article covers part 1.
Question
Facebook is launching Instagram Hashtags for the first time. What is the goal and metrics you would set for your team?
What is this question about?
The “set goal(s) and metrics” question is part of the execution interview at Facebook. Other companies are also asking these questions, following the Facebook interviewing style.
This question tests the following:
- Do you have a thoughtful process of defining and prioritizing product goals?
- Do you have a method for choosing top metrics for the product goals?
- Can you define key product metrics?
- Can you rationalize tradeoffs?
- Can you debug product issues?
The interviewer may ask you to determine a goal for a product that does not exist, a goal for an existing product, or the goal of a product when it was launched. In the case of an existing product, they want you to set a new release goal. In the case of a product that does not exist, they want you to identify the key problem to solve and define a goal around it. And for a product launched in the past, the interviewer wants you to infer what the goal was when it was launched. How the interviewer frames the question will reveal which of these scenarios is at play. In this particular case, the question refers to a specific feature of Instagram and asks you to look back and think about the product goal when the product was about to be launched?
We have split this answer into two parts. The first part answers the central question of setting a goal and defining metrics. And the second part answers a follow-up question dealing with how to diagnose a product issue.
What is the interviewer looking for?
The interviewer evaluates you in several areas: motivation, goals, metrics, tradeoffs and problem diagnostics, and communications. And here is what they are looking for:
Motivation: Can you infer the reason behind building the feature?
Goals: Can you propose a few product goals and prioritize the most important?
Metrics:
- Can you define a set of metrics that provide a holistic view of the product’s health if the product exists? Or metrics that measure the success of a new product launch, or measure the success of a new product release?
- Can you reason and argue what a good North Star metric should be for your product?
- Can you define counter metrics to keep your North Star in check?
Tradeoffs and problem diagnostics: Are you able to identify and clearly articulate a key tradeoff, and do you have a systematic way of identifying product issues?
Communications Are you concise and able to deliver a narrative that shows that your thought process is logical?
How to structure your answer?
Clarification
Ask questions to remove anything that seems unclear or ambiguous. For example, this question is posed in the present tense, as if the feature does not exist. But in fact, it has been around for a while. So ask if they mean for you to go back in time when Instagram launched the product.
Motivation and Goals
Explain why you think the Instagram team built the Hashtag feature. To do that, infer what the Hashtag is helping to do. Then state the goal of this Hashtag feature and the goal of the launch.
Start by talking about Instagram. Explain what Instagram is, what user problem it solves, and its product goal. Defining Instagram’s goal helps you understand the purpose of the product that contains the new Hashtag feature. Next, talk about what problem this Hashtag feature solves within Instagram, and then define the goal of the Hashtag feature based on the problem to be solved. Finally, explain how both the Hashtag feature’s goal and Instagram’s goal are aligned.
It’s best to define goals at this stage because you will be defining metrics that measure these goals later.
To summarize, talk about the following points to help you infer the goal of the new Hashtag feature, when it was launched.
- Instagram: What is Instagram? How does Instagram work?
- Instagram User problem: What is the user problem Instagram solves?
- Instagram goal: Define the goal based on what user problem Instagram solves. The goal is to solve that user problem.
- Hashtag: What is a Hashtag? How do hashtags work?
- Instagram Hashtag User problem: What is the user problem the Hashtag solves?
- Hashtag goal: Define the hashtag goal based on what user problem the Hashtag solves. The goal is to solve that user problem.
Goals: How does the Hashtag’s goal align with Instagram’s goal?
Metrics
Explain to the interviewer first how you think about metrics. In general, the interviewer will want to see how you define metrics that monitor the product’s health. Including metrics for all types of product users, if there are multiple types. For example, in Facebook Marketplace, there are two types: buyers and sellers. You could say that you want to monitor the product’s health via metrics that give you a holistic picture. In other words, metrics that tell you if the product is growing, if users are engaged, and if they are coming back (e.g. metrics of growth, engagement and retention). And to define these metrics, you will talk about which user behaviors are important to track because they impact growth, engagement, and retention. In summary, to define metrics of success for your product, follow the following steps:
- Talk about the users and their motivations for using the product. Most Facebook products are two-sided (e.g.Instagram has influences and followers), so talk about all the sides involved.
- For each user type, describe the users’ behaviors that signal growth, engagement, and retention when they use the product. After describing each behavior, define a metric that measures the actions and things derived from that behavior. List the behaviors that take place from the beginning to the end of a session (e.g., from top to bottom of the funnel). For example, an Instagram influencer would first compose the post, then publish it, and respond to comments. Think of people, things, and actions you could measure. For example, think of the number of influencers (growth metric), the number of posts (growth metric), and the number of replies (engagement metric). On the other hand, followers or fans would do the following through their Instagram session: view the Instagram feed, click on a post, engage with the post (e.g., like it, comment, share or save).
- Define metrics that measure the product’s goal (Instagram’s goal), the feature’s goal (the Hashtag’s goal), and the launch goal.
- Define counter metrics—metrics that keep your North Star and goals in check. These metrics signal that something is wrong with your product, even though the North Star metric is positive. For example, user behavior or activities that go against your product and business goals. For example, take Facebook Events. The events can be categorized as private and public events. The goal of the private side of Facebook events could be to help users organize events to meet and connect with friends in those events. A North Star metric could be the number of RSVPs to an event because if people RSVP, there is a likelihood that they will go to the event and meet. So one behavior that would be contrary to this goal and the North star metric could be if users stopped organizing events. The number of RSVPs could be going up, but if the number of events are on the decline, then there is something wrong. So the number of events would be a counter metric.
Tips
Avoid using averages as metrics. Instead, use counts.
The reason is that outliers easily skew averages. For example, you may have 10 users; 9 scored very low (1 in a score of 1 to 30,000), while one scored super high (30,000). Therefore, the average score will be high 3000.9 (30,009/10 = 3000.9). And one would get the impression that the majority received a score or 3000.9, which is much higher than the score most everyone got, 1.
Counts, on the other hand, are absolute numbers. They give an insight into how a metric is trending, whether it is going up or down. However, there are exceptions where the average is the right metric to use. For example, when you want to know the conversion rate on an ad. This metric is critical for estimating ad revenue.
Answer Example
INTERVIEWER: Facebook is launching Instagram Hashtags for the first time. What is the goal and metrics you would set for your team?
Clarify
INTERVIEWEE: The Instagram Hashtag feature already exists, so is the question about what was the goal of the product when it was launched?
INTERVIEWER: Yes.
INTERVIEWEE: Are we talking about the Hashtag feature for the Instagram Feed or across all Instagram features? Today, Instagram has IGTV, IG Live, IG Stories, IG Reels, and IG Shops. By the time the Hashtag feature was launched, the Feed was the only feature of Instagram. So should I assume that the context is the launch of the Instagram Feed Hashtag?
INTERVIEWER: You can assume that.
INTERVIEWEE: Thank you for clarifying. Could I have a moment to structure my analysis?
INTERVIEWER: Sure, go ahead.
Motivation and Goals
Before talking, the interviewee jots down the following bullet points on a piece of paper to delineate her talking points:
Why was the IG Hashtag built?
- Instagram: What is Instagram? How does it work?
- User problem: What user problem does Instagram solve?
- Instagram goal: What is Instagram’s goal?
- Hashtag: What is the Hashtag feature? How does it work?
- User problem: What user problem does the Hashtag feature solve?
- Hashtag goal: What is the Hashtag’s goal? How does it align with Instagram’s goal?
- Metrics: What could be metrics to measure the success of Instagram and the Hashtag?
INTERVIEWEE: To identify the goal of the Hashtag feature when it was launched and the goal for the launch, I need to provide some context about Instagram. And then go deeper into what problem did the Hashtag solve for Instagram. With this in mind, I will talk about Instagram, how it works, and its product goal. Then talk about the Hashtag feature, the reasons for introducing it on Instagram, and how it improved Instagram’s user experience. I will then infer what the product goal of the Hashtag feature is, based on what problem it solved. I will also talk about the goal for the launch of this feature. How does that sound?
INTERVIEWER: That sounds good. Go ahead.
INTERVIEWEE: Great. So let’s start with Instagram. I have used Instagram on occasion, and my understanding is that before Facebook acquired it, it was a mobile application for photography lovers to share their photos with their fans. So it was a platform that connected people around their love for photography. After Facebook acquired Instagram, Instagram became more of a place where famous people or influences and their followers connected. And the connection was around moments in their lives captured in photos or videos. Regular people also shared their lives’ moments on Instagram, however much of the engagement is around influencers. Am I correct in this assessment so far?
INTERVIEWER: Yes.
INTERVIEWEE: So, if I were to describe Instagram in one sentence, I would say that it is a mobile application that connects influencers with their followers through photographs or videos about moments in their lives. Would you agree?
INTERVIEWER: Yes, that is a concise way of describing it.
INTERVIEWEE: I think that influencers want to have a way to share and connect with others through their photographs or videos. And fans want to stay abreast of what influencers were doing and connect with other fans. So, I think that the goal of Instagram is to enable people to connect with their favorite Instagram influencers through photos and videos about their lives and to connect with other users around these moments. Would you agree?
INTERVIEWER: Yes, that is an excellent way of putting it.
INTERVIEWEE: This goal fits very well with the mission of Facebook, which is to empower people to build communities and bring them closer together. Instagram does this by bringing people together around these photos and videos of their favorite influencers.
Now that we have aligned on the goal of Instagram let’s talk about the Instagram Hashtag feature. I want to talk about what problems it solves, the impact it has on Instagram’s goal, and from this analysis, state what the goal of the Hashtag feature is. Does this sound reasonable?
INTERVIEWER: Sure.
INTERVIEWEE: The Instagram Hashtag feature is a label that a user can create and attach to their posts to associate them with a specific topic. The objective is to make it easier for users to find relevant posts when searching for keywords or hashtags with words similar to the label. Would you agree?
INTERVIEWER: Yes
INTERVIEWEE: So the hashtag feature is good for creators because it enables them to make their posts discoverable based on general interests around a topic. And it is good for Instagram users because it allows them to surface posts related to topics that interest them.
Therefore, the Instagram Hashtag feature solves the problem of improving the discoverability of posts. Based on this, I would infer that the Hashtag goal is to help users increase the discoverability of their posts and the relevancy of their search results. Would you agree with this goal statement for the Hashtag feature?
INTERVIEWER: Yes, that sounds reasonable.
INTERVIEWEE: Going back to when the Hashtag feature launched, I think the expectation was to increase posts’ engagement. If this feature helps people find more relevant posts, users are more likely to like them, write comments, and share them. This engagement, in turn, would encourage influencers to post more, feeding into the flying wheel of creating and engaging with posts, which triggers more content creation.
So to summarize, the goal I would have set for the launch of the Instagram Hashtag feature would have been to help users increase the discoverability of their posts and the relevancy of their search results to increase user engagement.
Metrics
INTERVIEWEE: Now, let’s talk about metrics. If I were the PM of Instagram, I would define a few key metrics to help monitor the product’s health and measure the success of the goals.
To monitor the product’s health, I want to define metrics that signal whether the user base is growing, engaging with the product, and returning to Instagram.
Instagram is a two-sided application, with creators or influencers on one side and followers on the other. Therefore I will define metrics that consider both sides.
Side note: When defining metrics, it is helpful to think about the user experience through the product (aka the funnel) and how the users interact with the product. Usually, activities at the top of the funnel signal growth, for example, how many people visit the app. As the user goes through the funnel, activities will be more about engagement, for example, liking a post or commenting on a post. Then you can define metrics that measure those activities. Define metrics that count people, things and user activities (e.g., spending time reading posts or liking posts).
The interviewee starts by describing user actions at the top of the funnel for each type of user; and writes down one metric at a time, explaining why that metric is essential. Below, we categorize the paragraphs about the metrics based on whether they are about growth, engagement, and retention. And as a sub-category, whether they are about counting people, things or activities.
INTERVIEWEE:
Growth Metrics about people
In terms of growth, I would like to know if the number of users coming to Instagram daily increases. DAU would be a good metric for this.
Growth metrics about things
I would also keep track of the number of users that post daily. The number of Instagram posts should also increase if more users use Instagram. I would benchmark this metric with a similar metric from similar Facebook features such as Newsfeed.
Engagement metrics about activities
Moving on to engagement metrics, I would track the total time spent. If people are engaging more with Instagram, this metric should increase.
Engagement metrics about people
Knowing how many Instagram users engage with posts would signal deeper engagement than just viewing. Therefore I would like to measure the number of users that interact with a post, in other words, the number of users that like a post, comment, or share.
Engagement metrics about things
Overall, I would like to see an increase in engagement with the Instagram posts. Therefore, I would track the number of likes, comments, and shares.
Retention metrics about people
And to check if users are returning, I would measure the number of users that return month over month and DAU/MAU.
Summary List of Metrics
DAU
# Number of people that post
# Number of IG posts
Total time spent
#Number of users that interact (e.g., like, comment, share)
#Number of likes + comments + shares
#Number of users that return month-over-month
DAU/MAU (Daily Active Users / Monthly Active Users)
Besides these metrics, I would define metrics that tell us if our goals are being reached. We want to achieve three goals: Instagram’s product goal, the Hashtag’s feature goal, and the launch goal.
North Star
Let’s start with Instagram’s goal. The goal was to enable people to connect with their favorite Instagram influencers through photos and videos about their lives and connect with other users around these moments.
Candidates for a North star could be:
- The number of users that like or comment. This metric counts people. It counts the number of fans who connect with their favorite influencers by liking and commenting on a post. Therefore it is a direct indicator that the goal is being achieved
- The number of Instagram posts that have likes or comments. This metric counts posts. It measures if posts are engaging.
I think the first metric, the number of users who like or comment, is a more accurate success measurement of Instagram’s goal. It counts people; it provides a direct measurement of whether people are connecting through the posts. While the second metric, the number of posts with likes, shares, or comments, tells us if Instagram’s posts are attractive to users, it doesn’t tell us how many people are engaging and connecting. One concern is that a count does not tell us if most people are engaging with the posts. A percentage would tell us that. But if I were to choose between a count and a percentage metric, I would choose the count; because percentages don’t tell you about growth. You can have 100% of 1 user, but what good is that? However, counts tell you about trends, whether you are growing or declining, and whether the volume is growing, which is a much more insightful signal of success.
Counter Metrics
Using the North Star metrics as a single measurement of product success is dangerous. One single metric is susceptible to being gamed or provides insufficient visibility. Therefore we need to define counter metrics to keep us honest. For example, some situations where the North Star could be misleading are, for which I would define counter metrics are:
- Suppose fake users count for a significant number of engaged users. The North Star, the number of engaged users could be increasing, but it would be counting the wrong type of users. So I would define the number of fake accounts as a counter metric.
- Or if the intensity of engagement declines even though the number of engaged users is not. This situation may happen if users get bored or spend more time on another feature. Therefore I would define the number of interactions, including likes, shares, and comments, as a counter metric.
Or if the number of posts decreases even though the number of users engaging with posts is still increasing. That may happen if there is a new competing product and users are starting to spend more time with those products. Therefore I would define the number of posts as a counter metric.
Hashtag Success Metric
Let’s talk about a success metric for the Hashtag feature and launch goal. To recap: the goal of the Hashtag feature is to help increase the discoverability of users’ posts and the relevancy of users’ search results. And the goal of the launch is to increase engagement by virtue of the Hashtag feature.
A few metrics that are good candidates for measuring the success of the Hashtag goal could be:
- The click-through rate of search results.
- Or the number of Instagram posts with likes, comments, or shares.
- Or the number of likes, comments, and shares.
Of these three metrics, I think the metric that more closely measures discoverability and relevancy of the Hashtag feature is the click-through rate of search results. If the hashtag feature makes relevant posts easier to discover, users will click more frequently on the resulting posts. The other two metrics measure the volume of engagement activity, but they don’t directly attribute that to the Hashtag feature. Would you agree?
INTERVIEWER: Yes, I agree with that statement.
INTERVIEWEE: Okay. So, if the launch of the Hashtag feature is successful, we can expect an increase in user engagement with the Instagram posts. A few metrics that may measure engagement could be:
- The total number of likes, comments, and shares.
- Or the number of users that click on likes, comments, or shares.
Of these two, I think the total number of likes, comments, and shares gives a better sense of how much more engagement there is overall. The second metric, the number of users that click on likes, comments, or shares, only tells us if people are doing something but not its intensity. Of course, the number of likes, comments, and shares doesn’t tell us if a majority or a minority of users are doing this, but that is what the North Star measures.
INTERVIEWER: Thank you for that coverage of metrics. Now, imagine that the launch happens, and your North Star, the number of users that engage with posts is increasing very nicely, and your click-through rate for the Hashtag feature is also increasing. But they start to decrease in the fifth week, declining 20% by the sixth week. How do you figure out what is going on and what would you do?
This follow-up question is a diagnostic question typically used as a follow-up question to the metrics. The answer to this question will be published in the second part of this article.
