## How Are You Being Evaluated

The interviewer is assessing your ability to provide quick estimates on any business related issue. The question could be about the size of a market, revenue estimates, number of daily active users, etc.

In product estimation questions, interviewers are evaluating your problem-solving and quantitative skills. They are looking for a ballpark number, not an accurate number. What matters is whether you are logical, can explain all your assumptions clearly, are organized in your work, and good with numbers.

## Answer Structure

The book *Cracking the PM Interview* by Gayle Laakmann McDowell and Jackie Bavaro provides a good suggestion on how to structure an answer for this type of question. It summarizes as:

**Ask clarifying questions.**Clarifying eliminates any ambiguity of what should include in your calculations.**Make an equation.**Consider edge cases or alternate sources of data and writing any facts that you know helps with calculations.**Break down the equation into components.**Write your assumptions next to the components. Doing this reminds you to explain the assumptions clearly to the interviewer.**Do the math.**Calculate the result of each component and compute the result.**Do a sanity check.**Do your results make sense? If not, recheck your equation, assumptions, and arithmetic.

## Answer Example

**INTERVIEWEE:** Should I estimate mobile ad revenue for the US or the entire world?

**INTERVIEWER:** For the world.

**INTERVIEWEE:** Okay, thanks. I know that Facebook currently offers four bidding options as ways to pay for ads: cost per 1,000 impressions (CPM), cost per link-click (CPC), cost per mobile app install and cost per Page like.

Facebook’s ad buying system is an auction-based bidding system. The winning bid depends on different factors besides the highest bid. It also depends on the quality of an ad, the target audience, the target industry, the country, and location where the ad appears among other properties. On mobile, the location where the ad appears is going to be the newsfeed or inside an app.

CPM, CPC, cost per mobile app install and cost per Page like, vary according to these factors. The same factors affect click-through rates (CTR) — the number of clicks divided by the number of impressions per ad.

I will use these metrics to estimate global revenue for mobile ads. But, to make a quick estimate, I need to use single numbers for these metrics; therefore I will use estimated averages in my calculations. Does this sound reasonable?

**INTERVIEWER:** Yes, go ahead.

**INTERVIEWEE:** Okay. Revenue from ad impressions and ad clicks can be calculated using two different equations. The impressions equation is based on cost per 1,000 impressions (CPM) while the clicks equation is based on click-through rates CTR and cost per click; where the click can be a link click, a mobile app install click or a Page like click.

**Equation A:** Annual revenue from impressions:

[$ / year] =

[#impressions / 1 person] X

[#persons / day] X

[cost per 1,000 impressions (CPM) / 1000 impressions] X

[360 days / year]

**Equation B:** Annual revenue from clicks:

[$ / year] =

[#clicks / #ads] X

[#ads / day] X

[cost per click] X

[360 days / year]

There are different click-through rates and cost per click for the Link click, App Install and Page like bidding options, so the revenue for clicks equation can be split into three:

*Annual revenue from link clicks:*

[$ / year] =

[CTR] X

[ #persons per day X #link-click ads / person] X

[CPC] X

[360 days / year]

*Annual revenue from cost per mobile app install:*

[$ / year] =

[app install rate] X

[#persons per day X #app install ads / person] X

[cost per mobile app install] X

[360 days / year]

*Annual revenue from cost per Page like:*

[$ / year] =

[page like rate] X

[#persons per day X #Page like ads / person] X

[cost per Page like] X

[360 days / year]

Now, I will use these equations to estimate the annual revenue of mobile ads. To count for mobile revenue alone, I will count only the daily mobile users of Facebook. From my recent reading of Facebook news, I recall that the number of daily mobile users is about 1B.

**INTERVIEWER:** Sounds good.

*(The interviewee starts making the calculations below and making side notes about the assumptions he is making as he enters numbers into the equation. Once he finishes each calculation, he walks the interviewer through his calculations.)*

**INTERVIEWEE:** Let’s start with annual revenue from viewed ad impressions. I check Facebook on average twice in an hour when I am not with my phone or exercising. So, say I am active with the phone 12 hours a day, and I check Facebook twice per hour, that is 24 times. Assuming I am shown an ad impression every time I log into Facebook then that makes 24 times I see an ad impression. Does this sound reasonable to you?

**INTERVIEWER:** Let’s go with it.

**INTERVIEWEE:** Okay, so 24 ad impressions multiplied by 1B persons results is 24B impressions per day. Now, CPM will vary widely, but I recall reading a report where a global average of $1 was estimated, so I am going to assume $1 for CPM. Multiplying 24B impressions per day by a cost per thousand impressions of $1 and dividing by 1000 results in $24M per day. And, multiplying that by 360 days results in $8,640M or about $9B a year.

*Annual revenue from impressions*

= [24 impressions per person]

X [1B persons / day]

X [$1 for thousand impressions / 1000]

X [360 days / year]

=~ $9B / year

Let’s estimate annual revenue from link-click ads. Again from my readings about Facebook ad benchmarks, I will assume a CTR of 1% and CPC of $0.2. Now, to estimate the number of link-click ads Facebook shows a person per day, I am going to guess that it is half the number of viewed ad impressions for which we just calculated revenue. Half of 24 is 12, so I am going to guess 10 link-click ads, to simplify things.

Applying these numbers to the equation we have 1% CTR X 1B persons/day x 10 ads/person X $0.2 CPC X 360 days = 10⁷ X 2 X 360 = 720 x 10⁷ or about $7B a year.

*Annual revenue from CPC*

= [1% clicks / link-click ad]

X [1B persons / day X 10 link-click ads / person]

X [$0.2 / click]

X [360 days / year]

=~ $7B / year

Let’s estimate annual revenue from cost per mobile app install ads. In general, I think users click less on ads to install apps than other ads, so I am going to assume a very low CTR of 0.01%. Now, I know that cost per click for app install ads are higher than regular link-click ads, so I am going to assume a cost of $0.8. I am guessing the reason why cost per app install is so much higher than a regular link-click ad CPC, is because the click-through rate of app installs is so much lower than link-click ads. The lower the click-through rate, the less value the ad has for a user since they are not clicking on them that much. Facebook’s bidding algorithm takes this into account because it tries to prevent unappealing ads from appearing on Facebook. Now, how many times are these ads shown to a user? I am going to guess twice a day. When I use Facebook, I rarely see these types of ads.

So putting these numbers together to calculate annual revenue from cost per mobile app install, we have 1% CTR X 1B persons/day x 2 app install ads X $0.8 cost per mobile app install ad click X 360 days = 10⁵ x 2 x 8 x 36 = 10⁵ x 576 which is about $58M a year.

*Annual revenue from cost per mobile app Install*

= [0.01% clicks / app install ad]

X [1B persons /day X 2 app install ads / person]

X [$0.8 / click]

X [360 days / year]

=~ $58M / year

Let’s estimate annual revenue from Page Like ads. I think the click-through rate for Page like ads is higher than app-install ads but much lower than regular link-click ads, so I am going to guess a CTR of 0.07%. In terms of cost per page like, I know the cost is also more expensive than regular link click ads, so I will use the same $0.8 estimate as the app-install ads. And in terms of how frequently users are shown these Page Like ads, I think they are shown more frequently than app-install ads, but less than link-click ads, so I am going to guess 5 times per person per day.

So putting these numbers into the equation, we have 7% CTR X 1B persons/day X 5 ads X $0.8 cost per Page like ad click X 360 = 7 x 10⁵ x 4 x 360 which is about 10⁵ x 7 x 1600 or 11200 x 10⁵ = $1B per year.

*Annual revenue from cost per Page like*

= [0.07% clicks / Page like ad]

X [1B persons X 5 Page like ads / person per day]

X [$0.8 / click]

X [360 days / year]

=~ $1B / year

So summing all these numbers up, we have $9B + $7B + $58M + $1B which is about $17B.

A recent eMarketer report estimated that Facebook would make $34B in advertising in 2017. Currently, mobile users make 54% of total users, so based on eMarketer estimates, mobile advertising should be roughly 54% X $34B or $18B. Since my estimate of $17B is very close to eMarketer’s, I feel more confident that I am in the ballpark.