When is the right time to use customer interviews, how to run them and how to use the insights gained from them?
I am totally aware that customer interviews is such a big topic that I could write 3 books on it and probably those would not cover it all — all the techniques and methods that can be used, all small biases that we should remember about, etc. However, I needed an accumulated piece of content that I can refer founders to — hence, this article. If you need more details on every subject covered below, don't hesitate to reach out to me via Twitter.
When to use customer interviews
The short answer will be: all the time 😅
Honestly, I can't think of any other type of market research or even wider, data extraction, that can produce as much value per effort as customer interview if it's done right. Google Analytics, traffic data analysis, heat-maps — it's all very useful. But nothing can beat the voice of real people.
Founders should definitely use customer interviews early on, on the idea validation stage.
I have an article that covers this subject in a more detailed way and explains why data analysis based on the data you can extract from online resources is not enough.
In a nutshell, when you research data online you should be aware of the fact that this data is heavily biased: it will follow the mood of the platform, so to speak. People on social media will exaggerate the size of the problem they have just because social media is a great place for rave and rant. Only conversations with real people (aka customer interviews) might reveal that the problem people are complaining about is neither important, nor urgent. Thus, you will be spared from investing your time and energy into solving the problem no one actually cares about that much.
Founders should definitely use customer interviews on the product launch stage while working on the landing page.
I have another blog post where I explain how to use customer interviews to create buying personas and find out your customers' implicit motives to buy. In a nutshell, your landing page should drive emotional response from your customers, therefore you have to find out what values resonate with them the most. The best way to do it — is to run customer interviews.
Founders should definitely use customer interviews when the product is launched at beta
to get insights on what features are more useful for customers and what are not so much, how they move through the onboarding process and what customers' expectations are. There are cases when founders pivoted to a totally different product due to customer interviews during the beta stage. One of the best recent examples here — Nira. Hiten Shah, co-founder of Nira, previously cofounder of Kissmetrics (an amazing analytics tool that connects all data to a real person), was working for almost 2 years on an app that helped with document search. The product was already in beta, and Hiten Shah and his team were running user interviews, talking to executives who were using the tool. During one of the interviews, the exec opened the laptop and went pale: "Why do I see these people? Who do they still have access to my docs if they are not working here for ages?" Product team quickly explained that he was seeing the current state of his documents and if he sees some people who the document is shared with then indeed, these people can still get this doc despite the fact that they don't work for the company anymore. It was a shock for the exec. The next day, he told Hiten Shah that he’d been up all night using Nira to find people with access to his documents, then going into G Suite to remove them. — This one conversation changed the course of our business, — explains the founder.
As a result of this one interview Nira was turned from the document search tool into the tool that helps companies track people who are not supposed to have access to some documents and block this access. Founders should use customer interviews later, when the product is up and running, to identify new opportunities and features customers would be happy about.
How do you interview customers to validate the problem?
When a VC asks you: "Did you talk to someone before building your product?" they refer to customer interviews. And almost every founder would give a positive answer to this question. However, 8 out of 10 founders I mentor presume that "talk to someone" means they show the prototype of their product to their friend, get feedback that sounds like "Wow! Wonderful idea!" — and that's the validation of their concept. Here's a big surprise: this is not a validation and the prototype demo is NOT a customer interview.
These are my favourite quotes from The Mom Test, a book that every startup founder should know by heart:
The conversations where you talk about our product only give us rope to hang ourselves
Whenever I hear a customer compliments me I hear alarms going ”you’ve messed it up”
People know what their problems are but they don’t necessarily know how to solve them.
If you go to Google and start researching the subject of customer interviews you'll stumble upon fancy techniques like "Jobs-to-be-done framework", "4 lists formula", "Stages of awareness differentiation" etc. My suggestion is: when you're on ideation stage — forget all these fancy words. Just go and interview people that you consider to be your potential customers.
But follow those simple rules:
NEVER ask a person what they think about your idea or product. NEVER ask them if they would pay someone for solving their problem. Instead, ask them to describe their existing process of solving the problem you think they have. Ask them yo walk you through their thinking process and the way they look for a solution.
If you're building a solution to the problem based on the tools that already exist and that you presume customers are unhappy about — ask, what exactly are they unhappy about. What "work around" have they found to deal with inconveniences they see in the existing tools.
Never ask a customer a straightforward question like "What are your existing problems?" Or "What do you feel stuck at?" People are horrible, indeed, with identifying their problems — if they were better at this , psychologists would not make a fortune helping people with finding out what their problems actually are. During customer interviews you have to be this psychologist: ask about current emotions, reactions and workflows. Never ask about future or probable actions.
I have a little e-book with illustrations about customer interviews. It explains 12 rules of customer interviews, what you should do and what should not be done under any circumstances. In a nutshell, those rules will help you build a conversation that is focused on the customer and their real problems, instead of you and your product.
How to identify a viable problem
Now, let's presume you have run 10-15 customer interviews (by the way, here's a useful article for you if you are not sure how and where to find participants for your interviews). How do you read the results? In some cases the results can be very straightforward — it's when you get about 90% of respondents reporting the same pattern. Sometimes, the split is closer to 50%. This is a reason to get suspicious, as 50/50 split tells you that what you're seeing is totally random results. You could flip a coin — and get the outcome with the same level of reliability. The answer to this might be:
— either the participants pool is too small
— or you have identified the wrong segment — it's too wide, customers show ambiguous behaviour, you have to narrow down the segment.
In any case, every interview should be recorded and transcribed (you can use Otter.io for that).
In every transcript highlight important quotes (adjectives and verbs customers use to describe their problem) and patterns (tools/ways/techniques they use to solve their problem now).
Mark where people report that the problem you're trying to solve is not urgent or important for them (more on it below)
Sort positive and negative patterns — this will give you an insight on what market segment you're not supposed to target and where you'd have more luck.
After every interview try to place a customer perception of the problem you're trying to solve, within the Eisenhower matrix. If you're not familiar with it - it's a beautiful concept that is used to increase productivity but I like to use it for problem differentiation too. If you place all existing products within this matrix you'll see that mostly, people pay for solving Important and Urgent problems. With Not Important but Urgent it works more or less too. But lower right squire is a dump for effort and energy — you won't be able to build a profitable business on it.
After a set of interviews and a thorough analysis you will be able to come up with a clear understanding of the following things:
— your customer segment (who has this problem you're trying to solve and perceive it as important and urgent or at least one of those)
— what are the competitive alternatives for your product? Competitive alternatives are wider than competition because they also include an option of doing nothing or doing something manually. For instance, you offer a fancy SaaS for budgeting — all the problems that you're offering to solve can be solved using Excel. If you're going to market it as a B2C solution you have to understand that thousands of people out there don't do their family budget at all and feel very comfortable with it.
How to estimate the size of the problem and if it's big enough to build your business on it
Now, knowing what you're up against and what will be the prize (total addressable market = your customer segment) you can calculate if this product worth your effort and energy.
Let's take an example of a founder I was working with in May, 2021.
He is building a platform that is very similar to AirBnB. It offers renting services for short-term stays. The basic difference is that it targets the commuting workforce — sales reps, consultants and other roles that have to spend 50% of time on business trips. Looks like a well defined audience, right? Let's see how he goes around with calculating the market size:
Americans make more than 405 million long-distance business trips per year. This means about 1.1 million people are traveling for business every day in the U.S.. TAM = 1.1 x 365 = 401 million people annually. Companies spend an average of $799 per person per day. About 34% of this amount is spent on accommodations (all this data is widely available online. Like here . TAM value = $108.9 B
The founder I work with wants to begin with the big cities and in particular, New York because that's where they are located. Target market value for NYC for business traveler in 2019 was around $60M (it is just the rough calculation of expenditure on the accommodations)
Now, the penetration rate. In case of apartment rentals this rate is basically the % of rooms that are occupied full year round. If you're dealing with, say, beekeepers of NJ your penetration rate would be for beekeepers that use applications like yours to improve their operations. How do you get this data? It obviously needs some creativity. The founder I work with joined Facebook and Reddit groups where AirBNB apartment owners discuss the downsides of their business and followed closely every conversation where occupancy rate is mentioned. We took the number 65% as a rough estimation (obviously, it was lower for 2020). Therefore, the SAM will be $70.7B.
What other paths can you pursue to come up with this number?
If you're thinking of a native app — check out how many times similar apps have been downloaded. Look up the app you want to check, tap on it when it appears in the search results, and it will take you to the download page. The number of downloads will be above the Install button and next to the app's size and age rating. Install the app and check it out. If it looks professional, with great features (you've been thinking of them yourself) and decent UX but download rate is very low — it is a reason to be worried. Perhaps, your target audience is not that tech savvy as you think.
Reddit and Facebook — again, can't stress enough the value of a proper research. You should have gone through this stage when doing initial discovery. Now, go back to the groups that you have found and see if people mention any software at all.
Google Keyword Planner — free option to figure out how often people look for solutions like yours. If you have not used it before — create your Google Ads account. In the Tools and Settings tab in the upper left corner you'll see Keyword Planner. Choose it, then follow the guidelines (specify the area of your search and language). Actually, I recommend doing this exercise anyway. Here, you see that the search for pitch decks is not so high in Canada. So if I wanted to build a service that provides customised pitch decks for Canadian startups, I would start thinking of adding something else to my product line. Because, I won't probably survive on a $5 service that is interesting to 1000 customers, taking for granted that there's someone else who wants to serve them and I won't get 100% of my addressable market.
Also, if you're dealing with physical products I have a great hack from my not very long DTC experience (yep, I've been doing Amazon sales too😂 )
Find a website that offers a product like you're planning to offer.
Place an order.
Record the order number.
Place another order in 24 hours and record the order number again.
Contact the service and say that it was a mistake and you'd like to get a refund.
What is it for?
Most online stores use a direct system to generate order numbers: they go one by one. Meaning, if your order #123 in the morning and #125 in the evening — it tells you that the company gets 2 orders a day. Add the average order size (usually, slightly less than the limit to get free delivery) — and it's a very rough and hacky way to evaluate the prospective revenue.
I'm not sure something like this can be done with subscriptions but there are obviously some numbers there. You can very well investigate using the same technique.
It is generally agreed that getting yourself 10% of the target market is a good bet. If we take my AirBNB-like service founder, it will be 10% of $70.7B = $7B. So, getting their business to $7B ARR is the best case scenario for them. But honestly, growing the business on a competitive and lucrative market like business travel to 10% of the market share is impossible if you're not backed up by very strong VCs. Therefore, the probable scenarios as I saw them were:
— total failure because the level of competition is too high. What keeps AirBNB from copying the offer? They have infrastructure, strong market presence, billions of dollars to run experiments.
— limit the market to a smaller niche where competition is not so strong and the maximum level of revenue is more humble (this will, theoretically, prevent big players from investing into aggressive efforts to monopolise this market).
My suggestion was to retarget the product to small cities with less business travel and where general attitude to AirBNB is not so good.
How to determine that you're qualified to solve this problem
This question is always asked by founders during the mentoring sessions, though I always presume that when you start building a solution to the problem you already consider yourself to be qualified
However, in a general, philosophical sense, and to maintain a holistic approach to product development it is recommended to choose a market segment and a problem that you have affinity for. In other words, your odds will be higher if you're scratching your own itch .
Having a strong affinity for the audience and a problem helps to keep a founder motivated and charged longer — and this is something that you will need, because early stage startup is a rollercoaster. Successful early-stage startup is a rollercoaster minus gravity — if you get a product-market fit you'll be growing like crazy and the risk of a burnout is very, very real. When there's a higher goal and implicit values, it definitely helps to maintain mental health. I have a little quiz that will help you to evaluate your affinity level, in case you're still hesitating if you've chosen the right audience to serve.
The recommended framework is to take what you know, combine with what you're good at, and add market demand.
The intersection works perfect for "productized" skills — you can see it pretty often with SaaS launched by agencies or professionals. For instance, Logology — a service that helps with creating unique yet not very expensive logos — was launched by a designer who had worked as a freelancer for years prior to this attempt of "productization". Classic example of super successful productization is Basecamp — a tool that the company was using for internal needs and when it was turned into SaaS became a huge success.
However, I'd say it is a recommended but not a necessary framework. There are multiple examples when founders managed to build a product or a service in a field where they had no first-hand expertise. For instance, Microacquire — a platform for selling and acquiring startups — was built by Andrew Gazdecki. He's never been in the SaaS business or in sales. He's just a person who sees trends and has a good network of partners.
Tools to track the voice of customer
Finally, let's cover what software you can use to stay up to date on the voice of your customer. In other words, how to avoid the "innovator's dilemma" and keep on solving the problems customers have. First of all, why is it important?
To answer this question let's just name a few companies that were on top of their game but ended up being complete losers:
Nokia — was one of the leading cell-phone manufacturers in early 2000. Completely shut down their phone manufacturing facilities several years ago.
Blockbuster Video — home movie and video game rental services giant, was evaluated over $500M in 2000. In 2010 they completely lost the game because of Netflix.
ToyRUs — one of the biggest US-based retailers that sold toys. It was supposed to feel great because in 2000 they signed a deal with Amazon that limited toy sellers on the e-comm platform to just 1 player - ToysRUs. However, in 2004 Amazon decided that it would be better to sue with retailers and pay the fees but let other manufacturers and retailers on the platform. ToysRUs was so focused on the contract with Amazon that it completely ignored the move of the customers online and did not make any attempts to build an e-comm branch. Had to file bankruptcy and close all the stores by 2018.
There are many others in this row — GM, Kodac, Compaq, Polaroid, Tower Records, etc. They all have the same reason for their failure — they stay ignorant about trends and the voice of customers.
What tools can you as a founder use to avoid thus fate? There are dozens of software solutions, most of which are enterprise level (like Medallia, Clarabridge, Qualtrics, etc.) As an early stage startup you don't need any of those. My recommendation for the founders — keep tracking Reddit subs, and influencers in your space. Whatever product you're dealing with there must be influencers on social media who cover this subject. Influencers are very responsive to change in customer demand and habits — if you see a change of narrative start getting worried. Also, every tool that will help you to collect user feedback — forms (Typeform, Tally, Google forms, Reform) chats, surveys — use everything you can. And obviously, keep on arranging customer interview sessions to assess how your customer natural workflow has changed and what improvements your product need to resonate with these changes.