فصل 3

CHAPTER THREE

Asking important questions

Once we know about The Mom Test and start trying to ask non-biasing questions, sometimes we over-compensate and ask completely trivial ones. Asking someone how old they are isn’t biasing, but it also doesn’t move your business forward. You have to apply The Mom Test to the questions which matter. Otherwise you’re just spinning your wheels.

In addition to ensuring that you aren’t asking trivialities, you also need to actively look for the world-rocking scary questions that you’ve been unintentionally shrinking from. The best way to find them is with thought experiments. Imagine that the company has failed and ask why that might have happened. Then imagine it as a huge success and ask what had to be true to get there. Find ways to learn about those critical missing pieces.

You can tell it’s an important question when the answer could completely change (or disprove or kill) your business. If you get an unexpected answer to a question and it doesn’t affect what you’re doing, then it probably wasn’t a terribly important question to begin with.

Every time you talk to someone, you should try to ask at least one question which has the potential to completely destroy your currently imagined business.

One of my companies had some legal ambiguities around content ownership. We thought it’ll be okay in theory, but it lacked strong precedents. And the few people we talked to, more or less told us the same. I was always a bit nervous that we would get “found out.” And so I kind of avoided talking to our customers, lawyers. And then, a key customer’s executives were very excited and their creatives were thrilled and we’d even brought the stubborn tech team around to our side. I spent months working on this complex sale and it finally seemed like the dozen people who needed to agree, all agreed. But in all those early conversations, I’d never asked to talk to their lawyers. It was an important question which for whatever reason, I shrunk from. And not asking it cost us the entire deal.

There’s no easy solution to forcing yourself to face and ask these scary questions. I once heard the general life advice that, for unpleasant tasks, you should imagine what you would have someone else do if you were delegating it to them. Then do that yourself.

And remember, you’re allowed to ask about money. You’re a startup. It’s okay.

Rule of thumb: You should be terrified of at least one of the questions that you’re asking in every customer conversation.

Love bad news

One of the reasons we avoid important question is because asking them is scary. It can bring us to the upsetting realisation that our beloved idea is fundamentally flawed. Or that the major client is never going to buy. Although this sort of news feels unfortunate, we need to learn to love. It’s solid learning and hearing it is getting us closer to the truth.

If you’ve only got one shot, then bad news is bad news. For example if you’re bungee jumping and your cord doesn’t work, then that’s bad cause you only get one attempt. If you’ve sunk your retirement savings into opening a cafe and it doesn’t work, then that’s bad. You only get one shot in pulling your retirement savings. If you save up a bunch of money for your business and you spend the entire thing on the first version of your idea and it doesn’t work, then that’s bad.

But on the other hand, the bad news stops being bad if you get multiple chances. If you save $50k for your business and you only spend $5k to learn you’re running down a dead end, that’s actually great news because you can use the rest to find a viable path to your goal with the advantage of all the extra stuff that you don’t know.

Similarly, if you have an exciting idea for a new product and go talk to people and they don’t really care about it, then that’s a great result. You just saved yourself however much time and money it would have cost to try building and selling it and you can instead put that energy towards an options more likely to succeed.

We go through the futile process of asking for fluffy opinions and fishing for compliments because we crave approval. We want to believe that the support and sign-off of someone we respect means our venture will succeed. But really, that person’s opinion doesn’t matter. They have no idea whether our business is going to work. Only the market knows.

Learning that you believe the wrong stuff is supper frustrating, but it’s still progress. It’s bringing you ever closer to the truth of a real problem and a good market.

The worst thing you can do is ignore the bad news while searching out some tiny grain of validation in order to celebrate. You want the truth, not a gold star.

Some of the most informative (and thus best) responses you can get are along the lines of, “Umm, I’m not so sure about that” and “That’s pretty neat.” Both are lukewarm responses and a lukewarm response is actually amazing because it’s a crystal clear signal that the person that you’re talking to does not care in the slightest.

In this context, “best” response means best in terms of the amount of information you’re learning, not best in terms of how likely you’re close to sale. In the early stages your focus is on learning, if you’re a sales guy with a mature company and you’re going door to door selling gadgets and if someone doesn’t care, then that is a bad result cause you’re not getting your sale’s bonus. But if you’re trying to decide whether to develop, build and promote those gadgets on the first place, then lukewarm is terrific because it gives you a crystal clear signal that the person does not care. It’s reliable information that you can take to the bank which is making you smarter and your business better.

The classic error in response which is a huge mistake to a lukewarm signal is to “up your game” and pitch them until they say something nice. However less they’re holding a check, the only thing you have to gain from “convincing” them are false positives. Remember you’re not here to collect compliments; compliments are fool’s goal. They’re worthless. You’re trying to learn the truth. And their lukewarm response already gave you the truth. They don’t care.

If the person you’re talking to is still engaged and happy to have the conversation, it can be worth asking a couple follow-ups to understand the nature of their apathy. Is it because the “problem” isn’t a big of deal? Are they different from your ideal customers? Do they not care about your specific implementation? Have they heard too many pitches in the space and they’re just warned out, like cafe owners in the aftermath of Groupon? Or are they just having a bad day and they’re not in the mood to hear it?

Once you’ve kind of understood their indifference, say a big thanks and leave them to their day. They’ve probably helped you out more than the guy who said he loved it.

Rule of thumb: There’s more reliable information in a “meh” than a “Wow!” You can’t build a business on a lukewarm response. Take that stuff seriously, fund another out.

Look before you zoom

Another way to miss the important questions is by obsessing over ultimately unimportant nuances. We let ourselves get stuck in the details before understanding the big picture.

Everyone has problems that they know about but they don’t actually care enough about to fix. And if you zoom in too quickly and lead that to that semi-problem, then they’ll happily spend all telling you about the unimportant details. Zooming in too quickly on the super specific problem before you understand the rest of the customer’s life can irreparably confuse your learnings.

So here’s a (really, really) bad customer conversation:

You: “Hello. Thanks for taking the time. We’re building phone and tablet apps to help people stay in shape and just wanted to understand how you stay fit.” This opening isn’t necessarily terrible, but opening, but if you can avoid mentioning the idea, that’s great.

And they: “Okay.” They’re thinking, I never exercise, so this should be quick.

And you continue with your conversation: “How often do you go to the gym?” This is a demographic question. It doesn’t give you any new insight, but it can be useful for figuring out which sort of person you’re talking to and so you can ask the right follow-ups questions.

They say: “Not really ever.” And they’re thinking, I guess we’re done here!

But you’re kind of not getting the signal: “What would you say is your biggest problem with going to the gym is?” Well, now starting to go horribly wrong. You’ve ignored their indifferent response.

“Yeah. I don’t care really about the gym! I don’t really go.”

And instead of trying to understand their indifference, you’re plowing ahead and pushing forward into your product and your problem: “So what’s your problem with going to the gym?” Anything we get is gonna be misleading here cause they just don’t care but since we asked the question, they’ll make something up to tell us.

So they reply: “Well, I guess the time it takes to get there. I’m kind of busy, you know?”

But this is crazy. This is a crazy response that we’ve forced them into it because they don’t even have a problem. They don’t care. They’re not trying to solve the gym issue. They’re not motivated. It’s like, it’s not in their life. And they’re giving you an answer which sounds plausible cause you force them to. But convenience, really convenience, it’s pretty convenient to do pushups at home and are they doing that? Well, like the way you frame the question prevents you from ever learning that. You zoomed in too quickly.

But entrepreneurs typically don’t notice. They’re like, convenience. That makes sense. It’s kind of validating the suspicion that they already held. So they say something like, “Perfect. That’s great. And could you rank these 4 in terms of which is most important to you in a fitness program: convenience, personalisation, novelty, or cost?”

It’s just so crazy at this point. You’re having this whole conversation. You’re collecting all these “feed backs” from someone who’s not even a customer. They don’t even care. But of course you’re asking, so they’ll answer: “Probably convenience, then cost, then personalisation, then novelty, I guess.”

The entrepreneur think: “Okay. Awesome. Well, listen, we’re working on an app. It’s gonna let exercise in the convenience of your own home. I think it matches the stuff you care about.” Which is obviously totally missing the point and mis-interpreting the bit we’re started with, which is that they don’t care. And now we’re on the fishing for compliments.

So what are they gonna say? Like, they’re gonna say: “Okay, cool. Let me know when it launches.” It’s a half-hearted compliment plus a stalling tactic.

And the entrepreneur leaves, thinking: “Great — I’ll send a beta key.” This is awesome. We got a user! We got a customer.

And they’re thinking, I am never gonna click on that link.

The reason this conversation is so bad is because, it seems like it went well if you weren’t paying attention. There’s like specific feed-back, all these ranked criteria for most to least important. They gave us the reason why they don’t go to the gym. It’s like, it’s really easy to mis-interpret this as being valuable. And then to use it to justify your product decisions? When in fact the whole conversation’s made up. You may as well dreamed it cause you basically ran with bad questions and force the person into giving you fake answers.

So let’s look at this problem with another version of the conversation and this time we’re trying to fix it when it goes off track. You can always fix these conversations. It’s not about never making the mistakes, it’s just about noticing and then bringing back to a good place.

So you ask: “How often do you go to the gym?”

And they’ll: “Um. Never. I don’t care.”

And you go: “Why not?”

“I don’t know, it’s just not something I’m that worried about, you know? I’m not trying to fix this. It doesn’t motivate me.”

So you might ask, cause again you can push back on these generics, these nevers, these always: “When’s the last time you did try? Have you ever joined a gym or taken up jogging or anything?” And they’ll say: “Oh yeah, I used to be into sports in high school. It just hasn’t been a big deal since I settled down. Running around after the kids gives me all the cardio I need!” And you’re like: “Alright, cool. I get it.” You could chat a little bit longer if they’re friendly but basically, my view at this point, they’re not a customer so I’m gonna leave them to their day. Go find who is a potential customer.

The premature zoom, like ignoring whether people care about the broader problem then zooming in to the specific details of your offering, it’s a huge problem because it leads to data that feels like validation, but is actually worthless. So it’s a big source of false positives.

So if we take the same conversation again, but this time instead of immediately zooming in onto exercise, we’ll start a little bit more generic. And this is gonna help us figure out whether fitness is even an important problem for them.

“Thanks for taking the time to chat to me. Could I ask what are your big goals and focuses right now?”

They’ll say: “The biggest one is finalising my promotion at work. And the wife and I just bought our first house, so we’re gonna get that all fixed up and ready to go. It’s exciting times. And of course I’m gonna find a bit more time for the wife, the kid, the family, my parents . I wanna see everyone. Things have been pretty busy lately.”

Okay, so his top of mind problems are work, house, marriage, family. Fitness didn’t even register. So, we can talk. You van banter. It’s not a big deal. It’s like talking to a friend.

“You’re buying a house and you’ll expect to be less busy?”

“Whatever. I’m hopeful. I’m optimistic.” And at this point, I’m pretty sure he doesn’t care about exercise but I might ask, skeptically, just to check.

“Is getting healthier on that list?”

They might say something like: “I’m actually feeling pretty good at the moment. Fine.”

We don’t always need to start the conversation from the square one of do-they-care-at-all like we did in this case. Sometimes we can assume that they care. Like if I’m talking to a small business, I can probably safely assume they would like to get more customers. I can probably safely assume that the small business owner would like to have better processes and more free time. I can look at industry reports and figure out what the major problems are. If those are the cases, then I can assume my problem’s important and get right into the details. But if you’re not sure that that’s true, either for the individual that you’re talking to or for the segment at large, then you might wanna start these conversations a bit more broadly and try to first figure out like, do they even care at all about what I’m doing. And if they do, then you know you can safely zoom in.

The phrase “if you could get me more gigs” is basically shifting the burden from the customer to your product. Even though you’ve found a pain, your success is dependent on a bunch of other factors, such as your ability to grow a healthy supply of paying gigs which are a good fit for this particular speaker. Will you be able to do that? It remains to be seen.

This situation is easier to spot in the online advertising industry. Imagine running customer conversations with an advertiser to try to understand their pains so you can convince them to advertise on your site. They’d sort of look at you blankly and say, “Listen, if you can get enough page views, we’ll pay you.” In fact, it’s such a well established would-pay-to-solve-problem that you don’t even need to talk to them to set it up. You just plug in an ad network and you’re done.

Similarly with affiliate commissions. If you sell a company’s products, you get a cut. That’s just how it works. You don’t need to explore or validate or understand their problems. The risk here, resides in your ability to get lots of traffic and sell lots of products or to get lots of speaking gigs. If you can do that, your users will pay you.

In all of these examples, the primary risk is in your product, not in the customer. They’ll just pay you if your product gets big enough, successful enough. If you can get the market place to scale.

So there’s these 2 different types of risks. There’s product risk, which is: Can I build it? Can I grow it? And in those cases you really have to build the prototype. You have to start small. You have to grow it from there.

And then you got the second type of risks which is your customer and market risk — Do they want it? Will they pay me? Are there lots of people like them?

You can’t overlook either one. But typically, most startups, most products will have more of their risks in one or other place. So prototyping is usually your tool for dealing with product risk whereas customer conversations are an incredibly useful tool for dealing with customer and market risk. But you can’t overlook either. I remember talking to a founder who had wasted 3 months on worthless customer conversations. He wanted to start a company building gadgets that track the fertility of farm animals, with the goal of ultimately boosting the birthrates cause you can catch some problems early and this would then farmer revenue. When he talked to farmers, he asked questions like, “Would you switch trackers if something cheaper and more effective was available?” Which is basically the same as asking people if they would like more money. The farmers say, “Listen, if you can build this, I’ll equip my whole herd.” But the problem was, he couldn’t build it. The risk in that case was in the product.

I’ve also seen this strike several of the recent companies who were trying to use mobile and realtime deals to drive foot traffic to bars and clubs. They run a bunch of customer conversations with bar owners who confirm that: yes, we would like more customers and if you could send us lots of customers, we will pay you. The founders take this as strong validation (“They have the problem and committed to pay!”) without recognising that the vast majority of the risk is in the product and being able to build the active engaged audience, not in the customer and can we get paid side of things. Bars will pay, but only if you can get tons of consumers on the door.

Video games are pure product risk. For example what sort of question could you ask to validate your game idea? “Do you like plumbers more than turtles?” Someone says, “Yeah, does that Mario’s gonna be a big success?” “Would you like to have even more fun than you’re already having in your leasure time?” In video games the way to deal risk the product risk is to build a prototype of the core game as quickly as possible, get it into people’s hands and see how much joy it brings them. You know people buy games. If yours is good and you can find a way to make them notice it, they’ll buy it. You don’t need to rediscover that.

This isn’t to say that you shouldn’t talk to anyone if you’re building something with product risk. In the case of the farm fertility monitor, it’s good to know that the farmer doesn’t have some reason to refuse to switch tech, for example. Maybe they’re all locked into 10 year contracts with their previous supplier. Or maybe there’s some issue with electronics that seems ridiculous from the outside but has prevented them from doing it in the past. You wanna know this stuff but it’s not gonna give you anywhere near enough certainty without accompanying it by actually building and hassling and growing the product.

What all this does mean is that if you’ve got heavy product risk (as opposed to pure market risk), then you just won’t be able to prove as much of your business through conversations alone. You’ll need to use both. The conversations give you a starting point, but you’ll have to build product earlier and with less certainty than if you had pure market risk. Pure market risk is exemplified in selling software to enterprises and businesses. They have a well-defined costly problem. They know what it looks like. They know what needs to happen. And they can tell you. If you can build it, they will pay you. In those cases, when the problem’s well-defined, you can learn with not perfect certainty but very good certainty whether or not it’s gonna work before you’ve ever written a line of code. So that’s why sometimes you get people who are saying, “Customer conversations were the key to my business” and other people saying, “Well, they were helpful but they were far from a magic bullet. It’s often because of the where the risk resides and their particular product.

Prepare your list of 3 big learning goals

Before you into a series of customer conversations you want to pre-plan the 3 most important things you want to learn from any given type of person.

For example the 3 things you most wanna learn from your customers, the 3 you’d like to learn from investors, from industry experts, from key hires, etc. As you learn more, you wanna update your list to reflect fresh questions about what you’re currently trying to learn. Pre-planning these big questions makes it a lot easier to ask your questions in a way which pass The Mom Test and isn’t biasing cause you had a chance to think about them in advance. It also makes it easier to face the scary questions, the ones that hurt the elephant in the room. When you through an unplanned conversation and impervise, it’s easy to get pulled into focusing on trivialities. And to keep the conversation fun and comfortable and easy. But that’s not necessarily what you wanna do. You shouldn’t make your customer feel uncomfortable, but you should be asking the stuff that matters. Decide on the tough questions a head of time in a calm environment with your team and then hold yourself accountable to asking them.

You’ll use different questions for each different type of person you’re talking to. For example you would ask customers different thing that you would ask partners.

Don’t stress too much about choosing the perfect questions. They’re gonna change. Just choose something that seems important for right now, a small list of important things. Answering those questions should give you firmer footing. It will help clarify your product features, your product road map, your marketing plan, your sales pipe line, your customer segment. You should choose things to learn which can meaningfully impact your business and the way you do things. And as you answer them, as you progress, it’ll give you a better sense of what you need to learn next. And then the following week you choose a new possibly overlaping set of 3 more questions.

You don’t need to go through the whole list with every customer. Foe example you might get a few answers from the first customer and the second kind of verifies that’s true and then when you talk to the third, you don’t even bother asking them those cause you already are confident on your answer. You move onto new stuff. There’s a bit of overlapping repetition but the time is valuable. And don’t feel obliged to push people through a massive laundry list of low-value questions.

Knowing your list of big 3, it allows you to take better advantage of serendipitous encounters. It’s so common to just randomly run into a dream customer, an industry event where ever. And instead of running into them and awkwardly asking to exchange business cards so you can “grab a coffee” (just like everyone else does), it’s really amazing to be able to just rattle off your most important question. It’s like, “Hey, wired question but how do you guys deal with this issue? I’m wondering.” That brings in the learning really quickly and it goes a long way toward keeping the conversation Casual. The advantages of which we’ll discuss momentarily.

Rule of thumb: You need a list of your 3 big questions.