---------- Forwarded message ----------
From: Sean K. Murphy
Date: Wed, Aug 12, 2009 at 7:15 PM
Subject: [lsc] Re: proof that we're not (completely) crazy
To:
lean-startup-circle@googlegroups.com Great write-up, candid and very useful. I have inserted some comments
and suggestions after some of your remarks but it's a much appreciated
narrative of a customer development process.
At 07:23 PM 8/11/2009, Sean Fioritto wrote:
We just completed a few rounds of customer interviews and I thought
I'd share this experience with all of you.
Round 1: A Friend of a Friend of a Friend
Everywhere I look I see customers.
You are probably correct,it's just that your application only brings
sufficient value to compel change from their current approach for a
few.
Our first experiment was a round of customer interviews. Our first
list of 'customers' to interview were mostly friends of friends. We
didn't know where else to start, so we emailed our most connected
friends, described Allicator and asked, "know anyone who might want
this?" We got a list of about 20 and just started calling.
This is a great start and because these were friends of a friend they
are more likely to tell you what's wrong or what's missing, the
average stranger who is unenthusiastic may instead appear lukewarm
"that's nice" to minimize the length of the interview.
Conversation not Presentation
1. Are you a customer?
We asked questions like, "How busy are you on a scale of 1-10?" or "Do
you spend much of your day coordinating the activities of others?" If
we were talking with a 'customer' these questions were usually
followed by a lively conversation. People love to talk about their
(most important, overwhelming), problems.
I think it's most useful to ask questions that are truly qualifying
and have yes, no, or a numeric answer. For example the question "Do
you want to save money on your car insurance" is worthless because the
only real answer is yes. If you question fails the Microsoft Error
Message Test (the phrase ", you moron" can be appended to the end of
it without changing it's meaning) then it's a poor question. Ideally a
question would elicit symptoms from someone in agony (a lot of pain)
with a condition that you can ameliorate, improve, or fix. Some
examples for Allicator (your mileage may vary):
Is the nature of your work such that you work to a "to do" list or
action item checklist?
How many action items do you check off in an average workday? An
average work week?
How many people do you need to "touch base with" on a common
project in an average work week?
Do you have to manage a shared list of action items with more than
3 people? (3 might be a breakpoint you uncover from earlier
interviews, it might also be 5,10,...
Are you managing a set of more than a dozen action items due this
week? (this might be derived as a breakpoint from earlier interviews)
At a higher level it would seem that the more action items that need
to be cross-checked with more people the higher the pain or need for
your offering. Above some threshold of group size (3,5,6,...) making
the completion list visible to everyone would enable "peer pressure"
instead of one person having to follow up. Perhaps above a higher
threshold (12,24,50) this would breakdown.
2. Do we understand your problem?
For us this set of questions was frequently answered without prompting
or we were presented with opportunities to ask these questions in the
natural flow of the conversation. One question we asked was, "What
tools do you currently use to help you with "problem description"?"
Some other questions that I also find useful in this regard:
How are you solving the problem now.
How well is that working for you.
What's good about that solution that a new solution should also offer
What's missing that would encourage you to try something new if it was
also included.
It's as important to understand their perception of the tool or
methodology as the tool itself. Many people may use Microsoft Excel,
but in different ways, to solve this problem. I think it's also
important to distinguish between ad hoc (e.g. pencil and paper,
E-mail, Microsoft Office tools) that are not purpose built to address
the problem but are being deployed against it because they are handy,
and more customized solutions.
3. Does Allicator solve your problem?
This part of the conversation was the most like a presentation. The
trick here is to describe the very smallest set of features you think
are good enough to solve the core problem. There is a major temptation
to sell at this point, so be disciplined. We were trying to learn
whether our current feature roadmap to get to our MVP would be able to
generate any sales at all.
I think it's more in the nature of
"if you could do x" on top of your status quo, or "X and Y" would you
be interested in trying something. Or can we get an example of your
current problem representation that we can map into our solution and
have you take a look at it.
4. Bonus questions.
"Have you ever looked for a tool to solve this problem? What did you
search for?"
What else have you tried and abandoned?
"What is your job title?"
"Do you know anyone else that would be interested in Allicator?"
Who else do you know who has this challenge/issue/problem (especially
quoting dimensional aspects: e.g. number of checklist items, team
size, ...)
"Would you pay "a number high enough that we are uncomfortable asking
for" / month for Allicator?"
How much is this costing you now: time, errors (quality), material
cost, money, headcount
individual contributors will tend to think in terms of time savings
and error reduction
managers will tend to answer in terms of headcount
exec/business owners will tend to answer in terms of money - cost,
savings, revenue, profit
Hey wait, was that a customer or my shadow?
It took dozens of phone calls before we managed to talk with some customers.
Technically these are prospective customers or prospects, being
old-fashioned I like to reserve the word customer for people who have
actually paid for the product or service.
Round 2: Ditch Diggers
The few customers we talked to had little in common except for the
core problem we were solving. Two had very similar job titles, (let's
call them Ditch Diggers), so we ran a facebook ad with the job title
at the top of the ad, which was roughly, "Ditch Digger? Feeling spread
thin? Click here to complete a survey and tell us about it." Facebook
ads were the easiest because we could pick types of people -- we have
yet to create an effective adwords campaign. We offered $10 Amazon
gift cards to complete a 15 minute phone interview.
I would think that there would be specialty media, websites,
newsletters etc.. that target ditch diggers that might be less of a
shotgun blast than a Facebook ad. You should certainly ask the ditch
diggers that you uncover what websites, magazines, newsletters, blogs,
etc... they regularly read for information on their
profession/occupation....Your mileage may vary.
What followed next was absolutely amazing. When we talk to a Ditch
Digger it's like every response has an exclamation point. "Yes, that's
me exactly!", "I can't believe you're building a tool for this, thank
you!", "Here are 5 emails of other people that will want this!", "It's
only (number that was so high we had to force each other to
ask)/month? Great deal!"
We've since switched to a simple web survey just to get a larger
volume of data and so far it looks consistent with our phone calls.
Someone apparently pasted a link in a Ditch Digger newsletter a day
after we ran our ad. It's really surprising to talk to someone and
find out they heard about you through a newsletter the day after you
run an ad. Now we have a way to repeatably find and talk to (and later
market to) a decently large pool of people that have a high likelihood
of being a potential customer.
Finally
After all our struggles with adwords campaigns, split testing landing
pages and the dozens of awkward phone calls we're finally talking to
customers. Proof that we're not crazy.
Here is a quick rundown of the tactics we used and a few tips:
-- Start your interviews with whoever you know
You should also enlist their support to get referred to folks who fit
a target profile for a prospect (one that you may refine as you go).
Don't share this initial profile with everyone you know (since it's
likely to be incorrect or deficient) but do a rolling set of requests.
Also remember that if someone you know refers you to someone you
don't, you should close the loop to tell them how you thought it went
and ask what the FoaF's impression was, they may be more candid with
the person in the middle than with you and it's all good data to help
refine your approach.
-- Keep your questions true to your hypotheses -- it's an experiment
not a sales call.
The challenge here is to avoid going into "objection handling" mode
and stay in an "appreciative inquiry" frame of mind. There is a
temptation to act as if "I am from the future and things work better
there. I only hope I can get you to abandon your ignorance and embrace
what's coming." If you find yourself wishing for "smarter prospects"
remember that means that your presentation is inadequate.
-- Have a conversation, don't give a presentation
This can't be said often enough. I think as software engineers we are
used to being in complete control of a program or a computer, and the
improv and unpredictability of a conversation can be difficult to
accept. But it's the only way to really learn something in the early
going.
Next step for us is to run some ads on LinkedIn and split test some
pricing data -- we'll also be collecting emails for when we finally
get our product out the door. We have some catching up to do in
product development. As soon as possible we're going to get something
out there and start charging for it.
I would think there might be specialty media/websites/blogs that might
be more useful than LinkedIn/Facebook etc..
I plan to share as much as I can with this group whenever I can. I'm
happy to give more detail, (maybe off list depending on the detail),
so please ask. We've just had some success so I feel like a genius,
but I know that's not even close to true so I'm very, very open to
feedback on this entire process.
- Sean Fioritto
This was a great write-up, thank you very much for taking the time to share it.
Sean Murphy
www.skmurphy.com /
www.bootstrapperbreakfast.com
