Planning an MVP Pilot: Write the Learning Plan Before the Launch Date

Most MVP pilots are planned around a date. The build is scheduled to finish, the first users are lined up for the week after, and the plan ends there. What happens during the pilot, and what would count as a result, is left to be worked out once people are using it.

That order is backwards. A pilot is the first time the product meets people who did not help design it, and it is short. If nobody decided in advance what it was meant to establish, the weeks produce impressions rather than evidence, and the decision at the end gets made on whichever user was loudest.

A pilot needs a learning plan, not just a launch date. The plan has six parts: what the pilot should teach, who is in it, how they get started, what the boundaries are, how they get help, and how you hear from them. Then a decision, on a day chosen before the first invite goes out.

#What should the pilot teach you?

Write down two or three questions, no more. Each should be something the team is genuinely unsure of, and each should change what gets built next depending on the answer. For example:

• Will people who have this problem finish setting up without help?
• Do they come back in the second week without a reminder?
• Is the part we think is the core the part they actually use?

These usually descend from the question the MVP was scoped to answer in the first place, which is how we approach building an MVP without building the wrong thing. The pilot is where that question finally meets real use.

For each question, write down what result would mean carry on, what would mean change course, and what would mean stop. Do it now, before any data exists. Once people are using the product, almost every number can be read as encouraging, and a threshold chosen afterwards tends to land just below whatever happened.

Leave sign-up counts off the list. How many people joined measures how well they were invited, not whether the product works for them.

#Who to invite, and who to leave out

Invite people who have the problem now and are handling it some other way today: a spreadsheet, a shared inbox, a manual routine somebody resents. Their current workaround is the benchmark. The pilot is answering whether the product beats it, and they are the only people who can tell you.

Leave out friends, family, and investors. They are generous testers and unreliable evidence, because they want it to work. Leave out anyone whose main need is a feature that was deliberately cut; they will spend the pilot testing what is missing. And leave out anyone whose participation rests on a promise made to win them, because their feedback is about the promise.

Keep the group small enough that the team can speak to every person in it, and large enough that one person's habits are not mistaken for a pattern. Recruit a few more than you need. Some will never start, and that is a finding too.

#Onboarding is the first thing you are testing

The first session is where pilots lose people, and it is the part teams rehearse least, because the team no longer sees it. Everyone building the product set it up months ago.

Onboard the first few users yourselves, on a call, watching rather than steering. Note every point where they hesitate, ask a question, or do something nobody expected. Resist the urge to explain. Every explanation you give is one the product will not be there to give later.

Then fix what the first sessions exposed and let the next users set up on their own, with only what a real customer would receive. That second group tells you whether the fixes held. The measure worth tracking is the time from invitation to the first useful result, not the time to an account existing.

#Boundaries that protect the users and the result

Put the terms in writing before anyone starts, in plain language:

• How long the pilot runs, and the date it ends
• What the product does and does not yet do
• What data participants put in, where it is held, and what happens to it when the pilot ends
• Whether they can keep using the product afterwards, and on what terms
• How either side can end it early

The team needs a boundary too: no features built for one pilot user during the pilot. A request is valuable data. Building it midway changes the experiment, and the results no longer describe the product you set out to test. Log the request, and bring it to the decision.

#Support is a research channel with a response time

Give pilot users one channel, a stated response window, and somebody actually watching it. Then log every request with a note of what the person was trying to do when they got stuck.

That log is often more honest than any interview. People report what blocked them in the moment, not what they remember a fortnight later with the frustration worn off. Read it weekly and group it. The requests that repeat are the findings.

One caution. Pilot users usually get far more attention than paying customers will, and that attention quietly improves the results. Note which outcomes depended on the team stepping in, so the decision accounts for the help that will not scale.

#Watch what people do, then ask about it

Opinions about a product are unreliable; behaviour is less so. From the first day, record the handful of actions that map to the pilot's questions: setup completed, first useful result, return visits, the core action used or ignored. Tracking added in week three loses the weeks that mattered most.

Then hold short conversations at fixed points, one after the first week and one near the end, and ask about specific, recent use. "Walk me through the last time you used it." "What did you do instead, the week you did not?" Specific questions get specific answers.

Avoid "would you use this" and "what features should we add". The first is answered politely. The second produces a wish list, not a finding. One question worth asking everyone: if this disappeared tomorrow, what would you go back to?

#Decide on the day you fixed in advance

A pilot ends in one of three decisions, made against the thresholds written at the start.

Carry on. The questions came back the way the criteria said they should. Widen access, and expect the next stage to test the parts built for a small group. That is when first-release architecture gets its real test; we wrote about that decision in monolith or microservices for your first release.

Change course. Something specific was learned and points at a specific change. Make it, then run a second short pilot with a sharper question.

Stop. The product did not beat the workaround, and nothing in the logs suggests a change that would. This is a result, reached in weeks rather than after a year of building on the assumption.

The pilot that quietly gets extended because the results are unclear is the common failure. An extension without a new question is a delay. If the answer is ambiguous, the question was probably too loose, and the fix is to sharpen it, not to wait longer.

If a first launch is coming up and its plan currently ends at the launch date, this is the part worth settling now, while the questions can still shape who gets invited. Discuss the first launch with us before the first invite goes out.