AI can build your app. How do you know it solves a real problem?

An app can open, look polished, and pass its technical checks while leaving its intended customer with the same problem. AI can help you make a first version, but the important next question is whether that version makes a useful difference.

Separate the work into three questions: Does the problem matter? Does the product work? Does someone choose to use it? Each needs different evidence. Here is a practical way to collect it, using Wealth Experiment’s First Experiment Planner as a working example.

1. Find a problem worth understanding

Start with a recent situation, a specific person, and the workaround they already use. “Freelancers need productivity” is difficult to test. “Independent workshop instructors struggle to turn participant questions into a useful follow-up resource” gives you something to investigate.

Ask about the last time it happened: What were they trying to do? Where did the work slow down? What did they do instead? How often does it happen? What did it cost in time, missed work, or money? Ask to see a non-confidential example when appropriate.

Record what people actually describe. A suggestion from an AI assistant is a hypothesis until you check it. Three conversations can expose useful questions; they cannot establish the size of a market. Look for repeated situations and meaningful consequences before investing in more features.

2. Define an outcome before a feature list

Write one sentence: When [person] faces [situation], this first version helps them [observable outcome]. Then choose the smallest artifact that lets someone attempt that outcome. It might be a manual service, a document, a clickable prototype, or a small application.

The First Experiment Planner is a real example on this site. Its starting hypothesis was that a broad library of resources could leave a beginner unsure which action to take first. The first version uses three short steps to produce a downloadable plan: identify an audience and problem, choose a test, and set limits.

The published build record documents that scope and its implementation decisions. It does not report reader outcomes, sales, or allocated project costs. The useful distinction is between a tool we can inspect and a benefit we still need to demonstrate.

3. Give AI a brief it can be checked against

A clear brief makes review easier. Include the person, their task, the required output, the information the tool may use, and the conditions that count as completion. Describe what is outside this version, especially when a feature adds accounts, payments, sensitive information, or ongoing support.

For the planner, the immediate output is an editable experiment brief. Its current guidance uses prepared rules; visitors do not need an AI account. AI assistance during development and an AI-powered customer experience are separate choices.

Help me create the smallest useful version of this idea. First separate my evidence from my assumptions. Ask about the intended user and one outcome. Propose a limited scope, acceptance checks, and a user test. Identify missing information. Do not invent interviews, revenue, or results. Explain what needs human review before I share it.

Review the proposed scope before building. A convincing response is still a draft. If you cannot explain how a feature helps someone complete the task, set it aside for now.

4. Check the experience from beginning to end

Technical verification answers whether the agreed behavior works. Follow the complete user journey: arriving, understanding the promise, starting, correcting an error, finishing, and using the output afterward. Try a small screen and keyboard navigation as well as your usual setup.

For a planner, useful checks include blank answers, invalid numbers, going backward to edit, preserving input, exporting readable text, and handling unavailable storage. For a paid service, they might include a clear scope, a working request form, and a deliverable that matches the offer. Test the actual journey you intend to invite people into.

Ask AI to help identify edge cases, then verify what happens. A generated checklist is a plan for testing, and a test report needs observed evidence. Record the date, version, attempted task, result, and anything you could not check.

5. Watch someone use it without coaching them through it

Invite a small number of willing people who resemble the intended audience. Give them a realistic task and observe. Let them explain what they expect and where they hesitate. If they need help, record it; completing a task after substantial coaching is a different result from completing it independently.

For the First Experiment Planner, a proposed task is: “Use this to decide what you will test next week, then explain your next action.” Observe whether the person can finish, whether the plan is specific, and whether it fits their available time. A later conversation could ask whether they actually attempted the test.

These are proposed checks, not completed research findings. A pilot with a few readers helps you identify issues and refine your questions. Report its size and context, and avoid treating it as a population-wide success rate.

6. Test commitment separately from usability

“That looks nice,” completing a worksheet, returning to a tool, requesting a pilot, and paying for an offer tell you different things. Choose the behavior that answers your current question. A free resource may be useful without becoming a paid product; a paid offer needs evidence about the purchase decision and delivery costs.

Before running the test, write down the audience you will reach, the invitation, the action you will observe, and the review date. For example, you might invite five relevant people to try a sample and look for two requests to discuss a pilot. Those numbers are an illustrative decision rule, not a forecast or proof of demand.

Include no response, incomplete attempts, and objections in the record. If people never see the invitation, the result may say more about distribution than the product. If they try it and cannot finish, investigate usability. If they finish but see little value, revisit the problem or outcome.

7. Count the effort you will have to repeat

Track research, briefing, building, review, corrections, outreach, support, and maintenance. Record software and hosting costs with a reasonable allocation to this project. Keep measured effort separate from a budget you hoped to meet.

A tool that takes ten minutes to generate can still require substantial review and support. Your first cost record may be incomplete. Mark missing entries and improve the record before publishing a savings, productivity, or profit claim.

Use the workflow run log for actual attempts and the time-and-cost worksheet to organize the work. The purpose is to decide whether the next version deserves more of your time.

What the music-equipment example teaches us

The Modded Cycles project extends an existing musical instrument with community firmware. Its documentation distinguishes individual feature checks from the status of the combined release. That makes it a useful example of both discovering a specific unmet need and checking whether the complete experience holds together.

You can apply that discovery pattern to a small business workflow: listen for repeated workarounds, build a narrow improvement, and verify it in context. This example demonstrates an approach to building; it does not establish that your proposed tool has demand or will make money.

Choose what to do with the evidence

Continue when the observed behavior supports your current hypothesis and the next test fits your limits. Change when you can identify a specific obstacle, such as an unclear question, the wrong audience, or an offer that is too broad. Pause when the problem is weak, reaching buyers is impractical, or delivery effort exceeds what you can sustain.

Write one next decision, the evidence behind it, and what would change your mind. Keep the first version small enough that learning something inconvenient is affordable.

Sources and evidence

• Wealth Experiment: First Experiment Planner build record — an implementation record with unresolved questions clearly identified.
• The working First Experiment Planner — inspect the current experience.
• Modded Cycles repository — project scope and testing status; these can change as development continues.

Reviewed October 21, 2026. The exercises and suggested thresholds are teaching examples. No reader-study, revenue, or time-savings result is claimed here.

Continue the launch path and get the worksheets