Measure whether an AI assistant actually saves you time

Pick one task and define finished

Choose a task you do often enough to compare: preparing a customer meeting brief, summarizing research you have already collected, or formatting a weekly business update. Keep its scope stable. Comparing a simple AI-assisted task with a complicated manual task will tell you very little.

Write the acceptance criteria before starting. A meeting brief might need an accurate company summary, three relevant questions, and links to the supplied evidence. Decide who will check the result and which errors would make it unacceptable. A fast result that fails the criteria does not count as completed work.

Measure comparable work using both methods

Collect several examples of the task and note their rough complexity. Assign comparable examples to your usual method and to AI assistance. Randomizing the order, where practical, reduces the chance that the easier cases or your freshest hours all favor one method. Do not redo the exact same task and assume the second attempt is a fair comparison; familiarity itself can save time.

Time the active work from preparation through final correction. Record elapsed time separately if waiting matters. Pause the active-work timer when you switch to unrelated work, and note interruptions. If several agents run simultaneously, avoid counting the same human minute multiple times.

Include the hidden minutes

Log setup, prompting, reviewing, correcting, and transferring the result into the system where it will actually be used. Include unsuccessful attempts. Keep initial setup separate from recurring effort so you can see both the first-week cost and the potential ongoing value.

In a hypothetical example, the usual task takes 45 minutes. AI preparation takes eight, generation needs two minutes of your attention, review takes twelve, and corrections take ten. Total active effort is 32 minutes, so the observed saving is 13 minutes per comparable completed task. If setup took 90 minutes, it would take about seven similar runs to recover that setup time, assuming the saving persists.

Prompt to try

Help me design a fair comparison for this recurring task: [task and acceptance criteria]. Propose a simple log with task complexity, method, setup time, active work, review, corrections, elapsed time, and quality outcome. Identify likely sources of unfairness. Do not predict a time saving. Separate one-time setup from recurring effort and define what counts as an acceptable completed result.

Check quality before celebrating speed

Use the same rubric for both methods. Review factual accuracy, completeness, usefulness, and avoidable rework. If another person can review without knowing the method, that can reduce expectations influencing the judgment. Record the defects that matter rather than treating every typo as equivalent to a wrong customer detail.

Keep the first results descriptive. A handful of personal tasks can guide your next choice, but it cannot establish how much AI helps everyone. METR’s research on experienced developers illustrates why measured productivity and expectations deserve separate attention; its 2026 update also describes selection and measurement problems that limit simple conclusions. Tools, people, and tasks all change.

Calculate the result you can defend

For each acceptable task, compare the complete active effort. Report the individual results and a median when a few unusually long cases distort the average. Preserve failed attempts in the total workload and report their frequency. Otherwise, the assistant gets credit for successes while you absorb its failures invisibly.

For cost, separate cash spent on tools from the value you assign to your time. Time returned is capacity, not cash income. It becomes commercially useful only if you apply it to valuable work or genuinely reduce an expense. Do not count the same recovered hour as both lower cost and extra revenue without a clear explanation.

Prompt to try

Analyze this task log: [log]. Check whether the tasks and quality standards are comparable. Calculate observed active time for each method, including failed attempts and corrections. Show setup separately, describe uncertainty, and flag missing data. Distinguish time returned, tool spending, and actual cash effects. Recommend continue, revise, or stop based only on this evidence.

Keep the useful part and change the rest

Look for the stage that helps. The assistant may accelerate preparation while making final writing harder, or produce a strong outline that still requires expert fact-checking. Keep that useful stage instead of forcing an all-or-nothing workflow.

Save the task definition, rubric, prompt version, and results. Repeat the comparison after a meaningful workflow or tool change, or when quality slips. A modest, measured saving on work you actually do is more valuable than an impressive productivity claim you cannot reproduce.

Try it on one real task

1. Choose one recurring task and write three acceptance criteria.
2. Log a small set of comparable tasks using both methods, including every correction.
3. Calculate the observed saving and choose where the recovered time will go.

Check before you use the result

• Both methods face the same quality standard.
• Setup, failed attempts, review, and corrections are counted.
• Active effort and elapsed time are separate.
• Time returned is not presented as guaranteed income.

Take the next step

Download the Wealth Experiment AI practice pack for these prompts, exercises, and review checklists.

New to AI? Take the free first-workflow course at AI Teachers. Already using it? Practice building a repeatable workflow.

Sources and further reading

• METR: Updating its developer productivity experiment design, February 2026
• Microsoft Learn: Monitor, measure, and report value

Sources support the principles discussed; the examples and suggested workflows are original teaching exercises.