AI Anything

The easiest AI job to automate is the one you can check cheaply

Estimate the cost of reviewing an answer before you estimate the cost of generating it.

By Ashley Kays Rabbitt · November 18, 2026

A team wants to automate two jobs. One converts a list of approved dates to a calendar format. The other decides which client should get a discount. Both can be described in a one-sentence prompt. Only the first has a quick, objective check against an existing source.

For the date task, prepare twelve example rows. Include an ambiguous date, a missing year and a deadline that crosses a time zone. Decide whether each should convert, ask for clarification or remain blank. Someone can compare the output with the answer key in a few minutes. For the discount task, possible answers depend on authority, current agreements and commercial judgment. A confident answer may cost more to review than the original work.

A batch of twenty accepted items
5 mingenerating drafts
80 minreview and correction
4.25 minper accepted item

Calculate the cost per accepted result

Use fictitious numbers. A batch produces twenty drafts in five minutes. Reviewing each takes three minutes; four require five extra minutes of correction. Total working time is 85 minutes, including generation. If all twenty are eventually accepted, that is 4.25 minutes per accepted item. Compare it with the current time to complete the same items to the same standard.

Anthropic’s agent evaluation guidance describes assessing results across trials and using graders that match the task. A team can use the same principle without a large evaluation system. Anthropic • Evaluating agent outcomes Pick the task with a clear output, an owner who can judge it, and a remedy for uncertain inputs. That is where a pilot can produce an answer you can trust enough to act on.