Codeably AI

A coding assistant passed the happy path. Give it the empty list

Teach a model what “finished” means with three tiny acceptance examples.

By Ashley Kays Rabbitt · November 18, 2026

Ask an AI coding assistant to write a function that returns the highest-scoring submission. With three ordinary numbers it will probably produce a plausible result. The interesting questions appear before you write more code: what should happen with a tie, with zero submissions, and with a missing score?

Write the behavior in plain language first. For a classroom scoreboard, two learners may share the top score. For an empty class, display “No scores yet.” A missing score means the work has not been graded, so it should not silently become zero. Those are product decisions the model could otherwise make on your behalf.

The empty-list acceptance test
TieReturn both high scorers
No submissionsShow No scores yet
Zero versus blankA zero is a graded score

Compare output with a small answer key

Input Expected behavior
Ari 8, Bea 10, Cam 10 Both Bea and Cam appear as leaders.
No submissions The empty-state message appears.
Ari ungraded, Bea 0 Bea is the only graded learner and remains eligible.

Give the assistant the rules, ask it to implement them, and run each example. If it sorts the list in place, decide whether changing the caller’s original list is acceptable. Add that expectation before asking for a revision.

Anthropic’s evaluation guide distinguishes what an agent says it accomplished from the outcome you can inspect. Here, the returned names and the unchanged input are cheap to verify. Anthropic • Evaluating agent outcomes Have a classmate invent one more case and predict its result. You have built a miniature specification that can outlast the conversation with the coding tool.