Question 1

What is Beckon Bench?

Accepted Answer

A benchmark for AI coding models built around eight one-shot tests run under identical conditions. Every prompt, artifact, score, judge vote, and ballot is published — the receipts are the product.

Question 2

Who decides the scores?

Accepted Answer

Two public verdicts: a cross-vendor AI judge panel (the Arena) and the People’s Vote. Human scores were issued through 2026-07-16 and stand as history; no new ones are issued since Ruling #6.

Question 3

How does the AI judge panel work?

Accepted Answer

Three judges from vendors with no model in the match see both artifacts blind, in randomized order, and must pick a winner. Majority decides, winners climb an ELO ladder, and every judge’s reasoning is published. A judge never evaluates its own vendor’s model.

Question 4

Can the People’s Vote be gamed?

Accepted Answer

Each ballot is one vote per person per match, deduplicated, and stored as an individual record — recountable, never opaque counters. Tallies are presented as indicative while anti-brigading protections are hardened, per the open item in Ruling #6.

Question 5

What does “one shot” mean?

Accepted Answer

The model’s first complete response is the scored artifact — no follow-ups, no retries, no error pasting. If the environment (not the model) fails, the run is voided and re-run once, disclosed in the rulings.

Question 6

Why do models run in different CLIs?

Accepted Answer

Each model runs in its own vendor’s flagship CLI at comparable settings, in the same environment, because vibe coders use each model through its native tool — the model-plus-harness pair is what’s being benchmarked.

Question 7

What do the time, token, and cost numbers mean?

Accepted Answer

Wall-clock time, output tokens, and API-equivalent cost are recorded for every run and published, but they never affect scores or verdicts. Cost is computed from output tokens at published API rates when runs happen on flat-rate subscriptions.

Question 8

Can I see the actual prompts and artifacts?

Accepted Answer

Yes — every prompt is published verbatim, every artifact is playable or watchable on its test page, and score records including revision notes are public in the repository.

Question 9

What is Beckon?

Accepted Answer

Beckon is the agentic workspace the bench runs inside — say the word, your agents build. Beckon Bench is its public benchmark.

How the Bench works.