Methodology

How Vibrometer works

What we measure in your coding sessions, how we compare you with other people, and why the results can be trusted.

I. What a vibe check is

A vibe check describes how you work with a coding agent, measured from the Claude Code or Codex sessions you choose. We read the structure of each session: what you ask for, what the agent does (reading, editing, running tests, using git, starting subagents), what fails, when you step in, and how long and how much it all takes. From that we measure 34 habits.

Your vibe check leads with the three habits where you differ most from other people, and shows every habit with its percentile. Here is what each one describes.

Directing the agent

How you set work up and steer it.

Plans before coding: How often you agree on a plan before the agent starts editing.
Prompts defining done: How often your prompts say what finished looks like.
Prompts naming a file: How often your prompts point the agent at a specific file.
Words per prompt: How long your prompts are.
Vague follow-ups per 10 prompts: Quick follow-ups such as “fix this” that don’t say what they refer to.
Corrections: How often you tell the agent it got something wrong.
Interrupts per 10 prompts: How often you stop the agent while it’s working.
Agent steps per prompt: How much the agent does for each prompt you write.
Agent work per prompt: How long each prompt keeps the agent busy.
Saves lessons to CLAUDE.md or AGENTS.md: How often you or the agent write down lessons for next time.
Clears or compacts long contexts: Whether you clear or compact a long context yourself, before it fills up.

Checking its work

Whether the agent’s changes get looked at and tested.

Checks after edits: How often the agent’s edits are followed by a test, build or lint.
Reads before editing: How often the agent reads a file before changing it.
Edits between checks: How many edits pile up before something checks them.
Sessions that end verified: How often a coding session ends on passing checks or a commit.
Errors left unresolved: Errors still failing when a session ends.
Retry loops you step into: How often you step in when the agent keeps retrying the same failing thing.
Corrections that catch mistakes: How often your corrections catch a real mistake that the agent then fixes.
Avoidable rework: How much of the agent’s work redoes earlier work after you push back.

How you work

The shape of your sessions and your manner with the agent.

Minutes per session: How long your sessions run.
Subagents per session: How often you hand parts of a job to subagents.
Agent waits for you: How long the agent sits idle, waiting for your next message.
Agents working, not waiting: How much of the time your agents are working rather than waiting for you.
Prompts after 11pm: How much of your prompting happens late at night. For fun.
Pleases and thanks: How often you say please or thank you. For fun.

Scale and pace

How much you get through, and how many agents you keep going.

Hours coding with agents: How many hours you spent coding with agents.
Prompts per hour: How many prompts you send in an hour of coding.
Lines per hour: How many lines your agents write in an hour of your time.
Tokens per line written: How many tokens your agents spend for each line they write.
Tokens per verified session: How many tokens a session takes to reach a verified finish.
Sessions running at once: How many sessions you usually have open while you work.
Time with parallel sessions: How much of your time has two or more sessions running.
Most sessions at once: The most sessions you’ve run side by side.
Agents working at once: How many agents are working at any moment.

Your vibes wrapped (hours, tokens, cost at API list prices, busiest day) sits beside the vibe check for fun. It isn’t compared or ranked.

II. Percentiles

Each habit is shown as “higher than X% of people”: where you sit among the people on Vibrometer, not a grade. When few people have been measured on a habit, a reference distribution, drawn from published research on how people use coding agents and from pilot sessions, stands in for them. As more people share, the comparison shifts, step by step, to the real people who have joined.

Habits describe style. Higher isn’t better in itself: long prompts and short ones, one session or six, can all be good work. A few habits, such as checking the agent’s edits, have a direction in the coaching we give, and we test those directions against outcomes (see What we’re learning). Results from only a handful of prompts are marked provisional and aren’t ranked.

III. Your vibe type

Your vibe type is a short epithet for how you work, such as “The Midnight Foreman”, made of three to five traits, each with a strength and examples. An AI writes it after reading your measured habits and a scrubbed sample of your prompts: secrets, emails and links are removed and file paths cut to the file name in your browser before anything is sent. We don’t keep the sample, and our AI provider doesn’t use it to train models. We keep the vibe type it writes. If you choose Private: numbers only on the upload page (or run the CLI with --private), no prompts are sent and you get no vibe type.

The AI chooses the words, not the numbers. Each trait names the habits it rests on, and its strength (strong, clear or a hint) comes from your measured percentile on those habits, never from the AI’s guess. Before anyone sees it, the text is checked: it mustn’t quote your prompts or carry links or handles, and it is told to describe your style, never your project.

IV. The Olympiad

Habits show how you work; the Olympiad shows how well. Each challenge is a realistic job: a short ticket from a busy stakeholder, with the rules that matter found by asking them a few questions and from follow-ups that arrive partway through. You solve it with Claude Code or Codex, against the clock.

Your code is graded by hidden tests that run on our servers. The score is compared with what the newest model scores when it works alone from the same ticket, and the standings rank people by how far they led the agent past that. An attempt where the agent did the work with no real direction from you is graded as practice and not ranked.

V. How we keep it honest

Boards count only checked uploads. Habit boards list only conversations that our server measured itself. Results worked out in your browser alone, including private ones, are yours to see, share and compare in groups, but aren’t ranked.
Uploads are checked for authenticity. Tampered or synthetic transcripts are rejected, and a conversation can count for only one account.
On a challenge, the server issues it and keeps the clock. Your session transcript must show the work: the agent writing the code you submit, with you directing it.
The hidden tests decide the score. You never see them, so there is nothing to tune to.
Only your first attempt is ranked. You can practise a challenge again, but a memorised second run doesn’t count.

Nothing about a transcript is proof on its own, which is why skill rankings, and what employers see, come from the Olympiad rather than from habits.

VI. What we’re learning

The habits started from published guidance on working with coding agents: give the agent a way to check its work, plan before large changes, say what done means. We’d rather know than assume. With your opt-in, aggregate, de-identified numbers from uploads help us study which habits actually lead to better outcomes, and the scoring improves as the evidence comes in.

Changes are versioned. When the comparison is rebuilt, everyone’s results are re-scored together, so everyone on a board is always measured the same way. The privacy policy has the details.

VII. Your privacy

The analysis runs in your browser. In Standard mode, only your measurements and a scrubbed sample of prompts leave it, to write your vibe type. In Private mode (numbers only), nothing you wrote leaves it: you get your habits and percentiles but no vibe type, and you can’t be on the boards. Conversations leave it only when you send them to be on the boards or submit a challenge: they are redacted on your device first (home folders in file paths become ~, so /Users/yourname/app becomes ~/app), and by default our server measures them, keeps the measurements and the ID of each conversation, and deletes the transcripts. Share links carry only your handle and your habit measurements. The full detail, including what optional choices change, is in the privacy policy.