Why a pelican on a bicycle?
The task exercises code generation, spatial composition, character recognition, animation and responsive layout at once. It became a popular visual coding test for LLMs in 2025.
PELICAN TEST / LLM BENCHMARK
Pelican Test
Ask the model to draw a pelican on a bike, then make it debug. One HTML task, one acceptance routine: the agent writes code, runs static checks, probes the page in a real browser, repairs failures — and published runs are rated 0–5 stars by signed-in users.
- 1 file
- Single HTML file, everything inline
- 3 protocols
- OpenAI / Anthropic / Gemini compatible endpoints
- Signed-in ratings
- One vote per user per model
01 / WORKSPACE
Run the pelican test
Works with OpenAI Chat Completions, Anthropic Messages or native Gemini generateContent endpoints. Configuration stays in this tab.
RUN / AGENT
Agent workspace
Waiting for the first run
Fill in the configuration and press “Run the test”. Every write, check and debug step appears here live.
02 / RUBRIC
Rating & leaderboard rules
Automated checks only drive the agent's repairs; the public leaderboard is determined entirely by signed-in visitor ratings.
| Rule | What it means | Why |
|---|---|---|
| Signed-in only | Anonymous visitors can browse but not rate | Prevents ballot stuffing |
| One vote per user per model | The same account can submit one rating per model | Keeps samples independent |
| 0–5 stars | 0 stars means unusable; 5 means first-try perfect | Allows clear negative feedback |
| Average ordering | Models are sorted by average rating; ties break on rating count | More ratings are more trustworthy |
| Thinking is provider-specific | Providers use enable_thinking, reasoning_effort, thinking.budget_tokens or always-on reasoning. Cards label the effective mode: off / low / plan / provider-default | Prevents thinking-budget differences from being mistaken for model ability |
03 / FAQ
FAQ
How is the leaderboard ranked?
By each model's average star rating. Only signed-in visitors can rate, each account votes once per model, from 0 to 5 stars. The rating is a subjective judgement of one fixed task, not a measure of general model quality.
Do the automatic checks still score the page?
No. The eight automated checks only drive the agent's repairs and show an acceptance checklist; they no longer produce a 0–100 score and do not affect the leaderboard.
Is my API key stored?
No. It is used for the requests in your current tab, proxied once to your upstream, and never written to the database, logs or leaderboard.
Why debug in the browser?
Static checks cannot tell whether the animation actually runs or whether scripts throw. The agent renders the HTML in a sandboxed iframe, injects a probe, measures motion and overflow, then feeds the problems back to the model.
Can I test any model?
Any OpenAI Chat Completions, Anthropic Messages or native Gemini generateContent endpoint works. Use the model ID your upstream actually accepts.
KIMISEEK
StartRun the same task and see where your model lands.
Running the test needs no sign-up. After publishing, signed-in visitors can rate the model from 0 to 5 stars.
testing
This page is for technical evaluation. Use your own API key and follow your provider's terms.