Run LocalMode Bench on your device

Pick a suite, keep the tab visible, and press Run. Models download once (they cache for next time), the protocol runs warmups and timed iterations per the published methodology, and when the run completes the result publishes automatically to the public leaderboard (switch publishing off to keep a run local; you can always export the raw JSON).

Configure the run

Suite

About 10 min with downloads: the SmolLM2 135M chat pairing and the embedding pairing. Runs on phones.

Adds tinyMMLU and STS-B scoring to check that each runtime keeps model accuracy.

Series

1 to 30 runs with these settings, each on a fresh page load.

min

Idle time after each run so the device can cool (0 to 30 min).

Each next run downloads its models again, for cold-start timings.

Set Runs above 1 (up to 30) to run a series with these settings, one fresh page load per run.

Publishing

The result uploads to the open dataset when the run completes.

Tools

Deletes the stored model files now; asks to confirm first.

Link presets

A link can prefill these controls, for example to send study participants the same settings. It never starts a run: a click on Run benchmark is always needed. On a study link with a completion code, the link fixes the suite, the quality-fidelity lane, Runs, the cool-down and clearing caches after each run.

  • tier=quick|standard|thorough: the suite
  • quality=on|off: the quality-fidelity lane
  • runs=N: runs in the series (1 to 30)
  • cooldown=M: cool-down between runs in minutes (0 to 30, rounded to the nearest half minute)
  • cold=on|off: clear caches after each run
  • publish=on|off: publish to the leaderboard
  • ccmode=full: on a study link with a completion code, the code is issued only for a finished, uploaded run, and an interrupted run restarts by itself (up to 3 times, and not after two interruptions in a row at the same step); kept in the link below when this page was opened with it
/bench/run?tier=quick&quality=off&runs=1&cooldown=0&cold=off&publish=on

Model lanes

SmolLM2 135M (MLC q0f16)webllm · 259 MB · q0f16unavailable
unavailable
SmolLM2 135M (GGUF Q4_K_M)wllama · 101 MB · Q4_K_Munavailable
unavailable
BGE Small EN (ONNX, WebGPU)transformers-webgpu · - · q8unavailable
unavailable
BGE Small EN (ONNX, WASM)transformers-wasm · - · q8unavailable
unavailable
BGE Small EN (GGUF Q8_0)wllama · 35 MB · Q8_0unavailable
unavailable
Universal Sentence Encoder (MediaPipe)mediapipe · -unavailable
unavailable
SmolLM2 135M (GGUF WebGPU, Q4_K_M)wllama-webgpu · 101 MB · Q4_K_Munavailable
unavailable
BGE Small EN (GGUF WebGPU, Q8_0)wllama-webgpu · 35 MB · Q8_0unavailable
unavailable

Probing device capabilities…

0 lanes · est. download - (cached models skip the download)

Models download only when you press Run. Keep this tab visible and your device plugged in - hidden tabs invalidate timed runs. With publishing on, the result uploads to the open dataset automatically when the run completes: timings, device environment, and the generated text for the fixed public prompts. No personal data. Turn the toggle off to keep the run local (JSON export only).