LocalMode Bench
One protocol, five browser AI runtimes, real devices.
- Measures TTFT, decode throughput, and cold vs warm model loads for LLMs and embeddings.
- Runs the same weights family across WebLLM, wllama, Transformers.js (WebGPU and WASM), LiteRT, and Chrome Built-in AI.
- Every submission publishes its raw timing trace to the open dataset; every number here is recomputed from those traces.
Leaderboard
466 verified submissions under localmode-bench/5 and localmode-bench/4 (rows never mix versions; the previous protocol stays beside the current one because only the llama.cpp lanes changed) · medians of per-device medians · devices grouped by GPU model where the browser names one, otherwise by platform and WebGPU architecture · rows need 3+ submissions to leave provisional status
| Protocol | Device class | Model | Runtime | Workload | TTFT | Decode chars/s | Embed | Load cold/warm | Quality | Runs |
|---|---|---|---|---|---|---|---|---|---|---|
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (ONNX, WASM) | transformers-wasm (wasm) | embed-batch32 | — | — | 13/s | — / — | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (ONNX, WASM) | transformers-wasm (wasm) | embed-single | — | — | 135 ms | 3691 ms / 1171 ms | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (ONNX, WASM) | transformers-wasm (wasm) | quality-sts-100 | — | — | — | — / — | 0.940 | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (ONNX, WebGPU) | transformers-webgpu (webgpu) | embed-batch32 | — | — | 9/s | — / — | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (ONNX, WebGPU) | transformers-webgpu (webgpu) | embed-single | — | — | 110 ms | — / 3988 ms | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (ONNX, WebGPU) | transformers-webgpu (webgpu) | quality-sts-100 | — | — | — | — / — | 0.620 | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (GGUF Q8_0) | wllama (wasm) | embed-batch32 | — | — | 5/s | — / — | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (GGUF Q8_0) | wllama (wasm) | embed-single | — | — | 233 ms | 2965 ms / 674 ms | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | BGE Small EN (GGUF Q8_0) | wllama (wasm) | quality-sts-100 | — | — | — | — / — | 0.930 | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | SmolLM2 135M (GGUF Q4_K_M) | wllama (wasm) | chat-pp128-tg128 | 5166 ms | 66 | — | 6805 ms / 617 ms | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | SmolLM2 135M (GGUF Q4_K_M) | wllama (wasm) | quality-mmlu-25 | — | — | — | — / — | 0.320 (96% parsed) | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | Universal Sentence Encoder (MediaPipe) | mediapipe (wasm) | embed-batch32 | — | — | 20/s | — / — | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | Universal Sentence Encoder (MediaPipe) | mediapipe (wasm) | embed-single | — | — | 48 ms | 175 ms / — | — | 16 |
| v5 | android/adreno-650android/qualcomm-adreno-6xx | Universal Sentence Encoder (MediaPipe) | mediapipe (wasm) | quality-sts-100 | — | — | — | — / — | 0.780 | 16 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (ONNX, WASM) | transformers-wasm (wasm) | embed-batch32 | — | — | 24/s | — / — | — | 39 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (ONNX, WASM) | transformers-wasm (wasm) | embed-single | — | — | 45 ms | 2554 ms / 430 ms | — | 39 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (ONNX, WASM) | transformers-wasm (wasm) | quality-sts-100 | — | — | — | — / — | 0.940 | 31 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (ONNX, WebGPU) | transformers-webgpu (webgpu) | embed-batch32 | — | — | 60/s | — / — | — | 39 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (ONNX, WebGPU) | transformers-webgpu (webgpu) | embed-single | — | — | 19 ms | — / 1350 ms | — | 39 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (ONNX, WebGPU) | transformers-webgpu (webgpu) | quality-sts-100 | — | — | — | — / — | 0.940 | 31 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (GGUF Q8_0) | wllama (wasm) | embed-batch32 | — | — | 12/s | — / — | — | 39 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (GGUF Q8_0) | wllama (wasm) | embed-single | — | — | 82 ms | — / 557 ms | — | 39 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (GGUF Q8_0) | wllama (wasm) | quality-sts-100 | — | — | — | — / — | 0.930 | 31 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (GGUF WebGPU, Q8_0) | wllama-webgpu (webgpu) | embed-batch32 | — | — | 15/s | — / — | — | 39 |
| v5 | android/adreno-830android/qualcomm-adreno-8xx | BGE Small EN (GGUF WebGPU, Q8_0) | wllama-webgpu (webgpu) | embed-single | — | — | 76 ms | 1823 ms / 397 ms | — | 39 |
Recent submissions
What each device disclosed to the browser. The full capture (WebGPU adapter, WebGL, WebAssembly features, API availability, display, network) is in each run's JSON.
| Run | Suite | Device class | Device | GPU | Browser | OS | Cores | Memory | Storage quota | Isolation |
|---|---|---|---|---|---|---|---|---|---|---|
| ef25fe3c2026-11-05 | quick | windows/nvidia-geforce-gtx-1060-6gbwindows/nvidia-pascal | desktop | NVIDIA GeForce GTX 1060 6GBpascal | YaBrowser 26.10.0.1854Blink | Windows 19.0.0x86 | 2 | 8 GB+ | 10 GB | isolatedWebGPU |
| febbece72026-10-13 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 39f759082026-10-13 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 651a18d72026-10-13 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 40aaa83a2026-10-13 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 3b06e45a2026-10-13 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 094063b82026-10-13 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 1b5e99ab2026-10-13 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 42a9f6ba2026-10-12 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| eafd44482026-10-12 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 8f7f6b492026-10-12 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 27 GB | isolatedWebGPU |
| 514bf1702026-10-12 | thorough | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| 98c2bfe32026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| ad6798332026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| e047c7672026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| 8c2f7e7c2026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| b67d512b2026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| 7d2f41d02026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| e6f5ca6e2026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| 5ccb275f2026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| ca12eb1f2026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| 5c66b68f2026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| 992c1f5c2026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| 7ebff8142026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |
| 8317f52a2026-10-12 | quick | windows/nvidia-geforce-rtx-3080-tiwindows/nvidia-ampere | desktop | NVIDIA GeForce RTX 3080 Tiampere | Google Chrome 154.0.8037.58Blink | Windows 19.0.0x86 | 16 | 32 GB+ | 12 GB | isolatedWebGPU |