9jaBench
The open, reproducible benchmark for how well AI understands Nigerian languages — translation (chrF) and knowledge (AfriMMLU) across Pidgin, Hausa, Yoruba, Igbo and Nigerian English. This site needs JavaScript to run interactively; the frozen results are available as a downloadable JSON.
9jaBench-Core v1.1 — top models by translation avg chrF, frozen 2026-06-16
| Rank | Model | Avg chrF | AfriMMLU % |
|---|
| 1 | Claude Opus 4.8 | 47.3 | 95.3 |
| 2 | Gemini 2.5 Flash | 46.5 | 62 |
| 3 | GPT-4o | 44.7 | 90 |
| 4 | DeepSeek-V3.2 | 43.8 | — |
| 5 | Llama-3.3-70B-Instruct | 40.2 | 77.3 |
Download the results data