Haytham sees what is there.
A compact vision model that answers questions about images accurately, quickly, and on a single GPU.
See the resultsVQAv2 pairs
81.3%
#1 of 9 models
MME
89.9%
#1 of 9 models
POPE
88.4%
Accuracy
Parameters
6.8B
#1 of 9 models
Peak memory
18.7 GB
#2 of 9 models
Forward latency
163 ms
One image, H100
Tested on five benchmarks.
Answers follow the image.
VQAv2 pairs check whether the answer changes when the image does.
| Model | Pair accuracy (%) |
|---|---|
| Haytham v3 | 81.3 |
| Clef 27B | 79.2 |
| Clef Omni | 78.2 |
| Clef Flash | 77.2 |
| Qwen3-VL-8B | 77.0 |
| MiniCPM-V-4.5 | 76.9 |
| Qwen3.5-9B | 76.5 |
| InternVL3.5-8B | 74.7 |
| Gemma-4-12B | 69.0 |
Broad perception and reasoning.
MME covers fourteen perception and reasoning skills.
| Model | Question accuracy (%) |
|---|---|
| Haytham v3 | 89.9 |
| Qwen3-VL-8B | 89.8 |
| Clef Omni | 89.6 |
| MiniCPM-V-4.5 | 89.2 |
| Clef 27B | 89.0 |
| InternVL3.5-8B | 88.1 |
| Qwen3.5-9B | 87.3 |
| Clef Flash | 86.6 |
| Gemma-4-12B | 77.8 |
Grounded in the image.
POPE checks whether the model reports objects that are not in the image.
| Model | Accuracy (%) |
|---|---|
| Qwen3-VL-8B | 89.2 |
| Clef Omni | 88.7 |
| Qwen3.5-9B | 88.7 |
| Haytham v3 | 88.4 |
| InternVL3.5-8B | 88.4 |
| Clef 27B | 87.9 |
| MiniCPM-V-4.5 | 87.2 |
| Clef Flash | 87.0 |
| Gemma-4-12B | 84.4 |
Right four times over.
NaturalBench asks four linked questions, and every answer has to be right.
| Model | Group accuracy (%) |
|---|---|
| Clef 27B | 40.7 |
| Haytham v3 | 40.0 |
| Clef Flash | 37.8 |
| Qwen3.5-9B | 37.0 |
| Clef Omni | 35.3 |
| Qwen3-VL-8B | 33.6 |
| MiniCPM-V-4.5 | 33.1 |
| Gemma-4-12B | 29.1 |
| InternVL3.5-8B | 28.3 |
Charts are the hard part.
HallusionBench uses charts and edited figures that trip up models leaning on language priors.
| Model | Question accuracy (%) |
|---|---|
| Clef 27B | 79.2 |
| Clef Flash | 78.4 |
| Gemma-4-12B | 73.7 |
| Qwen3-VL-8B | 73.1 |
| Clef Omni | 72.8 |
| Qwen3.5-9B | 72.3 |
| MiniCPM-V-4.5 | 72.2 |
| Haytham v3 | 71.4 |
| InternVL3.5-8B | 70.5 |
Charts get a specialist.
Haytham routes charts, maps and documents to a specialist model and lifts HallusionBench. Photos still answer in tens of milliseconds.
HallusionBench
51.5 ms per photo · 75.8 ms handed off · 512 image tokens, one H100, median
+5.6 pts from hand-off · 95% CI +3.3 to +7.9
| Benchmark | v3 | Hand-off | Change |
|---|---|---|---|
| POPE | 88.4 | 88.4 | 0.0 |
| VQAv2 pairs | 81.3 | 81.5 | +0.2 |
| NaturalBench | 40.0 | 40.1 | +0.1 |
| MME | 89.9 | 89.8 | −0.1 |
| Model | Accuracy (%) |
|---|---|
| Haytham v3 | 71.4 |
| Haytham with hand-off | 77.0 |
Small footprint. Fast answers.
Haytham runs on one GPU with modest memory and responds quickly to a single image.
All nine models, memory and latency
| Model | GB | Forward ms | End-to-end ms |
|---|---|---|---|
| InternVL3.5-8B | 18.1 | 141 | 175 |
| Haytham v3 | 18.7 | 163 | 239 |
| MiniCPM-V-4.5 | 20.6 | 127 | 225 |
| Qwen3.5-9B | 21.4 | 218 | 254 |
| Qwen3-VL-8B | 21.6 | 205 | 256 |
| Clef Flash | 21.8 | 211 | 289 |
| Gemma-4-12B | 24.2 | 44 | 64 |
| Clef 27B | 58.3 | 441 | 514 |
| Clef Omni | 64.0 | 101 | 156 |
Every model saw the same items on one H100 in bf16 at batch size 1, using our own prompt protocol, so results are not comparable to public leaderboards.