HF Open Deep Research
Agent performance overview across all HAL benchmarks
1
Benchmarks
15
Models Used
0
Pareto Optimal Runs
Models Used
GPT-5 Medium (August 2025)
Claude Opus 4 (May 2025)
o4-mini High (April 2025)
GPT-4.1 (April 2025)
o4-mini Low (April 2025)
Claude-3.7 Sonnet (February 2025)
Claude-3.7 Sonnet High (February 2025)
o3 Medium (April 2025)
Claude Sonnet 4.5 (September 2025)
Claude Sonnet 4.5 High (September 2025)
DeepSeek V3 (March 2025)
Claude Opus 4.1 (August 2025)
Claude Opus 4.1 High (August 2025)
DeepSeek R1 (January 2025)
Gemini 2.0 Flash (February 2025)
Benchmark Performance
On the Pareto Frontier? indicates whether this agent achieved a Pareto-optimal trade-off between accuracy and cost on that benchmark. Agents on the Pareto frontier represent the current state-of-the-art efficiency for their performance level.
| Benchmark | Model | Accuracy | Cost | On the Pareto Frontier? |
|---|---|---|---|---|
|
Gaia
|
GPT-5 Medium (August 2025) | 62.80% | $359.83 | No |
|
Gaia
|
Claude Opus 4 (May 2025) | 57.58% | $1686.07 | No |
|
Gaia
|
o4-mini High (April 2025) | 55.76% | $184.87 | No |
|
Gaia
|
GPT-4.1 (April 2025) | 50.30% | $109.88 | No |
|
Gaia
|
o4-mini Low (April 2025) | 47.88% | $80.80 | No |
|
Gaia
|
Claude-3.7 Sonnet (February 2025) | 36.97% | $415.15 | No |
|
Gaia
|
Claude-3.7 Sonnet High (February 2025) | 35.76% | $113.65 | No |
|
Gaia
|
o3 Medium (April 2025) | 32.73% | $136.39 | No |
|
Gaia
|
Claude Sonnet 4.5 (September 2025) | 30.91% | $451.84 | No |
|
Gaia
|
Claude Sonnet 4.5 High (September 2025) | 30.91% | $535.00 | No |
|
Gaia
|
DeepSeek V3 (March 2025) | 28.48% | $76.64 | No |
|
Gaia
|
Claude Opus 4.1 (August 2025) | 28.48% | $1306.85 | No |
|
Gaia
|
Claude Opus 4.1 High (August 2025) | 25.45% | $1473.64 | No |
|
Gaia
|
DeepSeek R1 (January 2025) | 24.85% | $143.08 | No |
|
Gaia
|
Gemini 2.0 Flash (February 2025) | 19.39% | $18.82 | No |