中
◀ Back to feed
HF Space · Joël Niklaus

编码 Harness 排行榜:同一开源模型换外壳,成绩差可等同换模型;mini-swe-agent 总体约 51.4%

Collected 2026-11-07 00:00 AI news #ai#tech
Full text

改进什么:大家常只比模型,忽略 Agent 外壳。怎么做:固定模型、换 9 种 harness,在 Terminal-Bench 2.1 / DeepSWE 等上跑约 2.8 万次试验。结论:最佳外壳依赖模型;mini-swe-agent 领先,Pi 接近且更省。对企业:选型时要把 Claude Code / Codex / 自研 harness 与模型一起评,而不是只看榜单模型名。

Source
Title
编码 Harness 排行榜:同一开源模型换外壳,成绩差可等同换模型;mini-swe-agent 总体约 51.4%
Author
—
Site
HF Space · Joël Niklaus
Published
—
Collected
2026-11-07 00:00 · 2026-11-07-00-ai.json
Link
https://huggingface.co/spaces/joelniklaus/harness-leaderboard
Open original ↗

Reposted. Will be removed on request. All rights belong to the original author.

More from this batch

00:00batch · 2026-11-0712 items
>

Ramp data shows AI spending stalling just as Anthropic nears its IPO

Ramp reports token spend grew only 1.5% across August–September while usage rose, as price cuts and cost controls cap vendor revenue ahead o…

TechCrunch
>

Anthropic's $1.5 trillion IPO hits resistance as investors balk at valuation

Investors push back on Anthropic's $1.5T IPO valuation amid tech selloff and missing October revenue in the S-1; a delay could shake the bro…

The Decoder
>

Frontier Radar #5:若 AGI 已近,为何大佬还在出走办「新实验室」?梳理 45 家 neolab 的六类技术赌注

专题把独立研究型 AI 公司称为 neolab,按「更好的学习方式 / 不只靠语言 / 独占真实数据 / 算力效率 / 自动化科研 / 不拼第一名也赚钱」六类赌注整理约 27 家详档加近 20 家简述,并点出英伟达几乎处处下注。适合做战略扫描:看清大厂产品化压力之外,资本在押哪条…

The Decoder
>

Stratechery Sharp Tech 上视频:美国选民如何投票「数据中心」、对大科技与 Google AI 回应意味着什么

Ben Thompson 把 Sharp Tech 首次做成视频,讨论中期选举里数据中心议题如何影响大科技叙事,以及 Google 对 AI 竞争的回应。偏宏观产业与资本开支政治风险,适合对照本周公投与抛售新闻。

Stratechery
>

After blocking Muse, Amazon now lets users text Alexa to help them shop

Amazon is rolling out texting with Alexa for Shopping so U.S. customers can search, compare and buy over SMS, after previously blocking Meta…

Amazon / TechCrunch
>

Visa Agentic Commerce Benchmark: almost half of AI shopping test runs fail

Visa's first agentic shopping benchmark scores 22 models at a 57.2% pass rate; 54% of failed runs still ended in a purchase.

Visa
>

Google Antigravity adds remote agents and voice mode

Antigravity v2.29 starts rolling out cloud remote agents for AI Pro/Ultra and Gemini Live voice mode, plus redesigned Automations and a nati…

Google Antigravity
>

Xiaohongshu's Mishu Personal Agent launches via Rokid AI glasses

Xiaohongshu frames Mishu as a Personal Agent and pairs it with Rokid glasses for progressive on-site sensing grounded in community notes.

界面新闻
>

China Mobile open-sources JoinAI-Agent engine

China Mobile released Jiutian Juzhi JoinAI-Agent, a multi-agent engine that extends MCP tools and A2A agents with zero-code workflows.

中国移动
>

Physicl raises $10M to make the digital world real enough for robots

Physicl's Studio turns high-fidelity 3D pipelines into synthetic data tooling for robot and world-model post-training.

TechCrunch
>

ToyBlocks: A controlled world for robot learning and evaluation

Ai2's first FoundationWorld is an open tabletop block-manipulation sim with automatic demos, metrics, and early sim-to-real transfer results…

Hugging Face / Ai2
>

Nace.AI 开源 Drex 决策模型族:输入状态 + 题型,一次前向给出各选项概率,不对文本「生成」

改进什么:很多路由/风控/分类只要校准概率,不必生成长文。怎么做:Drex Max(27B 多模态)、Drex v2.0(35B MoE/约 3B 激活)等,兼容 TypeSafe 风格 `/v1/systemone`,可本地 Python/llama.cpp/Ollama。许可…

Nace.AI