Artificial Analysis

@ArtificialAnlys

Independent analysis of AI

San Francisco

artificialanalysis.ai

回复转帖媒体文章

已置顶
Artificial Analysis
@ArtificialAnlys
2024年7月6日
Thanks for the support @AndrewYNg! Completely agree, faster token generation will become increasingly important as a greater proportion of output tokens are consumed by models, such as in multi-step agentic workflows, rather than being read by people.
Andrew Ng
@AndrewYNg
2024年7月6日
Shoutout to the team that built artificialanalysis.ai . Really neat site that benchmarks the speed of different LLM API providers to help developers pick which models to use. This nicely complements the LMSYS Chatbot Arena, Hugging Face open LLM leaderboards and Stanford's HELM
68
Artificial Analysis
@ArtificialAnlys
14小时
In AA-PersonalityBench, the top open weights models on average rank lower than the top proprietary models on truthfulness, pragmatism and structure, and higher on slop and summarizing the bottom line We compared the 10 open weights and 10 proprietary models measured with the
17
Artificial Analysis
@ArtificialAnlys
14小时
Knowing when to stop is hard for language models controlling a robot arm ➤ Claude Opus 5.5 completed the task in 23 minutes, and said so: "The crocodile looks fully inside now" ➤ It then spent 22 minutes pushing the crocodile back and forth, only to finish almost exactly
55
Artificial Analysis
@ArtificialAnlys
14小时
We’ve updated the six Artificial Analysis Capability Indexes to v2.0, aligning them with evaluations from the Artificial Analysis Intelligence Index v5.0 and placing greater emphasis on Non-Hallucination and Agentic Tool Use The Capability Indexes map tasks from O*NET
38
Artificial Analysis
@ArtificialAnlys
17小时
We’ve worked with the CritPt authors to finalize the CritPt v2 grading server and will be making this available to the community via Artificial Analysis’ API tomorrow. As part of this process the authors made robustness improvements to the grading server logic, which we have
27

Artificial Analysis

Artificial Analysis’s posts