A study by Vals AI indicates that multi-agent AI systems significantly increase costs while providing little to no improvement in performance compared to single agents.

Research from the evaluation company Vals AI suggests that deploying teams of AI agents yields negligible quality improvements over single agents. Testing GPT-6 Sol and Claude Opus 5.5 models, the study found that agent teams cost between 1.8 and 5.1 times more than individual agents while failing to provide consistent performance advantages.

Performance and Cost Analysis

In four comparisons conducted by Vals AI, only one instance showed a statistically significant improvement: GPT-6 Sol at medium reasoning, which saw a 7.3-point increase. At maximum reasoning levels, neither GPT-6 Sol nor Claude Opus 5.5 gained a performance advantage from a team setup. The findings indicate that the additional expenditure for agent teams is generally not justified, particularly when models are already operating at maximum compute capacity.

Expert Perspectives on Scaling

OpenAI researcher Noam Brown noted that multi-agent systems primarily improve speed rather than quality, with four agents solving tasks twice as fast at double the cost. Brown highlighted that the effectiveness of scaling depends on the task, noting that while web research and math parallelize well, creative tasks like writing a novel do not. OpenAI developer Eric Provencher cautioned against using agent swarms, citing a 'coordination tax' where the breakdown of communication between agents leads to wasted financial resources.