AI search agents are increasingly being used to perform complex research tasks, but a new benchmark called DiscoBench reveals a critical limitation. The study, conducted by researchers at Tencent Hunyuan and Tsinghua University, found that these agents rarely fail due to their search capabilities but often struggle with ambiguous queries. Instead of seeking clarification from users, they tend to proceed with unclear instructions, leading to suboptimal results.

DiscoBench is a comprehensive test framework designed to evaluate how AI search agents handle ambiguity. It contains 211 tasks across eleven knowledge domains, such as video games, sports, music, film, science, and politics. Each task includes 463 ambiguous points, making it a robust tool for assessing the performance of search agents under complex and unclear conditions.

The researchers tested eleven models released in the past six months, including Claude Opus 4.7, Gemini 3 Flash, and others. Despite their advanced capabilities, even the most sophisticated models performed below 50 percent accuracy on tasks involving ambiguous queries. This suggests that the ability to handle ambiguity is a significant challenge for current AI search agents.

The inability of AI search agents to handle ambiguous queries effectively has broader implications for their practical use. Users may face inconsistent results, and businesses relying on these tools could encounter inefficiencies. Additionally, the lack of clarity in query handling may lead to increased costs and potential vendor lock-in as organizations seek more reliable solutions.

The findings from DiscoBench highlight a critical gap in the current capabilities of AI search agents. While progress has been made in search technologies, the challenge of ambiguity remains unresolved. This benchmark provides a clear framework for evaluating and improving these systems, ensuring they can better handle the complexities of real-world queries.