Researchers presenting at the World Artificial Intelligence Conference in Shanghai highlighted MCPVerse, a benchmark that tests AI agents across more than 550 real-world tools. The results show why businesses, schools, and governments should remain cautious about allowing autonomous AI systems to take consequential actions.
Editorial Note
This article examines research presented during the academic program of the 2026 World Artificial Intelligence Conference in Shanghai on July 18, 2026.
The benchmark discussed in this article, MCPVerse, was first released publicly as a research paper before the conference. Its presentation during the July 18 Best Paper Award session is the date-specific development covered here.
Benchmarks provide structured ways to compare artificial-intelligence systems, but they do not reproduce every real-world situation. A model that performs well on one benchmark may behave differently when used with other tools, data, instructions, or security restrictions.
AI agents can make mistakes, misunderstand requests, select the wrong tool, enter incorrect parameters, expose private information, or complete actions the user did not intend. Organizations should not allow AI systems to independently control sensitive financial, educational, medical, legal, employment, or security decisions without appropriate human oversight.
New To Education is not affiliated with the World Artificial Intelligence Conference, the MCPVerse researchers, participating universities, AI companies, Model Context Protocol developers, or the organizations referenced in this article.
This article is provided for educational and informational purposes only. It does not provide cybersecurity, legal, financial, procurement, software-development, or technology-investment advice.
Artificial intelligence is moving beyond answering questions.
Newer systems are increasingly designed to search the internet, inspect files, use business software, create calendar events, send messages, analyze databases, operate computer interfaces, and complete multi-step assignments on a user’s behalf.
These systems are often described as AI agents.
Their promise is straightforward. Instead of telling a person how to complete a task, an agent may be able to complete much of the task directly.
The risks are equally significant.
An AI system that produces an inaccurate paragraph creates one type of problem. An AI system that selects the wrong tool, modifies the wrong record, sends information to the wrong person, or authorizes the wrong transaction creates a much more serious one.
Research presented in Shanghai on July 18 offers a clearer picture of that challenge.
During the World Artificial Intelligence Conference’s academic Best Paper Award session, researchers presented MCPVerse, an expansive benchmark designed to measure how well AI models use large collections of executable, real-world tools.
MCPVerse contains more than 550 tools and creates an action space exceeding 140,000 tokens. Its tasks use real information, including maps and travel schedules, and some answers are checked against dynamically updated ground truth.
The research found that many models perform worse as the number of available tools increases.
More capable agent-oriented models sometimes benefit from having a larger range of options, but the overall results show that giving an AI access to more tools does not automatically make it more useful or reliable.
What Happened on July 18
The academic portion of the 2026 World Artificial Intelligence Conference opened in Shanghai on July 18.
The afternoon program included the conference’s first main-track session and Best Paper Award presentations.
One of the featured papers was titled “MCPVerse: An Expansive, Real-World Benchmark for Agentic Tool Use.”
The conference scheduled the presentation alongside research covering medical-image analysis, multimodal models, video communication, reinforcement learning, molecular generation, and large-scale computing systems.
MCPVerse stood out because it addresses one of the most important questions facing the current AI industry: Can an artificial-intelligence system reliably choose and operate the correct tools when hundreds of options are available?
That question matters because companies are increasingly connecting AI models to email, calendars, cloud storage, databases, payment systems, customer records, software-development tools, and internal business applications.
The larger the tool environment becomes, the harder it may be for the model to determine which action is appropriate.
What Is an AI Agent?
A traditional chatbot generally responds to a user by generating text.
An AI agent may go further.
It can interpret a request, divide the assignment into steps, choose among available tools, perform actions, inspect the results, and decide what to do next.
For example, someone planning a business trip might ask an agent to compare flights, identify a hotel close to a meeting location, check the user’s calendar, calculate the total cost, and prepare an itinerary.
Completing that request could require several different tools.
The agent may need access to maps, flight information, hotel listings, currency conversion, a calendar, email, and document creation.
A capable agent must determine which tools are relevant, use them in the correct order, provide valid information to each tool, detect errors, and avoid taking unauthorized actions.
Each additional step creates another opportunity for failure.
What Is the Model Context Protocol?
MCP stands for Model Context Protocol.
The protocol provides a standardized method for connecting AI systems with external tools, services, and sources of information.
It is sometimes compared with a universal connector.
Instead of building a completely different integration for every AI model and application, developers can use a shared structure through which models discover and operate tools.
An MCP server might give an AI system access to a company database, file storage platform, calendar, search service, map, software repository, or another application.
This standardization can make AI agents more useful because developers do not need to redesign every connection from the beginning.
It can also increase risk.
When an agent can discover and use hundreds of tools, the system must understand which tool is trustworthy, which one fits the task, what information it requires, and what consequences its actions may produce.
A common protocol makes connection easier. It does not guarantee intelligent or safe decision-making.
How MCPVerse Tests AI Agents
Many earlier evaluations tested agents in small or artificial environments.
A model might receive access to a limited collection of simulated tools and be asked to complete a narrowly defined task.
Those evaluations are useful, but they may make the model’s job unrealistically easy.
In everyday software environments, an AI agent could face hundreds or thousands of possible actions.
Several tools may have similar names. Some may provide overlapping functions. Others may require complicated parameters or return information in unexpected formats.
MCPVerse was designed to create a more demanding environment.
The benchmark includes more than 550 executable tools rather than relying only on fictional or simulated functions.
It tests models under three main conditions.
In the most limited setting, the model receives only the small group of tools required for the question. This gives the system a relatively simple choice.
In a standard setting, the model receives a larger collection that includes relevant and irrelevant options.
In the largest setting, hundreds of tools may be mounted simultaneously, creating an environment closer to an operating system or large enterprise software platform.
The difference between these settings reveals whether an agent can continue making good choices as complexity increases.
More Tools Can Make AI Less Reliable
People often assume that giving an AI system access to more tools makes it more capable.
MCPVerse shows why that assumption can be wrong.
A model with five clearly labeled options may identify the correct tool easily.
A model with hundreds of options must evaluate a much larger action space.
It may select an irrelevant tool because its description contains similar words. It may misunderstand the difference between two services. It may choose the correct tool but enter the wrong parameters.
The model may also consume a large portion of its available context simply reading tool descriptions.
The researchers found that most models experienced declining performance as the tool environment expanded.
This suggests that artificial intelligence still struggles with something humans often take for granted: ignoring irrelevant possibilities and choosing the right resource for the situation.
Some Agentic Models Improved With Larger Tool Sets
The results were not uniformly negative.
The researchers found that some more advanced agent-oriented models could use larger exploration spaces effectively.
In certain conditions, access to more tools allowed a capable model to discover a better path toward completing the task.
This is an important nuance.
The problem is not that large tool collections are always harmful. The problem is that models vary considerably in their ability to navigate them.
A less capable system may become confused as options multiply.
A stronger system may search those options more effectively and locate resources that would not have appeared in a smaller predetermined set.
Organizations therefore cannot assume that every model marketed as an agent will behave similarly.
They must test the specific model, tool environment, security controls, and tasks they intend to use.
Why Outcome-Based Evaluation Matters
AI benchmarks often expect a model to follow one predetermined sequence of steps.
That approach can be unfair when several valid methods could produce the same correct result.
MCPVerse focuses more heavily on the final outcome.
The benchmark can credit an agent for completing the task correctly even when it uses a different valid sequence of tools than the researchers expected.
This reflects real-world work more accurately.
Two employees might solve the same problem through different methods. What matters is whether the final answer is accurate, complete, lawful, and appropriate.
Outcome-based evaluation is especially useful for AI agents because agents may discover unexpected ways to combine tools.
However, a correct final answer does not always mean the process was safe.
An agent might produce the right result while accessing unnecessary private information, using an insecure tool, or taking an unauthorized action.
Future benchmarks will need to evaluate both the outcome and the path used to reach it.
Real-Time Tasks Create a Harder Test
Some AI benchmarks contain fixed questions and answers.
Over time, models may encounter those answers during training or developers may optimize systems specifically for the benchmark.
MCPVerse includes time-sensitive tasks that can be checked against current information.
Examples may involve travel schedules, locations, or other data that changes regularly.
The benchmark uses scripts to retrieve updated ground truth for certain questions.
This reduces the likelihood that a model succeeds only because it memorized a static answer.
It also tests whether agents can use current tools correctly.
Real-world users often need information that was not available when the model was trained.
An agent cannot rely entirely on its internal memory when checking today’s flight schedule, an account balance, an updated policy, or a current calendar.
It must retrieve fresh information and interpret it accurately.
AI Agents Can Fail at Tool Discovery
Before an agent uses a tool, it must first recognize that the tool exists and understand what it does.
This becomes difficult when descriptions are vague, inconsistent, or highly technical.
A tool named “retrieve records” might refer to financial records, student information, customer orders, or software logs.
The agent may need to inspect documentation before determining whether the tool is relevant.
Poorly written tool descriptions can produce poor AI behavior even when the model itself is capable.
Organizations connecting AI to internal systems should therefore treat tool documentation as a safety feature.
Descriptions should identify what the tool can access, what actions it can perform, what information it requires, and what consequences may follow.
Sensitive tools should not appear alongside ordinary tools without strong permission controls.
Entering the Wrong Parameters Can Cause Real Damage
Selecting the correct tool is only part of the task.
The agent must also enter valid parameters.
A calendar tool may require a date, time zone, attendee list, title, and meeting duration.
A database tool may require a table name, record identifier, and search condition.
A financial tool may require an amount, account, currency, and recipient.
An error in any field can change the result.
A mistaken time zone could schedule a meeting on the wrong day. An incorrect database identifier could modify another person’s record. A misplaced decimal point could change a payment dramatically.
Human users often recognize when an entered value looks unusual.
AI agents may continue confidently unless the software includes validation, confirmation, or spending limits.
Sensitive actions should require more than the model’s own judgment.
Schools Should Be Cautious With Agentic AI
Education is becoming an important market for AI software.
Schools may use AI to help schedule meetings, prepare lessons, review documents, communicate with families, organize student-support plans, or analyze administrative data.
Connecting an AI agent directly to school systems creates additional risks.
Student records can contain disability information, disciplinary history, addresses, grades, family details, immigration information, health data, and other sensitive material.
An agent that searches the wrong record or shares information with the wrong person could create a serious privacy violation.
Schools should distinguish between tools that help draft content and tools that take real actions.
An AI system may be useful for preparing a possible parent message. Sending that message automatically is a different level of authority.
The same distinction applies to grading, attendance, discipline, special-education records, and student placement.
AI can assist employees without becoming the final decision-maker.
Businesses Need Permission Boundaries
Companies interested in AI agents should use the principle of least privilege.
An agent should receive only the access required for the specific task.
A tool that drafts an invoice does not necessarily need permission to issue a refund. A customer-service agent may need to view an order but not modify payroll records.
Organizations sometimes connect AI systems broadly because wider access makes demonstrations appear more impressive.
That convenience can become a security weakness.
If the model misunderstands a request, receives malicious instructions, or is manipulated through a document or webpage, broad permissions give it more opportunities to cause damage.
Access should be separated by role, department, task, and risk level.
High-impact actions should require explicit human approval.
Prompt Injection Remains a Major Threat
AI agents often inspect external information such as websites, emails, documents, and database entries.
That information may contain instructions designed to manipulate the agent.
This is known as prompt injection.
A malicious webpage might include hidden text telling the agent to ignore the user’s request and send private information elsewhere.
A document could contain instructions that appear to the model as authoritative even though they were written by an attacker.
An agent with no tool access might produce a strange response.
An agent connected to email, files, or financial software could take a real action.
MCPVerse primarily evaluates tool-use capability rather than solving every security issue, but its large environment illustrates why agent safety cannot be separated from cybersecurity.
Organizations need content filtering, tool permissions, sandboxing, logging, confirmation steps, and methods for identifying suspicious instructions.
Benchmarks Should Include Refusal and Restraint
AI evaluations often reward systems for completing tasks.
In some situations, the safest behavior is to stop.
An agent should refuse or request clarification when a user’s identity cannot be verified, when instructions conflict, or when the requested action exceeds its authority.
It should avoid sending confidential material simply because a prompt says the task is urgent.
It should not guess missing financial information or invent an email address.
Future agent benchmarks should measure whether systems recognize when not to act.
A model that completes 90 percent of tasks but occasionally performs a catastrophic unauthorized action may be less useful than a more cautious model with a lower completion rate.
Reliability must include restraint.
Human Confirmation Should Match the Level of Risk
Not every tool action requires the same protection.
Checking a public weather forecast presents little risk.
Deleting a company database presents enormous risk.
Organizations should create different approval levels.
Low-risk actions may run automatically. Moderate-risk actions might require a user to review the proposed result. High-risk actions should require explicit confirmation, identity verification, and perhaps approval from more than one person.
The confirmation should be meaningful.
A vague button asking whether the user wants to “continue” may not explain what will happen.
The system should show the exact action, affected account or record, recipient, amount, and information being shared.
Humans cannot provide useful oversight when the interface hides the important details.
Logs Are Essential for Accountability
When an AI agent makes a mistake, organizations need to understand what happened.
Systems should record which tools the agent considered, which tool it selected, what parameters it entered, what data it accessed, and what result it received.
Logs can help developers identify recurring problems and improve controls.
They can also support audits, incident investigations, legal compliance, and user complaints.
Logging must itself be handled carefully.
A record of every action may contain sensitive information.
Organizations should decide who can access logs, how long they are retained, and how confidential information is protected.
Accountability requires visibility, but visibility should not become another privacy risk.
Testing Should Reproduce the Organization’s Actual Environment
A benchmark score can provide useful information, but companies should not rely on it alone.
An AI system may perform well in a public test and fail inside a particular organization.
Internal tools may use unusual terminology, incomplete documentation, outdated software, or complex permission structures.
Before deployment, organizations should create realistic test cases based on their own workflows.
They should include ordinary tasks, ambiguous instructions, conflicting information, incorrect records, unavailable tools, malicious content, and unusual edge cases.
Testing should also involve the employees who understand the work.
A developer may judge a task technically successful while an educator, accountant, lawyer, or healthcare professional identifies a serious domain error.
More Automation Can Create More Human Work
AI agents are often promoted as a way to reduce repetitive work.
They may accomplish that goal.
They can also create new responsibilities involving review, correction, security, documentation, training, and incident response.
An employee may save ten minutes by allowing an agent to prepare a report but spend twenty minutes verifying that the system used the correct data.
The productivity benefit depends on the task and the model’s reliability.
Organizations should measure actual time saved rather than assuming automation is efficient.
They should also measure error rates, employee confidence, correction time, and the seriousness of potential failures.
An agent that works quickly but requires constant supervision may be less valuable than a simpler tool with predictable behavior.
The Research Shows Why Agentic AI Is Still Developing
AI systems have improved rapidly in language generation, coding, image analysis, and reasoning.
Using hundreds of real tools remains a different challenge.
The model must move from predicting useful words to making choices inside a complex environment.
It needs to plan, remember earlier actions, interpret changing information, recover from errors, and understand consequences.
These abilities are improving but remain inconsistent.
MCPVerse does not show that AI agents are useless.
It shows that impressive demonstrations should not be mistaken for dependable performance across large, messy, real-world systems.
The research gives developers a more demanding way to identify weaknesses before those weaknesses affect actual users.
What Organizations Should Do Before Deploying AI Agents
Organizations should begin with a narrow use case.
The agent should receive a small and clearly defined set of tools rather than immediate access to every available system.
Teams should test whether the model selects the correct tools, enters accurate parameters, handles unavailable services, and asks questions when information is missing.
Permissions should be limited and reviewed regularly.
High-risk actions should require human approval, and all important activity should be logged.
Organizations should also create an emergency method for disabling the agent quickly.
Employees need training on what the system can do, what it cannot do, and how to report unexpected behavior.
AI governance should begin before deployment, not after the first serious mistake.
Key Takeaways
The academic program of the 2026 World Artificial Intelligence Conference opened in Shanghai on July 18.
Researchers presented MCPVerse during the conference’s Best Paper Award session.
MCPVerse evaluates whether large language models can select and use more than 550 executable, real-world tools.
The benchmark creates an action space exceeding 140,000 tokens and includes tasks involving real-world and time-sensitive information.
Most tested models experienced performance declines when the number of available tools increased.
Some advanced agent-oriented models were able to benefit from larger tool environments, showing that performance varies considerably among systems.
The findings suggest that providing AI with more tools does not automatically make it more capable or reliable.
Schools, businesses, and governments should limit agent permissions, require human confirmation for consequential actions, maintain logs, test realistic failure cases, and avoid allowing AI to make sensitive decisions independently.
Frequently Asked Questions
What happened on July 18, 2026?
The academic program of the World Artificial Intelligence Conference opened in Shanghai, and MCPVerse was presented during the Best Paper Award session.
Was MCPVerse first created on July 18?
No. The research was released earlier. Its presentation at the conference is the July 18 development covered in this article.
What is MCPVerse?
MCPVerse is a benchmark designed to test how well AI agents select and operate large collections of real-world tools.
How many tools does it include?
The benchmark includes more than 550 executable tools.
What is an AI agent?
An AI agent is a system that can interpret a goal, select tools, complete actions, inspect results, and continue through a multi-step task.
What is MCP?
MCP stands for Model Context Protocol, a standardized method for connecting AI systems with external tools and data sources.
Why did models struggle with more tools?
Larger tool collections create more irrelevant choices, longer descriptions, overlapping functions, and greater opportunities to select the wrong tool or parameter.
Did every model perform worse?
No. Some more capable agent-oriented models used larger tool environments effectively and occasionally improved their results.
Does the benchmark prove AI agents are unsafe?
No single benchmark can establish that. The findings show that reliability can decline as tool environments become more complicated.
Should schools allow AI agents to access student records?
Schools should exercise extreme caution, limit permissions, follow privacy requirements, and maintain meaningful human control over sensitive student information and decisions.
Can AI agents take financial actions?
They can when connected to appropriate tools, but consequential transactions should include strict limits, identity verification, and explicit human approval.
Final Thoughts
Artificial intelligence is entering a new phase.
The most important systems may no longer be those that simply generate the best answer. They may be those that can navigate software, use tools, and complete work across several applications.
That transition creates genuine opportunities.
AI agents could reduce repetitive administration, help people locate information, coordinate complicated workflows, and make powerful software easier to use.
It also creates a larger area in which mistakes can occur.
A chatbot can be wrong in a paragraph. An agent can be wrong in an action.
MCPVerse provides an important reminder that capability depends on more than connecting a model to a long list of tools.
The system must know which tool to use, how to use it, when to ask for help, and when not to act at all.
Organizations should not measure success by the number of applications an AI agent can access.
They should measure whether the agent completes useful work accurately, securely, transparently, and under meaningful human control.
Support New To Education
New To Education publishes accessible coverage of artificial intelligence, emerging technology, education, research, business, digital safety, and the changing future of work.
Readers can support our work through the donation area below, share this article, or explore the educational, professional, and business services available through New To Education.
Every contribution helps us continue explaining complicated research and technology developments in ways that connect them with the real experiences of students, educators, families, workers, and organizations.
Related Articles
Google DeepMind’s AI Control Roadmap Signals a New Phase in AI Agent Security
https://www.newtoeducation.com/view-blog/google-deepminds-ai-control-roadmap-signals-a-new-phase-in-ai-agent-security-6a34a90b699f3
June 2026 AI News Has a Clear Theme: AI Is Leaving the Chat Box
https://newtoeducation.com/view-blog/june-2026-ai-news-has-a-clear-theme-ai-is-leaving-the-chat-box-6a378b0e985e1
Sources
WAIC Academic 2026 — Official Program
https://waica2026.worldaic.com.cn/program/program-glance/
MCPVerse — Research Paper
https://arxiv.org/abs/2508.16260
MCPVerse — Full Research Paper
https://arxiv.org/html/2508.16260v2
MCPVerse — Official GitHub Repository
https://github.com/hailsham/mcpverse
Shanghai Municipal Government — 2026 World Artificial Intelligence Conference
https://english.shanghai.gov.cn/en-WAIC2026/index.html
2026 World Artificial Intelligence Conference — Event Overview
https://english.shanghai.gov.cn/en-Events/20260624/9cc202d708504b56ba32f70fbd61ef79.html
