A future where artificial intelligence agents talk to each other is right around the corner. According to Anshuman Chhabra, an assistant professor in USF’s Bellini College of Artificial Intelligence, Cybersecurity and Computing, agentic AI is the next frontier for machine learning models.
These agents will be able to autonomously act on a user’s behalf, from writing emails and making dinner reservations to coding alongside a software developer. With agents interacting with other agents, there is a need for a better and more robust framework of engagement.
Chhabra’s recent paper, “Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges,” published in IEEE Access, outlines a taxonomy of threats from agentic AI as well as opportunities to improve trust and safety.
Chhabra wants to find ways to orchestrate better cooperation between models while ensuring that confidential data remains private and secure. As the leader of the Pioneering Advancements in Learning Methods (PALM) Lab in the Bellini College, he focuses on enhancing trust and safety of machine learning models and using these models in interdisciplinary applications.
AI trust and safety involve a goal of interpretability, meaning that a human can understand how and why a model made a specific decision or output.
“My work seeks to characterize the information flow in these AI models to understand what is driving them internally,” Chhabra said. “We want to see what the problematic behaviors are and how we can improve them by doing 'surgery' on the models.
Building trust in autonomous systems
The risk of agentic AI is higher than that of traditional chatbots without autonomy, especially in areas like medicine or cybersecurity. Agents, performing tasks on behalf of a user, can make mistakes such as accidentally misleading a doctor with regards to a diagnosis or deleting files in restricted codebases, leading to system failures.
“At this point in time, a user should assume that agents are more prone to failure in ways that humans might not be owing to the need for accountability in human society,” Chhabra said.
As agents become more integrated with society, they will ideally follow a trend in technological history and become safer as more effective guardrails are developed. Chhabra points to the seatbelt, which was introduced decades after cars were invented.
In the same way, he advocates a tiered governance framework for AI agents that considers risks such as hallucinations and reasoning failures. Under that approach, the most autonomous systems would face the greatest scrutiny, as fully autonomous AI is not yet ready for widespread use.
“We’re missing foolproof auditable systems with regard to these agents, understanding why they made a decision,” he said. “Agents should ask whether to proceed in high-stakes decisions. This is also why it is important to have open-source AI models, which can accelerate research progress on these fronts.”
Applying AI beyond security
Chhabra’s research has interdisciplinary applications. Much of machine learning can be used to improve modeling in other fields. He believes the onus is on computer scientists to bring the technology to other disciplines. As part of that, he collaborates with researchers around the country.
Examples include a flood depth prediction model, spurred by his arrival at USF during hurricane season and the sight of damage to communities. Using social media data, hydrological measurements and statistics, Chhabra and his team are building AI that can work in spatial and temporal domains to show real-time flood depth predictions.
Another project compressed AI models up to half their size without a significant performance reduction, with potential applications in small devices like AI-powered hearing aids.
“We work with USF faculty in communication sciences and disorders,” Chhabra said. “At a concert, a hearing aid should be operating differently than at home, or at a party during conversation. Hearing devices need to be adaptive to the listener’s communicative intent. Hearing aids don’t currently account for this variability.”
With faculty from the University of California, Davis, Chhabra and his students work on social media content moderation that can protect users on YouTube or TikTok from harmful content. To do that, they use models that can operate on videos at scale and provide a detailed rationale for why a decision about safety was made, while allocating compute power adaptively and efficiently depending on video complexity.
“Safety and innovation are complementary,” he said. “The way the models are designed with ever-increasing scale, it is harder for us to understand what is going on when the model makes a decision. We’ve had some success, but it still is a daunting research problem, and that’s what excites me the most.”