AI Governance , AI Technologies , AI-Based Cyber Defense/Cyber Offense
Proof of Concept: Boosting Security and Taming AI 'Lies'
Troy Leach and Avani Desai on Risks of AI Hallucination and Misleading Outputs
In the latest "Proof of Concept," Troy Leach of the Cloud Security Alliance and Avani Desai of Schellman discuss the risks of AI hallucinations, As AI models advance, hallucinations pose serious threats to security, especially when quick and accurate decision-making is essential.
See Also: OnDemand: AI Model Security Challenges: Financial and Healthcare Data
Risks are growing as AI models develop more humanlike thinking and reasoning capabilities, Leach said. While AI may be able to understand what the prompter or software developer wants, it could provide the wrong answer "because it thinks it meets your objective better."
"Trust in AI systems … has to be built and maintained through a really rigorous process and continuous oversight," said Desai, pointing to the need for proactive strategies such as AI red teaming, to understand the vulnerabilities.
In this panel discussion, Anna Delaney, director, productions; Tom Field, vice president, editorial; Troy Leach, chief strategy officer, Cloud Security Alliance; and Avani Desai, CEO, Schellman - discussed:
- How AI hallucination could disrupt decision-making and cybersecurity operations;
- Strategies to detect and mitigate risks of "lying" AI models, including AI red teaming;
- The balance between leveraging "good AI" for threat detection and maintaining human oversight.
Leach has spent more than 25 years educating about and advocating for the advancement of responsible technology to improve the quality of living and parity for all. He sits on several advisory boards as an expert in information security and financial payments. Leach also founded a consulting practice that advises on the opportunities to leverage blockchain technology, zero trust methodology and various cloud services to create safe and trusted environments. Previously, he helped establish and lead the PCI Security Standards Council.
Desai has domestic and international experience in information security, operations, profit and loss, oversight, and marketing involving both startup and growth organizations. She has been featured in Forbes, CIO.com and The Wall Street Journal and is a sought-after speaker on a variety of emerging topics, including security, privacy, information security, technology trends and the rising number of young women involved in technology.
Don't miss our previous installments of "Proof of Concept", including the May 22 edition on ensuring AI compliance and security controls and the July 25 edition on how to outpace deepfake threats.
