What potential vulnerability in AI systems could be exploited for malicious purposes?

Adversarial manipulation of data inputs refers to attackers deliberately crafting input data to mislead AI/ML models, causing them to make incorrect predictions or classifications. This is a well-known AI/ML vulnerability that can be exploited for attacks.

“Adversarial examples are crafted inputs to ML models that cause the model to make a mistake. Protecting against adversarial manipulation is critical for AI system security.”

— ISO/IEC 27001:2022, Annex A 8.19; ISO/IEC 27005:2022; NIST SP 800-53 Rev. 5