Red Teaming AI Applications with Prisma AIRS

We are accustomed to performing security tests on applications. We look for vulnerabilities, check APIs, permissions, infrastructure, and network exposure.

But what happens when the application itself is AI-based?

An AI system can be secure in terms of its infrastructure and code, yet still behave in ways we didn't intend.

Key AI Behavior Risks to Assess

For example, it's crucial to examine whether the model can be prompted to:

• Disregard its predefined instructions?
• Alter its behavior through Prompt Manipulation?
• Reveal information it shouldn't return?
• Unexpectely use tools, APIs, or data sources to which it's connected?
• Bypass guardrails defined within the application?

This is precisely where AI Red Teaming differs from traditional security testing.

The goal is not just to find weaknesses in the code or infrastructure, but to thoroughly examine the behavior of the AI system across a variety of attack scenarios.

AI Red Teaming with Prisma AIRS

With Prisma AIRS, the AI Red Teaming capability can profile the AI system, understand its context, and accordingly execute attack scenarios relevant to the system being tested. These tests can be performed against models, AI applications, and agents, examining a wide range of techniques and attack scenarios.

Upon completion of the testing, it's possible to understand which scenarios succeeded, what their risk level is, and to receive information that can assist in the remediation process.

This approach allows organizations to proactively try to make the system fail in a controlled manner before it goes into production, rather than waiting for problematic AI behavior to be discovered afterward.

The Growing Importance of AI Security

As AI systems gain more access to information, APIs, tools, and actions within the organizational environment, the potential impact of unexpected behavior grows proportionally. Therefore, just as we test applications, infrastructure, and code before production, we must also test the AI itself.