Safety and MLOps

AI red teaming

AI red teaming systematically searches for harmful, insecure, or unreliable model behavior.

Plan attacksTry them on the systemRecord failuresFix, then test againrepeat
AI red teaming systematically searches for harmful, insecure, or unreliable model behavior. Values shown in the worked visual are illustrative unless the labels state otherwise.

How it works

Testers construct adversarial cases around a threat model and record reproducible failures. Findings become mitigations and regression tests.

A concrete example

Illustrative: 4 testers run 1,200 cases and reproduce 37 unique failures.

Do not confuse it with Penetration testing

AI red teaming includes model behavior and social harms beyond conventional software vulnerabilities.

Study the full lesson