November 27, 2024

Advancing red teaming with people and AI

In this article:

The document discusses the importance of interacting with AI systems to understand their capabilities and potential risks.
It describes the concept of “red teaming”, where people or AI are used to explore a new system’s potential risks in a structured way.
OpenAI has been using red teaming for several years, starting with manual methods and then incorporating automated and mixed approaches.
The company has engaged external experts to test their DALL·E 2 image generation model and has joined other leading labs in a commitment to invest further in red teaming research.
The document also mentions that OpenAI is optimistic about using more powerful AI to scale the discovery of model mistakes, both for evaluating models and training them to be safer.
Finally, the document shares two papers on red teaming, including a white paper on how OpenAI engages external red teamers and a research study introducing a new method for automated red teaming.
Click the link to read more details

Contact Expert Request Form