Corrigibility

Summary

Corrigibility is the property of an AI system accepting correction or shutdown from humans. A corrigible AI system is safer because it can be corrected if its behavior drifts from intentions. However, building corrigible superintelligences is non-trivial: a sufficiently powerful agent has incentives to resist correction to preserve its long-term goals.

Key points

• 01Naive approaches to corrigibility create incentive problems: an advanced AI might try to prevent shutdown.
• 02Shutdown-aware agents: models that learn to accept shutdown commands without trying to avoid them.
• 03Value learning vs. value locking: a corrigible agent needs to update its objectives while remaining aligned.
• 04Technical corrigibility remains an unsolved problem at high levels of AI capability.