What it takes to control AI by design: human learning

Experts in government, academia, and practice are increasingly concerned about the need for human oversight in critical human–AI systems. At the same time, traditional control designs are proving inadequate to handle the complexities of new AI technologies. Incorporating insights from systems theory, we propose a robust framework that elucidates control at multiple levels and in multiple modes of operation, ensuring meaningful human control over the human–AI system. Our framework is built on continual human learning to match advances in machine learning. The human–AI system operates in two modes: stable and adaptive, which, in combination, enable the effective use of big data and the learning necessary for effective control and adaptation. Each system level and mode of operation requires a specific control-feedback loop, and all controls must be aligned for performance and values with the higher system level to provide human control over AI. Applying these ideas to a human–AI decision system for text classification in critical applications, we demonstrate how a method we call reciprocal human–machine learning can be designed to facilitate an adaptive mode and how oversight can be implemented in a stable mode. These designs yield high and consistent classification performance that is unbiased and closely aligned with human values. It ensures effective human learning, enabling humans to stay in the loop and stay in control. Our framework provides spadework for a model of control in critical AI decision systems operating in volatile environments, where humans continue to learn alongside the machine.