The Moral Compass as a Design Constraint: Ergonomic Principles for Building Safe AGI

The transition from narrow AI to Artificial General Intelligence (AGI) presents a shift that necessitates moving beyond purely algorithmic solutions toward a sociotechnical perspective. Our article argues that determining the moral compass of AGI is fundamentally a problem of cognitive compatibility and design constraints. We integrate established Human Factors and Ergonomics (HFE) principles, specifically Cognitive Work Analysis (CWA), Joint Cognitive Systems, and Distributed Situation Awareness, to address the process gulf arising from the opacity of AGI goal pursuit. We critically examine seminal AI safety thought experiments through the lens of automation surprises and mode error. Furthermore, we adapt Rasmussen’s Risk Management Framework to model AGI alignment as a hierarchical control problem, introducing the concept of Ecological Alignment Interfaces to visualise the safety envelope. Finally, we argue that AGI safety requires Bidirectional Human-AI Alignment and the rigorous, non-iterative safety protocols found in High Reliability Organisations (HROs). We conclude that the HFE discipline offers the essential toolkit for defining the design constraints necessary for safe AGI.Practitioner Summary:This article argues that Artificial General Intelligence (AGI) is a sociotechnical challenge requiring Human Factors expertise. This article demonstrates how practitioners can apply Cognitive Work Analysis and Systems Theoretic Accident Model and Processes to define the moral compass of AGI as a set of rigorous design constraints. We provide a framework for using Rasmussen’s Risk Management model to design governance structures that maintain human control over opaque, autonomous systems.