The Objects and Actors Metaphor in Extended Reality

Understanding the world requires us to create mental models based on experience. In Extended Reality (XR), this becomes more challenging as digital and physical realities blend. Digital intelligence can transform inanimate objects, whilst entirely virtual characters and objects can seem real. This paper examines this new blended reality through the lens of human cognition and perception, arguing for a shift away from outdated user interface metaphors such as the desktop metaphor from the personal computer era, the cards and stacks of mobile devices, and the timeline streams of social media. Instead, an emerging “Objects and Actors" metaphor better aligns with advances in XR, Spatial Computing, and Artificial Intelligence (AI). We face a collective challenge to shape this paradigm shift responsibly, ensuring that design prioritizes human well-being over pure economic interests. Context-aware systems offer significant potential but require personal data. Who controls this data, and how is it used? We explore these concerns and propose guidelines inspired by Calm Technology to design tools that assist rather than overload humans. Additionally, we present an implementation (‘Reality2’) using semi-embodied sentient digital agents (‘Sentants’) on a distributed platform that prioritizes privacy, distributes computation and storage, and enhances the intuitiveness of human-technology interactions. This approach paves the way for tools that respect users’ autonomy whilst enriching their experiences in an increasingly blended world.