#consumer electronics #gaming #human-machine interaction #learning #storytelling #interfaces #social science #machine learning #affective computing #ethics #industry #interactive #computer science #interfaces #learning #entertainment #art #artificial intelligence #design #creativity #history #technology #archives
Vision Simulation, 2025
To augment or expand sensory experiences requires a combination of sophisticated hardware and deep understanding of the neuroscience underlying human perception. Recent advances in XR pass-through technology have proven to be a powerful tool to explore how the brain perceives the world around us. Such tools hold great potential to impact how cognitive scientists can approach vision rehabilitation through the introduction of vision augmentation systems.
Working with a transdisciplinary team at the MIT Media Lab and the MIT McGovern Institute, I designed five real-time spatial frequency filters (Pierce et al., 2025)., each of which remove or enhance certain spatio-temporal frequencies of visual information, enabling wearers to appreciate new dimensions of perception in their experience. A tangible input mechanism enabled users to toggle between various vision simulations, altering their environment and perception of objects.
Vision Simulations
Built using the Varjo XR-3 headset, custom C++ and HLSL shaders, and a handcrafted controller, the system allows real-time switching between visual modes. High-resolution pass-through cameras capture the environment, translating it instantly into new perceptual states.
Museum Exhibitions
This project, showcased to hundreds of visitors at the MIT Museum, demonstrated the implications of vision simulations on well-being and vision rehabilitation, exploring the integration of multisensory AI within vision simulation systems.
Future work will expand this approach toward adaptive, personalized vision-augmentation systems that integrate physiological and affective feedback for clinical and creative applications.
Applications in XR
Participants wear a mixed-reality headset that transforms their view of the real world using five visual filters. Each filter alters perception—softening detail, highlighting edges, inverting color, or generating kaleidoscopic reflections. Visitors move freely, interacting with artifacts and light, discovering how filtering vision reshapes awareness.
Inspired by research in visual rehabilitation, this project reframes scientific methods for improving vision in children recovering from blindness as a creative encounter with altered perception. By filtering frequencies of light and texture, the system invites participants to notice how the brain reconstructs space, form, and meaning.
