#consumer electronics #gaming #human-machine interaction #learning #storytelling #interfaces #machine learning #affective computing #ethics #industry #interactive #computer science #interfaces #learning #entertainment #art #artificial intelligence #design #creativity #history #technology #archives
Return of the Jibo: Generative AI & Social Robots for Virtual Production Education
D. Pillis, MIT, USA
This paper explores the integration of Jibo, a social robot, with generative AI in film-making education. Deployed in college classrooms, Jibo supported creative workflows by helping students write scripts, storyboard, and act in AI-enhanced films.
The study documents increased student engagement, emotional bonding, and reduced anxiety, suggesting social robots can enhance AI-assisted learning in creative disciplines.
Over the course of two years, I developed collaborations with members of Lifelong Kindergarten and Personal Robots at the MIT Media Lab to create interventions across campus using Jibo. For example, some of these events programmed Jibo with Scratch to create performances and drag events for LGBTQIA+ students. Other interventions focused on deploying Jibo with other robots to create robot collaborations.
These projects included a Boston Dynamics Spot robot as well as an Aldebaran Nao robot in the public outreach events. By facilitating social interactions with robots and generative AI, these outreach events fostered communities of engagent with issues surrounding the introduction of social robots into our lives, as well as life with AI.
Images above show dozens of family and children interacting with Jibo, interactive AI visualizations, and other social robots.
Human Robot Interaction
Storytelling Robots
I collected public domain 2D animations comprising a geneology of CGI from 1963 to 1987, and used fragments of the animations to create a 3D environment for storytelling.
Jibo & Virtual Production
Human Robot Interaction
Users explored a chronology of CGI history by retargeting their hand gestures via a Leap Motion Controller onto Catmulls’ “hand”.
The dialogue of the story was told by two bookended iMac G3 monitors, each animated by Parke’s faces, re-animated with the original Apple text-to-speech engine, “MacinTalk”.
Jibo Performs
Using media artifacts with contemporary technology, the story of “The Machine Stops” was told through the medium of media archaeology.
A video recording of a popular performance by Jibo at the MIT Gender Fluid Dynamics Event in Spring of 2024.
Human Robot Interaction
Full Paper
At the 2025 MIT AI and Education Conference, I presented research on integrating Jibo, a social robot, with generative AI in film-making education. The project examined how Jibo supported students in writing, storyboarding, and performing in AI-assisted films, highlighting new possibilities for creativity and collaboration in the classroom.
