Pass Again Through The Heart: Gesture, Memory, and Food was a live-stream performance and workshop series, presented at the 2020 ISEA Symposium, an international symposium on electronic arts. The virtual workshops featured a guest speaker who cooked virtually alongside a host chef and moderator (Professor Immony Mén), whose movements were captured and animated and broadcast in real time. This project was a series of workshops that utilized technologies like photogrammetry, motion capture, videography, and audio recording to support transmedia storytelling centering around food as a cultural practice.

The aesthetic design of the digital environment was influenced by the cultural contexts of the recipes selected by guest speakers. The digital kitchen set was constructed from photogrammetry scans of raw ingredients drawn from the corresponding recipes, such as soy bottles, chicken, vegetables, etc.

The recording and streaming setup included both a physical set and a virtual kitchen, each equipped with virtual and live-action cameras. Motion data of the host was sent to a digital avatar via a Noitom Axis Neuron mocap suit. Multiple PCs were used to manage the setup, with one for the virtual set and cameras and another for live-action cameras and switching between feeds. The entire pipeline was created and managed in TouchDesigner.

Role: Touchdesigner & UnrealEngine Developer
Organization: Public Visualization Lab (OCADu)
Technology: Unreal Engine 4.25, OBS, Touchdesigner, Axis Neuron Motion Capture Suit