Cadence, an Immersive Media Exhibit

Interaction Design, Physical Product Design, Projection Mapping , Multi-Modal sensing

I designed cadence during my immersive design course. This project brings together the fundamentals of experience design to create an immersive interactive
Cadence was the most fun project I

Collaborators Aaditya Shete

Cadence, an Immersive Media Exhibit

Role: Creative Technologist, Fabricator, Sound Designer

Inspired by my avant-garde visions of interactive environments, I designed Cadence to dissolve the boundaries between spaces and their inhabitants.

In this living exhibit, your presence and movements coalesce with evolving visuals and soundscapes, creating a dynamic tapestry of emergent interaction.

Signal flow diagram

Concept sketch

Final set up

Goal

This was my first project in this domain, and my goal was to harness new technologies to create an immersive installation from conception to completion.

  1. Sketching and vision

  1. Learning the technologies

  1. Prototyping and testing

4. Showcase

Unclear affordances and emergent systems

Testing showed that the control gestures weren't obvious, but I refrained from displaying instructions— this led users to interact with the space in raw and interesting ways.

The empty exhibit pulsed with a latent energy that awakened the moment someone stepped inside, prompting further exploration.

  1. Activation on detection

  1. Changing scenes based on player count (up to 6)

Ideation and learning the ropes

For my initial idea, I wanted to create a space that creates a trail of disappearing noise behind you, inviting you to explore the themes of digital decay and revisitation.

Envisioned layout

Top down concept

I explored using stable diffusion for generating the trail, but quickly realised that the laptop we were given couldn't handle running such a big simulation in real time.

StreamDiffusion in Touch Designer

With my first time doing things, I had to get hands on and rapidly learn a bunch of supporting technologies. With Aditya assisting with the particle system, I was able to focus on directing the overall vision while getting my hands dirty with developing the soundscape and skeleton tracking integrations as well as the projection surface design and fabrication.

For my first venture in this field, I quickly mastered essential technologies. With Aditya handling the particle system, I focused on shaping the overall vision while actively developing the soundscape, skeleton tracking, and designing the projection surface.

  1. Skeleton tracking

  1. Projection Mapping

  1. Particle generation and physics

  1. Soundscape, scene, and interaction design

  1. Interfacing with Ableton (Click to unmute)

  1. Coding with my best friend

With my first time doing things, I had to get hands on and rapidly learn a bunch of supporting technologies.

Projection mapping

Fabrication and set-up

With all the different components ready, all we had to do was to put them together. This was my favorite part of the process.

Combined TouchDesigner file

Taping projectors down to limit misalignment

"DON'T BLOCK THE PROJECTOR"

Testing

Learning TouchDesigner from scratch was a challenging endevour.

With Aditya's assistance, I developed the touch designer file, which communicated with the peripheral tech to produce a reactive partical system.

2 projectors, kinect azure, touch designer, ableton

Reflections

Learning TouchDesigner from scratch was a challenging endevour.

With Aditya's assistance, I developed the touch designer file, which communicated with the peripheral tech to produce a reactive partical system.

2 projectors, kinect azure, touch designer, ableton

Learning things

Touch Designer, Projection Mapping,

I wanted to do something with stream diffusion, but settled for a particle system generated with noise and affected with physics based on the movement of the users

Playtest

I wanted to do something with stream diffusion, but settled for a particle system generated with noise and affected with physics based on the movement of the users

Development

Due to differences in direction and ambition, we decided to part ways and pursue our own projects.

This made me move toward a topic I have been thinking about for a long time, "living environments" and emergent experiences

What if the environment reacted to your movement?

Finally, I teamed up with Aditya to create Cadence.

My vision, sound design, and hands on skills, paired with Aaditya's programming skills

We demoed our concept to a bunch of different stakeholders, and received great appreciation and feedback.

Some of the feedback was that users couldn't understand what the change was/ how their movements affected the visuals/ music

Affordances

But overall, people felt immersed and very engaged and had a great time messing around

We built the projection surface using cardboard, foam, and spray paint.

We didn't have enough people to user test our creation since it was upto 6 people. Unfortunate.

Concepts Ideas

Learning things

Touch Designer, Projection Mapping,

I wanted to do something with stream diffusion, but settled for a particle system generated with noise and affected with physics based on the movement of the users

Playtest

I wanted to do something with stream diffusion, but settled for a particle system generated with noise and affected with physics based on the movement of the users

Final Build

Final Build

The final projection surface was built by assembling cardboard covered with white sheets. Some of it was planned, but mostly I winged it as I went

fig 01. Activation on detection