Lotus Echoes

Lotus Echoes

An interactive installation that encourages participants to find calmness through physical movements

#physical-computing #creative-coding #machine-learning-gesture-recognition




Class Project


Katherine Liu

Kanchan Pathak


Particle Photon




ADXL362 accelerometer
Edge Impulse

Laser Cutting


Lotus Echoes is an interactive mindfulness installation that encourages participants to find calmness through physical movements. Using the floral controller, the player can interact with the lotus on screen and enjoy an immersive experience with soothing visuals and music.

My Contribution

Designed interaction flow

Programmed visual interaction using p5.js

Programmed Particle Photon in VS Code

Trained gesture recognition ML model

Produced movement music in GarageBand

Designed and fabricated the physical controller

Interaction Flow

The experience involves two primary interaction flows. The first captures raw accelerometer data from the controller, which influences the lotus's movements on the canvas, creating ripples in the water. The second flow involves processing the raw data through a machine-learning model to interpret specific player gestures.


As it took a long journey for our accelerometer data to arrive in the p5.js code, especially through the ML model, latency and asynchronism have been our biggest challenges for the project. We decided to send the raw data and gesture types separately. The raw data transmits faster and is used for real-time control of the lotus’ position, while the gesture types take a bit longer to arrive and will trigger non-time-sensitive interactions.

To smooth out abrupt changes and noise in raw accelerometer data, I used interpolation to create transitions between the current flower position and the target position.

I also implemented non-blocking delays using timers in both the microcontroller code and p5.js to minimize the chance that the longer task of gesture recognition blocks or delays the shorter task of transmitting/receiving raw data.

Physical Controller

Instead of a standard mechanical remote control, we decided to have an organic and gentle design of a floral-shaped controller, which can evoke a sense of calm and relaxation, aligning with the game's objective of promoting mindfulness and tranquility. I referenced this flower model and designed a base to store our Photon at the bottom and ergonomic handles that can fit comfortably in the hands.


Within the interaction, each movement direction correlates with unique musical melodies, enhancing the immersive experience.The pitch and tempo of the music vary with the lotus's movement direction, simulating the sensation of pluckingstrings. For instance, when players move the lotus up-left, the music's pitch rises and tempo slows down. I crafted fourdistinct melody categories using the Chinese traditional instrument Guzheng in GarageBand.

See Full Video