FanRows is a browser-based system for movement-controlled, embodied sound interaction — developed as both a research platform and a creative tool for artists. Participants navigate and shape evolving sound environments through posture, movement, stillness, direction, and duration.
The clip shows a short excerpt from a live FanRows session, where changes in body movement become part of an evolving relationship between movement and sound.
Video processing and audio synthesis happen strictly on your device — no biometric data leaves your browser. FanRows does not identify or infer emotions, intentions, identity, health status, personality traits, or psychological states. Body movement is used only as a creative control signal for sound and visual interaction.
Many interactive audio systems are event-based: a gesture starts a sound, a hand movement activates a sample, or a pose switches an effect.
FanRows works differently. It interprets changing qualities of movement — speed, stability, direction, posture, and duration — and maps them across several layers of a sound environment. A shoulder angle might gradually bring a texture into the mix; body velocity might alter its character; stillness might allow other elements to become more prominent.
The mapping can be asymmetric. One movement does not need to correspond to one parameter, and left and right do not have to mirror each other. The intention is not to turn the body into a collection of virtual sliders, but to create relationships that can be explored, learned, revisited, and combined in different ways.
FanRows forms a body–sound feedback loop: movement changes the sound, the changed sound affects perception, and perception can influence what happens next in the body.
This relationship is central to the project. The participant is not simply operating an interface from the outside, but becomes part of an evolving interaction in which movement and sound affect one another over time.
Recurring movement–sound relationships can emerge through exploration. Some may become familiar and repeatable, while others remain open enough to invite new movement.
FanRows treats sound not as a fixed composition, but as a space that can be navigated and reshaped through movement.
A FanRows session such as Ambient combines several movement signals within one evolving sound world. Shoulder and elbow angles can reveal or reshape individual sound layers, while torso movement and body velocity influence other aspects of the overall sonic character. These relationships are not presented as a list of instructions. They are discovered through movement.
A particular posture may expose a texture. A repeated arm movement may return to a familiar sonic state. A different combination of joint angles may reveal another relationship. Slowing down or becoming still can change how the sound develops. The mappings remain stable enough to be explored and revisited without reducing the body to a set of one-to-one controls.
Other FanRows sessions can use the same underlying system with different sound material, movement mappings, and interaction qualities. The aim is not to define one universal gesture language, but to create distinct responsive sound environments with their own character and possibilities.
This also makes FanRows relevant to movement artists, dancers, and choreographers. Movement does not have to follow a predefined choreography or gesture vocabulary. A performer can improvise, repeat a motion, hold a position, or change direction and hear how those choices affect the surrounding sound. FanRows can therefore become a space for sound and movement improvisation, where movement influences sound and the changing sound can in turn affect the next movement.
For artists — FanRows offers a movement-responsive environment for performance, installation, ambient music, sound art, and audiovisual work: a space of relationships to design rather than a fixed sequence of musical events. Ambient producers and sound artists can explore how sound material becomes an interactive environment on Create Sound Spaces →
For movement artists, dancers and choreographers — FanRows creates a different relationship between movement and sound. Body tracking is not limited to recognizing poses or activating musical events; movement can influence the sonic environment throughout a performance. This opens possibilities for interactive sound for dance, movement-based sound, embodied performance, movement improvisation, and experimental choreography. The body–sound relationship itself can become material for exploration.
For researchers — FanRows provides a real-time environment for investigating embodied interaction and perception–action coupling: how does an unfamiliar movement–sound relationship start to feel predictable? Can repeated interaction produce stable patterns without explicit instruction? How does perception change when sound responds to the dynamics of the body?
Technically — FanRows combines real-time pose tracking with MediaPipe and browser-based audio processing with Web Audio and Tone.js, without requiring specialized motion-capture hardware. The webcam provides movement information from which FanRows derives signals such as joint angles, body velocity, direction, posture, lean, and stillness. These signals can then influence different properties of the sound environment, making interactive audio and body tracking for dance, performance and sound possible directly in the browser.
FanRows sits at the intersection of embodied interaction, cybernetic feedback systems, and non-symbolic interface design. Real-time body tracking is used not to classify a person, but to generate changing signals that participate in an interactive sound system. A shoulder angle is treated as a physical relationship that evolves over time rather than as a discrete command; stillness can be as relevant as movement.
Early studies focus on how stable interaction patterns, exploration, perceptual adaptation, and emergent dynamics develop through repeated interaction.
The technical structure and system architecture are documented here: View System Architecture →
FanRows — Continuous Embodied Audio Interaction
Next page About FanRows