FanRows is a browser-based system for movement-controlled, embodied sound interaction — developed as both a research platform and a creative tool for artists. Instead of triggering sounds with predefined gestures, participants navigate and reshape an evolving sound environment through posture, movement, stillness, direction, and duration.
The clip shows a short excerpt from a live FanRows session in which body movement continuously reshapes the sound space.
Video processing and audio synthesis happen strictly on your device — no biometric data leaves your browser. FanRows does not identify or infer emotions, intentions, identity, health status, personality traits, or psychological states. Body movement is used only as a creative control signal for sound and visual interaction.
You stand in front of a camera. You slowly raise your arm. The sound becomes brighter. You shift your body. A different layer begins to emerge. When you become still, the sound field stabilizes. When you move differently, it reorganizes.
In FanRows, movement does not simply trigger sounds — it continuously reshapes a responsive sound environment.
Most interactive audio systems are event-based: a gesture triggers a sound. A hand movement starts a sample. A pose activates an effect.
FanRows works differently. It continuously interprets qualities of movement — speed, stability, direction, posture, duration — and uses them to reshape several layers of sound at once. A shoulder angle might bring a layer into the mix; body velocity might shift its texture; stillness might let it settle.
The mapping is deliberately asymmetric — one movement doesn't need to correspond to one control, and left and right don't have to mirror each other. The intention is not to turn the body into a set of sliders, but to create a sound space that can be explored and revisited, where returning to a posture can mean rediscovering a sound rather than repeating a command.
FanRows operates as a continuous body–sound feedback system: body influences sound, sound influences perception, perception influences movement. This circular process is what distinguishes FanRows from systems where pose tracking simply replaces a button or a MIDI controller.
Over time, the interaction can develop self-stabilizing dynamics, recurring movement–sound relationships, and patterns that emerge through exploration rather than being prescribed in advance.
FanRows treats sound not as a fixed composition, but as a space that can be navigated and reshaped through movement.
A FanRows session such as Ambient combines several continuous movement signals within one evolving sound world. Shoulder and elbow angles can reveal or reshape individual sound layers, while torso movement and body velocity influence other aspects of the overall sonic character. These relationships are not presented as a list of instructions. They are discovered through movement.
A particular posture may expose a texture. A repeated arm movement may return to a familiar sonic state. A different combination of joint angles may reveal another relationship. Slowing down or becoming still can change the way the sound field develops. The mappings remain stable enough to be explored, learned, and revisited, but they do not reduce the body to a set of one-to-one controls.
Other FanRows sessions can use the same underlying system with different sound material, movement mappings, and interaction qualities. The aim is not to define one universal gesture language. It is to create distinct responsive sound environments that can be explored through the body.
This also makes FanRows relevant to movement artists, dancers and choreographers. Movement does not have to follow a predefined choreography or gesture vocabulary. A performer can improvise, repeat a motion, hold a position or change direction and hear how those choices reshape the sound. In this sense, FanRows can become a space for sound and movement improvisation, where movement influences sound and the changing sound can in turn affect the next movement.
For artists — FanRows offers a responsive, movement-controlled sound environment for performance, installation, ambient music, sound art and audiovisual work: a space of relationships to design, rather than a fixed sequence of events to trigger. Ambient producers and sound artists can explore how sound material becomes a movement-responsive environment on Create Sound Spaces →
For movement artists, dancers and choreographers — FanRows creates a different relationship between movement and sound. Instead of using body tracking only to recognize poses or trigger musical events, continuous movement can reshape the sound throughout a performance. This opens possibilities for interactive sound for dance, movement-based sound, embodied performance, movement improvisation and experimental choreography. The performer does not have to learn a fixed gesture language; the body–sound relationship itself can become material for exploration.
For researchers — FanRows provides a real-time environment for investigating embodied interaction and perception–action coupling: how does an unfamiliar movement–sound mapping start to feel predictable? Can repeated interaction produce stable patterns without explicit instruction?
Technically — FanRows combines real-time pose tracking with MediaPipe and browser-based audio processing with Web Audio and Tone.js, without requiring specialized motion-capture hardware. The webcam provides continuous movement information. FanRows derives signals such as joint angles, body velocity, direction, posture, lean, and stillness and maps them to changing properties of the sound environment. This makes movement-controlled, interactive audio possible directly in the browser and provides a lightweight approach to body tracking for dance, performance and sound.
FanRows is developed by Jürgen Berentz, a software architect and digital artist exploring new relationships between body movement, sound, and interactive systems.
The project combines software architecture, browser-based audio, real-time motion tracking, and embodied interaction in an independent experimental platform.
FanRows is being developed both as a creative system and as an environment for exploring how technology can respond to the body without reducing movement to a sequence of commands.
FanRows sits at the intersection of embodied interaction, cybernetic feedback systems, and non-symbolic interface design — using real-time body tracking not to classify a person, but to generate continuous signals that participate in an interactive sound system. A shoulder angle is treated as a changing physical relationship, not a discrete input; stillness can itself become meaningful.
Early studies focus on how stable interaction patterns, exploration, and emergent dynamics develop over time within this feedback loop.
The technical structure and system architecture are documented here: View System Architecture →
FanRows — Continuous Embodied Audio Interaction
Next page About FanRows