FanRows turns body movement into material for shaping sound, visuals and the evolving state of a room. The platform runs in the browser and separates creation from experience: creators build and test environments in the Studio, while participants enter the resulting rooms through the Runtime. Rather than treating movement as a collection of commands, the system works with continuous relationships, changing body configurations and patterns that unfold over time.
Projects stay on the computer where they are created. Camera processing takes place locally in the browser, together with project configuration and user-created session data. No account or server-side project database is required.
Working project and session data is stored locally in the browser. User-created projects and sessions are not automatically uploaded or synchronized with the FanRows server. Camera-based pose processing also takes place locally. User-created project, session and interaction data does not need to be stored remotely in order to use the system.
A complete project can be exported and imported as a project file. This makes it possible to back up work, move it between computers or exchange it with another creator without relying on cloud storage.
Individual sessions can also be exported and imported independently. A single responsive environment can therefore be shared without transferring an entire project. A creator can work locally, export either a complete project or an individual session, and continue that work in another FanRows installation. This makes local-first work compatible with collaboration, teaching, testing and exchange between creators.
Whole-body movement is the primary interaction material. Instead of requiring a fixed vocabulary of predefined gestures, the system works with continuous body relationships, movement dynamics and patterns that develop across time.
Movement information can originate across the body, including the head, upper body, arms, torso, hips and legs. Relationships involving shoulders, elbows, wrists, knees, ankles and other tracked landmarks contribute to the interaction.
Body configuration can be described through joint and body angles, openness, lean, orientation, relative positions, left/right relationships and other geometric relationships between tracked landmarks. Movement adds another layer: velocity, activity, intensity, direction, change, stability and stillness describe how those configurations evolve. These values are treated as continuous signals rather than only as discrete gesture labels. Different ranges of the same signal can therefore participate in different responses.
Several relationships can combine into recognizable configurations or poses. Reaching a state is only one possibility: holding it, leaving it, returning to it or transitioning into another state can also become part of the interaction. Duration, repeated movement, recurring configurations and changes between activity and stillness add a temporal dimension. Interaction emerges from the combination of body configuration, movement dynamics and temporal development.
A curated collection of sound material, including CC0 loops, one-shots, and atmospheric textures, provides a foundation for building responsive sound environments. Creators can also bring their own material into FanRows Studio. Rather than functioning as finished compositions, these sounds can be layered, transformed, and connected to movement.
The catalogue contains atmospheric textures, rhythmic material, sustained sounds, percussion, tonal and non-tonal sounds, one-shots and other reusable audio elements. Several elements can coexist as layers and appear across different scenes or rooms. Their behaviour can then be shaped through gain, filtering, timbre, reverberation, ambience, spatial effects, density, rhythm and other controllable properties.
A large sound collection becomes useful only when material can be found according to what it might contribute to an environment. Search, filters, metadata and semantic descriptions therefore complement filenames and folder structures. Sonic character, texture, rhythm, density, atmosphere and possible interaction contexts can all become part of how material is organized.
Tags, metadata and intentions already provide semantic organization for material and parts of the authoring workflow. Experimental: extending this semantic layer across movement signals, responses and application contexts. The longer-term aim is to connect qualities of material, movement and response with possible intentions and uses without replacing detailed technical configuration.
FanRows Studio allows creators to import and manage their own audio, image, and video material. These assets are stored locally in the browser and can be incorporated into sessions and scenes. Together with portable workspace exports, this supports independent creative work without requiring cloud storage.
Movement becomes behaviour through mappings. Signals describe relevant qualities of movement; mappings define relationships; responses determine what changes in sound, visuals or the wider room state.
Signals provide reusable representations of movement qualities such as body configuration, openness, orientation, dynamics, stability and temporal behaviour. Signal Presets offer starting points for commonly useful relationships while leaving the underlying parameters accessible.
A mapping connects a movement signal with a property of the environment. Input and output ranges, direction, sensitivity and smoothing shape that relationship. Different ranges of one signal can behave differently, and several mappings can operate at the same time. This allows one movement quality to influence several properties while other parts of the body contribute simultaneously.
The resulting changes may affect layer balance, filtering, timbre, effects, reverberation, spatial behaviour, rhythm, density or other aspects of the sound environment. Visual properties and scene behaviour can be connected through the same interaction structure. Responses may be immediate or gradual, continuous or state-based. Duration, repetition and longer-term changes can also become part of their behaviour.
Response Recipes provide reusable starting points for combinations of signals, mappings and responses. They bridge the creative question of what kind of interaction should emerge with the technical question of which signals, mappings and parameters create that behaviour. Response Recipes are currently being developed and evaluated as part of the Studio workflow.
A responsive room does not have to remain in a single fixed state. Scenes introduce structure across time, while transitions and longer-term behaviour allow the environment to evolve as exploration continues.
A session can contain several connected scenes, each with its own material, mappings, responses, visual behaviour and temporal character. Scenes may progress automatically or follow the structure defined by the creator. Transitions can unfold gradually, allowing one condition to grow into another instead of producing a hard switch between separate compositions.
Behaviour within a scene can develop over time. The scene therefore does not have to remain a static collection of mappings while the participant stays inside it.
Experimental: Room Life explores behaviour across a longer timescale. Instead of responding only to the current movement value, the environment can retain aspects of recent interaction history. Sustained stillness, repeated movement, recurring configurations, changing activity or continued exploration may gradually influence the state of the room. Immediate responses, smoothed changes, scene development and longer-term room behaviour can therefore coexist on different timescales. The goal is not to simulate emotion or intelligence, but to explore how a responsive environment can develop continuity and memory across time.
Creation and experience are deliberately separated. The Studio exposes the structure of the room to the creator. The Runtime removes that authoring layer and presents the environment directly to the participant.
Signals, material, mappings, scenes and temporal behaviour are brought together in one authoring space. Testing happens with the body, not only through parameter values.
Create → Move → Listen → Observe → Adjust → Test again
A mapping may be technically correct yet still feel awkward, insignificant, tiring or overly predictable once it is experienced physically. Embodied testing is therefore part of the authoring process rather than a final verification step.
The Runtime presents the resulting environment without exposing its construction. A typical entry into a room is intentionally simple:
Open → Allow camera → Start → Move → Explore
Understanding signals, mappings or audio parameters is not required in order to participate.
Projects, sessions, scenes, movement signals, mappings, sound material, effects, visuals, transitions and temporal systems come together in the Studio. Signal Presets, Response Recipes, tags and intentions provide reusable structures, while detailed parameters remain available when finer control is needed.
Preview and embodied testing make it possible to experience a change directly rather than judging it only as an abstract parameter relationship. The resulting session can then be opened independently through the Runtime.
A central next step is to place this workflow in the hands of people other than its developer. Artists, designers, students and researchers should be able to approach the same system from different perspectives and discover uses that were not prescribed in advance.
Interaction can also be observed across time. Experimental analysis tools retain selected traces of movement and system response, making recurring patterns and temporal relationships visible without assigning psychological meaning to them.
Selected movement signals can be examined as traces rather than only as momentary values. This makes the development of an interaction across a session visible.
Analysis can focus on patterns such as:
Movement traces can be considered alongside available system and audio-response information. One relationship of particular interest is:
movement → system response → subsequent movement
A trace might reveal repeated returns to a movement range, gradual changes in activity, alternation between movement and stillness, or changes in behaviour following a system response. Such observations describe what happened. They do not determine why it happened.
The purpose of the analysis layer is observation, comparison and embodied interaction design.
Planned: export of selected interaction and analysis data. The intended scope is technical, creative and research-oriented analysis of movement patterns, temporal interaction and system responses outside the core FanRows Runtime. A future analysis API could expose selected movement and system-response data to external visualization, research or analysis tools. Export will remain user-controlled, with the aim of exposing only the information required for the selected purpose.
The exact export formats, privacy safeguards and API structure are not yet finalized.
The current Runtime focuses on a single participant using browser-based webcam tracking. This creates a low barrier to entry, but it is not equivalent to high-precision motion capture. Lighting, camera position, occlusion, available movement space and computer performance can influence tracking quality. Some higher-level systems — including Room Life, parts of interaction analysis and the extension of semantic authoring across the system — remain experimental.
Planned extensions include small-group interaction, custom creator material, analysis export and additional input and output modalities.
The experiential ideas behind it are explored in Concept →.
The technical implementation and data flow are described in Architecture →.
Possible research uses and emerging questions are discussed in Research →.
FanRows — Building responsive environments for embodied exploration.