Building FanRows
A longitudinal case study of human–AI collaboration in a complex creative technology project
FanRows has been under continuous development for roughly eighteen months. It started as a browser experiment connecting body movement and sound and has gradually grown into a substantially larger creative technology system.
AI was present almost from the beginning, although its place in the work was very different then. Early conversations dealt with programming questions, APIs and debugging. Later they extended into architecture, interface design, experimental planning and, eventually, coding agents working directly across the repository.
Following that development over time is more revealing than looking at any single AI-assisted task. The same project remained in place while both the software and the available AI systems became more capable. Ideas could be implemented, tested in the actual system, rejected and revisited months later. The technical architecture and research questions are documented elsewhere on the FanRows site; this case study concentrates on what happened during the development process.
In brief
Over eighteen months, AI made implementation, investigation and the exploration of alternatives considerably faster. More of my own work consequently moved toward defining problems, preserving project context, judging consequences and testing whether a technically plausible result actually improved FanRows. The project also exposed a persistent boundary: some information only appeared when the software was used with the body or encountered by somebody who did not already know how it worked.
FanRows and the role of AI developed in parallel over roughly eighteen months.
Contents
- From prototype to evolving system
- AI becomes part of the development loop
- From assistant to agentic development
- Failure, revision and human judgement
- Testing beyond the developer
- What eighteen months of human–AI development changed
- Beyond FanRows: open questions
1. From prototype to evolving system
The first FanRows prototypes were easy to describe. A webcam observed a person, pose tracking supplied information about the body, and movement influenced sound in real time. An arm angle might alter a sound parameter or a detected pose might trigger an event. With relatively few relationships active at once, I could still understand most of the system directly from the code. AI was useful in much the same way as any good programming assistant. I asked about unfamiliar APIs, discussed implementation details and brought it errors that I could not immediately explain. The relevant context usually fitted into one conversation because the individual problem was local and I still carried most of the project structure in my own head.
The harder problems appeared after the basic interaction was already working. A movement could be detected reliably, yet after repeating it a few times the resulting sound became predictable. In another case, a mathematically smooth relationship between body position and an audio parameter still felt strangely mechanical when I actually performed it. The software had no obvious defect; the problem lay in the experience it produced.
One early direction relied heavily on clear movement controls because they made cause and effect easy to understand. That worked well for demonstrations. With more of those controls active at once, however, I began to feel as if different parts of my body had become invisible knobs and sliders. I was operating the system rather than exploring it. That was useful feedback because it pointed to a design problem that would have been difficult to identify from the implementation alone.
FanRows gradually moved toward continuous movement and slower changes in sound. Layers could remain present rather than repeatedly starting and stopping, and the system needed ways to regulate how strongly or quickly a bodily change affected what was heard. The exact technical mechanisms evolved, but the reason was practical: the interaction had to remain interesting after the first obvious connection between movement and sound had been discovered.
Those experiments made the software larger. Different sound environments needed different settings, and embedding each experiment directly in application code became cumbersome. Configuration was separated from the part of FanRows that actually runs an experience. Later, editing increasingly detailed configuration files became its own obstacle, which led to the development of FanRows Studio, a browser-based creator environment for assembling and adjusting those experiences.
There was never a moment when I stopped and redesigned everything according to a finished master plan. New structures tended to appear because an existing approach had reached a limit. Some survived. Others were revised again a few weeks later. The present architecture therefore contains traces of that history.
FanRows Studio during a live session. The creator environment brings body tracking, sound material, scenes and the relationships between movement and sound together while the result can be tested directly in the browser.
The screenshot also shows why development eventually became harder to reason about locally. A change to the way body movement was interpreted might later be visible in an editing control and finally affect what happened when a session was played. The code behind those steps lived in different parts of the project. Our AI conversations began to follow those connections. Instead of only asking how to implement a function, I increasingly discussed where a responsibility belonged, why state was being lost between components or whether an apparently clean refactoring would make another part of FanRows harder to change later.
There was a catch. The code never contained the whole reason why FanRows looked the way it did. Some decisions came from experiments that had already failed. Others reflected things I had noticed while moving in front of the camera or adjusting sound. Two pieces of code that looked unnecessarily different could represent a distinction I had deliberately kept after earlier testing.
As FanRows grew, supplying that background became part of working effectively with AI. Without it, a model could offer a sensible solution to a slightly different version of the project. By then, the software was no longer a sequence of independent programming problems. Interaction decisions affected architecture and architectural decisions could return later as something I felt or heard while using the system. That interdependence also changed what AI assistance could usefully mean.
2. AI becomes part of the development loop
Many FanRows problems do not arrive in the form of a specification. I notice them while using the system. “The sound changes too abruptly when I move” is a typical starting point. That observation does not yet say where the problem is. The camera tracking might be noisy, the movement values might need different smoothing, the conversion from movement to sound could be too sensitive, or the audio behaviour itself might create the impression of an abrupt response.
Discussing such observations with AI became surprisingly useful. I could begin with an imprecise description of what I had experienced, then work through plausible technical explanations. Sometimes the first explanation was wrong, but even that narrowed the problem. A change could be implemented and tried immediately in FanRows, where it either improved the experience or produced another clue.
That pattern appeared often enough to become part of ordinary development. Observation came first, followed by analysis and implementation. Then I ran FanRows again. The result, rather than the conversation, determined what happened next. Some cycles were short. Others exposed a deeper problem. An interface irritation might turn out to involve state handling across several components; a sound problem might lead back to the way movement values were interpreted. The distinction between debugging, interaction design and architecture was often much less clear in practice than it looked in a project plan.
The development loop that emerged around FanRows: observation and embodied experience feed into AI-supported reasoning and implementation, while the running system and human judgement return new information to the next iteration.
I do not regard this diagram as a formal development method. It describes what repeatedly happened around the running project. The longer that process continued, the more expensive missing context became. A discussion about one function might need only a few files. A discussion about Studio behaviour could depend on decisions made months earlier, terminology used elsewhere in the project or a distinction between two similar concepts that was intentional rather than accidental.
I therefore started externalising more of that information. Architecture notes, GitHub issues, explicit task descriptions and development summaries were useful for me, but they also allowed later AI conversations to start closer to the actual state of the project. The aim was never to preserve every conversation. Most of them did not deserve to become permanent project memory. What mattered were decisions that would otherwise have to be rediscovered: why a boundary existed, which approach had already failed, what a term meant inside FanRows, or what behaviour had to remain stable even while the implementation underneath it changed.
The repository alone is not the project.
The repository contained the current implementation. It did not automatically contain the path that had produced it. AI was gradually becoming less like an expert brought in for a question and more like a reasoning partner that could re-enter the same project again and again, provided enough of that history was available.
Coding agents extended the idea further. Once an AI could inspect and modify larger parts of the repository directly, missing or badly framed context no longer affected only the quality of an answer. It could affect the codebase itself.
3. From assistant to agentic development
Repository-level coding agents changed the scale of work I could hand over. Earlier AI assistance usually ended with a suggestion or some code that I still had to integrate. An agent could follow a problem through several files, modify them and run checks before returning the result.
A real FanRows issue shows the difference better than a general description. On 10 July 2026 I opened GitHub Issue #18: “Fix Studio V2 material sliders resetting during live preview.” The visible symptom was quite mundane. While editing an active sound loop in Studio V2, the Presence and Brightness sliders could jump back instead of remaining at the value I was trying to set. That made live editing frustrating because a change I was listening to could disappear while I was still adjusting it.
The cause sat deeper in the component behaviour. MaterialLoopCard was re-synchronising its local draft when values associated with the live preview changed. Moving the slider caused preview updates; those updates could in turn trigger the synchronisation that overwrote the edit currently in progress. By the time the issue was ready for implementation, it contained more than the original complaint. It described the root cause, the conditions under which the draft should and should not be synchronised, expected behaviour for Save and Cancel, and regression tests for both sliders.
GitHub Issue #18 from FanRows Studio V2. The observed behaviour, technical cause, required fix and regression coverage were recorded before implementation.
This is closer to how useful agentic work developed in FanRows than the idea of typing a broad prompt and waiting for finished software. A task became easier to delegate after the uncertain part had been reduced: what was actually wrong, which behaviour had to remain intact and where the change was allowed to reach. The agent still had freedom at the implementation level. It could inspect related code and make changes across files without requiring me to dictate each edit, but the task had a boundary.
That boundary mattered because speed works in both directions. An agent can carry a correct assumption through several files quickly. It can do exactly the same with a mistaken assumption. A refactoring may compile, pass its tests and still make an older FanRows session behave differently because the task description failed to mention an implicit constraint.
For that reason, clearer architecture became useful in a new way. Terms such as Studio and Runtime describe real responsibilities in FanRows: Studio is where an experience is created and adjusted; Runtime is where that experience is played. If a task belongs to Studio but must not change runtime behaviour, that distinction gives an agent a meaningful boundary without requiring a complete explanation of the entire codebase.
Review changed as well. With a larger agentic task, watching every line appear is less useful than checking whether the task itself was understood, whether the change stayed inside the intended boundary and whether the behaviour that motivated the issue was actually fixed. Issue #18 was closed on 22 August 2026. The important part for this case study is not the slider bug itself. It is the path from a concrete problem during live use to a bounded technical task with enough context for an agent to work on it reliably.
There were also failures. Sometimes an agent produced a cleaner implementation that removed a distinction I still needed. In other cases, a solution was technically reasonable but solved the wrong layer of the problem. Those cases were usually more instructive than obvious syntax errors because the output looked convincing.
Git history, issues and working versions became useful checks against another AI tendency: retrospective neatness. A model can explain an architecture as though it had always been moving toward its current form. FanRows did not develop that way. There were detours, experiments that were later abandoned and decisions whose original reason was pragmatic rather than theoretically elegant.
Agentic development made larger delegations possible. It also made the quality of task framing and review much more consequential.
Working with AI agents on a growing codebase?
Questions of project context, architectural boundaries, task framing and review also arise in existing software projects. Discuss your project →
4. Failure, revision and human judgement
One of the recurring difficulties in FanRows is that technically correct behaviour can still produce a poor interaction. Consider a movement that the tracking system measures reliably and that changes the sound exactly according to its implementation. During a short development check it can appear finished. After several minutes of actual use, however, holding or repeatedly reaching the required position may become tiring. The problem only becomes obvious because a body has been doing it for long enough.
That distinction matters in FanRows because I often test while I am still building. I may change a sound relationship at the computer, move back in front of the camera, listen for a minute, return to the controls and alter it again. A relationship that feels immediate and satisfying in one of those short checks can behave very differently during an uninterrupted session.
The video below is therefore less a performance demonstration than a piece of development evidence. It shows the physical situation in which some design decisions have to be judged.
Embodied testing during development. Running FanRows with the body reveals qualities that cannot be determined from source code alone, including physical effort, timing and whether an interaction remains engaging over time.
AI-generated solutions encountered the same boundary from another direction. A model might propose a technically elegant way to make behaviour more uniform across the system. Whether that uniformity was desirable depended on why the difference existed. Sometimes it reflected unfinished code. Sometimes it had survived because two experiences were intentionally meant to feel different.
The useful part of the collaboration was often the disagreement around such cases. AI could expose an assumption in my design or suggest that something I regarded as necessary was really historical baggage. I could then try the alternative rather than defend the existing implementation in the abstract. Occasionally the AI was right and an old structure could disappear. At other times I reverted part of the change after using the result. A failed implementation was not wasted if it clarified which property of the experience I had been trying to protect.
My own judgement had limits too. After working with the same system for months, I knew how to make it respond. I anticipated sound changes, unconsciously favoured movements that produced good results and could overlook awkward behaviour because I had learned to work around it.
For that reason, the goal was never to establish a hierarchy in which either AI or human judgement had the final word by default. A proposed change had to return to FanRows and survive contact with the thing it was supposed to improve. As the implementation work became faster, more of my attention went into this part of development. The difficult decision was increasingly less about how to produce another version and more about whether that version deserved to remain.
5. Testing beyond the developer
Self-testing eventually reached a fairly obvious limit: I could not become a first-time FanRows user again. If I knew that a particular movement affected the sound, I would try it. After repeatedly tuning a threshold I could sense approximately where it was. Even when I tried to behave neutrally, I brought months of knowledge into the session.
In August 2026 the first two external tests gave me another view of the system. The sample was far too small for general claims, but the behaviour was already useful. The test notes recorded exploration and repetition, periods in which participants paused and listened, search-like movements and different reactions to different sound rooms. These were things that could be observed without first asking participants to explain the internal logic of FanRows.
The testing setup deliberately avoided teaching people how to perform. Participants could know that movement influenced sound and visuals, but they were not supposed to receive a list of gestures or an explanation of what they were expected to discover. An observation sheet and a short questionnaire gave the sessions some structure. During the interaction, factual observations could be recorded before trying to interpret why somebody had behaved that way. Afterwards, the questionnaire provided another perspective on what the participant believed they had noticed.
The distinction between observation and interpretation turned out to be useful. If somebody repeatedly returned to a particular movement, that was an observable event. Whether they had discovered a stable relationship, were simply enjoying the sound or were testing a hypothesis of their own required more caution.
After those first two tests I deliberately did not treat the initial reactions as a pattern. On 22 August, GitHub Issue #24 was opened with a non-technical next step: recruit three additional naive participants and increase the initial test pool from two people to five while keeping the test setup comparable.
That issue is unusual in a software repository because its acceptance criteria contain no implementation task. It asks for participants who have not worked on FanRows, have not been taught the detailed movement relationships and can encounter the frozen test version without being coached toward a particular behaviour. The reason for recording the task alongside technical issues was straightforward. By that stage, gathering better evidence was sometimes more valuable than adding another feature.
The external tests also widened the human–AI development loop. Behaviour observed during a session could be brought back into an AI conversation and examined alongside the implementation. AI could help look for a technical explanation or suggest what to test next, but the original evidence had come from another person encountering the system.
Five participants will still not constitute statistical validation, and the current work is explicitly exploratory. The point at this stage is to find out whether some of the behaviours seen in the first sessions recur often enough to justify a more formal question later.
For the case study, even the small beginning matters because the development process now includes observations from people outside the developer–AI relationship, adding information that neither side could produce alone.
6. What eighteen months of human–AI development changed
Looking back across the development of FanRows, the clearest change is where effort is spent. Routine implementation has become cheaper. So has investigating an unfamiliar API or asking an agent to trace behaviour through a part of the repository I have not touched for several weeks. Trying two technical approaches before committing to one is much more realistic when producing the alternatives does not consume days.
The bottleneck moved.
The difficult part increasingly lies in deciding which of those possibilities belongs in the project. That sounds like a small distinction, but it affected my work quite substantially. When creating another implementation becomes inexpensive, adding complexity also becomes tempting. A new abstraction can be produced quickly and still leave FanRows harder to understand six months later. An agent may make a requested feature work perfectly even though the better decision would have been not to add the feature.
Architectural experience therefore remains important, although I use it differently. I spend less time proving that I can manually implement every layer and more time asking what a change would couple together, whether a new concept deserves to exist and what future work it will make easier or harder.
Problem descriptions have acquired similar weight. With an agent capable of making a substantial change, an imprecise task can produce substantial wrong work. The reasoning that precedes implementation — identifying the actual problem and deciding what must remain stable — is part of engineering rather than administrative preparation for it.
Project memory became another practical concern. The current source tree can show how FanRows works today, but it does not necessarily show why a particular design survived. Notes, issues and architectural documentation now carry part of that history. Some of the information is intended for me months later; some makes future AI work more reliable.
There is a balancing problem here. Preserving every old explanation would eventually make the context worse, because previous intentions can become obsolete. A long-running project needs some way of distinguishing a current decision from an old experiment that happens to be well documented. I do not think FanRows has completely solved that problem.
More capable AI also produces more convincing mistakes. An obviously poor answer is cheap to reject. A polished implementation based on one incorrect premise may survive longer precisely because so much of it looks right.
Returning to different kinds of evidence became the practical counterweight. A unit test can tell me whether a saved value survives correctly; running Studio reveals whether the parts still work together during editing. Using FanRows with my body exposes fatigue or monotony that neither of those tests contains, and another person can behave in ways I have stopped being capable of producing because I know the system too well. Those forms of evidence are not interchangeable, which matters whenever an AI explanation sounds more complete than the available evidence really is.
My role in the project has consequently become broader even as some implementation tasks have become easier. I still read code and debug technical problems. I also spend considerable time deciding what deserves to be built, what context an AI or agent needs, whether an apparently successful change has created a new problem and when there is not yet enough evidence to make a decision.
FanRows does not show that a developer with AI has become equivalent to a multidisciplinary team. The first external tests already point in the opposite direction: other people contribute information simply by not sharing my history with the project. Specialist expertise and genuine disagreement between humans remain different from asking an AI for another perspective.
What the project does show is a substantial expansion in the range of work one developer can attempt. FanRows has moved across browser software, real-time audio, body tracking, creator tooling and exploratory research without each new area requiring a completely separate development process. AI made some of those transitions far easier.
The most consequential gain has therefore not been a count of generated lines or features delivered. It is the ability to keep a relatively complex experimental system moving while shifting more human effort toward direction, context and evaluation.
7. Beyond FanRows: open questions
The experience so far leaves several questions that the current implementation cannot answer. Authorship is one of them. FanRows contains code and architectural ideas that emerged in conversations with AI, alongside texts and conceptual distinctions developed through the same process. I still choose the direction of the project, decide what remains in it and take responsibility for the result. Describing AI simply as a tool misses part of the interaction, while calling it a co-author would imply responsibilities and agency that it does not have. I do not yet find either category entirely satisfactory.
There is also a more technical question inside FanRows itself. AI currently participates mainly in development. It helps build the body–sound relationship but is not an adaptive actor inside that relationship while somebody is moving.
A future experiment could change that. Instead of playing relationships designed entirely in advance, FanRows might contain an adaptive process that observes how somebody has been moving and alters aspects of the sound environment over time. The resulting system would have to do more than reward whatever movement it detected most often. Otherwise adaptation could easily narrow the experience instead of enriching it.
The same problem becomes harder in a shared environment. If several participants influence one sound space, an adaptive system could begin responding to patterns across the group. It could also reinforce whoever moves most strongly or most consistently until the rest of the group follows. Preserving diversity rather than producing convergence would become part of the design problem.
These questions are currently research directions, not claims about what FanRows already does. Keeping that distinction visible is important because the conceptual possibilities have grown faster than the evidence.
FanRows remains unfinished. For a longitudinal case study, that is useful rather than inconvenient: the questions described here can continue to be observed as both the project and the AI systems involved in building it evolve.
About this case study
I am Jürgen Berentz, founder and developer of FanRows. My background in software architecture and product development informs the technical side of the project, while FanRows has increasingly expanded into creative technology, embodied interaction and experimental research.
Working on a complex project with AI?
Long-running AI-assisted development raises practical questions about architecture, project memory, agent workflows and human judgement. Those questions are not specific to FanRows.
If you are developing a complex project with AI, introducing coding agents into an existing development process, or trying to keep a growing AI-assisted codebase coherent, I can support you in structuring that collaboration, maintaining architectural boundaries and building workflows in which AI assistance remains connected to human judgement and real-world testing.
FanRows — Embodied Interaction in Responsive Sound Environments