Hacking the Kinect to Prototype New Connections In Urban Spaces

In September I was invited to participate in “Builders at Play”, a quick-hit session in Amsterdam which functioned as something of a public space hackathon, finding ways in which connected technologies can transform public space.

My starting point was something I’ve long been obsessed with – desire paths. Desire paths are the footpaths created by the public when the formal paved routes of a space don’t represent the most efficient, or desired, route between points A and B for the public, and they chose their own paths, their desired paths, as shown above. I wanted to find a way to capture the desired movement of people in a public space and use this incidental movement as a tool to enable a sense of contribution, connection and ownership between individuals and the shared spaces of the city.

Working with designer Jon Stam (centre, above) and programmer and Studio MSP guru Mattijs Kneppers (far left), we identified a space in the NDSM area of Amsterdam and set out to create a system to transform it into a platform for aesthetic narratives created by the movement of the public.

The beauty of only having a couple days to create a prototype for a project is that you instantly shift to hacking and rapid prototyping mode. So we grabbed a Kinect, some cables, a few blocks of Styrofoam, booked some time on a CNC machine, and went at it to communicate our idea. The full narrative outlining the concept and the playback system we devised for the space during night time follows. For those who want the quick walk-through of the concept, our summary presentation can be found at the bottom of this post; for an even quicker bite, here’s a video of us demoing it:

Concept Overview
We hacked the Kinect to serve as a prototype of a public camera overlooking the space, layering it with the ability to pick up a single object or colour on a visitor when they enter the space, and then remember that object and track it as it, and the person, moves through the space. If visitors to the space want to remain anonymous, they simply enter the space and walk through. If someone wants to take ownership of their movement in the space, they touch in at one of the check points with an RFID tag on their key fob, and they are given a unique colour path, which which they can draw something in the space, and then download an image of what they’ve drawn with their movement if they wish.

As the visitor walks across the space, his path leaves a trail in the grid of LED lights are embedded in the surface of the space. The next visitor to enter the space will see the trail left by the previous visitor, and other recent ones as the ghost of their travels are held by the LED lights in the pavement, each previous path fading slowly with time. The space holds a living history of its use throughout the day, and becomes a canvas for those who have ID’d themselves when they entered the space.

Night Time Use
The space holds the memory of its use at night. When dusk falls, a playback of the daily travels through the space begins, each path glowing and traffic patters growing as the timeline of the space’s use from the previous day is played back over night, as this video shows:

Through this playback of the day’s movement in the space, a relationship builds between those who use the space during the day and those who use it at night. The brightness and density of the light paths in the space at night will depend upon the traffic patterns in the space during the day – the more traffic during the day, the brighter the space will be at night. Ultimately, the normally ephemeral qualities of safety and security in the space at night become factors created and influenced by its use during the day.

For those who use the space both during the day and at night, the relationship between the two realities will form. Perhaps a few co-workers will be gathered to create some crazy shapes in the space during the day for them to enjoy as they come home that night. Or paths will be formed in circles to see if pedestrians at night will follow.

From a purely aesthetic point of view, every night, the design of the space will be different, directly correlating to the use of the space during the day, its animations, patterns and luminosity being the creation of those using the space 12 hours earlier.

Platform for Play
People are not the only thing able to be tracked in the space. Someone can place a ball on the square to be ID’d, and it will be tracked as well as it moves throughout the space. Children can enter from different entrances and invent a game where their movement become games pieces on a large urban board.

The space no longer becomes a passive area for its population, but an active participant in their daily lives, enabling creativity, connection, safety and narrative to be built during its daily use.

Platform for Data
In addition to the personal and narrative relationships enabled by use of the space, the space will generate data for use by the city. Data created by the space will be visual use patterns; data of the visual flow and use of the space.

So – there’s the outcome of an enjoyable couple of days in Amsterdam. Well, every day in Amsterdam is quite enjoyable, but these, especially so.

For the technical minded who’ve stuck with it this long, here is Mattijs’s Max/MSP map of what the Kinect was doing behind the scenes (click to enlarge):

Mattijs would like to add that his Max/MSP set-up was supplemented with CV jit.