The line between the digital world and the physical one is imaginary

There have been some really interesting projects spring up recently that make use of physical objects in three-dimensional space to negotiate or interact with digital media. This seems important.

It’s something I’ve been giving a great deal of thought to recently – and I’ve been paying close attention to this development, especially with respect to music making and music consumption – but it applies right across the realm of things we like to think of as ‘creative’ or ‘playful’.

To show you what I mean about the erasure of the line between digital and physical, take a look at Osmo – an iPad toy that uses wooden puzzle pieces to engage children in creative play and group activities.

As you see from the video, users arrange the wooden shapes on a table in front of the iPad and get instruction and feedback to let them know when they have it right. It puts the play in physical space rather than simply on the screen, but also uses the intelligence of the iPad to make the experience entirely interactive.

My favourite example of this sort of thing right now is the Qleek project, which uses beautiful wooden hexagons as playlists and holders of other intangible digital media such as video. The technology behind it is the NFC (near-field communication) chip – the same one employed by Samsung smart phones to pick up simple bits of information and direct the device to an online link.

Like a QR code, only actually useful…

The thing I love most about Qleek is the thought that’s been given to the way in which we make meaning from physical objects that contain intangible media. Just as vinyl records are collectible, personal, ritualistic – and can be displayed to reflect identity, Qleek ‘tapps’ fulfil a very similar function in a way that is sensory and experiential.

Of course, the wooden hexagons with the NFCs are entirely ‘unnecessary’ in that they simply provide a link to a URL where the media is actually stored and do not ‘contain’ any media themselves – but the design, feel and magnetic ‘fit’ of the physical media artefact into the ‘player’ provides a rich and rewarding experience.

At Music Tech Fest in Boston, Norbert Schnell of IRCAM threw some soft, coloured balls into the audience, who were encouraged to play with them. The balls contained accelerometers that detected movement and spin on multiple axes. They were also connected to a piece of software that created responsive music, communicating this time via wireless transmission.

See also Modular Musical Objects.

The balls did not ‘contain’ or ‘produce’ the music, but the way in which you threw, bounced, passed or otherwise interacted with them dictated the quality and the nature sounds that you heard. You were making data by engaging with the physical object and the computer was making music dictated by that data.

As an experience, there was no separation between the physical object and the digital media phenomenon – just as there is no separation between pressing the keys on the piano and hearing the sound of the strings being hit with felt hammers.

And this is central to the idea of Internet of Things – a current buzzword usually reserved for discussions of home heating monitoring, transportation, stock management and other industrial and prosaic uses.

But we also have the opportunity within the creative arts to erase the artificial boundaries of ‘real’ and ‘virtual’ by applying principles of design innovation to bridge the gap between the ways in which human beings like to experience the world and the affordances of digital technologies.

What Michela Magas calls the Internet of Music Things.

We like things to feel nice. To smell nice. To be beautiful. We like to share them, hold them, display them and play with them. We finally seem to be getting over the idea that being digital means that we have to abandon ‘stuff’. We like stuff. Stuff is lovely.

I feel like negotiating everything via a screen has been a necessary transitional phase. That we’re the unlucky generation for whom most digital interaction has, for the past 20 years, been primarily via keyboard and mouse.

But we’re now at the beginning of a world of experiences that are hand carved from wood, hung on the wall, gift wrapped, treasured, moved around on a table with our friends or thrown about the room. And what we’re seeing now is just the tip of the iceberg. You won’t need to throw out your computer, but all going well you may not notice that it’s even there a lot of the time.

There’s a long way to go yet. A great deal of media design innovation still to take place. But it’s encouraging to see it becoming a primary concern.

Digital technology was supposed to make our lives better – and it has in all sorts of ways. But now it’s finally starting to make up for taking some of our lovely things away.

Leave a Reply