Being Decimal: The Anticipatory Pleasures of the Thicket App

Morgan Packard, the sound half of the development duo that produced the 10-finger interactive audio-visual iOS app Thicket, on composing for interactivity

The Thicket app can be understood as many things. An interactive audio-visual delicacy programmed by Morgan Packard (the audio half) and Joshue Ott (the visual half), it is composition and instrument, toy and tool, video art and record album. It runs on the iOS suite of gadgets — the iPad, the iPhone, and the iPod Touch — and turns each of their respective screens into a rarefied sonic playground, one with its own rules and its own rewards.

Thicket presents itself, initially, as a black screen covered with what seem like a nanotech vision of pickup sticks. These myriad razor-thin white lines glisten and bounce around the screen to an elegant score that seems halfway between the gauzy blankness of ambient and the automaton funk of techno. Had Jackson Pollock lived long enough to be a developer on the Atari arcade classic Tempest, it might have looked like this.

Digits, All: Two hands and up to ten fingers can be used to manipulate the Thicket app on the iPad (shown here), the iPhone, and the iPod Touch.

But to touch the screen is to break the score’s fourth wall — to touch the screen is to alter the sound, and the visuals along with it. With each additional finger, the pace of the piece is altered — dragging fingers across the surface brings new patterns. Hold them long enough in the first version of Thicket, and a whole new mixture of sound and image comes into view.

While debugging the second edition of Thicket, version 2.0, which was released today (November 8, 2010), Packard participated in an extended conversation about what Thicket is and what it isn’t: “I don’t want to cross over from creating an interactive art piece which people can explore,” he says, “to creating a completely open-ended tool.”

Perhaps what Thicket is is a peek into a possible future for music, and for video — a future in which a release isn’t a static recording but a malleable one, designed to be played with, prodded, explored. Thicket’s pleasures are real and formidable, but they are also anticipatory, hinting at cultural norms yet to be fully imagined, let alone codified. There’s always a lot of talk in the world of interactive multimedia of 19th-century opera legend Richard Wagner’s Gesamtkunstwerk, the total work that combines all art forms. Thicket is such a work, yet one that never loses sight of its own economy, its own modesty.

iOS Intuition: Video overview of Thicket’s multi-touch functionality by developer Morgan Packard; “one finger works great — up to ten fingers works even better”

Ott, who studied computer graphics, lives in Brooklyn. Packard, who studied anthropology in college and spent some years in the jazz department at the New England Conservatory in Boston, lived in Brooklyn for a decade, but these days makes his home in Denver. Like a lot of people who program, they got their primary education by working. Says Packard, an accomplished electronic musician with several albums to his credit, “I think both of us really cut our teeth coding in the Internet industry.”

Over the course of the interview, Packard talks about how composing music for interactive applications differs from traditional composition (“The idea of retro-fitting a studio production for interactivity gives me shivers”), explains why the second version of Thicket is purposefully less puzzling than its predecessor, and provides a peek inside the Thicket code.

Level Up: A run through Thicket’s new version (2.0), which went live on November 8, 2010.


 

Marc Weidenbaum: The Thicket app — I’m still sorting out whether to call it a “piece” or a “song,” or what exactly to call it — seems to have a few modes, primarily the white-lined one and then the blue-lined one. I’m trying to get a sense of whether I think of the blue-lined part as a “chorus,” versus the white-line “verse,” or a “bridge.” Do these comparisons sit with you?

Morgan Packard: Honestly, I’ve never really identified strongly with any verse-chorus type of music. My first, intense love was jazz, which tends to have all different sorts of structures, and then I went straight in to focusing on underground dance music, which tends to be steady state music, about the groove, the moment, rather than larger structures like verse and chorus. But the analogy does fit. Especially with the new version of Thicket [version 2], which has more modes and an easier way to transition between them; I find myself alternating back and forth in a way that feels like different sections of the same musical piece.

Weidenbaum: Regarding the new version of Thicket, what did you learn from the first version that led you to update it as you did?

Packard: Some of the changes in the new version are in response to problems we identified, and some of them are just us wanting to add more stuff, make it a fuller, deeper experience. The biggest difference is in the mode system. In the first version of Thicket, there were three distinct modes, which you entered based on a fairly obscure calculation based around how active your fingers were. This worked OK, but was confusing to people, and sometimes felt arbitrary. You’d be having fun in one mode, and all of a sudden, you’re thrown into a different visual and sound world. I sometimes found it annoying, actually. The new version has none of those surprise mode changes. You change modes by simply rotating the device. It’s much nicer now, being able to control how the modes change.

Additionally, we’ve added two new modes to the three existing ones. Josh has done a great job dreaming up two dazzling new visual worlds, and I’m happy with giving people two entirely different ways to control the sound. I wanted to give people a little more freedom to be creative with the sound, to expand the palette a little bit. I prefer to be careful with how much freedom we give people, because I don’t want to cross over from creating an interactive art piece which people can explore, to creating a completely open-ended tool. So the new modes have a wider range of the possible sounds you can create, but it’s still a very focused experience. You can be more creative, but it’s still inside the paradigm of exploring the world that we’ve created, rather than building your own world from the ground up. Nothing against actual software tools, of course. That’s just not what Thicket is.

Weidenbaum: It seems in the revised version there are, correct me if I’m wrong, now five stages — the three original ones, and two more. Does the initial, “white lines” one still serve as the “main” or default mode, or do they all have equal standing?

Packard: The initial white lines mode is the most Thickety of all of them. It’s the first one we built, and it’s the heart of the experience. The other modes are supporting characters. And there are five modes now. A default mode, plus a mode for each vertical orientation you can hold the device in. You get back to the default mode by shaking.

Mirror, Mirror: This main screen is the most “Thickety” of all the app’s stages, according to developer Morgan Packard.

Weidenbaum: Before one ever touches the screen, there is already a lot of visual activity. What is the hands-free correlation between sound and the lines?

Packard: Generally what happens is the audio part of Thicket sends messages to the visuals part indicating the intensity of notes as they strike. Then the visuals part can use this message both as an indication of the rhythm — “something happened,” and the intensity — “whatever happened happened with X degree of intensity.”

Stage Diver: The four secondary stages of Thicket, as of version 2.0

Weidenbaum: If this would work for you, could you select a piece of coding from the project, show it, and explain what it is that you’re proud about in it? And with a non-programming reader in mind, could you annotate the code a little?

Packard: That’s a tough one, because interesting software is often made up of a bunch of very simple, boring components and procedures. Also, well engineered software looks simple and boring when you look at it up close. The beauty is in the connection and arrangement of the components, and the programmer’s ability to keep them simple and boring.

Here’s the code which is responsible for slightly tweaking the main sequence of Thicket so that it gradually evolves, randomly:

De-Code: Sound developer Packard has graciously agreed to annotate a snippet of Thicket’s programming.

-(void)evolvePosition:(float)evolveDist{

   evolveCursor++;

   [self checkBounds];

   SequenceState* state = &states[evolveCursor]; // here 
   we pull the current step in the musical sequence out 
   and keep it locally so I can do stuff to it

   SequenceState* excitedState = &excitedStates[evolveCursor];

   int signedRand1000 = (rand() % 1000) -1000; // this 
   generates a random number

   float change = signedRand1000 / 1000.0; // this makes the 
   random number smaller

   (*state).position =  fabs( // "position" corresponds to the start 
   point on the chords sample I play back. The sample is several 
   seconds long, and gets more and more intense. So a larger 
   "position" value will mean a brighter, more intense chord 
   sound. In the following bit, I add the random number I 
   generated to the "position" value of the current sequencer 
   step, so that the sound drifts and changes subtly. If you 
   leave Thicket alone in the default mode, you'll notice that 
   the sound sort of loops, but there are subtle variations in 
   the sequence. The sequence evolves. This code is how that
   happens.

      (*state).position 

      + ( evolveDist *  change )

     ); // 

   (*excitedState).position =  fabs(

            (*excitedState).position 

            + ( evolveDist *  change )

            ); // 

           }

I’m not particularly proud or ashamed of that code, but I’m proud of the idea behind it. And the code gets the job done. The language is Objective-C, which is the default programming language for the iPhone/iPad platform. It’s all written with XCode, which is Apple’s development environment. I’ll write some comments in the code below explaining what it does. I’ve commented with the English language text that begins with “//” [above].

Weidenbaum: Did the piece of music pre-exist the project?

Packard: The visuals, existed before, in a way. When discussing where to start with Thicket, Josh and I both agreed we’d like to start with with a piece from a DVD we did a few years ago called “Unsimulatable.” There was one in particular we affectionately called “the ball of string.” I took some of the sound from the ball of string, but mostly created the sound in response to the specific challenge of creating an interactive piece of music with no visible controls. So everything in the audio reacts just to finger speed and number of touches. I definitely didn’t repurpose an existing, static composition for Thicket. The idea of retro-fitting a studio production for interactivity gives me shivers. For something like Thicket, I’d only want to work with sound generation processes which were fluid, flexible. If I was going to use an existing song, I’d just be chopping it up in to samples and triggering the samples. I’m interested in making something richer and more dynamic than that.

Weidenbaum: You refer to “the ball of string” that preceded the development of Thicket. Is that related to the stage in the new Thicket that has large circles in it? Also, is that video available online?

Packard: No, the ball of string is the default mode, what you see when you launch the app. It’s the dense cluster of tangled lines that hints at complexity to the point of oblivion. “Unsimulatable” is available as a DVD, packaged with my Airships Fill the Sky album. I don’t think the ball of string bit is online anywhere.

Weidenbaum: I appreciate your apprehension regarding re-purposing studio production for interactive purposes. Could you describe how composing for interactivity differs?

Packard: I’ve been into the idea of functional music for quite a long time. The idea of music not being just for music’s sake, but to facilitate something larger than itself. That’s what drove me away from an early interest in jazz, and toward club music, years ago. So, I really like the idea of creating music with the thought of “what does this music accomplish, how will it be used?” I like the idea of form following function. I certainly pulled from my standard musical palette when creating sound for Thicket, but imposed very few actual structures on the user. My purely musical composition process is about creating larger structures out of smaller things, and also about editing — generating lots of ideas, and deciding which of the final particular expressions of the ideas are just right. For example, in creating a piece in the studio, I might compose a nice long melody, or a specific bass line, or come up with an intricate beat that I really like. The balance of those structures is often so delicate that if you change them just a little bit, they don’t work any more. So for an interactive experience, I need to guide the user toward simpler structures, sounds, and techniques that aren’t so delicately balanced.

Weidenbaum: There’s something about the self-contained-ness of Thicket that makes me think about remixing, how much remixing is part of music distribution these days — it’s a promotional tool, a social tool, a functional tool. I wonder if part of what’s cool about Thicket is how it makes remixing part and parcel of the cultural object itself.

Packard: Actually, I’d say that Thicket is decidedly anti-remix in philosophy! I’ve distanced myself artistically from the idea of sample and remix culture. I don’t deny the importance of it historically, and love a nice breakbeat as much as the next guy, but I think the focus on sampling, remixing, whatever, basically musical collage, has gotten us to a place where the sparkle and life is missing from a lot of music. I try to combat that with my own electronic music. I work basically only with synthesized sounds or recordings I’ve created myself. I try to avoid looping my own sounds, and design software that while pattern-oriented, makes introducing variations in the patterns extremely easy. So, with Thicket, we want it to feel like there are no pre-built “chunks” that you’re moving around, as you do with remixing, or collage. Instead, we want you to feel like you’re interacting with a living organism, like every aspect of it is malleable, and never existed before you created. There is something nice about the remix thing, that it allows people with little training and expertise to have the experience of creation. I think that’s great, and that’s what Thicket does. So in that way, I’ll agree that there’s maybe something “remixy” about Thicket.

Weidenbaum: What’s the compositional and programming division of labor between you two?

Packard: So far with Thicket, it’s been: Josh comes up with some interesting visual ideas and programs that end of things, then I program sound for it.

VGA Jockey: Video by developer Joshue Ott showing how the iPad can project to a larger, secondary display when using Thicket.

Weidenbaum: What work have either of you have done in interactive apps previously?

Packard: Josh has loads of experience with Flash development. We’ve both been composing and performing using home-made software for years, which I suppose counts as interactive apps. Josh does something called “multi-draw” with his superDraw software, which allows people to interact with his software as he’s performing, using their iPhones. So people can fiddle on their iPhones and see their movements all projected together on the same screen. Thicket is the first full-on interactive art creation I can take any real credit for.

Weidenbaum: Could you name other music/sound-related apps that you admire?

Packard: I never spend as much time checking out other people’s work as I’d like to. So there’s probably some good stuff out there that I’m totally unaware of. I’m very focused on a couple of criteria that would need to go in to an app that I’m really thrilled by. One is gestural control. I like to feel music actually respond to my movements. And the two-dimensional interfaces of the iPad and iPhone are so nice, they’re really begging to have fingers dragged across them. The other thing I’m looking for is some sort of interaction paradigm that goes beyond the traditional instrument paradigm. In order to make sound on a traditional instrument, you have to bow, pluck, blow, or bang on it. If you stop moving, it stops making noise. Part of the fun of computers is that they can keep working even after we’ve stopped.

So I want something that will give me something back, something I don’t have to put as much effort in to as a real instrument (otherwise I’d just play a real instrument — they sound and feel better), but also something that will allow me to move my fingers, to feel a bit expressive, that doesn’t feel too fiddly. Bloom accomplishes this well. Amit Pitaru’s Sonic Wire Sculptor meets some of my criteria as well.


 

Related Links: Get the Thicket app at itunes.apple.com. More on Morgan Packard at morganpackard.com. More on Joshue Ott at intervalstudios.com. There’s also a great document at vagueterrain.net, collecting excerpts from the development duo’s chat logs.

2 thoughts on “Being Decimal: The Anticipatory Pleasures of the Thicket App

  1. A collaborator of mine (and great musician/mind) K.M. Krebs told me a few years ago that he had crafted a generative drone piece for C-Sound that shifted and changed in a live fashion when accessed. That made me really excited! He sent me a sample, and it sounded great. I wanted to release it on Treetrunk (my netlabel), but I realized that, unless everyone who surfed in could access C-Sound, the full effect of the piece might be lost. Still, it seemed promising.

Leave a Reply

Your email address will not be published. Required fields are marked *