The ShapeSeq of Things to Come

Developer Paul Apfrod on how programming is like composing, the need for a "software art" iTunes subcategory, and how how octagons look nicer than squares

ShapeSeq for the iPhone and iPod Touch produces four rudimentary noises, which makes it sort of like the ukulele of music apps — well, if there weren’t already a lot of ukulele apps.

Each noise in ShapeSeq is one of the basic waveforms: sine, square, triangle, saw. And each waveform is symbolized by a basic shape-color combination: a blue circle, a red pentagon, a green triangle, an orange square.

Placing or moving one of those symbols on the screen (ShapeSeq also works on the iPad), you can roughly determine a given shape’s pitch (along the vertical axis) and volume (along the horizontal). It will then repeat. The manner in which that selected pattern repeats is determined by two factors: the pace of the loop, and whether it’s a four-beat or eight-beat sequence. These options can be altered in realtime, which makes ShapeSeq as much a performance tool as a compositional one.

This all seems fairly straightforward, right? But play with ShapeSeq for even a short period of time, and the expansive variety of potential patterns becomes apparent. Says the app’s British developer, Paul Apfrod, “One thing I like about what emerges from the simple system I created is the blurring of boundaries between what would be considered a note or a single beat of a sound and of a rhythm or melody of several sounds.” From little things big things grow.

After & Before: Screenshots of the iOS app ShapeSeq (above) and the earlier, pre-iOS version, called Shape Sequencer, which involved PlayStation 2 controllers (below, in a still from video footage)

To trace the course of ShapeSeq’s own development, you have to go back to 2004, several years before the 2007 launch of the iPhone. ShapeSeq had an earlier life as an art-installation project, when it was called Shape Sequencer. It was built in Pd, or Pure Data, a visual programming language that has a shared lineage with Max/MSP, and it was played on Windows machines using PlayStation 2 controllers (the PlayStation 2 at this point would have been a half-decade old). It also allowed for multiple players in a kind of cyber-jam.

Since then, the application has made two significant evolutionary jumps: from open-source art-hack to the commercial world of the iTunes Store, and from site-specific installations to global distribution. The website creativeapplications.net, which tracks the intersection of interaction and artistry, listed it among the “15 Best and Must Have iPhone Apps of 2009.” As Apfrod puts it, his project has gone from “art installation” to “software instrument.”

ShapeSeq is one of a host of music apps that have helped feed a growing popular interest in playing with, manipulating, sound. Such sonic play is an inherent part of contemporary experimental electronic music — from so-called “handmade music” tools to the home-brew software patches developed by laptop musicians. What’s remarkable about the music app field is that everyday consumers are participating in what previously would have been an esoteric affair.

Apfrod, who was born in Scunthorpe, North Lincolnshire, and currently lives south of London, studied interaction design, and says most of his programming skills are self-taught. A musician himself, his earliest music-making experience took place, foretelling his current activity, on the Amiga personal computer. Over the course of an interview about ShapeSeq, he talked about, among other things, the cultural and technological shifts between Shape Sequencer and ShapeSeq, how programming is like composing music (and vice versa), and the need for a “software art” subcategory in the iTunes app store.


 

Marc Weidenbaum: How did ShapeSeq originate?

Paul Apfrod: ShapeSeq for iPhone is the child of the earlier Shape Sequencer project, which was made in Pd — Pure Data — and allowed four players to jam using four PlayStation 2 controllers. Shape Sequencer came from an investigation into notions of “playing” in the sense of musical instruments and video games, trying to address the problems of immediacy, intuitiveness, and expressivity for electronic music performers and audiences. It became its own type of musical instrument, with simple geometric shapes on screen representing different sounds. As the players move the shapes around, shrink them and rotate them, so the sound of each shape is changed in pitch, volume, and timbre. Thus, the players have their movements magnified into a live video abstract representation of the sounds being created. This visual feedback is good for the practice of the player, and satisfying for the audience, as they can follow the usually obscured practices of the electronic music performer by watching the screen.

PlayStation Jam: Video from 2005 of various installation events that featured the (pre-iPhone) Shape Sequencer, evidencing the collaborative-play aspect of the original software

In making Shape Sequencer and enhancing it for my own use in performances, I added the ability to record movements to a constantly playing live loop, and to resize and divide this loop, all from the PS2 controller. This mechanism leads to some of the more unique sounds the Shape Sequencer is capable of. The loop is represented by a regular polygon, which can be changed in size and number of points, and will play back or record on those points as divisions of the same pattern.

Years later, the iPhone came out, and eventually people started making interesting sound apps and musical instruments for this new platform. ShapeSeq seemed an obvious candidate for my first app. I aimed to make the iPhone version just as intuitive and fun to play, but adapted for the screen size and single player. I also added the ability to save and recall with four memory slots, so players can store their best loops or switch patterns quickly in live performance.

Weidenbaum: What’s the timeline between the development of Shape Sequencer and the development of ShapeSeq? Was the original more of an art experiment, and then the iPhone OS came along and made it possible to do this sort of work for a broader audience?

Apfrod: Shape Sequencer was first developed between 2004 and 2005, and was used for performance and art installations. I started work on an iPhone version around Sept 2009, and released it in December 2009 to reach a broader audience, and just to make something cool as a first iPhone app.

Weidenbaum: Did you sort of put aside Shape Sequencer between 2005 and 2009, or was it still a big part what you were working on, and where your head was at?

Apfrod: I was working on other stuff and developing my software-making skills. I knew Shape Sequencer wasn’t really finished, and still isn’t, but didn’t think about it for a bit while I got on with better paid work where I could develop my ability.

Weidenbaum: You did Shape Sequencer in Pd, which has a somewhat shared lineage with the software Max/MSP — what did you run it on? A Mac, a PC? Did it not matter?

Apfrod: I chose Pd because it was open source and cross-platform. This assures me that my works will be portable, and hopefully available in future. I used Windows at the time and tried porting to Mac and Linux, but had problems with drivers for the PS2 controllers.

Weidenbaum: During what situations do you imagine people using ShapeSeq?

Apfrod: ShapeSeq is a musical instrument that happens to be a piece of software. I hope to be surprised by what people create with it, and how they enjoy it.

It is an interesting question — what makes people play musical instruments in the first place? I designed the Shape Sequencer out of a desire to explore sounds in a performance, and to share the joy of just playing — in the sense of being playful — with electronic sounds. Putting this into people’s hands and pockets with the iPhone opens up vast new performance and practice opportunities.

Weidenbaum: What, if anything, have you learned from iPod/iPad users of ShapeSeq since its release?

Apfrod: Some love it, some completely don’t get it. It’s too radical for a lot of people. Maybe the App Store should have a section for “software art” apps. I’d appreciate some more feedback from users.

Weidenbaum: The instructions are fairly minimal for ShapeSeq, which is not uncommon in music apps. The best instructions on your site are in the video — do you think video is a better mode for app instructions than written-out instructions?

Apfrod: For some, videos are best. The first set of instructions for Shape Sequencer were a laminated A4 sheet, and players could actually try it there and then. I guess people need some instruction about techniques and ideas, since it really takes some dedication to investigating the possibilities to discover all the tricks and styles possible.

Demo Scenes: This demonstration video doubles as a detailed introductory tutorial on how to use the ShapeSeq app. Note this is one version prior of the app, and therefore doesn’t show the “M” button for the memory slots. (Compare with the screenshots up above.)

Weidenbaum: How do you balance your time between iterating updates to ShapeSeq and developing new projects?

Apfrod: I just do it when I fancy. Right now there’s more demands on other stuff I’m doing.

Weidenbaum: Version 1.2 of ShapeSeq, released this past June, added memory slots. Had that been on your mind from the beginning?

Apfrod: When I started to take Shape Sequencer seriously as a software instrument, rather than an art installation, it seemed obvious that it needs a cross-platform file format for exchanging patches or performances. This seemed like a complex problem, so I put it off from version 1, and hoped to make a good file format for file exchange in a future version. But the demand for some sort of saving was strong, so I put a simple slot-based load/save mechanism in there to at least partially satisfy this requirement.

Weidenbaum: I think it’s interesting you made the octagon the default instead of the square. What trial and error, or general insight, led to that decision?

Apfrod: I think the octagon looks nicer than the square! There may also be something innate in the way I make beats that leads me to think of eight beats rather than four.

Weidenbaum: One of my favorite things to do in ShapeSeq is to draw an initial pattern at a large or a small number of steps, and to then scale back and forth to see how your algorithm breaks it up, compresses it, or expands it. It’s a neat way to introduce a sense of variations on a theme. That’s not a question in itself, but could you talk a bit about how things like the steps or the ability to load and save banks can be used in performance?

Dr. Octagon: At left, the initial screen, in which the small blue triangle demarcates the current spot along the eight-beat loop, which is depicted as an octagon. At right, a second loop, also an octagon, is added.

Apfrod: The way steps of a loop are divided can lead to some very interesting sounds and patterns, since the same loop is used at different parts along its length according to how many steps are in the loop. In ShapeSeq, a loop has a length and a number of steps, where the loop is divided along its length by the number of steps. Patterns can be recorded in the gaps between other patterns, or split up with silence between them to make gated effects. By carefully building up patterns at different resolutions of steps, interwoven sequences can be made out a single shape.

One thing I like about what emerges from the simple system I created is the blurring of boundaries between what would be considered a note or a single beat of a sound and of a rhythm or melody of several sounds, since this is really only a distinction of the length of certain intervals.

Being able to load and save banks allows the player to instantly recall a favorite pattern or sound. In a live performance, this can be used to save a theme or riff, play variations over it, and quickly return to that theme later on, or swap between several themes built earlier to build up a song. The emphasis is still very much on live performance, but with some of the benefits of digital music (perfect recall) combined with some of the joys of real playing (sloppy input and immediacy).

Weidenbaum: What kind of thought went into the sounds you selected for ShapeSeq? What were sounds you’d tried out but decided not to use?

Apfrod: I made the four basic waveforms — sine, square, triangle, and saw — for ShapeSeq for simplicity, and out of affection for these basic sounds. My approach is to provide simple building blocks so that the complexity can emerge in the playing.

Weidenbaum: Any plans to port it to another OS, like Android, or Windows 7, for example?

Apfrod: I could make versions for OSX and Windows, if there’s demand.

Weidenbaum: Are there particular iOS apps that you especially admire?

Apfrod: In terms of music/sound apps, I like Sampletoy by Marek Bereza (mrkbrz.com), Tweakybeat by Rodrigo Yanez (itunes.apple.com), and RjDj by Reality Jockey (rjdj.me). I also love the style and innovative gameplay of Eliss by Steph Thirion (toucheliss.com).

Weidenbaum: What tools did you use for writing (and, now, improving upon) ShapeSeq?

Apfrod: ShapeSeq is written using openFrameworks.

Weidenbaum: What can we expect from the next version of ShapeSeq?

Apfrod: I would like to add realtime sampling and sample manipulation, if I get around to it. Maybe one day I will have desktop versions and then I would like to put in file transfer between desktop and mobile. If I ever get an iPad, that is also likely to lead to some innovations too, as I foresee the larger touchscreen as a great potential interface for music.

Weidenbaum: The early version of ShapeSeq, Shape Sequencer, before the iOS edition, involved multiple players. Is this something you plan on bringing to the iOS version?

Apfrod: Since iOS includes iPad, maybe. But for iPhone/iPod Touch, I can rather imagine several players with their own device, just due to the size.

Weidenbaum: Do you also make music, perform music, regularly?

Apfrod: I’ve been making music for many years, starting on the Amiga as a young teenager. Some years ago I ran a tape label — called blank audio — and sold and swapped tapes on the fanzine scene. I’ve had tracks on a few fanzine-related compilations, which are probably impossible to get hold of now. Some of the blank audio back catalogue is available on archive.org, however. Some of my more recent work is a available to hear on soundcloud.com/apfrod, but I’ve spent more time learning how to make software over the last few years.

Weidenbaum: Can you talk about any parallels between the processes of making music and making software?

Apfrod: Since I have spent so much time making music in software, making music could be like making software sometimes. There is a lot of listening and tweaking when making patches or patterns, just as there is continual testing and tweaking with code. There’s also a kind of “object oriented” approach to electronic music making, where you will spend a lot of time on a component like a snare or a tone which will be reused over and over again in a track. Personally, I love the “pure techno” approach where every sound is generated algorithmically, just as I love procedural and generative software approaches.


 

Related Links: Get the app at itunes.apple.com. Follow on twitter.com/shapeseq. Visit the app’s home page: apfrod.com/shapeseq. Early coverage (December 2009) of ShapeSeq at creativeapplications.net by Filip Visnjic (fvda.co.uk). More on Apfrod’s chosen development platform at openframeworks.cc.

Leave a Reply

Your email address will not be published. Required fields are marked *