Using an iPhone or an iPod Touch as a music/multimedia controller (with example Pd patches)

Submitted by Cesare on Wed, 05/07/2008 - 17:35

[This tutorial is still a draft]

There are two applications available at the moment (that I know of) to do this: mrmr and akaRemote.app. Both are native apps for (jailbroken) iPhone/iPod Touch and both use the OpenSoundControl protocol.

mrmr
mrmr is an ongoing open-source project by Eric Redlinge. You can create your own interfaces with a text editor or with a custom GUI builder provided with the project (which, unfortunately, is OS X only). You can also start with an empty interface on the mrmr client and populate it by sending the widgets descriptions via OSC messages. The available widgets are: buttons, slider and x-y controllers. Each project can be divided in four pages.

[mrmr screenshot]

akaRemote.app
akaRemote.app (by Masayuki Akamatsu) is not open-sourced but it also listens for messages so you can add visual feedback to your projects. With the 'Grid' page (one of the four built-in interfaces) you basically have the same functionalities of the monome 64 controller.

[akaRemote.app grid interface]

Controlling Pure Data

To make these two apps talk with Pure Data we can use the OSCx externals (dumpOSC, sendOSC and OSCroute).

Open Pd and create a patch with just [dumpOSC port] connected to [print]. In this way you can see how the messages sent by the apps are formed. Set your computer's IP address in the application and try to press some button or move a slider. You will see the messages printed in the Pd log window. Otherwise you can read the documentation that comes with these apps, but it would be less fun.

One of the most versatile widgets available in both apps is the x-y controller. In mrmr, load the 'Performance.mmr' file and switch to the fourth page. When you drag your finger on the controller, you should see the messages received in Pd printed in the log window:

print: /mrmr/tactilezoneX/44/iPod 490
print: /mrmr/tactilezoneY/44/iPod 516

Here's a patch to get the pad x and y coordinates:

The same method can be applied for akaRemote.app (and for any other application sending OSC messages).

Let's try the monome-inspired buttons grid. By pressing the first button (top-left) I get:

print: /event 2061 3 10 1 1 1

when the button is pressed, and:

print: /event 2061 3 10 1 0

when it is depressed. Now the button is 'on'. If I press it once again to turn it off I receive:

print: /event 2061 3 10 1 1 0

By experimenting with the other buttons it becomes clear that they are identified by the third numerical value. They are indexed from 10 to 73: 10 is the first button from left in the first row, the first button from left on the row below is 18 and so on.

Here's a patch to get the number of the pressed button and its status: