I proposed that we device at least one ‘effort’ gesture continuum. At one end of the continuum, we have a physical gesture that appears to require little energy exertion while at the other end we have a gesture of great vigour and energy. In between these extremes, we have numerous gestures, each one requiring more effort than the next.

I followed these steps while developing a possible continuum.

1.  I chose physical gestures that required the least and most effort based on musical and theatrical concepts required for Luna’s composition.

Least effort:  horizontal stick / cradling / no movement / no pressure (no squeezing)
Most effort:  continually changing angularity / alternating wide grip and baseball bat position / kayak and  lasso like movements / pulsating squeezes

2.  I categorised the different components of these two gestures (the least and most effort gestures) into four fields:

(1)  Angularity, orientation and position of stick
(2)  Touch or contact
(3)  Expanse of movement, activity
(4)  Squeezing

3.  I looked at these four fields and considered the sensing mechanisms involved in each. For instance, angularity is about measuring tilt via the accelerometers. Squeezing involves the pressure sensor. Measuring the expanse of, say, a lasso gesture requires a reading of combined gyroscope data.

4.  Next, I programmed ‘two’ different algorithms for extracting gesture in each field (e.g., two methods of measuring effort in relation to changes in angularity).  The two algorithms are designed to respond with different sensitivity so that one method is more representative of effort that is exerted in short bursts while the other method indicates a longer energy accumulation, as long as the gesture is repeated/maintained.

A preliminary ‘effort’ gesture continuum is listed here, from least to most effort exertion. One performance goal may be to combine all of these and so, as each gesture is added, there is an accumulation of movements and also exerted effort.

(1) Rolling slowly
(2) Tilting slowly
(3) Movement along the frontal plane (forward and backward motion)
(4) Changing hand position (widening and narrowing) and brushing
(5) Squeezing with a sustained degree of force and/or rapid pulsating squeezes
(6) Spinning: lasso, fan, airplane, majorette
(7) Concentrated and highly vigorous spinning
(8) Throwing

While performing, the prompt score appears as a downward scrolling on-screen graphic that is positioned next to the standard t-stick interface window. The animation of this graphic is controlled by Max/MSP.  A vertical scrolling display – music notation is normally read from left to right, but not in this case – was chosen for two reasons.  First, because the size and lay-out of the t-stick interface window commands two thirds of the laptop screen, only a thin vertical space on the screen is available for the prompt score.  Second, and more importantly, I believe that the full-body movement required to play the t-stick can be better depicted by a score lay-out that, in itself, may more accurately enclose the geometry of the performer’s body. In other words, it is easier to show an image of a standing human form in a vertical window space than in a horizontal – perhaps physically reclining – window.  In addition, I was influenced by the study of Laban dance notation, which also uses a vertically distributed notation system.

To facilitate t-stick notation in this fashion I have included a downloadable package (tstick_score_template.zip or see Software Downloads, on the right) containing templates for OmniGraffle, Word, Pages, Finale and basic JPG files that can be used with any graphics editing software.  I have also included an additional Finale document containing many of the standard music notation symbols used in previous prompt scores. Typically, I create and then export these symbols from Finale, using them in OmniGraffle to assemble the final score.

If you like this way of notating, but you want to use another application, please use a page size of:

405 X 800 (pixels)
=  14.29 X 28.22 (cm)
=  5.626 X 11.11 (inch)

In addition, remember to lay out your symbols and graphics from top to bottom and that each page must be placed ‘below’ the previous page (read from top to bottom). The final version of your score must be an image file that is readable by Max/MSP and also a simple text document containing the time cues for each page.  The time cues should represent the ‘running’ time of the piece (i.e., cumulative time).  Do not concern yourself too much with the formatting of this text document.  However, the following format would be useful:

1, 1 scrollwork_page;
123, 2 scrollwork_page;
137, 3 scrollwork_page;

The first number of each line represents the cumulative time (e.g., 123 = 1 minute, 23 seconds).  The number before “scrollwork_page” is your page number.

Musical notes and thin vertical blocks on the staff indicate an approximate placement of single fingers (traditional note-heads) and hand grips (vertical blocks) on the t-stick. The range of sounds is variable and depends upon a musician’s control of timbre, which is indicated by t-stick tablature grids located above the staff. I speak more about the t-stick tablature  notation in relation to Examples 2 and 3, below. A slash through a note-head specifies a thrusting or jabbing motion with the t-stick and consists in: (1) selecting hand position; (2) tilting and rotating the instrument (and one’s own body); and (3) applying a proper degree of force not only in the direction of the jab but also to grip pressure. An encircled ‘X’ below the staff specifies a technique known as a ‘thrust-sustain’, which is an adaptation of the jabbing technique. The thrust-sustain requires a minimum of a 0.75-second preparation time during which the performer must maintain a consistent degree of pressure (on the pressure-sensing side of the DMI) before executing the jabbing movement. The result may be anything from a series of sustained cacophonous bell-like tones to a brittle and woody bubbling, depending on the degree of pressure used. Changes in volume are traditionally notated with standard dynamic symbols:  ƒ  ,  p , crescendo, etc.. In addition, the lv symbol, which is a standard mark for percussion music, is found above the staff and specifies that the sound of the t-stick be allowed to resonate.

The second component of t-stick notation concerns a graphical software interface for displaying a type of dynamically-changing tablature system. I invented both the interface and the tablature system. Generally speaking, the timbre of the t-stick results from both tilt and rotation; however several other factors concomitantly contribute to the resulting sound (e.g., degree and location of surface contact, pressure applied to surface). Symbols (Example 2) appearing on a computer screen and above the staff (Example 3) inform the performer about the current tilt and rotation of the instrument, as well as approximate contact positions (i.e., hand positions). In Example 2, we see three tablature grids. The circle and star contained within each grid correspond to control parameters of multiple synthesisers; the circle is related to one instance and the star, the other. The shaded top left corner of each grid has been used in previous versions of the on-screen interface. The shaded corner can be automated so that it moves from square to square.

During a performance, the grid elements (i.e., circle and star) shift up and down and from side to side corresponding to the physical handling of the t-stick. For instance, tilting the t-stick moves both elements horizontally. The star moves vertically as a result of rotating along the lateral access of the DMI while hand width, combined with hand position along the surface of the DMI, controls the vertical positioning of the circle. During a performance, a musician reads the notated tablature grids in the printed score along with information written on and below the staff. Next, he or she manipulates the t-stick in order to match the on-screen tablature to the notated grids. For instance, the three grids shown in Example 2 correspond to the notated musical score grids of Example 3. Furthermore, dotted lines appear between notated grids in Example 3 and indicate a gradual change from one grid to the next. T-stick playing technique, therefore, requires one to have a swift and accurate grasp of the tablature system so that one can smoothly shift from hand position to hand position while fluidly rotating and tilting the instrument.

One further symbol shown in Example 4 needs clarification. Throughout the development of the DMI, I found notated indications for t-stick orientation to be necessary (Example 4). While composing, I continued to use them even though some similar information was already conveyed by the tablature grids. From my experiences as both composer and performer on the DMI, I have observed that these orientation symbols provide a simple and coherent means of conveying basic tilt and hand position information. For instance, the first symbol of Example 4 specifies holding the t-stick upright (i.e., the top of the t-stick pointing upward) and vertical with the left hand on the bottom and the right hand on the top.

I have been working on a t-stick sound vocabulary that integrates well with winds – wood wind instruments to be more precise – capturing not only sustained ‘wind’ sounds but also a wealth of extended technique sounds (e.g., key clicks, tongue rams, flutter, air noise, etc.). In this video, I hope you will hear various wood wind instruments and also some of the more contemporary extended technique sounds.

For these extended technique sounds, I initially experimented with audio files – recordings of techniques played by actual wind instruments. I was not happy with the result seeing how sampled audio can lead to a very static aural experience especially when the samples are treated with very little processing, colouring, effects, etc. I find it more interesting to listen to, say, tongue ram sounds on an actual clarinet rather than replicating a single ‘ram’ sound over and over again.

My solution was to implement several physical models – my usual approach – that emulate or evoke extended techniques on wind instruments. This allows me the added bonus of being able to evoke an extended technique sound (when played with one particular gesture) and then to alter the sound quite dramatically (with another gesture). What pleasure!

In terms of gesture, we are exploring movements that co-ordinate well with David’s musical concepts of rhythmic unity.  For instance, his composition will involve a set of gestures that when combined, stabilise or bring a sort of periodicity to a bank of glitch-influenced sounds; t-stick movement will increase or decrease ‘metricity’, as David is calling it.

During the second half of our meeting, we explored shaking as a means of controlling metricity. I shook non-stop while David looked over the t-stick data streams coming into his custom-built synthesiser.  We made an effort – exerted an effort – to map energy exertion to metricity.  The harder I shook, the more stable the sounds became.  After some reflection, it seems that the ‘inverse’ may make more sense; the harder the shake, the less stable the sounds become.  Work in progress, as usual.

Henry has laid out some very interesting challenges for us to overcome.  He’s exploring a concurrence with gesture and sound spatialisation in his t-stick composition.  In addition to initiating and modulating sounds, he would like the t-stick to control the localisation of these sounds in the context of a multi-channel work.

I am reminded of a comment I received from a former teacher.  He appropriately speculated that the effectiveness of the t-stick may be diminished if I ask the stick (programme it) to do too much.  He was responding to ‘one’ of my definitions of the t-stick as an agent of both electroacoustic music synthesis and diffusion.  In short, if I use the t-stick to create (initiate and modulate) sounds in an interesting way via effective gesture-to-sound mappings, will I also be able to use the t-stick to simultaneously diffuse my sounds in a controlled and meaningful way?  Perhaps too much of one thing (either creating sounds or diffusing sounds) diminishes the other.  For instance, it’s very useful to be able to develop mapping strategies that seemingly allow the initiation of a sound to be separate from its modulation;  or that one can initiate new sonic events while at the same time modulating a presently sounding event – permitting the possibility of polyphony.  What happens to my control over polyphony if I also have to concern myself with controlling where the sound is placed in the performance space?

To begin problem solving the challenges posed by Henry, we implemented some Spat (http://www.forumnet.ircam.fr) examples that he had prepared and brought with him for the workshop.  Because we are continuing to use the extremely useful DOT Mapper software to convey t-stick gesture extraction data to synthesis or in this case, Spat parameters, we first had to write up an XML document that declares the Spat parameters to the mapper.  Second, we included the necessary dot.admin Max object and the message routing in Henry’s example patch.  This part of the process was swift and yielded immediate results.  Third, we made some preliminary mapping tests with a five-channel system provided by the EMF.  We primarily experimented with ‘throwing’ the sound to specific loudspeakers through rotational movements of the stick.  This lead to a data recording segment, during which I recorded the t-stick datastreams of specific t-stick gestures (recorded using my Datastream Max patch).  Henry can now simulate some of the t-stick movements in an effort to see what gestures may be effective spatialisation gestures.

Throughout the workshop, I believe Henry and I were contemplating the task of simultaneously playing and diffusing sounds – which I mention above.  To this dilemma, I added a former proposition.  I have suggested to Henry that he concerns himself solely with the diffusion aspects of his piece.  This isn’t to say he would develop his spatialisation regardless of the sounds of the t-stick; rather, I have proposed he uses sounds and sound-to-gesture mappings that I have ‘already’ established for the stick. Consequently, the creative challenges of his work may be more restricted, and I would say more limitations in-play allows the imagination to wander more freely.  Plus, I’d be happy to illustrate how a fixed voice for the t-stick can be used to compose ‘different’ works.

I was content to see/hear Bryan’s progress and to find out that he’s being consistent in his compositional project, which explores the separation of the human performer from his instrument amid a soundscape of vocal utterances.  I’m wondering what the final theatrical effect of his composition will be.  For example, how will the audience perceive the clearly vocal sounds in relation to the stick?  Will the t-stick appear to declaim – a ‘talking stick’?

Bryan started the workshop by showing me his approach to synthesis.  He has been designing a synthesis engine in Max/MSP.  It combines two sets of tools for granular synthesis (the Granular Toolkit 1.49X and Smooth Overlap Granular Synthesis or sogs~), in addition to a groove~ playback generator.  Next, we wrote up the necessary XML document for the DOT Mapper.  That is to say, because I am using the mapper to convey t-stick gesture extraction data to synthesis parameters, it was necessary to generate an XML document that declares the synthesis parameters of Bryan’s patch to the mapper.  After that, we explored gesture-to-sound mapping strategies for creating some basic playing functionality:

-  Initiating a sound, emitting a vocal utterance
-  Articulating a vocal utterance, short staccato or long sustained sounds
-  Modulating the loudness of an utterance
-  Modulating pitch and grain quality

With some preliminary mappings in place, Bryan played the t-stick while I recorded datastreams coming from the instrument.  These streams will permit him to replicate playing gestures, via my Datastream Max patch, without having a stick.

We’re still roughing out the staging for the composition.  Bryan’s initial idea was to present the t-stick as a table object – placed on a table.  As I understand the theatrical concept, Bryan wants the possibility of rolling the stick on the table and also letting it passively rest in place without any manipulation by the performer.  We are running into some physical constants, though.  My own t-stick has curved end caps that interfere with rolling.

It occurred to me that there is something of New York City in the t-stick, although this impression may be the result of the new sounds and sound-to-gesture mappings on which I’m currently working. I’ve been using physical modelling to develop sustained sounds that I hear as quite compatible with the timbre of woodwind instruments. In addition, I’ve been using abrupt rotational gestures and shaking to access a wide ‘wide’ variety of audio samples. Patrick Hart was kind enough to lend me a large collection of sounds after laying them out in Logic’s EXS software sampler. I found it quite easy to import the sampler into one of my own projects and to begin playing/manipulating the collection of sounds. Briefly, with my sustained and full-bodied wind-like timbres, I clearly imagine both the endlessly stretched avenues of Manhattan and the constant intensity of the streets (e.g., the quantity of people, inexhaustible and contrasting movement trajectories of city traffic, also sound trajectories and smell zones). With regard to the large array of audio samples, I sense a metaphor that compares the diversity of samples with the diverse origins of New York City’s inhabitants.

In these presentation-demonstration situations, I don’t often remember to address the point of computer-assisted performances. I am sometimes under the impression that my audience may believe I’m controlling a set of preprogrammed sequences and that the timbral subtlety I strive for is only the result of organising a set of studio-designed audio files. I would like to assure everyone that the sounds ‘flow’ from my instrument as they would resonate from a normative acoustic instrument – this takes some explanation and defining of the word ‘instrument’. Nonetheless, I wish to state that I do not control predesigned computer sequences and for the most part, the sound you hear is entirely a consequence of my real-time manipulation of the stick.

In the case of triggering (predesigned) audio files, I am constantly exploring new methods of handling the continuant segment of a triggered sound. In my opinion, the terminology ‘trigger‘ implies a rather static duplication of audio. My colleague Joe Malloch recently made an insightful suggestion on terminology. Instead of speaking about triggers, we should consider envelopes. The common use of ‘envelope’ in acoustical foundations is the ‘amplitude envelope’ (i.e., ADSR). Enveloping, break-point filtering or function multiplying are not limited to modulating amplitude – far from it. So in Joe’s statement, I infer two things: (1) a sound that the t-stick initiates, for example, may have a controllable and malleable continuant and decay structure; (2) the shaping of the decay structure can be a result of modulating amplitude in addition to a host of other synthesis parameters. There should be nothing static about playing the t-stick.

Software setup for “power of one” – How to set up the t-stick and mapper and load mappings
Datastream record & playback – Description of patch for playing back t-stick gesture data
DOT Mapper – Introduction to the mapper (e.g.,  making mapping, loading, saving, etc.)

Software setup

Follow the instructions shown in the video, entitled Software setup for “power of one”.  You can open my programming for my piece, Everybody to the power of one, but later you will obviously want to open your own ‘performance patch’ and also your own synthesis (e.g., in Logic, Battery, Max/MSP).

To facilitate a move to your own programming, instead of opening my Power of one ‘performance patch’, open up the generic workshops performance patch, entitled workshop_performance.maxpat.

In the T-Stick Repository, go to:
MaxMSP Components/performance/workshop_performance.maxpat

This performance patch contains the t-stick programming and also my signal2midi patch for converting data from the t-stick into standard MIDI messages.  In addition, the performance patch allows you to call up an instance of DOTmapper in order to make, load and save your own mappings.  It also allows you to open an instance of Datastream for playing back (emulating) t-stick gesture data.  Click on “Mapper” and “Datatream” in to top righthand corner of the GUI.

In addition, you may eventually want to have more editing capabilities over the t-stick Max patch, in which case you will have to open up individual patches and initialise them, instead of opening up a ‘performance’ patch.  Contact me if you want to try this option.

Video and Datastream file

The main reason for posting this video, and accompanying datastream file, is to give you some material to work if you want to emulate t-stick gestures using my Datastream Max patch.  I hope you will be encouraged to send me your gesture ideas.  In turn, I can record the datastreams of these gestures and send them back to you to work with.

This short 35-second video clip illustrates a few simple playing techniques.  The playing techniques in the video are:

-  Framing and then fingering a couple of ‘notes’ with the right hand
-  Tilting
-  Releasing the ‘notes’

Click on this still to play the video in a new browser window:

Crtl-click and “Save linked file…” the accompanying Datastream text file:
datastream_test_video.txt