Jason graduated from
Yale University with the music and

then received his master's degree in
composition from Columbia University.

Music has been performed by groups such
as the American composers orchestra and

the SO percussion group and his
interactive installations and software art

has been exhibited at the Lincoln Center
Festival in New York the Boston cyber art

festival and has been featured in The New
York Times and on National Public Radio.

We're very lucky today.

Jason is going to talk to us
about composer performer.

Thank you.

Thanks very much.

Laura I really appreciate the invitation
to come and speak here today and

also thanks to Christine for
facilitating this well I mean

talk about a few different pieces
of mind today that show some of

the sort of ideas that I've had about
music and its creation performance and

dissemination over the last few
years of the before I do that.

I just want to make a few observations.

Just share a few right.

Thoughts with you that I've had recently
and the first one is that I kind of

suck and
I mean this in a very specific way.

I think I'm actually a fundamentally
a pretty decent person but I don't really

subscribe to this whole composer is genius
idea you know I went to Laurie allude to.

I went to some very traditional music
programs to receive my education.

But this idea of the composer is genius
is one that really kind of difficult for

me to stomach and I'm not the only one who
has trouble with this in French Schubert

said famously you know who can
compose music in the shadow

of Beethoven Actually I was in Vienna
a number of years ago at the central fried

off where thirty two B.
all the famous composers are buried there.

It's actually quite unusual all
the composers are in this one area of

the cemetery because they moved the seven.

Out of town they wanted people to
be like OK with being buried there.

So they moved all the composers there.

So they're all right next to each other
and literally like this you know this

statue coming up above Beethoven's grave
the day I was there cast a shadow on

the Schubert three of so so so
this is not just figurative it's

also quite literal and more recently
composers like a Terry Riley.

For instance one.

You know composers music I admire
greatly American who was one

of the sort of founding fathers
of minimalism and music.

So I'm not good at telling
people what to do.

That's sort of what a composer was always
supposed to do I joke to people on the ask

me you know what I do
I say I'm a musician.

So what do you play and I say I don't
really you know play an instrument very

much anymore I used to play
saxophone I used to play piano but

now I just tell other
musicians what to do.

That's sort of what a composer does but.

It's a little weird here because
this is sort of how I think progress

is there's like this moment of divine
inspiration or something that comes down

into Beethoven's brain and he like
translates that into this score and

then performers translate that score and
a sound that we listeners hear sitting

in the audience but this is kind of
a there's a separation here between what

the composer is doing what the performers
are doing versus what the audience or

is doing these are fundamentally very
active activities writing down music going

to piece of paper playing your instrument
to interpret it in performance and

the audience does it can be a fairly
passive activity you can just be sort of

sitting there listening on your i Pod or
listening in a live concert hall or

wherever hearing this music and you can
be very engaged in that process and

you can actively try to come to an
understanding of what that music means to

you but nothing that you do
in that sort of process of

listening ends up affecting
what the composer do or

what performers are doing you know that
is a story I sort of always tell is.

How many of you have been to the orchestra
concert in the last five years

a lot of you.

It's great.
I like orchestra concerts too but

it was a little funny in fact there was
just an article in a web blog I read I

read a lot this morning that I
was looking at and it was yet

another tirade against people
coughing at a concert.

OK And you know they hand you cough drops
on your way in to try to limit this but

you know someone coughs or sneezes or

something everyone kind of looks at you
and it's like you know who do you think

you are to interrupt this sacred
moment in music history and that sort

of just hints of the dynamic here that
you're a spectator when you're coming.

You're witnessing something
happening on stage and

something really special and amazing.

But there's this barrier is sort of
between what's going on on stage and

what's happening with your experience
of taking that in what you do unless

you sneeze or cough or
something like that or yell fire I

don't know what doesn't really go back and
affect what's happening on stage.

This is particularly so in classical music
in particular orchestral performances

you think of going to a jazz club or
something.

It might be a little different.

You applause you're talking to people.

There's a real kind of vibe in the room
the performers may pick up on music

we've kind of shot away from that a law.

So anyway as I said I don't
get divine inspiration.

And I'm not a genius.

So it's kind of hard for me to.

So you know to look at that whole.

The whole chain from composer to
performer to listener in the way I just

put it up on stage.

So it's crazy to observations too
which is that you may not suck I

don't actually know I only know
like three people in this room so

I really don't want to jump
to conclusions here but.

I'm a pretty optimistic person I
think that most of you are really

creative that your musical even
if you claim that your tone deaf.

You have some innate musicality.

And you have something to contribute to
musical experience other than sitting

there of worrying that you might cough or
sneeze or that you might applaud between

movements when you're not supposed to or
something like that so any.

Way The basic idea that's driven a lot
of my work over the last seven or

eight years is shouldn't we try
to make some music together.

Shouldn't I try to include
listeners in the compositional

process in the performance of processes.

So we take this kind of
feed forward network.

Just hit OK there.

I'm not sure what just happened.

I'm not even on the internet but
it's giving me an error here.

There we go.

Thank you so take this process of
going from composer to performer and

audience and
two hundred more into a feedback loop.

You can imagine during a concert music is
being created performers are interpreted

not music the audience
is responding to it and

then somehow their response is feeding
back into the music itself it's shaping or

affecting the music through
what the audience is doing.

And says word Georgia Tech here.

I'm sorry I have one more slide first.

So this all goes back to one
of my heroes American composer

named John Cage how many of you have
heard of John Cage show of hands.

What's John Cage's most
famous piece anyone.

Yes it's a piece called four minutes and
thirty three seconds and

the pianist comes up on stage and
sit down and.

Then after Foreman's thirty of his aims
they get up to about it and everything so.

So you can thinking this is a joke.

You can think of it as an extreme
statement about what music is but

if you think about in terms of this
cage quote I think it's going.

Another thing so he was composing
is one thing performing is another

listenings the third what can they
have to do with each other and

for Cage these are very distinct processes
as a composer in that piece for example.

He's creating a framework within which
to experience sound the performer

is I guess executing that framework but
then your experience as a listener is

to just be there in the space
instead of listening to.

The pianist is playing said
Listen to what you're being told

you know this is what's important.

I'm walking you through
this piece step by step.

Here's the introduction.

Here's the development section here's
the recapitulation so on and so forth.

You're listening to that and

the sounds in the concert maybe there's
an ambulance driving by in the street.

Maybe the air conditioner or

in this room maybe the projector is making
a lot of noise or there is something

coming out of the fluorescent light
bulbs at a steady frequency and so on.

If you you pause from and
you listen to all these sounds.

You can try to interpret them and

make sense out of them as a musical
composition in and of themselves.

So that when.

Cage is really trying to do is to in
a lot of ways empower his listeners

to develop their own idea about a piece
of music their own interpretation of it.

And not to show people this is you
know me walking you through and

this is how it's all supposed to work.

So anyway like I was saying before
the third observation I want to make

is kind of specific to being a Georgia
Tech here which is that this feedback loop

I was talking about is kind of connected
to the way people write software.

And people the way people
use computer software.

So in the speed back loop composer writes
music performers before music audience

reacts to music goes around and
around and around during a performance.

I mean think of a developer write software
which creates output the user response to

that output in some way creates new input
and goes around and around and around it.

Basic level you're using Microsoft word
you type in the letter A the letter

A shows up on screen you say OK now I'm
going to type in the next line eventually

you type in your documentary whatever
you can also think of using a graphics

program drawing things manipulating
things in different ways.

Adding plug ins in effects and
so on and so forth.

As you sort of see what
the computer does as you tell it to

execute all these different operations see
how it comes out and sort of shape the.

You know the draw.

That you're making or whatever over time.

There's lots of different ways to think
about this feedback loop where different

has a connection to the feedback loop what
I'm thinking about in terms of law of

musical performance.

So for music the input into this interface
is some kind of musical interface it may

be me singing it may be me running around
the room maybe me waving a light stick or

you for that matter.

It may be taping on a keyboard or
using a mouse.

But some kind of interface it generates
input into the software system

on the software consists of some kind
of real time algorithm that analyzes

that data that's coming in and

it transforms in some way it maps it onto
another Jimmy it creates some kind of

musical output which could be electronic
sound that comes out of speakers.

It could be music notation.

That's generated for
musicians to play on that could be video.

It could be control
messages that controlled

robot like of one of my colleagues a log.

It could be basically anything is
eventually going to lead to the creation

of music either directly or
through speakers or

kind of indirectly through
some other mediator and

the user of the software
is you as the listener and

audience and a group of people
in a lot of situations as well.

So it's sort of how I'm
thinking about all of this.

That's a good question.

I'm not even sure sometimes if
composer is the right word anymore or

software developer or interaction designer
or something is a better term and

I think that varies a lot
from piece to piece.

In this sense the developer who
write the composer is the one who

writes the software.

So you can think of like something
this is what's happening during

the performance before the performance the
composer created that software instead of

creating a score.

They created the software.

Yes.

Yes So there's definitely a sense
of a shared authorship here.

Between whoever develop software or
whoever's using it.

And again it's kind of a continuum
in some cases the authorship may lie

primarily with the software developer
slash composers or you might think of it.

Traditionally and sometimes it may
lie almost entirely with the users.

I mean you know no one would say if you
made a photograph in Photoshop that.

It was created by the people
that wrote Photoshop.

It was created by you.

But you can think of.

Some other interfaces where you don't
have nearly as much control as a user and

much more the program is
exerting itself on you.

There's a wonderful program called all
those shop by this software artist named

Adrian ward and it kind of.

It makes it very very hard to use.

It looks like Photoshop on the surface but

as you start putting things in
like you click the text object and

you go to put in text and you can
actually control what goes in there.

It's to cast a CLI generates the text in
the phone and all this other stuff it.

You know when you insert like
a selection rectangle or something.

It takes something from your desktop and
it puts it in there.

And sometimes I'll just start you know.

Generating plugins and doing processing
and stuff in it's own way you have no

control over it and just like
mangoes you're drawing beyond any.

Any point of recognition from where you
started or where you intended to go.

So I said bugs that crawl across it and
things.

All kinds of crazy stuff but

a lot of artists actually like to use it
because it helps them generate stuff.

They might not have thought to do
otherwise they can sort of extract it and

work with it.

So there's a very problematic
authorship situation because

the software is making it difficult for
you to do what you want and

really exerting its own control over you.

But you're still having
some influence of the much

more of a shared authorship there.

So the other is a really
complicated question.

So what I'm going to talk.

About today are three
recent pieces of mine.

And the first one of the piece called
Glimmer that I wrote in two thousand and

four is actually my doctoral
dissertation and it was for

a twenty five player Chamber Orchestra and
audience participation.

Using late sticks.

The second piece I don't talk about is
one that was just premiered in December

called Flock it's for a Saxophone Quartet
audience participation dancers.

Electronic sound and video and
the final piece on talk about is called

Graph Theory which has a slightly
different direction moving away from

live performance moving
towards online environments.

And as for solo violin there's a Web
component goes along with that.

So that's sort of where I'm going today.

Are there any other questions or anything
anyone wants to to say before I move on.

OK but feel free to interrupt me anytime I
have a tendency to talk kind of quickly.

And I know these pieces very well.

So if I leave out any critical details and
you're confused or

you just have something you want
to ask just interrupt me any time.

So whoever or chamber orchestra audience.

Is take this idea of this feedback loop.

I was talking about before and
realizes that in this way so

that everyone in the audience is given
a light stick just from these handheld

battery operated light sticks like you
know you're used to seeing around that

people wave around and
they make funny like patterns and stuff.

So everyone gets one of those and

they wave them back and forth during
the course of the piece there and

I'll get into more details about
what that actually does in a minute.

There are four video cameras in the hall
that capture images of the whole audience

and a computer run software
that analyzes the video images

to extract information about what those
audience members are actually doing with

their light sticks basically looks at
different groups of audience members and

figures out how quickly they're
waving them back and forth.

Then that.

That then gets mapped onto music notation
of sorts for the orchestra to play.

Each musician has a little
colored light to go

on their music stand this computer
control in the color of that light and

the brightness of it tells them what
note to play and how loud to play it.

Is also a little video projection that
helps the audience figure out what

is going on.

So the orchestra response to the lights
they don't really have a score that they

play off of they just responding to
the lights the audience hears with

the orchestra plays they respond to each
other they respond to video projection so

on they change the way that
waving their light sticks and

we begin in other iteration
through this whole loop so

I want to go through each of these
stages in a little bit more detail now.

The light sticks of there
are four inches long.

They're actually marketed as a novelty
cocktail stirrers like it was to put them

in a drink at a bar and their lights
up the whole drinking is cool but

they were the cheapest
thing we could find.

So three years it will action shot
here people waving them around.

So everyone gets one of those and
I'm to divide into groups.

We've done this performance
a couple different times and

the sizes of the audience
are varied a lot or

six hundred people when we
premiered it in New York.

There were fewer when we did it
in Israel a couple years ago but

basically we divide the audience into five
six seven groups something like that.

So there's around fifty sixty
people in each group and

each group serve alternates
what color sticks they have so

there's a blue group over here in a red
group you've heard so on and so forth.

And then we divide the orchestra
up into groups as well.

And actually orchestra sits
in a very unconventional way.

Now with these clumps of
players around the stage.

Instead of traditional seating in
a semicircle or something like that.

So there will be the first violins will be
sitting there in the second violins will

be sitting there and
there are another third by the violas and

the cellos in the double.

Bases.

You know and so on and so forth.

So each group is kind of this clump of
three or four musicians sitting together

and they're linked to one of
these groups of audience members.

So what happens from there.

And I like I said the organ
each orchestra musician

has a light and every musician
actually plays four different

notes over the course of the piece
it's pretty minimalistic music but

the color of the Light tells them which
note to play and the brightness tells them

how loud to play it and there are some
flashes that tell them when to accent or

you know get ready to play and
stuff like that.

But this is basically all that they
get in terms of a printed music.

It just tells them how
to interpret the light.

So to speak briefly about how this
gives it generated by the software.

We can talk about more later.

If you're interested but they're basically
two ways of the they're waving good mapped

to to the notes of the musicians play.

One is this kind of direct mapping so
everyone in the group waves their light

sticks really fast back and
forth that group of musicians will play

much louder and if they slow
down they'll play much softer.

And then there's also this
kind of competition that

goes on among all the audience groups and
so.

The problem that I had when I
was in design is pieces of you

want your group to be loud so
you can hear them and everything so

everyone is going to wave their
light sticks all the time.

That's not going to be so interesting.

So I want to groups to be able to
work together to sort of change

what their group is doing over time and
so look at the derivative is basically

the change over time in this
light stick waving a balance and

when groups are successful in
changing as a group so everyone stops

then everyone starts then everyone stops
again that kind of thing they're rewarded.

And when they don't do such a good job of
that they're they're basically punished.

So what do.

Happens is when your group.

When your group has run these
dramatic changes the musicians

change the note they're playing your group
the scaling of the dynamics gets higher so

they become louder relative to all
the other groups and so on and

have doesn't do so well they get quieter
they don't change what they're doing

as much and they also eventually
drop out of the piece all together

on the video clip of players from the end
of the piece where one by one the groups

kind of get knocked out of the piece
to this one like winning group left and

you hear people are kind of screaming
at each other and getting in here and

stuff so that's the basics of
how this math thing works.

So here's a shot of
the the light that the musician

each musician has on their music stand.

Giving them cues and here's a video so
if you wouldn't mind

clicking on the video on this is just the
last couple minutes of the performance.

The boy with

the

new on

the be the

was OK you get the picture.

So I kind of like a fighter
having my accident.

You know when musicians light
goes on that the stage is dark.

Otherwise it sort of lights them off and

you can see the instrument as
a airplane engine It's very pretty and

performance that doesn't really
come across on the video.

The premiere was by the American
composers orchestra of the commission

the piece in August in New York.

That is far as you might imagine dedicated
solely to performing music of American

composers and they their one of the
ensembles in residence at Carnegie Hall

from your residence and Kohol which is
the new undergrounds face at Carnegie.

And there were six hundred people in
the audience there twenty five players

on stage.

The video screen there.

It was showing the the audience basically.

As each group waved their sticks faster.

There was a box representing their
group and it got brighter and.

Also showed them how their competitive
element was doing he was in first place.

He was moving stuff like that the other
performers were gay was in Jerusalem and

it was with in a much smaller space
it only had about two hundred

two hundred fifty people and we we had
fifteen musicians who were students of

the conservatory in Jerusalem play at
that show any other questions before and

move on the yeah the music is different
every time I mean it's it's different.

Within a range of possibilities.

So you know the overall arch of it
remains the same because there are some

structural things and some harmonic things
that I've written in the software but

the specific notes
are different every time

then

yeah and

it has the same problems in
successive some democracies as well.

Yeah

and that's what I'm going
to be getting to you

more in the next thing that I
think you're definitely right.

So there are some great things about
these performances in this piece.

I thought people had a lot of fun and
I think as you probably got that out of

the video they were screaming
cheering it and so on.

They got really into this competition this
year which was kind of unexpected on my

part I don't know why I
should realize that of and

they did some creative things in
New York people did the wave with

their lights take sort of hearing
him back and forth in Israel.

People just took charge
you know they would stand.

Updates started yelling at everyone
in their group stop start so and

you could really see you know
some coherent group collaboration

New Yorkers are not as good that don't
know what that says for the but anyway.

So yeah there was some good stuff here.

And I thought visually it was interesting.

A lot of the time to see the stick
movements and see the lights on stage and

all that going on.

And sometimes good music came out of it.

There were moments I thought
were really beautiful.

There were moments I thought
were not as beautiful as you.

But that's part of the.

The risk.

I guess but it was not perfect of there
were too many people in this whole thing

six hundred people in the audience
twenty five musicians on stage and

there was too little time my
contract with the orchestra when

they commissioned this was that the piece
could be no longer than ten minutes.

So it was exactly ten minutes but you know
I think that six hundred people are going

to walk into a concert hall and I explain
to people how things work for five or

six minutes first and then we're
going to play this piece together and

have ten minutes to figure it all out and
practice it in no practice and

expect great music to come out of it.

Would be kind of I leave and

you had a point here was more that it
was about being part of the process than

creating a piece of music that you
don't want to like so on a CD.

It was about being part of this and being
a part of the experience and shaping what

was going on building this community with
your peers but given these constraints.

Unfortunately you know
I didn't think that you

know the product of what came
out of it was as good as it.

It might have been and
the other constraint was that a orchestras

are unionized and I'm not saying anything
for against unions but this was a problem

for me I had one hour of her soul with
the orchestra and that's not unusual.

That's how you know you get a piece
premiered by the Atlanta Symphony.

It's probably a similar situation.

And the orchestra musicians came
from a classical background and so

I couldn't really ask them to improvise or
do anything to really you know put their

own interpretive stamp on this getting
back to what you were saying before what

if someone started collects all this
feedback and does something really.

You know interesting to come by.

It's kind of intuitively or musically Well
you've got twenty five players on stage.

Who are used to just playing
the score in front of them and

you know one hour of rehearsal.

It wasn't really possible.

I didn't think under those
circumstances to tell them to

really you know be creative
in interpret ing so much.

What was coming through or
turning it into something different.

So they played just the notes
that were in front of them.

And so there was that limitation there
they didn't really have a chance.

So much of a chance to respond in this
feedback loop the audience was responding

to the music the orchestra was really
just creating the music they want so

much responding to the audience.

It was just the algorithm sitting
between them was doing that.

So you know there was good and

bad about this piece of the I
thought that I could do better.

And so right around the time I got to
Georgia Tech two and a half years ago I

got a commission from Big Art Center in
Miami to create a full evening piece for

them and what I came up with
was this piece called flock.

For Saxophone Quartet electronic sound
video dancers and audience and Q.

and a half years later it finally
premiered in December big sigh of relief.

So I want to show you actually I haven't
really shown anyone this piece I think is

the first talk of you in about
this piece since it premiered so

this will be kind of fun.

So the basic idea here is you've got
the sax on quartet go for dancers and we

actually worked at college dance students
in Miami is the dance of the saxophonist

were all great jazz musicians taught at
University of Miami an audience of one

hundred people an hour for the performance
approximately So in C. already I'm.

Trying to play around with some of
the limitations from glimmer here

working with far fewer musicians fewer
audience members and a lot more time and

I chose saxophonists because saxophonist
usually have a background in Cosco music

and in jazz and so
they're really good at improvising and

doing sort of weird things and much more
open mind say violinists you know just to

throw out a broad stereotype there I don't
want to offend anyone in the room but of.

So if that was in the reasons
I work with saxophones

There's also the instrument I used
to play back when I played so it.

I've always had a soft spot for
her saxophone and Saxophone Quartet.

So all the way the venue was set up.

You can see from this diagram it was a big
room about the four times the size of this

room maybe it was an experimental black
box space space of the art center and so

we took all the fixed seating alum and
we put chairs around the edges.

So the audience sat sort of in this this
big rectangle on the edges of the space

there were four big projection screens and
the four walls and then.

Everything in the middle
was the performance space.

And the reason for this is that the
interface here was not waving light sticks

around there something like that but
was actually getting up and moving around.

So we tracked the locations of people as
they moved around the space the performers

the dancers and the audience members and
use that to generate all the music and

video that was going
on in the performance.

So if you look at this kind
of feedback loop here.

The audience is living around location
data is collected by our computers out

about where everyone is in the given time.

It maps that on to
performance instructions for

the musicians who didn't interpret that to
create music the audience hears the sound

changes what they're doing and
removed this feedback loop.

And how it's a lot more complicated
in that there's all kinds of

other connections between people.

There's a visual element people
are responding to the eyes phones to each

other so on and so forth.

The basic idea that feedback loop and
all these connections is still there.

So I walk through these
elements step by step here.

So the first thing is.

We've got to figure out
where everybody is.

And we do that with computer vision
started off working with a wonderful

professor in college a computing friend
to layer a couple of his students here.

And then one of my students in the Masters
of Science program in the other technology

sort of went off running with but
they started out and

we developed this stuff that takes an
image from a single overhead video camera

mounted on the ceiling looking down that
captures a video of the entire performance

space and takes people's takes people's.

The images of people in that image and
figures out through an X.

Y. coordinate of where everyone standing.

So being one direction of the floor

being the other we don't care if
people are jumping up and down or

one person six feet tall on the ones
five foot two or something.

We just want to know where
they're standing or.

On.

The musicians instead of seeing flashing
lights or something like this on glimmer

I want an interface to convey some
more information to them and so

they actually have these Pocket P.C.

P.T.A.'s that are mounted on their
saxophones using marching band equipment.

So when you go see the Georgia Tech
band play at half time we

borrowed a few of their liars and
used a lot of ball current and

just stuck that on so so that way
as the musicians are moving around.

They can see this image of music notation.

I'll show you what
the notation looks like here.

So you can you click on this first.

Image there.

So sometimes the notation looks
pretty straight forward like

this it's not exactly like traditional
notation but you've got trouble club.

You've got no heads and
accidentals and stuff and

the way this works of the musician.

Played the green notes and then the pink
notes are showing what the other musicians

are playing so
they can try to sort of sync things up and

coordinate and the blue scrolling thing
shows them where they are in the measure.

And sometimes it looks a lot stranger
if you can click on this one of

the right here.

Because I don't want the musicians
to be forced to play just the notes

on the page so much.

And so.

Squiggles in different things sort of
develop over time to show gestures and

occasionally there will be a pitch in
there or an accent or something where they

get an exact note for a lot of the time
they're playing more of a gesture.

It gets much crazier than
that actually they are and

you have got one more example
here you can click on this one.

Yes this is a C. on these little pocket
P.C. screens that they're supposed to

somehow play and so you can see.

It's very nonspecific and
it's deliberately nonspecific in your nice

second and third examples
because I didn't want them to

just play what was there I wanted
to force them to improvise and

to use these to get ideas for gestures for
register for a kind of melodic

contours you know rising up falling
down all kinds of stuff like that.

So again they're playing the green stuff
in the pink in the background the show

with the other musicians are playing and
we use a lot of different strategies

to generate is based on the position the I
was going to to much detail right now but

basically you know sometimes you
just say well this axis is time

you know position within the measure this
axis is pitch someone standing here.

Well there's a note here.

And then sometimes you build up crazy or
textures over time like this by saying

what I've not interested just in
where someone standing right now.

Shinn looking at where
they walk over time so

that whole path that they'd of go across
is like a finger painting across this

image and since I view some math things
polar coordinates and some other stuff.

It's based more on proximity of people
to each other and stuff like that.

And there's electronic sound that
is also generated in response to

other positions of audience members
of various points in the performance.

I won't get into the details of that
right now but it's mapped in the same

kinds of ways you know X. and Y.
being positioned in those are in pitch.

And all kinds of other parameters control
aspects of the Tambour that sound

of it and then the big question here was

just how to structure this
whole thing because you know.

My my initial They have the most
basic thing you could do

would be to just say let
people in the room have them.

You know run around and
do stuff for an hour put.

That would get really boring pretty
quickly and I don't think most people

would stay for the hour and a lot of
them might ask for their money back.

So we came up with a lot of strategies in
terms of how to engage the audience over

time and one of those was that we
invited these dancers to join in

the performance and act as kind of
facilitators to get the audience invite

them to participate to show them different
ways that they could move around to.

Group them together in
groups of people and

then they also handed out these
instruction cards about a dozen different

instruction cards they hand out over
the course of the performance to different

people giving AUDIENCE MEMBERS of you
know sort of their own scored a real job.

These are just sheets of paper.

And this is an example of one of them so

they give this to someone who's in a group
of people dancers it kind of created and

you say you move your group
towards a saxophonist so they give

this to this group this group moves
towards one of the saxophone is then.

This card ends up going to
another group that group or.

Towards one of the sacraments and so on.

So there's always this kind
of movement going around and

it's you know it's not like they're
being told exactly what to do but

there's a framework being provided for
them.

Here's another example which is
obviously much more open ended.

They just give it to a group of
people standing in a line and

say you move around with one
foot stay with your group.

So.

There's also four screens of video as you
saw in the that layout of the beginning

and the video tries to take
the data of where everyone is and

you know where people are moving and to
show it in a sort of more artistic way and

I worked with this wonderful video
artist named the uber Boris solve.

Who's a professor at the province
to two in New York and

so he wrote this software
that generates these for

screens a video in the nation real time
you click on this image somewhere.

They'll show you a few things
of what this looks like this

is a white dots here
represent audience members.

There's some Aleck tronic sound going on
in this section which is represented by

these kind of rising
pillars as people approach

them that activates the sea
from parts of the sound.

Here's another example here.

If you click on this.

This is a section where only the saxophone
a certain stage the audience is actually

seated a saxophone This is represented by
one of these colors as they move around

and one of the bubble kind of lights up
as one that saxophone is plays a note.

And here's a more complicated thing
if you could click on this one which

shows the saxophone is in the white
dots being the audience members and

it kind of shows the connections between
them as notes are being generated

by an audience member for
the performer to play.

So getting back to this
question of structure here.

The basic idea was to
start off with very simple

things at the beginning of the hour and
then she moved to.

To more of those total chaos by the end.

So it starts I would just a Saxophone
Quartet kind of in a solo on stage and

the audience is seated and and
then eventually the dancers join then and

serve had their solo moment with the
saxophone is but it's all about service

stablish ing how the mappings work how
you know the piece is set up and so on.

By having his very few number
of people on stage and

having the audience is seated and
watching.

And then this actually does get a well
deserved rest for a few minutes

the audience comes on stage and they're
just generating electronic sounds and

then sort of goes into various degrees
of chaos first with kind of this

structured movement with those cards I
was telling you about and then eventually

moving into more free movement where
everyone is sort of on their own.

To the basic idea of being you know going
from highly structured of movement.

Very simple mappings.

So to.

Much more free movement much
more complicated mappings much

denser musical textures and so on.

So I'm going to sort of go
through this pretty quickly here

in terms of the details of how those
work because I don't know how many

of you are actually interested.

Like I said there's no overhead camera
that figures out where everyone is.

The basic way that we do that
makes it much easier if we

make everybody wear a hat.

Because it turns out if
you're not wearing a hat.

Then you know you cast a shadow and
the camera sees your shadow or

if you're standing right next to someone
then it comes to together into one

giant person and stuff like that and.

You know it works reasonably well there's
ways to try to deal with that stuff.

It was so much easier if we just had
people have that little hot with a little

light on top.

Plus you know people walk in
the ushers give them a hat.

They start laughing it's or
gets them a little more relaxed and

we ask them to do crazy things
later in the performance or

a little more open to it on
the saxophone is all they were

happy to each had was a different color
which held the computer figure out which.

Which saxophonist was which the audience
members of his were White House.

So we do all kinds of filtering in stuff

to take an image like this
it looks very messy and

just have the lights are all that are left
basically once we filter everything out.

So from there you can imagine
it's still a little tricky but

a lot easier to actually
track what's going on

in my student Mark Godfrey wrote some
wonderful software to do all of this.

This is a shot of the software.

Then there's this control
software I wrote and

that's what does the mapping you take
all this data and turn it into music and

turn it into the tronic sound format his
message is to generate this video and

it's got a real time interface that I
can use to actually control everything.

During the performance and respond
a little bit to what's going on to step

through presets but also change all kinds
of printers the algorithm on the fly.

So I don't actually expect to
be able to read any of this.

I don't think it's particularly
necessary but but basically here.

You've got all the data that's coming in
where everybody is where the saxophone

those are and so on in this section
this is harmonic information what chords

are active that piece that the things are
being mapped onto what are the voicings of

those chords and stuff like that so that
actually evolves over the pieces well up

here without the notation being generated
that moment and is sort of sparse or

one of the right of the electronic
sound being generated.

And then all of this stuff here
are all just parameters to control

aspects of the algorithm real time there's
a lot of preset settings that I step

through over the course of the piece where
I was just respond to what's going on.

If things are getting too dense or
they're not dense enough or whatever where

the mappings are making sense can change
things actually have a functionality in

here where I can send little text messages
to the performers that will show up

on their on their screens and
it's turned out to be really useful.

Because A you know like if there's a one
point you know they're playing too loud

and I can't you know the balance against
the electronic sound is all messed up.

I send a message saying play quieter and.

You know it takes them

about five seconds to notice
the message and everything's good.

But then they play quieter and
I say thank you.

In there and we're set so.

So there's a lot of flexibility to play
with this stuff in real time thinking back

to that feedback loop of you know
composer performer and listener.

Well you know the performers have a lot of
freedom here to interpret the notation.

So they're putting a lot into
the audience is you have

a lot of free in the move around and
sounds they're playing on into it.

I want to make sure that I had a chance
to put things into it to use or

tweak this him on earth and

you know and sort of keep tabs on
a during a performance and then just

hit a button that said go and you know
hope that everything would turn out well.

So that's what this
interface is really about.

Yes yes yes.

So as I said these displays
in these Pocket P.C.

issue they're Taishan musicians.

They're just like you know Pocket P.C.
P.D.F. like you'd buy a Circuit City.

Or something like that and

they just receive stuff over why
five years is marching band lawyers.

Electronic sound.

We use a lot of different techniques and
I see there's only one music technology

student here so I'm not going
to speak out on that too much.

We're going to hire up later if anyone's
interested I'll tell you I will say one

thing which is that the sound
comes over eight channels.

So there are eight speakers
around the room and

you know if I'm an audience
member standing here.

It's going to send the mics to these eight
speakers in such a way that it sounds like

that note that's being played is actually
coming from more of this room standing.

Whereas if I were standing that corner
kind of speakers over there and so on and

so forth so
that adds a nice dimension to the music.

And the video animation
I talked about already.

So yeah.

This brings me to the video and
this is actually a kind of montage of

clips from the we did five
performances in Miami in December.

And so these are these
are bunch of different clips

as about five minutes long.

I don't know who played the whole thing or
not.

Well if you can just click on it and
I'll get started.

Sorry

these images are so dark but
it's a dark and the light.

Yes

he's a dancing going on here.

So the music plays actually as people walk
in right before the performance starts

as they're walking to their seats

the space is responding to
the generating electronics.

There's a section with just
the audience dancers on stay

here.
I think I do hear pretty

good sense when I skip ahead.

There are lots of people involved in this
I was going to mention a few of them.

I haven't mentioned already the Adrian
Irish Center for the Performing Arts

is the group that commissioned it was
produced by a local group in my on a call.

I saw that in interdisciplinary sound
arts workshop at the Miami sax far ahead

of dancers from the college division your
old school the arts Mark Godfrey must.

And here in the a group or so.

The professor probably mentioned.

And just to give you a sense
of what was involved in this.

I've never done a sixteen minute piece
before that was actually the scariest

thing for me as a composer.

But we develop this over two years
there were seven different of

Tech students involved with software
development in various stages ranging from

undergraduates their masters students.

There's about twenty thousand lines
of code just in the mapping and

notation software there's a lot more
if you add in the computer vision and

all the other components and

we did for test events at Georgia Tech
over the last twelve months

where we've brought in lots of students
and guinea pigs and we failed miserably.

And we learn from it and
we went back to the drawing board and

changed a bunch of things around and
we're still working on it.

The next performance
is going to be at the.

Zero one festival in San Jose in June.

In San Jose making a lot of changes
between now and then we distributed

feedback forms there on the audience in
Miami and got a lot of good feedback and

we do the sort of informal Q. and
A sessions with all the audiences as well.

To get some some verbal
feedback from them.

So we're making a bunch of changes
particularly not so much the technology

it's all pretty stable of changing a lot
of the interactive structure of what

some of those instructions on
the cards tell people to do.

How many audience members we have on stage
at a time and how the dancers sort of

bring people up and put them away and back
in their seats not put them away put them

back in their seats and organize them into
groups and show them what to do and so on.

We learned a lot about what worked
well and what works less well and and

sort of mean a lot of
changes based on that.

So before I go on are there any
questions about this piece.

I want to show you one of
these briefly to this.

Oaf log so that.

The idea is you know if you're if you're.

I had this experience maybe five or six
years ago when I was living in New York.

And I lived on you know this tree with
lots of tall apartment buildings and

stuff that are you know on the street and

there was this flock of birds and
they I stood there watching them for

like ten or fifteen minutes because
they just you know they didn't move.

You know they did they were moving and

you know they weren't going anyplace
they're just kind of hanging out doing

their thing and you know on
the corner of this intersection.

But the pattern is that they were flying
around and we're constantly changing

everything incredibly complex
relationships between these birds and so

I guess that's why I was
in the thing you want.

I thought a flock was that we're
looking at this you know this group

of people that we're letting loose in this
space seeing how they move around how

their relationships with each other.

Change over time and
using that as a basis of generating music.

Of sorry.

The A.P.
critic who came father was a lot of fun.

She had a good time she had I'm not sure

I'm not sure that she was convinced that
you know this is the future of music but

I'm not sure that I'm
trying to say that either.

It's just interesting
direction to explore this.

It was not as pronounced but
the competitive aspect was basically that

you is one audience member
didn't have a ton of control.

You know you were just one note of many.

So the way to really influence
the music was to convince other

people to follow you.

And so you know if you go for
this one Saxons you generate one note for

them to play basically if you get
fifteen people to flock around

the sanctions are going to start
playing this incredibly dense.

Complex stuff so that was the that.

As a competitive aspect it
was just finding in place but

it was much less pronounced and slimmer
I I honestly I felt like in a glimmer

it kind of usurped
the piece a little bit and

it became too much of the focus and
I wanted to be careful not to make it

not to make it too dominant in
the interaction in this piece.

And your questions.

OK but I want to show you to finish.

I want to show you something
completely different here.

Not completely different but another
question I asked myself a lot is how it

take these ideas I've been
talking about the last hour.

It was outside of the concert hall and
the reason that I ask that is because

I don't know why do you how many of you
have been to a concert in the last week.

Well OK Last month the same people.

OK last six months.

OK still not a ton of it.

How do you view a listen to music and
why some other means in the last week and

pretty much all of you.

Yeah so.

So it's just a fact of the world these
days that most of the music that we listen

to is not in my performance in concert
halls we listen on our i Pods we listen

in a car radio.

We listen on sometimes on our stereos
at home and so on and so forth.

I want.

I've always been interested in seeing
how we can take these kinds of

paradigms are thinking about these
feedback relationships involving composer

followers and listeners outside of just
live concert performance and taking them

into these contexts are people listen
to music on a on a more regular basis.

And so the web is a natural
place to go with that.

So I wanted to show you one of
several projects I've done simply at

some possibilities in this direction
it's a piece called graph theory and

a good old friend of mine a violinist my

a sorority been bugging me to
write a piece for her for.

Several years.

And we finally got the money to do it.

And the money came from a commission for
a piece of Internet art.

So here I was writing
a solo violin piece but

I had to somehow make it a website and
so what I did was.

I sort of build this kind of model
here where there is it's a combination

of a live performance and
a web interaction and so what you have

is you have an online component where I
wrote some software and I worked with

a wonderful visual designer named Patricia
Wrede who designed the interface for it.

And so people use that software online

to kind of explore their own
path through this piece.

So that's the that's the interactive
feedback loop through portion of this

whole thing but then the things that
people do is they use this website

actually change how the music is
performed in concert the decisions people

make online are collected on the server
server continually generates new

versions of the score and when
someone goes to perform it in concert.

They go to the website they hit
print score and they get a score and

they go on a performance and then
the next time someone goes to perform or

they hit print score they get
a different version of the score.

Well based on what people been doing more
recently and they go on they perform that

so there's no technology in the concert
performance of this piece it's just

a violinist and audience the feedback
actually comes to the online interaction.

So there's this kind of
unfinished score that I wrote.

There are sixty one and
usable fragments one measure each.

I was even a little bit so
you can almost see them.

So each of these are little fragments I
just sat down I wrote them on pen and

paper and then I connected them all
together on this graph structure so

each one of these fragments
is connected to three or

four other fragments on the graph.

So as you're going through this.

It's like a Choose Your Own Adventure or
like web surfing or something.

Everything.
It's linked to other things.

So you know there's ways to get from
any place to any other place but

you may have to take some
intermediary steps to get there and

similar things are linked
directly to each other.

That's sort of how I
decide what to connect.

And then there's web interface
that Patricia designed and so

you see that same graph
structure there and it's hot.

You see these visual representations of
each of the Freidman and it's kind of

piano roll style in there Taishan shows
the pitches in the durations and dynamics

and see here recordings of my of playing
each of these and they loop in a loop

in a loop and then on the right there
are your choices of where to go next.

So you click on whichever
one you want to go to next.

And that becomes the next Very where you
can actually click directly on the graph

to internet again.

That way.
You don't like what you did you

go backwards and so on.

So this is what I mean by the kind of
choose your own adventure piece you

know a video of the web interaction here

and

you can actually review the path
that you've taken as well.

And so on and so
that's how it works on the web and

then there's the downloadable score and
since every night the server looks at

all the navigation choices people
made compain different votes

for what order things should be in and

then it prints out the score that shows
all these fragments in a particular order

and that's where the violin is actually
plays in concert and what's interesting to

me is of course it sounds fairly different
in concert than on the web because someone

actually thinking of these fragments in
a particular order and really is your

interpretation and musically I think
click on that little speaker there.

This is a recording out.

Helen came flying this piece
down Georgia State last year

and so on.

So you can hear the difference between
just playing these audio clips and

looping on the website but

I mean really put that together into
a musical interpretation in concert.

So that's a piece I'll just have if
I have a few final thoughts here and

then I'll open up to any other questions
your house on basically you know

getting back to the question you asked
at the beginning of the talk here.

One of the key questions online is what
does it mean to be a composer these days

there is composition the act of
designing software designing interfaces.

Well is it writing a score that people
play through from start to finish or

is it designing an environment for
people to explore and then what

was the emphasis is it creating a finished
product that ends up on a CD this or

this fixed object or
is it about defining a process to create

such a product where the emphasis is
really more on the process of creating it.

And the collaboration between people
involved rather than what comes out

the experience of creating music rather
than the experience of listening to music.

What does it mean to be a performer.

There's a line between sort
of where composition ends and

performance begins and
that lines always been flexible.

So with every piece it's
a it's a challenge for me and

important for the process to decide
what I want to keep control over as

a composer what I want to give control
to the performers to interpret and

what I want to give control
the audience to do.

And you know when does the performer
really become a composer as well or

compositional collaborator rather than
just someone who's interpreter in

the music improvisation obviously plays
a big role into a lot of this as well

from what are people doing.

I've in concert.

How is it being shaped by
the notation in front of them and

what I mean to be a listener.

There is you know we can think of all
kinds of ranges of things from you know

sitting in the elevator standing in
the elevator trying to tune out the music

that's coming through to being at
a concert where we're not doing anything

else and are really focusing on that music
trying to come to an understanding of what

that music means to us as listeners.

So there's this passive to active
search continue in there but

there's also an individual
to social continuum.

So there's that individual experience of
listening to something where it's really

just you know you thinking in your head
versus a more collaborative way of

listening to music like some of the things
I've been talking about today and

there's an obvious tie in to
the way we share music online.

You know if you look at Facebook or
My Space I think it's all about what music

in my listening to what music in my
buying what music do I like last F.

and it is even more Pandora these
kind of places are even more

explicitly built around musical tastes and
recommendations.

One of the things I'm interested in as we
move on law is not just thinking about

sharing listening preferences but actually

sharing collaborative music making
experiences online there are actually some

interesting kind of fledgling commercial
products in that direction and

there are a lot of sort of you know
more weird experimental kinds of.

Things like what I just showed
you with graph there is well and

also how is your engaging with
the music shaping that music

should it be shaping that music and
if so how does that to do that.

So I always want to thank all the people
who made these works possible

glimmer as I said was commissioned by
the American composers orchestra we had

some foundational support
behind that piece.

Flocke was commissioned by the aging R.
Center in Miami.

And we got some support some small small
support from the Georgia Tech foundation

in the G.B.U. center to make some of that
possible as well as a foundation and

in Miami Dade County and graph theory is
commissioned by Miriam performing arts for

its turbulence website turbulence is one
of the most interesting collections of

internet based art online and I say that
as a totally biased person because they

have a lot of my work on it but they're
wonderful people and they do a lot of

good stuff and I have a couple blogs too
if you're interested in any of this stuff

good turbulance dot org And there's links
to network performance blog and network

music blog that have links to all kinds
of other people doing this kind of stuff.

So this point I'll just open up to any
other question than eighteen out of.

Yes.

Who participates.

I know the number of
people who participate but

I don't really know who they
are they don't have to sign in for

a username or
give me any personal information.

And I'm trying to remember.

And I are in the numbers I had
somewhere between five and

ten thousand over the fifteen months or
so it's been around.

It's weird with these web things and
sometimes I'll do a piece and

I'll be here five hundred
people that use it and

sometimes I'll do a piece I'll be
two hundred thousand to use it

a lot of it depends on sort of how hip or
catchy it is I did this i Tunes P..

Isa couple years ago which you know all
of a sudden ended up on these social.

News websites and so like in ten minutes
twenty thousand people use it and so

on and the several crash and and
it all kind of go down but it's

always hard to know what you know what
was going to take off and what is not and

different pieces that's more or
less of a concern to me with you know with

this graph here I don't really care of
one hundred thousand people use it or one

hundred people use it because they can all
have meaningful experiences in tribute.

So it's not really quality.

Question.

Christine.

Not through the online interface
because you made a decision to

record everything as audio files and
so you're just looping through

a prerecorded audio file there's there's
not very much you can do with it.

In the live performance I was by
those can do whatever they want.

And you know it.

We've done.
There's a project I worked

on actually with the with one of
the people in back of the room here in my

network music class this fall where
there's a lot more control over dynamics.

It's a project called fluid and
it's a three D.

interface with your fly
a spaceship through this world and

as you fly through objects you kind of
pick them up and you add them to your mix.

So that kind of more general way of
thinking about through you playing a game

online or flying through space as being
a D.J. and mixing things together and

there you have more dynamic control
because it mixes the level of the sound

file depending on how close you
are to that object and so on and

you can also add effects and
things which which change the those

prerecorded audio files for
this piece it's quite long.

And I'm

sort of like a letter from him

this half of the head

that this just so you

know you can't which I which I allude
to the beginning which is that

this extreme sort of separation
between audience forming and

posers is is unique in a way to
classical music and it's not true.

Nearly so much in jazz performance or
in or I guess it

in a lot of other musical styles as well
but there's two things there for you.

I mean first of all I'm particularly
interested in in dealing with it in

a closet around because that's
the world that I came out of but

also as people are moving more and
more away from live performance and

you're listening to that music on
according to how you how are you

getting engaged at the same
layer How is it going to you.

How do you get that sense of being at one
of these incredible performances where

everything locks into place and
everyone is into it without actually being

there and you have an answer
that I would love to hear.

Yes but it's really just their first step.

At this point I think you
know if I can retire and

feel like I've actually made some
good progress toward solving some of

these problems and continue questions
I'll feel good about my career.

No no that sounds a little
too trendy in the

church

or I haven't because you know we
sell tickets and so it's very earth.

Curve.

No I didn't because you know
we you know we were trying to

get as many people as we could.

So we took everyone but that would be
really interesting to know one thing that

we did ask people on the surveys we give
them out to the Miami performances and

we asked them all these questions and some
of them were they you know on a one defied

a scale or whatever it is to say
how well they're you know how I

guess it they were suppose a grid is
here is here and I am a musician and

I regularly listen to
experimental music and

I haven't done the correlations on
the idea to the other responses but

just from glancing through them.

It didn't seem to have a huge effect
on their responses to their questions.

Anything else.

OK Well thank you all for coming in.

Thanks again.