For almost twenty years we work so
the time flies torture and.

Work three years to work.

Here one of the first rid off.

That was the time.

He was already.

Really you know what if you
do all this reality or.

Your.

Work that you find is.

Very.

Slow to move so you will go or
so you know fifty percent.

Thought we.

Were like the free.

Market based.

Thank you very much appreciate
the invitation to come here and

I was going to get a little more doubts
about myself and one hundred a nice job of

that still showed a couple of slides
anyway also sort of like powers of two so

I was thinking it was about sixteen years
ago that I had this idea of rapidly

exploring random trees for
motion planning and then eight years ago

was exactly when I was here down to
the month giving a seminar here so.

And I was about my book on planning and at
that time this was the first slide I had

and it was the planning algorithms
past present possible future and

one of the futures I was talking about was
sensing and filtering in something called

Information spaces I'm very interested
in that side of the problem because

planning often assumes that you have
a perfect representation of the world and

most of the difficulty just deciding
how to move through that world without

colliding with your robot
colliding let's say and

I became very interested in
where does this information.

Come from and then how to be
minimalist with respect to that

that led me into something called
Information spaces and in two thousand and

twelve I completed a book left
on a small book on sensing and

filtering and then my plan was to
expand that into a larger book

occur while I was on sabbatical from two
thousand and twelve to twenty thirteen and

then I would be talking about that
today but when I went away to

northern Finland for my sabbatical to
work on on this new book on sensing and

filtering I was in over it which is up
here near the top where we're Sweden and

Finland come together it was still only
about an hour and twenty minutes to

the nearest IKEA though because Sweden
is is just right up there so you know so

it's quite civilized a lot
of things you can do.

Yeah.

And so
I was working on this book and then.

Well along came Oculus and this nineteen
year old disrupted my life with

a disruptive technology as they say and
I got interested in virtual reality so

Homer started a company in the summer
of two thousand and twelve and

it was generating a lot of excitement
in the videogame industry and

he built us just prototype system

mainly in his garage tinkering
around with smartphones screens and

memes sensors that are available from
the smartphone industry as well some.

Magnifying lenses like you
can buy at the if you go to.

A.
Where would you go for

that maybe Walgreens or something I
don't know just typical magnifying lens

as you put these things together and
you're getting very close to

a high field of view a very compelling
virtual reality experience.

Just with standard stock components that
was very interesting I had this memory

of virtual reality I was born in one
thousand nine hundred sixty eight so

it was all way back to about my birth when
I haven't Sutherland had the first vision

of this on the sort of Damocles virtual
reality system that he built was this

heavy thing that weights down on you
with a table holding up to four to

alleviate some of the.

Pressure and it inspired a lot of people
but it really didn't take off fully

until into the eighty's Jaron Lanier.

Pioneered the term virtual reality and
a lot of people interested in this idea of

having data gloves and interacting in
virtual reality I was a graduate student

the University of Illinois where they
developed the cave system in the early

ninety's we had one of the back Institute
there where I did my Ph D. and so

I was always kind of close to people doing
virtual reality in some ways after that

time I was a professor at Iowa State and
I had a big virtual reality applications

center there that I was affiliated with
I got talk to a lot of people there but

I wasn't centrally in it but
always kind of paying attention.

Eventually the hype ended with.

Around the mid ninety's especially with
this product called Nintendo Virtual Boy

and that was what industry kind of knew
as the cautionary tale don't do it

again it's doomed to failure so
that was ended up being a big disaster and

mainly I think the reason why after being
so overhyped was that the compost and

technology just wasn't ready yet and
people in robotics can appreciate that

we're always borrowing company technology
from other places you hand or Obama says

to connect sensor they're very excited
right they can do all kinds of things and

it wasn't developed for them is
WANT TO USE IT same thing is it happened

even back in the ninety's when Henrik and
I were playing or off mobile robots.

German company made this thing called the
SIC laser which provided laser scanning

data that was very accurate and very fast
in very dense way better than you can do

a sonar is at the time in a revolutionized
a lot of people's research so

I was quite willing to believe that yeah
the complement to technology could be

right for virtual reality it seemed right
another personal motivation I had was

what when I tried their prototype
back in September of two thousand and

twelve I thought I always thought
virtual reality and this place and

I really started to believe in this that
you could bring people closer together so

my my my grandma and her sister were
separated after World War two and

saw each other face to face twice in.

Years and one of those times was in ninety
three another time was in the seventy's

when I was a kid but I brought my
grandmother to to see her sister

in the in the early nineties she
was seventy three at the time and

I kept I went to Poland in the village
where her sister lived and.

I went back there many many times after
that I never could drag my grandma along

she got older you know stubborn frustrated
didn't get on a plane to all these things

and so I thought would be great
to have technology to make people

feel like their present together
in the same physical space so

maybe they could attend each other's
grandkids birthday parties and

special events and things like that
remotely so I thought there was a lot of

great applications beyond video games
which were the main motivating.

Market I would say at that time
something personal that I found

was a great possibility for
this technology.

So we spent a lot of time trying to
improve the experience of my wife on

your show and I were working in in our
apartment in Finland you can see the white

cabinets and everything's it looks
looks like a part of the world and

we had a U R five robot with what
the censor board on it we were doing.

We did the head tracking code and

a large part of it and also we're studying
the sensor noise and calibration and

things like this by the way
Finland's an excellent place to do

temperature dependent studies of sensor
can you just put it out the window and

then bring it back in and thirty seconds
you have a huge range of temperatures.

So along came the rapid rise of Oculus so
so

we started working for the company in just
a few days after their Kickstarter in.

In two thousand and twelve in September
and then you know at the Kickstarter they

raised a couple million started consulting
for them from Finland work with them for

about six months and then moved
straight to California from there so

that was like opposite ends and many many
ways a huge a shift and joined as their

head scientist and remain in a position
up until the acquisition and this company

just grew at a dramatic pace it was it was
crazy the things were happening so grew

in terms of number of people the amount
of visibility went through a series.

A series B. and then eventually a Facebook
acquisition for a couple of billion

dollars even though the company was
still not yet two years old so.

And so very recently we shipped
a second developer's kit and

I'll go into some of the details
of some of the things we did and

what I think some of the interesting
future research problems or.

Less When you give us the roadmap
was incredible the rate of things so

how does this device work well on
the screen you present two images one for

the left eye one for the right eye and
then you're looking at them when you put

this up to your face through magnifying
lenses these lenses distort the image so

one thing you can do now
the hardware is fast enough so that.

So that the graphic
card was fast enough so

that you can render the image in a
distorted way so that that distortion and

the distortion of the lens cancel
each other off hopefully perfectly so

that you don't perceive distortion when
you look through the lenses that's very

nice a lot of older virtual reality
systems especially as they try to maintain

a very high field of view and this one
goes up to one hundred ten degrees

diagonal field of view if you put the
lenses right up to your eyelashes so then

you still have optical distortion problem
but you can fix that in software and

also it's very very neat So a lot of
things working for to our advantage with

regard to the latest technology on
this was the main censor board.

That was used as an inertial
measurement unit it provides.

Gyro an accelerometer from invent sense
and this one was used in our first

developer kit that we shipped last year
in March and has a had a magnetometer and

there's a microcontroller that
communicates with the sensors and

sends information over U.S.B. So that was
the main thing we first started working we

just had another sensors a Hillcrest
sensor that was available lot of people

use it for projects and things on
the sensor has this board that we made has

in the company has some better
performance characteristics but

still relies on the same kind of
complements that are floating around.

And are very low cost so

we had was a gyroscope that provides data
at a thousand hertz so it's pretty amaze.

Giving you angular velocity in all three
dimensions we also were able to measure

acceleration linear acceleration
along three axes and

at about a couple hundred hertz we had
a magnetometer again three three axis.

So we did head tracking work so
as you move your head around you have to

change the images that
are presented to your eyes so

that everything matches up perfectly
right so if I want to look over a hundred

there's a v our version of him the viewer
version doesn't catch up one hundred

milliseconds later that causes lots
of trouble things get into but

if we want that to be spot on everything
feels like it's in the right place and

then your brain is fooled into believing
that you're somewhere else and so

that's the idea one of things that
as a roboticist I know very well is.

The space of three D. rotations right and
people in graphics they're often working

with transformations of things but they
might not often deal with the space of all

transformations and worrying about
the particular parameterizations and

representations of that in
the geometry of that and

the different kinds of singularities
that you can get into so.

So that's one thing that became
very important even though I

can break through the rotation
of the YA pitch and

roll finding the right representation
ends up being very important

as I like to say to a lot of people in the
game industry I see that quite tourney and

there are not just for breakfast anymore
there's sort of an all day long thing and.

Some interesting stories I know that
people in that industry some people like

to work with or their angles are very
simple other variants of that

a few people learn quite tourney and then
say I've I've I've developed these methods

that use quaternions they're there
what you should be using and

then people push back and say No I don't
need them for my application anyway and

most often the videogame character can
only pitch within a certain angular range

and so because of that they don't end up
near the kinematic singularity anyway so

they're right they can get by with your
patrol and they may be just fine for

what they're trying to do is soon
as you go into virtual reality and

you map all the game player controls
on the geometry everything over into

a virtual reality application all you
have to do is look straight up or

straight down in the world starts spinning
around in some strange way if you don't

get these parameterizations right.

So it's a it's been a very interesting
challenge sometimes to just

get people to switch to either this or
exponential horded representations or

other similar things that are saying that
singularity free just explain how the how

the tracking works I guess I'm so
very simple complement of it.

So.

Imagine a merry go round spinning around
to make it to the version imagine

it's just spinning around at some
constant angular rate Omega and

then I'm just trying to update the angular
position of the total cumulative

angle assuming I knew it was on the start
let's say then all you need to do is for

each delta T. I just take the this angular
velocity is just a numerical approximation

to the integral if it's if if the velocity
is not changing then it's exact and

I will give you exactly how far
the this merry go round has

has gone has gone around at least down
to the levels of delta T. informants and

if it's varying in its velocity then
the accuracy depends on this delta T.

HERE AND course your ability to measure
Omega if you're using sensors but you know

that's very simple oil integration and
of course you start to think about what

happens if your measurements are off a
little bit over time and you have to deal

with the time quantization issues you have
drift errors everybody in robotics faces

drift errors as robot localization and
you have dead reckoning era so

it's the same kinds of things
that we're familiar with but

just another application or problem.

So what I thought was interesting is
there's a lot of ways to go from two to

three D. for these problems and make it I
would say severely over complicated and

possibly introduce singularities and
so it's amazing to me how simple it

can be I can boil down to just one line of
code if you could get it right and that is

let's take a Omega to represent the three
directions of angular velocity and

then say what happens if you're
rotating around some arbitrary axis

you have some angle velocity you have L.

the length of that vector so that's
just the angular speed if you like and

then it turns out that
the math works out that.

If I observe omega then the axis of
rotation is just the normalized Omega.

And the amount that you're rotating or
the rate of

rotation is the length of the vector so
I really find that fascinating so

that if you want to just update for
the three D. case assuming you have a nice

quaternion library let's say So it's one
of the things we developed as quaternion

libraries everywhere then
you just pass in the.

The axis and which I calculate V.
and theta there you just passes right in

construct a quick turn in for that in just
multiply it on and that very simple step

updates the three dimensional orientation
for you in the same way that this did for

two D. and of course you can experiment
with higher order integration methods

a lot of other things but I think this is
fair it's amazing to me how simple that is

because that omega is directly what comes
out of the gyro at a thousand hertz and

so you just have a very
simple up to law so.

And if you look around on the Internet you
search for various class projects forums

people giving advice it's wrong
almost everywhere so and and

way more complicated than needs to be so
it's just one of these things where

if you dig in books talk to mathematicians
whatever you need to do there's very very

elegant perfect simple solutions for
this in retrospect it's just obvious and

people will probably do this for
forever after if they seen this but but

I'm all right so Nevertheless even if
you do this there still drift air so

the angular error will drift over time
this is a graft in seconds here some plots

that my former student Max Katz of did
he work as an intern for Oculus and

stay permanently kind of a rumor that
the interns Oculus never go back so

they've to keep busting.

You know get to that.

So there's a lot of energy
things that happen so

that we found that the drift rate is worse
the faster you're moving your head so

all kinds of annoyances like
this happen so we just of course

it may depend on temperature and other
things like that you can compensate for

it but there's some
fundamental problems like this.

So how do we fix drift error
in this kind of regime

Well first thing you do is suppose
you have a constant vector.

FIELD So you have field lines just
poking through your head right and

it's constant How can I use that well
I can align myself with respect to

that field is the idea if I have a sensor
that can measure that field in the local

coordinate frame that's attached to
my head or head mounted display so

so if I can sense this field I can just
look at how that feels varying right and

I can use that as some kind of
absolute angular reference notice that

the in this kind of scenario the length
of the vectors doesn't matter all matters

which way they're pointing and what they
are all pointing in the same direction so

it's a constant field doesn't matter
where I'm at in particular position and

I want to use that as a reference to
align myself so if you have that.

Then what we did is we can tweak we
compensated for tilt error so errors in

pitch and roll right are just like you
would think tilting like like exactly

in the cases where a marble would start
rolling right for tilt airs we compensated

for that with well color here are gravity
sensor right anybody have one of those.

Anyway those are nice and we then
compensated for the remaining comp on it

which was your error which is which
direction you're facing we had trouble

when people were making games they
had a cockpit in front of them and

some of the games and in the cockpit would
just slowly start drifting off to the side

and if you're in one of these nice
rotating office chairs you can just

rotate with it but it's pretty silly right
to have to start having the person using

the rift eventually coiling themselves
up in a cable you know for no reason.

So we solve that with a compass and
one of the things we quickly learn is that

these nice terms we use especially compas
but gravity sensor sort of made up as part

of the comparison here is that these are
the things we ideally want to measure and

maybe in a textbook we could measure
them but in reality we have sensors that

usually measure some vector sum of
a bunch of things that includes this and

a bunch of other annoying stuff and
so in the case of the in the case

of a gravity sensor we don't have
that what we have is an Excel or.

Comet or so it measures acceleration
he said That's great we know it's

nine point eight roughly meters per
second squared pointing upward but

what happens if you're moving your head
around then you're measuring the vector

sum of the acceleration of your head and
gravity.

And then you start to say OK
well I'll try to align myself

only when I have a good
gravity measurement.

And so OK how about what it's really close
to nine point eight then you say OK I

better hope my calibration is very good
and then after that I start thinking about

it and this goes right back into might
this work I did on a preimage is and

sensing and filtering all of this ahead
and thinking about in the last six or

seven years I thought you know there's
got to be a set of things I can do

that will produce the same sensor I'll
put in a kind of thought about it yeah

let's see if I go up like this then
the two vectors will combine positively

if I go straight down I can cancel off
gravity completely if I'm dropping so

I kind of do a nice maybe diagonal motion
I should be able to accelerate just right

so that the system will think that it's
tilted and start trying to correct

if you use this you're a stick of look for
it being nine point eight so

I started practicing a little but
I got good at doing that and

found ways to mess that So
it's very interesting in that these

problems tend to be fairly under
constrained you might use assumptions

on kinematics of the human body and
motion limitations and

things like to try to make some purist
it's about acceleration but it's a very

tough challenge a tend to be fairly under
constrained consumer products space where

you just just throw up the track system
down other things like that right.

Or it and the magnetometer was the same
way because magnetometers are not compas

as they measure magnetic fields I'm
standing right here that's going to be

the vector sum of whatever magnetic
fields are generated by this building and

by the Earth's magnetic field and
you better hope they don't cancel

each other off in some way so that
the resulting vector is really tiny And

there's vector field on the board itself
and they're also causing trouble and

there's hard and bias and soft and
bison on all kinds of problems anyway so

that's a lot of stuff we struggle with I
could give a whole talk about that but

I want to start getting closer to humans.

Assumption because that's the the
interesting thing that I got into so

in the beginnings of perception or one of
things we can talk about is latency so

when I turn my head how long does it take
for the correct image to show up and

be presented to the AI as stimulus so
people sometimes call this motion to

photons so I do the motion how long before
the photons are hitting the retina.

This is a nice lie the Oculus folks made
so lets see you turn your head it goes

through the sensor sensitive goes
through U.S.B. the game engine as a game

engine does some calculations and figures
out what the scene should look like

they stay on the new orientation
then you have to scan it out and

then the pixels in the L.C.D.
display for example in

the DK One developer kit one that we made
have to switch it turns out that for

some ranges of colors when you switch from
one to another it could take as long or

longer as the entire frame time could take
longer than sixteen milliseconds just to

do the switch that's annoying so that
causes trouble I'll get back to that one

of the film is perfectly fine if it's for
your smartphone it's optimized for

low power and not you know staring
at it with a magnifying glass and

having all kinds of different images
presented to your eyes quickly right so

that was made for something else.

Well all you have to to fix this latency
problem is a couple of things you can do

one of them is predictive tracking which
I'll talk about is something I worked on

and then there's time warping or which
is also so it's called Image shifting or

image warping where it's a kind of let's
say graphics or image processing technique

or trick where you you use
the game engine to generate.

The best guess you have at the time and
then do some post-processing to fix

the result maybe shifted a bit warped
a bit to something so that it still

looks perceptibly fine right and it has
no perceptible problems hopefully and

in that case you can effectively
reduce latency so John Carmack and

his team and Oculus work a lot
on that method and I work mainly

on predictive tracking with with with with
with other other folks of Oculus and I'm.

Kind of eating up the latency
from both ends let's say right

if you have very high latency it tends to
make people uncomfortable very quickly

especially if the latency varies so
it turns out your brain can compensate for

maybe consistent latency but
if it varies a lot you're in worship.

So here's something interesting you know
that I've heard of predictive filtering

it's been around forever I
don't even have a year for

linear predictive filtering it's got
a been around since the thirty's or

something I don't know common
folders been around for

over fifty years there's a whole bunch
of virtual reality in augmented reality

research on predictive tracking there's
a lot more references I could and

should add there here's one thing I
didn't know because I never worked in VO

as I didn't know that predictive tracking
was considered a double edged sword people

said well when it's working it's helpful
and then when it screws up it's way worse

so we don't like to use a very much
I found out after I did this work.

Which I was just kind of a naive
robot of so as food was fun so and

so I started thinking about things and
I wondered so I ended up with pretty good

performance of predictive tracking just
very simple kinds of techniques and

I can't figure out why it seemed to
contradict what the wide widely widely

held beliefs were in virtual reality
research these are very smart people and

they've tried a lot of things and
you know they were right in some sense.

So I thought about things and one
question is how far do we need to predict

into the future and then another
question is how far into the past

do we need to look to get the trend
to predict into the future and

in the one nine hundred in one
thousand nine hundred five for

example in a in a Ph D. thesis of Ron
Azuma from a universe in North Carolina.

Eighty milliseconds was a short
prediction interval and

a lot of friction of all maybe could have
been up to one hundred one hundred fifty

milliseconds I consider that.

We were having problems around the order
of twenty to forty milliseconds and so

that got shorter That's helpful.

One of things I like to say is that you
know there's there's no free will that

twenty milliseconds right so you might
feel like you can do whatever you want but

the momentum of your.

Head takes over largely at the scale
of twenty milliseconds and

you see if you can extrapolate the trend
estimate velocities pretty well then

you're not going to move
too much away from the.

So that's the idea so if you use
an optical tracking systems only like

people are using back to the ninety's then
maybe at sixty hertz you get some data so

that's the upper part you have to
look at some data points in the past

it's noisy from vision and you have to
extrapolate way into the future people

start building very complicated maybe
kinematic in dynamical system models maybe

trying to get the intent of the person
all kinds of things whereas in our case

we had a thousand hertz gyro that's
directly measuring the angular velocity so

I don't even have to take
noisy position estimates and

make even noisier derivative
estimates of that but

I have a very fast accurate fairly
accurate measurement coming from the gyro

and I don't have to predict as far so
that worked out pretty well.

Then I end up making trying out some
different methods method one was no

prediction that was the thing that
was done first and basically it's

equivalent to just the model says that you
instantaneously stop your head during that

entire interval which ends up leading
to exactly the latency of the system so

first thing I thought
was Why don't we just.

Take the most recent gyro reading and
just extended over the latency interval

our best guess of whatever the latency
interval is that's another problem in

itself as measuring what that is it's
very hard to determine sometimes but

that works fairly well but it seemed to
make it kind of jittery when we started

seeing going to vibration so we did some
simple smoothing they used to go away

filters that were popular in chemistry
it doesn't matter as all kinds of simple

filters you can use of that state and
that's sent in to smooth it out a bit and.

You know help reasonably well then I tried
to get clever that was method to wind

up using it for a while and then ended
up trying to switch the method three

which estimated angular acceleration and
then just propagated that forward so

this isn't velocity space and it's angular
So it's actually three dimensional but

I'm representing it as one dimension and
I just mean to say that we estimated

the angular acceleration of
the beginning and then just use that to.

Strappy laid a linear
The Los Angeles lost the change and

then you know you integrate that
again to get the angular position and

I was very happy I did
an experiment where.

I put on the wrist I gathered up some
data from the sensor and went like this.

That's about what I did about that
piece and I looked at the angular air I

tried it with no prediction and
it was just the angular error

you can measure very nicely in quick turn
in space so it's exactly the quad turning

that rotates from one orientation to the
other just what's the angle of that and so

that's the air this was the air
when I was turning my head back and

forth not sure what I did there but who
knows where I was turning my head back and

forth maybe I reached over to
stop it then I don't know and

the newer method we were using is the red
arrow here that was the one that just

extrapolate forward the angular velocity
so that worked a lot better but I was so

excited one day because I made one
that had the yellow curve here and

I was like yeah this is
great it's better and so

I started showing it around and one of
the guys I work very closely with and

near the beginning is a game developer
named Peter Geo chorus and he's.

He's a very.

Well was say he's an audio file so
any Also a D.J.

is on the side of very very close
attention to musical details and

he turned his audio file kind of
nature into being a video file and

like looking very carefully for flaws and
he said yeah your new method is worse

my god it's better see you know I
use science and engineering and

in all this it's better the plot say
it's better and yet it's fine but

it's worse and and so that's what happened
and what was interesting about it is

he said OK you know put it on look at the
wall it look at some structure in V R and

just stare at it and he says with your
new method you can see a little bit of

twitching so there's a tiny bit of jitter
when you're not moving your head but

he said when you turn your head while
it looks great you know like I turn and

everything's in the right place so
that really caused me to think and

it has changed my whole course of way
of thinking about development and

now research and I'm interested in and

being very strongly human perception
centered on that while this is very

interesting this is a filtering try.

Problem or human perception is at the core
of what you're trying to optimize here

because if you're just staring then you
can perceive these small motions but

you can't perceive latency because
nothing's changing anyway but when you

turn your head it's all about latency
because you want the world to catch up and

your brain essentially dumps all
the information in between while

you're turning your head anyway so
it mainly where you end up

is in terms of your your
attention your visual attention.

So we ended up making a method
that was perceptually two and

filtering that kind of gives the best of
both worlds it doesn't try to predict

when you're stationary and
then smoothly ramps up to the full

interval that you're supposed to be
predicting at when you rotate quickly and

so by blending these together we had
that was something that we felt was.

Perceptually a better tracking method and
of course the best way to do that is to

have a very complicated very sophisticated
I should say user studies with a large

number of people and see what people
prefer and if people are more comfortable

one of the other and so forth and that's
all stuff that that can and should and

will be done but but at this point we
were just a small group of people and

just trying to improve the experience so
on so I thought that was very interesting

and so you end up having a trade these
things off one of the reasons why a trade

off like this is because the more
smoothing you do on a signal the more

latency you introduce right because you're
using old data and it looks like it goes

in the opposite direction the further you
try to predict the more it adds jitter So

it actually increases noise because the
sensitivity goes up in that direction so

one such is something to pay attention
to so this might be the basic signal and

this is how it looks after
some kind of smoothing soap so

that you're trading off smoothing
versus prediction in some ways and

it's hard to get around those
those fundamental tradeoffs.

That's pretty much how
the sensor data looks but

I think that's actually stock market data
it looks it looks surprisingly similar So

imagine like a twenty day moving
average it's a very similar Let's see.

Perceptually to filtering so that's
what I just said that I'm jitters easily

perceived but not the latency and then we
need for stationary and for quick motion

late these perceived but not jitter So
that's what we ended up doing as I.

Said predict further when moving
quickly and it needed little or

no prediction when stationary and then we
just move the transition between these.

That can be further thinking OK I've
been working on sensor characterization

calibration and filtering was also called
sensor fusion by a lot of folks I had

tracking and
it got me into perceptual psychology.

I ended up hiring couple of
perceptual psychologists and

we started working on all sorts of things
because it became clear that not only does

it affect these aspects but it affected
all aspects of our system so so

many different kinds of
problems not just latency and

tracking jitter but optical distortion and
the distance between your pupils I.P.T.

focus of the lenses brightness all
kinds of artifacts of the graphics and

rendering and scan out like tearing if you
don't have the sink on all kinds of things

like this and of causing trouble screen
door effect the way textures appear on and

on there's all kinds of
problems some texture maps

might look really nice some maps textures
might look really nice on a screen and

then a virtual reality painted cardboard.

But when you look at them in stereo
everything's you know very nice right in

front of you doesn't look right so a lot
of strange perceptual things are going on

I find that interesting and so perceptual
psychology especially vision science in

this context is is very very important to
people who have done a lot of systematic

studies of the human body the brain down
to neural science up to the phenomenology

of the highest levels so very interesting
and very relevant to what we're doing

some may have thought about its
connections to virtual reality before but

most have probably not so very interesting
but tensions for collaboration's and

universities many other places so
if you don't get all these things kind

of right and comfortable you end up
with Ask adverse effects like disbelief

doesn't feel right doesn't look right or
fatigue your brain may compensate for

a lot of the problems and you don't even
realize it but for some reason you get

a headache or you feel you feel really
tired after a little while or the worst

is a simulator sickness of the end if
you don't pay attention these things and

it gets very complicated because
your brain is always adapting

to new situations and so it may comp and.

For the flaws in your system and then
reward you later with some discomfort.

And it varies across people somebody will
say what the heck you're talking about

this is fine this is better this is fine
let's ship it you know this is great and

then no there could be twenty percent of
the people have a bad experience with it

so it's very interesting very difficult
problems now there's human factors in here

all the way down at the lowest levels of
the engineering just which is pretty cool

and very interesting all right the
vestibular system comes into play here.

So the human vestibular system we have two
pieces of that the two main parts are air

inside of your inner ear canal and
what I found really fascinating about

it is you know the auditory part is in
here as well but the vestibular part

is like a low frequency version
of the of the auditory system and

what really amazed me is to see there's
two parts this is the otoliths organs here

the utricle and secular and each one
of those is essentially a two axis

linear accelerometer and
they're in two different planes and so

you get four axes total with one redundant
axis but your brains filter all this

stuff out anyway you don't know where it's
coming from Right but but that's gone and

I found that really amazing and then
there are these semicircular canals and

you've heard this before there's fluid
inside of little hairs and neurons and

on and on and I will go down the details
of what I found really interesting is that

there are three semicircular canals and
they are mutually orthogonal down to

within a few degrees and
I thought of while it's amazing I have

a three axis gyroscope on the rift that
we're doing sensing and filtering on and

I have a three axis gyroscope here this
WANT TO turns out measures angular

acceleration so we measure the integrated
version of that but it's measuring sort of

almost the same thing we have
an accelerometer so does this.

I guess we don't have a magnetometer as
far as I know but I'm pretty much the same

compound there so that isn't as beautiful
we have a competition between the.

Inertial measurement unit and the inertial
measurement unit that would have evolved

inside of our ears and there at conflict
a lot of times and that's very important.

To add to the things we're dealing
with another thing I learned about

was the vestibule ocular reflex
if you if you fixate So for

example you can you can look at me and
start moving your head like this.

But I give it a try it won't hurt so
you can keep me fixed but

what why is that why is that you
keep me fixing your eye is perfectly

counter-rotating it's using
your vestibular system it's

in a very Stipe loop with the muscles of
your eye so that your eye will counter

rotate as you rotate your head to maintain
a stable image it's really fascinating and

it does that down to I would I think it's
the latency is below ten milliseconds so

it's doing better than we are with
with our engineering systems it's

quite fascinating and that comes to play
in virtual reality it's happening all

the time and your perception of whether or
not some object looks correct and

stable in virtual reality you
have to run it through this too

you know it's a very nice control
system here running at a fast rate.

Or so if you're thinking about how
the physical world in the virtual world

are related it's one thing we can do is
they can sit in a chair put on a headset

and then just in virtual
reality be sitting in a chair

in a similar looking space you move around
and it's all just matched perfectly or

you can start changing it so
if I want to move in the virtual world

I could just move in the physical world I
could have I could have a stand up version

with a long area where I can walk around
large area where I can walk around or

I can just grab something
like a game controller and

move myself around like I'm
just a video game avatar.

I can do with a keyboard or
mouse as well or

of course I can simulate physics I can
throw things organize other things moving

around so it's another way motion
can happen but the first two

are probably most interesting because they
correspond ego motion or self motion and

the second one is can be quite
disturbing in virtual reality right.

Kinds of questions like if I want to
rotate my avatar in virtual reality how

do I do that can I just grab onto
a controller and move the knob and

just rotate there's going to be a mismatch
between your vestibular system and

what you perceive with your eyes.

So you can do all kinds of different
things here you can change your scale that

should be very interesting right nice to
be down the size of an insect running

around those lots of things I was studying
just mapping from the physical to virtual

world I find fascinating just like I once
found fascinating the mapping from I don't

know let's say the robots work space into
a configuration space that you planning

and there's a really nice mapping from
the physical space into the virtual world

space that you're making and there's all
kinds of parameterizations you can do in

there knobs you can turn and some of which
will make people happy some will make them

excited some will make them sick and
it's very interesting.

So we think about what your
mobility reason for the person

we were now we've been in an ocular been
focused mainly on sitting in a chair.

Because of safety if nothing else
because the first versions of this or

essential like a blindfold covering it.

So if you get out walk around it's a lot
more dangerous under schedules anyway and

such but of course as time goes on
people will want to walk around and

feel more immersed are present in
some in some other space but there's

there's serious safety considerations of
things that be worked out as well for

that and it's a lot of set up cost right
not everyone can build cool labs like you

can a Georgia Tech where you can
put cameras all over the place or

and and instrument the environment
to make a huge space area and

maybe you can do it lower cost sensors and
technology but maybe not yet with the kind

of accuracy we would like a need for human
comfort for perceptual comfort let's say.

So moving the Avatar around you go
forward backward Maybe side to side

called strafing in games torso
rotation as I said walking up and

down stairs I find one of the most
nauseating experiences so you see stairs

going by very fast it makes me it's very
difficult for me so a lot of people will

have a favorite direction like to say I'm
better going up the stairs than down so

it's amazing but it'll be different people
will like different ones all sorts of

stuff happens so one interesting
concept I learned about is veteran so

vaccine is the illusion of self motion so
if you've ever been I know you

could have traffic you're probably right
there in a city for that so so if you're.

Like in traffic you're not moving and
then maybe a truck starts pulling past you

in the in the neighboring lane you ever
get the feeling like you're moving

backwards a little bit and it's a little
bit nauseating sometimes it's kind of whoa

what just happened to it so
it's a strange feeling that's exactly what

you're experiencing you you perceive an
optical flow right there's there's motion

in your vision field if I walk up towards
the audience near my periphery I see a lot

of motion outward like this that's this
approach in case so you get this optical

flow when you start moving your avatar
around with a controller for example but

that's not matched by
your vestibular system.

So I found this interesting every year
people want to talk about the levels of

learning in others taxonomies all
sorts of things I've been going to

do that I could go off on a philosophical
rant but what I did have different

levels of understanding for this problem I
think it's kind of fun to be like almost

assign a number of each person wants or
talking to them about the R.

and such about where they're at like
first or like what that matters what is

vestibular mismatch right at the first
level want to I started myself you know

Sure you can move your avatar around who
cares right then you learn Aha OK so

if I'm moving you know if I'm
sitting motionless in my chair and

I'm moving in view are that's bad and
I thought well is it always bad.

I thought about it more like well then
I start thinking about it you know with

enough physics background robotics
background you think about how these

sensors are working and you think all well
wait a minute maybe it's only bad when

the accelerations and angular velocity
is mismatch right because if I'm moving

along at constant velocity How can
an inertial measurement unit even sense

that right whether it's in my ear or in an
ion you that I build you can't detect that

can you right so I remember those kind of
things right these kind of relativity and

all these sorts of you know
basics very much you can do so.

If you have a sensor that's measuring
something like maybe the air flow

rate the air pressure is a moving
then I can detect that but

otherwise I can't detect that so sources
so people at level three are saying OK so

it's fine as long as you
don't have this kind of miss.

That's right as long as you're
not changing the velocity and

then I thought about even more I'm like
Are we really sure that that's still

comfortable so if I'm moving my avatar
along a track at constant velocity and

I'm like yeah this is great I can
look around any direction I want and

like I don't do that very often in the
real world at a perfect constant velocity

with no other vibrations anything
else going on no other feedback and

I'm just being dragged along at a speed
I'm looking all around all I can look down

no problem I'm not completely sure you
know I think the there's still more and

more human factors that come into that and
then there's all kinds of research you

can do you can say well find these
are problems I still want to move

my avatar anyway what should
I do maybe I can point and

click where I want to go just teleport
there but then someone will say yeah but

then you don't feel immersed anymore you
don't feel present in that place and

I don't know maybe I can feel
like that if I could point and

click where I want to go in this
world I would gladly do it you know

in a sense I might start to prefer the
virtual world you know it it may be fine

one of things we found out is that if you
want to get up to a certain speed like

I want to accelerate your character is it
better to do it gradually or just do it in

one spike a given impulse and you're just
at some velocity it turns out the spike is

better the brain tends to reject it
as an anomaly like an outlier but

if you have a sustained mismatch that's
smaller over a longer period of time

not good.

It was a very very tiny mismatch
over a very very long amount of time

maybe that's fine as your brain
won't even perceive the mismatch but

then you get tired of waiting for
your character to start moving and

so it's all kind of interesting trade
off here people came up with all kinds

of different methods you could you could
tilt your head to move your character and

then that causes the stimulus stimulation
and might distract your brain long enough

with the mismatch and things that might
be OK So a lot of things you can do.

When out of time so let's get to other
parts here a lot of interesting problems

of adapting games like people won't take
the geometry exactly out of games that

is even high level human perception kinds
of problems as well like if you take like

this was this game was made to render on
a screen but if you put it into virtual.

Ology then your eyes will be looking
a ball here I think right so

all of a sudden people asked Why do I feel
so short and I say well this was made to

render somewhere else if you put the
geometry exactly the same won't be right

and then people say well this guy is
too tall he's standing too close to me.

And is very intimidating and
that maybe wasn't even intended so

if you start doing social interaction
all kinds of things like this in virtual

reality space it a lot of
counterintuitive things happen

because of this additional
feeling of presence.

Let me get back to kind of lower level
stuff so what about head position so

I talk only about orientation what without
position you cannot do Parallax or

I can do this motion back and forth where
the people in front of me seem to be

moving quickly with respect to the people
way back in the back as they move back and

forth right so
if you don't track that position and

if you move like that then there's
more mismatch so it makes sense to

try to develop even more we tried some
tricks like just making a fictitious and

of head model just move the at
the center of rotation somewhere else so

that if you're keeping your head in
position you're rotating around it's

not bad but if you do anything
restart to move your head.

Translate the base of your head then you

we could not find any ways out of
it by just using more sensors.

In this case we developed a system
that has that uses vision

uses a camera and infrared L.E.D.S.
about forty that are positioned around

the surface of the headsets so it's sort
of it's computer vision but cheating right

it's not a hard computer vision problem we
want to be one hundred percent reliable or

at least as reliable as we possibly can so
we get lots of these L.E.D.S.

within the field of view of the camera
you sit between a half meter and one and

a half meters from the camera inside of
that cone by integrating this vision data

together with the new data you get
pretty good sixty or freedom tracking so

that's what our latest system has
this is the prototype version and

we made last year in the in the spring
what I should think I thought

was kind of interesting about
it was that a dove cats do.

Mainly division work I did some of
the vision work as well that time and

two other people max Katz of my wife
Ana and I all over the main people who

contributed to the tracking of this and
roll from robotics so that's pretty

interesting writes a dove is a student
of Oliver Brock and some of you know.

And your very own Andrew Mellon was was
was a interned in at our company did a lot

of great work beyond this and
is again in robotics so for

the time being right and
so and so that's a great.

So so I just want to say that robotics
have a lot to offer in this field

more than you might realize but

I would also strongly suggest putting the
human put perception come poured into it.

All right let me make sure I get
to some of my final things here so

as we go forward what do we need we
need faster switching in display that

should appoint a Sol one of the one of
the High Performance things here is that

the display inside here uses all
the technology it's a ten A.T.P.

panel so it's higher resolution because
staring at you know screens with

magnifying lens this causes you to see
the pixel structure so this is better and

it's going to keep getting better and
better as we go down the road

there's not really any fundamental
technological structures to that

another thing I was important about it
was low persistence which is that in the.

Witching speed that you have for
L.C.D.'s you can switch these in about

that it's eighty microseconds or
so you can turn these on or off so

what you do is you pulse them on for a
couple of milliseconds and pulse them off.

If you leave them on for too long then
you get an effect called judger where you

keep a he looks like you see a sequence of
stationary frames you may have seen that

with some high speed cameras and things
like that so so we just pulse it but

then if you don't make the frame rate high
enough then you start having discomfort

because a flicker perception and there's
flicker fusion threshold in your back and

divisions in perception again so
this one's at seventy five hertz but

they really should be at ninety
Hertz we think for full comfort

these are almost equivalent to the earth
that were recommended for C.R.T.

monitors back in the ninety's as people
started sitting in front of bigger and

bigger monitors so you may remember.

One of those things because of
the same kind of flicker problem.

Let's see so we have new technology
demands faster switching pixels

higher resolution displays higher frame
rates improved sensors new optical systems

faster rendering pipeline to do all
of this all there is heavier demands

on graphics so
I think that's really interesting.

There's new platform research so
basic user interfaces and

controllers of course there are people who
do work on three D. interfaces and in V.

are all throughout academia but
now I think is going to be

an enormous growing effort in industry
around these kinds of things and

increased activity in universities so
for this platform research again high

performance graphics high highly accurate
tracking and all kinds of interesting

problems in developing middleware
maybe for helping developers to

make content easily whether trying to make
a kind of a panoramic movie experience or

a new kind of game or what social
experience whatever it might be

house of the middleware look it took a
long time to make effect of middleware in

the game industry people made their own
game engines and then a lot of the big

companies started selling them to other
developers because it was very valuable so

there will be things like that coming
along is currently a growing ecosystem so

there's four headsets around at least so

that the Oculus a DK to get
to came out a few weeks ago

Sony Morpheus you may have seen around
Google cardboard it's pretty funny right

you drop in a smartphone and there's
the lenses and such you don't need too

much more again and it's leverage off
the same kind of technology it's very

hard to control the quality of that though
because I don't know if the sensors and

screens are going to be good enough
in your particular phone and

it's hard to write the software in a way
that works with high performance for

all these and then I'm very happy
to say what was just announced

in the news in the last couple
of days is a Samsung Gear V.R.

which probably can't see that but
it's is powered by so

we've been in a collaboration
with them for

quite a time and they've just
analysis as an add on product for

the note for a phone that's coming out
soon so you drop your smartphone and.

You get a I was a reasonably good virtual
reality experience with the same tracking

technology that I've been presenting
here but it doesn't have the camera and

full six degrees of freedom but
has a three degrees of freedom so

it's a kind of entry level view our
experience that I think should be

should be should be interesting and nice
so that's where things are with a growing

ecosystem of a lot of people getting into
this and we have very good relations with

various folks the Oculus of Sony People
are always talking a lot of shows and

we go into each other's back rooms and
show each other the TAC and it's been very

very nice kind of camaraderie in
the field and stuff so this is great.

There's all kinds of new content so.

You know take to start with something
simple like Google Street View and

now think what kind of panoramas can
I make live panoramas virtual travel.

Three hundred sixty degree
movie making a lot of folks in

Hollywood are interested in this.

Social interaction and of course different
kinds of chats that gets back to the kind

of application like I was talking about
in the beginning with my grandma and her

sister so I'm thinking about what kind of
interactions going to have in this medium.

I'm I'm putting a few favorites is
not my grandma in this case but.

And so so I want to now just show a few
quick slides of what people have done out

in the wild so we sold about seventy
thousand of the first developer kits

we may have already sold around that
month many of the of the newest one or

more I don't know the numbers exactly.

And so different ones that were my
favorite just look for gram Oculus And so

somebody way back is but have been like in
March of last year they just strapped on

their nine year old grandma and she had
a wild fun time running around in this

the Tuscany environment so that was
great fun a very inspiring because

it kind of confirms this this kind of
interest that I had in this technology.

People might wish their
home were bigger so so

imagine going home to relax this is
particularly popular in East Asia where

you may be crammed in a small apartment
in Tokyo or Seoul for example.

And so OK Manhattan I don't know.

So so those are going to have.

A place to relax maybe
listen to some music and

you can even have that space
specialized through the heads that or

you may want to give people architectural
walk through in real estate so

all kinds of things you can do in this
in the space people are doing that.

One of the first applications that came
out that really transformed the way we

thought about virtual reality was a thing
called virtuality cinema it was made in

a few weeks by a developer a guy in
Korea I was in his early twenty's and.

The idea is that there's a virtual movie
theater you pick what seat you want to sit

on it and you stream movies that are on
your hard drive to the screen and

there are three D. movies that work
fine either way and that's kind of

going to hurt my brain a little to think
about it works you can look back and

see the projector and the acoustics of
the room or even taken into account

I thought that would be kind of pointless
and then I tried and I thought all

over there right away it was it was one of
the most comfortable view our experiences

we had seen up to that point it wasn't
making people sick it was very nice and

very comfortable experience and I thought
that was amazing and it's also a kind

of gateway into other kinds of experiences
you can enhance this virtual reality

experience but imagine you're watching
a movie about zombies you don't

even have to touch the movie about zombies
but you can put zombies in the theater or

you can socialize with your friends who
are in other parts of the world and

sit in the theater together and then you
know come up with ways to harass each

other throw things at each other I don't
know you can do all kinds of fun stuff

in these theaters but leveraging all
this content that already exists or

that was brilliant so the guy who
made that works at Oculus no course.

You know telepresence
panoramas things like that you

know what as one the first thing someone
did was connect to Google Street View.

First person shooter games and go pretty
fast through these I mean my last couple

of minutes we had a very hard game
jam this is one of my favorites

you basically just have
a big elephant trunk and

you're like moving on a track and you just
gotta swing around knock stuff down so

you're using your head as a controller and
just flopping around it was a lot of fun.

And did make people sick really cause
they're just there just to have.

Yes with plopping there
are other trunk around.

Of course everyone wants some people want
to have the extreme experience there's all

kinds of virtual newsmen park rides there
are some brilliantly disturbing videos.

Going to look up the Oculus guillotine.

Even shows you even shows you what it must
look like when your head is rolling on

the ground in the last
moments of your life.

And this is done by some art students I
believe in I think Denmark if I'm not

mistaken so I'll just say it's Denmark.

Could have been France I think at
least one of them might might and for.

Alternative body experiences so what would
it what would what would it be you know

what would it feel like to be somebody
else right you can do all kinds of

interesting experiences you may change
your height or race or gender all kinds

of things in a kind of social interaction
experience or just simply what would be

like to be somebody else in this case they
put cameras on the front of the rift so

that you can look out through the eyes of
the other person then you move together in

some kind of synchronous way
this was done in by students and

at a university in Barcelona now they have
all kinds of activities beyond that so

I found that interesting.

To see what else was there socialising in
virtual spaces so things like open sim

which by the is a very distant open source
project where it's just like Second Life

but fully open is develop by a lot
of people in universities and

there's all kinds of servers for this and
you can interact as even an entire

conference they're going to have a couple
of months that's it's all done in

a virtual conference center that they made
all kinds of interesting opportunities for

social interaction that's
been connected to the rift.

There's no shortage of testers So
there's people out doing things all

over the place with this technology but
the problem is that we don't have what

I would call a good perceptual training
program which is getting people to see

what the flaws are in the system of
their developing like do you know that

the latency is terrible and what you
just developed you know do you know the.

There's some distortion problem the frame
rates down the stereos offers all kinds

of different things that
can be going wrong and

most people start off perceptually
blind they don't see these things and

I were I was blind to a lot of them when
I started working on these problems but

now I see all kinds of things
that I didn't see before so

it's a kind of training perceptual train
exists in other kinds of fields and

context we're going to need
to do something about that

these are the swear words
that came up a lot of Oculus.

Jittery leg of it all and why it seems
like you know I don't like this experience

is too jittery laggy swimmy just
be blurry flickery I know they're

like seven dwarves or something and
there's eight now that the list continues

to grow I think is a couple of new ones
but of it's amazing how often I've heard

those around the company usually
they're used fairly consistently but

sometimes not I think swimmy had
multiple definitions sometimes so

it's very very difficult problem I
think that's the beginnings of trying

to do this perceptual training that
we're communicating what the flaws are.

Future research so
better visual engineering so I've already

gone over some of these kind of things
before better tracking systems and.

There's more sensory stimulation
beyond vision audio touch haptic

I don't You can even go to smell and
taste if you like and

you can stimulate your vestibular system
if you want but I don't recommend it.

And each one of those is going to have
some perceptual psychology come pointed to

it that's at a very low level I would say
and then there's the higher levels of

psychology and human interaction and
sociology interfaces H.C.I.

communications educational sorts
of things at this level as well so

there's human aspects from the lowest
levels all the way up to the highest

levels which I find very interesting.

So it's human centered research
all around I would say.

You know a lot of these interfaces took
a long time to develop it's hard to know

what would have worked well in advance I
think virtual reality is that that kind

of stage and finally of course especially
being here at Georgia Tech and

myself being a robot of this for a couple
of decades I'm really excited about

what can happen mixing
virtual reality with robots.

And the first thing that happens if you're
not careful is that you'll make people

sick because robots don't move
in a nice comfortable way and

so you have to do some kind of interesting
stabilization or something to make a good

experience so I think I should do
one final thing Oculus is hiring and

in many places we're building a big R.
and D. center in Seattle headed by Michael

a brush I'm building a smaller R. and D.
lab at Illinois in the research park and

broadly we're hiring in
a lot of areas Thank

you.

These kind of human perception
kind of mismatches.

You know it's it's so difficult I really
think that the biggest barrier right now

it's interesting and maybe it's because
I'm a professor it's an educational

problem it's just just getting developers
or potential developers to understand

the bad ramifications of some
experience they're building.

Yeah you can do it can be quite fine
yeah that's right that's right so

I mean it's just been diverts rally
cinema's a good one it's not bad but

some people just don't necessarily
want to do that right and

it has to be installed correctly and
run of the right frame rate and

a latency has to be everything has to be
done as it was designed so that the system

configuration is difficult sometimes it
might fail to work like it was expected so

reliability is an issue here rather than
just breaking it instead sort of works and

gives you a bad experience and

if it's your first of your experience ever
you might think wow this is great why do I

feel sick after five minutes and because
it was a set up I can look at it fast and

tell you what's wrong with
it in a lot of cases.

I got you I would say that.

For Health and Safety which includes human
comfort is the biggest issue right so

and I think there's solutions for
that and I think I don't.

Anyone's on the business side we're not
terrified over these are solvable things

I think it's quite fine it's just.

It's more a problem of providing the right
warnings the right education the right

guidance to developers and so forth and we
also need to go through this together and

as a technology keeps improving all the
things we get better faster frame rates

and stuff like that will help you
feel more immersed may have less

eyestrain things like that some things are
hard to fix like the separation between

vergence and accommodation so right so so
you know about that your eyes focusing and

where your eyes are rote they're always
coupled in the real world they get

separated in virtual reality
unless you unless you change

the display technology to life feels and
that might be more comfortable if you can

get high enough resolution and stuff like
that so a lot of interesting issues are.

Going

to.

Get.

This vision work or
not work what you think.

So I mean if so the biggest problem
with with with vision of is always been

as long as I can remember as a researcher
I worked on image segmentations the back

in the ninety's is it works fine for some
number of cases and there's reliability

issues so you have to have very high
accuracy first of all both temporal and

spatial because of the human perception
problems with much higher than the kinds

of tracking that you typically see in
other kinds of systems like if I was doing

some basic hand tracking or the kind
of tracking you see with vision and

such it's got to be higher accuracy
than we're used to and if it's not very

you know if it drops out here and there
what are the fallbacks if the fallbacks or

something else perceptually bad you're in
trouble again so if the reliability is

high and if you're sure you're not
facing you know perfect white walls and

that whatever other kinds of things
then it should hold a lot of promise so

it's an interesting research challenge a
lot of people going to be looking at that

and I'm not sure what's

going to happen you know because I really
think I really I am always a stickler for

reliability we accept lower and

lower levels over liability as
time goes on you know take the.

The phones they always the landlines
worked perfectly all the time when I was

a kid and now I can barely control
my cell phone it starts making calls

stopped making call There's all kinds of
stuff I have no idea thought of controls I

don't know what level of unreliability
are willing to tolerate but I can say that

some of that unreliability leads to human
discomfort and that has to be taken very

seriously.

Yes.

Or just I'm just looking at the angular

velocity.

Yeah I see what you mean I mean it's just
it's just done with simple filtering so

I mean it's just a gradual you know we
try to number of different things and

I probably don't go too far into it
because it's not something that I would be

like the method itself I would not be so
proud of there was a here's

here's the scientific paper and
this is exactly how you should do and

I can prove smoothness that's
better than your smoothness or

something like that it's not so much
like I said it's just the insight that

we need to switch between these two modes
and it just involves a simple kind of

scaling of the angular velocity is over
some range from a minimum to a maximum and

so I I increase the latency interval
as the angular velocity increases and

nothing really more sophisticated
than that at that stage

a lot of more interesting research
could be done but I think some

hard core vision scientists need to
get in the mix and some very serious

studies with a lot of human subjects need
to be done to make improvements on that

as well because if we just say it's
better because it has better smoothness

properties based on the engineering
math that I'm worried again because I've

already been shown that that
doesn't always matter and OK.

All right thank you very much.