[00:00:05]
>> Thank you for inviting me can you guys hear me OK in the back there. OK. All right so many going to have 3 parts of the talk 1st I'll give you over you the status of robot this is that there be. So you can see how the fields evolved of the last 20 years the 2nd part I'm going to talk about some of the neural control learning mechanisms that are played at play when humans interact with robots for the purposes of rehabilitation and they'll talk a little bit about where the fields headed.

[00:00:42]
Some of the things I want to talk about I have a financial interest in. And invention which is our Max's skeleton that's being used about 700 hospitals now and some and then some things from a start up company average. So amazingly about one in 6 people worldwide will have a stroke and this is a.

[00:01:04]
Video of a volunteer from our lab and I'm asking you to touch your Chad. And you can see it's very difficult for live to our so my lab I really became obsessed with the question how do people get into this state. Can you prevent them from getting to the state how can you get out of the state once you're in it so we've really been focused on how much plasticity is possible.

[00:01:29]
When you when you have an impairment like this. We started asking a question about 3 years ago when I was a grad student Berkeley surprisingly there was very little work on using robots to automate physical therapy at that point. Hogan had started with an S.F. grad at MIT.

[00:01:49]
In the late 1980 S. It was really the kind of 1st attempt to do it so you can see plotted over here the. This is the number of publications per year so it's if you if you google search for robotic rehabilitation and it's up to about a over a 1000 publications per year and you know it's an interesting lesson because there's no reason that had to start then they could have started earlier so it makes you think well what could have been started right now right now that hasn't started already but the technology was available on the so it's just having the idea that you could apply robots to automate aspects of physical therapy that started that the other lesson there is like when I published the 1st paper or one of the 1st papers there you know.

[00:02:35]
It wasn't well no one really was interested in it early on and our 1st grants were completely rejected by and I H But you can publish a relatively crappy paper that cited thousands of times later if it's you know early on in the field so it's kind of kind of an interesting that you make.

[00:02:54]
So what everyone tried to do initially is build robots that can physically assist people in moving so this is called active assist there. This is a known technique in physical therapy it was sort of the obvious thing to do with the robots. So. Most of ice is used this type of technique now and part of a big part of the talk will be trying to understand you know does this work and how does this work certainly there are other things that you can do with robots as well such as augmenting errors as a person moves.

[00:03:32]
Don't work in that and other people don't work in that not really in the talk too much about it this is a typical robotic therapy result. So this is from a study that we did in my lab. This is something called the the score so I'm going to refer to that several times through the talk so I need to explain it to you but it's it's a clinical measurement of how well you can move your arm so if you get a 0 for completely paralyzed and you get a 66 if you're if you move normally and the way it scored is a trained evaluator ask you to do 33 different tests movements and then rates each movement $1.00 or $20.00 meaning can't do it very well one you can do it to some criterion to do a perfectly and these are movements like lift your arm without bending your elbow and it's based on sort of this idea that after you have a stroke you progress through stages of recovery where initially your movements are are synergetic you can individuate movement more proximal movement they have less There's still movement and then as you recovery you gain more individual movements and more does this new movement.

[00:04:42]
So in this study we had people who were fairly impaired baseline for 23 so that looks like the person I showed you in the video and they did 241 hour training sessions with this pneumatically actuated exoskeleton this robot is learning a model of. This system's forces that you need in real time using a sliding adaptive control controller and building up using a regular basis function that work sort of a representation of the static assistance that you need to move affectively So the red is the robot group the blues the control group which is getting conventional tabletop therapy one on one with a physical therapist and you can see we have a significantly improved response these are 2 baseline sessions post there a P 3 months later.

[00:05:35]
And so there's a small positive benefit to doing the robotic therapy there is that the standard deviation of our is very large this is the box of box score this is how many blocks he can lift and pick up and move over a divider in 60 seconds so some of our subjects couldn't do any of this this sort of a floor effect here but but you can see that the ones that could actually are better at it yeah.

[00:06:05]
Yeah the non robotic comparison there was conventional tabletop therapy with the physical therapist that's the blue line. Yeah and same here this is the largest study that's been done this was sponsored by the V.A. this used the MIT menace robot so this was funded originally and it's a player to joint mechanism here that says you move your arm across the table top and you move to targets and they developed and algorithms sort of adaptively assist you moving to the targets and the compliant way and here this is the change in form our score this is 127 patients and you can see compared to usual care which is in this case you're not getting additional therapy these are chronic patients usual care is just coming to see the dye for every once in a while but if you do the robotic therapy it's better if you can and then this is compare it to I.C.T. which is intensive comparison therapy and there they taught the therapist to ask the patient to move very you know as frequently as the robot was asking the patient to move so.

[00:07:13]
You might see calls as Charlie Chaplin modern times there where. It is very sort of atypical what's done in physical therapy that the normal physical therapy session it's been measured you do about 30 practice movements that's it here you're doing hundreds and the purpose just trying to get you to go very quick so when you do that you see a pretty similar result with the robot.

[00:07:39]
And then this this is sort of getting at the point could added complexity help so here's a more sophisticated our Mexico thing. Called. Doing. 3 dimensional tasks in a sort of pseudo virtual environment and here you see very similar results again I want to point out the extremely high variability here so you have the therapeutic benefit which is a small benefit and you have this massive variability so a big part of what robotic therapy is that right now is trying to understand where does the variability come from.

[00:08:18]
So we ask the question could you you know could you get someone results if you just use a simpler device that isn't technically a robot in my definition because it can't move on its own. So here we just have a rubber band driven device that is providing counterbalance against gravity and this was based on the device designed by.

[00:08:42]
Dupont hospital for children he's got this 3 D. printed exoskeleton that kids with our weakness can wear literally kids that can't lift their arms they print those to conform to their body and that they can lift and start immediate playing with their hands and arms a great videos on that which I'm going to show you to the courage to look that up but we scaled it for use by stroke patients we put a pressure sensitive group here that could detect even trace levels of gravel and we put people into those sort of simulated activities the daily living so now you have this arm that's really pretty much nonfunctional and once you go into the environment you can you know lift and release and drop.

[00:09:21]
Move around virtual objects so we compared it to in this case. Conventional exercises on the table top of this would be like a home program that you get prescribed that no one actually ends up doing here they did in a clinic so we know they were doing it with a therapist who's a stopwatch to time so for the 1st week for both either group she spent time with them training them how to use it and then after that she only needed to spend about 4 minutes with each person and you can see we've got quite similar results here the reg Ruby in the robotics technology 6 month follow up they have a a.

[00:09:58]
A. Sustained benefit in this for them are a score. When you ask people which type of therapy do you prefer So fairly strongly people prefer the robotics technology and they like the one we ask why they like the quantitative feedback they like the computer gaming but interestingly they also like the sort of the proof f itself advocacy so this quote I think is very revealing someone said you know if I can't do it once why would I do it $100.00 times so you're reconnecting volution the movement through this physical assistance and that has a motivational effect and.

[00:10:38]
Bring that up a little bit more as we go so this device over spring it's become I think the best selling robotics technology for rehabilitation sells for $60000.00 a Swiss company license the patent from U.C. Irvine. Interestingly the therapist said that you know we want to use anything if we have to spend more than 5 minutes setting it up so this was company really worked very hard to have a very highly adjustable device for different body shapes that you can fit someone into and yet even still when I go to different rehabilitation facilities some if you have a person who takes the time to learn it therapist then yeah it's widely used but sometimes people still just prefer to bring someone up and put them on an exercise cycle you know because you don't have to turn on the computer you want the boot software you have to log in and you want to just anything just go up to it so if you're thinking about.

[00:11:36]
Rehabilitation technology that can actually used by people you have to really think about the time constraints that people are under in the simplicity that that's that's demanded. And you can see the penetration here it's only 700 of them that's probably the most widely used device right now so it's mainly bought by flagship hospitals it's used in marketing materials a lot of times to show like a we're using high tech at our hospital but it's but it's and there's publish studies showing its efficacy in different different groups.

[00:12:09]
But compared to the potential there's still a huge way to go and order to effectively automate the repetitive aspects of realization therapies and I'll show you some some of our attempts to do that later so we had a. Contract for him and I as these initial results were coming out and you see these small changes and for them our one of the thoughts was if you made that tat made the a therapy more functional Perhaps you would bump up the effectiveness of it and so we developed an exoskeleton with a parallel mechanism back there this was of Jim Bob or of Steve Spencer Julius Klein Steve Kramer is the neurologist that work with Ki-Chan as our physical therapist who runs all of our studies and then real level 0 was a post doc physical therapist that sort of supervise a study and did all the brain imaging you can see now we've added a pneumatic pneumatic wrist and hand part to it which is the magic because we can get a lot of power is a very lightweight package to put on the end of the the exoskeleton here we have an innovative parallel mechanism here this guy used to ride a motorcycle before he had a stroke you can see how engaged he was driving a motorcycle through Death Valley in the simulation and the robot is sort of learning how much help he needs to stay on the robot not overly helping him.

[00:13:36]
And we said OK So so there's this functional called the functional mantra in rehabilitation therapy which is that it comes on a motor learning research over the last 40 years which says that when you practice a motor task you get you get better at that task but it doesn't generalize very well to other tasks OK So the idea is that you need to practice the things that you want to learn and so we have this device that allowed us to do this multi-unit practice a more functional task and we decided to set up a study to prove our point and we just compared it to doing individual joint training so people would come in and they would move one joint at a time in the robot for a few weeks and then they'd take a break and then they would do the joint functional training and so much for surprise this is sort of a principle components analysis of like 30 or 40 outcome measures but the individual joint training appeared to be better both in the 1st phase of the 2nd phase of the training.

[00:14:33]
So there was really made us think so it doesn't appear to be the complexity that matters something else matters in terms of the therapeutic efficacy of it all right so that's that's sort of. Sort of the 1st 1st part so how does it work. So we have a couple ideas hypotheses about how there are people works one is the what I call the search for local bypass So you have a your brain connection into the spinal cord to drive the motor all polls it's highly redundant there's many not only are there are multiple sort of groupings of neurons but they're also very to drive you don't have a single wire that says produce the amount of force and reproduce for Shep that activate many many neurons at the same time.

[00:15:21]
OK so here is sort of analogy it works here is U.C. Irvine down in Orange County by Newport Beach and Disneyland going up to U.C.L.A. this is the busiest stretch of freeway in the country but you can see all the options that you have for going. We don't have the ability to google an alternate route so the idea is that it has to search through trial and error with what groups of neurons are going to allow it to get there so we're working with a model we call cassock search model.

[00:15:51]
Essentially of different ways of activating that are neural pools and what we're assuming is that there's variability on these neurons and functionally or sort of the computational level your brain is trying a command see if it works well. If it works well in terms of say generating more force extension figure force than Then you have some sort of process that cements that activation of memorizes.

[00:16:17]
So it's it would be rate related to work reward based learning recent research is talking about something called synaptic eligibility traces where you can have a heavy in process at work appear. That is really only activated when there's a reward that sent seconds later and there's a neural modulator released into the potential synapses synapses.

[00:16:43]
So we've shown that. This can explain several observations of Stroke One is that people after they have a stroke they always have this sort of residual capacity meaning they seem to plateau but if you then you stop and they give and then give some more in intensive therapy they'll get a little bit better so this type of network shows that property another of the major observations after stroke is they have a shift of activity the secondary neural error areas and this explains that it can it explains other things I'm not going to dive into it but for the your purposes as a group of people interested in robotics.

[00:17:22]
In computation a neuro science. Were advocating the start of a new field which we call computational neural rehabilitation so amazingly no one is attempted to mathematically model rehabilitation the model I just showed you is one of the 1st maybe 3 or 4 models. Just briefly described and again it's one of those things where it's completely possible it's a complex system but but it's difficult but it's possible so I think that's going to be a burgeoning area now is trying to bring the mathematical mathematics of neural simulation and human robot interaction into rehabilitation.

[00:18:02]
OK so now I'm going to talk about some specific mechanisms that we've discovered as we've gone along. And studied how people interact with robots so the 1st is slacking so we've discovered that all people are slackers and we actually nervous system implements a algorithm for us for slacking so we have this parallel robot here where we kid it was very back drivable.

[00:18:28]
And we could measure walking with it and then we could drive walking in perturb walking as well and really we started with a very simple question which we wanted to know. If you apply in the novel dynamic environment to the leg do you build a model of that dynamic environment and this had been well studied already for the arm and we're curious if it was true for the leg.

[00:18:50]
So we had people walk in this robot and the robot played a novel dynamic force field which is it that pushed upward on the leg with the force proportional to the forward velocity of the leg during the swing phase so what I'm showing here is the step by.

[00:19:08]
Reference to your normal step and as you're walking along we turn the forceful and it causes you to step high which is the blue trajectory there you adapt of the course of about 10 steps. OK but you don't fully adapt and then when you turn off the force field you have an after effect and you lose that and it takes 5 or 10 steps to to wash out now I can tell you and I did this experience experiment 100 times on myself and you can know this is going to happen but you can never quite turn this off OK so you adapt to it sort of automatically.

[00:19:43]
So what we said was we notice a people don't fully adapt so we said well perhaps are trying to memorize minimize a cost function that has to do with the kinematic there but also with the amount of effort there that the putting into overcoming the forcefield So that's the top there.

[00:20:00]
Is the step there are minus 60 is the step by there and you is the effort and is the cost there so if you minimize that agree fashion you get this error based learning law which is shown right here where your next command will equal listen at those one for a minute so you're your command your leg on your next step will be the previous command minus the error on the last command so it's air based learning so what we discovered was that F. was less than one.

[00:20:30]
And so what happens there is if your arrow small minus 60 is 0 then you get an exponential decay and predicted in the amount of force that you're exerting this model predicts very well when we applied different strengths of force fields to predict very wall predicted very well the final error that people would exert but what it implied is that you have this sort of latent tendency to reduce your effort whenever you could that was from a trial the trial basis.

[00:21:01]
With this pneumatic robot we found that you also slack continuously. And so here we had implemented the light emitted at the controller and we took the standard textbook adapted controller has to learn a model as a person trying to track targets about how much help they need static support forces in order to affect the track and when we did that we found that people automatically decrease their force.

[00:21:29]
And we found that by building error making the robot sloppy having the robot slack then all of a sudden it allows a little bit of kinematic error and then people would put out a lot more effort to my going to go into a lot of detail on that but that's the main message is that we found in continuous time people are always trying to reduce their forces as well so explain it I think this is the best way to explain it so you can you can all understand slacking so Brinton Smith this is actually just published own journal of their physiology.

[00:22:00]
So he came up with this paradigm where we we just looked at GRIP force tracking so you you have to. Squeeze this transducer it moves a cursor up to a target OK now on the half of the trials when you get into the target on half of the trials we simply disconnected the force transducer from the cursor so now you're not you're getting non-vertical feedback OK and your cursor is staying in the target no matter what you do.

[00:22:32]
Now in order to make sure that people don't know that we're doing that we preserve the high frequency component of the grip force so all the little wiggles are still there so you see it we're going along in the target but now you if you were sort of lay involved here you could just touch the side I'm not going to grip it I'm not going to squeeze anymore because I don't need to and this is fairly effortful some of these targets OK So the question is what what do people do no one realizes that we turn off the we we turn off the vertical feedback it's never cognitive or realized and what you see over here these are the average force traces going up to say 30 percent of your maximum voluntary contraction and once you get up to the target you turn the knob really cool feedback on it's the black line and after one reaction time so as soon as your last really cool information vanishes you start systematically slacking and decreasing your forces so really as opposed to a robot where you program it often to have a command force and you expect the robot to produce the force a human you have a command something and it generates a force but immediately it starts trying to get away with doing less force putting less effort out it seems which is really really interesting so it's a much more sort of transit device now why don't know why had no one else ever realised slacking it's because slacking creates errors so the like great trace at the top that's when you have visual feedback so you can see you get into the target you start trying to come out of the target but then you realize I'm going to come out of the target so that you make a correction and you stay in the target OK so it's very slacking is usually not apparent because it causes errors and you're correcting for those errors is what we're thinking.

[00:24:17]
We if that's true then slacking should account for forced variability right because slacking while it makes us more energy efficient it should increase or variability because we're always we're always have to correct for it so we each have our personal slacking rate and we measure the personal psychic rate of each person in terms of percentage of maximum voluntary contraction percent and the this is the variability coefficient of variability for your force and you can see that your personal slacking rate accounts for about 30 percent of your variability of your force tracking so we believe that slacking is the cause or of variability of the tradeoff between variability in efficiency so back to rehabilitation does that matter for real Taishan well at the extreme case here we have the study.

[00:25:11]
Very nice study done where we had a robot wrist robot that was triggered by E.G. activity and then provide assistance to the wrist and they compared it to just having the patient put their arm in the robot having the robot move the patient's wrist the same number of times and this is full of fire again and you see that the blue group where the patient was passive didn't benefit at all from the type of therapy so the extreme.

[00:25:36]
Slacking which becomes completely passive it. Certainly certainly is not therapeutic We also have this study this is the only study I know of in Robot therapy where the robot did not do as well as the comparison therapy OK so I called them the tortoise Seidler study and this is the local MOT.

[00:25:56]
Robot same company that licensed army of spring and here it's a hip hip actuator knee actuator that assist you in walking on a treadmill. The original version was relatively rigid and you walked with Gary Colombo who is one of the founders of the company you walked with his step pattern so there was no no variability built.

[00:26:20]
They compared this to conventional gay training this is the change in your date's speed these are for chronic stroke patients and you can see here that the people who got the conventional training had a higher or greater improvement in their gait speed with their training and then the group that did the local training.

[00:26:39]
Now the look did help people learn how to walk faster just one of the affected so Israel measured energy consumption in the local and in this version of the local You use about 50 percent less energy than you do compared to a therapist assisting you as you're walking OK so this kind of so we think that the slacking mechanisms are probably out of play as you're walking in the in the side that decreased their peak of fact so yeah.

[00:27:13]
So why was the gate restricted to the creator of the device I have never asked Gary that question it's probably simplicity of interface. So what I mentioned earlier is that when you have a device that you want to use clinically it has to be easy to set so if you ask the other question is like well what if you allowed.

[00:27:33]
The question what trajectory would you have used you have maybe 3 would have been better exactly based on height or something or something. So needless to say this you know there's now algorithms built into it to change the directories to provide feedback about how much effort the person is exerting and things like that so it's a constantly evolving thing but yeah that's a good question I think it's simplicity drove that probably that decision.

[00:28:04]
And then kind of mentioned the study already but we had the pneumatic robot that did the anti slacking control and it wasn't a homerun or a miracle it was still about you know we don't we still only got 2 or 3 points of the floor so I think slacking is important I think you have to prevent slacking at its extreme forms it's not a magic bullet we're going to really help everyone recover.

[00:28:25]
All right so what so what else could be going on so presumably are kind of building some new pathways as well we know that after a stroke you're there's a different. Tissue microenvironment that you express so whole different array of genes that are related to development and you have a exuberant exuberance of some synaptic formation that then gets pruned sort of simulating something happens in development so what we we sort of this is sort of a crude mathematical model that sort of can give a S. what we think is the factors that might be a play here so what we say is that the change in your outcome is a function of the number of practice movements that you make as your motor effort so you have to motor effort and that's what I've been talking about but that has to be correlated was some sort of sensory feedback some rewarding sensory feedback that's temporally correlated with the motor effort and fundamentally the aphids it's it's a constraint of the residual anatomy so if you blow out all of your cortical spinal track you can't there's not much recovery possible so there's an atomic will constrain here so we're interested in the S. part here so as would be some out of sensation coming from the movement and you know if we could sort of control the other things could we detect that this S. has an effect on the outcome and so what we did was.

[00:29:56]
We wanted to study this for the fingers because the representation in the cortex for your hand is quite large so we thought if it mattered it would matter the most for your for your hand so we're talking with Eric wall Brecht the University of Idaho. We have this a bar mechanism that can all assist you and naturalistic function extension of your finger and we have this actuator high bandwidth actuator so it's only a single degree group for a mechanism but we can really control it nicely and we.

[00:30:30]
Connected it to Guitar Hero 3rd most popular video game invented. It was programmed by a U.C.I. student. Here we have to mechanism so you have to move one figure or the other figure about fingers in time to the music in order to hit the notes as they come by and we work very hard the robots compliant we require that the person trigger the motion of the force threshold before we would assist them.

[00:31:01]
So we tried to have an anti fracking type of approach where. We divided people into 2 groups so we randomized 30 people astray in a chronic phase of stroke who plotted the recovery supposedly and we randomized them to get a high amount of assistance from the robot or a lower amount of assistance they did about $8000.00 practice movements over 3 weeks the highest this group in the lowest this group and then this is the this is the success rate when you turn the robot assistants on so this is the percent of notes that you're hitting successfully so with the highest that you're getting 80 percent of your notes across the song Lois's to about 50 percent when we turn the robot off people can only get about 20 percent OK So this is really frustrating to play game where you only have one out of 5 right this feels about right 8 percent feel good when you're playing it so you still can learn.

[00:31:56]
So. This is when we turned the robot off across the 3 weeks of the therapy you can see people got better higher success as they went along we readjust of the success. Of assistance of the robot applies Each week we found that if we did it we could do it in real time every session but if we did that we would it would be super frustrating to the person because they could never get better at it so we decided we'll just readjust it once a week and recalibrate it back down to 80 percent of it so if you've got a little bit better than 80 at the end of the week you get pushed back at the next beginning of the next week seem to work well similar results for the full Mar score here we're training only the fingers but we see a whole arm effect we see a better retention with a high or high assist at the fall we believe that may be related to dopaminergic mechanisms where when you have a more rewarding activity it releases dopamine and there are studies that show that that has a effect to some motor memory so we think this so here roboticist you attach a robot to a person that helps them to move that stimulates dopamine and has potentially a long term motor learning effect beneficial moldering effect OK So that's that's a hypothesis that's very interesting it really goes into the demands of neural modulators in psychology when you're physically attaching something that someone when we plot the change in the form our score versus the baseline we found that the high level of the systems was much much more effective for the lower level people OK So that's kind of an interesting result as well.

[00:33:39]
OK So but our main reason for doing the study was the look at that as term the sum out of sensory feedback and did it matter turns out there was no good way to measure fingerprint reception the way the way a doctor does it is he said close your eyes and move your finger tell me if I moved it up or down very crude So what we did is develop the same called Morgan ignorance and just a rote they did this thing or crisscross with this robot Now we could slowly move the fingers past each other and without vision we asked people to push a push a key when the fingers are overlapping OK So that's that's our finger perception measurement and remarkably we found that the amount of perception error that you have a baseline so this is on the X. axis here is the average angular error that you're making when you push that button in the crisscross task so high error means poor or poor pro preception and then here we have the change in our primary outcome measure which was 0 the functional measure which is the bottom block score and you can see it explains owns 40 percent of the variance of the training effect so something measured at baseline which is simply how well you know where your fingers are explains 00 most we could say almost half of how much you can benefit from this there be importantly if you have.

[00:35:06]
A large purpose up the air or you have just wasted 3 weeks of your time roboticists there be OK So I think that you know this is where the game is right now the robot says that there is explaining who can benefit from and why and how you can sculpt what the robot does to what the person needs to recover so we have a grant funded now to see for these people that sort of fail the purpose of the baseline could be engage them in purpose Cept of training and there's been very very little done on sensory transmitter sensory training of humans.

[00:35:43]
So we are looking for devices where you can think about what that means what do you need to do it's a matter of sensory training device so you need a video game you need the thing is to be a gaming paradigm it's a video game and for someone who's blind or it's a video game that doesn't use visual feedback to play it so the decisions that you make in the game have to be based on the movement of your fingers that you're not controlling that are being proposed imposed on you so it's an interesting you know sort of as a design problem for how how do you how do you design a game like that that's engaging that someone is going to do hundreds of thousands of practice movements or thousands.

[00:36:32]
Yes of course is you know what's the intuition for why a purpose option is so important so that intuition relates to have been learning so have been learning means why are you no neurons. Fire temporally coincident with each other tend to either turn to strengthen their connections let's just simplify it OK So neuron a fire is neuron B. fires temporally coincident with each other and then they strengthen so.

[00:37:00]
What that. Yes well that's a secondary question so why why wouldn't you just come to rely on vision to cement that some have been learning and so maybe it's because you have a dramatic enervation of some out of century Africans into the motor cortex in the visual input somehow as an F F F It is.

[00:37:24]
Affective in doing that yeah I don't know that's a great question actually yeah but it doesn't seem like the vision alone is capable of doing it. Something to be followed up there and we don't know if and then we don't know if this is measuring perception which is the ability to say well your fingers are in space we don't know how that relates to tack to all input which is another form of cement a sensation of the 2 correlated you require the tactile component as well we don't know.

[00:37:58]
So we also working with Steve Kramer a neurologist we've mapped out. And it's how Michael injury to the primary some out of sensory cortex the secondary cortex and so the level cortical pathway which is conveying the sensory information from from the perf per free up to these core disease and we found that the total in we looked at 60 different measurements of motor function different sort of demographic or clinical predictors and the ones that most strongly predict are the injury to the sensory system.

[00:38:32]
Predict your ability to benefit from the robotic therapy and we and the other one that was strongly predictive was sort of in balance of sense about a sensory input. Matter sensor activity measure with F. M.R.I. as you're doing sort of a Guitar Hero but impressing task as well so if you have this sort of abnormal lateralization of sensory input so it's at least for the head what's really interesting is that the sort of sensory some out of Century thing seems to really account for a large amount of the variance.

[00:39:02]
So simply if we break people into 2 groups that were based on a better baseline measurement you could see that we could double the effect of robotic therapy and we could prevent people from wasting their time so we're following up on that now interestingly we I think for the 1st time measured.

[00:39:22]
Regularly validated scale how sort of a survey thing how motivated people were to to go in the training so again the robots that are. More highly assisting produced more motivated more motivation for the therapy which is we think is probably producing this dopaminergic fact and we measured efficacy which is here we asked like how how how many blocks more do you think you will be able to lift up and we found that the people who had the highest actually thought that they would do better higher sense of self because he as well so get a robot that attach to people and augment them help you to move better have these motivational effects.

[00:40:04]
Are just briefly mention dirtied the sort of the opposite we've always thought if we can get a human robot interaction system that improves your golf putting then then we can we can have a funded lab without writing grants anymore so. Here we're putting in a virtual environment and we program the robot to physically assist.

[00:40:27]
You in putting So if you're going to hit the ball too hard we can sense that from your backswing and we have the robot slow you down so now all sudden we turn it on you're sinking every. Opposite we can also if you're going to hit it too hard we can increase your air so it will air augmentation we thought that would be a better way to learn putting right so we're highlighting your errors and causing to respond to them.

[00:40:53]
We had people come in day one. Day 2 we turn on the air augmentation or happy guidance and day 3 they come back and we see what they learned without any robot intervention both arrogant Taishan and haptic assistance they work about the same amount and they have a very small effect So unfortunately I'm not fully funded by Callaway but.

[00:41:18]
Really interesting was when we asked the motivational questions the group that got their augmentation from the robot they did they reported substandard substantially reduced motivation so I think this is an important activity that this activity was enjoyable when you turn on their augmentation it goes down that's not surprising what was prizing is on day 3 when they're putting just as well as the other groups and the augmentation is not on anymore they still report that they don't like it so there's this like persisting effect from robots that you know make your errors larger.

[00:41:55]
So just to finish up. Where's robotics Where is this headed So one thing I mentioned is you know there's this big opportunity now I think to do robotic therapy. Customized robotic therapy based on competition modeling there's a huge effort Greg So what he and others here were working on wearable robots.

[00:42:16]
Which combine potentially assistive and therapeutic functions there's combination therapy so we're doing a brain computer interface based robot therapy where the patient has to create a brain state favorable the movement before we allow them to move. There is Spinal stimulation Reggie Edgerton Gregor Corteen U.C.L.A. and E.T.H. respectively where if you apply.

[00:42:45]
Stimulation to the to the back of the spinal cord you can raise the excitability of afferent see again the sensor neurons and that seems to have a dramatic effect on allowing people walk and learn to walk and then we're coupling also robotic therapy with stem cell regenerations who have a project now with Mark as in ski so he has a stem cell modelers getting robust bridge formation across spinal cord lesions and we have a robot that gives nonhuman primates the opportunity to practice thousands of hand practice movements as they're getting the stem cell therapy the last thing I'll mention is consumer start tech So we're trying to start this new area of devices that people can just take home they cost $2300.00 they can paddle pocket and just continue so anytime freemen.

[00:43:34]
Was my Ph D. student working on the project I told you and then Professor Marc Bach then so he said let's just take this Guitar Hero idea and make it with conductive fabric that senses different groups and you play play Guitar Hero and I'll show you every researcher has to have their or one of their their favor top 10 favorite graphs of all time.

[00:43:57]
And this is one of mine so we gave it to 18 people with stroke to take home this is the total number of grips they completed over the 3 weeks that we let them use the device at home the red is what we asked them to do and so you can see during the 1st week they're sort of sticking with what we asked and then on average they started getting addicted to the game and doing even more so now we have a 1000 users of those globes so far.

[00:44:24]
It's therapeutically affective I won't go over that but that's that's basically clinical data showing that if you do that much better than having a book of exercises. Interestingly the company as they started selling this. They would people would call and say I'd like a device to continue my therapy you know my insurance ran out and I want to keep doing it and they had to screen people so you had a little bit of finger movement left to build to use the music love and it turns out only about like one in 8 people calling the calling had enough finger movement to use the glove so they came up with this idea which I thought was a horrible idea which has since arise PA They said script force and they have I am used in them and they work with physical therapist to design about 40 exercises so in the back there someone doing something with their torso you could put them on the floor and stop there called fit me well it turns out now they've sold out 4000 of those and people love it so just simply a simple sensors things that are keeping you sort of accountable to do a set of exercises seem to be extremely.

[00:45:28]
Popular and are currently we're doing a randomized controlled trial in this device this is a magnet and here we have a array of magnetometers So it's kind of a fit for the fingers so people after stroke they stop using their hand because it's clunky and it's hard to use the hand and what we do here as a measure this measure is how much you use your fingers every day we make about 122-0000 finger movements.

[00:45:52]
And you can do finger sprints which is the low Moji feedback here it will stop and give you positive feedback if you do have a hand Sprint which is moving your fingers a lot for about 5 minutes. Yeah I won't I won't go into much detail Suffice it to say that when we're giving at least for our 1st 6 subjects when we they're wearing it they're willing to wear it 8 hours a day they're performing substantial number of extra movements per day when they have their way.

[00:46:23]
So that kind of last idea is. You know I started realizing well we have this company they've sold over 5000 devices and this data is the logs there's now 30000000 private practice movements that been logged so we're we're moving into this new era of rehabilitation where it's possible to do big data I think for the 1st time in rehabilitation in this link back into this idea of computational modeling it could drive those types of models.

[00:46:51]
This is just showing there's a the debate about the appropriate dose that you need to achieve rehabilitation there is this wonderful paper published by Jeffers this year they were doing a rap model a stroke they had done hundreds of rats they put them all in the same plot and they found that the rats that did less than $300.00 reaches per day really had no effect of rehabilitation you had to do more than 300 and there was a large randomized controlled trial funded by recently to do high dose therapy and it was right here OK and everyone super depressed in the field right now because of that because of this trial because it was a high dose right it was supposed to solve it so I think music love we get about here theoretically the main new meter is going to be out here now those are very specific and directed movements so we'll see what their therapeutic effect is but there's still a lot of work to be done and automation of rehabilitation and I'll just finish with this.

[00:47:46]
Class with junior level lab class where they build a pneumatically powered robot at U.C.I. and so 100 robots there thank you very much. Higher. I 100 percent agree with that statement I think it remains to be demonstrated that you need the physical actuation Now I did show you data that.

[00:48:34]
Physically assisting improve motivation and I did show you data that integrity of purpose of the pathways is required to benefit and so that would imply that it's purpose up to stimulation that sort of driving the recovery and we know the robot is providing purpose up to stimulation. It could yeah you know the argument would be the robot might be applying sort of more normative pattern of purpose at the stimulation because when you're not you know it's helping you move in a different sort of state space of perception Yeah.

[00:49:13]
Yeah yeah. That's very Yeah exactly yeah that's exactly right yeah robots for you know for people are less impaired don't make any sense to me at this point but other people and less they're augmenting or something like you imagine something you're wearing that slightly perturbing your balance as you go and making a better balance or as you go yeah.

[00:49:39]
Yeah. Yeah but I completely agree with you is that you know what we haven't seen the killer experiment the show that the actuations necessary I have one design for this next script I hope but will see and will see their way yeah. Yeah are better. Yeah safer Yeah I think that one reason that the army of Spring has sold better than the other ones is because it isn't actuated and it just subliminally feel safer it's not going to move on its own it only moves if the patient drives it right so question.

[00:50:18]
It's support right it's just not motorized driving around yes. Right. Yes So that would be your it seems like you want to add have control difficulty or to put it another way I think you should have about 8680 percent success when you're playing these games so it's great so.

[00:50:54]
You can get what we do so we just analyze data from the. Home use of the guitar who are the people self adjusts the difficulty as they go along they make it more difficult so you can choose song complexity and you can choose the number of groups that your and people do on average increase their difficulty self increase now you could do that with an algorithm as well you don't want to do it too frequently as I mentioned because it's self-defeating to the person they never see themselves getting better but it's a level up as is what it is but yeah you can provide the higher success and reward through appropriate leveling up.

[00:51:31]
Yeah. Yeah I think I've seen that yeah yeah. Yeah in terms of like so 102030 percent improvements that kind of thing or yeah. OK having his arm extended like those to hold the bike. Break OK yeah the tree. Yeah. Couple companies doing sensory stimulation a camera 15 minute tech Yeah yeah yeah I think I think.

[00:52:40]
I'm even more so after we did this last study the sensory stimulation idea is really important I think to follow up on it sounds like I should read more of your stuff with the the vibration it's showing sort of vibration demonstration to show which finger to do that OK yeah.

[00:53:07]
Yeah I like this it yeah like this is I like to try it out at some school for Jane Yeah I met her and she said Janie are yeah it's created Jr is. It's a 3rd ranked rehabilitation Journal and our are sort of sweet spot is technology innovative technology coupled with some neuro science coupled with rehabilitation and so our and our associate or combination of clinicians and engineers and neuro scientists actually so so yeah it sounds like it would fit for it really well for that.

[00:53:55]
Yeah the music love business model is completely new so it's if you pay out of pocket We're not trying to get insurance reimbursement we're trying to. Yeah yeah. Yeah I didn't understand the last part. Yeah we talk to you know therapist or if they're in private practice they have some leeway if they're in a facility and I can speak to this usually there's a there's a person in charge of buying equipment and they have an annual budget and they can select things if it's under some level they can sort of do it without too much approvals and if it's over some level they need approval.

[00:54:47]
But there's a lot fewer. Yeah. Yeah. Yeah. Yes. Our neural new neural network your muscle a church or spine for example in your bones all that that's good yeah. Yes that's right. Yeah yeah. Comets are fine. Yeah. That's right yeah yeah yeah. It's a case of the you know the work there's been a lot of work on in term while formation generally that adaptive system at least from what I know it forms models on orders of tens hundreds of movements maybe a couple days with PRISM adaptation or something like that when realisation really had time constants that look like 15 days 20 days OK So there's a fundamental mismatch already between it's not the old reality is that fundamentally in fact we just published a paper on this and we looked at people learning to use the arm in spring and there's 2 time constants there's a quick one that corresponds to learning the dynamics of the exoskeleton that happens over days couple days and there's a longer one that happens is the recovery time constant OK So ensure a model formation and that part of it to me is a spur smallish part probably of what's required to recover Yeah yeah yeah slower process yeah I think that's right yeah yeah what is the prism out of Titian time constant you know.

[00:58:05]
OK. Yeah. Yeah Yet no one really knows what the learning mechanisms are that are in play during And personally I think it's not one it's probably 10 that are sort of interleaving and that's why it's so complicated to know what optimize the robot because what it does to help one process might not be helping another one of the I'm using a robot generally here as a technology that's giving you an environment to train and yeah.

[00:59:00]
Yeah yeah yeah. Yeah. Yeah. Yeah. Yeah well I mean that's a wonderful question and that's the type of question right path is this we need now with this big database that we have we've got 30000000 movements 5000 people so we don't have permission and we don't have a personality test but you could design it for the next 5000 people you know or you could say sign up for this study we'll give you 50 bucks and you give that you felt this test and keep doing your therapy but now we know I don't know anything about that so.

[00:59:56]
Yeah yeah I'm trying to think if I have heard about I don't really know do you know about psychological testing for people benefit. Yeah. Yeah yeah. Yeah. It you know it's probably is part of why a robot will never replace the robot can take take some of the automated things from a physical therapist but physical therapist is a good one like a coach and a look at you figure out your personality and then do exactly what you do to need to be motivated and so that sort of human intelligence part of it social interaction part which is maybe more difficult to automate I think is a big part of what makes a good physical therapist Yeah yeah.

[01:01:15]
Like.