[00:00:05]
>> So it is my pleasure today to introduce Dan Farris who is the Robert w. 8 and professor of engineering innovation at the University of Florida in the department of biomedical engineering so Dan got his ass at the University of Central Florida. Miami and then his Ph d. from Berkeley then he went on to do 2 postdocs want us u.c.l.a. in the partment of neurology and a very different University of Washington compartment of engineering electrical engineering so he's got a pretty well rounded.

[00:00:41]
Education before Michigan where he was out for 1617 years before the transition to University of Florida pretty recently and. I think 2017 so he's really Carvel himself as a leader in human machine interaction to improve human performance in mobility he really started in the robotic exoskeleton world where he was really the 1st of quite a few people from the early 2000 to serious physiology approach to measurement and performance which a lot of the field later followed years down the road but was he was kind of in the only one in that class at the time which really helped distinguish his program and then credit to him he expanded and went to doing beyond it prostheses and especially mobile brain imaging with which he's being a lot of notoriety and more recently and so he's received a lot of prestigious awards including a fellow of the National Academy of.

[00:01:41]
The American Institute for Medical and bio biological engineering editor in chief of Tripoli transactions on Neural Engineering and rehabilitation and he's standing member of the I know each musculoskeletal rehabilitation Sciences study section and one in other words he received a n.s.f. career early on it and later got the Founders Award through s.p. which is quite prestigious and another thing that I just want to mention about Dan So he's been a mentor to a lot of different students including myself I was a post-doc with him a few years ago before coming here and he's he takes a very serious role in terms of mentoring and a lot of his students are now.

[00:02:24]
Out a lot of the prestigious institutions as faculty members which really kind of shows his capability on the mentoring front with that like to welcome Dan for and I seminar today for him. Thank you Thank you Aaron very much. So. If I could leave you with one piece of advice if you're going to go into academia definitely find that nobody else in the world is doing because that really helped my own career when I started out as Aaron mentioned we really started combining biomechanics and physiology with exoskeletons like nobody else did so it was really easy for me to say I was the best in the world at what I did I was also the worst in the world that what I did but I didn't highlight that portion of it because I really was the only one and since then I've had my former students like Greg who Akiane Aaron Young go often do better work so I've moved over to more neural engineering with a lot of my research but today I'm going to talk about the robotic interfaces and some of the work that we still do.

[00:03:30]
So when I actually started out very fortuitous this hit the airwaves yesterday it's the latest video from Boston Dynamics highlighting. The robot and why is this so surprising and so. Popular in the social media anybody got a suggestion why is this particular so amazing to see is it because it's using little energy to do it.

[00:03:59]
It's because of the agility we did not expect that by people robots would have this level of agility and that's going to be another common theme that I talk about as we move along I actually got into science because I was an athlete growing up played football in high school and college not for Florida but University of Central Florida but Emmitt Smith back then was a contemporary of mine to play for us and I was amazed at the level of the.

[00:04:28]
Physical capabilities the high level athletes can perform and I was really curious is how do you study humans as a machine understand what's going on from a biomechanical from a neurological from a controls from a energetic standpoint and so I originally started out thinking during my Ph d. at Berkeley that this was sort of the the domain I wanted to learn I wanted to understand humans as machines from the mic biomechanics from the motor control in the energetic standpoint and that's where I sort of went from doing a master's in exercise when I started in undergrad in math master's in exercise physiology ph d. in biomechanics post-doc in neuroscience 2nd post-doc and robotics because I really want to understand humans as a machine but over the years it really has morphed into how do you combine human and machine together I grew up reading comic books was youngest to 6 kids and still my older brother's comic books all the time and it's nice to now go tell tell my mother see it all of those years of reading comic books wasn't a waste of time it actually had a positive impact on me because now I really do research in the domain of robotic exoskeletons.

[00:05:38]
And when we think of robotic exoskeletons right this is the science fiction mass media perception of what an exoskeleton should be most of you know Tony Stark and. I mean but there's other movies like Edge of Tomorrow with Tom Cruise in this was at least him with Matt Damon but this was a better picture of his rival that's got the new updated exoskeleton the bad guy in the movie this is really what science fiction wants from us they want these exoskeletons that we can slap on that make a super human and can do many many things multi-tasking.

[00:06:13]
The reality is we don't have anything approaching that yet but we do have the capabilities to build a lot of hardware. If you look at all these real exoskeletons everything in the top row is designed for individuals that have no disabilities this is giving them superhuman strength or superhuman insurance somehow allowing them to do something that humans normally couldn't and everything in the bottom row are exoskeletons designed for people with disabilities and their either to replace walking ability that they don't have any more like a wheelchair would would or they're designed for therapeutic intervention so can we help a physical therapist by assisting them to practice walking and then take the exoskeleton off and let them walk.

[00:06:59]
All of these devices are out there and there's many more this is just a sampling I would say that the limitation right now is the control we can build hardware what we don't understand is how do you allow a human to wear them and to move fluidly and to be agile most of these devices are severely limited in how agile they are and so they get rejected none of the companies working on these devices are yet producing profit with their devices they may be selling them or leasing them but the amount of money they've sunk for research and development has not been recovered to the point where they're actually a profitable company yet and it's still unknown how soon it's going to be till we get there.

[00:07:43]
There are many ways to sort of gauge this if you're not familiar with the Gartner hype cycle there is a consulting company that specializes in emerging technologies and forecasting where technologies are going to advance. They have this nice hype cycle where the x. axis you have the maturity of a given technology and on the y. axis you have visibility how common is it to see in mass media.

[00:08:08]
Generally what you see is there's a normal trigger some innovation that allows the technology to start being built you get up to a peak of inflated expectations where it's all over the place think nanotechnology and then eventually it falls down through the trough of disillusionment and it's not as common in the mass media and then you get the people keep working up the slope of enlightenment and eventually get to a plateau of productivity and the plateau of productivity stage is really commercially profitable companies Ok it's actually allowing companies to sustain themselves they make a prediction every year and human exoskeletons is bet on that prediction for a number of years the latest $28.00 team version that I have if you look right now human augmentation which encompasses everything want to talk about today both by Anik prostheses and robotic exoskeletons we're still only halfway up the slope of inflated expectations Ok so there's a lot of work to be done right now I would say you are if you're working in this is a graduate student or beginning assistant professor it's a great time to be in the field there's a lot of really cool advances still left to be made.

[00:09:19]
And if you notice they give it a little yellow triangle which means that it's got more than 10 years till you reach that plateau of productivity I think that's a fair assessment is that there's not going to be really profitable companies for about 10 years but you'll start seeing that transition about 5 years from now with more and more around it's a good time to be in the field.

[00:09:42]
One of the points that I made is we need more focus on agility generally the gold standard has been let's put an individual on a treadmill ethic at a set speed constant speed walking and try to reduce energy cost that's one advantage of having exoskeleton if you can reduce energy cost but I really like this graph even though it's a little busy from Michael or indoor from colleagues where they actually measured humans walking around in the real world in there every day for a week and he what they wanted to know how do people actually walk and so what you have on the x. axis is the number of steps in a row that they took in a walking about versus the y. axis which is the frequency of that means how much of all their walking of the bouts was done at that number of steps and I'll point that there's a quickly declining trend is that the most common walking about was 4 plus or minus one step so when we think about building an exoskeleton for walking for a long distance at a constant speed that's really not how humans move around in our world they generally take 75 percent of all bouts were less than 40 steps you're not even getting up to steady state in terms of energetics with 40 steps so one thing we really need to think about is let's build exoskeletons and prostheses that actually benefit the user the way they want to use it which isn't walking on a treadmill a constant speed for a long distance but allows them to be agile and starting and stopping and having a lot of capability if we look at the real world there are lots of obstacles and there are changes and terrain that we deal with they can be naturally going on a natural train rocky terrain.

[00:11:29]
Putting your feet and given the stepping stones or foot placements even in a city urban environment you see changes in curb changes in the material of the sidewalk versus the road and if you start getting into a crowded urban setting and then you have obstacles people that are walking around you that you have to avoid you're not walking in a straight direction at a constant speed you're moving left and right you need to be agile to move through your environment that's the way people move Ok so how do we get there.

[00:12:02]
Early on if you know the history of robotic exoskeletons DARPA had a large role to play around 2000 they decided they were going to start a robotic exoskeletons for human performance augmentation project where they give out millions of dollars to people to build superhuman exoskeletons but the teams that got it were very much focused on the hardware in engineering and did not take into account the end user how they were going to use it what the movements were like and what the physiology and biomechanics of them were so you actually look at how successful they were and neither one of them actually ended up with a commercial project product.

[00:12:41]
Might take back in 2000 as I was starting as a faculty member was to learn how to build these robotic exoskeletons and do it in the lab to try different controls and look at the physiology can I study the biomechanics can I study the motor control can I study the energetics all together because they impact each other with various robotic interventions and this is a hip exoskeleton that we had tested we've built many of them over the years but the controller even then was still a limiting factor how do you build something that moves with a human in a very fluid way.

[00:13:14]
Traditionally the control approaches have been taken from by people robotics require Pead robotics where they want to do these finite state machines they put a bunch of sensors on a robot that measures foot placements joining goals. Trying to infer what the state of the locomotion cycle is and then manipulate the impedance the stiffness the damping of the various joints and provide a trajectory of control that was the original way that a lot of the exoskeleton start even go back to the 1970 s..

[00:13:46]
And we're still sort of stuck on it and I would argue that what we really need to do is get 2 other alternatives that allow for biological input so when you're sensing motion. Kinematics and kinetics it's going to happen after the user had already decided this is what I want to do let me start that motion that exoskeleton then has to infer what the human wants and then move along there's always going to be this inherent delay no matter how good we get at these finite state machines and the other alternative is to take some neurological signal in from the user and maybe cortical implants where you implant electrode in the brain it may be e.g. where you put electrodes on the head and figure out what's going on at the cortical level that's not invasive or maybe where you put sensors either on the skin or you implant them inside underneath the skin in the muscle and transmit the signals wirelessly out all of those allow you to get feed forward commands from the user that's the way we control our bodies we have these neurological signals we send from the brain to the spinal cord to our muscles and that's what causes the motion and if we're going to have more success my one of my arguments is to get agility we need some of these signals and I think that outcome is really going to end up being some sort of hybrid control taking advantage of what we can from machine learning techniques taking advantage of hybrid state machines but using some sort of neurological input.

[00:15:19]
So over the years we've built a lot of different robotic exoskeletons and our intent was never to build it exoskeleton to sell we didn't really want to do the commercial and we knew that there was a big enough leap that need to happen in the control and that other people working on the hardware let's build something that works in the lab and then try to control structures and measure the physiology biomechanics find out what works and what doesn't here's an example of some of the early ones AHIP exoskeleton in the ankle and ankle exoskeleton a lot of times in our early stuff we use these robotic artificial pneumatic actuators sometimes called McKibben actuators they're like specialized balloons you pump air and they get short and fat you let the air out they get long and skinny I like them because they produce high power mechanical power they're very compliant so they're safe for human interaction.

[00:16:11]
And they're very lightweight because you can put the compressed air off the human if you're going to study in a laboratory or a clinic put the compressed air off and pump it in and it's very easy to use it's not very portable so if you want to build a device that goes around campus or goes to the mall or works at home you need a scuba tank on your back not so viable But again early stages we know the limitations are control and interaction Let's get that right 1st one of the strategies we came upon is something called portion of my lecture control where if we put electrodes on the skin we can record the electrical activity that activates the muscle to create motion and so here if we take the soleus electrum agrah fi we send it through a computer interface we send it through a pressure regulator.

[00:16:59]
We can actually activate the artificial pneumatic muscle in a feed forward pattern based on what the humans intent is so this is what it looks like in action. So what you see is that it's actually Keith Gordon's legs going up on the toes the electrodes on the skin in between the scope you see.

[00:17:37]
Because you see the bursts of electrical activity and then on the side is the ankle exoskeleton showing it contracts and real time the electro mechanical delay between the start of the electricity that we were cord and the burst of the mechanics the increase in force in the exoskeleton is on the order of 40 milliseconds which is very similar human electromechanical boy so we 1st brought in a bunch of young undergrads and who have never walked an exoskeleton put them on a treadmill and tried to figure out how they adapt to this is what looks like.

[00:18:22]
You'll notice it almost knocks them off the treadmill. The other thing you'll notice she's walking on her toes she's getting so much power and torque out of her ankle she doesn't know how to control it so. It actually takes about 30 to 45 minutes to learn to walk normally using this type of actuator controller and we're going to take the power away in 30 minutes later after she's starting to walk normally so it looks like.

[00:19:00]
You'll notice she stuck her steps and if you look she's moving left and right she's very unstable and she makes the comment. While your leg feels heavier what's happened in that 30 minutes is you can learn to turn your own muscles down and you rely on that exoskeleton and then almost And when you have they're so skeleton gone you feel like that's the same signal I was sending but I'm not walking the right way Thankfully it's only 2 or 3 minutes before you're back to normal because that's your standard mode but you do have to learn to turn those muscles up again so some of this was our early work we've gone a lot farther over the years one of the one I recent students Jeff KOHLER who's now Villanova worked out of the depth of game controller So essentially what happens if you learn to turn your muscles down now you're getting less power out of your ankle the exoskeleton So what if you had an adaptive game that learns to adjust on the fly to get maximal power as you need it and so what he did is he came up with a way to do this dynamic game calculation based on a running average of the past few steps and basically we wanted to know do people adjust differently when they work with walk with an adaptive gain so these subjects had one on each leg and what you're seeing now is pite charts that show the mechanical power of walking.

[00:20:23]
With each of the slices of pot for a different joint the ankle is the darkest The knee is the medium gray or the medium color and then the hip is the lightest what you see is that in the average unpowered condition your hip does most of the work for walking on level flat ground at about preferred walking speed when you start learning over time with an adaptive gain you realize you can get a lot more power out of the tech so skeleton than what you were using and you learn even though you're turning your muscle activation down you get more work and so if you look at the percentages you shift from a strategy of 28 percent of your work coming from the ankle to 41 percent of the work coming from the ankle because the exoskeletons boosting it so better to walk with a different pattern that is relying on the ankle than the hip because you get it for free with the exoskeleton So this is a good evidence that people do adapt to long term especially if you can give them adaptive controllers so I really am a strong believer we need to increase agility and task to see how well it works in the real world new batiks is not going to work out in the real world and so what we're now is we're working with defy which is a really cool set of ankle exoskeletons and the nice thing is they let you hack the controller so in this particular angle exoskeleton Rachel one of my Ph d. students at Florida has put a controller on it so we're actually activating it based on the same way that we did the old ones but we can walk around campus and do different things here's an example of one of our subjects walking on a treadmill using control and again it's just on a treadmill so that we can make sure they're adapting but then we're going to take it out into the real world have them walk around campus have them walk in different trains start to measure not just metabolic cost on a treadmill but how they are adapting to obstacles and to being able to change their direction.

[00:22:22]
Finite state machines work really well especially if you do human in loop optimization for steady state locomotion but in the real world if people are starting and stopping and going left and going right a finite state machine that's got a limit cycle control is not going to be able to adapt unless you've got some really good machine learning that's on the front end teaching it how to adapt the limit cycle for all the different conditions so let's get to agility How do you measure agility so if you live strictly in the robot world one way to measure how agile are your robot is is to take it on different trains give it different conditions so this is Kasey University Michigan going on snow and sand and grass and pavement trying to figure out.

[00:23:12]
You'll see that it's fi or sinking really heavily and so they actually come up with a novel solution sometimes simple mechanics can help you a lot so this is barefoot and then they realize because of the small feet that it wouldn't really need with shoes. This is some really cool work out of just a grizzles lab small changes in the mechanics can make the control work a lot better.

[00:23:41]
So our in the lab 1st approach at this is to build treadmills that actually can do a lot of different types of terrain. So here's an example from another series of studies of one subject walking on treadmills So this is a really neat solution where we actually print hard foam.

[00:24:02]
Circles and discs of different heights and we can Velcro them right to a wood way treadmill about so that we're walking in a very uneven terrain this particular study was not about exoskeletons but more about brain activity with high density e.g. we can actually measure what's going on in your brain while you're walking around and this idea was to try to 1st get a baseline of what's going on in your brain while you're dappling do different types of training the next stage that we're going to get you down the line is then using.

[00:24:31]
Robotic exoskeletons that at the ankle and try different controllers to see how well different controllers allow for adaptation to an even train we also work with companies it turns out a lot of these companies invest millions of dollars to try to get the hardware better and it's much more than we can do in our own lab if we're going to look at the physiology the biomechanics the motor control and change different controllers I would much rather have a system that we can hack and put different rollers on so one of the ones we work with is a company in Canada called the team this is their k s r d nice stress relaxation device which is actually a really nice piece of machining an engineer engineering it's a knee exoskeleton it doesn't help with the ankle it doesn't help at the hip but it can actually provide pretty high torque so about 30 new meters about 30 watts of power which is fairly large especially for walking on level ground and it's fairly light weight so their goal is to use this for the military to carry heavy loads so long distances try to provide some assistance does it actually do that.

[00:25:41]
So we designed a quick little experiment knowing what we know about the biomechanics of human locomotion the knee actually doesn't do that much work if you remember the pie charts I showed you the hip in the ankle where the 2 big contributors to work work over level locomotion the me is a relatively small amount it does mostly negative work so we actually were not expecting this device to work really well to reduce energy costs on level but where it should shine and do better is going up inclines or carrying loads when the knee has to do more positive work so we set up an experiment with subjects with no xo no load x. so with a load $35.00 pounds in a backpack and then incline and not incline this is what it looks like this is Andrew Norden one of my he was a post-doc at the time he's now a research scientist with me in Florida he's walking at the level with a backpack load and going up hill.

[00:26:45]
What have the results show it turns out exactly as we predicted in the hypothesis is if you look at the difference between how much does the exoskeleton help and not using the exoskeleton So knowing that there is a penalty for having the extra weight of the exoskeleton on were including that in this calculation comparing are 4 conditions no load on the level load on the level no load incline and load incline the biggest benefit by far and the only one that was to sickly different than 0 was when you're carrying a load up an incline and you're getting the benefit and again it you have to take into account the biomechanics What does the knee actually do don't just build the hardware but think about what niches you can fill with the system that you have.

[00:27:29]
They were adamant that no out in the real world when you have an even train this is going to benefit a lot more so we said fine let's do it another way instead of in the in the lab on the treadmill let's do a natural outdoor train if you've ever been to University of Michigan you know they have an arboretum on campus so we set up a fairly long path about 1.3 kilometers around natural train up top is the loop that they took down here is the elevation so it actually shows that they go a steep decline in the incline their path and there's Andrew walking through it and we measured energetic so if you know anything about the physiology of measuring metabolic you normally set up your condition so they reach steady state out in the real world you don't reach steady state so you have to take.

[00:28:16]
With a grain of salt the data to show you but essentially this is what we found so what we have is the metabolic power on the y. axis versus the walking course. And then at the end a little bit to equal amount of time of standing what you see is that the exoskeleton was always greater than 0 exoskeleton throughout the condition so it's actually not helping you without the backpack load but even perhaps a little bit more problematic is even when you throw the backpack load on.

[00:28:45]
There was no statistically significant difference so it wasn't hurting you so much there was a little bit of a increase with red but statistically it wasn't there so what you end up with is yeah you could use it or not use it outside but it's not really helping you unless you're climbing up a steady incline the whole time.

[00:29:04]
So now we're working with Lockheed Martin they've been a really good partner they have a modified version of the old k s r d they've leasing it from the team made some modifications working on with them to try to measure not only level walking but we're actually doing a host of different things some Gelati drills about getting up and down from a prone posture to standing up doing things like climbing up and down the stadium steps at Florida stadium the swamp.

[00:29:34]
Trying to figure out when it works when it doesn't work measure the biomechanics measure the physiology Ok so let's take a little side note here and I want to explain to you why you can't tickle yourself this is actually some really cool work by Daniel Wolper and his crew a while back but it was very clear you cannot take yourself in the same way that somebody can tickle you and why is that he did a series of experiments and basically it comes down to the control theory of how we process sensory information.

[00:30:10]
So if you have a desired motion I want to reach over and pick up this. Water in my nervous system I have an inverse model of how my nerve my muscular skeletal system works so that I can send in the motion and get out a control signal that needs to go to my muscles I send it to my muscles the motion happens.

[00:30:32]
I get sensory information back about that motion and then I actually compare the expected sensory information with another pathway the events copy so when I get the signal that should go to my muscles I send it to another part of my nervous system which interprets. With a forward model what the expected sensory information is so I compare the 2 and any errors go back to my inverse model and update this is how we move and there's been a whole lot of really cool studies that show this when you try to tickle yourself you are predicting that sensory information that you're going to get so it doesn't tickle Ok the point that I want to me is that for your body to work in its normal mode of physiological control you need a internal model of how your limb is going to respond you're going to send a command signal you want to know then how the musculoskeletal motion should happen and what the sensory information should be so they did a really cool experiment where they basically had.

[00:31:41]
A while though system a tell operated hand try to tickle you that you were in control of so instead of you directly tickling your hand you tickle operator and then the machine tickles you and they entered in different lags of time because they wanted to know if they entered more and more lags at some point do you start tickling yourself and the short answer is yes so if you look at the tickle rating rank which I love.

[00:32:14]
Versus different time lags going from 0 self produced 102-0300 milliseconds to purely robot produce so you had nothing to do with it if you have a 300 millisecond lag or even a 20 millisecond lag it's just sickly very similar to the robot produced. So when we have these robotic exoskeletons if we enter in lags between our intent and the outcome it makes it hard for the nervous system to adapt and it makes it hard for them to break that sensory information Ok we're having pretty good success with a robotic exoskeleton to understanding basic principles and me being a child of the seventy's wanted to move in this direction how many people in this room know who these 2 people that raise your hand you're basically doing an age test we have 3 of us.

[00:33:06]
Ok I grew up watching Steve Austin the 6000000 Dollar Man and Jamie Summers the by Anik woman as a kid and I thought this was super cool is that we can build robotic parts that replace the limbs they've lost for some of you that are a little bit younger you probably know the same sort of mindset is how cyborg from Teen Titans or Teen Titans Go.

[00:33:30]
Up or rates he's lost his arms and his legs and now he has robotic parts that assist this Ok can we use similar types of control to produce control by Anik limbs there are actually more and more devices that are being sold that are really by Anik in the sense that they are electro mechanical they add mechanical power to the gate and that you can wear to replace a limb Here's just a couple of them biome pre-meditation spring active.

[00:34:05]
Be typical way that these devices operate though is back to that state based control with intrinsic sensing So what they do is that there is a mechanical linkage between amputee in the processes there is a mechanical connection between the prosthesis on the ground when it's on the ground there are sensors on the prosthesis that sense it's position it's kinematics it's velocity acceleration that goes to the control board.

[00:34:31]
Determines what impedance position trajectory should have then that goes back to the processes you will know in this whole control loop the human cannot change what is going on if you are running along with a prosthesis that operates in this mode and you see that there's a step down in the curb you can't do anything to your prosthesis to change it but you have that vision in the information that you normally have why not find a way to alter the control of that processes using some neural signal so what we've figured out is that we can actually mount electrodes inside the socket and use that to deduce a signal that can go to the processes.

[00:35:14]
Again I go back to my 1st generation build something in the lab let's test it out see if it works here is our pneumatic. Artificial muscle prosthesis this is work of stuff and he was wrong for her dissertation stuff and he's now off an apple she says she's doing something similar to this work but she can't tell me so I'm curious to see what eventually happens.

[00:35:34]
But she's working on emoji control prosthesis this was her dissertation and here's an example of how it works. You'll notice the subjects can actually control their ankle go up and down on their toes this was the 1st prosthesis in the world that would allow that you had volitional control of the ankle yet.

[00:36:00]
None of the other finite state machines could allow this because it was relying on filling the limit cycle of what it expected you wanted to do. So it worked as we expected which was very good and we liked that aspect but was more interesting to us was the comments the qualitative comments we got back from the users having trained with it and I'll give you an example of some of them it's about the same way as the other in this case the other was the biome but it looks and feels more natural walking than the other one so I figure out what it's doing it can figure out how to operate and it picks up really quickly what I want to do you have that.

[00:36:52]
What I will call him body meant that the user is adopting this prosthesis into their body schema and trying to control it and adapt it on its own. Another one Friday session was extraordinary for me just the brief experience of moving a limb I had not had contact with since 970 was a total tree picture Christmas for a 5 year old very pure Great stuff it was mostly it was an extremely obvious but both my eyes were tearing all of them had this strong emotional connection to their device one of them said I haven't seen my ankle moving in 8 years and we're like that's not your ankle but go back to the way we control our bodies and we're sending this electrical signal through an internal model and we think that if I send this electrical signal and I see the motion then it starts to be adopted as to I'm in control of this and it's happening so I'm a big believer that there's a lot of researchers that are trying to add sensory information from the prosthesis back to the user that's great but I think don't discount the control we want to put the users in control of their device not to be a writer Q her colleague of mine that actually we were working on a related project is a bilateral amputee I had been trying to convince him for years to do you control he wasn't a big believer he finally did emoji control on himself for his own by across the seas he's 1st comments to me was the differences between riding in a car and driving the car you can get car sick riding in a car if somebody is turning left and right because you don't have a good prediction of exam even though you know where they're going you don't have the millisecond timing of the expected sensory information if you're driving you have a model of that car in your mechanics to know if I turn I get that sensory information so I'm a big believer we need some sort of neural signal going to the processes.

[00:38:43]
Ok so what's the next step we want to take this mobile we want to go to different terrain we want to measure agility Nicole Stafford whose mother actually is a faculty member here Georgia Tech look in the back of the room there's presser staffer down so to call is a great ph d. student of mine in mechanical engineering at Florida she's working on a portable electronic chemical powered prosthesis with a series last actuator we bought the actuator from after onic love and they do great work.

[00:39:11]
And we've put it on the biomechanical we can actually get very good range of motion I think numbers roughly around 35 degrees of planar flexion and around 15 degrees a door selection so we get really good range of motion with this system we can put our electrodes in the socket inside the socket.

[00:39:30]
Dorsal flexors implant if lectures and then control it this is the initial data coming out of Nicole studies one of the subjects and funny enough this subject is a professor and so he's it's so fun to work with him because he has such a great attitude about the learning aspects of it.

[00:39:48]
But we're working on implementing the emoji control so that we can get to measuring agility as we move on. Long term I don't think the solution is going to be use electrodes on the skin there's a lot of really cool things like these flexible electronics where you can mount electrodes on the skin but I think long term if you really are an amputee and you need a prosthesis the solutions kind of probably be something like what Richard Ware is working on a universe Colorado Denver implantable my electric sensors are big grains of rice size you inject them with a hypodermic syringe into a muscle and they transmit the electrical activity wirelessly through the y. fire blue to through the skin and can be picked up by the processes a lot of potential turns out he's already started doing this with some of their r. and d. efforts you can go online and actually if you do a search for.

[00:40:39]
Processes see some of the videos of them in action I definitely think there's a lot of potential there so wrapping up I'm a big believer that it's a great time to be working and Humans are machines we're not cyborg and Iron Man yet but we're working there and we're getting closer every day.

[00:41:00]
To take home messages we don't want to design exoskeletons that can do everything like Iron Man suit we want to do niches find something where you can be successful and build it so the user will be happy with it for the cyborgs for building by and it prostheses get the user intent somehow some neural signal coming out of the user so that they can modify it it's probably going to be a hybrid controller in the long term but you're going to need some feed forward commands from the user to be successful.

[00:41:30]
And then lastly if you do work in this field and I've talked you into it please realize that there will be some that will take these exoskeletons and turn them to criminal use so with your great power of being able to create exoskeletons comes great responsibility. I want to give a shout out to a lot of the people that worked on this you'll notice.

[00:41:51]
Greg is up there Jeff KOHLER Keith Pichon And Rachel Jeff when Sam and Kara Nicole. Nicole Stephanie and the funding sources lucky to see d.m. r.p. and s.f. and I manged thank you very much for listening to. Questions constructive criticism. So where do we incorporate. Sensors into the control framework and the answer that I can tell you is we do it lots of places there's not one path that we think is best a lot of our designs right now are testing out different strategies.

[00:42:55]
There is a non-linear relationship between muscle recruitment and the torque produced at a joint So if you give the nervous system some path that it realizes that if you increase emoji you get greater torque it can actually back learn whatever relationship you give it. So it can be a simple relationship where you take the m g you low pass filter it you do it a threshold you have to be above a certain level you put again on it kaput out at some level and turn that into a torque signal from 0 to 10 amps of control right that's just very straight forward linear transit translation that's one possibility it doesn't have to be that way we've tried lots of different possibilities I can tell you that there's not a lot of variation because the human learns So if you give them something to learn they're really good at getting better at it over time and so I don't know what the best humor has an m.g. controller now that they use but they actually do it a different way they use a finite state machine to figure out the relative damping in the stiffness.

[00:44:02]
Of the joint at different phases of the gate cycle it 3 or 4 or 5 phases and then they use the to amp that stiffness or amp that damping up so that it uses the same state machine but it moves it up or down that's another way that's different from just using it as a pure torque control system.

[00:44:32]
That's a really good question and I think you could ask that to lots of different people and nobody would agree. If you actually use athletics as a good test case right because they've been doing this for years go to the n.s.a. n.f.l. draft or go to college football.

[00:44:50]
Camps where they're bringing high school recruits in there trying to figure out who's going to be better they really have vested interest millions of dollars at knowing who's more agile and they've come up with a lot of tests I'm not saying that's the gold standard but there are tests out there.

[00:45:06]
3 cone drills where they actually have them navigate move around and come back right. There are other agility tests like I showed you would Cassie put them on different train can they deal with the fact that each foot fall is going to be different in terms of the interface that's another test for Jodi presumably that you're dealing with this an even trained surface there are people that do robotics that like to put them on moving surfaces so if I put it on me like a teeter totter foot then will it adapt to that changing all of these are valid I don't have a a one that I like best but we are definitely trying out these various things you want to differentiate agility from speed so you don't want to just know how fast are they but you want to know if they're going to walk at a given speed and they deal with these types of obstacles that's why I like the terrain it's a really good measure of how quickly they can adapt to different types of folds.

[00:46:07]
Contest work if you want to do it in a certain time so if you look at the path of moving around a bunch of obstacles but say here is the speed we need you want to complete this course 10 seconds and then look at the pathway do they have the path of least distance from beginning to end in that time or do they take really wide turns things like turning circumference is a really never good measure of.

[00:46:32]
Agility young he Chang who's one of your faculty in Applied biological sciences now has some really cool stuff on turns and the turning radius that's actually really good measure of agility a great example is go online and look at. Cheetahs versus because owls Ok Cheetahs have gree top speed gazelles are more agile and so when you see cheetahs go after gazelles they want to go up as stealthily as possible to get as close as possible at a high speed because they know if they give rise to start changing direction they can't keep up so what is good.

[00:47:11]
You know if you're going to succeed in evolved for many many years being able to evade and have high Gelati is really good for prey and I think what we sort of want to do is take that mindset is we want to have a high level of agility being able to change direction being able to have low returning radio.

[00:47:40]
So we've tried e.g. we're really we figured out. And again I can probably say that I'm the best of the world at Mobile u.g. just because I started with the only one doing it again and eventually people will catch up and I'll go do something else but. Mobile e.g. we can get signals out of your brain while you're walking and running and jumping over obstacles the problem is that there is a lot of steps between the computation of what goes on in your brain and the activation in the muscles and the motion so the spinal cord is not just transmitting signals it's not just cabling it does a lot of control it does a lot of calculations.

[00:48:22]
What you see it this the brain doesn't necessarily reflect the kinematics and kinetics of what's going on at the lower limbs and in fact when we try to control it our biggest problem is the variability if I measure the variability from step to step I can have 10 to 20 percent variability in my motion kinematics and in that emoji but I'll have 120 percent variability in the Igi So the brain does so many other things that are not just that motion and there's so much processing between the brain and there it's not really good at the 2nd to 2nd understanding of your brain your emotion dynamics it's much better at a high level higher order control so like when I'm walking along and I see an obstacle Yeah I can get the signal out of the brain great I can use that for the exoskeleton I'm not going to tell you exactly when to start the motion and how high to go so it's better at higher order control because of the variability in the way it works.

[00:49:47]
That. We're starting to figure that out and I'll give you 2 really good examples so this original work with Stephanie Wang where we gave them the pneumatic prosthesis and they had to walk. We're like here is their torque pattern of their ankle and it's of their passive prosthesis it would very low torque and the negative work in the positive work were very similar because all it was was a spring and then you look at their active leg and there was a very high peak of torque high power output we gave them their prosthesis and asked them to walk guess what they've made their prosthesis their new processes the pneumatic one walk just like their old one.

[00:50:39]
They were so intense this is the way I walk this is my motor pattern they've been doing it for years and years that's the way so they actually didn't get to being symmetric between their able bodied leg and their prosthetic leg until we pushed them into that space we actually had to give bio feedback and say Here is what I want you to do produce this torque this power profile they saw it they experienced they're like wow this is great take the Wii feedback away and they kept it up they knew once you pushed them outside their area of.

[00:51:15]
Preferred walking that they could learn it Max Donnelly and Simon Fraser has done some really cool stuff with a knee brace that has active damping he can basically increase or decrease the amount of resistance in the knee brace and he found the same thing he could have it coupled to like their stride frequency or their stride length and they don't get outside their normal domain all of the subjects they stick to what they know even though there's a better solution.

[00:51:49]
Energetically unless you push them there and so he's got a series of papers showing that there's a lot of people you have to push into that other space before they learn it so I would say the take home message from all those studies is you can't just throw an exoskeleton on and expect they're going to learn it I actually think the training is a really important parameter that not many people have.

[00:52:11]
Fully appreciated and there's a lot of research to go in that I mean is there any way to incorporate. So I'm going to try to interpret your question tell me if I got this right so is there a way to incorporate some of the approaches from era names and just the grizzle in terms of the way they do control robots with how we work with exoskeletons and.

[00:52:53]
My funny some of my funniest experiences being a young faculty member was sitting at a table with quo and Jesse Grizzle who are both controls experts but from very different backgrounds Ok Jessie is an electrical engineer give me any hardware and I'll build a controller that works and I can prove that it's stable it works are says let me design the hardware and you won't need a controller so building a passive dynamic robot that can walk without hardly any control those are very different approaches but funny enough the solution that they both come up with looks very similar in terms of bipedal locomotion in terms of the kinematics in the motion patterns and you know I think that.

[00:53:36]
With that in mind there are some approaches where you can do any type of control and that's how the exoskeleton started they're like don't worry about it if we can build the hardware fancy thing I'll just put a really good controller on and I'll move with the person and history of exoskeleton after exoskeleton showed that it failed is that the human doesn't move smoothly smoothly and fluidly It's like walking in the last us it's always trying to catch up with you and so it doesn't work and so I think you need to start from the concept of what is the human want to do how do they want to move and then take that hardware and build it to move as fluidly and smoothly.

[00:54:16]
I don't think the approach of Jesse and Aaron will work as well for exoskeletons I think it's great for. Pure robots where you have control of everything but it's less helpful when you're in or facing with a human and the problem really is I can't write the equations of motion of the human and know that they're accurate and they're not going to change because any equations that I can model of a human change the minute I throw them into a different situation they have so much variability that they can control that I can't document that you can apply the approach of air names or Jesse Grizzle where you know your system you can write the equations and you can calculate an ideal controller thanks for coming.