That was great because I feel like Seth gave you the abstract of this talk very, very efficiently. So you could just eat your lunch and leave and you get the key idea. Please don't read that 51 page paper. There's a six page version or a single slide version that I'll present today. But if you really, really like different, not worried about publication count. And you don't want to produce three conference papers. You could just write 150 page. It was actually 60 pages when we were first submitted it and we cut it down. This is a terrible advice. Yeah. Alright. Exactly. Okay, so before I start, you know, who all is in the robotics program here. Awesome. Okay, Cool. Usually I give this talk to a mix of robotics and non robotics people. And this is like at the interface of robotics, control engineering, mechanical engineering, but it's a very broad sense. But this is my target audience. I'm very excited. Alright, so this is a question you may have asked yourself. You know, where's my self-driving car? It's a thing that we've wanted. We keeping promised them. These are four years of articles of people promising them. Volvo has recently even backed off from this, is an asterisk, two asterisks in this one. But if you look on the website today, they have a vision in the next few years optimistically for maybe getting a self-driving car on the road. And there's a lot of challenges to this. I can't obviously solve all of the challenges, but I like to think about them, to think about the broader context of what do we really proposing to do as roboticists. We are usually thought of as the engineers, but there's also the society that these things interact with. And then there's actually how we legislate them or how we think about when something goes wrong. So to look at this holistically is I think, very, very informative because it can make better engineering decisions if you're able to really think about everything. This is, this is something I'm excited to touch on, especially in future research, but I think shades how I've done my past work. I think there's also really interesting when we think about who we're engineering for. Like do people from all different communities really want robots everywhere, especially anything about surveillance, it can get very tricky. So this is something I've also started working on more recently. And if you're curious about talking, especially about the interface of DEI engineering and societal trust. That's like a thing that I have learned a lot about that is not in this talk, but I'm very excited to talk about it. In this talk. I really, I'm going to sell this idea, right? How should we contribute to an overall society that may or may not benefit from widespread automation? We can formalize and implement safety. And what safety is, I'll define shortly. But first, what does it look like? What are some implementations? So this is in collaboration with the University of Sydney in Australia. And this is the algorithm, one of the algorithms I'll be talking about today called RTD running. This is at Forex because it was a full hundred and ten meter stretch of very heavy concrete planters, that would be very, very bad if we hit them with our brand new shiny autonomous golf cart. And the cool thing is like, this was not the first tried. On the first try we breaks us we breaked to a stop safely. But on the second try we got all the way down, dodged everything. The car didn't crash and we had a formal guarantee like on the whiteboard that it wouldn't crash, that moved all the way onto the hardware. The fun thing is we can start to exploit that safety guarantee. And so I'll talk about this Bernstein algorithm or BA, to increase robot performance. So we usually think about safety at odds with performance or we have to make a trade-off. In fact, that was the big gap of the state of the art that a lot of my research has addressed so far. It's not always a trade-off if you're very careful about it. And so in this case, the robot is able to handle all the different scenarios that it's never seen before. And I'm just honestly, I'm like really moving fast to make sure it doesn't catch up to me. Alright, so the methods I'll talk about today, and not just for autonomous driving. This is because for any kind of robot, and this is a little bit admittedly a bit of a contrived scenario, but this is from an out-of-the-box solver for manipulator trajectory optimization. And it just seems like a pretty bad trajectory. It was suffering from singularities and the two don't then the start and goal position and it just went through the obstacle it converge to an infeasible solution. We were able to solve that previous scenario just fine, but we're also able to solve dynamic scenarios with the safety guarantees. So here we're asking manipulator to basically break to a stop safely and then go around the obstacle. And it's planning, It's maneuvers autonomously in real time. In this case, someone bravely donated a glass vase that we kept as our obstacle. So the idea is that across a wide variety of robot morphologies, we can do better than just safety versus performance. And it's not always necessary, necessarily about saving the day. So this is on game day at Michigan, and this is our Cassie maze and step into crack and fell over basically immediately in front of millions of people on national television. So sometimes when you run these out-of-the-box methods there, It's not, it's not like you're saving the world, but sometimes it's just embarrassing. You want your robot to behave how you expected, right? Don't worry, the Cassie here has never done this probably. But in the past year or so. But one of the students I worked with a Michigan has figured out how to take a lot of the algorithms I'll present today for autonomous driving and extend them to, in this case, digit, where digit is walking around obstacles in a crowded room with the safety guarantee of not hitting stuff and not falling over, which is pretty exciting. So admittedly, this digit is not yet sensing tiny, tiny cracks in the ground. But that's one of the next things that a lot of folks in Michigan are working on. The big idea again, is to formalize ideas of safety to make sure that robots do exactly what we want within the specification is the hard constraints that we care about. And that I think can enable widespread deployment that can let us test whether or not we actually want robots everywhere. In this talk, I'll think about how to define safety, but more importantly, and this is really the crux of my research. How do we represent safety? Because you can write down whatever you want on the whiteboard, but the numerical representation is absolutely key to making anything work on hardware. My answers in this talk, we'll use tools from reachability analysis and like eighth grade geometry, it turns out that, that 50 page paper is actually just a bunch of geometry proofs one after another. Alright, so how do we define safety? Let's think about things in terms of the autonomy stock. And I'll reuse this autonomous car example because it's two-dimensional and easy to plot. So imagine we have our autonomous car. It's trying to get down the road to a goal. We have some obstacles. One way to think about safety is due to sensory limitations. We have to know what's around us. So we have to correctly detect and predict the world around us. This is something I've worked on more recently but will not show up in this talk. This is a key future direction that I'm working on. And if you're excited about provably correct sensing and prediction methods and dynamic environments. Come talk to me afterwards. The focus of this talk we'll really talk about if you have, let's say an estimate of the world around you. How do you plan and control your robot? To do this correctly? One way to think about this is you can create intermediate goals. This is a common planning strategy. Use a simplified, what we'll call a planning model, denoted with a little f. And don't worry, the notation will be, you'll see it a lot of times you'll get used to it. Then we know that the models that we can use that are fast enough to plan with are typically not fast enough numerically to generate really high-quality representations of our motion. So we have this issue where if we were to describe the robot really accurately, we couldn't necessarily use that to plan arbitrary motion in real time. I'll call the other model that's more accurate, a high-fidelity model. If we can compensate for the error between these two models. From planning and control, we can make a strong safety guarantee in this particular case for collision avoidance. So for this talk, let's say safety means collision avoidance. I'll present one case where it's slightly different or slightly extended beyond that. But you should really think of this as a particular beginning choice of hard constraints that I really, really have to enforce on my real system at runtime. Safety with collision avoidance is interesting because it's non-convex. The programs they have to solve a nonlinear. So it's a really hard one, even though it feels kinda easy to define. So in this talk, I will discuss safe, real-time, optimal planning and control. This is the crux of my work that's planning and control space with a focus on getting things working on real hardware. But like I said earlier, I'm moving more into the sensing world. And really the long-term vision is to look at this whole autonomy stack. And I'll talk at the end of the, at the end a little bit about this future vision. Since I can't cover everything, I'll focus on these three big ideas. In particular, reachability based trajectory design. What is this paradigm, this way of thinking about enforcing hard constraints at runtime? And then how do you exploit the safety guarantee that you get in RTD to improve performance in this case with a provable optimality guarantee. Then finally, what happens if you give up and decide to deep learn everything? So in this case, we'll bring learning and safety together in a framework that is called RTS or reachability based trajectory safeguard. Getting Started. Rtd. Also point out, I really like to work with people like none of this is I didn't just go off to do this than just disappear for years. All of this is collaborative research. It takes a lot of people to get a real robot working. And I think that's something that's really cool about Georgia Tech. This is my excitement to be here. This is a very collaborative space and you should absolutely take advantage of those opportunities. In this case, they flew us to Sydney. Let's go back to this safe planning problem, the left half of the autonomy stack. One way to think about it is I want to minimize some trajectory x and the control signal you subject to an arbitrary cost. My dynamic model has to be enforced, have to respect the physics of the robot. And I want to be outside of obstacles for all time. Each of these components is quite confused, are quite challenging to deal with. Especially the fact that you have to enforce these constraints at all points in time. You really want to consider continuous time to respect the reality of physics evolving for your robot. Often we can break this down into a hierarchy to handle these challenges. So one way to think about it is at the highest level, we generate waypoints or cost functions for our robot to optimize to do It's real-time planning. These waypoints are handed to a trajectory planner or trajectory optimizer to find a specific small time horizon trajectory that the robot will try to execute by tracking it with a tracking controller. And you can choose to enforce safety at any point in this autonomy stack. Since you're all robotics people, you may be familiar with RRT or a star as high-level planners, These typically if you want them to run really fast, you just ignore physics and you, you just generated a bunch of configurations that can run really, really quickly, but you can't necessarily avoid obstacles and collisions if you're also trying to get a real-time guarantee. A particularly interesting high-level planner in the vehicle driving case is called a state lattice. And I think this is like widely deployed now in all of the autonomous driving stacks. They made me don't call it this exactly, but the idea is you can parameterize the space k on the left, we'll call them parameters. And the state space on the right is big X. If I pick a point. So you can think of these as like a cloth void curves are basically polynomial coefficients on the left parameterize motions of a vehicle in the space on the right. At runtime, I can intersect my forward moving trajectories with obstacles and then back out, which are the parameters that I sampled are unsafe to eliminate them from my optimization program. In this case, we're simplifying the dynamics of the robot to enable this to run really, really fast. So we have to potentially dilate obstacles to compensate for the fact that we're not perfectly modeling the physics or the fact that our lower-level parts of the autonomy stack may not perfectly execute these types of motions. A different way to think about enforcing safety is in the tracking controller. And many of you here at Georgia Tech of value are familiar with CDFs and malpractice control. So what can a CDF look like in a safety context? Imagine that we have a nominal control input on the left and our corresponding nominal trajectory on the right. And we want to stay outside of an unsafe set. If we can compute a CBF that will always keep us in a particular choice of a safe set. We can basically create a bunch of polygon shaped or polytropic constraints in our control and the left. And instantaneously correct at the speed of how fast we can solve a quadratic program, our control input to always stay in the safe set. The trick is we have to assume often that all the higher-level parts of the autonomy stock are, in the worst-case, actually actively planning against us. Or a different way to think about it is if we want more flexibility in our choice of control, we often actually suffer a smaller, limited set. So if I want to be able to move in more places, I want to guarantee that they're all safe. I have fewer choices when I have to make it safe, which is basically trying to compensate for all possible future disturbances. A different way to think about this and where RTD starts is in the model predictive control or MPC framework. So here we have an initial guess. We solve this optimization program I pointed out earlier where I just replaced the dynamics with f. Hi, this high-fidelity are complicated model of the vehicle will get an output if this converges some control trajectory and some corresponding state trajectory. Often we can't directly solve this quickly if we implement our non-convex safety constraints. So one way to think about this as generating a convex tube of constraints to convexified the program. Often this becomes a mixed integer formulation. Which tube are you in at which time? And then we have to discretize things, especially time to enable representing this numerically. In most cases. Both of these introduced challenges. One is like how do you compute these tubes and how do you discretize time correctly to enable safety? This is exactly where RTD comes in. We tell you exactly how to discretize time. We tell you exactly how to represent obstacles around you to enable this non-convex nonlinear program at runtime. And it can run fast enough to get robust to do real things. The challenges in particular or that model mismatch I pointed out earlier. And sharing things are working in continuous time and space. And then making sure that we represent, remember the key idea is always representation. We represent things to enable really fast trajectory planning contributions. Yeah. Alright. So how does RTD work? Were actually more recently we've started moving away from an offline competition. We're doing this IFRS or for unreachable sets online, but I'll talk about the original implementation that method first. The idea is that before you run your robot, you would pick a space of trajectory parameters such that points in this space correspond to short trajectories for your, in this case, autonomous car. If you were to sweep out the volume of all of those trajectories, you'd have this, in this case, a bell-shape or basically a swept volume, which we'll call the forward reachable set, or IFRS. If we pre-compute this object. And make sure that it includes the tracking error that we care about. Then at runtime we can intersect it with obstacles and map that intersection into our parameter space. And then given a cost function from a higher level planners such as reaching a waypoint, the black star up there. That cost function can also be mapped to our parameter space and a pretty straightforward way. And then we can just solve an optimization program in this lower dimensional parameter space. So instead of solving over our entire infinite number of time points and our trajectory and the resulting reachable tube, the subset of the IFRS corresponding to that particular trajectory, as long as it's feasible for the constraints, is guaranteed to be outside of collision when you implement it on the actual uncertain system. The idea is then to rinse and repeat this procedure. So identify constraints, create a cost function, perform trajectory optimization, and going to result in reachable tube to stay inside. This talk, we'll do this every T seconds. In all the research papers, tea plant is usually 0.25 or 0.5 s. So planning at two to 4 hz, engineers at Ford in the past year have gotten it up to like 50 hz. So it's no longer just a research idea. The idea is to really, really quickly generate safe trajectories for our real, in their case, autonomous car. By rinsing and repeating this procedure, we can eventually reach a goal position. What does this parameterization look like? What are these magical k numbers are really useful parameterization in the autonomous driving case is actually just a speed and your rate profile where up to time t plan, I'm re-planning every T seconds, I'm just picking my speed and you, alright? And then, because I have to solve a non-convex nonlinear program over and over and over, It's very much a probability that I'll not be able to solve it all the time, I may converge and feasible solutions which are often unsafe. So as part of the parameterization, we also include a braking maneuver here, the speed and you alright, break down to zero. The idea being that if in my current planning iteration, recall this is in a receding horizon way we're planning over and over as the robot moves. My current iteration, if I can't find a solution, my previous iteration was guaranteed safe. So I just continue executing my previous trajectory and continued replanning from wherever I end up. With this idea of a parameterization in mind, now, we can start to think about what the reachable sets should look like. So here suppose I have a speed on ERA profile parameterized by a single point in the space k. This is expressed as I mentioned earlier, by a low-dimensional planning model with this little f. And then I've replaced the control input with k. But since I can't always track these plans perfectly, I want to compute a reachable set, or some of you may call it a reachable tube of the error between my two models and the uncertainty in my system surrounding my plan trajectory. The reason being if I can keep this reachable set out of collision, I've enforced my safety guarantee. In this particular case, we'll introduce what we call a tracking error model function g times another function d, where d is just mapping from time or short-time horizon to basically minus one to one. So if you were to just integrate f plus g, the dynamics, you will have a single trajectory. But in this case, we want to sweep out the volume of all possible trajectories subject to tracking error, which we'll represent as GI. If the map doesn't do it for you. Here's a visual representation. So we know something about the dynamics of the robot. And we can prove that we can compute an over-approximation of this g function for certain physical systems by looking at the dynamics offline. In this case, the tracking error can drop down to zero because we have a lower level controller attempting to drive us towards our trajectory. But if we over bound the error between our two systems and incorporate that into our overall planning, then we have a strict safety guarantee. So now we'll just think about this model, f plus g times d. Again, this is planning plus tracking error. With this model in hand, we can compute the forward reachable set by leveraging, in this particular case, a sums of squares program to find a polynomial representation such that if I have a point that's in the IFRS, in the Cartesian product of these two spaces. So you're looking at two different projections. If a point in the joint space is within the IFRS than this polynomial evaluates to greater than or equal to one. And the key idea is, this is something that we would run offline. It takes hundreds of gigabytes of RAM. It takes 30 h. But at the end of the day it spits out a quite tidy polynomial that guarantees this property. Now, how do we use this property at runtime? In particular, how do we represent obstacles? And this is really the crux. What we want to do is intersect are reachable set with obstacles and map that intersection to our planning space k. So then we can pick a low-dimensional point and k. And we know that for our high dimensional model, we're safe. We're again solving this type of optimization program that's optimizing over an arbitrary costs subject to being safe. But this constraint has to be checked for every single point in the obstacle. So just like we had to think about continuous time in this case, we have to think about continuous space. How do you evaluate a polynomial on every single point within a polygon? It turns out you can do this with semidefinite programming, but it's very, very slow. Here. I propose to instead discretize very, very carefully. We can parameterize for this 2D case with a number r and that's a point spacing and a buffer distance b. That's how far away from the obstacle to move. The physical intuition being that if I can discretize and this way my vehicle can ever squeezed between any pair of points. Critically, these two values exist in the 2D case and they're bounded for, however, I represent my robot as long as it's a compact body. And I've recently started extending, this is now also working in 3D, so we're going to use it for manipulator arms and drones. But the whole idea is like this is a geometry puzzle that lets us suddenly infinite. Take an infinite number of points and break it down to a finite number that we can put on a computer and just evaluate a polynomial to check safety. In particular, if our polynomial value evaluated on all of those points is less than one, then we are not in the unsafe set. Of course, in practice you can, you can't evaluate like less than, you have to do, less than or equal to one minus epsilon. So there's some little technical details, but the idea is, I can cover my unsafe set by mapping all of my finite set of discrete points into my planning space on the left. And the resulting choice of plan will be out of collision. This is the summary of that 50 page paper on one slide. The nice thing is, we were able to extend this to a dynamic case. So if you're very careful about how you discretize in time and space. And you cover the whole prediction. Where you assume the prediction is given to you by a perception part of your stack, then you can guarantee a different type of safety. In this case, we took mobilize idea of not at fault safety and collisions. So in this case, if I come to a stop safely and you hit me, it's your fault. That is one way to think about it. I'm certainly going to be a bit more careful when incorporate interactions. Are you paying attention to me? And these are interesting avenues of future work. How do I compute these safe trajectories when I'm not certain whether or not you know that I am behind you or in front of you, e.g. and so does the starting point forcing hard constraints on these dynamics scenarios. But I think there's a lot of work to be done in what it means to be safe in an interactive scene. Alright, so that's the theory of this idea. And critically, it outperforms other methods that are trying to enforce safety as well. The funny thing was because we incorporated uncertainty into the very beginning of creating this method. To get it to work on hardware, we just had to dilate everything because our sensors were a little bit off sometimes. And that was it. Like this just worked on the hardware. So the sim to real gap was overcome by directly incorporating uncertainty in this case. Of course, you don't want to just directly put this on a car. So here for donated a Carson model, so it's a full hybrid power train simulation. And here we're, we're driving on this 55 km kilometer per hour speed limit like loop with random obstacles while planning and real-time. Every in this case, I think every 0.3 s or something like that. And the car is autonomously deciding, it's detecting these obstacles and an autonomously deciding to go around them to slow down for turns. And it's all by solving exactly that optimization program over polynomials you saw earlier. In terms of comparisons, we looked at implementing an RRT from one of the arteries from the darpa Urban Challenge. And then also a non-linear model predictive controller or giving it hi fidelity representation of the car. And this is sort of our correctness check. If we can do modal afraid of control with a really, really high fidelity model of the car and non-convex obstacles. Without a time limit, it works perfectly all the time, which makes sense like you expect it. If you don't cut off your solver, it will find a good solution because you're perfectly representing physics. And so that was the bar you wanted to meet. In this particular case, our solver, the model predictive controllers Solver, couldn't solve that problem at all if you enforce a real-time constraint, which is, I think a good way to think about why you would change how you're approaching trajectory optimization. With RT, this is me hand tuning a trade-off between safety and performance. I tend to err towards safety, but I still actually, my RRT caused one crash where the car tried breaking to a stop and just slightly bumped into an obstacle. But when especially when you enforce a time limit, it can be very challenging. Certainly, there is a better way to do this in ROT, but this was a good baseline to start out with and I'll talk about a better baseline next. This is the method then that you saw earlier, but now you have an idea of what's going on in the left side, you can see the forward reachable tube being recomputed over and over and over, and critically staying outside of the discretized obstacle points. So that's, that's exactly what the car is doing, is exactly the same algorithm. In random environments with predictions of the obstacles around us, we're able to avoid the full prediction. So here's the time-varying case. In this case I compare it against a state lattice approach, which is definitely better than RRT. You're using a much lower dimensional parametrization that is directly related to the physics of your vehicle. But you still have to tune, unless you're being very, very cautious, you can, you really have to tune for this safety versus performance tradeoff? Interestingly, in our case, enforcing a safety guarantee for this particular type of scenario. Causes to get to the goal faster. So there's this decoupling of the performance versus safety trade-off that we didn't anticipate. We really thought we would be much more cautious. But after, I think 1,000 simulations with random obstacles, this actually worked out quite a lot better. We also tried this unstructured scenes. So this is an unprotected left turn. And we validate the entire left turn maneuver before beginning it, and then begin our regular planning once we cross the intersection. In this case, we compare because we know the exact trajectory of priority. We just use a model predict controller and like a safety check, that easy safety check, it turns out it was very, very conservative. So if we backed off a little bit, we started to see crashes even though we're getting to the goal lot. But with the RTD approach of doing this, we basically were more patients. So here was our hypothesis that we'd be a little bit more conservative actually played out. We were just a little bit more patient and we were able to in all but 100, 100 trials, completely cross the intersection safely. That one trial and we just stopped because there was no safe way to cross the intersection. And these were like delight. You can see that these are deliberately aggressive, really, really challenging scenarios where none of the other cars, nobody each other, they're all just doing whatever they want. So it's a little bit less structured than real life driving. It's very, very aggressive to make a left turn in this type of setting. The fun thing is, this does work on a wide variety of wheel hardware. So here the rover is seeing these boxes for the first time and it's able to plan. And we can basically just rearranged box as much as we want. But the idea is across a wide variety of comparisons. So in this case on a Segway and a rover, the Ford Fusion, that electric vehicle, a turtle bot, the fetch manipulator. And the point is not for you to read the slide. The point is for you to be like, wow, we did a lot of comparisons. We really, really, really tried to make sure that this thing actually works. I don't have time today to talk about the fetch manipulator or the drilling, but I'll show you some videos later. The idea is not just that we're safe, but also even in comparison to other provably safe methods, like in this case, FastTrack. We were able to get to the goal more frequently. So we have in some sense decoupled the safety versus performance tradeoff. Not fully. You saw that there was conservativeness in for mutable sets. But we're getting, we're getting to fight that intuition that, oh, if I'm safer and much, much less likely to complete my tasks, which I think is really, really cool. Now, I mentioned societal trust is a thing I'm interested in earlier. So I think this is the, this is the litmus test required. We told the robot to come towards the camera and we turned away from the robot and we were the obstacles. This is, this is the first, the first try. This is very, very scary. Look at it. Let it go. Good robot. Yeah, that was a good time, right? Yeah. Alright, so we've got to put yourself you've got to put yourself out there and little, a little. Okay, So we solved a bunch of problems, right? We plant with respect to tracking error, we create a collision avoidance guarantees and continuous time and space. We represented things to enable this to solve in real time really fast on real robots, but there's never a perfect solution. There's always things that you can improve on. In this case, our reachability analysis only worked in very low dimensions are interaction model is fully open loop. We just receive a prediction and playing with respect to it. Actually, in the past year, I've worked a lot on trying to close this loop and it's very, very hard. Happy to talk about that later. But for now I'll talk about solving this non-convex nonlinear program at runtime. How do you do this with a guarantee of performance, guarantee that you will always solve it optimally and within a finite amount of time. It turns out it's, it's hard. But we created the parallel constraint Bernstein algorithm. It's funny because it's puncturing and Bell house. So as the PCB or puncturing Bohai algorithm, which apparently fit into, but I thought it was cute. These guys are really, really good to work with. And what we did is we took this convex nonlinear problem, mapping all these discrete finite points to our parameter space. And we notice that it's a very low dimensional problem. We can parameterize our trajectory is in the vehicle case in two-dimensions. But we have hundreds and hundreds of these polynomial constraints. Recall the W is a polynomial. Our cost is also a polynomial. We noticed we can leverage the structure of this polynomial optimization problem to basically guarantee our solvers performance. And to give you a context of what I mean, we'll consider a variety of other solvers in contrast to PC VGA, we care about it being fast, fast enough, or real-time operation. Optimal, meaning our robot is making the best decision at Cannes. And low memory because it has to fit on real hardware. In all of the trials you've seen so far, we use FMN con in MATLAB. So it's just a regular interior point nonlinear programming solver, which is super fast. But it doesn't have an optimality guarantee. And it definitely uses very, very little memory. Another way to think about this is by solving a bounded sums of squares program or a semi-definite representation that is guaranteed to find the optimal solution despite constraints on a polynomial program, but it's very slow and it uses a lot of memory. And finally, we also considered this dividing rectangles. So your standard branch and bound way of thinking in a low-dimensional space. We noticed that the implementation is available on line, basically required you to wait your constraints. They didn't do hard constraints, it was only soft constraints. And it wasn't super fast and the online implementation because it wasn't paralyzed. So we took advantage of these gaps in the literature and we parallelized it. And we made sure that it actually obeys constraints. So we leverage a particular polynomial format called the Bernstein polynomials. But the cool thing is we hit all three of these points and we see it working on a real robot. In particular, here's a picture directly from Wikipedia about the Bernstein polynomials. They had this particular form, but really, this is just to amuse you. The idea is, this is the critical property. If I represent my polynomial and the Bernstein basis instead of the usual monomial basis. The value of the polynomial on a finite compact interval is bounded by the coefficients in the branch to representation. In other words, we have this, this property is very, very tempting for optimization because we just get them in and the max value. This of course is not, we're not the first people to think of this. The Bernstein algorithm has been around for a little while. The idea is that we can compute these coefficients by performing a linear map. On the right, we can compute them as a linear map from the monomial basis. If we shrink the size of the space we're considering, we get tighter and tighter bounds. So visually in this one-dimensional unconstrained optimization problem, you can imagine that we split this interval in half and we get tighter balance in each half. We can take our, our lowest upper bound as our estimate of our optimal value and the center of our whichever interval contains that as our optimizer estimate. Branching and bounding exactly as you'd expect. You can start to discard parts of your search base colored in pink here. And then you're guaranteed to find an optimal solution to within numerical tolerances. So what we did is we parallelize this, we got this weekend or GPU in a constraint format. We prove the convergence rate, which was surprisingly hadn't been done yet. But most importantly, the most exciting thing is that it worked on a real robot. The robot didn't run me over. So why is this, why is this important from a convergence rate perspective? You can read the theorem, but really the takeaway is the robot can plan optimally in real time. When you're branching and bounding with these constraints in a compact space, you can upper bound how many iterations you need to solve to within certain tolerances. And that upper bound grows logarithm logarithmically. So, you know, before you run the robot, exactly how long the solver will take to solve a non-convex nonlinear program, which is super hard. We also compare this against the other three solvers I showed you. So we had a live variety of 1-4 dimensional benchmark problems that were taken from the optimization literature. And these are notoriously difficult problems with many local minima or extremely small feasible regions. What we were aiming for, again, it was like salt time similar to f min con. So in this particular problem, our average solid time was only six milliseconds slower than f min Khan. And recall, recall that F mankind is the standard we're trying to hit in terms of speed. The provably optimal algorithms took much longer and sometimes had willed enough worse optimality error because of how you set your optimization parameters. So we were able to outperform these other solvers on enough benchmark problems that we felt justified putting this on a real robot. If you want more, there's way more details in the paper. So now we can look at this video again, but a direct comparison against f min com. So here I basically played around with the random obstacles, a whole bunch. And I realized that I can actively construct a scenario where the robot has to emergency brake very often because the scenario is very non-convex and very tight to navigate through. Our metric for how bad is the robot doing is it's how long it takes and how many times it stops. I drove the rollback, which is actually very hard to do, and then counted the same duration and number of emergency stops. So we were able to get to the goal much more quickly because we know we have an optimality guarantee and with much fewer emergency stops. In this case, in this deliberately challenging scenario. I decided then to really, to ramp it up, make the robot navigate a very tight corridor. And the nonlinear programming solver f min can just immediately started making bad decisions and then got the robot stuck. So here the robot is wiggling back and forth because it's just trying to see if that program will solve anywhere. I take over and drive it a little bit to get unstuck and it gets itself stuck immediately again, when I say stuck, I mean that the non-linear programming solvers converging to infeasible solutions over and over and over. And so all the robot can do is rotate in place. This is the first take with the Bernstein algorithm on the exact same scenario. And it just worked. I was very confident in this, but I was surprised that I didn't have to pick up the controller at all. The cool thing is, when we're considering this optimality guarantee of dealing in these non-convex, non-linear situations, were able to do this safe, safely in real-time with an optimality guarantee. And in general, this paper provides a solver that maybe you have a similar problem that you can use it on. Again, didn't solve all the problems in the world. We're using a GPU, though we never use more than 11 mb of GPU memory, which is, which is good. And then recall that at the beginning of the talk I mentioned that we have a high level planner that's generating these intermediate waypoints for us to track. If that high-level plenary does a bad job, no matter how good our trajectory optimizer is, we're always going to make bad decisions. This is me like hand tuning, an RRT, your hand tuning and A-star algorithm for months to get it to help the robot gets to its global goal. This is exactly the right place I think, to bring learning into the fold. So the final technical contribution here, what we call reachability based trajectory is safe guard in collaboration with Simon and chow. The idea is to learn to leverage our safe representation. Here. Remember that we've mapped the intersection of our forward reachable set in state-space to the corresponding unsafe planning parameters. But then if I hand tune something, degenerate these waypoints between where I am and the goal. That hand tuning process is quite painful and it doesn't always produce a good result. When you have a bad cost function, your solver is very likely to converge to a bad solution. The nice thing is, we can adjust bad guesses with gradient descent. We can learn to make better initial guesses. So one way to think about this is I want to solve a non-convex nonlinear program. I have to make an initial guess. Usually, let me use RL to make that initial guess. So our reachability based trajectory safeguard replaces hand tuning with reinforcement learning. It basically penalizes the RL agent for how unsafe It's guess was and for how far the adjusted safe guess was from its initial guess. In particular, were able to directly create a feedback signal for learning based on our safety constraints. And we still have to tune in RL agents. So I think the common complaint about this work is, hey, you took a low-dimensional hand tuning thing like an a star algorithm entered into a very high-dimensional hint hitting thing. It turned out that the safety feedback signal, it worked pretty well and it made this hyperparameter search. I won't say easy but a little bit easier. So if you consider the baseline RL agent where it's, it's considering sort of a soft constraint or a cost or an objective penalty for not crushing, it has to crash to learn about why it shouldn't crash, right? So even in very simple scenarios that just, just like bumbles into obstacles and eventually learns to be safer. But what we really do is by imposing the safety layer on top of that, you can see that there's a difference between the yellow dashed line, which is the RL agents guests, and then the solid black line, which is the guests with us or the adjusted, the guaranteed safe trajectory. So the RL agent is still able to learn when it makes an unsafe choice. But the vehicle, the robot never has to execute the unsafe choice. And in terms of training, we noticed that we are able to accumulate better award, were able to reach more goals than in comparison to RTD. So in comparison to my actively, aggressively hand-tuned waypoint generator, which I thought was doing great. It was getting us to the 90% of our goals. And RL agent was like No, no, no, every single goal could be reached if you are just better at your job, so, well, it's the right place to bring RL in. The coolest thing is it's making a really good initial guess for an MPC solver basically. So it cuts are planning time down. We have to take much fewer iterations to get to a good guess. Because you have a safety guarantee. We know we're not going to crash upfront even though the baseline basically decided that sometimes it's worth it to crash. The reason that the baseline decided to crashes because we penalize it for waiting around too long, but we ended an episode when it crashed. It learned sometimes in some cases that if it crashes early it doesn't accumulate any negative reward. Bump, bump. So the, the other authors, Simon and child will, may disagree with slightly with the statement. But I thought this was kind of a cool takeaway, at least for the vehicle case. That tuning the RL agent was not as hard as we expected it to be because the safety guaranteed took care of that safety performance tradeoff. We try this out on a drone. Here I'll point out that the green trajectory you see is the final realized trajectory that was planned in real time. It is not given to the grown-up priority. It's just that our plotting tool decided to plot the whole thing. But the cool thing is that in this scenario, this is also a deliberately easier scenario for the purpose of visualization because it's quite hard to visualize super tight corridors with these drones. But the idea is that across a wide variety of scenarios, by enforcing the safety guarantee, the drone was able to fly down the full tunnel over and over and over safely. Whereas the RL agent on its own just learned to crash immediately and then the episode. Okay, So that's that's everything I had in terms of this story of decoupling safety and performance and thinking about how to do safe planning. But where do we go from here before I proceed, are there any questions? I realize I've just been talking at you a little bit. Cool. We'll have more time for questions later and shout them out if you feel the urge. But why am I here at Georgia Tech? What am I trying to do? What have I solved? I think you saw that, but what do I think is the right direction to go forward? We want to enable this widespread deployment by enabling safety throughout the autonomy stack, not just in planning and control. I think that there's still a lot of work to be done in terms of learning to plan safe motion. So I have started working on this in terms of the manipulator space and an interactive scenes. But we have to couple that with robust perception. What does it mean to make a perception guarantee? This is still a very hard question to answer. I think even creating the right plot problem statement and the perception world can be quite tricky. Because often it's tempting to just go for existing baselines. And those don't necessarily incorporate this idea of guaranteeing that you will see something given enough time. But the long-term vision, the reason to do all this, this is really to build coupled representations where you can have them Coltrane each other or co-design the entire autonomy stack, the whole robot to be safe all the time. And to give you a feel for what these might look like. So one of the challenges is really performing this reachability competitions for very high-dimensional system models. The idea is, in this case we had a little tiny drone and it's running RTD. I asked my labmates to very carefully walk deliberately in front of the drum. So the reason they're walking slowly is because I asked them to, not because the drone can't plan around them. We forced this interaction and the drone is still able to go safely. But then as soon as it reached the goal, the AC event was open above it and it blew it almost out of the reachable set where the onboard feedback controller wasn't fully able to compensate for this sudden very large disturbance. So how do we incorporate this type of unexpected disturbance without making the reachable set is so large that the robot can't plan. And how do we compute them for these really, really high dimensional systems. So like not just the drones dynamics, but also wind dynamics or ground effect or rotor wash off of obstacles that are nearby. These are super, super hard to model and usually we have to use a finite element solver. So there's a lot of room to be done in very high-dimensional reachability. There's a paper actually we just put on archive. This is like a year old first attempt, but the fetch robot in this case doesn't know how heavy the dumbbell is. It has a broad range of dumbbell masses. And it's trying to just maneuver around the obstacles without hitting them. So in this case, we have a safety guarantee of not hitting the obstacles on this fold dynamic manipulator using the recursive newton euler algorithm in a set-based way to guarantee safety for the uncertain dynamics of a manipulator. So this is a direct key future direction robustness to uncertainty, much more complex dynamics. The other thing I think is really interesting is how do we leverage these safety tools and planning and control to communicate safely to people where we have to solve problems where we wanna make contact. Like if I want to help somebody get out of bed, I have to make contact with them, but that contact has to be constrained and careful. And long-term. I really want to work on this adaptive intuitive idea, which are very dangerous words to use in the planning and control contexts. But that's a future vision. In terms of robust perception. Can we verify perception models? In one way I'm thinking about doing this. So we had a paper at ITS see about a smart sensor network that teaches itself to predict cars. Now we're making planning and control into this idea. So what autonomous cars are connected vehicles may miss in an urban area, we can protect, potentially provide the infrastructure. But the representation should really be the same safety ideas that we've been talking about from planning and control. How do you get that idea out of the perception stack? It's, it turns out there's a lot of work to be done. And we've also noticed that people have started representing in implicit scenes. So, so people have represented scenes themselves using neural networks. And usually you throw away any safety guarantees when you try to represent scene geometry in a learned way. But that learned way can actually be very, very useful for feedback control. It can be really useful for object detection. So there's a bunch of benefits you can perception if you do, if you choose to go down this particular representation at the expense of safety, I'm very curious to merge these two ideas together. This is the cute tagline. Does my robot know where it is and why? Can we think about things also semantically in the longer-term? The reason to do all of this is really two on real robots make some interesting guarantees. So you may have seen this video from a few years ago, This one hand solving the Rubik's cube and it does really well. Maybe, maybe they do better now, but it will also drop the cube a lot of the time that it wouldn't solve the Rubik's cube. So creating that contact model with a super high dimensional robot and planning to never drop the thing is something that we can do with our hands very, very easily by leveraging not just the hardware or wet where that were built out of, but also our planning and control algorithms very, very carefully as humans, how do we end our robot to that same ability? And maybe more exciting. We've started to see extreme vehicle dynamics. So here, this car is running an RTK GPS. It knows exactly where it isn't, knows exactly where all the obstacles are, the boundaries of the radar. And it's able to do this really, really extreme maneuvers. I'm very curious to then leverage this type of thinking to generate these type of maneuvers and arbitrary scenarios that we've never seen before. The idea being that our robot should be extremely, in a superhuman way capable. If we want to test the hypothesis that that's something that we actually want in the real-world. In terms of working towards this. I think coupling, perception and planning is the right future direction. And something I started working on, just barely touched on is verifiable neural network training. So the idea is, can I train my neural network with a hard constraint on its performance? The idea in the long term is that robots should be able to teach themselves how to operate safely in human environments. And it should feel intuitive and seamless, which is, I personally think very scary. But because it's scary, I think it's very important to work on it. So a question hopefully that you'll just be asking your autonomous car sometime is, are we there yet? Right now the answer is no. But if we formalized safety that can enable widespread deployment. And start to answer the really hard questions about putting robots in the real-world. That's everything I have for you today. I'm happy to take questions and I want to thank my various funding sources, but yeah, thank you so much for your attention and care. Sure. Okay. More of a different direction. So you've mentioned there's engineering department legal or illegal? I know a little bit about recent laws in Kansas. Works for the company. I'm wondering. That's a great question. So the question is, how do I how do I say safety, which has heretofore been ignored in existing legal frameworks, entering into the future law governing autonomous systems. One thing I noticed while making these robots work is that when it goes wrong, like sometimes I get the dynamics wrong or get a minus sign wrong, my reachable side is incorrect and my robot crashes. I've had that happen on the small robot and then we figured out our bugs before putting it on the full-sized car. But what we noticed is that when we have a safety guaranteed, we can basically backtrack through the steps of the algorithm and figure out exactly what went wrong very efficiently. And yes, we introduce some conservativeness that relies more cautious. But I think that, that is, is one way of thinking about it very logically or sometimes in the longer-term legally. Can I give the people who are potentially dealing with the consequences of these systems a much clearer path to understanding exactly what went wrong. If I especially have a learned model or something. So that's where I see the math or the algorithms carrying over into a legally is potentially, it's a logical prescription of what should have happened, and then something different happened. And I know basically exactly where, where to penalize. Now, the the way that the law will evolve. I think it's very, very hard to say actually, I went to a legal conference and every single speaker got up to the podium was like, I was like, Cool, Okay, That was very, it was very informative that the legalization around this is very tricky. But I think this is where the societal trust aspect comes in. And what we're seeing right now is that people give autonomous cars in San Francisco more space. People are very wary because you know that the car may like brake suddenly or you know, that the car will drive slower than the speed limit for apparently no reason. Or you may have a pile-up of cars and a random hill in San Francisco. So people are not trusting these systems fully. But the reason those cards are on the road is because people are getting in them and the companies are paying a lot of money to get them out there. So partially just through exposure, we will get used to the odd behaviors or the unexpected behaviors of these robotic systems. And then our legal code maybe a little bit more lax about what we what we require them to do because we expect them to perform poorly. I think a safety guaranteed can be really, really interesting because we can revise our expectations about what the car should be doing at all times. We can have a much more stringent expectation if we know that it's going to meet some hard constraints that we as a society identified. That get to your point. Yes. Yeah, that's a great question. So what are the advantages of this, let's say this particular reachability best method. So. The trajectory planning level versus a control barrier function. I don t think it's always very cut and dry because controlled barrier functions can provide very, very strong performance guarantees. But often you have to, just based on the problem setup, you have to consider any possible future disturbance of your system. So with a control barrier function, the typical setup is that I'm correcting my instantaneous control input at each point in time. Right? Now this is not always the case. People have used CBS as a constraint within an MPC framework. So you're looking at a longer-time horizon. But the CBF itself, when you're computing it, it's often just computed with that potentially very conservative way of thinking about future disturbance. I think that in this case, are reachable tubes are considering the full forward trajectory of the robot and it can make it much, much less conservative, at least in terms of these mobile robotics applications. This is, this is what I've seen. I do think though, that there's different ways to bring the temporal correlation of uncertainty into the CBF. Or even like Hamilton Jacobean Bellman solver frameworks to share some of the same benefits. So like thinking about the evolution of uncertainty in your CBF formulation. So this is why I don't, I don't want to like hard, fast, say like, Oh, it's more conservative or less conservative. I think that there's in both cases, a lot of work to be done to bridge the gap between these two families of methods. So how easy would it be the case? So the question is, the millennia, make sure I understand that you're asking. I claim and this is exactly the claim I'm making, is that in certain scenarios we can decoupled performance from Safety, at least partially, meaning that we can increase performance without losing a safety guarantee, right. But how does that safety guarantee appear for different classes of performance? Like, what if I want a super, super high performing robot, like an acrobatic drawn, what is the safety guarantee look like versus perhaps lower performance, like milder dynamic case. I actually, my take is if you, if you can write down, imagine that you had your dream MPC solver and you can put in any constraints you wanted and you could put an infinite number of decision variables. What program, what MPC problem would you write down for your particular use case? My goal is to make sure that you satisfy those hard constraints. The idea is like the safety guarantees should get you to achieve the performance you want and potentially exceed it. But that means carefully crafting the theoretical and numerical representation of that safety guarantee regardless of what your desired performance is. I think a really, really challenging future example is like how do I guarantee that my robot makes contact? Like I want to guarantee that I gripped something correctly and that I manipulate it like like I pick up my cup and keep it a certain orientation and hand it to a human and make contact with that person. That's really hard. Those hard constraints are really hard to write down. Once you've written them down often you can't even solve the resulting program. So then there's an additional step. How do we represent those hard constraints correctly to enable that solution? Does that answer? The answer is a slightly different question for me were asking, but I hope it gives you my perspective on the framework. Overlap of the screen. Is that process. The idea that well known volume might currently overlap with that, since they're both moving in that trajectory might reject intersection. So let me try and rephrase that just to make sure I understand. The question is, is there a time varying hard constraint? So that intersection? Yes. So I think I'm going to enter the wrong question, but I want to answer it as a setup for the more interesting question. So like suppose I just took the entire future prediction of some other vehicle and ignored time and just treated it as a static obstacle. We did compare against that and that works very poorly. It really impacts your porphyrin, your performance that you never hit anything, which is great. But we discretize in time to enable considering a time-varying safety, just like a little safety bubble around each obstacle. It turns out that the discretization in time is also very conservative because it introduces like there's an exponential growth in the number of constraints and it's very, very hard to solve the optimization problem at runtime. So we decided a pre-specified partition of time is actually much, much more effective, where we partition the time into like four or five intervals. In this case, if you're considering a five to five to 7 s long trajectory for your autonomous car. And you split that up into little 1 s chunks and then treat, treat each 1 s chunk as a static obstacle that's not, not so conservative but solves much, much, much faster. So we are incorporating this idea that like as you move. And you can see this in not the demos, I hadn't this talk, but there's some videos online where you'll see our car very closely follow another car. But then we can always break to a stop safely. Or if the car is behind us, we would know not to brake to a stop until we've cleared it. What's really interesting though, and what hasn't, what we've been beating our heads on and been unable to solve so far is suppose that I choose by future trajectory. I am assuming everybody else keeps their future trajectory fixed for that whole duration. And that's I think it's an unreasonable assumption, though it turns out in autonomous driving, it's a pretty safe assumption because people tend to mostly just follow the rules. So autonomous driving is actually surprisingly, it's easy in the sense that people are very predictable, but it's very hard because when people are unpredictable, they're extremely unpredictable. They're very far out of distribution. But the cool thing is, I want to really think about how do you incorporate this interaction? Like how do you, how do you adjust not just your reachable set, but your estimate of other people's reachable sets together within a single optimization problem is probably the right way to think about it. That's very, very challenging ongoing work. So I think it's a key directions for future work that we're out of time. Yeah, we're basically extra excuse view of what to expect from Rogers will evolve. Mommy, listening. Thank you.