Join me in, welcoming. Exactly. Thanks a lot. Alright. Hey everybody. Thanks. Thanks again for the intro. Thanks so much for having me. It's, it's really great to be here and finally be interacting with humans and verse. And again, that's said, Yeah, On that topic of human interaction, please, please interrupt me, speak up, raise your hands, whatever you wanna do. That's what makes this fun. And you know, better than me talking at you earn on a screen. So I really welcome the audience. Participation. Cool. So I'm going to talk about work from my lab over the last couple of years. It's become an obsession of my philosophy here is trying to, trying to figure out how to apply ideas that have worked really well in other areas, trajectory optimization model, predictive control, these kind of things to systems with discontinuous contact interactions. It's been a bit of an odyssey for me that started about five years ago when I was a postdoc. And that's what we're gonna talk about. So I'd like to kick this off as, as was alluded. I have an aerospace background and I'm a huge space nerd. And this is one of my favorite robots, Mars Curiosity. This has been on Mars for ten years. And this is super impressive. This thing has been operating in one of the harshest environments reliably for this amount of time. But I want to zoom in on another statistic here in ten years, this robot has covered 28 km. That sounds pretty impressive until you divide those two numbers and realize this works out to be less than 8 m per day of progress. So I want to ask the question, why, why is this robot only covered maybe the distance across this room in every day? And the answer is that this robot is really not very autonomous at all. In fact, almost nothing about its motion planning is autonomous. The way this actually gets done is perception data from the robot is sent back to Earth. A bunch of engineers at JPL recreate physically the environment around the Rover in a giant sandbox at JPL that they call the Mars Yard. And down to every rock, right? They like manicured the sand and place boulders and exactly the same places as they are on Mars around the robot. Then they meticulously planned and rehearse every move, move the robot makes in the Mars Yard. With people, they're sort of teasing out all these little details and all these little backup plans and everything that could possibly go wrong before they then beam these commands up to the robot and have the robot execute them in a very sort of canned manner. So very little autonomy, lots of humans in the loop. And I'd argue that at the end of the day, this, this lack of autonomous motion planning and control capabilities in these kind of complex environments is what's holding back this number. And at the end of the day, if we could do this, we'd be able to get a lot more science out of the missions like this and cover a lot more ground. And to that end, where I'd like to see these robots ago. Thinking about like the rough terrain on Mars and locomotion, I want are robust to be more like this. I love watching these videos of mountain goats. This is insane, right? Like I couldn't do this. These guys are able to traverse near vertical rock faces. Reason about complex, really complex contact interactions, insane, rough terrain right there, super agile and super robust. They're able to recover from foot slips. And this is, this is just dramatically beyond anything we can do with robots right now in terms of locomotion capabilities. So hugely inspiring. Goats are impressive. Okay, so enough, enough code videos. So part one of the Tuck, I'm now going to try to convince you that just about every aspect of this problem, of this kind of locomotion planning control problem is or can be posed as an optimization problem. So let's see if you believe me in the next segment. So the first one, I think most of you will probably agree with that. We can treat control as optimization. We do this all the time. This is very standard. Probably many of you who've taken an optimal control class, this one is not a tough sell, right? So this is an optimal control problem. This is sort of a classic trajectory optimization problem. Write a deterministic optimal control problem. We have a model of your system down here and a cost function that's saying what you want the robot to do, maybe minimize fuel, energy, time, get to some goal state and all this kind of stuff, right? So we can write this down and then sort of in a whole bunch of different ways that probably many of you are familiar with. We can discretize this and try to solve with a computer, you can do all kinds of things, right? Okay, so control is optimization. The next thing I'm going to try to convince you of is that we can also deal with physics as optimization. This may be familiar to you if you've taken some kind of advanced graduate level like mechanics or physics course. So this thing at the top here, this equation, this is called Hamilton's principle, or the least action principle. And it turns out that the sort of true trajectory that a system follows under standard physics is the one that minimizes this cost function. And if you stare at this for a little bit, This looks a whole lot like that trajectory optimization problem. Hint, hint just to kind of fill in some details. Q of t is the pose trajectory of the system. This L is the Lagrangian, the normal sort of thing for mechanics, kinetic minus potential energy, that kind of thing, right? And I'm sort of looking at this really. If I imagine that Q as like an x and that Q dot is like a U. This kind of fuzzy, you know, think about that in my imagination were, but this really, really looks like in a trajectory optimization problem. In fact, it is exactly the same structure. And unsurprisingly, we can actually take this and discretize it and turn it into, we can use basically the same kind of direct transcription, co-location, whatever ideas that we use to solve optimal control problems to do physics. You might ask, why would I ever do that when I have all these nice tools that can spit out ODEs. Well, this is where it starts to get interesting. So I can do this. I can solve this as an optimization problem. What that lets me do that I couldn't do before with my nice F equals MA, second-order ODEs, Euler-Lagrange equation. All that continuous death is. I can actually now, just like I could with trajectory optimization, I can put constraints on this problem. In particular, I can put inequality constraints on this problem. I can solve it just like I would any other inequality constrained trajectory optimization problem where I would have say, state constraints, right? Or control limit constraints. And what that lets me do is if I now put this constraint over here, Let's see laser pointer. This, if this FI guy over here is a signed distance function, so the distance between this block and the floor, and I put this inequality saying that this has to be greater than equal to zero, which says the blocks not allowed to go through the floor. If I just put this inequality constraints on this problem and use any standard numerical optimization technique to solve this. What magically falls out Impacts. And if you think about this a little bit, it's not too surprising. Like the constraint Jacobian here ends up being the normal vector. And the Lagrange multiplier associated with this constraint ends up being your impact force or impulse. That's kinda cool. I should say also, if you use optimization techniques to solve this, you get perfect. No interpenetration artifacts. You get hard impacts. And you don't get any issues with stiffness or ill conditioning. Because we know how to solve these problems with interior point methods, whatever you like, kinda standard optimization techniques. So it's got some really appealing properties compared to other methods that are maybe more common in robotics for handling discontinuous contact interactions. Tagging onto this, another aspect of the physics that it can also be posed as an optimization problem, friction, coulomb friction. So this is called the maximum dissipation principle. Probably a lot of you heard of this too. If you haven't, what this says is that so Coulomb friction, in particular stick slip interactions. So if I push, push, push, and then we break static friction and slide, that is also discontinuous, right? I have this stick slip transition, same thing if I'm sliding and then all of a sudden stick and stop. There is this discontinuous discontinuity in the velocity due to Coulomb friction. So it turns out to deal with that. A way of capturing that is to write down this optimization problem which says, so b here is the friction force in the tangential direction. This is called the friction cone, that says the friction force has to be less than or equal to some friction coefficient mu times the normal force. It's like high-school physics stuff, right? But this is how you generalize it to 3D. So it says that the true friction over here is the one that minimizes or sucks out the most kinetic energy from the system. So this is like instantaneous rate of decrease in kinetic energy. So we're trying to suck out the most kinetic energy at a particular instant in time, subject to lying within this friction cone. And so practically what happens here is I have sticking friction. That means I have enough friction to totally oppose the motion and keep, keep this, keep my kinetic energy is zero. That means I'm inside the cone. And when I slip is when I push harder and harder, harder, the friction has to increase when I hit the boundary of the cone and I can't have any more friction. That's when I start to slip, right? So this really cleanly in nicely captures this discontinuity associate with Coulomb friction, I just plug everything in here, solve this optimization problem and outcomes. I don't have to worry about any stiff ODEs, weird smoothing things are creep artifacts that show up in other ways of doing this. Okay, So that's kinda the, most of the physics we care about. The last one I want to mention is really recent work from. From my lab on collision detection. So if you've also worked in this area, you'll know that even just deciding when parts of your robot or touching things in the environment or other objects is a non-trivial problem. There's a lot of work on detecting collisions are intersections between, usually this is posed as between convex primitives, convex objects, and what you do typically as decompose your complicated robot into a whole bunch of convex chunks or primitives. So tons of work over decades in the computer graphics community on solving this problem. Obviously, this shows up in all kinds of video games and animation. And those techniques are generally used also allow the robotics simulators. The most famous one might be a, G j k, if you've heard of this or enhanced GJ K, most of these techniques like JK, they work by actually decomposing these convex objects. They deal with polytopes. So there's sort of a mesh, right? And they work by actually like sort of traversing mesh points and a very discrete fashion to try to figure out if there's a collision. And it turns out so they work, they can be fast, but they are not differentiable. What we've done recently is figured out how to pose this collision detection problem. So the red dots here are the closest points between the objects. We figured out how to pose this as a convex optimization problem. It's not too surprising that you can do this. You can imagine sort of, okay, I have a point that's gotta be in both convex sets. And I tried to find the distance. That's a convex objective. It turns out getting this to work though, is kinda subtle. We can go into the details later. If anyone is curious, come talk to me. But the upshot is we can pose this as a convex optimization problem and we can get really nice closest points, differentiated all, etc. etc. So hopefully this kind of sets the stage a bit and shows that most of the building blocks here of this kind of bigger contact rich motion planning control problem. A lot of the individual pieces can be well-posed as optimization problems from control to physics to really like low-level collision detection stuff under the hood. Okay, so the next part two is, if I can do all of these individual things by solving optimization problems, ultimately, what we want to do is this guy up here control, but all these other pieces are individual little optimization problems. So now the challenge is, how do I, if I ultimately want to solve this guy, I have to somehow take all these individual optimization problems that might have quite different solution techniques from interior point methods or whatever optimization solver techniques you would typically use to solve these. I somehow have to take these and squish them all together. And they all depend on each other in various ways. Input parameters from this problem might come from this problem. Solutions from this problem show up in this problem and vice versa. So somehow I have to like, figure out a way to not only solve each of these problems individually, but also how to glue their solutions together. Glue those solvers together at a deep level and makes sure that they can all converge simultaneously and then I can actually get good solutions. So that's, that's what we're going to hopefully go for next. I'm going to start off with actually showing again some motivation for why we do this and maybe some existing stuff that has some issues. So here's back to that optimal control problem that we would typically solve. Here's a little legged robot walking around. Why is this sort of contact rich thing hard? I'm sure many of you who've worked in this area can appreciate this, but basically it comes down to you when I tried to do optimization. I need gradients of everything. And at the end of the day, I need all these functions really, but this one's the one that's problematic. The dynamics here, we want these to be smooth and at least have continuous first derivatives so I can differentiate them and have gradients for my optimization solver. And ultimately in this context setting, impacts and Coulomb friction like we talked about or just non-smooth and don't have these derivatives. And as a result, if you naively try to solve something like this with these kind of contact dynamics, you run into all kinds of problems. How we do this now, the standard approach that's very widely used in the legged locomotion community is so-called hybrid approach, where you basically, if you can pre-specify when the feet of that robot will make and break contact at particular timesteps. If you pre-assign that, which comes down to making some assumption about a gate. If you can prescribe a gate and know when these foot touchdown and lift officer going to happen. Then what you basically do is you just kinda bake these into the problem and remove the discontinuities from the problem. And then really what you're doing is just optimizing over the smooth arcs between these impact events. Now you've kind of like essentially removed the discontinuities and gotten yourself back to a smooth optimization problem that we know how to solve. This has been super successful, like pretty much all the state-of-the-art legged robot results you see from Boston Dynamics to MIT Cheetah. These guys are all doing some version of this where they're assuming some gate and then like solving some kind of smooth optimization problem over the chunks between the impact, right? So can work really well. You can solve these things in real time if you do it right. The problem here is that it requires this pre specification of a gate or a contact mode sequence. And it starts to break down when you start to need to really reason about. In the legged context, foot placement on very rough terrain, say, or reacting to very large disturbances where you have to put your feet somewhere else and not be in that periodic gate all the time, right? Okay. So the alternative to this, which is the rabbit hole we're going to go down, is so-called contact implicit optimization. So has anyone here worked on this kind of stuff? I know, yeah, in particular has done some of this stuff. A couple of people, so not too many. So the idea here is what we ultimately really, really wanna do is I want to get rid of that assumption where we're pre-specifying the gate or the mode sequence, right? So that we can have the robot reason fully about its physics and the places in the environment that it could touch potentially. You can imagine, right, in rough terrain, this is very important also in manipulation. This is everything, right? We, we really can't do this mode, pretty specification stuff super well, and a lot of manipulation contexts. There's, there's people who've tried doing this for a long time. I've tried doing this for a long time. And the dream here is that this can just do everything. It can reason about all the physics, all the contact, all the geometry, and just give you the answer. The problem is, right now the state of the art here, there's this kind of There's various techniques for smoothing people have tried which ultimately hinder your accuracy and lead to a really big seemed to real gap where these things can maybe work in simulation with a soft contact model or something like this with a bunch of smoothing. But they don't transfer to hardware very well because of all the smoothing approximations and assumptions you're making. And then also the more realistic you try to make your model there. Again. So we might smooth the dynamics to avoid these discontinuities and get nice gradients so we can do optimization. But then at the end of the day, you're not getting an answer that respects the true physics, right? Then the closer you try to crank that smoothing down to get closer to the heart answer the worst or numeric get, you start to get stiff ODEs, numerical ill conditioning and your optimization problems, you start to not be able to solve the problems anymore robustly and efficiently on your computer as you crank harder and get closer to that hard contact cancer. Okay, so we're now going to try to I'm going to now try to show you how this like composable optimization viewpoint addresses some of these problems and tries to get us closer to the, oh, that's fun. Okay. I think we're back. Okay. So back to this picture. So we're now going to, I'm going to now try to show you some a big picture viewpoint here on how we make this diagram where the key idea is that rather than try to smooth the physics or any of that stuff, what we're going to try to do is use the real physics like I just showed you with these optimization formulations. And we're going to develop Solver tools that let us solve each one of these problems individually, really efficiently and really accurately. And then differentiate through the solutions to those individual problems in such a way that we can get gradients that are nice, numerically nice, and sort of glue all these things together. And at the end of the day ultimately converge towards the real true hard contact answer without any smoothing. Okay, so that's where I'm heading towards. So the key piece here is inside each one of these blocks, what we're gonna do is this. So we're gonna take each one of those little optimization problems. So our least action with impact problem or are Coulomb friction, maximum dissipation problem. All of these are inequality constrained nonlinear optimization problems at the end of the day. So I'm going to write them generically like this. So this is my objective function. So this might be the minimize kinetic energy thing. Say in the friction case, that's probably a good one too to go with. And then this would be in the friction case, say that friction cone constraint right here, what we're doing is the x is the variable we're minimizing over. So in the friction case, this would be that friction vector Theta here are all the other problem parameters that show up in that problem that are not being optimized directly, right? So when the friction case, this would be x is the friction vector. The Theta in that case would be then the other terms in that kinetic energy. So they'd be like the robot's current velocity and pose that kind of stuff, right? So all the other stuff that's not being optimized in this block, but that's coming from another block. So that current velocity and pose stuff that's coming from the other block that's doing the rest of the physics. And ultimately some of that. That problem is coming from the collision detection block, right? So they're all coupled. So the stuff that's coming from the other blocks is, is basically encapsulated in the theta. The stuff that's being optimized in this block is in the x. Okay, So what we do is we take this problem and this is a very classic, this is probably a lot of you know, this stuff, but like this is the baby version of how interior point methods work. We're going to take this inequality constraint and we're going to smoosh it into the objective function with this term here called the log barrier. So the cartoon version of this is here. So this is what that log term looks like. This is minus log of x. In this case, this log blows up as I get close to the constraint boundary. And so the idea is if I start in the interior over here, if I'm just minimizing this objective with this log barrier, the log, it's called the log barrier because it makes the cost blow up to infinity at the constraint boundary. So it keeps me inside the feasible set of the constraint always, right? That's the idea. Okay, So that's how that works. So I can transform the sort of constraint problem into an unconstrained problem with this log barrier. And ultimately what's going on here. I also have this row, this kind of tuning parameter. And what this really does is it just how steep the barrier is, right? So this is very classic stuff. This is how primal-dual interior point methods work roughly, the idea is I start with a relaxed row, so things are very smooth and soft. And that lets my gradient to be well conditioned and things are nice and I can solve the problem. I solve it there. And then once I get a solution, I crank down the barrier parameter and get closer to this hard corner answer. And I can then warm start and resolve a couple of times. Well, I crank this down. This seems like a heuristic, but I'm going to argue that this is a really theoretically well understood thing that has had decades of work from the optimization community. And there's lots of mathematical guarantees and lots of really rigorous ways of cranking the row and solving this such that we're able to solve these kind of problems extremely reliably and really well. This is basically how all of modern convex optimization works, right? So we have like decades of nice theory from the optimization community on how to do this. Okay? So the next step is when I do this, this thing is now totally smooth and continuous, at least on the interior here. And what happens is as I crank row all the way down, I gradually approach that hard answer, that hard discontinuous answered. This basically is the real trick. This lets us encode all of those discontinuous things with this kind of smoothing effect, with the smoothing and there were now differentiable. So I can now take this thing, I can differentiate the solutions. This is called the implicit function theorem. All this says is that smoothing stuff. I can take my solution to this optimization problem with that log barrier trick and get nice gradients out by just giving the solutions. At the end of the solve process. The upshot is now I can take this entire block that has this complicated optimization thing going on inside it. And I can treat it just like any other continuous function. I can treat this as a function like x of Theta with a nice continuous smooth first derivative. There's, there's lots of magic going on under the hood here. But what I'm trying to convince you is that we can basically use this as a mechanism for turning complicated, messy, discontinuous stuff into nice differentiable functions. So now each block there turns into a differentiable function. The magic is in how you crank the row such that you can make the whole thing converge. The, trust me, we can do this and there's like good theoretical backing for how you do that coming from the convex optimization community that we're, we're gonna kinda gloss over here. Okay? So that's how we do this at a very high level. And please yeah, come bug me. If you want to know more details on this. Then sort of coming at this from an application standpoint. Here's a couple of things we've done with these tools. The first one is we've done simulation. So we've built a simulator for contact rich robotics, stuff that we call Dojo. And this is kinda hooking together this chunk of those blocks. So we've got the, the sort of least action principle with impact problem, this coolant victim problem and this collision detection stuff. We glued all this together. And we've got this simulator. Sort of some of the, we, by the way, I'll mentioned we did all this. We started this project before mood gioco got acquired by God open source. So originally when we started this, there was maybe a much better incentive for having a nice open-source robotic simulator. And some of that incentive went away when that happened. But still i'd, I'd argue, at least in terms of like physics accuracy seem to real gap all that stuff. I think we're, we're still quite a bit ahead. So what we've just some of the properties we get as a result of formulating the problem this way. First is we get hard contacts, so we get no interpenetration or creep. Artifacts that it's real hard contact, real Coulomb friction, a lot less of this kind of weird symptom, real gaps stuff. We also, because it turns out the way we do that sort of discretized least action thing to simulate the physics. That stuff turns out to be unlike standard ways of writing out ODEs and using Runge-Kutta methods, etc. It turns out if you do it that way, you automatically get really, really nice conservation properties. You get like energy momentum in general, symmetry conservation properties just kinda fall out of doing it that way. I'm happy to talk more about that later. That comes from this line of work called discrete mechanics. If anyone's heard of that, just kinda cool. And lastly, we get the smooth gradients for optimization. So I'll mention also the way we're doing this. I mentioned things are smooth when we have the log barrier parameter kinda relaxed. And then as we crank it down, they get less and less smooth, right? And they sort of approach that hard answer which is ultimately will want. So what we actually do here, we have some tricks. So we, in the simulator we always return, like when we give you the next state, we always return the hard contact solution to the physics. But when we evaluate the gradients, we have, we let the user basically decide that value of rho, the gradients computation. So in practice, what we do is as we crank down on row in the solver, we cache previous results. So we can give you the heart answer with the gradient evaluated from maybe the previous or any of the previous row values. So we have this knob, you can turn and you can give. So this is just showing impacts and friction. Here's the heart answer with row all the way crank. And we get this sort of continuum of solutions with various levels of smoothing. So the idea here is we can give you smooth gradients while still giving you the hard contact physics answer. It turns out this is actually, this works great for lots of optimization things and it lets you converge on real answers to a lot of problems. So stuff we've done with this, It's just a smattering of examples. This is doing so. This one is actually MPC. This is just taking this simulator and plugging it into IL QR. If you're familiar with this very generic kind of simple trajectory optimization technique, nothing fancy. Just throw the dynamics and therein do LQR. This stuff is kind of policy gradient, open IgM stuff. Again, just throw it in there and let it go. You very standard things. And this one is kinda fun. This is going system identification. We take a bunch of examples of like throwing a block and we tried to regress the shape of the box and the friction coefficient, a bunch of the physics parameters. So you're seeing the actual a block and then the regressed model parameters from running kind of very standard least squares estimation on the observation data. From that, we can run like all the things you would like to do from control to system identification and estimation to RL on top of this and get nice results and get to take gradients. Okay, So moving a little bit more forward into fancier stuff, harder stuff. This is where we kind of tried to put all the pieces together and try to solve some hard stuff, particularly here on legged robots. So we can solve. The whole motivation for this was solving these kind of trajectory optimization problems. Again, the goal is we just write down all the physics for the robot. No contact mode, whatever, say go and we get stuff like this out. Climbing stairs, walking behaviors that just kind of emerge. I want to now kind of point out a couple of key things we can do here that are fun and kinda where we've taken this more recently. So I'm gonna show a very terrifying slide full of math. I regret doing this, but please don't be too scared. Here's, here's the, here's the only thing you need to know about this. This is like the full set of messy stuff that isn't all those blocks. Here's what I want to tell you. This is the whole mass. I want to ask the question, what can we linearize in here? If we wanted to control, we love to linearize things. So here's the robot dynamics, including all the contact and friction stuff. We can linearize all the robot dynamics about some nominal trajectory. That's great. We can also scale. You can also take all the contact stuff, i e, the terrain, stairs, whatever you want. I can locally linearize that contact manifold like this, basically the signed distance function. I can take that and also linearize that about some reference trajectory. The most important thing though is I'm actually keeping all of the stuff that encodes stick, slip and discontinuous impact interactions. So I'm linearizing a bunch of things locally around some reference, but I'm holding onto all of the key, like contact physics stuff, right? So I can approximate a bunch of things while still keeping the flavor of the ability to reason about making a breaking contact and stick slip interactions. So sorry for that. That's terrible. I apologize. Okay, so the next little tidbit here, so we do this, we linearize locally about some reference or some set point for computational reasons. The next thing we do, This was a lot of under the hood work. Essentially, this whole thing has to get solved ultimately with some kind of Newton's method. And it turns out there's lots of gains to be had there. If you're clever about matrix math. What we did here is take advantage of some of the structure in that big Jacobian and develop a custom linear system solver for those who are into this sort of thing, we do some really fancy like QR decomposition stuff. I like learning about this that I'll, I'll tell you some of the tricks. Basically, it turns out if we're doing this where we've linearized about some reference trajectory, a lot of the information in this matrix is known ahead of time and is actually fixed online, like we know it when it's basically all evaluated about the reference trajectory. If we're doing some kind of tracking thing. The interesting bit is, if you know anything about how these like in particular here we use QR. If you know anything about how those factorization methods work. They basically walk left to right along the columns of the matrix when they do the decomposition. So here what we did is take all the stuff that we know ahead of time and can pre-compute and permute the matrix. So all that stuff's on the left. And then we pre-compute the QR factorization left to right on all the stuff we knew ahead of time offline online. All we have to do is plug-in the stuff that we only get to evaluate online and finish the factorisation over the last few columns. So this ends up being a gigantic when we get to speed up in solving these under the hood. Just the linear system solves in the optimizer by doing this, by linearizing offline pre computing some stuff and doing this partial factorization ahead of time. So all we have to do is finish off the last columns of the matrix. That's cool. Okay, So putting all this together, we've gotten a bunch of this contact implicit stuff, this reason bottle of the contexts have to work at real-time rates. So this one is showing you that we call this the push bot. It's an inverted pendulum with a little arm push her thing at the top. This is all linearized about the upright equilibrium. And we're just randomly hitting it. It has no information at all about the disturbances ahead of time and has no information about contact modes or anything. So it's entirely reasoning about these contacts online. So it just decides to push against 1 mol of the other, etc. Again, we can run this it like hundreds of hertz using all the tricks I just showed you cartoon example that's showing you that we retain the ability to reason about these contact modes in real time without having to pre-specify modes or anything like that. Some more fun stuff. So this is another example. This is tracking a walking gait on a 2D quadrupeds. This is the Monte Carlo run when we just drop it from a bunch of initial conditions and show that it converges to the nominal walking it. You're trying to track with all kinds of different contexts that happening. You can tolerate all kinds of model mismatch. This is showing you like put a big weight on the robot, it's fine. This was originally it's just tracking a gate for a flat ground and it works with whatever disturbances. This is maybe the most fun offline example. This is showing us just like giving the robot a giant kick. And it's trying to track this nominal trotting gait. And as you can see, while it's flying through the air, it's completely around. It's able to, again, this is linearized about this trotting in place at that spot. And all we do is like fling it, run this MVC controller. It's reasoning about all this complicated contact stuff, all this push recovery stuff, where to put its feet, when to touchdown with its feet, et cetera, et cetera, with no pre specification of anything. So it's pretty robust. I think this is maybe my favorite demonstration of the robustness of this thing and how well it's able to reason about these contacts. Okay, So this is not going to show you some very fresh results, so don't be too harsh. This is like us putting this stuff on the hardware. This stuff is not, not yet published, so it's it's not, it's also, I'd say we're not at the level of state of the art kind of classic hybrid MPC yet this showing just that we can do trotting, whatever and kick the robot, the usual, usual stuff. And it does kind of okay, Again, this is not at the level yet of what the state of the art MPC stuff on like cheetah and all those guys are able to do yet, but we're still still working on it. We're gearing up to do some where we're going next with this is we're trying to, again, take advantage of this ability to reason about arbitrary contacts with complex geometry. So that's where we're headed next, is kinda starting to get into that. We're trying to do things like reason about contacts with other parts of the robot's body, exploit contacts with other things in the environment. So not just flat ground. Maybe getting into a little bit of manipulation stuff. That's where we're headed. We're just getting this stuff on the hardware, so stay tuned. Hopefully in the next couple of months, you'll see some more fleshed out demos of this. And the last thing I want to just plug, I really focused on this kind of legged locomotion stuff. But I wanted to zoom out really quick at the end and just give a little tiny teaser of some other things where we're using this composable optimization idea and other places it shows up in robotics. I'm one that we've worked on a bunch is game theoretic, planning and control. So it's like autonomous driving where you're reasoning about what other drivers are gonna do. This is actually solving for Nash equilibria. If you know what that is online real-time rates. And again, it's an instance of this composable optimization problem where we're taking the optimization problems for a bunch of other drivers and coupling them together. And then the other one is, we've done some work on Robust control for a bunch of aerospace applications where it's kind of this idea as well, where you're reasoning about a lot of the classic kinda minimax optimization. Things that show up in robust control are also instances of this or can be posed in this kind of composable optimization way. Teaser stuff. Happy to talk about this stuff offline. I wanted to give a brief plug to my amazing students who've really driven this work. Taylor and Simon have done all the work on Dojo or stimulator, and also a lot of work on the contact implicit and PC stuff. Brian Jackson who just defended a couple of weeks ago, is the main sort of numerical optimization guy in the lab. We developed a lot of our solver infrastructure. Kevin Tracey is the guy who did all this, is still doing this differentiable collision detection work, all that geometry stuff. And then show and chin or the guys who've been making our hardware work and how the experimental work was really driven by these guys have last few months. And with that, I would like to talk to you guys, take questions. Now, hopefully get into an argument. So we have many questions. Very interesting, cool. I guess one question I had, you mentioned you wanted answers, but the real heart attack cancer, that's what you want it. Working with real robots and building robots. I don't want I don't want the vibrations. I don't want the chocolate. Yeah. I'm wondering, even in your example, you had rubber feet on the floor. So that actually all the contact you're going to deal with in any feasible trajectory. Does that change your approach or so? Yeah, thanks for this question. I really like I'd love to dig into this one. So that's all that hard contact term is very loaded, right? So what I'm trying to do is do things that do not have inner penetration artifacts. And I'd argue that's a completely separate phenomena than having compliant sort of pieces on the environment around the robot. You'll hear people say, Oh yeah, we have soft feet. So therefore a soft contact models appropriate here. These are actually very different things. So the soft contact models, what they're giving you is actually force at a distance off the floor before you touch down. And they're giving you interpenetration after you touched down, neither of which are happening in real life, right? So especially the force at a distance stuff, right, is even if you have compliant feet. So we actually want the compliance to for exactly the same reasons and the rubber feet and all that. I have lots of thoughts on this. I'm sure we can have a really long discussion. So I totally agree with you that we want this compliance. Where for me the way I think about that is what the compliance is giving you is low-pass filtering. And what that lets us do is run our control. So if you think about this at the level of pure rigid body mechanics with impacts, you're getting like impulsive accelerations and step functions and your velocity and all this stuff that you think about that from a frequency domain perspective, you're getting like infinite frequency. So your bandwidth has to be gigantic and your controller, whatever. And what the compliance and the robot or the environment is doing for you is basically banned, limiting that, right and giving you some low-pass filtering, which ultimately is how you run a controller, a digital controller at some finite frequency. So it turns out we have mechanisms for getting it that in this stuff. And essentially what, what falls out here is ultimately when we run the simulator, what we're implicitly doing with all of this stuff is smoothing the contact impulse over a timestep. So effectively, to make this stuff match reality, we have to have the timestep sort of on the order of whatever the real bandwidth and compliance and the robot matches up to. If we sample, if our sampling rate is too low, we're basically getting more smoothing or more of this sort of band bandwidth limiting, factor eight. And I think I'm really interested in this sort of thing. And I've sort of been pushing my students are doing the hardware work to better characterize the flip compliance and stuff like this. And I think there's, there's kind of two ways, ways of getting other. One is baking it into the hardware level, which I think is super effective. And there's been a lot of kind of series elastic, everything, rubber feet, et cetera. But I think there's a way to make that more rigorous from this band limiting frequency domain perspective. And ultimately that should inform, we should be able to say, Oh, given this robot hardware with these compliance properties, whatever, we should be able to rigorously say, here's what the resulting bandwidth. We'll be at the motor that the controller needs to reason about. It should basically put a lower bound on the sample rate and you have to run your impedance Controller ad or whatever, right? That's, that's the way I think about that or that we're trying to think about that. That's separate wondering. Yeah. Yeah. So rho is doing physically speaking in the impact case, which is maybe the easiest to understand, it's giving you force at a distance off the contact manifold. It doesn't allow inner penetration because it's got this strict interior point kind of thing going on, though, actual physical interpretation of that and at least action setup. It takes the role of a it takes, so ideally, when it's cranked all the way, it takes the role of a brick wall potential. It shows up like a potential function from physics and it looks like a brick wall potential and it's all the way cranked. So it's basically smoothly approximating a brick wall. And then as you crank, you get closer to the real brick wall potential. Yes. You absolutely do. You can crank it down and it turns out at the levels of smoothing we have here in like real legged robot case, it's equivalent to a couple of millimeters. We can crank it like arbitrarily low. And the upshot here is the reason this is okay and the reason we can crank it arbitrarily low and maybe like things like Luke gioco and these other methods can't, is because we're leveraging all the nice numerical methods and theory that underlie primal-dual interior point methods. Whereas they're using kind of more serious sticky spring damper. Whatever potential things from like more, more intuitive physics, understanding. And ultimately if you kinda don't take the numeric seriously, those things all result in numerical ill conditioning and really poorly poor numerics that you can't solve the problems down to those really stiff levels. Thank you. Yeah. We'd like to follow up more on this. I'm really into this. Excellent. Yeah. It's not a version of g, j, k. It's totally different, works completely differently. We basically take, we can take in that method arbitrary convex shapes. So we can do cones, pills, cylinders, whatever. So they can be like round things, not just polytopes. The only caveat is that you have to be able to write it down mathematically. So some shapes are gonna be really inconvenient to write down. So we have a handful of primitives that we've written down constraint functions for it that are nice. We can do like ellipsoids, Cone baba, blah. And then there's a picture. And if you want to know how that thing actually works, It's sort of subtle. There's a picture in here that I threw up but I didn't explain. This is the picture of how that thing actually works. So it turns out like the naive thing you might do is say, Okay, I have, I need to find a point P1 and P2. P1 has to be in set one P2 USB and set to, and want to minimize the distance between them. So this is all convex. You can do that. That's sort of where we started with this. The problem with that formulation is a breaks down when you have, when you have contact and interpenetration works great if the sets are disjoint, if they're out here, but it becomes ill-posed when they start to intersect. So that's why that doesn't work. Other people have tried that too, by the way. So the thing that works is this, This is Kevin, my student came up with this after it kinda thing about this a lot. Here's what's in this picture. So basically we take the two shapes and we say, we want to find a single point p that has to lie in both sets. So we have T and set one and set two. And what we do is we apply a uniform scaling to the two shapes, alpha. And so we minimize alpha subject to the point being in both sets. And what this does for you is it doesn't change at all if they're interpenetrating or not, it's greater than one if they're not penetrating, is less than one if they are, right. And it turns out that works great. It's bounded and always well-posed, blah, blah, blah. So we can give you all kinds of guarantees on the formulation has no edge cases. And and it's differentiable. We can use all the tricks to it. And we can, at the end of this, extract the closest point and the normals and the tangent basis and all that good stuff. So that's how it works. So what we do here is ultimately, for a complicated thing, we're decomposing everything into convex primitives. So we basically have to evaluate everything pair-wise. So if I had like three of these guys, I would end up having a pairwise solve. I guess, in that case, three problems for each pairwise set of interactions, which I can do, right? So it's fine with the multi contact stuff. You just have to evaluate it for each. If you have a nonconvex thing, you decompose bulb on and solve it, so it doesn't scale horribly, but you get this kind of n squared flavor scaling for many objects, which is what? All the standard stuff now how's anyway? Fabulous. Question about that. So basically you add requires feasible solution initially, which is probably the hardest, solving the original problem. How do you deal with that? And then just solve the problem itself? Yes, there's lots to say about this. So it depends, I guess is the answer. So some of those problems are convex and well-posed. This one in particular is beautiful, always convex, always nice. It turns out the friction problem is also of that flavor. It's always a second-order cone program. All kinds of guarantees always nice. The ones that are not nice. The physics with, with impacts problem is non-convex in general and not always nice. It turns out the reason that one works well is that if you're doing simulation, you're always warm starting on the previous solution. So because of this, if you take small time-steps, it's basically always, it always converges in a couple of Newton iterations and practice because you're always sort of super close to the previous timestep, that one starts to fail if you try to take a really big timesteps that are sort of which is not unreasonable anyway, because if you're using any integrator, if you take two big timesteps, bad things happen, right? So that's what happens there. Then ultimately that outer optimal control problem is the hardest one. It's always terrible, non-convex. And I'd argue that what we've done here is tried to make a math problem like no worse than other non-convex trajectory optimization problems for snooze systems that we're already solving, like the hybrid problem. So that one is obviously that one is the bad one, that one's non-convex, but it was already, even in the hybrid case, already kinda hard, non-convex. The topic of like the initial feasible iterate. So it turns out that's not necessarily the case. Some of these we can do it, right, and we do, but there's this whole body work in the interior point optimization community on infeasible start implementations of these. The trick is what you basically do is you take the stuff that always has to be feasible with respect to the convex set. So the variables that have to be in the cone you introduce, you basically make a set of dummy variables called slack variables. Those you constrained to always be in the cone. And then you introduce a set of equality constraints between those dummy variables and the actual variables in the problem. This lets you initialize these ones trivially to always be in the set with these ones, be whatever they want, and then tries to glue them together as the solver converges. This is how the IP OPT and these other nonlinear programming solvers. They use interior point methods. Do it hack for getting around this issue? Basically, it makes the problem a little bigger, but it makes it always well-posed. We do that for the trajectory optimization problem. In particular, for these other ones that tends to actually we can we can actually do it. Yeah. Any other questions? Yeah. Yeah, so it's really just the classic any anytime we linearize to do control where it really breaks down is when you get knocked too far off your point, you linearized about if you're trying to track some reference trajectory, you've got a really huge disturbance that knocks me really far off that reference than my linearization is going to break down. That's really it. The idea there is we are linearizing strategic pieces of the problem for computational reasons while keeping the overall structure. So I'm really, what I'm doing there is I'm keeping the block structure of that composable optimization thing and I'm linearizing chunks of the inside each block. I keep the overall structure that lets me reason about all these discontinuities and all this contact stuff. While like making each chunk easier to evaluate and faster. One way of thinking about it as some of those chunks are non-convex and hard to solve, maybe I'm basically making all the chunks convex and nice. So I'm taking all these, these blocks that might've been non-linear problems and converting them to like LPs, QPS, things like that, that are really easy to solve, right? That makes sense. As far as the sensory relay thing, that's always an issue and that can be modeled other ways basically, right? That are kind of independent of that linearization thing. So I'd say, yeah, it's really this idea of like large disturbances. We get knocked far off the linearization reference or where you're going to have problems. Okay? So here's a constant. Yeah, so we actually don't work very hard at this. It's so we use the standard approaches in the optimization community for these primal dual interior point methods. We generally initialize it to one and then crank it down. So we use what's called the marrow tra predictor-corrector method. This is like how all these convex solvers work in practice. If you're familiar with E, e-commerce or any of these conic solvers. Basically, there's like a lot of good theory from that community on how to do this well. And what we've done is really just figured out how to take all are nasty contact problem problems and pose them as interior point stuff so that we can go and steal all these nice interior point algorithms from the optimization community that have provable, nice convergence results and things like that. So roughly speaking, if you can, if you can smoosh your problem into an interior point method, then we can just leverage all these tools off the shelf more or less and that's really what we're doing. Oh yeah. Yeah, I mean, so like this can reason about all the contact physics. I mean, that's kinda the idea. So if you have something that's like soft, all, I mean, all you have to do is model it died. Sorry. If you want to interact with some soft thing, if you can give me a model of said soft thing that includes the compliance, then this will just reason about it. Right? And I yeah, The air quotes are very big there for the as anyone who's worked on any of this will tell you, the modeling problems in soft systems are huge and daunting. So, so that's kind of, you know, I can't fix everything, but if you can give me a model than we can do it.