[00:00:05]
>> It was that I didn't look up every year but it was put on the generous donation from Dr Kelly who's here in the front Dr Kelly in the long term George has been a factor he's part of the sermon by the freeboard. And you just have maids in career you have that Darkover every year to start a new program as well as the Information Technology program that might not sound like very much to do but that is the program that bit all of your hard work and the.

[00:00:39]
So everything that we're going to talk about today came from the programs that got us started at DARPA So it's a good interesting history to have there was also the vice president that is a lot of city has really been a wreck and I forgive her. Leadership of technology do you think knowledge he Street and government so is just a wonderful house that you know part of your church that we just want to thank you very generous to.

[00:01:09]
Them have this event. Thank you very much. And I also just want to say are you did not need secretly worthwhile you organizing this event these events always take a lot of people who don't think that that means you really want to do that event have to organize and this is making all this happen so special thank you Governor as well.

[00:01:40]
If I'm going to our guest of honor it's really is a pleasure to introduce someone new as that there's nothing new career there. Is no one or No they can't start ups and the problem is transportation the nation probably the world preserves and got this car you know and that would be also work at robotics and.

[00:02:14]
Destructs area so. For those. Stars. It's really interesting that. There. Are No. Space. Really. Or there. Are probably. So. There's a sense. Thank you thanks very much thank you. Thank you so much and thank you all for coming to listen to drone on for the next you know forty five minutes or so.

[00:03:16]
It's incredible honor and privilege to be here when I got my Ph D. The first job I got out of the program was thanks to Clinton and so when I was given the opportunity to come and speak a lecture series name for him it seemed like you know there was absolutely state could do so thank you all for coming today and indulging indulge me.

[00:03:38]
I'm going to talk today a little bit about the history self driving cars at least the part of it that I've had a chance to see over the last fifteen years talk a little bit about what we're doing with Aurora and then talk about where this technology is going so much challenges we face along the way.

[00:03:53]
But before we get into that I'd like to talk about why we're doing it because I think that's really the most important part and really. Think about it back in one nine hundred thirteen Henry Ford. If the world a Gift right he figured out how you could make the automobile democratized so that everyone can get access to it and that shaped the basically the American nation in the world as everyone was able to get cheap high speed mobility and it's been about one hundred years and there's been significant events in the vehicle but we're kind of on the cusp of transforming mobility again and through the transformation we're going to see an awful lot of benefits.

[00:04:30]
For was at Aurora It starts with safety and these statistics are pretty incredible So if you look at the two point four million injury collisions that happen every year in America the forty thousand people that die on the roads every year that's a profound cost and anyone who talks about going to the risks of automated vehicles and you know can we get there forget the status quo is fairly horrifically broken.

[00:04:57]
Turns out that. The cost to the nation of this is about one trillion dollars This is about one hundred fifty billion dollars of direct cost and the other eight hundred fifty billion dollars is indirect cost of pain and suffering and whatnot around that and so if we can actually move the ball forward we can move the American economy forward dramatically here as well and the good news as a robot assist is that ninety five percent of these accidents are caused by human error so we've got a shot you're saying there's a chance.

[00:05:28]
The next part of it is kind of how we think about the asset that we're we're we're up to eating so today's car sits idle ninety four percent of the time right you drive to work you drive it home maybe go get some groceries the rest of the time the second biggest thing you spend your money on is sat there doing nothing so we can find a way to utilize that more effectively that's that's incredible congestion cost the U.S. economy one hundred twenty four billion dollars in terms of lost labor time and environmental impact and fuel burnt right which Professor which going to come out with environmental impact and then finally the amount of real estate we spend in this country is somewhere.

[00:06:05]
On the order of the size of I looked this up earlier today I think it's about the size of Delaware. Right the parking state of Delaware is paved across the effectively across the nation for each vehicle there's practically three parking spaces there's the one it home there's the one at work and then there's one of the shopping center So imagine if we can get to a shared mobility platform based on autonomy that allows us to recapture that landscape.

[00:06:31]
And then there's some very human elements of this so the technology we're developing will expand access to mobility there's three million Americans today that don't leave their home there's six million Americans today that can't drive because of disability if we can give them the same freedom of movement the human I take for granted it's profoundly important and then for the rest of us like I'm not really a car guy I did at one point discover recently what drive when driving feels good I turns out I own a convertible I was driving on this very beautiful road I had my music playing the light was to take that magic evening light and I thought I get it this is this is why people love driving.

[00:07:13]
And then I got to San Francisco and the fog and the traffic stopped it's like this is why we need help in cars and so anyone who says that they love driving. Doesn't think about the time they spent commuting because anyone who says they love driving while they're commuting probably should get locked up and so if we can give people back those eighty minutes a day that they on average waste commuting right here back to them to rest so that when they get home instead of being keyed up to spend time with a family get it back to them do a little more work give it back to them to you know read a book or enjoy media write we can kind of make the world a happier place which sounds a little corny but I actually think it's important.

[00:07:57]
And then of course I work a company starting a company and it turns out we you know we to have a company you have to make money. And it turns out that U.B.S. estimates that by twenty thirty. Cell phone cards can be a two point three trillion dollar industry I think that's a little optimistic but they're probably smarter than me so this is an incredible amount of economic opportunity up for grabs and so that's why you're seeing an awful lot of activity in the space for me this journey started in a desert in the out of comma the other comedies are back in two thousand and three and this was wrote called Hyperion and we were down there to look for signs of life in this incredibly ared place and this vehicle would drive at fifteen centimeters a second which is about this fast.

[00:08:46]
And the funny thing was there was a buzzard that lived on that hill. And every day it would come out and circle overhead and then go back because it was pretty convinced anybody anything moving that slow in the desert was about to die. And you know became increasingly depressed over the couple of weeks we were there but as we were doing this my Ph D.

[00:09:10]
advisor read Whitaker came down and said. We're going to do this DARPA Grand Challenge and the idea is going to drive from Los Angeles to Las Vegas and we do that less than ten hours and you know we're going to drive fifty miles per hour and I thought that sounds really cool and so that's when I got a chance to in and work as part of the red team building these vehicles and this was the first vehicle we built a Carnegie Mellon's called Center for the grand challenges so this is called sand storm and I kind of think that this is one of the missing links.

[00:09:44]
In the car. Kind of history so Clint helped found Thomas fickle industry and L.V. back in one thousand nine hundred four had lasers cameras and radars on it. One of the innovations that came with this vehicle was not only do we have leased a radar camera but we added maps to it and that kind of prior knowledge and the ability to build you know have a database of where the vehicle was going was something that unlocked the capability of the system and so we built this thing up and in a matter of few months we took it out to the desert and we tested it.

[00:10:21]
And we tested it hard. And you know this was a this rough day. But in about ten days we got the thing back together we ended up taking it down to the Dr competition site the kind of call fire we had on the road we qualified first we sent it out in the desert there was a bunch of you know generals and other military folks there and they watch as this thing took off into the desert and it was it was really like seeing your child go out for school for the first day right we we really were not allowed anywhere near it and off it went in the desert.

[00:10:53]
And about seven point two miles in sorry for those you wonder that's what it looks like on a cell phone car rolls over I wouldn't recommend it. Some point two miles into the race this happens so the vehicle goes around the turn gets a little bit on the inside the corner gets wedged up on the side almost literally burst into flame.

[00:11:17]
Kill it snapped the wheels and the poor thing is dead out in the desert. The good news was that we made some point two miles which was the farthest anyone went. The great news for the guys behind it as we drove through three fenceposts on the way there because they're tired tracks were right in the center of ours.

[00:11:36]
And they were much smaller. But at the end of the day we had gone basically an order of magnitude farther faster than anyone before has it done in this kind of environment and so while the media crucified us and kind of rightfully so imagine Olympic marathon where the best runner went to of the twenty six miles.

[00:11:54]
That somebody did something wrong. DARPA looked at it and said you know what this is actually a step forward this is great so they said come back in another year and instead that one million dollar prize we offered you will offer you two million dollars and we were like that's Crites two million dollars.

[00:12:13]
So we came back for another run at it. Turns out the black part supposed to be down. So this by the way was the lesson for me on this was the whole move fast and break things. Is not the right model for self driving cars. So at the end of the day again this was about ten days before the competition we got the thing flipped over put back into it in the competition and at the end of this race in two thousand and five five finished it and this is vehicle from Stanford on the left here it won the two vehicles from Carnegie Mellon came second and third there's a giant truck from a company called Oshkosh which makes giant trucks.

[00:12:57]
And it finished basically the day later had to overnight in the in the in the desert because of. Because it was so slowly I guess and they had a insurance company and and so we were incredibly excited about this undoable thing was done and we got on our soapbox and we said you know what we're going to be able to save young men and women's lives on the supply line in the military and we're going to you know take this to the next level and DARPA said that's great that you can drive across the desert and when there's nothing moving and not hit so many things but let's see if you can actually drive in traffic as they came up with the concept of the Urban Challenge and the idea here was instead of just driving one hundred fifty miles down the road in a static world we're going to drive on our side of the road and we're going to interact with other traffic along the way and deal with stop signs and they're going to put other people out in cars to create traffic to see how the vehicles behave and so.

[00:13:54]
We got around to the competition and this is an example from that so that's mit. And the black vehicles Cornell. And that is the first self driving car crash in that we're aware of. And I figured it was safe to show that here this place they give this talk where the at that.

[00:14:17]
But what's really what's really actually interesting about this is this is kind of. The canonical problem that we have to deal with today and this is the thing that is really limiting the ability to herself driving cars so if we are going to play the video again here and what you'll see is the Cornell vehicle is stop on the soft on the side of the road.

[00:14:41]
And the MIT vehicle approaches it and figures the Cornell vehicles not going to go anywhere so I'm going to go around it. And then the Cornell vehicle says Actually I'm going to go and then they have a little bit of a coming together. And so what's interesting here is modeling having to model the behavior of the other actors on the road and particularly in kind of off nominal situations so I'm sure I'm confident that if the MIT vehicle been following the Cornell vehicle down the road it would behave perfectly well but in this weird situation understanding what the other actor was going to do is hard and this is one of the things that kind of holds me is one of the core bits of technology we're still working on today of course with human drivers instead of other robotic drivers let's see if we can do that again there you know so at the end of the day.

[00:15:29]
I think there were six teams that actually finished that urban challenge there was Carnegie Mellon this time actually won the event. Stanford finished Virginia Tech finished. Cornell MIT there after they pried them apart they also finished and so this was again a pretty exciting day for the industry because now in this kind of somewhat limited world right there weren't pedestrians there weren't traffic lights but there was a lot of traffic moving around we were able to do this thing on a given moment we didn't get to kind of run it long enough and see if we get a video of it working we had to that day show up and have it work and that was exciting and again we got our soapbox and said we're going to solve this self driving cars are coming and doctor said you're right the problem solved you go find your own funding to take IT industry to commercialization and so at that point what had been this kind of very vibrant community kind of started to fade a little bit and go different ways.

[00:16:29]
Until two thousand and nine Google stepped up and said we'd like to build Well they didn't actually say this publicly at the time but we're going to build a self driving car company and or at least honestly at the time it was were to see if there's a company to be built around self driving cars and so I went to Google to head up the the engineering team there and we set for ourselves two goals on day one and that was to drive one hundred thousand miles of public roads and then to drive a thousand miles of interesting roads so that we kind of understand how hard the problem was and whether we could actually work make this work in the real world now the hundred thousand miles was intend to get as kind of a volume of data right this was an order of magnitude more miles than anyone had driven before and the thousand miles was to kind of keep us honest because the right way to solve driving one hundred thousand miles is you get a few vehicles on a stretch of road you go for four football fields right you gather the data and you see you know like everyone manages to there met.

[00:17:23]
EX I can tell you we certainly did and then the thousand miles though was let's see what the real the breath of the problem was and within about a year and a half we were able to complete those milestones and convince ourselves that while the technology wasn't ready yet it was kind of on the cusp and so by two thousand and eleven we kind of turned the crank at the company and decided that we were going to go build something interesting here build a commercial venture and along that path and I spent about seven and half years ago we had a bunch of worlds first so this was the first time a self-serving card pulled over by the police.

[00:18:01]
And what was really fascinating about this was the officer said he pulled over the vehicle because it was obstructing traffic. That we had the data and there was no traffic backed up behind the car it was not it was just driving down the road turns out the coughs or was just curious this was self driving car you want to learn something about it and so this isn't there another lesson from this was you this is a technology that touches everyone right everyone can kind of get it viscerally and you need to help bring the community along with you you need to put effort into the education so people understand the impact the technology understand where it's going why you're doing it so we can actually build and see the benefits of this.

[00:18:44]
This is one of the last things that happened before I left the company and this gentleman here turns out is blind. And he got to go for a. About a three maybe five mile drive around Austin Texas by himself in a self driving car for the first time ever.

[00:19:02]
And that was truly you know an exciting day because this is a person who relies on public transit and you know leaning on friends and others and you know it for in his mind inconveniencing his friends to get where he wants to go and for a while he actually had the freedom to get where he wanted to go and so that was exciting to see.

[00:19:23]
The the personal connection and the meaning of this this technology has. And of course over the last few years the industry has exploded so it's gone from. You know feeling like a crazy person. You know running across the desert to you know a crazy person running at the Wal-Mart you know Boxing Day sales.

[00:19:48]
And we're seeing interest from the tech companies from the automotive companies and from everyone else in between and what the question I think a lot of people have is why this moment right why now and part of it is the technology is advanced to the point where it's relevant machine learning has become incredibly capable our understanding and ability to use your cloud computing the events the more SUA the advances in sensors these are all kind of pulling together but in parallel with that the business world the business environment has set itself up for this so the automotive world is right now the set by the challenge of electrifying the drive change drive train to deal with environmental impact it's beset by the implications of connectivity and what that means for its business model so instead and I'm not talking about on Star We're talking about things like Uber lift which are fundamentally changing the way we interact with vehicles and then thirdly it's been beset by autonomy and so the big companies out there understand they maybe need to do business a little differently than they have done in the past to make something happen.

[00:20:56]
And so a little under two years ago. We found the company Aurora and the mission of companies to deliver the benefits of having technology safely quickly and broadly and all talk well you heard about the benefits we'll talk a little bit about how we're doing it and it started with three of us so this is your bag now on the left here he is.

[00:21:17]
Up until very recently in fact a member of Carnegie Mellon he and I have known each other since grad school so about twenty years ago. And he's one of the world's experts in machine learning robotics and that's a key part of. Solving this problem in the gentleman over here in the white shirt is Sterling Anderson he's an MIT Ph D.

[00:21:36]
It's been a couple of years at McKinsey and then he helped launch model X. in auto pilot at Tesla so he kind of understands a little bit about what it takes to get technology out the door into the world in a in an automotive production environment and so we thought this was kind of a decent seed to go into this space and help build the company and of course building a company is hard and you need a lot of great people and so we've been building building that team over the last couple of years for about one hundred eighty people today and we've got some some awesome folks so this is the Space X.

[00:22:09]
rockets our vice president software engineering used to be the vice president offer engineering at Space X. So he's helped launch a land rocket so we're like that's cool we you can work here. And then this thing in the middle here is a dump truck the size of a house so before I left her email to go to Google we were working with Caterpillar and it turns out that today if you own a large open pit mine in South America or other places like I'm sure many of you do if you wanted to have a giant dump truck that drove itself you go to Caterpillar and buy it and so a bunch of the team that happened to develop that helped Caterpillar develop that now come and now work in a Rover and are helping us bring other technologies to market we also have one of the co-founders of Ross.

[00:22:56]
Which is kind of a big deal as we think about the infrastructure we need to make this work. So what are we building we're building the world's safest driver and we think about that as the software that kind of understands the world around the vehicle the hardware that allows the car to see the world around it and then compute whatever the software and whatever the software system is and then the data services that run on that notably we have very little interest read zero interest in actually building cars turns out there are people out in the world who know how to do that.

[00:23:29]
And we're not those people so let us focus on what we think we can be world's And world's best that and for that and go find some awesome partners who kind of share a vision of seeing this technology come to market. This is an example of kind of how our software system works so it starts says the vehicle understanding roughly where it is in the world we then layer on top of that map so in this case the curves are in red the double yellow lines are the lanes center marks from the from that map the purple stuff that just layered in here is data from the laser system that's on the vehicle and then finally the perception system turns that into classifications and tracks of other be other objects in the road so here the cyclists behind the vehicle the blue boxes or other cars on the side of the road once we have this kind of understanding of what's happening in the world around the vehicle we can then turn that into action what's the safe trajectory to go get we're going in the world.

[00:24:23]
So this is a cartoon that drew every time I shows it says it's completely wrong but it's approximately right. You know our system starts again with maps from that we build a localization engine that allows us to know precisely where we are in the world so to something like ten centimeter resolution and a few Miller radians.

[00:24:44]
Those two inputs are used by the perception system along with the sensor data understand what's going on around us we then handed over the planning system that turns into controls we think of the car and of course a bunch of other stuff that you know even in the cartoon diagram you know it's more complicated but what's kind of fun is I remember way back when I started graduate school it was a three tier architecture this is it right it you know the core of this really hasn't changed in a very long time.

[00:25:15]
One of those parts is our mapping system so at Aurora we're pretty convinced that you need to engineer the mapping in the motion planning systems together because it turns out engineering is hard particular if you want to make it really work and engineering is really about cheating and the best way to cheat is if you kind of control all parts of the game so if we can take the map system and figure out what the right schemas are to make the problem as easy for the motion planning guys as possible then that's great and when the motion planning guys think they are on the map guys hit a particularly hard thing we can then push that over the motion planning team and shift the complexity around in the system as we as we want so this is our mapping the different layers that are not map in the top left this is a vector representation of the world so these are the lane markings and whatnot that we saw in the video a moment ago the top right this is kind of a three D.

[00:26:04]
geometry of the kind of load bearing surfaces that the vehicle operate on and then the image in the bottom here is what we call a surface tile model so this is you specifically for localizing the vehicle is driving along so it looks models and ugly but it's a key part of the technology we have so that we can very precisely in six degrees of freedom localize the vehicle and this is what that map looks like so again it's where zooming down a freeway here you can see different chunks of that surface tile model.

[00:26:36]
It's basically little patches of a certain radius that in real time we're matching the point cloud data to the. Moment you will see a slightly more complicated intersection that we've built this model. And what this allows us to do very precise driving so it's a parking lot obviously in the snow the interesting thing here is look the tire tracks so we're driving exactly in our tire tracks and here's a tunnel in Pittsburgh if you try to drive through this tunnel using G.P.S. positioning you're probably closing the tunnel.

[00:27:08]
So by using the techniques we're using today we're able to position the vehicle very precisely the wind despite that and operate through the tunnel and autonomous mode. One of the deeply held beliefs we have a river of. Is that safety is a statistical game and you should be using everything you can to your advantage and so we look at some companies that say almost kind of religiously.

[00:27:35]
People drive with two eyes therefore we can just use cameras because that just makes sense and we look at kind of the world we say computer vision is really hard right and it's come a long way in the last decade but is it going to get to six nines or seven lines of reliability in the next five years we're not convinced of that so let's throw everything we can at solving this problem because it's so profoundly important and so when they come when it comes to sensors what that means is we use a combination of camera laser radar because laser radar because they fail in different ways so in the camera image here this is actually a decent automotive quality camera but as it approaches the intersection you'll see it's very hard to pick out that there's a cyclist on the side of the rating there and if you did see it a moment ago you probably thought it was a pedestrian.

[00:28:24]
So it's hard using just passive imaging to see the world this is the vehicle driving through a tunnel again that's where a whole tunnel and what you're seeing is the energy coming back the kind of our image showing the radar returns through that tunnel and what's problematic is if we look towards the top of this image in the middle of the road there's a whole bunch of crud there and it turns out there is not a whole bunch of current there.

[00:28:46]
So we need to be able to pick out what are the actual strong returns from the wall versus the basically the energy that bouncing around in the tunnel and generating spurious returns to do that we probably need other signal than just that the radar data itself and then finally is the small video of our car driving and it's autonomy Asli in a snowstorm and it's a little hard to see and hear because of the exposure but if you look closely you'll see there's again a cloud of returns just around the vehicle and that's energy bouncing off the snow in the atmosphere and coming back to a car obviously we want to build a drive through the snow but we don't want to drive through that car next to us or the one in front of us so be able to differentiate the the different failure modes and be able to combine data from different sensors allows us to mitigate that that problem.

[00:29:34]
One of the other beliefs that we have is that while machine learning is great an incredibly powerful tool is really just a tool and so we believe that you have to take the machine learning systems and you have to look dispassionately at the places where we can engineer hand engineer systems that we know how to engineer and then put the best of the two together and this video kind of captures this so what you're what you're looking at the bottom here is our machine learning detector seeing things in the world and you see this flashing yellow boxes those are things that it's detected it turns out if you drove based on just that data you'd have a rough day but if we can then take that data and pass it through something that looks perhaps like a extended common filter that's been thoughtfully engineered with decent motion models we can actually get really crisp clean track of a vehicles on the road and then we can operate safely and robustly.

[00:30:29]
And so we believe in using all the tools in the tool box not just the ones that are kind of shiny and new. And so here are a couple of examples so on the left the vehicle is making a left turn across traffic and the kind of a fun thing to see here is just the number of vehicles we can track simultaneously right dozens of vehicles through this maneuver and then on the right here what you're seeing is a motorbike splitting lanes so in California motorbikes are allowed to use the dotted line to go faster than traffic which seems crazy to me.

[00:31:04]
But there again I don't ride a motorbike. And so what you can see here is even if that motorcycle gets extremely close to the other vehicles in light next to it we're able to track that despite the proximity and this has to do with the way that we think about how we segment the world and we're not and of this early preset minting that we're detecting interesting actors first.

[00:31:27]
And this is just fun so if you happen to have a gang of mean bicyclists driving around yourself driving a car we can track them too and you're kind of seeing here is again those the combination of detection and thoughtfully to. Comment filtering. So going to talk a little bit now about some of the challenges in motion planning so one of the hard things emotion planning is dealing with other people and doing that a natural way people don't always do what you expect and there's often a lot of subtlety in the decision boundaries about should I make this maneuver or not and people are disappointed cutely aware of these things but when I get it I'm coming in a little too hot that vehicle in front of you feel very uncomfortable Tingley if it's done poorly and so one of the things we have been putting effort into is developing a framework by which we can learn some of those decision boundaries so particular here whether we should stop or go through an intersection and so we've go out we have experts drive our vehicles around and then we reconstruct what we basically record that data and then pass it off to our machine learning system and it then learns the cues as to whether this is time to proceed through the intersection or not and so the the video on the left here is a novel intersection that the vehicle had never experienced before complicated lots of traffic people sometimes not taking their turn and he was actually able to wait until the appropriate gap and then through it the video on the right here and I apologize for the.

[00:33:06]
Difference in contrast so this vehicle is approaching an intersection it into the stop sign intersection and there's a vehicle coming out in front and the for pedestrians to the left of first the car think that might be able to go than it realizes the pedestrians there and this is not coming vehicle so it yields and then takes a moment and then goes after that and this actually ends up feeling quite quite natural.

[00:33:30]
This is one of the actually kind of things that feels the most scary if you get it wrong so this is merging onto a freeway and so our car is here the white thing on the right on this green carpet of where it's planning and to our left you can see just up in front there's a big yellow dump truck behind that I think that's a minivan and so at this moment the car has three choices.

[00:33:51]
Kind of gun it and try to get in from the big truck. Looking at that big space to our left going to maybe accelerate a little bit and tuck in there or fall back behind all three and instead of trying to just drive the speed of the road decelerate to fit their.

[00:34:08]
And it turns out this is this is actually really quite hard to encode there's a lot of subtlety in it so again this is a place where we have gone out and had our expert operators on the road in the vehicles and model and taking that data back and then develop machine learning system to extract it so here's the magic moment words basically I'm going to slow down and instead of trying to slip between I'm going to flop behind and instead of Eckstein you exhilarate up to sixty five miles an hour I'm going to decelerate from the fifty five or so that I met down the fifty three to fit this very natural looking merge into my repertoire so that's a little bit about the technology and to talk to you now a little bit about some of the challenges we faced and really most of these are centered around the interaction between humans and the vehicle and so the first of these is kind of the social model of autonomy and self driving so it turns out this is meme that the reason self driving cars get hit is because they drive weirdly.

[00:35:06]
And it's a very like it's an it's intuitive right if somebody is doing something weird on the road of course other drivers are going to understand them and they're going to walk into what turns out that that's almost never the problem the problem almost always looks like this which is car every car is behaving perfectly reasonable reasonably and then someone bumps into them.

[00:35:27]
And here's a video of this is from publicly on Google's website so let me just orient you to this for a moment so there's a white car here that's a self driving car and then there's two kind of purple cuboid the things in front of them so there's one vehicle that come up to this green traffic light out in the world but is looking across the intersection and the cellphone care can't see it but there's somebody stopped on the far side of the intersection so that first driver right at the intersection line right now is come to a stop so they won't block the box the driver behind them came to a perfectly gradual stop the car behind that came to a perfectly gradual safe spot stop.

[00:36:11]
And then the driver behind them didn't even hit the brakes right and they were checking their phone they were doing something in the car that meant they weren't paying attention the traffic in front of them the medic Lanston seen a green light thought OK this is just going to keep going like normal and then whack into the back of the vehicle and almost all of the self driving car accidents look like this and the thing to take from that is that our understanding of the rate of accidents on America's roadways is actually off substantially because the statistics you hear quoted about rate of accidents or about.

[00:36:46]
Either injury or fatality accidents or police reported accidents and it turns out an almost all of the accidents that we encountered at Google and certainly at Aurora never would have raised the level of the police reportable and so that's kind of this this kind of dark matter of accidents that are out there in America's roads.

[00:37:07]
This is the one accident this is not turning kind it's the social interaction between drivers was the first examiner's of this this was the time that a Google self driving car hit a bus. And what happened in this situation is the car this is a super wide lane the Google cell phone car was human reasonable thing and point to the edge of it to make a right turn it gets mostly to the intersection and sees that there's sandbags in the road and I says OK I'm going to pull out around them it looks behind it and sees a public transit bus coming along.

[00:37:37]
And it looks at the space in the lane the other vehicles how much lane it's occupying and it says that bus can't possibly fit and so it's I'm going to now slowly pull out go around the sandbags because the bus is already going to stop the bus driver from his vantage looked out and said I can make it right and went and saw the still very car then bumped into the side of the bus at about three mph and so this kind of understanding of the intent of other drivers again led to this accident.

[00:38:08]
Fortunately was a very minor accident there's a little more severe accident. So this accident is a similar kind of social convention problem so this was the reaction in Arizona it was not the fatality and what happened in this situation was. The vehicle was driving down a three lane road so the three lanes in the same direction the two outside lanes were basically slowed to a stop and it was a thirty five forty five mph road we were vehicle continues in its lane which is unimpeded and it's OK I've got a clear lane in front of me I'm going to book it whatever the speed limit is a woman was making a left turn out of a driveway saw the stopped traffic and assumed that like any other human driver on the road which would kind of slow down out of caution because it was going to stop traffic to the right that it would be OK to kind of poke her nose out and to make that left turn well turn out you were vehicles moving at speed she didn't anticipate it bumped into the side of the uber vehicle now one hand she should have made the term right it was not a safe maneuver to make on the other this kind of a social wisdom about how the people will drive that was broken in this case because of the behavior of the over software.

[00:39:17]
And then finally this is the completely different class of human factors challenges so this is the Tesla that was it I'm. Josh Brown fatality in Florida and what happened in this case is this gentleman had used autopilot a lot right tens of thousands of miles and he believed that he understood the performance and capabilities of the system and so he was on this this highway in Florida thought that it was like every other fly highways Bedard turns out in this case it had crossroads where trucks would enter the road and as he's driving along paying attention to not driving whatever it was he was doing a truck pulls into the road the autonomy system on the Tesla vehicle the autopilot did not have the ability to detect the truck despite it being a gigantic truck in the radar returns probably coming back from it and so we didn't see anything.

[00:40:12]
The driver wasn't paying attention because he over trusted the system to do the right thing and then unfortunately he went right under the truck and you know a tragedy ensued and so this challenge of understanding what the responsibility of the driver was the responsibility the technology is a fundamentally hard problem that we're going to have to work through as we bring this to technology to market.

[00:40:33]
And so those are the kind of the biggest set of social challenges we face let me talk a little bit about the future so I think this technology is going to split into two fundamental classes of capability one in which it's augmenting the driver and so the person is still driving the vehicle and then the other broad class a few calls where you are being driven you're riding in vehicles going forward.

[00:40:56]
And there's a couple reasons for this one is that it's really hard to take the technology that's used for driver assistance make the leap over to fully self driving capability and the intuition for this comes from from kind of how the nomics in the space work so it turns out if you're building a driver's test and system say for closing that agree mitigation braking right which is really important it's this thing where if you're about to hit a car in front of you it'll hit the brakes for you bring it to a stop it turns out if you develop that system there's a test you want to pass it's the end cap and they basically take you out to a test track that puts them you know foam objects out there and you're not supposed to hit them but I'm underselling it but that's effectively it it's a it's a relatively narrowly defined test and then the other thing that you have to do is you have to not fire on false positives you know basically it's something like one in every hundred thousand miles you're allowed to have a false positive because it turns out if you're driving your fancy B.M.W. down the freeway and it hits the brakes at random you're not happy and then you call the dealer and the dealer is not having they call the yeah right and then if you just sell so when you talk to Tier one is building these systems they can tell you what the false positive rate is and they can tell you they can pass and cap they cannot tell you what the false negative rate is and it turns out they have no incentive to because if it works say half the time which feels like a low false negative rate right that would be an incredible advantage for safety right that would cut out half the accidents or at least the complexity of the damage to do so by them so they really are going to just try and drive the cost out of that system and drive the false positive rate down and get in as many vehicles as possible and so that is a very different technology path than something where you need to keep false positives and false negatives to a minimum.

[00:42:42]
So our belief is this technology comes to market first in right Helen. And the reason for this is it's really expensive to put twelve thousand dollars of extra stuff on a car and then expect or it's really unlikely to put twelve thousand dollars worth of stuff on a car and then expect people to buy that as a feature in contrast if you look at something like lift or movers cost structure it costs them something like a buck sixty mile somewhere maybe to fifty a mile but after their take about a dollar per mile goes to the driver and about sixty cents per mile is the depreciation maintenance fuel etc for the vehicle insurance so imagine a world where you're able to take that driver and instead of paying a dollar per mile you're able to pay ten cents a mile first for kind of the the cost of putting the electronics and software in the vehicle to drive itself that's ninety cents of every of the dollar could have up for grabs.

[00:43:43]
That we shared the consumer and sure to the people selling the technology if it's ten cents to ten cents per mile and you multiply that by the three trillion miles that are driven in the US That's three hundred billion dollars of revenue kind of in play. Another way to think about this is if you're a car company why are you excited about this so I went and looked at Volkswagen looked at their public numbers.

[00:44:09]
And it turns out that they make about four thousand five thousand dollars per car right which is not bad. In contrast imagine if they offer their car for service and they were competing against people that had human drivers operating a surface where the human drivers would cost them about seventy thousand dollars a year.

[00:44:31]
The driving kit let's say were a few years from now that's down to about ten thousand dollars apiece per vehicle so it turns out that the opportunity created by taking the drivers out of the vehicle and operating the service is now three hundred forty thousand dollars per car instead of four and a half thousand dollars per car or five thousand dollars per car with this kind of two orders of magnitude.

[00:44:54]
That's really interesting and so particularly for industries where the margins are very tight this looks like an incredible new opportunity to rebuild your company. For roar we obviously think about the economics we are a company we think that matters a lot but we really the reason why people kind of get up in the morning come to work or is about this kind of core part of the mission delivering the benefits in the world and to talk about you know the opportunity change cities and take that parking space and use it for green space or office space or housing that profoundly important giving more people access and making their day better is important and then saving the forty thousand Americans that are dying on our roads every years lives right that opportunity is incredible.

[00:45:38]
For me there's a lot you know there's some kind of different personal connections to this so this is Steve Mann He's the gentleman I talked about earlier that got the right in Austin but director of Sarah clearance to you for the blind he's an artist he's incredibly interesting guy.

[00:45:55]
And the thought that there's six million other Steve's out there who could have the opportunity to get around that we we all have and be able to touch their lives and just see the joy of giving that freedom back to people is kind of profoundly important. This guy is Steven Fletcher turns He's a Canadian politician he is the brother of the best man at my wedding when he was in his early twenty's he was a geological engineer he was driving up a mine site north in Canada and there's a moose in the road he swerved around the moose and then hit the other moves.

[00:46:32]
And he was instantly paralyzed became a quadriplegic now he's going to have to have an incredible career as a politician he's been in the Cabinet but I magine if we had this technology on that day right and that it had done the right thing and he still had it you know all his all his capability move around or given today again the flexibility where he doesn't have to have another person with him any time he wants to go somewhere and have moments of privacy alone if that's profoundly impactful.

[00:47:02]
And final is my family and these are my two boys and Ethan. On the left here is fifteen in a bit which means within the next six months in California he does learner's permit. And that is terrifying right he's a he's a wonderful guy he's very thoughtful very you know pays a lot of attention to detail but if you look at the statistics for teen drivers it's tragic and so if we get this technology to market and keep my kids safe and keep your family safe right and if we give them our one year sooner that's one point two million people around the globe that don't die right that TO was there's a reason for it and seem to work hard to get this technology to market is so closing I would just ask you to think about kind of joining our mission.

[00:47:49]
Right this is what we're about is getting this technology out in the world doing it fought fully and safely and seeing these benefits realized so thank you very much for your time and I'd love to take your questions was thank you for a very provocative presentation really mistake about a lot of things so thank you very much that we do have some time for questions so we'll start with you please.

[00:48:26]
Yes So the question is how much how much computer power do we have in the car. So. I mean think about what I can say so a lot is the short answer. So we there are numbers that have been put out in the press around somewhere like a kilowatt for the cars through the computer the sensor on the vehicle that's about right.

[00:48:49]
And a lot of that is actually the computer one things to that that you hear a lot about with technology is how expensive the sensors are it turns out that that's not really going to be the heart of the problem the heart is going to be the computer right that the most expensive thing is going to be the whether it's G.P. use or until you know whatever's or custom a six and hopefully the six and drop the cost but that's the most expensive and biggest power hungry part of vehicle.

[00:49:20]
There. Are. Yeah. Yeah I think there's there's a bunch right so one is the housing market will shift. So today if you own property near a train station right within a half mile of that it's much or a transit hub that's much more valuable property then you know they're happier about a mile away.

[00:49:54]
And so as you make the broader part of the city more accessible more is will become desirable I think in the long term this incredible opportunities in public transit so today public transit agencies about ten percent of the cost is actually the terms that are sent ten percent of the cost of operating the service is paid for by turnstile revenue the other ninety percent comes from subsidy subsidies if you imagine taking the driver out of these vehicles and then sharing smaller less expensive vehicles you could get down to a point where we could actually operate transit if not at a profit at basically zero subsidy so I think that would be a really interesting way to kind of provide more mobility into the labor market.

[00:50:35]
Environmental impact you know these things over as we get to density should reduce congestion and so that you know hundred twenty four billion dollars worth of congestion costs I think should get mitigated so there's a lot of a lot of positive I see out there. You know. Yes or so the question is What about interferes with communication D.S.R. cv to V.V. to X.

[00:51:09]
So our belief my belief is that it's a nice to have. So it turns out my kids aren't walking on the street or biking on the street and they do not have transponders on them so the vehicles will need to be able to see other actors on the road even if today we mandated every new vehicle had a vehicle to.

[00:51:29]
Transponder D S R C Get on it can take at least fifteen years and they'll still be stragglers at the end so fifty years to see the market replaced with vehicles that have this so I think that we can't wait for that but what does provide is super powers so in situations where human driver also would not be able to kind of perceive what's going on you can imagine short range communication supporting So we think of it as it's a nice to have extra sensor that has little value today but as the technology deploys you might see more value from a.

[00:52:06]
Question. Yeah so the question is do we have a time frame for releasing a product and what kind of. Milestones do we have to pass through so can't share the first. You know we're working at it is diligently and quickly as we can what I can say is you're starting to see early versions of this technology come to market so in Phoenix for example where most of the company that I used to be part of.

[00:52:39]
Has got a handful of vehicles driving around with the driver in the seat. You know in a in a neighborhood things Chandler so you can see these vehicles on the road today I think you need a ride in them if you're one of the people within the sample group so the technology is kind of getting there.

[00:52:58]
In terms of safety milestones there's a bunch of work that you have to do so. When you know the overall process is called functional safety the idea here is you're looking for ways the system can fail and then figuring out you're going through the rigorous processes how you mitigate those and drive that to an acceptably low risk.

[00:53:17]
Conventional kind of automotive function safety you know follows what is a relatively recent standard goal iso to sixty six to challenge with the sixty six to another function safety systems again it's about things breaking. When the heart of parts of self driving cars is that the world didn't break its behavior will differently than you expected and so there's a concept or a term out there called so if which is safety of the intended function and this is a kind of a more nuanced type of analysis work that we have to go through to kind of convince ourselves that we're ready to put the vehicle out there.

[00:53:58]
Yes. Goes. Yeah yeah. Yeah man. Must. Have some. Of those actors get it yet. You know. Or have so so the question is. I will. And there's the question is how do we make sure we don't end up with a kind of a self driving disaster on the roadway right where we end up with a bunch of empty cars getting rich people.

[00:54:42]
Ice cream and tying up extra you know this kind of shared resource which is a roadway. I think it's a real you know it's actually a really important problem because otherwise we'll lose a lot of these benefits so first I think a lot of the parking will move out of city because while the vehicles moving we can kind of keep it on the road and then we can move it a couple miles out of city in the small overhead in doing that around how do we mitigate the kind of the bad outcomes on this one is that the economics for right Hayling are so compelling relative to partial personal car ownership I think the economics are going to steer this in the right direction to begin with this kind of it's nice when the business model and kind of the desired outcome look a lot alike I think the second is that there's a place where we have to engage with civic leaders about kind of pulling levers of the World Economic Forum just did a really interesting study on the impact of automated mobility in Boston and it's actually a fairly fairly detailed one chunk of work and what they found is basically with no levers most of Boston would see something like I'm making numbers up.

[00:55:51]
But roughly this order of magnitude something like an eight percent decrease in travel time across the Great greater Boston area but within the core of downtown Boston they would see something like a five percent increase in congestion there if you did nothing but then there were a variety of different levers that the city could put in place through taxation and other things that would actually one relative with relatively modest pulls on the levers get it up to something like a twenty percent net decrease in congestion across the whole of the Boston area so really interesting work it was you know it's about.

[00:56:29]
Thinking about current pricing thinking about congestion pricing and multi occupancy Ben kind of an inducing companies to have multi occupancy vehicles there's a lot of interesting ideas about how you manage this kind of in the community. Here's. Your. So the question is we're going to use deep learning and something cars know where it's a dead end and.

[00:57:07]
So we want to be for are going to know. Now we use it we use it actually fairly broadly so. Right now deep learn models are the best. Tool we have for pulling signal out of the sensor data so we are putting a lot of effort into that particularly some interesting work three D.

[00:57:28]
point cloud data we're also. Looking at how do we construct the kind of the motion planning decision space in a way that we can apply machine learning to that deep learning to that so our hope is that we're using it kind of in pieces throughout the whole system.

[00:57:51]
That. You know. That. You know. You know yes so today honestly we are trying to make it work at all so it's there in the question was I quote a big number one point three million people on the world's roads die every year a lot of those happen in particularly low income countries newly mechanizing countries I would describe it as.

[00:58:32]
Well and the answer is. We can't solve all of that yet. You know frankly we can't get the car to drive sufficiently safely today you know is in a very you know relatively benign environment right that's that's the kind of the place to get to first. So we see it rolling out honestly through the developed world probably the more developed world first and then into these lower income countries over time one of the interesting things that may happen.

[00:59:03]
You know if we move fast enough is that much like the way that the cell phone kind of pervaded Logan countries rather than installing fixed telephone line infrastructure there may be an opportunity to catch some of these nations in a point where they're going to just turn the curve America mechanization and you know drop in a replacement mobility system before they get to our ocean because it's you know it's more than just safety that the location right is if you've been to China recently.

[00:59:34]
That the roads just can't deal with the you know kind of the public's demand for for personal car ownership based mobility so we have to find a way where these countries get the benefit that we see from personal car ownership mobility put in a different form factor so.

[00:59:49]
Sustainable. Let's watch. Really. Yeah so we have not really seen that so sorry the question was country companies are concerned about Level five automation because of the liability and maybe pack that in a little and tell you so so today most traffic accidents are due to the human driver right ninety five percent of them are due the human error.

[01:00:23]
It turns out that if you if we were successful in building a safe system we could dramatically reduce the number of accidents that happen but in the process of that the liability particularly if it's a fully automated vehicle like we're trying to build it Aurora the liability is then gone from the driver no accidents to the driver's fault all accidents are going to be tied to the system in the car and so the liabilities shifted in the risk of shifted.

[01:00:49]
I think what we're seeing is that that was a talking point for some of the O.E.M.'s three four five years ago and almost all of them have kind of come to realize that this is happening. You know I showed showed a slide a little while ago not here but if you look at the market cap of we were lifted Deedee which are three of the biggest fright held companies on the planet and you some that together it's the same it was at the time and it's now larger and it's for General Motors and Chrysler.

[01:01:18]
Right and these are companies have been around for a hundred years these are companies that are losing money. And one of them was going to double in the next five years at least and the other one was not at least with what they're doing today so they've kind of they've realized that they need to kind of change their business model and they want to basically they look at those business models they say we want some of that and to do that means they need to get involved in nation.

[01:01:45]
So thank you all for this is. Very interesting presentation about about what the lecture series lecture series was originally intended to be about robots and jobs because of the city it's about the people it's about so much more than just a single topic of the growth and I think that this presentation is just an excellent topic for this series so we want to thank again Dr Kelly for generous donation and for Chris or simply coming and we have a small guest as a token of our creation you can take it on your way to Detroit thank you very much a small gift of thank you very much so much for coming if you are one of the THANK YOU THANK.