back to index

Vijay Kumar: Flying Robots | Lex Fridman Podcast #37


small model | large model

link |
00:00:00.000
The following is a conversation with Vijay Kumar.
link |
00:00:03.160
He's one of the top roboticists in the world,
link |
00:00:05.800
a professor at the University of Pennsylvania,
link |
00:00:08.760
a Dean of Penn Engineering,
link |
00:00:10.680
former director of Grasp Lab,
link |
00:00:12.880
or the General Robotics Automation Sensing
link |
00:00:15.320
and Perception Laboratory at Penn,
link |
00:00:17.560
that was established back in 1979, that's 40 years ago.
link |
00:00:22.600
Vijay is perhaps best known
link |
00:00:24.720
for his work in multi robot systems, robot swarms,
link |
00:00:28.520
and micro aerial vehicles.
link |
00:00:30.880
Robots that elegantly cooperate in flight
link |
00:00:34.040
under all the uncertainty and challenges
link |
00:00:36.200
that the real world conditions present.
link |
00:00:38.760
This is the Artificial Intelligence Podcast.
link |
00:00:41.960
If you enjoy it, subscribe on YouTube,
link |
00:00:44.320
give it five stars on iTunes,
link |
00:00:46.080
support it on Patreon,
link |
00:00:47.560
or simply connect with me on Twitter
link |
00:00:49.480
at Lex Freedman spelled FRID MAN.
link |
00:00:53.280
And now here's my conversation with Vijay Kumar.
link |
00:00:58.680
What is the first robot you've ever built
link |
00:01:01.080
or were a part of building?
link |
00:01:02.840
Way back when I was in graduate school,
link |
00:01:04.760
I was part of a fairly big project
link |
00:01:06.760
that involved building a very large hexapod.
link |
00:01:12.000
This weighed close to 7,000 pounds.
link |
00:01:17.520
And it was powered by hydraulic actuation,
link |
00:01:21.600
or was actuated by hydraulics with 18 motors,
link |
00:01:27.760
hydraulic motors, each controlled by an Intel 8085 processor
link |
00:01:34.200
and an 8086 co processor.
link |
00:01:38.160
And so imagine this huge monster that had 18 joints,
link |
00:01:44.840
each controlled by an independent computer.
link |
00:01:47.000
And there was a 19th computer
link |
00:01:48.560
that actually did the coordination
link |
00:01:50.160
between these 18 joints.
link |
00:01:52.360
So as part of this project,
link |
00:01:53.760
and my thesis work was,
link |
00:01:57.960
how do you coordinate the 18 legs?
link |
00:02:02.120
And in particular, the pressures in the hydraulic cylinders
link |
00:02:06.360
to get efficient locomotion.
link |
00:02:09.240
It sounds like a giant mess.
link |
00:02:11.680
So how difficult is it to make all the motors communicate?
link |
00:02:14.480
Presumably you have to send signals
link |
00:02:16.880
hundreds of times a second, or at least...
link |
00:02:18.720
This was not my work, but the folks who worked on this
link |
00:02:22.800
wrote what I believe to be
link |
00:02:24.200
the first multiprocessor operating system.
link |
00:02:26.640
This was in the 80s.
link |
00:02:29.080
And you had to make sure that obviously messages
link |
00:02:32.240
got across from one joint to another.
link |
00:02:34.640
You have to remember the clock speeds on those computers
link |
00:02:37.960
were about half a megahertz.
link |
00:02:39.640
Right.
link |
00:02:40.480
So the 80s.
link |
00:02:42.200
So not to romanticize the notion,
link |
00:02:45.320
but how did it make you feel to make,
link |
00:02:47.960
to see that robot move?
link |
00:02:51.040
It was amazing.
link |
00:02:52.240
In hindsight, it looks like, well,
link |
00:02:54.160
we built the thing which really should have been much smaller.
link |
00:02:57.240
And of course, today's robots are much smaller.
link |
00:02:59.080
You look at, you know, Boston Dynamics,
link |
00:03:02.200
our ghost robotics has been off from pen.
link |
00:03:06.000
But back then, you were stuck
link |
00:03:08.640
with the substrate you had, the compute you had,
link |
00:03:11.120
so things were unnecessarily big.
link |
00:03:13.640
But at the same time, and this is just human psychology,
link |
00:03:18.000
somehow bigger means grander.
link |
00:03:21.280
You know, people never have the same appreciation
link |
00:03:23.600
for nanotechnology or nano devices
link |
00:03:26.320
as they do for the space shuttle or the Boeing 747.
link |
00:03:30.120
Yeah, you've actually done quite a good job
link |
00:03:32.720
at illustrating that small is beautiful
link |
00:03:35.960
in terms of robotics.
link |
00:03:37.680
So what is on that topic is the most beautiful
link |
00:03:42.520
or elegant robot emotion that you've ever seen.
link |
00:03:46.200
Not to pick favorites or whatever,
link |
00:03:47.880
but something that just inspires you that you remember.
link |
00:03:51.000
Well, I think the thing that I'm most proud of
link |
00:03:54.000
that my students have done is really think about
link |
00:03:57.200
small UAVs that can maneuver and constrain spaces
link |
00:04:00.360
and in particular, their ability to coordinate
link |
00:04:03.640
with each other and form three dimensional patterns.
link |
00:04:06.760
So once you can do that,
link |
00:04:08.920
you can essentially create 3D objects in the sky
link |
00:04:17.000
and you can deform these objects on the fly.
link |
00:04:19.800
So in some sense, your toolbox of what you can create
link |
00:04:23.520
has suddenly got enhanced.
link |
00:04:27.400
And before that, we did the two dimensional version of this.
link |
00:04:29.920
So we had ground robots forming patterns and so on.
link |
00:04:33.760
So that was not as impressive, that was not as beautiful.
link |
00:04:37.080
But if you do it in 3D, suspend it in midair
link |
00:04:40.480
and you've got to go back to 2011 when we did this.
link |
00:04:43.640
Now it's actually pretty standard to do these things
link |
00:04:45.960
eight years later, but back then it was a big accomplishment.
link |
00:04:49.800
So the distributed cooperation
link |
00:04:52.440
is where beauty emerges in your eyes?
link |
00:04:55.640
Well, I think beauty to an engineer is very different
link |
00:04:57.960
from beauty to someone who's looking at robots
link |
00:05:01.520
from the outside, if you will.
link |
00:05:03.400
But what I meant there, so before we said that grand
link |
00:05:07.920
is associated with size.
link |
00:05:10.480
And another way of thinking about this
link |
00:05:13.640
is just the physical shape and the idea
link |
00:05:16.480
that you can create physical shapes in midair
link |
00:05:18.320
and have them deform, that's beautiful.
link |
00:05:21.520
But the individual components,
link |
00:05:23.000
the agility is beautiful too, right?
link |
00:05:24.840
That is true too.
link |
00:05:25.680
So then how quickly can you actually manipulate
link |
00:05:28.400
these three dimensional shapes
link |
00:05:29.560
and the individual components?
link |
00:05:31.200
Yes, you're right.
link |
00:05:32.200
Oh, by the way, said UAV, unmanned aerial vehicle.
link |
00:05:36.760
What's a good term for drones, UAVs, quadcopters?
link |
00:05:41.840
Is there a term that's being standardized?
link |
00:05:44.520
I don't know if there is.
link |
00:05:45.440
Everybody wants to use the word drones.
link |
00:05:47.880
And I've often said there's drones to me
link |
00:05:49.760
is a pejorative word.
link |
00:05:51.000
It signifies something that's dumb,
link |
00:05:53.960
a pre program that does one little thing
link |
00:05:56.320
and robots are anything but drones.
link |
00:05:58.600
So I actually don't like that word,
link |
00:06:00.680
but that's what everybody uses.
link |
00:06:02.960
You could call it unpiloted.
link |
00:06:04.880
Unpiloted.
link |
00:06:05.800
But even unpiloted could be radio controlled,
link |
00:06:08.080
could be remotely controlled in many different ways.
link |
00:06:11.560
And I think the right word
link |
00:06:12.680
is thinking about it as an aerial robot.
link |
00:06:15.040
You also say agile, autonomous aerial robot, right?
link |
00:06:19.080
Yeah, so agility is an attribute,
link |
00:06:20.600
but they don't have to be.
link |
00:06:23.080
So what biological system,
link |
00:06:24.800
because you've also drawn a lot of inspiration
link |
00:06:26.880
with those I've seen bees and ants
link |
00:06:28.640
that you've talked about, what living creatures
link |
00:06:32.400
have you found to be most inspiring
link |
00:06:35.280
as an engineer, instructive in your work in robotics?
link |
00:06:38.560
To me, so ants are really quite incredible creatures, right?
link |
00:06:43.480
So you, I mean, the individuals arguably are very simple
link |
00:06:47.920
in how they're built, and yet they're incredibly resilient
link |
00:06:52.400
as a population.
link |
00:06:54.000
And as individuals, they're incredibly robust.
link |
00:06:56.800
So, if you take an ant with six legs, you remove one leg,
link |
00:07:02.080
it still works just fine.
link |
00:07:04.160
And it moves along,
link |
00:07:05.800
and I don't know that he even realizes it's lost a leg.
link |
00:07:09.800
So that's the robustness at the individual ant level.
link |
00:07:13.480
But then you look about this instinct
link |
00:07:15.400
for self preservation of the colonies,
link |
00:07:17.760
and they adapt in so many amazing ways,
link |
00:07:20.440
you know, transcending gaps by just chaining themselves
link |
00:07:27.680
together when you have a flood,
link |
00:07:30.400
being able to recruit other teammates
link |
00:07:33.160
to carry big morsels of food,
link |
00:07:36.560
and then going out in different directions,
link |
00:07:38.240
looking for food,
link |
00:07:39.560
and then being able to demonstrate consensus,
link |
00:07:43.880
even though they don't communicate directly
link |
00:07:47.240
with each other the way we communicate with each other,
link |
00:07:50.400
in some sense, they also know how to do democracy,
link |
00:07:53.840
probably better than what we do.
link |
00:07:55.480
Yeah, somehow, even democracy is emergent.
link |
00:07:59.000
It seems like all of the phenomena
link |
00:08:00.640
that we see is all emergent.
link |
00:08:02.480
It seems like there's no centralized communicator.
link |
00:08:05.600
There is, so I think a lot is made about that word, emergent,
link |
00:08:09.840
and it means lots of things to different people,
link |
00:08:11.560
but you're absolutely right.
link |
00:08:12.600
I think as an engineer, you think about what element,
link |
00:08:17.600
elemental behaviors, what primitives you could synthesize
link |
00:08:22.720
so that the whole looks incredibly powerful,
link |
00:08:26.640
incredibly synergistic,
link |
00:08:27.920
the whole definitely being greater than some of the parts,
link |
00:08:30.960
and ants are living proof of that.
link |
00:08:33.760
So when you see these beautiful swarms
link |
00:08:36.280
where there's biological systems of robots,
link |
00:08:39.920
do you sometimes think of them
link |
00:08:41.560
as a single individual living intelligent organism?
link |
00:08:45.920
So it's the same as thinking of our human civilization
link |
00:08:49.400
as one organism, or do you still, as an engineer,
link |
00:08:52.920
think about the individual components
link |
00:08:54.560
and all the engineering that went into
link |
00:08:55.840
the individual components?
link |
00:08:57.280
Well, that's very interesting.
link |
00:08:58.600
So again, philosophically, as engineers,
link |
00:09:01.440
what we want to do is to go beyond the individual components,
link |
00:09:06.800
the individual units, and think about it as a unit,
link |
00:09:10.240
as a cohesive unit,
link |
00:09:11.480
without worrying about the individual components.
link |
00:09:14.240
If you start obsessing about the individual building blocks
link |
00:09:19.680
and what they do, you inevitably will find it hard
link |
00:09:26.400
to scale up.
link |
00:09:27.920
Just mathematically,
link |
00:09:28.960
just think about individual things you want to model,
link |
00:09:31.560
and if you want to have 10 of those,
link |
00:09:34.000
then you essentially are taking Cartesian products
link |
00:09:36.400
of 10 things, and that makes it really complicated
link |
00:09:39.280
than to do any kind of synthesis or design
link |
00:09:41.800
in that high dimension space is really hard.
link |
00:09:44.160
So the right way to do this
link |
00:09:45.840
is to think about the individuals in a clever way
link |
00:09:49.080
so that at the higher level,
link |
00:09:51.160
when you look at lots and lots of them,
link |
00:09:53.400
abstractly, you can think of them
link |
00:09:55.320
in some low dimensional space.
link |
00:09:57.120
So what does that involve?
link |
00:09:58.680
For the individual, you have to try to make
link |
00:10:02.160
the way they see the world as local as possible,
link |
00:10:05.160
and the other thing,
link |
00:10:06.440
do you just have to make them robust to collisions?
link |
00:10:09.560
Like you said, with the ants,
link |
00:10:10.880
if something fails, the whole swarm doesn't fail.
link |
00:10:15.320
Right, I think as engineers, we do this.
link |
00:10:17.760
I mean, you know, think about,
link |
00:10:18.840
we build planes or we build iPhones,
link |
00:10:22.240
and we know that by taking individual components,
link |
00:10:26.280
well engineered components,
link |
00:10:27.600
with well specified interfaces
link |
00:10:30.080
that behave in a predictable way,
link |
00:10:31.680
you can build complex systems.
link |
00:10:34.400
So that's ingrained I would claim
link |
00:10:36.840
in most engineers thinking,
link |
00:10:39.400
and it's true for computer scientists as well.
link |
00:10:41.600
I think what's different here is that you want
link |
00:10:44.760
the individuals to be robust in some sense,
link |
00:10:49.480
as we do in these other settings,
link |
00:10:52.000
but you also want some degree of resiliency
link |
00:10:54.480
for the population.
link |
00:10:56.320
And so you really want them to be able
link |
00:10:58.720
to reestablish communication with their neighbors.
link |
00:11:03.840
You want them to rethink their strategy
link |
00:11:07.320
for group behavior.
link |
00:11:09.040
You want them to reorganize.
link |
00:11:12.440
And that's where I think a lot of the challenges lie.
link |
00:11:16.120
So just at a high level,
link |
00:11:18.400
what does it take for a bunch of,
link |
00:11:22.400
what should we call them,
link |
00:11:23.560
flying robots to create a formation?
link |
00:11:26.920
Just for people who are not familiar with robotics
link |
00:11:30.400
in general, how much information is needed?
link |
00:11:33.000
How do you even make it happen
link |
00:11:36.040
without a centralized controller?
link |
00:11:39.720
So I mean, there are a couple of different ways
link |
00:11:41.320
of looking at this.
link |
00:11:43.400
If you are a purist, you think of it as a way
link |
00:11:50.040
of recreating what nature does.
link |
00:11:53.800
So nature forms groups for several reasons,
link |
00:11:58.680
but mostly it's because of this instinct
link |
00:12:02.200
that organisms have of preserving their colonies,
link |
00:12:07.280
their population, which means what?
link |
00:12:11.200
You need shelter, you need food, you need to procreate,
link |
00:12:14.640
and that's basically it.
link |
00:12:16.480
So the kinds of interactions you see are all organic.
link |
00:12:20.120
They're all local.
link |
00:12:22.320
And the only information that they share,
link |
00:12:25.760
and mostly it's indirectly,
link |
00:12:27.800
is to again preserve the herd or the flock
link |
00:12:31.040
or the swarm and either by looking for new sources of food
link |
00:12:39.400
or looking for new shelters, right?
link |
00:12:42.960
As engineers, when we build swarms, we have a mission.
link |
00:12:48.280
And when you think of a mission,
link |
00:12:52.080
and it involves mobility,
link |
00:12:54.360
most often it's described in some kind
link |
00:12:56.840
of a global coordinate system.
link |
00:12:58.800
As a human, as an operator, as a commander,
link |
00:13:03.080
or as a collaborator, I have my coordinate system
link |
00:13:07.120
and I want the robots to be consistent with that.
link |
00:13:11.120
So I might think of it slightly differently.
link |
00:13:14.720
I might want the robots to recognize that coordinate system,
link |
00:13:18.960
which means not only do they have to think locally
link |
00:13:21.360
in terms of who their immediate neighbors are,
link |
00:13:23.160
but they have to be cognizant
link |
00:13:24.640
of what the global environment looks like.
link |
00:13:28.320
So if I go, if I say surround this building
link |
00:13:31.080
and protect this from intruders,
link |
00:13:33.280
well, they're immediately in a building
link |
00:13:35.160
centered coordinate system
link |
00:13:36.520
and I have to tell them where the building is.
link |
00:13:38.720
And they're globally collaborating
link |
00:13:40.080
on the map of that building.
link |
00:13:41.360
They're maintaining some kind of global,
link |
00:13:44.240
not just in the frame of the building,
link |
00:13:45.560
but there's information that's ultimately being built up
link |
00:13:49.040
explicitly as opposed to kind of implicitly,
link |
00:13:53.320
like nature might.
link |
00:13:54.400
Correct, correct.
link |
00:13:55.240
So in some sense, nature is very, very sophisticated,
link |
00:13:57.720
but the tasks that nature solves
link |
00:14:00.480
or needs to solve are very different
link |
00:14:03.040
from the kind of engineered tasks,
link |
00:14:05.160
artificial tasks that we are forced to address.
link |
00:14:09.800
And again, there's nothing preventing us
link |
00:14:12.560
from solving these other problems,
link |
00:14:15.200
but ultimately it's about impact.
link |
00:14:16.640
You want these swarms to do something useful.
link |
00:14:19.400
And so you're kind of driven into this very unnatural,
link |
00:14:24.400
if you will, unnatural meaning, not like how nature does,
link |
00:14:27.840
setting.
link |
00:14:29.000
And it's probably a little bit more expensive
link |
00:14:31.720
to do it the way nature does,
link |
00:14:33.560
because nature is less sensitive to the loss of the individual
link |
00:14:39.280
and cost wise in robotics,
link |
00:14:42.080
I think you're more sensitive to losing individuals.
link |
00:14:45.280
I think that's true, although if you look at the price
link |
00:14:48.800
to performance ratio of robotic components,
link |
00:14:51.320
it's coming down dramatically.
link |
00:14:53.640
I'm interested.
link |
00:14:54.480
Right, it continues to come down.
link |
00:14:56.040
So I think we're asymptotically approaching the point
link |
00:14:58.920
where we would get, yeah,
link |
00:14:59.960
the cost of individuals would really become insignificant.
link |
00:15:05.080
So let's step back at a high level view,
link |
00:15:07.600
the impossible question of what kind of,
link |
00:15:11.680
as an overview, what kind of autonomous flying vehicles
link |
00:15:14.400
are there in general?
link |
00:15:16.200
I think the ones that receive a lot of notoriety
link |
00:15:19.720
are obviously the military vehicles.
link |
00:15:22.560
Military vehicles are controlled by a base station,
link |
00:15:26.280
but have a lot of human supervision,
link |
00:15:29.640
but have limited autonomy,
link |
00:15:31.800
which is the ability to go from point A to point B,
link |
00:15:34.760
and even the more sophisticated vehicles
link |
00:15:38.320
can do autonomous takeoff and landing.
link |
00:15:41.760
And those usually have wings and they're heavy?
link |
00:15:44.400
Usually they're wings,
link |
00:15:45.360
but there's nothing preventing us from doing this
link |
00:15:47.440
for helicopters as well.
link |
00:15:49.000
There are many military organizations that have
link |
00:15:53.440
autonomous helicopters in the same vein.
link |
00:15:56.560
And by the way, you look at autopilots and airplanes,
link |
00:16:00.080
and it's actually very similar.
link |
00:16:02.800
In fact, one interesting question we can ask is,
link |
00:16:07.160
if you look at all the air safety violations,
link |
00:16:12.120
all the crashes that occurred,
link |
00:16:14.080
would they have happened if the plane
link |
00:16:16.880
were truly autonomous, and I think you'll find
link |
00:16:20.200
that in many of the cases,
link |
00:16:21.960
because of pilot error, we made silly decisions.
link |
00:16:24.600
And so in some sense, even in air traffic,
link |
00:16:26.920
commercial air traffic, there's a lot of applications,
link |
00:16:29.760
although we only see autonomy being enabled
link |
00:16:33.920
at very high altitudes when the plane is an autopilot.
link |
00:16:41.160
There's still a role for the human,
link |
00:16:42.520
and that kind of autonomy is, you're kind of implying,
link |
00:16:47.640
I don't know what the right word is,
link |
00:16:48.680
but it's a little dumber than it could be.
link |
00:16:53.480
Right, so in the lab, of course,
link |
00:16:55.720
we can afford to be a lot more aggressive.
link |
00:16:59.200
And the question we try to ask is,
link |
00:17:04.600
can we make robots that will be able to make decisions
link |
00:17:09.600
without any kind of external infrastructure?
link |
00:17:13.680
So what does that mean?
link |
00:17:14.880
So the most common piece of infrastructure
link |
00:17:16.960
that airplanes use today is GPS.
link |
00:17:20.560
GPS is also the most brittle form of information.
link |
00:17:26.680
If you've driven in a city, try to use GPS navigation,
link |
00:17:30.480
tall buildings, you immediately lose GPS.
link |
00:17:33.720
And so that's not a very sophisticated way
link |
00:17:36.320
of building autonomy.
link |
00:17:37.880
I think the second piece of infrastructure
link |
00:17:39.560
that I rely on is communications.
link |
00:17:41.920
Again, it's very easy to jam communications.
link |
00:17:47.400
In fact, if you use Wi Fi,
link |
00:17:49.680
you know that Wi Fi signals drop out,
link |
00:17:51.880
cell signals drop out.
link |
00:17:53.560
So to rely on something like that is not good.
link |
00:17:58.600
The third form of infrastructure we use,
link |
00:18:01.240
and I hate to call it infrastructure,
link |
00:18:02.960
but it is that in the sense of robots, it's people.
link |
00:18:06.400
So you could rely on somebody to pilot you.
link |
00:18:08.760
Right.
link |
00:18:09.960
And so the question you wanna ask is
link |
00:18:11.600
if there are no pilots,
link |
00:18:13.400
if there's no communications with any base station,
link |
00:18:16.200
if there's no knowledge of position,
link |
00:18:18.720
and if there's no a priori map,
link |
00:18:21.640
a priori knowledge of what the environment looks like,
link |
00:18:24.880
a priori model of what might happen in the future.
link |
00:18:28.280
Can robots navigate?
link |
00:18:29.560
So that is true autonomy.
link |
00:18:31.440
So that's true autonomy.
link |
00:18:33.240
And we're talking about, you mentioned
link |
00:18:35.040
like military applications and drones.
link |
00:18:36.880
Okay, so what else is there?
link |
00:18:38.280
You talk about agile autonomous flying robots, aerial robots.
link |
00:18:43.480
So that's a different kind of, it's not winged,
link |
00:18:46.320
it's not big, at least it's small.
link |
00:18:48.120
So I use the word agility mostly,
link |
00:18:50.800
or at least we're motivated to do agile robots,
link |
00:18:53.480
mostly because robots can operate
link |
00:18:57.960
and should be operating in constrained environments.
link |
00:19:02.120
And if you want to operate the way a global hawk operates,
link |
00:19:06.960
I mean, the kinds of conditions in which you operate
link |
00:19:09.120
are very, very restrictive.
link |
00:19:11.760
If you wanna go inside a building,
link |
00:19:13.720
for example, for search and rescue,
link |
00:19:15.600
or to locate an active shooter,
link |
00:19:18.120
or you wanna navigate under the canopy in an orchard
link |
00:19:22.120
to look at health of plants,
link |
00:19:23.880
or to count fruits to measure the tree trunks.
link |
00:19:31.240
These are things we do, by the way.
link |
00:19:33.240
Yeah, some cool agriculture stuff you've shown in the past,
link |
00:19:35.920
it's really awesome.
link |
00:19:36.760
So in those kinds of settings, you do need that agility.
link |
00:19:40.400
Agility does not necessarily mean
link |
00:19:42.560
you break records for the 100 meters dash.
link |
00:19:45.440
What it really means is you see the unexpected
link |
00:19:48.040
and you're able to maneuver in a safe way,
link |
00:19:51.520
and in a way that gets you the most information
link |
00:19:55.440
about the thing you're trying to do.
link |
00:19:57.720
By the way, you may be the only person
link |
00:20:00.520
who in a TED talk has used a math equation,
link |
00:20:04.280
which is amazing, people should go see one of your TED talks.
link |
00:20:07.720
Actually, it's very interesting
link |
00:20:08.840
because the TED curator, Chris Anderson, told me,
link |
00:20:13.560
you can't show math.
link |
00:20:15.400
And I thought about it, but that's who I am.
link |
00:20:18.240
I mean, that's our work.
link |
00:20:20.800
And so I felt compelled to give the audience a taste
link |
00:20:25.800
for at least some math.
link |
00:20:27.680
So on that point, simply, what does it take
link |
00:20:32.680
to make a thing with four motors fly, a quadcopter,
link |
00:20:37.120
one of these little flying robots?
link |
00:20:41.560
How hard is it to make it fly?
link |
00:20:43.800
How do you coordinate the four motors?
link |
00:20:47.360
How do you convert those motors into actual movement?
link |
00:20:52.400
So this is an interesting question.
link |
00:20:54.600
We've been trying to do this since 2000.
link |
00:20:57.840
It is a commentary on the sensors
link |
00:21:00.360
that were available back then,
link |
00:21:01.880
and the computers that were available back then.
link |
00:21:05.640
And a number of things happened
link |
00:21:08.080
between 2000 and 2007.
link |
00:21:11.640
One is the advances in computing, which is,
link |
00:21:15.560
so we all know about Moore's Law,
link |
00:21:16.840
but I think 2007 was a tipping point,
link |
00:21:19.760
the year of the iPhone, the year of the cloud.
link |
00:21:22.800
Lots of things happened in 2007.
link |
00:21:25.680
But going back even further,
link |
00:21:27.640
inertial measurement units as a sensor really matured.
link |
00:21:31.440
Again, lots of reasons for that.
link |
00:21:34.000
Certainly there's a lot of federal funding,
link |
00:21:35.480
particularly DARPA in the US,
link |
00:21:38.360
but they didn't anticipate this boom in IMUs.
link |
00:21:43.800
But if you look subsequently,
link |
00:21:46.080
what happened is that every car manufacturer
link |
00:21:49.000
had to put an airbag in,
link |
00:21:50.120
which meant you had to have an accelerometer on board.
link |
00:21:52.720
And so that drove down the price
link |
00:21:54.080
to performance ratio of the sensors.
link |
00:21:56.280
I should know this, that's very interesting.
link |
00:21:57.960
It's very interesting, the connection there.
link |
00:21:59.480
And that's why research is very hard to predict the outcomes.
link |
00:22:04.920
And again, the federal government spent a ton of money
link |
00:22:07.760
on things that they thought were useful for resonators,
link |
00:22:12.360
but it ended up enabling these small UAVs, which is great,
link |
00:22:16.920
because I could have never raised that much money
link |
00:22:18.600
and told, sold this project,
link |
00:22:20.800
hey, we want to build these small UAVs.
link |
00:22:22.280
Can you actually fund the development of low cost IMUs?
link |
00:22:25.520
So why do you need an IMU on an IMU?
link |
00:22:27.720
So I'll come back to that,
link |
00:22:30.440
but so in 2007, 2008, we were able to build these.
link |
00:22:33.400
And then the question you're asking was a good one,
link |
00:22:35.280
how do you coordinate the motors to develop this?
link |
00:22:40.320
But over the last 10 years, everything is commoditized.
link |
00:22:43.920
A high school kid today can pick up a Raspberry Pi kit
link |
00:22:49.520
and build this,
link |
00:22:50.600
all the low levels functionality is all automated.
link |
00:22:53.240
But basically at some level, you have to drive the motors
link |
00:22:59.160
at the right RPMs, the right velocity,
link |
00:23:04.560
in order to generate the right amount of thrust
link |
00:23:07.480
in order to position it and orient it
link |
00:23:09.960
in a way that you need to in order to fly.
link |
00:23:13.800
The feedback that you get is from onboard sensors
link |
00:23:16.680
and the IMU is an important part of it.
link |
00:23:18.400
The IMU tells you what the acceleration is
link |
00:23:23.840
as well as what the angular velocity is.
link |
00:23:26.400
And those are important pieces of information.
link |
00:23:30.440
In addition to that, you need some kind of local position
link |
00:23:34.200
or velocity information.
link |
00:23:37.440
For example, when we walk,
link |
00:23:39.320
we implicitly have this information
link |
00:23:41.520
because we kind of know how, what our stride length is.
link |
00:23:45.800
We also are looking at images fly past our retina,
link |
00:23:51.440
if you will, and so we can estimate velocity.
link |
00:23:54.240
We also have accelerometers in our head
link |
00:23:56.280
and we're able to integrate all these pieces of information
link |
00:23:59.120
to determine where we are as we walk.
link |
00:24:02.320
And so robots have to do something very similar.
link |
00:24:04.320
You need an IMU, you need some kind of a camera
link |
00:24:08.160
or other sensor that's measuring velocity.
link |
00:24:11.600
And then you need some kind of a global reference frame
link |
00:24:15.800
if you really want to think about doing something
link |
00:24:19.480
in a world coordinate system.
link |
00:24:21.280
And so how do you estimate your position
link |
00:24:23.680
with respect to that global reference frame?
link |
00:24:25.160
That's important as well.
link |
00:24:26.560
So coordinating the RPMs of the four motors
link |
00:24:29.520
is what allows you to, first of all, fly and hover
link |
00:24:32.680
and then you can change the orientation
link |
00:24:35.640
and the velocity and so on.
link |
00:24:37.640
Exactly, exactly.
link |
00:24:38.480
So there's a bunch of degrees of freedom
link |
00:24:40.360
or there's six degrees of freedom
link |
00:24:42.240
but you only have four inputs, the four motors.
link |
00:24:44.960
And it turns out to be a remarkably versatile configuration.
link |
00:24:50.960
You think at first, well, I only have four motors,
link |
00:24:53.120
how do I go sideways?
link |
00:24:55.040
But it's not too hard to say, well,
link |
00:24:56.360
if I tilt myself, I can go sideways.
link |
00:24:59.200
And then you have four motors pointing up,
link |
00:25:01.200
how do I rotate in place about a vertical axis?
link |
00:25:05.400
Well, you rotate them at different speeds
link |
00:25:07.840
and that generates reaction moments
link |
00:25:09.760
and that allows you to turn.
link |
00:25:11.560
So it's actually a pretty,
link |
00:25:13.400
it's an optimal configuration from an engineer standpoint.
link |
00:25:17.960
It's very simple, very cleverly done and very versatile.
link |
00:25:23.800
So if you could step back to a time,
link |
00:25:27.320
so I've always known flying robots as,
link |
00:25:31.120
to me it was natural that the quadcopters should fly.
link |
00:25:35.840
But when you first started working with it,
link |
00:25:38.040
how surprised are you that you can make,
link |
00:25:42.040
do so much with the four motors?
link |
00:25:45.560
How surprising is that you can make this thing fly,
link |
00:25:47.640
first of all, that you can make it hover,
link |
00:25:49.800
then you can add control to it?
link |
00:25:52.920
Firstly, this is not, the four motor configuration
link |
00:25:55.800
is not ours, it has at least a hundred year history.
link |
00:26:01.080
And various people,
link |
00:26:02.480
various people try to get quadrotors to fly
link |
00:26:06.280
without much success.
link |
00:26:09.240
As I said, we've been working on this since 2000.
link |
00:26:11.560
Our first designs were,
link |
00:26:13.360
well, this is way too complicated.
link |
00:26:15.160
Why not we try to get an omnidirectional flying robot?
link |
00:26:19.160
So our early designs, we had eight rotors.
link |
00:26:22.760
And so these eight rotors were arranged uniformly
link |
00:26:27.520
on a sphere, if you will.
link |
00:26:28.880
So you can imagine a symmetric configuration
link |
00:26:31.360
and so you should be able to fly anywhere.
link |
00:26:34.160
But the real challenge we had is the strength
link |
00:26:36.280
to weight ratio is not enough,
link |
00:26:37.880
and of course we didn't have the sensors and so on.
link |
00:26:41.240
So everybody knew, or at least the people
link |
00:26:43.840
who worked with rotor crafts knew,
link |
00:26:45.680
four rotors would get it done.
link |
00:26:48.280
So that was not our idea.
link |
00:26:50.200
But it took a while before we could actually do
link |
00:26:53.480
the onboard sensing and the computation
link |
00:26:56.520
that was needed for the kinds of agile maneuvering
link |
00:27:00.400
that we wanted to do in our little aerial robots.
link |
00:27:03.800
And that only happened between 2007 and 2009 in our lab.
link |
00:27:08.320
Yeah, and you have to send the signal
link |
00:27:10.680
maybe a hundred times a second.
link |
00:27:13.200
So the compute there is everything
link |
00:27:15.400
has to come down in price.
link |
00:27:16.720
And what are the steps of getting from point A to point B?
link |
00:27:22.320
So we just talked about like local control,
link |
00:27:25.840
but if all the kind of cool dancing in the air
link |
00:27:30.840
that I've seen you show, how do you make it happen?
link |
00:27:34.480
Make it trajectory, first of all, okay,
link |
00:27:38.840
figure out a trajectory, so plan a trajectory,
link |
00:27:41.600
and then how do you make that trajectory happen?
link |
00:27:44.320
I think planning is a very fundamental problem in robotics.
link |
00:27:47.320
I think 10 years ago it was an esoteric thing,
link |
00:27:50.120
but today with self driving cars,
link |
00:27:52.360
everybody can understand this basic idea
link |
00:27:55.160
that a car sees a whole bunch of things
link |
00:27:57.320
and it has to keep a lane or maybe make a right turn
link |
00:27:59.720
or switch lanes, it has to plan a trajectory,
link |
00:28:02.160
it has to be safe, it has to be efficient.
link |
00:28:04.320
So everybody's familiar with that.
link |
00:28:06.120
That's kind of the first step
link |
00:28:07.400
that you have to think about when you say autonomy.
link |
00:28:14.320
And so for us, it's about finding smooth motions,
link |
00:28:18.600
motions that are safe.
link |
00:28:20.760
So we think about these two things.
link |
00:28:22.360
One is optimality, one is safety.
link |
00:28:24.160
Clearly you cannot compromise safety.
link |
00:28:26.720
So you're looking for safe, optimal motions.
link |
00:28:30.160
The other thing you have to think about
link |
00:28:33.160
is can you actually compute a reasonable trajectory
link |
00:28:37.360
in a small amount of time, because you have a time budget.
link |
00:28:41.560
So the optimal becomes suboptimal,
link |
00:28:44.360
but in our lab we focus on synthesizing smooth trajectory
link |
00:28:50.560
that satisfy all the constraints.
link |
00:28:52.360
In other words, don't violate any safety constraints
link |
00:28:57.200
and is as efficient as possible.
link |
00:29:02.200
And when I say efficient, it could mean
link |
00:29:04.600
I want to get from point A to point B as quickly as possible
link |
00:29:07.760
or I want to get to it as gracefully as possible
link |
00:29:11.200
or I want to consume as little energy as possible.
link |
00:29:15.360
But always staying within the safety constraints.
link |
00:29:17.600
But yes, always finding a safe trajectory.
link |
00:29:22.440
So there's a lot of excitement and progress
link |
00:29:24.440
in the field of machine learning.
link |
00:29:26.440
And reinforcement learning and the neural network variant
link |
00:29:31.440
of that with deeper reinforcement learning.
link |
00:29:33.440
Do you see a role of machine learning in...
link |
00:29:37.440
So a lot of the success with flying robots
link |
00:29:40.040
did not rely on machine learning,
link |
00:29:41.840
except for maybe a little bit of the perception
link |
00:29:44.440
on the computer vision side.
link |
00:29:46.040
On the control side and the planning,
link |
00:29:48.040
do you see there's a role in the future
link |
00:29:50.040
for machine learning?
link |
00:29:51.040
So let me disagree a little bit with you.
link |
00:29:53.040
I think we never perhaps called out in my work
link |
00:29:56.040
called out learning, but even this very simple idea
link |
00:29:59.040
of being able to fly through a constrained space.
link |
00:30:04.040
The first time you try it, you'll invariably...
link |
00:30:07.040
You might get it wrong if the task is challenging.
link |
00:30:10.040
And the reason is to get it perfectly right,
link |
00:30:14.040
you have to model everything in the environment.
link |
00:30:17.040
And flying is notoriously hard to model.
link |
00:30:22.040
There are aerodynamic effects that we constantly discover,
link |
00:30:28.040
even just before I was talking to you,
link |
00:30:31.040
I was talking to a student about how blades flap when they fly.
link |
00:30:37.040
And that ends up changing how a rotorcraft
link |
00:30:43.040
is accelerated in the angular direction.
link |
00:30:46.040
Does it use like microflaps or something?
link |
00:30:48.040
It's not microflaps.
link |
00:30:49.040
We assume that each blade is rigid,
link |
00:30:52.040
but actually it flaps a little bit.
link |
00:30:54.040
It bends.
link |
00:30:55.040
Interesting, yeah.
link |
00:30:56.040
And so the models rely on the fact,
link |
00:30:58.040
on the assumption that they're actually rigid.
link |
00:31:01.040
But that's not true.
link |
00:31:02.040
If you're flying really quickly,
link |
00:31:04.040
these effects become significant.
link |
00:31:07.040
If you're flying close to the ground,
link |
00:31:09.040
you get pushed off by the ground.
link |
00:31:12.040
Something which every pilot knows when he tries to land
link |
00:31:15.040
or she tries to land, this is called a ground effect.
link |
00:31:19.040
Something very few pilots think about
link |
00:31:21.040
is what happens when you go close to a ceiling,
link |
00:31:23.040
or you get sucked into a ceiling.
link |
00:31:25.040
There are very few aircraft that fly close to any kind of ceiling.
link |
00:31:29.040
Likewise, when you go close to a wall,
link |
00:31:33.040
there are these wall effects.
link |
00:31:36.040
And if you've gone on a train and you pass another train
link |
00:31:39.040
that's traveling the opposite direction,
link |
00:31:41.040
you can feel the buffeting.
link |
00:31:43.040
And so these kinds of microclimates
link |
00:31:46.040
affect our UAVs significantly.
link |
00:31:48.040
And they're impossible to model, essentially.
link |
00:31:51.040
I wouldn't say they're impossible to model,
link |
00:31:53.040
but the level of sophistication you would need
link |
00:31:55.040
in the model and the software would be tremendous.
link |
00:32:00.040
Plus, to get everything right would be awfully tedious.
link |
00:32:03.040
So the way we do this is over time,
link |
00:32:05.040
we figure out how to adapt to these conditions.
link |
00:32:10.040
So early on, we use the form of learning
link |
00:32:13.040
that we call iterative learning.
link |
00:32:15.040
So this idea, if you want to perform a task,
link |
00:32:18.040
there are a few things that you need to change
link |
00:32:23.040
and iterate over a few parameters
link |
00:32:25.040
that over time you can figure out.
link |
00:32:29.040
So I could call it policy gradient reinforcement learning,
link |
00:32:34.040
but actually it was just iterative learning.
link |
00:32:36.040
And so this was there way back.
link |
00:32:38.040
I think what's interesting is,
link |
00:32:40.040
if you look at autonomous vehicles today,
link |
00:32:43.040
learning occurs, could occur in two pieces.
link |
00:32:46.040
One is perception, understanding the world.
link |
00:32:48.040
Second is action, taking actions.
link |
00:32:51.040
Everything that I've seen that is successful
link |
00:32:54.040
is on the perception side of things.
link |
00:32:56.040
So in computer vision, we've made amazing strides
link |
00:32:59.040
in the last 10 years.
link |
00:33:00.040
So recognizing objects, actually detecting objects,
link |
00:33:03.040
classifying them and tagging them in some sense,
link |
00:33:08.040
annotating them, this is all done through machine learning.
link |
00:33:11.040
On the action side, on the other hand,
link |
00:33:13.040
I don't know if any examples where there are fielded systems
link |
00:33:17.040
where we actually learn the right behavior.
link |
00:33:21.040
Outside of single demonstration is successful.
link |
00:33:23.040
On the laboratory, this is the Holy Grail.
link |
00:33:25.040
Can you do end to end learning?
link |
00:33:27.040
Can you go from pixels to motor currents?
link |
00:33:31.040
This is really, really hard.
link |
00:33:33.040
And I think if you go forward,
link |
00:33:36.040
the right way to think about these things
link |
00:33:38.040
is data driven approaches, learning based approaches,
link |
00:33:43.040
in concert with model based approaches,
link |
00:33:46.040
which is the traditional way of doing things.
link |
00:33:48.040
So I think there's a piece,
link |
00:33:50.040
there's a role for each of these methodologies.
link |
00:33:52.040
So what do you think, just jumping out on topics,
link |
00:33:55.040
since you mentioned autonomous vehicles,
link |
00:33:57.040
what do you think are the limits on the perception side?
link |
00:33:59.040
So I've talked to Elon Musk,
link |
00:34:02.040
and there on the perception side,
link |
00:34:04.040
they're using primarily computer vision
link |
00:34:07.040
to perceive the environment.
link |
00:34:09.040
In your work with, because you work with the real world a lot,
link |
00:34:13.040
and the physical world,
link |
00:34:15.040
what are the limits of computer vision?
link |
00:34:17.040
Do you think you can solve autonomous vehicles,
link |
00:34:20.040
focusing on the perception side,
link |
00:34:22.040
focusing on vision alone and machine learning?
link |
00:34:25.040
So we also have a spin off company, Exxon Technologies,
link |
00:34:29.040
that works underground in mines.
link |
00:34:32.040
So you go into mines, they're dark.
link |
00:34:35.040
They're dirty.
link |
00:34:37.040
You fly in a dirty area,
link |
00:34:39.040
there's stuff you kick up by the propellers,
link |
00:34:42.040
the downwash kicks up dust.
link |
00:34:44.040
I challenge you to get a computer vision algorithm to work there.
link |
00:34:48.040
So we use LIDARS in that setting.
link |
00:34:53.040
Indoors, and even outdoors when we fly through fields,
link |
00:34:57.040
I think there's a lot of potential
link |
00:34:59.040
for just solving the problem using computer vision alone.
link |
00:35:03.040
But I think the bigger question is,
link |
00:35:05.040
can you actually solve,
link |
00:35:08.040
or can you actually identify all the corner cases
link |
00:35:11.040
using a single sensing modality
link |
00:35:14.040
and using learning alone?
link |
00:35:16.040
What's your intuition there?
link |
00:35:18.040
So look, if you have a corner case
link |
00:35:20.040
and your algorithm doesn't work,
link |
00:35:22.040
your instinct is to go get data about the corner case
link |
00:35:25.040
and patch it up, learn how to deal with that corner case.
link |
00:35:29.040
But at some point,
link |
00:35:32.040
this is going to saturate, this approach is not viable.
link |
00:35:36.040
So today, computer vision algorithms
link |
00:35:39.040
can detect objects 90% of the time,
link |
00:35:43.040
classify them 90% of the time.
link |
00:35:45.040
Cats on the internet probably can do 95%, I don't know.
link |
00:35:49.040
But to get from 90% to 99%, you need a lot more data.
link |
00:35:54.040
And then I tell you, well, that's not enough
link |
00:35:56.040
because I have a safety critical application
link |
00:35:58.040
that want to go from 99% to 99.9%,
link |
00:36:01.040
well, that's even more data.
link |
00:36:03.040
So I think if you look at
link |
00:36:07.040
wanting accuracy on the x axis
link |
00:36:11.040
and look at the amount of data on the y axis,
link |
00:36:15.040
I believe that curve is an exponential curve.
link |
00:36:18.040
Wow, okay, it's even hard if it's linear.
link |
00:36:21.040
It's hard if it's linear, totally, but I think it's exponential.
link |
00:36:24.040
And the other thing you have to think about
link |
00:36:26.040
is that this process is a very, very power hungry process
link |
00:36:31.040
to run data farms or servers.
link |
00:36:34.040
Power, do you mean literally power?
link |
00:36:36.040
Literally power, literally power.
link |
00:36:38.040
So in 2014, five years ago, and I don't have more recent data,
link |
00:36:43.040
2% of US electricity consumption was from data farms.
link |
00:36:50.040
So we think about this as an information science
link |
00:36:54.040
and information processing problem.
link |
00:36:56.040
Actually, it is an energy processing problem.
link |
00:36:59.040
And so unless we've figured out better ways of doing this,
link |
00:37:02.040
I don't think this is viable.
link |
00:37:04.040
So talking about driving, which is a safety critical application
link |
00:37:08.040
and some aspect of the flight is safety critical,
link |
00:37:11.040
maybe philosophical question, maybe an engineering one.
link |
00:37:14.040
What problem do you think is harder to solve?
link |
00:37:16.040
Autonomous driving or autonomous flight?
link |
00:37:19.040
That's a really interesting question.
link |
00:37:21.040
I think autonomous flight has several advantages
link |
00:37:26.040
that autonomous driving doesn't have.
link |
00:37:30.040
So look, if I want to go from point A to point B,
link |
00:37:33.040
I have a very, very safe trajectory.
link |
00:37:35.040
Go vertically up to a maximum altitude,
link |
00:37:38.040
fly horizontally to just about the destination
link |
00:37:41.040
and then come down vertically.
link |
00:37:43.040
This is preprogrammed.
link |
00:37:46.040
The equivalent of that is very hard to find
link |
00:37:49.040
in a self driving car world because you're on the ground,
link |
00:37:53.040
you're in a two dimensional surface,
link |
00:37:55.040
and the trajectories on the two dimensional surface
link |
00:37:58.040
are more likely to encounter obstacles.
link |
00:38:01.040
I mean this in an intuitive sense,
link |
00:38:03.040
but mathematically true, that's...
link |
00:38:05.040
Mathematically as well, that's true.
link |
00:38:08.040
There's other option on the 2G space of platooning
link |
00:38:11.040
or because there's so many obstacles,
link |
00:38:13.040
you can connect with those obstacles
link |
00:38:15.040
and all these kinds of problems.
link |
00:38:16.040
But those exist in the three dimensional space as well.
link |
00:38:18.040
So they do.
link |
00:38:19.040
So the question also implies how difficult are obstacles
link |
00:38:23.040
in the three dimensional space in flight?
link |
00:38:25.040
So that's the downside.
link |
00:38:27.040
I think in three dimensional space,
link |
00:38:29.040
you're modeling three dimensional world,
link |
00:38:31.040
not just because you want to avoid it,
link |
00:38:33.040
but you want to reason about it
link |
00:38:35.040
and you want to work in that three dimensional environment.
link |
00:38:37.040
And that's significantly harder.
link |
00:38:39.040
So that's one disadvantage.
link |
00:38:41.040
I think the second disadvantage is of course,
link |
00:38:43.040
anytime you fly, you have to put up
link |
00:38:45.040
with the peculiarities of aerodynamics
link |
00:38:49.040
and their complicated environments.
link |
00:38:51.040
How do you negotiate that?
link |
00:38:52.040
So that's always a problem.
link |
00:38:54.040
Do you see a time in the future where there is...
link |
00:38:57.040
You mentioned there's agriculture applications.
link |
00:39:00.040
So there's a lot of applications of flying robots.
link |
00:39:03.040
But do you see a time in the future where there is tens of thousands
link |
00:39:07.040
or maybe hundreds of thousands of delivery drones
link |
00:39:10.040
that fill the sky, a delivery of flying robots?
link |
00:39:14.040
I think there's a lot of potential for the last mile delivery.
link |
00:39:18.040
And so in crowded cities,
link |
00:39:21.040
I don't know if you go to a place like Hong Kong,
link |
00:39:24.040
just crossing the river can take half an hour.
link |
00:39:27.040
And while a drone can just do it in five minutes at most.
link |
00:39:32.040
I think you look at delivery of supplies to remote villages.
link |
00:39:38.040
I work with a nonprofit called Weave Robotics.
link |
00:39:41.040
So they work in the Peruvian Amazon,
link |
00:39:43.040
where the only highways are rivers.
link |
00:39:47.040
And to get from point A to point B
link |
00:39:49.040
may take five hours.
link |
00:39:52.040
While with a drone, you can get there in 30 minutes.
link |
00:39:56.040
So just delivering drugs,
link |
00:39:59.040
retrieving samples for testing vaccines.
link |
00:40:04.040
I think there's huge potential here.
link |
00:40:06.040
So I think the challenges are not technological.
link |
00:40:09.040
The challenge is economical.
link |
00:40:12.040
The one thing I'll tell you that nobody thinks about
link |
00:40:16.040
is the fact that we've not made huge strides in battery technology.
link |
00:40:21.040
Yes, it's true.
link |
00:40:22.040
Batteries are becoming less expensive
link |
00:40:24.040
because we have these mega factories that are coming up.
link |
00:40:27.040
But they're all based on lithium based technologies.
link |
00:40:29.040
And if you look at the energy density and the power density,
link |
00:40:34.040
those are two fundamentally limiting numbers.
link |
00:40:39.040
So power density is important because for a UAV
link |
00:40:41.040
to take off vertically into the air,
link |
00:40:43.040
which most drones do, they don't have a runway,
link |
00:40:47.040
you consume roughly 200 watts per kilo at the small size.
link |
00:40:52.040
That's a lot.
link |
00:40:54.040
In contrast, the human brain consumes less than 80 watts,
link |
00:40:58.040
the whole of the human brain.
link |
00:41:00.040
So just imagine just lifting yourself into the air
link |
00:41:04.040
is like two or three light bulbs, which makes no sense to me.
link |
00:41:08.040
Yeah, so you're going to have to at scale solve the energy problem
link |
00:41:12.040
then charging the batteries, storing the energy and so on.
link |
00:41:19.040
And then the storage is the second problem.
link |
00:41:21.040
But storage limits the range.
link |
00:41:23.040
But you have to remember that you have to burn a lot of it
link |
00:41:30.040
for a given time.
link |
00:41:32.040
So the burning is another problem.
link |
00:41:33.040
Which is a power question.
link |
00:41:35.040
Yes.
link |
00:41:36.040
And do you think just your intuition,
link |
00:41:39.040
there are breakthroughs in batteries on the horizon?
link |
00:41:45.040
How hard is that problem?
link |
00:41:47.040
Look, there are a lot of companies that are promising flying cars,
link |
00:41:52.040
that are autonomous, and that are clean.
link |
00:42:00.040
I think they're over promising.
link |
00:42:02.040
The autonomy piece is doable.
link |
00:42:05.040
The clean piece, I don't think so.
link |
00:42:08.040
There's another company that I work with called Jatatra.
link |
00:42:12.040
They make small jet engines.
link |
00:42:16.040
And they can get up to 50 miles an hour very easily and lift 50 kilos.
link |
00:42:20.040
But they're jet engines.
link |
00:42:22.040
They're efficient.
link |
00:42:24.040
They're a little louder than electric vehicles.
link |
00:42:26.040
But they can build flying cars.
link |
00:42:29.040
So your sense is that there's a lot of pieces that have come together.
link |
00:42:33.040
So on this crazy question, if you look at companies like Kitty Hawk,
link |
00:42:39.040
working on electric, so the clean, talking as the bashing through.
link |
00:42:45.040
It's a crazy dream, but you work with flight a lot.
link |
00:42:52.040
You've mentioned before that manned flights or carrying a human body
link |
00:42:58.040
is very difficult to do.
link |
00:43:01.040
So how crazy is flying cars?
link |
00:43:04.040
Do you think there will be a day when we have vertical takeoff and landing vehicles
link |
00:43:11.040
that are sufficiently affordable that we're going to see a huge amount of them?
link |
00:43:17.040
And they would look like something like we dream of when we think about flying cars.
link |
00:43:21.040
Yeah, like the Jetsons.
link |
00:43:23.040
So look, there are a lot of smart people working on this.
link |
00:43:26.040
And you never say something is not possible when you're people like Sebastian Thrun working on it.
link |
00:43:32.040
So I totally think it's viable.
link |
00:43:35.040
I question, again, the electric piece.
link |
00:43:38.040
The electric piece, yeah.
link |
00:43:40.040
For short distances, you can do it.
link |
00:43:42.040
And there's no reason to suggest that these all just have to be rotor crafts.
link |
00:43:46.040
You take off vertically, but then you morph into a forward flight.
link |
00:43:50.040
I think there are a lot of interesting designs.
link |
00:43:52.040
The question to me is, are these economically viable?
link |
00:43:56.040
And if you agree to do this with fossil fuels, it instantly immediately becomes viable.
link |
00:44:02.040
That's a real challenge.
link |
00:44:04.040
Do you think it's possible for robots and humans to collaborate successfully on tasks?
link |
00:44:09.040
So a lot of robotics folks that I talk to and work with, I mean, humans just add a giant mess to the picture.
link |
00:44:18.040
So it's best to remove them from consideration when solving specific tasks.
link |
00:44:22.040
It's very difficult to model.
link |
00:44:24.040
There's just a source of uncertainty.
link |
00:44:26.040
In your work with these agile flying robots, do you think there's a role for collaboration with humans?
link |
00:44:36.040
Is it best to model tasks in a way that doesn't have a human in the picture?
link |
00:44:43.040
I don't think we should ever think about robots without human in the picture.
link |
00:44:48.040
Ultimately, robots are there because we want them to solve problems for humans.
link |
00:44:54.040
But there's no general solution to this problem.
link |
00:44:58.040
I think if you look at human interaction and how humans interact with robots,
link |
00:45:02.040
you know, we think of these in sort of three different ways.
link |
00:45:06.040
One is the human commanding the robot.
link |
00:45:09.040
The second is the human collaborating with the robot.
link |
00:45:13.040
So for example, we work on how a robot can actually pick up things with a human and carry things.
link |
00:45:19.040
That's like true collaboration.
link |
00:45:21.040
And third, we think about humans as bystanders, self driving cars.
link |
00:45:26.040
What's the human's role and how do self driving cars acknowledge the presence of humans?
link |
00:45:33.040
So I think all of these things are different scenarios.
link |
00:45:36.040
It depends on what kind of humans, what kind of tasks.
link |
00:45:39.040
And I think it's very difficult to say that there's a general theory that we all have for this.
link |
00:45:45.040
But at the same time, it's also silly to say that we should think about robots independent of humans.
link |
00:45:52.040
So to me, human robot interaction is almost a mandatory aspect of everything we do.
link |
00:45:59.040
Yes.
link |
00:46:00.040
But to wish to agree, so your thoughts, if we jump to autonomous vehicles, for example,
link |
00:46:05.040
there's a big debate between what's called level two and level four.
link |
00:46:10.040
So semi autonomous and autonomous vehicles.
link |
00:46:13.040
And sort of the Tesla approach currently at least has a lot of collaboration between human and machine.
link |
00:46:19.040
So the human is supposed to actively supervise the operation of the robot.
link |
00:46:24.040
Part of the safety definition of how safe a robot is in that case is how effective is the human in monitoring it.
link |
00:46:33.040
Do you think that's ultimately not a good approach in sort of having a human in the picture,
link |
00:46:43.040
not as a bystander or part of the infrastructure, but really as part of what's required to make the system safe?
link |
00:46:51.040
This is harder than it sounds.
link |
00:46:53.040
I think, you know, if you, I mean, I'm sure you've driven before in highways and so on,
link |
00:47:01.040
it's really very hard to relinquish controls to a machine and then take over when needed.
link |
00:47:10.040
So I think Tesla's approach is interesting because it allows you to periodically establish some kind of contact with the car.
link |
00:47:18.040
Toyota, on the other hand, is thinking about shared autonomy or collaborative autonomy as a paradigm.
link |
00:47:24.040
If I may argue, these are very, very simple ways of human robot collaboration because the task is pretty boring.
link |
00:47:31.040
You sit in a vehicle, you go from point A to point B.
link |
00:47:34.040
I think the more interesting thing to me is, for example, search and rescue, I've got a human first responder, robot first responders.
link |
00:47:42.040
I got to do something.
link |
00:47:44.040
It's important.
link |
00:47:45.040
I have to do it in two minutes.
link |
00:47:47.040
The building is burning.
link |
00:47:48.040
There's been an explosion.
link |
00:47:50.040
It's collapsed.
link |
00:47:51.040
How do I do it?
link |
00:47:52.040
I think to me, those are the interesting things where it's very, very unstructured and what's the role of the human?
link |
00:47:58.040
What's the role of the robot?
link |
00:47:59.040
Clearly, there's lots of interesting challenges.
link |
00:48:02.040
As a field, I think we're going to make a lot of progress in this area.
link |
00:48:05.040
Yeah, it's an exciting form of collaboration.
link |
00:48:07.040
You're right.
link |
00:48:08.040
In the autonomous driving, the main enemy is just boredom of the human as opposed to the rescue operations.
link |
00:48:15.040
It's literally life and death and the collaboration enables the effective completion of the mission.
link |
00:48:23.040
So it's exciting.
link |
00:48:24.040
Well, in some sense, we're also doing this.
link |
00:48:27.040
You think about the human driving a car and almost invariably the human is trying to estimate the state of the car, the state of the environment, and so on.
link |
00:48:37.040
But what is the car where to estimate the state of the human?
link |
00:48:40.040
So for example, I'm sure you have a smartphone and the smartphone tries to figure out what you're doing and send you reminders.
link |
00:48:47.040
And oftentimes telling you to drive to a certain place, although you have no intention of going there, because it thinks that that's where you should be.
link |
00:48:53.040
Because of some Gmail calendar entry or something like that.
link |
00:48:59.040
And it's trying to constantly figure out who you are, what you're doing.
link |
00:49:02.040
If a car were to do that, maybe that would make the driver safer.
link |
00:49:06.040
Because the car is trying to figure out there's a driver paying attention, looking at his or her eyes, looking at circadian movements.
link |
00:49:14.040
So I think the potential is there.
link |
00:49:16.040
But from the reverse side, it's not robot modeling, but it's human modeling.
link |
00:49:21.040
It's more in the human, right?
link |
00:49:23.040
And I think the robots can do a very good job of modeling humans if you really think about the framework that you have.
link |
00:49:30.040
A human sitting in a cockpit surrounded by sensors, all staring at him, in addition to be staring outside, but also staring at him.
link |
00:49:39.040
I think there's a real synergy there.
link |
00:49:41.040
Yeah, I love that problem because it's the new 21st century form of psychology actually, AI enabled psychology.
link |
00:49:48.040
A lot of people have sci fi inspired fears of walking robots like those from Boston Dynamics.
link |
00:49:54.040
If you just look at shows on Netflix and so on, or flying robots like those you work with.
link |
00:49:59.040
How would you, how do you think about those fears?
link |
00:50:03.040
How would you alleviate those fears?
link |
00:50:05.040
Do you have inklings, echoes of those same concerns?
link |
00:50:09.040
Any time we develop a technology meaning to have positive impact in the world, there's always a worry that somebody could subvert those technologies and use it in an adversarial setting.
link |
00:50:23.040
And robotics is no exception, right?
link |
00:50:25.040
So I think it's very easy to weaponize robots.
link |
00:50:29.040
I think we talk about swarms.
link |
00:50:31.040
One thing I worry a lot about is, for us to get swarms to work and do something reliably, it's really hard.
link |
00:50:38.040
But suppose I have this challenge of trying to destroy something.
link |
00:50:44.040
And I have a swarm of robots where only one out of the swarm needs to get to its destination.
link |
00:50:49.040
So that suddenly becomes a lot more doable.
link |
00:50:53.040
And so I worry about this general idea of using autonomy with lots and lots of agents.
link |
00:51:00.040
I mean, having said that, look, a lot of this technology is not very mature.
link |
00:51:04.040
My favorite saying is that if somebody had to develop this technology, wouldn't you rather the good guys do it?
link |
00:51:12.040
So the good guys have a good understanding of the technology so they can figure out how this technology is being used in a bad way or could be used in a bad way and try to defend against it?
link |
00:51:21.040
So we think a lot about that.
link |
00:51:23.040
So we're doing research on how to defend against swarms, for example.
link |
00:51:28.040
That's interesting.
link |
00:51:29.040
There is, in fact, a report by the National Academies on counter UAS technologies.
link |
00:51:36.040
This is a real threat.
link |
00:51:38.040
But we're also thinking about how to defend against this and knowing how swarms work, knowing how autonomy works is, I think, very important.
link |
00:51:47.040
So it's not just politicians?
link |
00:51:49.040
You think engineers have a role in this discussion?
link |
00:51:51.040
Absolutely.
link |
00:51:52.040
I think the days where politicians can be agnostic to technology are gone.
link |
00:51:59.040
I think every politician needs to be literate in technology.
link |
00:52:05.040
And I often say technology is the new liberal art.
link |
00:52:09.040
Understanding how technology will change your life, I think, is important and every human being needs to understand that.
link |
00:52:18.040
And maybe we can elect some engineers to office as well on the other side.
link |
00:52:22.040
What are the biggest open problems in robotics in UV?
link |
00:52:25.040
You said we're in the early days in some sense.
link |
00:52:27.040
What are the problems we would like to solve in robotics?
link |
00:52:30.040
I think there are lots of problems, right?
link |
00:52:32.040
But I would phrase it in the following way.
link |
00:52:36.040
If you look at the robots we're building, they're still very much tailored towards doing specific tasks in specific settings.
link |
00:52:46.040
I think the question of how do you get them to operate in much broader settings where things can change in unstructured environments is up in the air.
link |
00:52:59.040
So think of the self driving cars.
link |
00:53:02.040
Today, we can build a self driving car in a parking lot.
link |
00:53:05.040
We can do level five autonomy in a parking lot.
link |
00:53:09.040
But can you do a level five autonomy in the streets of Napoli in Italy or Mumbai in India?
link |
00:53:17.040
No.
link |
00:53:18.040
So in some sense, when we think about robotics, we have to think about where they're functioning, what kind of environment, what kind of a task.
link |
00:53:27.040
We have no understanding of how to put both those things together.
link |
00:53:32.040
So we're in the very early days of applying it to the physical world.
link |
00:53:36.040
And I was just in Naples, actually.
link |
00:53:39.040
And there's levels of difficulty and complexity depending on which area you're applying it to.
link |
00:53:46.040
I think so.
link |
00:53:47.040
And we don't have a systematic way of understanding that.
link |
00:53:51.040
Everybody says just because a computer can now beat a human at any board game, we suddenly know something about intelligence.
link |
00:54:00.040
That's not true.
link |
00:54:01.040
A computer board game is very, very structured.
link |
00:54:04.040
It is the equivalent of working in a Henry Ford factory where things, parts come, you assemble, move on.
link |
00:54:11.040
It's a very, very, very structured setting.
link |
00:54:14.040
That's the easiest thing.
link |
00:54:15.040
And we know how to do that.
link |
00:54:18.040
So you've done a lot of incredible work at the UPenn University of Pennsylvania Grass Club.
link |
00:54:23.040
You're now Dean of Engineering at UPenn.
link |
00:54:26.040
What advice do you have for a new bright eyed undergrad interested in robotics or AI or engineering?
link |
00:54:34.040
Well, I think there's really three things.
link |
00:54:37.040
One is you have to get used to the idea that the world will not be the same in five years or four years whenever you graduate, right?
link |
00:54:45.040
Which is really hard to do.
link |
00:54:46.040
So this thing about predicting the future, every one of us needs to be trying to predict the future always.
link |
00:54:53.040
Not because you'll be any good at it, but by thinking about it, I think you sharpen your senses and you become smarter.
link |
00:55:01.040
So that's number one.
link |
00:55:02.040
Number two, it's a callery of the first piece, which is you really don't know what's going to be important.
link |
00:55:09.040
So this idea that I'm going to specialize in something which will allow me to go in a particular direction.
link |
00:55:15.040
It may be interesting, but it's important also to have this breadth so you have this jumping off point.
link |
00:55:22.040
I think the third thing, and this is where I think Penn excels.
link |
00:55:25.040
I mean, we teach engineering, but it's always in the context of the liberal arts.
link |
00:55:30.040
It's always in the context of society.
link |
00:55:32.040
As engineers, we cannot afford to lose sight of that.
link |
00:55:35.040
So I think that's important.
link |
00:55:37.040
But I think one thing that people underestimate when they do robotics is the importance of mathematical foundations,
link |
00:55:43.040
the importance of representations.
link |
00:55:47.040
Not everything can just be solved by looking for ROS packages on the Internet or to find a deep neural network that works.
link |
00:55:56.040
I think the representation question is key, even to machine learning, where if you ever hope to achieve or get to explainable AI,
link |
00:56:06.040
somehow there need to be representations that you can understand.
link |
00:56:09.040
So if you want to do robotics, you should also do mathematics, and you said liberal arts, a little literature.
link |
00:56:16.040
If you want to build a robot, you should be reading Dostoyevsky.
link |
00:56:19.040
I agree with that.
link |
00:56:20.040
Very good.
link |
00:56:21.040
So Vijay, thank you so much for talking today. It was an honor.
link |
00:56:24.040
Thank you. It was just a very exciting conversation. Thank you.