back to indexGavin Miller: Adobe Research | Lex Fridman Podcast #23
link |
The following is a conversation with Gavin Miller, he's the head of Adobe Research.
link |
Adobe has empowered artists, designers, and creative minds from all professions,
link |
working in the digital medium for over 30 years with software such as Photoshop, Illustrator,
link |
Premiere, After Effects, InDesign, Audition, Software that work with images, video, and audio.
link |
Adobe Research is working to define the future evolution of these products in a way
link |
that makes the life of creatives easier, automates the tedious tasks, and gives more and more time
link |
to operate in the idea space instead of pixel space. This is where the cutting edge, deep
link |
learning methods of the past decade can really shine more than perhaps any other application.
link |
Gavin is the embodiment of combining tech and creativity. Outside of Adobe Research,
link |
he writes poetry and builds robots, both things that are near and dear to my heart as well.
link |
This conversation is part of the Artificial Intelligence Podcast. If you enjoy it, subscribe
link |
on YouTube, iTunes, or simply connect with me on Twitter at Lex Friedman's spelled F R I D.
link |
And now here's my conversation with Gavin Miller.
link |
You're head of Adobe Research, leading a lot of innovative efforts and applications of AI,
link |
creating images, video, audio, language, but you're also yourself an artist, a poet,
link |
a writer, and even a roboticist. So while I promise to everyone listening,
link |
that I will not spend the entire time we have together reading your poetry, which I love.
link |
I have to sprinkle it in at least a little bit. So some of them are pretty deep and profound,
link |
and some are light and silly. Let's start with a few lines from the silly variety.
link |
You write in a beautiful parody, both Edith Piaf's and my web at Frank Sinatra.
link |
So it opens with, and now dessert is near. It's time to pay the final total. I've tried to slim
link |
all year, but my diets have been anecdotal. So where does that love for poetry come from
link |
for you? And if we dissect your mind, how does it all fit together in the bigger puzzle of Dr.
link |
Gavin Miller? Well, interesting you chose that one. That was a poem I wrote when I'd been to
link |
my doctor and he said, you really need to lose some weight and go on a diet. And whilst the
link |
rational part of my brain wanted to do that, the irrational part of my brain was protesting and
link |
sort of embraced the opposite idea. I regret nothing, hence. Yes, exactly. Taken to an extreme,
link |
I thought it would be funny. Obviously, it's a serious topic for some people. But I think,
link |
for me, I've always been interested in writing since I was in high school, as well as doing
link |
technology and invention. And sometimes the parallel strands in your life that carry on,
link |
and one is more about your private life and one's more about your technological career.
link |
And then at sort of happy moments along the way, sometimes the two things touch, one idea informs
link |
the other. And we can talk about that as we go. Do you think you're writing the art, the poetry
link |
contribute indirectly or directly to your research, to your work in Adobe? Well, sometimes it does if
link |
I say, imagine a future in a science fiction kind of way. And then once it exists on paper,
link |
I think, well, why shouldn't I just build that? There was an example where when realistic voice
link |
synthesis first started in the 90s at Apple, where I worked in research. I was done by a friend of mine.
link |
I sort of sat down and started writing a poem which each line I would enter into the voice
link |
synthesizer and see how it sounded and sort of wrote it for that voice. And at the time,
link |
the agents weren't very sophisticated. So they'd sort of add random intonation. And I kind of made
link |
up the poem to sort of match the tone of the voice. And it sounded slightly sad and depressed. So I
link |
pretended it was a poem written by an intelligent agent, sort of telling the user to go home and
link |
leave them alone. But at the same time, they were lonely and wanted to have company and learn from
link |
what the user was saying. And at the time, it was way beyond anything that AI could possibly do.
link |
But, you know, since then, it's becoming more within the bounds of possibility.
link |
And then at the same time, I had a project at home where I did sort of a smart home. This was
link |
probably 93, 94. And I had the talking voice who'd remind me when I walked in the door of what
link |
things I had to do. I had buttons on my washing machine because I was a bachelor and I'd leave
link |
the clothes in there for three days and they'd go moldy. So as I got up in the morning, I would say,
link |
don't forget the washing and so on. I made photographic photo albums that used light
link |
sensors to know which page you were looking at would send that over wireless radio to the agent
link |
who would then play sounds that matched the image she were looking at in the book. So I was kind of
link |
in love with this idea of magical realism and whether it was possible to do that with technology.
link |
So that was a case where the sort of the agent sort of intrigued me from a literary point of
link |
view and became a personality. I think more recently, I've also written plays and when
link |
plays you write dialogue and obviously you write a fixed set of dialogue that follows a linear
link |
narrative. But with modern agents, as you design a personality or a capability for conversation,
link |
you're sort of thinking of, I kind of have imaginary dialogue in my head. And then I think,
link |
what would it take not only to have that be real, but for it to really know what it's talking about.
link |
So it's easy to fall into the uncanny valley with AI where it says something it doesn't really
link |
understand, but it sounds good to the person. But you rapidly realize that it's kind of just
link |
stimulus response. It doesn't really have real world knowledge about the thing it's describing.
link |
And so when you get to that point, it really needs to have multiple ways of talking about
link |
the same concept. So it sounds as though it really understands it. Now, what really understanding
link |
means is in the eye of the beholder, right? But if it only has one way of referring to something,
link |
it feels like it's a canned response. But if it can reason about it, or you can go at it from
link |
multiple angles and give a similar kind of response that people would, then it starts to
link |
seem more like there's something there that's sentient.
link |
You can say the same thing, multiple things from different perspectives. I mean, with the
link |
automatic image captioning that I've seen the work that you're doing, there's elements of that,
link |
right? Being able to generate different kinds of... Right. So one in my team, there's a lot of work on
link |
turning a medium from one form to another, whether it's auto tagging imagery or making up full
link |
sentences about what's in the image, then changing the sentence, finding another image that matches
link |
the new sentence or vice versa. And in the modern world of GANs, you sort of give it a description
link |
and it synthesizes an asset that matches the description. So I've sort of gone on a journey.
link |
My early days in my career were about 3D computer graphics, the sort of pioneering work sort of
link |
before movies had special effects done with 3D graphics and sort of rode that revolution. And
link |
that was very much like the renaissance where people would model light and color and shape
link |
and everything. And now we're kind of in another wave where it's more impressionistic and it's
link |
sort of the idea of something can be used to generate an image directly, which is sort of the
link |
new frontier in computer image generation using AI algorithms. So the creative process is more in
link |
the space of ideas or becoming more in the space of ideas versus in the raw pixels?
link |
Well, it's interesting. It depends. I think at Adobe, we really want to span the entire range
link |
from really, really good, what you might call low level tools by low level, as close to say analog
link |
workflows as possible. So what we do there is we make up systems that do really realistic oil
link |
paint and watercolor simulation. So if you want every bristle to behave as it would in the real
link |
world and leave a beautiful analog trail of water and then flow after you've made the brushstroke,
link |
you can do that. And that's really important for people who want to create something
link |
really expressive or really novel because they have complete control. And then a certain other
link |
task become automated. It frees the artists up to focus on the inspiration and less of the perspiration.
link |
So thinking about different ideas, obviously, once you finish the design, there's a lot of work to
link |
say do it for all the different aspect ratio of phones or websites and so on. And that used to
link |
take up an awful lot of time for artists. It still does for many, what we call content velocity.
link |
And one of the targets of AI is actually to reason about from the first example of what are the
link |
likely intent for these other formats, maybe if you change the language to German and the words
link |
are longer, how do you reflow everything so that it looks nicely artistic in that way.
link |
And so the person can focus on the really creative bit in the middle, which is what is the look and
link |
style and feel and what's the message and what's the story and the human element.
link |
So I think creativity is changing. So that's one way in which we're trying to just make it easier
link |
and faster and cheaper to do so that there can be more of it, more demand, because it's less
link |
expensive. So everyone wants beautiful artwork for everything from a school website to Hollywood movie.
link |
On the other side, as some of these things have automatic versions of them, people will
link |
possibly change role from being the hands on artist and to being either the art director or
link |
the conceptual artist. And then the computer will be a partner to help create polished examples of
link |
the idea that they're exploring. Let's talk about Adobe products versus AI and Adobe products.
link |
Just so you know where I'm coming from, I'm a huge fan of Photoshop for images premiere for video,
link |
audition for audio. I'll probably use Photoshop to create the thumbnail for this video, premiere
link |
to edit the video, audition to do the audio. That said, everything I do is really manually. And I
link |
set up, I use this old school kinesis keyboard and I have auto hotkey that just it's really about
link |
optimizing the flow of just making sure there's as few clicks as possible. So just being extremely
link |
efficient. It's something you started to speak to. So before we get into the fun, sort of awesome
link |
deep learning things, where does AI, if you could speak a little more to it AI or just
link |
automation in general, do you see in the coming months and years or in general prior in 2018
link |
fitting into making the life, the low level pixel work flow easier?
link |
Yeah, that's a great question. So we have a very rich array of algorithms already in Photoshop,
link |
just classical procedural algorithms as well as ones based on data. In some cases, they end up
link |
with a large number of sliders and degrees of freedom. So one way in which AI can help is just
link |
an auto button which comes up with default settings based on the content itself rather than
link |
default values for the tool. At that point, you then start tweaking. So that's that's a very kind of
link |
make life easier for people whilst making use of common sense from other example images.
link |
So like smart defaults.
link |
Smart defaults, absolutely. Another one is something we've spent a lot of work over the last
link |
20 years. I've been at Adobe 19 thinking about selection, for instance, where
link |
you know, with a quick select, you would look at color boundaries and figure out how to sort of
link |
flood fill into regions that you thought were physically connected in the real world.
link |
But that algorithm had no visual common sense about what a cat looks like or a dog. It would just do
link |
it based on rules of thumb, which were applied to graph theory. And it was a big improvement over
link |
the previous work we had sort of almost click every everything by hand or if it just did similar
link |
colors, it would do little tiny regions that wouldn't be connected. But in the future,
link |
using neural nets to actually do a great job with say a single click or even in the case of
link |
well known categories like people or animals, no click, where you just say select the object and
link |
it just knows the dominant object as a person in the middle of the photograph. Those kinds of things
link |
are really valuable if they can be robust enough to give you good quality results.
link |
Or they can be a great start for like tweaking it.
link |
So for example, background removal, like one thing I'll, in a thumbnail,
link |
I'll take a picture of you right now and essentially remove the background behind you.
link |
And I want to make that as easy as possible. You don't have flowing hair, like rich at the
link |
moment. Rich sort of. I had it in the past, it may come again in the future, but for now.
link |
So that sometimes makes it a little more challenging to remove the background.
link |
How difficult do you think is that problem for AI for basically making the quick selection tool
link |
smarter and smarter and smarter? Well, we have a lot of research on that already.
link |
If you want a sort of quick, cheap and cheerful, look, I'm pretending I'm in Hawaii,
link |
but it's sort of a joke, then you don't need perfect boundaries. And you can do that today
link |
with a single click for the algorithms we have. We have other algorithms where with a little bit
link |
more guidance on the boundaries, like you might need to touch it up a little bit.
link |
We have other algorithms that can pull a nice mat from a crude selection. So we have combinations
link |
of tools that can do all of that. And at our recent Max conference at AB Max, we demonstrated how
link |
very quickly just by drawing a simple polygon around the object of interest, we could not
link |
only do it for a single still, but we could pull at least a selection mask from a moving target,
link |
like a person dancing in front of a brick wall or something. And so it's going from hours to
link |
a few seconds for workflows that are really nice. And then you might go in and touch up a little.
link |
So that's a really interesting question. You mentioned the word robust.
link |
You know, there's like a journey for an idea, right? And what you presented probably at Max
link |
has elements of just sort of it inspires the concept, it can work pretty well in a majority
link |
of cases. But how do you make something that works? Well, in majority of cases, how do you make
link |
something that works, maybe in all cases, or it becomes a robust tool?
link |
There are a couple of things. So that really touches on the difference between academic research
link |
and industrial research. So in academic research, it's really about who's the person to have the
link |
great new idea that shows promise. And we certainly love to be those people too. But
link |
we have sort of two forms of publishing. One is academic peer review, which we do a lot of,
link |
and we have great success there as much as some universities. But then we also have shipping,
link |
which is a different type of, and then we get customer review, as well as, you know, product
link |
critics. And that might be a case where it's not about being perfect every single time, but
link |
perfect enough at the time, plus a mechanism to intervene and recover where you do have mistakes.
link |
So we have the luxury of very talented customers. We don't want them to be
link |
overly taxed doing it every time. But if they can go in and just take it from 99 to 100,
link |
100 with the touch of a mouse or something, then for the professional end, that's something
link |
that we definitely want to support as well. And for them, it went from having to do that
link |
tedious task all the time to much less often. So I think that gives us an out. If it had to be
link |
100% automatic all the time, then that would delay the time at which we could get to market.
link |
So on that thread, maybe you can untangle something. Again, I'm sort of just speaking to
link |
my own experience. Maybe that is the most useful idea. So I think Photoshop, as an example or premiere,
link |
has a lot of amazing features that I haven't touched. And so what's the, in terms of AI,
link |
helping make my life or the life of creatives easier? How this collaboration between human
link |
and machine, how do you learn to collaborate better? How do you learn the new algorithms?
link |
Is it something that where you have to watch tutorials and you have to watch videos and so
link |
on? Or do you ever think, do you think about the experience itself through exploration being
link |
the teacher? We absolutely do. So I'm glad that you brought this up. We sort of think about
link |
two things. One is helping the person in the moment to do the task that they need to do. But
link |
the other is thinking more holistically about their journey learning a tool. And when it's like,
link |
think of it as Adobe University, where you use the tool long enough, you become an expert.
link |
And not necessarily an expert in everything. It's like living in a city. You don't necessarily
link |
know every street, but you know, the important ones you need to get to. So we have projects in
link |
research, which actually look at the thousands of hours of tutorials online and try to understand
link |
what's being taught in them. And then we had one publication at CHI where it was looking at,
link |
given the last three or four actions you did, what did other people in tutorials do next?
link |
So if you want some inspiration for what you might do next, or you just want to watch the
link |
tutorial and see, learn from people who are doing similar workflows to you, you can without having
link |
to go and search on keywords and everything. So really trying to use the context of your use of
link |
the app to make intelligent suggestions, either about choices that you might make,
link |
or in a more assistive way where it could say, if you did this next, we could show you. And that's
link |
basically the frontier that we're exploring now, which is, if we really deeply understand the
link |
domain in which designers and creative people work, can we combine that with AI and pattern
link |
matching of behavior to make intelligent suggestions, either through verbal possibilities or just
link |
showing the results of if you try this. And that's really the sort of, I was in a meeting today
link |
thinking about these things. So it's still a grand challenge. We'd all love
link |
an artist over one shoulder and a teacher over the other, right? And we hope to get there. And
link |
the right thing to do is to give enough at each stage that it's useful in itself, but it builds
link |
a foundation for the next level of expectation. Are you aware of this gigantic medium of YouTube
link |
that's creating just a bunch of creative people, both artists and teachers of different kinds?
link |
Absolutely. And the more we can understand those media types, both visually and in terms of
link |
transcripts and words, the more we can bring the wisdom that they embody into the guidance that's
link |
embedded in the tool. That would be brilliant to remove the barrier from having to yourself type
link |
in the keyword, searching, so on. Absolutely. And then in the longer term, an interesting
link |
discussion is, does it ultimately not just assist with learning the interface we have,
link |
but does it modify the interface to be simpler? Or do you fragment into a variety of tools,
link |
each of which has a different level of visibility of the functionality? I like to say that if you
link |
add a feature to a GUI, you have to have yet more visual complexity confronting the new user.
link |
Whereas if you have an assistant with a new skill, if you know they have it, so you know
link |
to ask for it, then it's sort of additive without being more intimidating. So we definitely think
link |
about new users and how to onboard them. Many actually value the idea of being able to master
link |
that complex interface and keyboard shortcuts, like you were talking about earlier, because
link |
with great familiarity, it becomes a musical instrument for expressing your visual ideas.
link |
And other people just want to get something done quickly in the simplest way possible,
link |
and that's where a more assistive version of the same technology might be useful,
link |
maybe on a different class of device, which is more in context for capture, say,
link |
whereas somebody who's in a deep post production workflow maybe want to be on a laptop or a big
link |
screen desktop and have more knobs and dials to really express the subtlety of what they want to do.
link |
So there's so many exciting applications of computer vision and machine learning
link |
that Adobe is working on, like scene stitching, sky replacement, foreground,
link |
background removal, spatial object based image search, automatic image captioning,
link |
like we mentioned, project cloak, project deep fill filling in parts of the images,
link |
project scribbler, style transfer video, style transfer faces and video with Project Puppetron,
link |
best name ever. Can you talk through a favorite or some of them or examples that pop in mind?
link |
I'm sure I'll be able to provide links to other ones we don't talk about because there's visual
link |
elements to all of them that are exciting. Why they're interesting for different reasons might
link |
be a good way to go. So I think sky replace is interesting because we talked about selection
link |
being sort of an atomic operation. It's almost like a, if you think of an assembly language,
link |
it's like a single instruction. Whereas sky replace is a compound action where you automatically
link |
select the sky, you look for stock content that matches the geometry of the scene.
link |
You try to have variety in your choices so that you do coverage of different moods.
link |
It then mats in the sky behind the foreground, but then importantly it uses the foreground
link |
of the other image that you just searched on to recolor the foreground of the image that
link |
you're editing. So if you say go from a midday sky to an evening sky, it will actually add
link |
sort of an orange glow to the foreground objects as well. I was a big fan in college of Magritte
link |
and he has a number of paintings where it's surrealism because he'll like do a composite,
link |
but the foreground building will be at night and the sky will be during the day. There's one
link |
called The Empire of Light which was on my wall in college and we're trying not to do surrealism.
link |
It can be a choice, but we'd rather have it be natural by default rather than it looking fake
link |
and then you have to do a whole bunch of post production to fix it. So that's a case where
link |
we're kind of capturing an entire workflow into a single action and doing it in about a second
link |
rather than a minute or two. And when you do that, you can not just do it once, but you can do it
link |
for say like 10 different backgrounds and then you're almost back to this inspiration idea of
link |
I don't know quite what I want, but I'll know it when I see it. And you can just explore the
link |
design space as close to final production value as possible. And then when you really pick one,
link |
you might go back and slightly tweak the selection mask just to make it perfect and
link |
do that kind of polish that professionals like to bring to their work.
link |
So then there's this idea of, as you mentioned, the sky replacing it to different stock images of
link |
the sky. In general, you have this idea or it could be on your disk or whatever. But making even
link |
more intelligent choices about ways to search stock images which is really interesting. It's
link |
kind of spatial. Absolutely. Right. So that was something we called Concept Canvas. So normally
link |
when you do say an image search, I assume it's just based on text. You would give the keywords
link |
of the things you want to be in the image and it would find the nearest one that had those tags.
link |
For many tasks, you really want to be able to say I want a big person in the middle or in a
link |
dog to the right and umbrella above the left because you want to leave space for the text or
link |
whatever. And so Concept Canvas lets you assign spatial regions to the keywords. And then we've
link |
already preindexed the images to know where the important concepts are in the picture. So we then
link |
go through that index matching to assets. And even though it's just another form of search,
link |
because you're doing spatial design or layout, it starts to feel like design. You sort of feel
link |
oddly responsible for the image that comes back as if you invented it a little bit. So it's a good
link |
example where giving enough control starts to make people have a sense of ownership over the
link |
outcome of the event. And then we also have technologies in Photoshop where you physically
link |
can move the dog in post as well. But for Concept Canvas, it was just a very fast way to sort of
link |
loop through and be able to lay things out. In terms of being able to remove objects from a scene
link |
and fill in the background automatically. So that's extremely exciting. And that's
link |
so neural networks are stepping in there. I just talked this week with Ian Goodfellow.
link |
So the GANS for doing that is definitely one approach.
link |
So is that a really difficult problem? Is it as difficult as it looks,
link |
again, to take it to a robust product level? Well, there are certain classes of image for
link |
which the traditional algorithms like Content Aware Fill work really well. Like if you have
link |
a naturalistic texture like a gravel path or something, because it's patch based, it will
link |
make up a very plausible looking intermediate thing and fill in the hole. And then we use some
link |
algorithms to sort of smooth out the lighting so you don't see any brightness contrasts in that
link |
region or you've gradually ramped from dark to light if it straddles the boundary.
link |
Where it gets complicated is if you have to infer invisible structure behind the person in front.
link |
And that really requires a common sense knowledge of the world to know what,
link |
if I see three quarters of a house, do I have a rough sense of what the rest of the house looks
link |
like? If you just fill it in with patches, it can end up sort of doing things that make sense
link |
locally. But you look at the global structure and it looks like it's just sort of crumpled or messed
link |
up. And so what GANs and neural nets bring to the table is this common sense learned from the
link |
training set. And the challenge right now is that the generative methods that can make up
link |
missing holes using that kind of technology are still only stable at low resolutions.
link |
And so you either need to then go from a low resolution to a high resolution using some other
link |
algorithm or we need to push the state of the art and it's still in research to get to that point.
link |
Right. Of course, if you show it something, say it's trained on houses and then you show it in
link |
octopus, it's not going to do a very good job of showing common sense about octopuses. So
link |
again, you're asking about how you know that it's ready for prime time. You really need a very
link |
diverse training set of images. And ultimately, that may be a case where you put it out there
link |
with some guard rails where you might do a detector which looks at the image and
link |
sort of estimates its own competence of how well a job could this algorithm do.
link |
So eventually, there may be this idea of what we call an ensemble of experts where
link |
any particular expert is specialized in certain things and then there's sort of a
link |
either they vote to say how confident they are about what to do. This is sort of more future
link |
looking or there's some dispatcher which says you're good at houses, you're good at trees.
link |
So I mean, all this adds up to a lot of work because each of those models will be a whole
link |
bunch of work. But I think over time, you'd gradually fill out the set and initially focus
link |
on certain workflows and then sort of branch out as you get more capable.
link |
So you mentioned workflows and have you considered maybe looking far into the future?
link |
First of all, using the fact that there is a huge amount of people that use Photoshop,
link |
for example, they have certain workflows, being able to collect the information by which
link |
they basically get information about their workflows, about what they need,
link |
what can the ways to help them, whether it is houses or octopus that people work on more.
link |
Basically getting a beat on what kind of data is needed to be annotated and collected for people
link |
to build tools that actually work well for people.
link |
Absolutely. And this is a big topic and the whole world of AI is what data can you gather and why.
link |
At one level, the way to think about it is we not only want to train our customers in how to use
link |
our products, but we want them to teach us what's important and what's useful. At the same time,
link |
we want to respect their privacy and obviously we wouldn't do things without their explicit permission.
link |
And I think the modern spirit of the age around this is you have to demonstrate to somebody
link |
how they're benefiting from sharing their data with the tool. Either it's helping in the short
link |
term to understand their intent so you can make better recommendations or if they're
link |
there friendly to your cause or you're tall or they want to help you evolve quickly because
link |
they depend on you for their livelihood, they may be willing to share some of their
link |
workflows or choices with the dataset to be then trained.
link |
There are technologies for looking at learning without necessarily storing all the information
link |
permanently so that you can learn on the fly but not keep a record of what somebody did.
link |
So, we're definitely exploring all of those possibilities.
link |
And I think Adobe exists in a space where Photoshop, if I look at the data I've created
link |
in OWN, I'm less comfortable sharing data with social networks than I am with Adobe because
link |
there's just exactly as you said, there's an obvious benefit for sharing the data that I use
link |
to create in Photoshop because it's helping improve the workflow in the future.
link |
As opposed to it's not clear what the benefit is in social networks.
link |
It's nice of you to say that. I mean, I think there are some professional workflows where
link |
people might be very protective of what they're doing such as if I was preparing
link |
evidence for a legal case, I wouldn't want any of that, you know,
link |
phoning home to help train the algorithm or anything.
link |
There may be other cases where people say having a trial version or they're doing some,
link |
I'm not saying we're doing this today, but there's a future scenario where somebody has a more
link |
permissive relationship with Adobe where they explicitly say, I'm fine, I'm only doing hobby
link |
projects or things which are non confidential and in exchange for some benefit tangible or
link |
otherwise, I'm willing to share very fine grain data.
link |
So, another possible scenario is to capture relatively crude high level things from more
link |
people and then more detailed knowledge from people who are willing to participate.
link |
We do that today with explicit customer studies where, you know, we go and visit somebody and
link |
ask them to try the tool and we human observe what they're doing.
link |
In the future, to be able to do that enough to be able to train an algorithm,
link |
we'd need a more systematic process, but we'd have to do it very consciously because
link |
one of the things people treasure about Adobe is a sense of trust
link |
and we don't want to endanger that through overly aggressive data collection.
link |
So, we have a Chief Privacy Officer and it's definitely front and center of thinking about AI
link |
rather than an afterthought. Well, when you start that program, sign me up.
link |
Is there other projects that you wanted to mention that I didn't perhaps that pop into mind?
link |
Well, you covered the number. I think you mentioned Project Puppetron.
link |
I think that one is interesting because you might think of Adobe as only thinking in 2D
link |
and that's a good example where we're actually thinking more three dimensionally about how to
link |
assign features to faces so that we can, you know, if you take, so what Puppetron does,
link |
it takes either a still or a video of a person talking and then it can take a painting of somebody
link |
else and then apply the style of the painting to the person who's talking in the video.
link |
And it's unlike a sort of screen door post filter effect that you sometimes see online.
link |
It really looks as though it's sort of somehow attached or reflecting the motion of the face.
link |
And so that's the case where even to do a 2D workflow like stylization,
link |
you really need to infer more about the 3D structure of the world.
link |
And I think as 3D computer vision algorithms get better,
link |
initially they'll focus on particular domains like faces where you have a lot of
link |
prior knowledge about structure and you can maybe have a parameterized template that you fit to the
link |
image. But over time, this should be possible for more general content. And it might even be
link |
invisible to the user that you're doing 3D reconstruction but under the hood, but it might
link |
then let you do edits much more reliably or correctly than you would otherwise.
link |
And you know, the face is a very important application, right?
link |
So making things work.
link |
And a very sensitive one. If you do something uncanny, it's very disturbing.
link |
That's right. You have to get it. You have to get it right.
link |
So in the space of augmented reality and virtual reality, what do you think is the role of AR and
link |
VR and in the content we consume as people as consumers and the content we create as creators?
link |
No, that's a great question. Let me think about this a lot too.
link |
So I think VR and AR serve slightly different purposes. So VR can really transport you to an
link |
entire immersive world, no matter what your personal situation is. To that extent, it's a bit like
link |
a really, really widescreen television where it sort of snaps you out of your context and puts you
link |
in a new one. And I think it's still evolving in terms of the hardware I actually worked on,
link |
VR in the 90s, trying to solve the latency and sort of nausea problem, which we did,
link |
but it was very expensive and a bit early. There's a new wave of that now. I think
link |
and increasingly those devices are becoming all in one rather than something that's tethered to a
link |
box. I think the market seems to be bifurcating into things for consumers and things for professional
link |
use cases, like for architects and people designing where your product is a building and you really
link |
want to experience it better than looking at a scale model or a drawing, I think,
link |
or even than a video. So I think for that, where you need a sense of scale and spatial
link |
relationships, it's great. I think AR holds the promise of sort of taking digital assets off the
link |
screen and putting them in context in the real world on the table in front of you on the wall
link |
behind you. And that has the corresponding need that the assets need to adapt to the physical
link |
context in which they're being placed. I mean, it's a bit like having a live theater troupe come
link |
to your house and put on Hamlet. My mother had a friend who used to do this at Stately Homes in
link |
England for the National Trust. And they would adapt the scenes and even they'd walk the audience
link |
through the rooms to see the action based on the country house they found themselves in for two
link |
days. And I think AR will have the same issue that if you have a tiny table in a big living room
link |
or something, it'll try to figure out what can you change and what's fixed. And there's a little
link |
bit of a tension between fidelity, where if you captured Senior Eye of doing a fantastic ballet,
link |
you'd want it to be sort of exactly reproduced. And maybe all you could do is scale it down.
link |
Whereas somebody telling you a story might be walking around the room doing some gestures
link |
and that could adapt to the room in which they were telling the story.
link |
And do you think fidelity is that important in that space or is it more about the storytelling?
link |
I think it may depend on the characteristic of the media. If it's a famous celebrity,
link |
then it may be that you want to catch every nuance and they don't want to be reanimated by some
link |
algorithm. It could be that if it's really a loveable frog telling you a story and it's
link |
about a princess and a frog, then it doesn't matter if the frog moves in a different way.
link |
I think a lot of the ideas that have sort of grown up in the game world will
link |
now come into the broader commercial sphere once they're needing adaptive characters in AR.
link |
Are you thinking of engineering tools that allow creators to create in the augmented world,
link |
basically making a Photoshop for the augmented world?
link |
Well, we have shown a few demos of sort of taking a Photoshop layer stack and then expanding it into
link |
3D. That's actually been shown publicly as one example in AR. Where we're particularly excited
link |
at the moment is in 3D. 3D design is still a very challenging space. We believe that it's
link |
a worthwhile experiment to try to figure out if AR or immersive makes 3D design more spontaneous.
link |
Can you give me an example of 3D design just like applications?
link |
Well, literally, a simple one would be laying out objects, right? On a conventional screen,
link |
you'd sort of have a plan view and a side view and a perspective view and you sort of be dragging
link |
it around with the mouse and if you're not careful, it would go through the wall and all that.
link |
Whereas if you were really laying out objects, say in a VR headset, you could literally move
link |
your head to see a different viewpoint. They'd be in stereo, so you'd have a sense of depth
link |
because you're already wearing the depth glasses, right? So it would be those sort of big gross
link |
motor, move things around, kind of skills seem much more spontaneous just like they are in the
link |
real world. The frontier for us, I think, is whether that same medium can be used to do fine
link |
grain design tasks, like very accurate constraints on, say, a CAD model or something. That may be
link |
better done on a desktop, but it may just be a matter of inventing the right UI.
link |
So we're hopeful that because there will be this potential explosion of demand for 3D assets
link |
that's driven by AR and more real time animation on conventional screens,
link |
those tools will also help with, or those devices will help with designing the content as well.
link |
You've mentioned quite a few interesting sort of new ideas. At the same time, there's old
link |
timers like me that are stuck in their old ways. I think I'm the old timer.
link |
Okay. All right. But the opposed all change at all costs. When you're thinking about
link |
creating new interfaces, do you feel the burden of just this giant user base that loves the
link |
current product? So anything new you do that any new idea comes at a cost that you'll be resisted?
link |
Well, I think if you have to trade off control for convenience, then our existing user base
link |
would definitely be offended by that. I think if there are some things where you have more convenience
link |
and just as much control, that may be more welcome. We do think about not breaking well known
link |
metaphors for things. So things should sort of make sense. Photoshop has never been a static
link |
target. It's always been evolving and growing. And to some extent, there's been a lot of brilliant
link |
thought along the way of how it works today. So we don't want to just throw all that out.
link |
If there's a fundamental breakthrough, like a single click is good enough to select an object
link |
rather than having to do lots of strokes, that actually fits in quite nicely to the existing
link |
tool set, either as an optional mode or as a starting point. I think where we're looking at
link |
radical simplicity where you could encapsulate an entire workflow with a much simpler UI,
link |
then sometimes that's easier to do in the context of either a different device like a
link |
mobile device where the affordances are naturally different or in a tool that's targeted at a
link |
different workflow where it's about spontaneity and velocity rather than precision. And we have
link |
projects like Rush, which can let you do professional quality video editing for a certain class of
link |
media output that is targeted very differently in terms of users and the experience. And ideally,
link |
people would go, if I'm feeling like doing Premiere, big project, I'm doing a four part
link |
television series. That's definitely a premier thing. But if I want to do something to show my
link |
recent vacation, maybe I'll just use Rush because I can do it in the half an hour. I have free at
link |
home rather than the four hours I need to do it at work. And for the use cases which we can do well,
link |
it really is much faster to get the same output. But the more professional tools obviously have
link |
a much richer toolkit and more flexibility in what they can do.
link |
And then at the same time, with the flexibility and control, I like this idea of smart defaults,
link |
of using AI to coach you to like what Google has, I'm feeling lucky button.
link |
Right. Or one button kind of gives you a pretty good
link |
set of settings. And then you almost, that's almost an educational tool.
link |
To show, because sometimes when you have all this control, you're not sure about the
link |
correlation between the different bars that control different elements of the image and so on.
link |
And sometimes there's a degree of, you don't know what the optimal is.
link |
And then some things are sort of on demand like help, right?
link |
I'm stuck. I need to know what to look for. I'm not quite sure what it's called.
link |
And something that was proactively making helpful suggestions or,
link |
you know, you could imagine a make a suggestion button where you'd use all of that knowledge
link |
of workflows and everything to maybe suggest something to go and learn about or just to try
link |
or show the answer. And maybe it's not one intelligent to pothole, but it's like a variety
link |
of defaults. And then you go, oh, I like that one.
link |
So back to poetry.
link |
We're going to interleave.
link |
So first few lines of a recent poem of yours before I ask the next question.
link |
Yeah. This is about the smartphone. Today left my phone at home and went down to the sea.
link |
The sand was soft, the ocean glass, but I was still just me.
link |
So this is a poem about you leaving your phone behind and feeling quite liberated because of it.
link |
So this is kind of a difficult topic and let's see if we can talk about it, figure it out.
link |
But so with the help of AI, more and more, we can create versions of ourselves, versions of
link |
reality that are in some ways more beautiful than actual reality. And some of the creative effort
link |
there is part of creating this illusion. So of course, this is inevitable, but how do you think
link |
we should adjust this human beings to live in this digital world that's partly artificial,
link |
that's better than the world that we lived in a hundred years ago when you didn't have
link |
Instagram and Facebook versions of ourselves and the online.
link |
Oh, this is sort of showing off better versions of ourselves.
link |
We're using the tooling of modifying the images or even with artificial intelligence
link |
ideas of deep fakes and creating adjusted or fake versions of ourselves and reality.
link |
I think it's an interesting question. You're all sort of historical bent on this.
link |
I actually wonder if 18th century aristocrats who commissioned famous painters to paint portraits
link |
of them had portraits that were slightly nicer than they actually looked in practice.
link |
So human desire to put your best foot forward has always been true.
link |
I think it's interesting. You sort of framed it in two ways. One is if we can imagine alternate
link |
realities and visualize them, is that a good or bad thing? In the old days, you do it with
link |
storytelling and words and poetry, which still resides sometimes on websites. But
link |
we've become a very visual culture in particular. In the 19th century, we were very much a text
link |
based culture. People would read long tracks. Political speeches were very long. Nowadays,
link |
everything's very kind of quick and visual and snappy. I think it depends on how harmless your
link |
intent. A lot of it's about intent. So if you have a somewhat flattering photo that you pick
link |
out of the photos that you have in your inbox to say, this is what I look like, it's probably fine.
link |
If someone's going to judge you by how you look, then they'll decide soon enough when they meet
link |
you whether the reality. I think where it can be harmful is if people hold themselves up to an
link |
impossible standard, which they then feel bad about themselves for not meeting. I think that's
link |
definitely can be an issue. But I think the ability to imagine and visualize an alternate
link |
reality, which sometimes which you then go off and build later, can be a wonderful thing too.
link |
People can imagine architectural styles, which they then have a startup, make a fortune and then
link |
build a house that looks like their favorite video game. Is that a terrible thing? I think
link |
I used to worry about exploration actually, that part of the joy of going to the moon
link |
when I was a tiny child, I remember it, and grainy black and white, was to know what it would look
link |
like when you got there. And I think now we have such good graphics for knowing, for visualizing
link |
experience before it happens, that I slightly worry that it may take the edge off actually wanting
link |
to go. Because we've seen it on TV, we kind of, oh, by the time we finally get to Mars,
link |
we're going, oh yeah, it's Mars, that's what it looks like. But then the outer exploration,
link |
I mean, I think Pluto was a fantastic recent discovery where nobody had any idea what it
link |
looked like and it was just breathtakingly varied and beautiful. So I think expanding
link |
the ability of the human toolkit to imagine and communicate on balance is a good thing.
link |
I think there are abuses, we definitely take them seriously and try to discourage them.
link |
I think there's a parallel side where the public needs to know what's possible through events like
link |
this, right? So that you don't believe everything you read and print anymore, and it may over time
link |
become true of images as well. Or you need multiple sets of evidence to really believe
link |
something rather than a single media asset. So I think it's a constantly evolving thing.
link |
It's been true forever. There's a famous story about Anne of Cleves and Henry VIII where,
link |
luckily for Anne, they didn't get married, right? So, or they got married and
link |
Oh, so Holbein went and painted a picture and then Henry VIII wasn't pleased and, you know,
link |
history doesn't record whether Anne was pleased, but I think she was pleased not
link |
to be married more than a day or something. So I mean, this has gone on for a long time,
link |
but I think it's just part of the magnification of human capability.
link |
You've kind of built up an amazing research environment here, research culture, research
link |
lab, and you've written that the secret to a thriving research lab is interns. Can you unpack
link |
that a little bit? Oh, absolutely. So a couple of reasons.
link |
As you see, looking at my personal history, there are certain ideas you bond with at a certain
link |
stage of your career and you tend to keep revisiting them through time. If you're lucky,
link |
you pick one that doesn't just get solved in the next five years, and then you're sort of out of
link |
luck. So I think a constant influx of new people brings new ideas with it.
link |
From the point of view of industrial research, because a big part of what we do is really taking
link |
those ideas to the point where they can ship us very robust features, you end up investing a lot
link |
in a particular idea. And if you're not careful, people can get too conservative in what they
link |
choose to do next, knowing that the product teams will want it. And interns let you explore the more
link |
fanciful or unproven ideas in a relatively lightweight way, ideally leading to new publications for
link |
the intern and for the researcher. And it gives you then a portfolio from which to draw which idea
link |
am I going to then try to take all the way through to being robust in the next year or two to ship.
link |
So it sort of becomes part of the funnel. It's also a great way for us to
link |
identify future full time researchers, many of our greatest researchers were former interns.
link |
It builds a bridge to university departments so we can get to know and build an enduring relationship
link |
with the professors and we often do academic give funds to as well as an acknowledgement of the
link |
value the interns add and their own collaborations. So it's sort of a virtuous cycle. And then the
link |
long term legacy of a great research lab hopefully will be not only the people who stay but the ones
link |
who move through and then go off and carry that same model to other companies.
link |
And so we believe strongly in industrial research and how it can complement academia and
link |
we hope that this model will continue to propagate and be invested in by other companies,
link |
which makes it harder for us to recruit, of course, but you know, that's the sign of success
link |
and a rising tide lifts all ships in that sense.
link |
And where's the idea of born with the interns? Is there brainstorming? Is there discussions
link |
about, you know, like what the ideas come from? Yeah, as I'm asking the question, I
link |
realized how dumb it is, but I'm hoping you have a better answer than a question I ask at the
link |
beginning of every summer. So what will happen is we'll send out a call for interns. They'll
link |
we'll have a number of resumes come in, people will contact the candidates, talk to them about
link |
their interests. They'll usually try to find some somebody who has a reasonably good match to what
link |
they're already doing, or just has a really interesting domain that they've been pursuing in
link |
their PhD. And we think we'd love to do one of those projects too. And then the intern stays in
link |
touch with the mentors, we call them. And then they come and in the first at the end of two weeks,
link |
they have to decide. So they'll often have a general sense by the time they arrive.
link |
And we'll have internal discussions about what are all the general
link |
ideas that we're wanting to pursue to see whether two people have the same idea and maybe they
link |
should talk and all that. But then once the intern actually arrives, sometimes the idea goes linearly
link |
and sometimes it takes a giant left turn and we go, that sounded good. But when we thought about
link |
it, there's this other project or it's already been done and we found this paper that we were
link |
scooped. But we have this other great idea. So it's pretty flexible at the beginning. One of the
link |
questions for research labs is who's deciding what to do, and then who's to blame if it goes
link |
wrong, who gets the credit if it goes right. And so in Adobe, we push the needle very much towards
link |
freedom of choice of projects by the researchers and the interns. But then we reward people based
link |
on impact. So if the projects ultimately end up impacting the products and having papers and so
link |
on. And so your alternative model just to be clear is that you have one lab director who thinks he's
link |
a genius and tells everybody what to do, takes all the credit if it goes well, blames everybody
link |
else if it goes badly. So we don't want that model. And this helps new ideas percolate up.
link |
The art of running such a lab is that there are strategic priorities for the company
link |
and there are areas where we do want to invest in pressing problems. And so it's a little bit of a
link |
trickle down and filter up meets in the middle. And so you don't tell people you have to do X,
link |
but you say X would be particularly appreciated this year. And then people reinterpret X through
link |
the filter of things they want to do and they're interested in. And miraculously, it usually comes
link |
together very well. One thing that really helps is Adobe has a really broad portfolio of products.
link |
So if we have a good idea, there's usually a product team that is intrigued or interested.
link |
So it means we don't have to qualify things too much ahead of time. Once in a while, the product
link |
teams sponsor an extra intern because they have a particular problem that they really care about,
link |
in which case it's a little bit more, we really need one of these. And then we sort of say,
link |
great, I get an extra intern. We find an intern who thinks that's a great problem. But that's not
link |
the typical model. That's sort of the icing on the cake as far as the budget's concerned.
link |
And all of the above end up being important. It's really hard to predict at the beginning of the
link |
summer, which we all have high hopes of all of the intern projects. But ultimately, some of them
link |
pay off and some of them sort of are a nice paper, but don't turn into a feature. Others turn out
link |
not to be as novel as we thought, but they'd be a great feature, but not a paper. And then others,
link |
we make a little bit of progress and we realize how much we don't know. And maybe we revisit that
link |
problem several years in a row until it finally we have a breakthrough. And then it becomes more
link |
on track to impact a product. Jumping back to a big overall view of Adobe Research, what are you
link |
looking forward to in 2019 and beyond? What is, you mentioned there's a giant suite of products,
link |
giant suite of products, giant suite of ideas, new interns, a large team of researchers.
link |
Where do you think the future holds? In terms of the technological breakthroughs?
link |
Technological breakthroughs, especially ones that will make it into product will get to
link |
impact the world. So I think the creative or the analytics assistance that we talked about where
link |
they're constantly trying to figure out what you're trying to do and how can they be helpful and make
link |
useful suggestions is a really hot topic. And it's very unpredictable as to when it'll be ready,
link |
but I'm really looking forward to seeing how much progress we make against that. I think
link |
some of the core technologies like generative adversarial networks are immensely promising
link |
and seeing how quickly those become practical for mainstream use cases at high resolution with
link |
really good quality is also exciting. And they also have this sort of strange way of even the
link |
things they do oddly are odd in an interesting way. So it can look like dreaming or something.
link |
So that's fascinating. I think internally we have a Sensei platform, which is a way in which
link |
we're pooling our neural net and other intelligence models into a central platform, which can then be
link |
leveraged by multiple product teams at once. So we're in the middle of transitioning from a,
link |
you know, once you have a good idea, you pick a product team to work with and you sort of hand
link |
design it for that use case to a more sort of Henry Ford, stand it up in a standard way, which
link |
can be accessed in a standard way, which should mean that the time between a good idea and impacting
link |
our products will be greatly shortened. And when one product has a good idea, many of the other
link |
products can just leverage it too. So it's sort of an economy of scale. So that's more about the
link |
how then the what, but that combination of this sort of renaissance in AI, there's a comparable
link |
one in graphics with real time ray tracing and other really exciting emerging technologies.
link |
And when these all come together, you'll sort of basically be dancing with light, right? Where
link |
you'll have real time shadows, reflections, and as if it's a real world in front of you, but then
link |
with all these magical properties brought by AI where it sort of anticipates or modifies itself
link |
in ways that make sense based on how it understands the creative task you're trying to do.
link |
That's a really exciting future for creative for myself too, the creator. So first of all,
link |
I work in autonomous vehicles. I'm a roboticist. I love robots. And I think you have a fascination
link |
with snakes, both natural and artificial robots. I share your fascination. I mean, their movement
link |
is beautiful, adaptable. The adaptability is fascinating. There are, I looked it up, 2900
link |
species of snakes in the world. Wow. The 175 venomous, some are tiny, some are huge.
link |
Saw that there's one that's 25 feet in some cases. So what's the most interesting thing
link |
that you connect with in terms of snakes, both natural and artificial? Why, what was the connection
link |
with robotics AI in this particular form of a robot? Well, it actually came out of my work
link |
in the 80s on computer animation, where I started doing things like cloth simulation and
link |
other kind of soft body simulation. And you'd sort of drop it and it would bounce,
link |
then it would just sort of stop moving. And I thought, well, what if you animate the spring
link |
lengths and simulate muscles? And the simplest object I could do that for was an earthworm.
link |
So I actually did a paper in 1988 on called the motion dynamics of snakes and worms. And I
link |
read the physiology literature on both Hale snakes and worms move and then did some of the early
link |
computer animation examples of that. So your interest in robotics started with graphics?
link |
Came out of simulation and graphics. When I moved from Alias to Apple, we actually did a
link |
movie called Her Majesty's Secret Serpent, which is about a secret agent snake that parachutes in
link |
and captures a film canister from a satellite, which tells you how old fashioned we were thinking
link |
back then, sort of classic 19 sort of 50s or 60s Bond movie kind of thing. And at the same time,
link |
I'd always made radio control ships when I was a child and from scratch. And I thought, well,
link |
how can it be to build a real one? And so then started what turned out to be like a 15 year
link |
obsession with trying to build better snake robots. And the first one that I built just sort of
link |
slithered sideways, but didn't actually go forward, then added wheels and building things in real
link |
life makes you honest about the friction. The thing that appeals to me is I love creating the
link |
illusion of life, which is what drove me to drove me to animation. And if you have a robot with
link |
enough degrees of coordinated freedom that move in a kind of biological way, then it starts to
link |
cross the Yankani Valley and to see me like a creature rather than a thing. And I certainly got
link |
that with the early snakes by S3, I had it able to sidewind as well as go directly forward. My
link |
wife to be suggested that it would be the ring bearer at our wedding. So it actually went down
link |
the aisle carrying the rings and got in the local paper for that, which was really fun.
link |
And this was all done as a hobby. And then I at the time that can onboard compute was incredibly
link |
limited. It was sort of yes, you should explain that these snakes, the whole idea is that you would
link |
you're trying to run it autonomously. Autonomously, on board right. And so
link |
the very first one, I actually built the controller from discrete logic, because I used to do LSI,
link |
you know, circuits and things when I was a teenager. And then the second and third one,
link |
the 8 bit microprocessors were available with like a whole 256 bytes of RAM,
link |
which you could just about squeeze in. So they were radio controlled rather than autonomous.
link |
And really, we're more about the physic physicality and coordinated motion.
link |
I've occasionally taken a side step into if only I could make it cheaply enough,
link |
bake a great toy, which has been a lesson in how clockwork is its own magical realm that
link |
you venture into and learn things about backlash and other things you don't take into account as
link |
a computer scientist, which is why what seemed like a good idea doesn't work. So it's quite
link |
humbling. And then more recently, I've been building S9, which is a much better engineered
link |
version of S3 where the motors wore out and it doesn't work anymore. And you can't buy
link |
replacements, which is sad given that it was such a meaningful one. S5 was about twice as long and
link |
look much more biologically inspired. I, unlike the typical roboticist, I taper my snakes.
link |
There are good mechanical reasons to do that, but it also makes them look more biological,
link |
although it means every segment's unique rather than a repetition, which is why most engineers
link |
don't do it. It actually saves weight and leverage and everything. And that one is currently on
link |
display at the International Spy Museum in Washington, DC. None of it has done any spying.
link |
It was on YouTube and it got its own conspiracy theory where people thought that it wasn't real
link |
because they work at Adobe, it must be fake graphics. And people would write to me, tell me
link |
it's real. They say the background doesn't move and it's like, it's on a tripod. So that one,
link |
but you can see the real thing. So it really is true. And then the latest one is the first one
link |
where I could put a Raspberry Pi, which leads to all sorts of terrible jokes about pythons and
link |
things. But this one can have onboard compute. And then where my hobby work and my work worker
link |
converging is you can now add vision accelerator chips, which can evaluate neural nets and do
link |
object recognition and everything. So both for the snakes and more recently for the spider that
link |
I've been working on, having desktop level compute is now opening up a whole world of
link |
true autonomy with onboard compute, onboard batteries, and still having that sort of
link |
biomimetic quality that appeals to children in particular. They are really drawn to them and
link |
adults think they look creepy, but children actually think they look charming. And I gave a
link |
series of lectures at Girls Who Code to encourage people to take an interest in technology. And
link |
at the moment, I'd say they're still more expensive than the value that they add,
link |
which is why they're a great hobby for me, but they're not really a great product.
link |
It makes me think about doing that very early thing I did at Alias with changing the muscle
link |
rest lengths. If I could do that with a real artificial muscle material, then the next snake
link |
ideally would use that rather than motors and gearboxes and everything. It would be lighter,
link |
much stronger, and more continuous and smooth. So I like to say being in research as a license
link |
to be curious, and I have the same feeling with my hobby yet. It forced me to read biology and
link |
be curious about things that otherwise would have just been natural geographic special.
link |
Suddenly, I'm thinking, how does that snake move? Can I copy it? I look at the trails that
link |
side winding snakes leave in sand and see if my snake robots would do the same thing.
link |
Out of something inanimate, I like why you put a try to bring life into it and beauty.
link |
Absolutely. And then ultimately, give it a personality, which is where the intelligent
link |
agent research will converge with the vision and voice synthesis to give it a sense of having
link |
not necessarily human level intelligence. I think the Turing test is such a high bar, it's
link |
a little bit self defeating, but having one that you can have a meaningful conversation with,
link |
especially if you have a reasonably good sense of what you can say. So not trying to have it so
link |
a stranger could walk up and have one, but so as a pet owner or a robot pet owner, you could
link |
know what it thinks about and what it can reason about.
link |
Or sometimes just meaningful interaction. If you have the kind of interaction you have with a dog,
link |
sometimes you might have a conversation, but it's usually one way.
link |
And nevertheless, it feels like a meaningful connection.
link |
And one of the things that I'm trying to do in the sample audio that will play you is beginning
link |
to get towards the point where the reasoning system can explain why it knows something or
link |
why it thinks something. And that again, creates the sense that it really does know what it's
link |
talking about, but also for debugging. As you get more and more elaborate behavior, it's like,
link |
why did you decide to do that? How do you know that? I think the robot's really
link |
my muse for helping me think about the future of AI and what to invent next.
link |
So even at Adobe, that's mostly operating in the digital world.
link |
Do you ever, do you see a future where Adobe even expands into the more physical world perhaps?
link |
So bringing life not into animations, but bringing life into physical objects with whether it's,
link |
well, I'd have to say at the moment it's a twinkle in my eye. I think the more likely thing is that
link |
we will bring virtual objects into the physical world through augmented reality.
link |
And many of the ideas that might take five years to build a robot to do, you can do in a few weeks
link |
with digital assets. So I think when really intelligent robots finally become commonplace,
link |
they won't be that surprising because we'll have been living with those personalities in the virtual
link |
sphere for a long time. And then they'll just say, oh, it's Siri with legs or Alexa,
link |
Alexa on hooves or something. So I can see that welcoming. And for now, it's still an adventure
link |
and we don't know quite what the experience will be like. And it's really exciting to sort of see
link |
all of these different strands of my career converge. Yeah, in interesting ways. And it is
link |
definitely a fun adventure. So let me end with my favorite poem, the last few lines of my favorite
link |
poem of yours that ponders mortality. And in some sense, immortality, as our ideas live through
link |
the ideas of others through the work of others, it ends with, do not weep or mourn. It was enough
link |
the little atomies permitted just a single dance. Scatter them as deep as your eyes can see. I'm
link |
content they'll have another chance, sweeping more centered parts along to join a jostling,
link |
lifting throng as others danced in me. Beautiful poem. Beautiful way to end it. Gavin, thank you
link |
so much for talking today. And thank you for inspiring and empowering millions of people
link |
like myself for creating amazing stuff. Oh, thank you. It's a great conversation.