The following content is provided
under a Creative Commons license.
Your support will help MIT OpenCourseWare continue
to offer high quality educational resources for free.
To make a donation or to view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: All right, so this brief reminder here
is about the basic requirements in this course, which
are that we have two written reports.
And that's the main thing I want to remind you of, one in vision
and one in audition.
And I presume all of you have the syllabus that
was a hand out.
And in that syllabus-- and if you don't have it,
we can provide you, of course, with a copy.
And on the second page of the syllabus,
it says written report, vision part,
and this report is one that is based on an article published
many years ago, which is very influential article back
in 1967 about the so-called accessory optic system.
And what I would like you to do is to read that article
but then proceed through the internet
to see what have become the most recent discoveries
about the so-called accessory optic system.
And that's what your report needs to be about.
You can be reasonably summary about it
so that you don't need to write 50 pages.
But you know, you can write this whole thing up
in four or five pages, taking into account
as I've said what has been discovered
since those days about the so-called accessory optic
system.
The reason I thought this would be a good assignment because it
will give you a good historical sense of how knowledge has
expanded, in this case, since the 1960s,
in uncovering yet another attribute
that you have in the visual system that
starts with a specialized set of cells in the retina
and then proceeds through a special series of steps
until it connects with the ocular motor system.
So anyway, that's the task.
And that will consist of 10% of the report,
of your total grade.
And the same thing is 10% also for addition.
Then the midterm exam is going to consist
of 25% of your exams.
And the final exam is a total 55, a which 15 is vision
and 40 is audition.
So if you add them all up, you have an equal distribution
in percentages for vision and for audition.
So that then is the basic layout.
And so what I would like you to start thinking about
is how you're going to put this report together.
I was hoping that perhaps you could get it done by midterm.
But I don't have a hard and fast rule about that.
As long as you get it in before the final exam, that's fine.
But it may make life easier if you
can start working on that while we are talking about vision,
rather than waiting until we are covering
the auditory system in the course.
Does anybody have any questions about this basic layout?
OK.
All right, now today we are going
to talk about form perception.
Let me say one more thing, that next time as you have it
on the syllabus, we are going to be covering illusions.
I think that'll be fun.
You'll see all kinds of interesting illusions
and some inferences as to what we
think about how those illusions come about by virtue
of the fact that there are all sorts of interesting rules
and facts about how the visual system works.
And then during the second half over the next lecture,
this is this coming Wednesday, we
are going to talk about visual prosthesis.
All right, so that in essence what is we
are going to cover next time.
So now let's get back to today.
We are going to talk about form perception.
This is a topic that, even at this stage,
we still only have rather limited knowledge
about how the brain is capable of carrying out
the perception of form.
And we are going to look at some inferences and some ideas
about it.
And I will also provide you with a brief historical background.
Because this has been a topic of tremendous interest
for people for centuries, trying to understand just
how we are capable of seeing forms, shapes, patterns.
All right, so the first influential idea
that emerged about this is called structuralism.
And this idea actually took place in the late 19th century.
And one of the prime proponents was a fellow called Titchener.
And they had, in trying to come to grips with how we do this,
had a rather basic idea, much like building
a house with bricks.
The idea was that perception is an aggregate
of simple elements.
We just have a whole bunch of elements that you put together.
And that generates a more complex precept, OK.
The problem with this approach, well, the first problem
was that they began to do experiments.
And they asked subjects to try to divorce
a personal impression of anything they looked at.
But to sort of physically describe it.
So for example, if you looked at an apple,
how do you have the idea that it's an apple?
And so they said, well, it has say, four colors.
And so the person would list four colors and said
would have several shades.
And put all those together, add them all up,
and that equals apple.
Well, that was the idea then.
And when they did this systematically,
and the thing became almost ridiculous.
Because they came up with more than 40,000
elementary sensations, thinking that somehow you
puts these elementary sensations together,
40,000 of them, and that gives us
a sense of an apple, or banana, or whatever.
Now as soon as this sort of became well known,
a lot of opposition arose.
And one of the most famous ones then
was the opposition that was brought
about by Gestalt psychologists.
And the first consideration that I want to sort of bring up here
is a rather simple picture.
Here's a picture everybody looks at that.
OK, what did you see there?
You saw a car, right?
Now if you look at it, I mean this is actually
a picture of a car, all right, but it's
interrupted by all these vertical bars here.
And yet we're able to infer that it's a car.
And so the question comes up, how do you do this?
I mean, you can't simply just add up a bunch of bricks
and say it's a car.
Because there's all this interruption.
And most things that we see in the world,
we see in a discontinuous fashion.
And as a result of this, the Gestalt psychologists,
which happened sort of in the 1920s,
began to think about this whole problem
in a very different way.
And they made a tremendous influence.
And they came up with these new ideas
of how we create the perception of form.
The founder of the Gestalt psychology
is a fellow called Max Wertheimer.
So now, you can look this up in Wikipedia, by the way,
if you look at them more closely,
they came up with a few basic principles of organization.
And they call this grouping.
They argue that in the brain, somehow we group things up.
And one of the things we group has to do with proximity.
Another one has to do with similarity,
another one with common motion, another one with closure,
and another one with figure-ground perception.
This last one I'm going to talk about at the end of today's
presentation.
But now I'm going to give you an example of these top two.
All right, so the most important conclusion
that they had come to which was very much against structuralism
is that the whole that you see is
different from the sum of its parts.
Somehow, there is an active process
that creates our ability to see something that is not
evidenced from its individual parts.
Now here is an example of what these principles are.
Grouping by proximity, so here we have a bunch of dots.
And if you'll put them closer together vertically,
you see a bunch of vertical lines essentially.
And if you put them closer together horizontally,
you see a bunch of vertical lines,
horizontal lines, vertical lines.
Yeah?
So we group things due to proximity.
Another reason we group things is shown here,
is we group things according to shape, or similarity of shape,
I should really say.
And here what we have, you can readily
see a group of nine disks here, and a group of nine triangles.
And if they're not all the same as down here,
then you have much more difficulty grouping it.
So there's a strong tendency that we group things together
that are similar.
So these are some general principles
of how we organize our visual percepts.
So now we can look at some more of these examples.
In doing this, we are now going to try
to be able to say a bit more about the brain itself.
And in doing so, it's evident that the three major theories
that try to deal with how the brain does it, according
to the first of these, form perception
is accomplished by neurons that respond selectively
to line segments of different orientations.
Now that theory, obviously, is the outgrowth
of what we talked about when it was discovered that in V1, we
have orientation selected neurons.
And this orientation selectivity is something
that you see in several progressively
higher visual areas, as if everything in the world out
there was broken down into oriented line segments, which
are then somehow put together according to there orientation
to enable us to see shapes.
So that's one theory.
Another theory is that form perception is accomplished
by spatial mapping of the visual scene onto the visual cortex.
And I will elaborate on each of these.
And the third theory is one that form perception is accomplished
by virtue of Fourier analysis.
So let us first then turn to form perception,
supposedly based on breaking down the visual scene
into oriented line segments.
Now, one of the big problems with this
is that when you look at a famous art work
that you see in every day in the Wall Street Journal,
there is an artist who created this originally.
And he created faces like this using only dots
and varied the spatial frequency of the dots,
as you can see here.
This happens to be a person called Larry Poons.
I'm sure none of you have heard of him.
But here was one of the pictures many years ago
in the Wall Street Journal.
And even today, as I say, you can see faces created this way.
And you can readily recognize them,
even though there are no oriented line segments here.
And in fact, if you sort of squint up and you three
quarter ways close your eyes, so you can't even
see the dots anymore, you can still
make out that face very, very clearly.
That's because the spatial frequency effect here
is very important, and the degree
of shading, meaning that this involves rather low
spatial frequency analyses, rather than
the analysis of particular orientations of line segments.
Now this particular analysis has its counterpart
in the observation that many of you have seen.
When you look at a person's face on television,
and they want to prevent you from being
able to see that face because it's confidential or something,
what do they do?
Anybody remember?
What you do is you put up a bunch
of squares of different spatial frequencies.
And each square is comprised of the mean illumination
level of the actual face.
If you do that, this high frequency information
is something that interferes with your ability
to analyze face.
So to give you an example of this, I'm sure all of you
have seen this, but mainly I show you this.
Here we have this example.
How many of you can tell who these people are here?
I guess nobody can, right?
Now then what we can do is we can
increase the frequency of these.
Here it is.
And it's still very difficult to tell.
And now I'm going to show you the actual photograph.
And what I'm curious about is how many of these two people
you actually know.
I can tell you right away, they're
extremely famous actors and actresses.
Who recognizes these people?
How many of you recognize?
Let me see your hand.
Only a few people recognize it.
That's how quickly time goes by.
These were the most central, most exciting, best known
people in the movies.
And this is Humphrey Bogart, Ingrid Bergman.
OK, so these are these two very famous people
whose faces were obstructed by this and by this using
these high contrast edges that obscure your ability
to smoothly analyze faces.
So that then suggests that this idea that we extensively use
oriented line segments to analyze faces at least
is an insufficient explanation of how
the brain processes shapes.
Now another idea is so-called topographic mapping.
This idea I'm sure you can already
reject on the basis of what I told you
about how the visual system is laid out.
But now I'm going to belabor that
so that you can follow it closely.
Here we have a runner, of course.
And here we have a monkey brain to make it easy to understand.
And the idea here there is that this image, once it
was discovered that the visual field is laid out
topographically in the visual cortex,
the idea was that what happens is somehow,
and I may be am unfair to poke fun of it,
but the idea is that the mind can
look at the creation of this image on the cortical surface
there.
And thereby, it can identify.
OK, it's almost like a photograph,
looking at a photograph.
In this case the mind looks at the photograph, so to speak,
on the cortical surface.
So they thought at the time is that this
is what you have there, OK.
That's the imprint of this image.
And indeed, you say, oh my goodness, that's
just like that.
Therefore, I can recognize that person, and so on and so on.
Well, I mean that's a cute idea.
But then when you take into account
this has been discovered subsequently to these ideas,
that the topographic layout in the visual cortex
is actually not one to one because of the magnification
factor.
So let's look at that in some detail.
Here is an actual reconstruction of the monkey area V1 here.
And here is the visual field.
And we are going to put these red arrows
in the contralateral hemifield.
Now remember what I told you before,
that if you look at the visual scene,
you can imagine your eyes being vertically cut in half.
And you have a nasal and temporal hemiretina.
And the one which is contralateral hemifield
crosses over and gets into this half of the brain.
And this one crosses over to that half.
So now if you do this and take the magnification factor
into account, you look at these arrows,
which are all identical in size, OK.
There are one, two, three, four, five, six, seven of them.
What you can see here is the actual impression
on the cortical surface of the neurons
that are being activated by this look like this,
nothing like those arrows there.
And in fact, the central arrow is much bigger
because of the huge magnification factor.
Now this is already creates a major problem
in trying to believe in that particular theory,
the topographic mapping theory.
But now, if instead, you put those arrows
halfway across each of the hemispheres like that,
then whatever is on this side goes here.
Whatever is on this side goes there.
And this is the impression that is created.
And my goodness, you create that impression.
And you say to yourself, my god.
If that's the case, how come I can
see seven straight arrows of equal size
when this is the impression that is being created in the brain?
So therefore, obviously, they are not using a topographic map
to analyze the visual scene at all.
Now this can be driven home even further.
Let me make another point here.
Here we have a monkey visual cortex again from the rear.
Here is the visual field laid out.
And if here we put in a bunch of dots in a circular fashion,
this is the activation there.
Now you say, oh, that's not bad.
That looks like a circle.
But then if you put it along the midline, half and half,
then this is the actual activation.
But you still see a circle, even though the activation,
in terms of the topography, is nothing like a circle.
So that indeed created a much, much greater degree
of skepticism among investigators
to try to understand how we process shape.
Now let me make one other point here,
which is a wonderful story, which
I think will be good for you to remember.
That's called the Giotto story, which
says that when Pope Benedict, that was
in the 12th century, or the 13th century,
he set out to have the walls of the great cathedral of Saint
Peter in Rome redecorated.
And so he sent out a bunch of messengers
to various artists in Italy and asked
them to provide some of their best work
so he could evaluate it and could
pick one guy to actually do the redecoration.
Well, one of these messengers went to Ambrogio Bondone Giotto
and asked him, he was a well known artist,
and asked him to provide a painting of his,
a drawing of his.
And Giotto said, oh, my goodness.
I just don't have anything around.
But I tell you what I'll do.
He took out a red pen and drew a perfect circle, OK.
And so the messenger took this to the pope.
And the pope said, my god.
I can't believe how incredible this is.
And so Giotto got the job of redecorating
the cathedral of Saint Peter.
To this day, there is this expression in Italy,
in Tuscany in Italy, which says the round O of Giotto, OK.
So this is the round O of Giotto,
which somehow, in a sense, denotes perfection, perfection
in sight and in perfection in your ability of execution
in terms of a drawing, for example.
All right, now to highlight this even further,
let me point out this to you here.
What we have here is a bunch of imperfect circles, one of which
is perfect.
So if you keep looking around, you
should be able to spot which one it is.
And you should tell me what letter denotes that circle.
Which one is a perfect circle?
AUDIENCE: C.
PROFESSOR: Very good, all right.
So we are incredibly good.
There's a slight difference here, OK, or even slight
difference here.
And yet, we can see this very slight difference.
And we can tell what a perfect circle is.
I mean, that's incredible given how the impressions are
made by those circles on the visual cortex.
So that is an incredible puzzle of how
we are capable of doing this.
And I'm afraid, even to this day,
we don't have a really good answer of how this happens.
OK, now the third theory that has become actually
probably right now, one of the most successful ones,
it claims that you analyze the visual scene
by taking into account the spatial frequencies that
are impinging on the retina, OK.
This was created by Fergus Campbell and John Robson,
very influential.
And they pointed out, first of all, that a very interesting
finding, which I think I've mentioned once
before, that if you vary the spatial frequency [? both, ?]
as well as the contrast, we have this sensitivity function
like this.
I did show you a picture of it the last time.
And then, furthermore, what they had shown
is that you can create all kinds of complex precepts
by varying the gratings.
Make them simple gratings, compound gratings,
and compound gratings with much lower contrast.
And this is simple different spatial frequencies.
These are compound.
And these, as I've said, are lower spatial frequencies.
So then if you again squint, then you
will see something much smoother.
So they actually carried out a detailed mathematical analysis
using Fourier analysis.
And what they did was quite remarkable.
They would down break down a visual scene,
like a photograph of New York with all the skylights,
OK, skyscrapers.
And then they would convert that using Fourier analysis.
And they could recreate the visual scene
with a high degree of accuracy using that procedure.
Now if indeed the visual system uses this,
you have a number of basic logical requirements.
And those basic logical, let me come back to that,
the basic logical requirements are
that you need spatial frequency analysis.
And I've shown you that already when we talked about V1,
that neurons there are spatial frequency selective.
Secondly, that there are contrast selective,
which you know already.
And of course, the orientation selective,
and they can tell you about phase.
As long as you have these four attributes,
you can perform a detailed Fourier analysis
to reconstruct the visual scene.
So now to stress this even further,
they did a series of experiments, in which they
asked a question, is it true that you
have a particular spatial frequency analysis,
that you can manipulate that?
And so they did an experiment, which
called a frequency-specific adaptation experiment.
They would present to a subject this display
and have the subject look at this for a couple of minutes
without having to fixate.
And then they would look at each of those.
And they found that this, which is the same spatial frequency,
they had difficulty seeing because of this adaptation.
And so then they did a series of careful studies
and carried out this analysis and showed
that you could get any kind of spatial frequency
to lose your sensitivity for it if you
had been pre-exposed to it.
So that's what that looked like.
And by doing this systematically,
they came up with the idea that what
you have in the visual cortex is a series of channels
that are spatial frequency-selective.
And I showed you we talked about V1,
that indeed, there are neurons there that are selected
to particular spatial frequencies.
And they proposed that you have a series of channels like this,
OK, that peak at different spatial frequencies.
And by activating them selectively,
you can reconstruct virtually anything out
in the visual scene using Fourier analysis.
Now OK, I think that then is the essence of that theory.
And I can tell you that some people avidly believe in it.
And there are some people who are highly skeptical.
I'm not sure where I stand at this stage about that.
But now, people began to study our shape,
the ability to see shapes by recording various visual areas.
And I want to tell you next about some studies that
had been done primarily in Japan looking
at inferotemporal cortex, which is [INAUDIBLE] already
mentioned that to be involved in the analysis of shapes
and in particular, in the analysis of faces as well.
So what these investigators did is
they would record to individual neurons
in inferotemporal cortex.
And when they did that, they would present various stimuli
to see how those cells responded.
And they found, they claimed to find,
that there was some incredible specificity
in inferotemporal cortex for shapes.
And so what they did, here's an example.
Here, we have the neurons' responses, histograms.
And on top, above it, we have the particular shape
that was presented repeatedly.
So this particular cell responded vigorously to this,
but very poorly to that, and so on down the line.
But you can see that it responded
to several different shapes, quite well to this one,
reasonably well to that one, so a whole range of them.
So it wasn't like this particular neuron responded
only to one particular shape.
But that idea that you have neurons
which are specific for certain shapes
did take on a lot of attraction on part of investigators.
And some of them indeed thought that you
have these neurons which are selective
to individual elements in the visual scene.
And those subsequently the critics refer to
as your having grandmother cells.
Somewhere in the brain there's a cell
that represents your grandmother, all right.
Now that idea was disabuse subsequently.
But it took very stronghold in many investigators' minds.
And here's another example of yet
another inferotemporal neurons, in this case again
for various shapes, showing that this shape was,
elicited a lot of responses, as well as did this.
The rest of them didn't respond, didn't elicit as much
of a response.
So these kinds of experiments then,
they tried to systematize.
And they came up with an idea which, at least to my mind,
borders on the absurd, which is shown here.
Here is a section of inferotemporal cortex.
And they argue that they are columns there,
and that these columns represent different percepts.
So here we have a bunch of percepts
that process the monkey.
This is a monkey, of course, that process monkey faces.
And by inference, therefore, there
must be some areas in inferotemporal cortex
that process human faces in humans.
And then others process different aspects
of the visual scene.
Now the big problem, technical problem, with this approach
is that when you record from individual neurons,
all right, typically in these experiments,
you can study a single neuron for a fairly limited time,
not like for months on end, just maybe a few hours or something.
And so because of that, you can only
present a limited number of visual stimuli.
Now there are millions of visual stimuli out there.
And so to really establish how specific this particular neuron
is, they say well, these are the only shapes it's showing us.
What if you use the cross, or who knows,
many, many other things, something
that is three dimensional?
How would these cells respond?
And so these cells respond to different degrees,
to hundreds and hundreds and hundreds of different stimuli.
And the real fact then is that anytime a stimulus
appears out there, you're activating tens of thousands
of neurons, maybe even more, in the visual system.
And each of those neurons, especially
in inferotemporal cortex, responds to different degrees
to the different stimuli.
It is the compendium of these many, many neurons firing
into different degrees to different stimuli
that gives you sort of an overall computational ability
to say what that stimulus is.
Now to be able to analyze that, it is that complicated,
that really takes a lot of effort.
Now some people are now trying to do this.
And the way you try to do this is
that you're recording from hundreds and hundreds
of neurons with multiple electrodes,
present various scenes.
And you see how these neurons respond as an aggregate,
so that you can determine whether or not there is indeed
the potential for some sort of computation
that takes place in gaining the impression
of a particular individual face.
Now one of the very important facts
here is, of course, that as you look around the visual scene,
let's imagine you're looking at a face.
You're looking at a face head on.
You're looking at a face in a profile.
You're looking at a face tilted.
You're looking at a face close.
You're looking at a face far.
And it's still all the same face.
And yet the impressions that face
makes on the visual system, in the retina,
in the geniculate, in the visual cortex, varies a great deal.
And yet we can come up with constancy,
which is sort of a higher level process.
And I'll come to that in short order.
All right, so now, therefore, we need
to start to talk about what we call intermediate level vision.
What is intermediate level vision?
So far, we've talked almost exclusively
about basic visual capacities, color, brightness, pattern,
texture, motion, depth, the very,
just the very basic types.
But now when we talk about intermediate visual capacities,
we talk about constancy.
One example of constancy, of course,
is that the face that you're looking at,
whether it's profile, or head on, or near, or far, it's
still the same face.
So we get constancy out of it.
Then there's an important necessity
to be able to select various aspects out
of the visual scene, to be able to select if you recognize
things, to induce transposition and variance,
and to be able to make comparisons,
and also lastly, to be able to say where things are in space.
So those are these intermediate capacities.
And we'll talk a little bit about them next.
All right, so here's an example of constancy.
What we have here is a bunch of words
for doubt in different sizes, and orientations,
and some handwritten, and so on.
And yet all of them say the same thing, doubt.
We extract that, OK.
Now to make this even more difficult,
I'm going to tell you that there's one of these words here
which is not doubt and see who can
find the word that is not doubt.
Everybody find it yet?
OK, right there, not doubt but doubts.
And so we are able to extract the difference
in this kind of visual scene, even
though it is incredibly subtle.
And yet, we are also able to say that this doubt,
and this doubt, and this doubt are all the same,
even though they're different size or different in print.
We can extract from that the common element.
So this is an incredible capacity
on part of the visual system.
And of course, the big question comes up, how is this achieved?
And at this stage, we only have very preliminary answers
to that.
OK, so now I want to show you a couple other cute things here.
This is another famous artist, Hirschfeld,
He is no longer with us, unfortunately.
He published hundreds of these little cartoons
in The New Yorker, all right.
Now, you see his named here, Hirschfeld.
What do you see at the end of his name?
What's that-- what is this?
What, that?
Huh?
A number three.
Why does it say Hirschfeld three?
His name isn't Hirschfeld three.
And then if you've seen several of these,
how many of you have seen these before?
Oh, you missed out on something very good.
At least look it up sometimes again on the internet.
Just type in Hirschfeld.
And some of these will come up.
So he says three.
And some of his pictures may have one.
Some of his pictures may have five or six at the end.
And you say, well, what on earth is that?
Anybody know?
Yes.
AUDIENCE: Doesn't he have other names hidden in the images?
So maybe that's the [INAUDIBLE].
PROFESSOR: Ah ha, you're thinking well.
OK, that's getting there.
OK, so he's telling us something that there
are three of in this picture.
That's what he's telling us.
OK.
So what is the three of that he has in this picture?
Ah ha, anybody want to come up with?
It
AUDIENCE: The line orientation?
PROFESSOR: Huh?
AUDIENCE: Like the orientation of the line?
PROFESSOR: Orientation?
No.
Mhm.
OK, I'll give you a clue.
Hirschfeld had a daughter--
AUDIENCE: Nina.
PROFESSOR: Who he was very fond-- hey, fantastic, Nina.
OK.
So had a daughter, Nina.
And he decided in his display to put her name up there.
And now he said three, that he put her name up
there three times.
Now who can find all three of them?
AUDIENCE: [INAUDIBLE] the arm [INAUDIBLE].
PROFESSOR: OK, here's one.
Here is another one.
OK, and here is the third one, in the arms.
And so if you, that's one of the fun things you can do.
Back when he was doing this actively,
every time he had a cartoon like this,
I would spend some time trying to see where were the Ninas.
[LAUGHTER]
So that I think is quite remarkable
and further highlights the complexities
and the remarkability of our visual system.
OK, so now, I'm showing you yet another picture.
In this case again, you have to be
knowledgeable about something.
Does anybody know who this is?
Oh, all right.
Well, let me see if I make it smaller.
Can you see that now?
Who is that?
OK, that's Voltaire.
All right.
Now Voltaire, 18th century, is a guy who had outdone all of us.
And he had written 2,000 books.
Can you believe that?
The guy had written 2,000 books.
But that's not the prime point here.
Most of us who know a little bit about history
immediately recognize him as, oh yeah, that's Voltaire.
But then if you go back here and you look at this more closely,
and so what I'm going to do here is just
to tell you what a remarkable artist.
The guy who did this is Salvador Dali.
This, by the way, is his wife, OK.
And so what I'm going to do here,
I'm going to blow up the center portion of this figure.
There it is.
Can you see this?
This is two faces, two nuns.
And this is their outfit.
So there's nothing there that is really Voltaire.
But he's such a remarkable artist
that he could create Voltaire by playing around with this, OK.
So let me go back to the display again.
OK, so this is Salvador Dali.
And certainly, it's an artist who is remarkable.
He has many paintings in which this kind
of a double confusing thing, playing on your ability to see.
So because of that, for people like me
who's interested in how we see things,
certainly I'm very intrigued by his artwork.
And you may enjoy looking that up again on the internet.
OK, so now I'm going to tell you yet another interesting aspect
of art involved in this.
And this one has to do with a book, a very clever book that
was written by David Hockney.
Has anybody ever heard of him?
I'm not surprised since he's not that well known.
But the book he wrote is called Secret Knowledge.
Now what was that all about?
Well, he analyzed how artists created paintings way,
way, way back when and in subsequent years
up to the present.
And now I'm going to show you some of his pictures
and give you a sense of what this is all about.
Here is a picture, OK, by Masolino da Panicale in 1425.
This was a typical kind of picture back then.
Artists had a very poor sense of how
to create depth perception in a painting.
That was before they came up with vanishing point.
And so this kind of looks flat and fairly expressionless.
This was in 1425.
And then Hockney noticed that just five years later,
another artist came up with a picture,
just five years later, that looks almost like a photograph.
And he said, what on earth has happened?
And that's why the book is called the Secret Knowledge.
So what happened?
Anybody know?
Ah, all right.
Well, let me tell you what happened.
What happened is that at that time, in the 1400s,
they came up with the lens, OK.
And so they created a device called the camera obscura.
Does anybody know what the camera obscura is?
It's essentially very similar to a camera, OK.
So here we have it.
Here's a camera obscura.
And what they did, these artists,
they would create a building.
So it will be dark inside.
And then they would put the lens here.
And they would put some object out here
that they wanted to make a painting of.
And that would be reflected onto a piece of canvas here.
Of course, it would be upside down, right?
And then they would paint it.
And so once it was painted, they could turn it around
and finish it and sell it.
Now, the reason they did this, the prime reason they did this,
is because it was much much, much quicker
to create a portrait, for example,
by using this procedure than to actually look at a person
and paint them on the canvas while looking at it.
OK.
So that was done.
And of course, this was sort of a no, no thing.
And because of that, it was kept secret.
All these artists who did this, and some very famous artists
did so, were very careful never to disclose to the public
that they did this kind of thing.
Because it was conceived to be kind of a cheating thing, OK.
So what happened then was that all these paintings were
created.
And here's an example by van Eyck in 1436.
Again, this looks much like a photograph.
But some other thing is a bit distorted.
So Hockney undertook a careful, detailed analysis
of how we could tell whether a painting was
a real painting or one that used the camera obscura method.
And that's also a real painting in a sense,
but a painting using the camera obscura method.
And he came up with a series of criteria
which are listed in the book.
But I'm only going to deal with one of them.
OK, so here is an example.
This is in 1597.
This is by Caravaggio, OK.
And what is notable about this person?
Well, to Hockney, what was notable
is that this person is holding the wine
glass in his left hand.
Yeah?
And he said, huh.
That's curious.
And then he looked at a whole bunch of other paintings.
And he had one in which there were
three people on the painting.
And all three of them were left handed.
Yeah.
And he said, my god, that is really curious.
And he said, well, let me analyze
what happens when you do this kind of stuff
with the camera obscura method.
So here's the example of this.
This is the original image, he claims,
meaning the person is right handed, is not left handed.
Then you put this person through the lens
and put him up here upside down like that, OK.
He's painted on the canvas like that.
And then what you do is you rotate this 180 degrees.
And when you do that, lo and behold,
the person becomes left hand.
And to make this clear, I added the F here.
So this is a normal F. This is what
is projected onto the canvas.
And this is when you rotate it 180 degrees.
So you reverse the left according to Hockney,
reverse the image.
And so it was a dead giveaway that most of the people who
appeared left handed in his paintings, in these paintings,
I should say, not just Caravaggio,
but several other people, used the camera obscura method
for creating the painting.
All right, so that is the process.
And then I want to show you one more example of this.
OK, this is a very famous painting also, all right.
This is the so-called marriage of Giovanni Arnolfini.
And this by van Eyck, again 1493, way, way, way, way, way,
way back then.
Now this is a famous painting.
Now there's one interesting thing
first of all I want to point out.
You see this here?
That's proof that in those days they
had come up with the lens, yeah.
Now what is wrong with this picture?
This guy who is about to marry this woman
is holding her with his left hand.
I mean, that's unacceptable.
You're supposed to hold it with the right hand, yeah.
Now the reason he's holding it with the left hand
is because the artist, van Eyck, used the camera, according
to Hockney, used the camera obscura method to take this,
to paint this picture, OK.
And then he rotated 180 degrees and became,
he became left handed as a result.
Now if instead of having done that, you would go back,
you don't have to go back.
Go back to here, and let's go back to this.
If you take this guy here and you create the picture.
But then instead of rotating it, if you could flip it,
then you would, he will remain right handed.
But of course, we can't do that because it's on a canvas.
It's not on a transparency.
So that then is the interesting story of artwork
that was created using the camera obscura
system that further the highlights
the amazing interestingly complicated manner
in which we can analyze the visual scene for shapes.
All right, now another factor that is in a similar vein
has to do with the recognition of faces.
Lots of experiments have been done, including several
in our department here, that has recognized
that, unfortunate use of terms, recognition, recognize,
became aware of the fact that facial recognition depends
very heavily on seeing faces right side up.
When faces are upside down, you have great difficulty
telling who is who, OK.
So let's do that.
Here are a bunch of faces.
And I bet you can you can tell this one, right?
Who is it?
And who is this?
Very good.
And those two you don't know.
The problem is that I am going to flip this over now.
And you still don't know who those two are.
OK, so this one is Norbert Wiener.
Now all of you know about Norbert Wiener.
He is one of the great geniuses of our time.
He came up with the digital code.
He used to be a professor at MIT.
And this here is Chuck Vest.
Anybody recognize him?
Chuck Vest was a president of MIT for what?
For eight years, I believe, maybe more, 12 years?
I forget.
And he has sunk into obscurity, even though he
was incredibly visible for many, many years.
If I had shown this to you say eight or 10 years ago,
you would immediately recognize him.
Because he, at that time, was a president at MIT.
So that then is an interesting fact
that even though we are capable of using intermediate level
vision in a very sophisticated way,
it does not seem to work that well for upside down faces.
You guys did pretty good with those.
But it takes a while.
If I had flashed that on in a tachistoscope,
you wouldn't have had a vague idea of who those people were.
But once it's on for a while and you can analyze it,
you can eventually tell even an upside down face.
OK, now in the same vein as we are talking about,
our ability to process shapes based on contours and whatnot,
one of the interesting set of experiments people had done
is to look at what is called subjective contours.
And so let me give you a couple of examples of that.
Here's an example.
Almost instantly, here you can see a disk.
And here you can see a square rotated
45 degrees, a diamond if you will.
But if you analyze this carefully,
you can see that about 80% of this border
here is not a border.
There's no border here.
There's no border here, here, here, here, here.
And yet we can see a square.
So there's some strange ability in part of the visual system
to complete inferred borders.
All right, I'll come back to that in just a minute.
Now another example of this is shown here.
Can you make out what it says here?
If you look a little bit, you should be able to see.
What is it?
Visual system, very good.
It was difficult to see.
But as soon as I turn this into color,
you have no trouble at all.
And that highlights another important reason
why color vision is so useful and has evolved.
Because it enables us to see borders,
where under certain conditions and lighting conditions,
borders would not be visible on the black and white.
OK, so now that's another interesting example,
has to do with further subjective contours.
And here's an example of one with a high contrast,
where you can readily see a cube.
Does everybody see a cube there?
All right, but if you look here, it's next to impossible
to see that.
Because here the stimuli are isoluminant.
So here you eliminated contrast, which
is a very important aspect of being
able to analyze the visual scene in various ways,
including three dimensions.
All right, so now here's a very interesting discovery that
was made by recording an area, V2.
The recordings were then made to see
whether those neurons perceive subjective contours.
So here's a receptive field.
And you take this bar and move it back and forth across it.
And you can see it gives a vigorous response.
Now you do the same thing.
But you don't, here you have only a subjective contour.
And then you move this back and forth across.
Somehow the information is added up from other areas,
so that this cell responds, not that well,
but responds reasonably well to the subjective contour.
So it's said that in V2, you can carry out
some of these higher level processes that
enables you to complete figures even
when they are incomplete like that.
OK, here's another example of that.
This is even more dramatic.
In this case, we take this bar back and forth across.
That's a vigorous response.
And then you do the same thing.
You create a bar here simply by these continuous horizontal
bars.
And still, the cell responds quite well,
as if it were seeing an edge here.
So we have this kind of completion in area V2.
So this is sort of an initial hint
then that already in area V2, you
begin to process higher level events,
among which is the fact that you can
complete incomplete contours.
All right, so now, the next thing we're going to turn to,
we're going to ask the question, when
we deal with these so-called intermediate level
visual capacities, what happens when you take out
such areas as V4 and MT to your ability
to see these intermediate visual capacities?
So let me describe to you some of these,
how one would do experiments like this.
First of all, monkeys are trained.
This is done with monkeys, of course, because we can't just
take a human and remove V4 MT.
And so what you do here is you first present a fixation spot.
Once the monkey fixates, you present a shape,
in this case a square.
And then you present a whole bunch of other ones, only one
of which is the same as the original.
And the monkey has to make a [INAUDIBLE] there
to be rewarded.
So he has to be able to detect identity, which
is an intermediate visual capacity.
So that's what this is like.
And so then what you can do here is
to vary the amount of information
you can provide, again by reducing the amount of contour
information that you can provide.
You can do this in several ways.
First of all, before we do that, I'll come to that in a second,
let's just see, how does the monkey do
when you do those very similar shapes I just have shown you
after you take out area V4?
What you find here is that the monkey initially, after you
take out the area V4, can't even do the regular task, which
is this one, right?
So this is identical to this one in this case.
And when you do that, he does very poorly to begin with
and then gradually, over many, many days,
improves a great deal.
Then if you put a new figure in, then it takes a while for it
to learn that, all right.
Now then, what you can do, now I come
to this business of having these same figures.
But you can vary them now, so that you
have to do a transposition to do an intermediate visual task.
In this case, what you do is that you vary the size.
In this case, it's identical just like it was.
In this case, you see that this is smaller than this.
But you say, oh, I have to go find the circle.
I'm not looking for identity.
I'm looking for something that's the same looking.
And here we have even bigger case.
This is a triangle.
And so the monkey makes a [INAUDIBLE]
to that one here to this one.
So that has to do a test position in size.
Another thing you can do that we talked about,
you can vary the amount of contour information
by doing this kind of occlusion.
And the degree of occlusion you can
vary by varying the spatial frequency of the display.
And lastly, you can also decrease or increase the amount
of contour information that you provide.
Now if you do with this, you get a huge effect after V4 lesion.
This was the normal condition with the varied object size.
This is the occlusion.
And here is the varied contour information.
You can see that there is quite a dramatic loss, not
a total, but quite a notable loss, in the monkey's ability
to perform this task.
And this is also reflected in the huge increase
in the latencies with which the monkey can perform the task.
So area V4 seems to play an important role
in these intermediate visual capacities.
And I come to some more examples of that in just a minute.
Now, yet another important factor
in analyzing the visual scene occurs
when it is our task to find something
out there that is less noticeable.
Remember, we talked a little bit about using camouflage,
in which case, you have to find something lesser to survive.
Now in this case, what we can do here,
do a similar experiment in a monkey.
On the left side, the target, the one
the monkey's supposed to select, has a much higher contrast
than the distractors.
And you can vary the degree of difference,
but always the target is brighter than the others,
so that it stands out.
But then, you must be equally able,
whether you're an animal or a human,
to be able to pull out something that's
lesser in the visuals field.
And you have to be able to do this
if you are going to survive in your environment.
Now here then, the task is to go to this lesser stimulus,
because it's the odd stimulus.
So what you're extracting is the odd stimulus.
Here it's easy to extract, and here it's difficult to extract.
And so now the question is what happens in the visual cortex?
What area plays a role in this?
And so it was discovered that area V4
is very important for this.
And let me explain this to you then.
You do an experiment in which you remove area V4
and see what happens.
And here what we have is when the star gets brighter,
you vary the luminance difference.
And you can see there's a mild deficit with the V4 lesion,
highly significant but still fairly mild.
On the other hand, when you make it dimmer,
the monkey is practically staying at the probability.
He cannot do the task at all.
So somehow, V4 plays a very important role
in being able to ferret out some subtle things
in the environment, lesser things.
All right, so this then brings us
to yet another way of analyzing this.
We can, in this case, go to the larger target,
in this case to the smaller target.
And you can recognize yourself that this is certainly
easier than that.
But a normal monkey can do both of these quite well,
shown here.
This is the normal monkey's performance,
the target larger, here the target smaller.
He does extremely well.
But with a V4 lesion when the target is smaller,
it's totally devastated, so as if area V4 were involved
in the analysis of subtle things, things which are lesser
then, rather than being reflex like
and going to the brightest, biggest thing in the world.
OK, so that then indeed highlights the fact
that we have some of these areas,
including V4 and of course, therefore MT,
that involved in these much more subtle types of visual analyses
that we need to perform.
Now capitalizing on these kinds of subtle things
that we are capable of doing, artists, in addition to
the ones that I've shown you before, also
created all kinds of percepts, I should say paintings, sketches,
precepts, that cause confusion by playing around
with these factors.
One of these here is a very well known audience called Escher.
This was so near the end of the 19th century.
And here what we have is this is sort
of a figure-ground confusion.
Here what we have is a bunch of birds that fly to the right,
and also a bunch of birds that fly to the left.
And so it's confusing.
It's alternating.
You don't know which one is which.
And it has to do with a very, very clever creation
of figure-ground confusions.
And actually, next time we talk about illusions,
I will bring in some more of these kinds of curious effects.
And then here's another one.
And it's very hard for you to tell are the stairs going up?
Are they going down?
What's going on?
It's the same kind of play with the paintings
to create confusion in your perceptions.
And then here is yet another one,
where you don't know is water running up,
or is the water running down here?
That's again an interesting confusion
that Escher has created.
All right, so that then is the essence
of what I wanted to cover today to highlight
the fact that our ability to extract shape
information in the world is absolutely incredible.
And it has triggered not only experiments
to try to understand that by scientists,
but it has created a tremendous amount of artwork
that played with these because it was so enjoyable.
And certainly, the several paintings that I have shown you
must have given you a pretty good sense
of how artists capitalize on these limitations in ability
to extract intermediate and high level vision
capacities from the visual scene.
All right, so to summarize them, first of all,
I mentioned three major theories that
have to do with form processing.
The one, the orientation of line segments.
And I pointed out to you that that theory is not
particularly powerful.
And then the one, the topographic theory,
which turned out to be bordering on the ridiculous.
And finally, Fourier analysis, which
seems to have a lot of power, even though it also
has created a lot of skeptics.
All right, then I pointed out to that are the V2, V4,
and inferotemporal cortex, play important roles
in intermediate vision, these intermediate visual capacities
we talked about in detail.
Then I pointed out to you also that in V2, it
was discovered that there are some neurons that
respond to subjective contours, indicating that already in area
V2, we can perform these incredibly higher level
abilities to extract information when
it is unclear in the visual scene.
Then I pointed out to you that recognition
of objects transformed in the various ways
is compromised by V4 and inferotemporal lesions.
And V4 lesions also produce major deficits
in learning and selecting lesser stimuli, which
is a very important attribute for us
to be able not the respond in reflex-like fashion
to what is most obvious out there,
but to be able to extract the subtle things
from the visual scene.
Some inferotemporal neurons are selective for objects
including faces.
But most respond to a variety of objects whose recognition is
based on the differential activity
of a great many neurons.
OK, so that then brings me to the last point, which
is how we process and deal with ambiguities in perception,
unfortunately still remains a mystery.
And so there's a lot of space here for new investigators
to come up with exciting, interesting new findings
about how the brain performs these kinds of subtle analyses
of the visual scene.
OK, I think I'll leave this until next time.
I'll talk about this next time.
OK, does anybody have any questions?
All right, I hope that you can take a little bit of time
out, look at some of these artists on the internet,
and look at Hirschfeld and the Ninas that I had shown you.