The following content is provided
under a Creative Commons license.
Your support will help MIT OpenCourseWare continue
to offer high quality educational resources for free.
To make a donation or to view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: All right, so today our topic
is going to be depth perception, which,
as I have mentioned to you before,
is certainly one of the most intriguing achievements
in vision.
Because the impressions onto the retinal surface
essentially two dimensional.
And from that, somehow the brain needs
to reconstruct the third dimension.
And what is interesting about this also,
that even in the most primitive animals, this is a must.
And so annals with tiny brains also
have mechanisms to be able to calculate depth
from the information that comes in through their eyes.
And to demonstrate that, I have here
a frog, which has a tiny little brain like that, has big eyes,
and this frog, for its existence,
needs to know exactly where things are in depth.
Because if he doesn't, he would starve to death.
And so what a frog does looks something like this,
very crudely.
He will stick out his tongue, grab a flying insect
and consume it.
And because of this incredible capability,
it is a well-adjusted, healthy animal
in most parts of the world.
Now the big question then comes up,
how do we carry out these computations.
What kind of mechanisms are involved
in being able to compute where things are in space,
either in absolute sense where it is from you
and in a relative sense where one object is
relative to another one.
Now it turns out that this became such a serious problem,
in the course of evolution, that actually
several different mechanisms have evolved
to make possible our ability to see things in depth.
And so when one looks at this as a list, as a fairly brief list,
we can make a distinction between so called ocular motor
cues and visual cues.
The ocular motor cues are accommodation and vergence.
So if various objects are at a very distance from you,
your eyes converge or diverge.
And you your lens gets thicker and thinner.
And that information can be utilized in a rather crude way
to tell you about where things are in that relative to you.
Now as far as visual cues are concerned,
the very significant one we are going
to talk about quite a bit, is a binocular cue,
which is called stereopsis as you all know.
And then we have a whole bunch of monocular cues motion
parallax, shading, interposition, size,
and perspective.
And so we will talk about many of these
to give you a sense of what it is like
and to give you a sense of what various brain structures do
with this as a result of extensive research
that had been done in this area.
So now, first of all, let's talk about stereopsis.
And we talk about stereopsis, we're
going to talk about the basic facts of it,
and then we are going to have some demonstrations.
First of all, the so called stereoscope,
of whichever modern version that has been handed out to you,
the stereoscope was invented in the late 19th century.
And when that was done, the initial approach to this
was to be able to present to each eye separately
an image that was taken by a camera that
has two lenses, which are apart about as much as your two eyes
are apart.
And each of those created a separate image
of what's out there.
And, of course, each eye gets a very slightly different
perspective of what's there.
And then when you present these two images
that you had collected separately to each eye,
you get a very strong sense of real depth,
as you will see in just a minute.
Now another way to do it, which nowadays is easier
because you can barely ever find even
one of these two-lens cameras, even in stores that
sell ancient materials, antique stores.
So sometimes what you do instead,
if you only want to take a picture of a static image,
that you can take a camera, put in a track,
and have it take two pictures in succession.
And then you can do the same thing
as you do with a serial camera.
You can present one to each eye.
OK, so, what we are going to do now,
we are going to have a series of demos.
And so we have a handout for each of you, the paper.
And that you can keep and take home.
But the stereoscope that I have for each of you,
that you're going to have to leave behind,
because I need to use that in other classes.
So what I want you to do then, there
are two pictures on the first page
that you put the stereoscope down onto the page
so that the vertical line cuts it in half
so that one goes into each eye.
And then you put your head right down to it
to look into it, all right?
And if you do that, if you have it properly sectioned,
you're going to have a sense that that image is actually
three dimensional.
It's an ancient, ancient old picture on purpose.
But you should still be able to see it in depth.
So that's the initial thing.
This became quite a parlor game and for that case,
for many, many decades, whenever you went to a party,
they would hand out to you a stereoscope, a handheld one,
and they would show you all kinds of images.
And you can even do this today when
you get on the internet to find such displays.
Now, then a very important discovery was made.
I shouldn't say discovery, really,
I should say an invention was made by Bela Julesz who came up
with a so-called random dot stereograms.
By the way, don't look at the bottom one, that just tells you
what it's going to look like.
There's nothing to look at the bottom, the bottom set.
Now if you look in the middle set,
that looks like a random dot stereogram.
And the idea here was that the only cue that you provide
is stereo cue, nothing else.
It's pure.
And so what can be done here, you
can take a section here, or the same on each side,
and simply move a few pixels, those images as a unit, over.
And when you do that, they're going to stick out in depth.
So now take you stereoscope and look in the middle display,
and you look through it, you should see something
sticking out in depth.
And the first question I'm going to ask you is how many of you
can see something stick out in depth?
What do you see?
You see
AUDIENCE: A square.
PROFESSOR: A little square sticking out?
All right, so now, don't try to look at the bottom one.
That simply tells you what the procedure was in that center
section where you see the square sticking out.
The pixels were moved a few steps inward from both the left
and to the right creating what is called a disparity.
And that's what the brain then can calculate for depth.
So now to provide you with the acid test,
go to the second page.
Now you look at the second page, everybody
see the letter on top?
You don't even have to look through the stereoscope,
obviously you see the letter E, right?
That's because that section is made darker.
But now if you do the same thing at the bottom, the only cue
you have is the disparity cue.
And the question comes up, what letter do you see there?
And let me just add, this can be used as a quick, general test.
You can present these two subjects,
and if it can present a whole bunch of different letters,
and if they can see the letters, that means they can see stereo.
If they cannot see the letters, then it looks like they may not
see stereo.
Now let me add one other fact here.
As you move these progressively closer to each other like this,
you increase the disparity.
And that causes the image to be seen at increasing depths.
Our sensitivity is so great that when
you look at this on a computer, a standard computer,
if you move those images just one
pixel from the left to the right, then
from right to the left, you will see it in depth.
And even monkeys can see as small step as one pixel.
So now how many of you can tell me
what was the letter at the bottom there?
AUDIENCE: H.
PROFESSOR: H, H, good.
Anybody not being able to see the letter?
Everybody sees it.
Well, you guys are lucky.
Because there are a significant number
of people in the world who lack stereopsis,
something like 5% to 10% of the population
lacks stereopsis for a variety of reasons.
We'll talk about that a bit more later.
But one is sometimes you're born,
and you're amblyopic in one eye.
Sometimes you are strabismic, which
means that your two eyes are not aligned,
which in commonplace language is often
called as being cross eyed or wall eyed.
Those types of people very seldom
will have stereoscopic depth perception,
even after it's corrected, especially
if the correction is made by the time you're 8 or 10 years old,
the correction won't help.
It has to be done much, much earlier.
All right, so that then is the very, very basics
of the stereo procedures.
And now another procedure that had
been developed more recently is one
which is called the auto stereogram.
So then if you go to the next page, and what you want to do
is you want to look at this horizontally like that
with the T on top.
And then just look at it at sort of, I don't know, maybe
about 20 inches from you, normal reading length.
And what you want to do is to look beyond it.
So stare beyond it.
And if you keep doing that for a while,
you will suddenly see an image, a three-dimensional image,
as this comes actually from a book called The Magic Eye.
There are several magic eye books
in which all kinds of displays are
done using these auto stereograms.
Does everybody see-- who can see what's sticking out?
OK, what do you see?
AUDIENCE: Was it a shark?
PROFESSOR: You see a shark?
OK.
Now let me see if any of you don't see it.
Keep staring at it.
Look beyond it.
Another thing that helps, if you look at it,
bring it a little closer to you so you
can look beyond it easier, gradually
move it back and forth.
And if you're patient, eventually you
may be able to do this.
The reason this is difficult is because you
have to uncouple the vergence in your two eyes.
You have to look beyond it slightly.
And, in fact, that is one of the reasons why
testing people for stereopsis, an auto stereogram
is not a very good procedure.
Whereas virtually everybody can use a stereoscope
without any trouble.
AUDIENCE: That's so cool.
PROFESSOR: Did you get it finally?
AUDIENCE: Yeah, that's so cool.
PROFESSOR: Yeah, all right so, if anybody
is really interested in this auto stereograms
is I say go to the store, the bookstore,
and get one of those magic eye books.
They're just a lot of fun.
And you can just leaf through it.
You don't even have to buy the book, just look through it
at the store.
And you'll see one interesting, clever image after the next.
All right, so that's the stereoscope.
And now let me explain to, I think
I've mentioned this briefly before, the principles involved
behind being able to see stereoscopic depth perception.
And what I've mentioned to you before was
that if you have the two eyes fixating
at a particular distance, if you then draw a circle around that,
that's sometimes called a Veith-Muller circle,
or the sometimes called a horopter, then any target,
like this one here, will hit equivalent points
on the retinal surface of the left and right eyes.
However, if you do the same thing,
and you put a target either beyond or closer
than the Vieth-Muller circle, then they're
going to hit nonequivalent points on the retinal surface.
So then by nonequivalency, we can do this and calculate this
as to where the image falls relative
to the central fixation spot in the foveola.
And so then, when these nonequivalent points are hit,
somehow the brain can measure this nonequivalence.
And that is then converted into an estimate
of where things are in depth.
Now the idea behind this was that
these nonequivalent points, that you
have on the retinal surface, can connect in the cortex
to single cells.
So they have a cell in the cortex that is binocular,
by virtue of the fact that inputs
from the left and right eyes.
But they don't necessarily have to come from equivalent points.
They can come from nonequivalent points.
And that may be then the mechanism whereby
it can tell you the degree of nonequivalence,
and, therefore, convert that into depth.
And, therefore, there could be single neurons
in the brain that are selective to certain depths.
And so people began to do all kinds of experiments with this.
And the way these experiments were done
is you presented images separately
to the left eyes and right eyes.
And you could then present them to both eyes at the same time
and vary the amount of disparity systematically
to see what kind of tuning function
you would get in the cortex.
So this, some of the most beautiful work of this
was done by a person called John [? Porgio ?]
And I will tell you briefly about some of his experiments.
So here we go with then.
We're going to look at the neural responses,
neural responses in V1 as initially done in the monkey.
So here's an example of a cell.
We have here different degrees of disparity.
And we have the neuron responding
each time they're four repeated trials.
And you can see the action potentials by these dark lines
here.
And what you do is you move the stimuli back and forth
across the two eyes, the way it's actually done,
you have a mirror.
And then you have two in this experiment.
You have two monitors, one to the left and one to the right.
And then you can set it up almost exactly
the same as what you would do with a stereoscope.
So this particular cell, as you can readily see,
responds best when there's zero disparity.
Now by contrast, here is a cell that responds vigorously
at the far disparity and not to the close one.
So then, when you do this, you can study hundreds of cells
to see what kinds of distributions
you have in the cortex for different degrees of disparity,
cell activity.
Now again, this hearkens back to what
I talked about with respect to color vision
where the question came up, if you want to see color,
how many receptors would you need that peak
at different wavelengths, right?
And one idea was that maybe you need as many photo receptors
as there are colors.
But in the end, it turned out that we have only three
of them.
And on the basis of that, we can recreate all the colors out
there.
Now the same thing applies to stereopsis.
So when this was done systematically,
here's an example of the tuning functions.
This one here is the same, very much the same,
as the first figure I showed you.
And so we have a bunch of different ones.
And if you then study, as I've said hundreds of cells,
you can come up with a distribution of this.
And what John [? Porgio ?] came up with,
he thought that there were four major classes.
And the relative amount of activity of these four
major classes then is used to compute all the very fine
differences in depth.
So there's a right on one.
This cell is right on the fixation spot.
And then you have near and far cells.
And you have some in between cells.
Initially he thought there were four classes.
Some people argue that there may be as many as six.
But at any rate, there's a limited number
on the basis of which you can calculate
almost an unlimited number of depths,
which is quite remarkable.
All right, so now, what you can next turn to
is to ask the question to what degree do various extrastriate
areas contribute to stereoscopic depth perception.
And some people thought that this
is a unique function for area MT,
some people argue that maybe it's
area V4, and so experiments were done in which it was examined
to what degree stereoscopic depth deception is altered when
you eliminate, say, area MT or you eliminate area V4.
So that is what's been done.
And you can think about it for a minute
and say, well, what do you think?
What do you think would happen in a monkey
once you no longer had area V4?
What you think would happen if the monkey no longer had
area MT?
Well, the results were actually quite surprising.
And they're shown here, same experiment as before.
The Bela Julesz random dot stereograms.
And then you're presenting one of four locations
of however many little area where,
like little square, that's sticks out in depth
and you vary the amount of depth that sticks out
by varying the number of pixels you
moved the images into this place.
And when that was done systematically,
this is what was found.
It was found that neither V4 lesion nor an MT lesion
cause a significant deficit.
The only deficit that was significant
had to do with a response latency.
And as I should have mentioned earlier,
like when we talked about the frog, one
of the very important things about processing depth,
again, is to be able to do it quickly.
So when you have that's frog and the fly is flying along,
he has to be very quick to compute it
so that he can catch it, right, as you had seen.
So in this case, what you see here,
that there is about a 20 millisecond difference
after V4 lesion, increase in latency, and quite a bit more,
almost 40, 30, 40 after and MT lesion.
So that contributes to some aspect of depth processing
in terms of being able to do it quickly.
But neither the MT or V4 are unique in processing
stereopsis.
It looks like that it is processed
in several different areas in the brain
and inspire conjoint computation that you
can arrive at the actual depth.
And it's by virtue of that joint computation
that you can do this very quickly.
So now you come to the next important depth
cue, which is called motion parallax.
This one is a capacity that we had acquired
in the course of evolution, which
is extremely potent and powerful.
And it's based on a very simple physical fact.
And the physical fact it that either when you are in motion,
or something in the environment is
in motion, the rate at which these images travel
across the retinal surface is heavily distance dependent.
And so let me demonstrate this concretely for you.
Here we have an eye that's fixed.
It's always looking straight ahead.
And we have two rods here, sorry, just one rod,
that we are going to move into position gradually from here
to here, and then back up as shown by the arrows here.
And we examine the range over which
these near and far and middle objects move
across the retinal surface when you engage in this motion
and the eye is stable.
And you can see that the far object moves
over a much shorter distance than the near object.
This you can readily do it yourself in an experiment.
You can stick out your thumb and move your head back and forth.
And you see that your thumb will move a lot more than the object
that you're looking at.
So now the same thing also applies when you actually
are engaged in the eye movements, which of course you
do all the time.
In this case, they eye is set up so that it's fixated initially
on this object and then tracks it to here
and then tracks it back.
And when you do that, you get the same kind of effect,
namely that the distance over which a far and a near object
move is quite different.
The near object moves over much, much greater distance
than the far object, even though the eyes are tracking.
So this, then, being a basic physical fact,
was then used in the course of evolution
to create mechanisms that are sensitive to this differential
motion.
And, of course, because of the rate of motion
also varies a little bit, it became
possible to create mechanisms to make that computation
to tell you where things are in depth.
So here I'm going to show you an actual demo of this
to make it clear to you.
In this case, again, we have a bunch of random dots, much
like in the Bela Julesz random dot stereograms
but just a single one.
And everybody agrees there's no depth here.
Is there any depth?
Do you see any depth?
So now what I'm going to do is I'm
going to set this image into rocking motion.
And when I do this, almost instantly, you're
going to see something in depth.
Are you ready?
So what you see here is are three levels, right,
very clearly.
In milliseconds, in 20 milliseconds, you can see this.
And let me explain to you why you see this.
If, OK, let me go back and do it again.
If I keep this stable, you can see that the dots move over
a great distance here, a lesser distance
here, and practically not at all here.
So there is a differential motion.
And the greater the motion, the closer the image
is in your analysis.
So that's called motion parallax.
And then what you can do actually,
you can play all kinds of games, do experiments
in which you can present this kind of image.
You can put this into each of the eyes separately.
And you can present this image alone
or you can present it paired with disparity for stereopsis.
And you can do each separately or you can do the two together.
So let's now first summarize the essence of motion parallax.
To derive depth information from motion parallax,
neurons are needed that provide information
about velocity and direction of motion
and perhaps also about differential motion.
Secondly, the majority of V1 cells
are direction and velocity selective,
as we had discussed before, and some appear also
to be selective for differential motion, which
I did not mention before.
But, indeed, there are such cells in the visual cortex.
Now, the third important point is
that such cells that are motion selective and direction
selective and selective for differential motion
are very, very common area MT.
So those are some of the very, very basic facts.
And now we can move on and ask what kind of brain activation
occurs by stereopsis and motion parallax
in normal and serial blind subjects
using a recently developed technique, which
is magnetic resonance imaging, functional, functional
magnetic resonance imaging.
So how do you do this kind of stuff?
Well, what you do here, here's an example,
you have a very large stereoscope
with a mirror at the end.
And you have a subject who is lying down.
And this whole unit, except not of course that part,
is put into the magnet.
And we have a magnet down here at MIT.
Most of you probably have seen that.
It's on the ground floor.
So you can do this, and then you can present those images here.
And so the stereoscope will present two images
and then you can vary this by rocking him back and forth
either to present only motion parallax
or present only stereopsis and to present both.
And so now the question is, this is a very primitive question
at this stage, where in the brain are
these processes analyzed?
And so you can find out what brain areas are
active by doing this repeatedly collecting the fMRI data
and then printing them out and looking
at them to see what happens.
So I'm going to show you a couple examples of that.
Here is the basic figure that the person
sees but done in such a way that you can see it.
Of course, he doesn't see anything like this.
He just sees different depths.
There are one, two, three, four, five, six,
seven different depths here.
And this rocks back and forth.
And then you can, as I say, present this only
with differential motion or you can present it only
with disparity or you can present it with both.
And then finally, as a control, what you do is
you can do the same thing.
But you don't have any depth of any sort.
You just have a flat surface rocking back and forth.
And then when you do data analysis,
you actually subtract the last one from the rest of the data
so that you're not looking at the data for the activation
just by the spots but for the activation that's
specific for stereopsis or motion parallax.
So now if you do this experiment,
here's an example of a normal subject
and a stereobind subject.
And we have here a sagittal cut adding up the images sideways.
And what you see here, this is posterior cortex, of course.
Here in the normal subject, when you present only motion
parallax, you only analyze motion parallax.
But you have a huge amount of activation in the visual areas.
And then if you do the same thing
when you do a binocular stereopsis only,
you also get a great deal of activation,
in quite similar set of areas.
And then the big crucial test comes up.
What happens if you present the stereo under monocular
conditions when you don't see stereo?
And if you do that, using this same calculation procedures,
there is no brain activation here.
And, therefore, what we see here is due, indeed, to the analysis
that we do for stereopsis.
Now if you do the same experiment
in a stereoblind subject who has been tested, on tests
similar we had shown you, when that person even
looks at it under binocular conditions,
there is no brain activation meaning
that this person doesn't have any mechanisms in the brain
to analyze stereopsis.
Now the fortunate thing is that we
have these several different mechanisms
for depth perception.
And so people who are stereoblind and have
no analysis for disparity, they can still see depth reasonably
well.
And, indeed, they can get driver's license and all
that, because we have all these other mechanisms that include,
that we have talked about, motion parallax.
So that then is one way of looking at it.
Now the other way to look at it, especially when I ask as well,
are the same brain areas doing both or what.
And so what you can do is instead
of doing a sagittal section, you can take sections coronally
like bang, bang, bang, bang like that
and see what that looks like.
And here's an example in which I've isolated the stereo.
And here are a bunch of sections.
And this shows the activation for stereopsis.
You can see there are all kinds of areas
that are being activated.
And then if you do the same thing
and just look at the parallax alone,
here we have the activation for that.
And then lastly here, we do both of them.
Now we can go back.
The way to look at the question, how
are these two areas, these two types of depth perception
analyses, differently activating in the brain.
And so I'm going to go back and forth
between stereo and parallax.
You can see, and that, you can see the difference.
Now some regions, which have a perfect
overlap, and there's some regions
that are quite separate.
Notable here are these areas here,
which are activated by stereo but not by parallax.
So this then can provide you with an initial idea
that there are some brain areas in which both of these
are analyzed together.
And there's some brain areas in which
are uniquely analyzed for either stereopsis or motion parallax
alone.
Now this tells you where it takes place in the brain.
But how it takes place requires a totally different approach.
Namely, the most comfortably to record
from individual neurons in various areas
just like I had shown you that nice work done
by John [? Porgio ?] recording from the one demonstrating
there that there are disparity selective neurons that
are tuned that then provide the hardware, if you will,
for being able to analyze stereoscopic depth.
So that then summarizes what I wanted
to tell you about motion parallax.
And now we are going to go on and talk
about yet another important depth
cue that is utilized by the brain, which is called shading.
Now remember that our ability to use light to illuminate things
is something that was practically nonexistent
for endless millions of years.
And so because of that, both animals and us,
we have to heavily rely on information
based on light coming from the sun, coming from above.
And shading is based on those millions of years of evolution
utilizing the fact that most of the light that
illuminates things comes from above.
So there are all kinds of nice examples of this.
And here is one of them.
What you can do here is you can take a bunch of disks
and set them up so.
You can do this on a computer to make the upper part
light and the lower part dark or the other way
around, the upper part dark and lower part light.
And all of you readily can see that these images,
the first and third row, seem to be protruding towards you.
And the images in the second and fourth row seem to be receding.
Now that is because the brain is interpreting
that on the basis of the fact that the light at least used
to come predominantly from above.
So that is the basic arrangement for seeing depth.
And now I'm going to give you some demonstrations
to indicate that this cue is actually quite powerful,
even when you would not necessarily expect it to be.
So these shading cues have also been extensively used
in art work to provide an impression of depth.
And I will show you some examples
that will give you a sense of how that is done.
So let me make one more point before I proceed that namely it
is, indeed, the degree of illumination
that's crucial here.
We have the same change from red to, sorry,
from some greenish to yellowish.
And you have no sense of depth here whatsoever.
In other words, you do need the shading information,
meaning the amount of light that's
being reflected from various surfaces, that
is crucial for perceiving depth.
So now what we are going to do is
we are going to present a series of slides that will highlight
the power that shading has for the perception of depth.
So here is an example of how we do this.
And the reason I'm showing this in some
detail because if you really are interested in stuff like this,
you can do all this on your own computer.
You can play games, endless games with it.
You can spend hours and hours having a lot of fun
thinking about how depth works on the basis of shading.
So what you have here are a whole bunch of disks.
And each of these can be shaded differently
by many different computer programs.
So that's what you can do.
That's the basics.
So now what we can do is we can play a game.
And we can say, present just two different objects here.
But we're going to present them repeatedly on a big display.
And then we can shade these differently as we please.
So here is a whole bunch of them.
And all the rows, this, the first, third, and so on rows
are the same shape and the second, fourth
and so on are the other shape, these two shapes.
So we only have two shapes here that are juxtapositioned.
Now what we can do is say, well, this is a peculiar sensation.
I have a vague sense that there's
something maybe in the third dimension.
But it's not too well defined, because this
is not in accordance with the rules and laws of shading
of light coming from above.
So now what we can do instead, we can selectively
shade these to be in accordance with the rules of light coming
from above to create shading and depth.
And when you do that, here's an example of that.
What you can see here is a very compelling image
of these protruding elements, sort of protruding to the left,
right?
Everybody see, have a strong sense of depth here?
So now what you can do is you can play with it
and decide, well, can we do something
that, keeping the very, very same shapes,
shade them differently and see what
it does to our perception of depth.
And so what we are going to do next
is we're going to take each of these elements
here, the same ones here, and we're
going to reverse the contrast.
You see the contrast here on top is white
and the bottom is black.
So we're going to reverse that contrast.
And when we do so, the question is what are you going to see.
And if you do that, low and behold,
you still have a strong sense of depth.
But it's a very confusing sense.
You may see sometimes these objects pointing to the left
and sometimes to the right.
It's unstable, because you're confusing those computations
that have evolved over millions of years
for interpreting depth in terms of shading.
Now you can play also some additional games.
You can make this even more complicated, make more changes,
and here is another one.
You still have a feeling of depth.
But it's totally confusing.
It's very hard.
You can't organize it any way, because it
is not in accordance with the law of light
coming from above to a real object.
And lastly, you can also make this
so that it would be in accordance with the laws.
But you can change it around so that you
get a completely different perception,
a strong sense of depth.
It's still the very, very same elements
that you had seen before.
But now the shading is done, again, differently.
And then now gives you, again, a unified sense
of a display that is not conflicting.
Because in this case, it's in accordance with that
some of the basic principles of shading.
So now, what we can do next, having talked about stereo
and we have talked about shading,
is to look at some more of the demos.
So let's go back to the stereoscope.
And let's go back to the handouts.
And so if you now come to the next page that has a heading
called stereo and shading.
So, again, take the stereoscope and we
are going to look at these in steps.
So let's start by looking at the top display
first, which is called stereo only.
So if you look at that, first of all,
if you just look at it without the stereoscope,
you see pretty much a sort of flat display
of a truncated pyramid.
Then if you put the stereoscope there and look through it,
you should see, if you look at it for a little while,
that one of those sticks out towards you
and the other one seems to recede.
Does everybody see that?
So let's stop there for a minute, because I
want to add one more fact here, which
I should have mentioned earlier.
So what you do here is-- so you have
these two displays like that.
And you have one image here and another image here.
When you-- this is greatly exaggerated,
these coming together like this.
That means it's going to stick out
towards you as you look at it.
But if you do the opposite like that,
they're further apart than the rest of them.
Then actually you see it receding.
And that's why, if you now take away the stereoscope
and look at it, you can see that the top left image is
in each, or facing towards each other,
whereas the other ones are facing away from each other.
And that's what creates, that's what the brain interprets,
as protruding versus receding using the stereoscope.
So that is very similar, in a way,
to what happens with shading.
So now if you look at the second image
there, first without the stereoscope, what you see here,
again, is one that sticks out just
like in the original display.
And the rest of them are receding.
That's because the shading, the one that sticks out
is light on top and dark on the bottom.
And it's the obverse for the other ones.
Now if you do the same thing looking
through the stereoscope, what you will see
is still some degree of depth, but it's not very pronounced.
Because there is no corresponding disparity
information.
But now if you look at the third display,
where stereo and shading are in harmony, then what you see
is an extremely compelling dramatic sense
of depth with the top left one sticking out
towards you and the other three receding.
So shading appeared to have added to the compelling nature
of the depth that you see through the stereoscope.
Now the last image in here is that we put stereo and shading
and conflict with each other.
And when you do that, you can look at it, first with just one
eye, then with the other eye.
When you look at it with both of them, you, for a while,
you see something unstable.
And when you see it well, eventually, you
realize that there is a conflict there
because of the shading and the stereo
being in opposition to each other.
Now then, this kind of effect, if you go to the next page,
we're going to go now to page five, six, and seven.
Again, what you need to do here is
to look at it sideways with the F's on top.
And when you look at this, those of you who can use your eyes so
that they are divergent, and you look beyond it,
then this is very much like, or it is the same actually,
as an auto stereogram.
So if you look at this for a while, and you look beyond it,
eventually it's going to gel.
And when it gels, what you should
see is where the F's are, the images
are protruding towards you, and the others are receding.
Now it may take you awhile.
It's much more difficult than what we just
did with the stereoscope.
But you should be able to see that.
How many of you are able to actually see those images?
A little bit di-- move it back and forth a little bit slowly.
And maybe eventually you manage it.
So as I say, where the F's are you
see these images-- these truncated pyramids protruding
towards you.
And the rest of them are receding.
If you have difficulty seeing this,
I'm not surprised, because it takes a lot of practice.
But once you get a sense of it, I
think that you will enjoy doing this and actually
showing it to some of your friends.
So then if you go to the next page,
there we have added shading.
And the shearing is the same everywhere.
But the stereo cues are not.
Once again, what happens is that they
are stereo cues where the F's are stick out
towards you much greater than the others.
They stick out a lot less because of the added stereo.
And then, in the last demo there, the last page, we
are putting, just like in that figure with a stereoscope,
we are putting them into opposition with each other.
And so when you look at these, this
would be very difficult to see for a while.
Because there's a tendency to see it differently
for stereo and for motion parallax.
And so it's going to be an unstable percept.
So what you can do then is you can play around
with this at your leisure and especially once you become more
proficient looking at all the stereograms,
if you go and get one of these magic eye books to look at,
you will be able to see these displays as well.
So now, this is the one was the first one that I showed you.
As I said, this one, with the F's, are the ones
that should stick out closest to you.
Once you see that, then you can go on to the next
to add the shading or subtract the shading from it.
So now, an interesting question that arises
is to what degree are we able, or are animals able,
to integrate these different kinds of depth cues.
And in particular, in this case, you're
going to ask the question, what about integrating stereopsis,
parallax, and shading.
So the experiment is one done on monkeys
in which you can present these cues either singly
or in combination.
And we can ask the question, well,
does the monkey do better with one or the other,
or does he integrate really and does really much better
when you provide all three cues.
And so here is a procedure.
What you do here, again, you have a rocking display
like this, and you can present this
either with shading as shown here
or with motion parallax where it rocks back and forth,
and lastly also with stereopsis.
So if you do that, the results you get are quite dramatic.
What happens is shown here as a percent correct performance
and here is the latency in milliseconds.
And it shows that the monkey does extremely well when
you present-- this is percent correct.
This is degrees of disparity.
The monkey does extremely well when you present all three cues
and does worse when you present each of those cues alone.
Even more dramatic is the fact, and this
is-- I keep coming back to this, that the ability for us
to respond quickly to things is very important for survival.
And here what we can see is that when you present all three
cues, performance is much, much, much, much faster than
when you present each of those alone.
And, of course, as you might expect,
when you present parallax only, because that's
motion over time, that takes the longest to do.
So even though motion parallax cues are great,
it became important in the course of evolution
to create mechanisms that can detect these things more
quickly and more efficiently.
So now we come to yet another cue
that we know very little about at the level of the brain
or single units, because it's so complicated, which
is called perspective.
But I want you to just be aware of it and have a sense of it.
And here is one of those cartoon examples
that gives you a very strong sense of depth.
And you almost cringe.
If you were there, you would worry
that you would be falling down.
This is done strictly by virtue of perspective.
It's very similar to what you encounter all the time when
you're driving down a road and the road seems to converge,
even though you're not aware of it.
But that's what's happening on the rental surface.
Because things further away are smaller than things
that are close by.
And that's when you look down a railroad track,
the same thing happens, even though you
know that the railroad track is not converging,
it's going parallel.
But because of the distances involved,
that's what falls on the retina.
And you're smart enough to know that even though that's
what falls on the retina, you can
make the right kind of interpretation.
Conversely, you can also compute the depth
on the basis of that kind of convergence.
Now here's another example of that, a much simpler way
that people can do with experiments.
Here we have a bunch of dots.
And we have two basic cues that have to do with perspective.
One of them is this gradually decreasing size of these dots.
I should say elongated disks, if you will,
and also that they are converging
much like a railroad track converges.
And so we have a very strong sense
of having a third dimension here.
Now the fact that this is so strong
can be mitigated by adding a few things here.
If you add some more dots, it's not question as dramatic.
And then if you start mixing up the sizes,
you are beginning to lose it.
And then if you totally mix it up,
then you have no sense of that left at all.
So it is that progression of steps and sizes
and whatnot that gives you the sense
of the depth of the images that you're looking at.
Now here's another converse example
of this that is an illusory effect that what you see here
is three barrels, if you will.
And this barrel is a lot bigger than this barrel, right,
or is it?
Well, so what we're going to do here,
we have an inducing element here by this hallway, if you will,
with a door at the end.
And we're going to remove this hallway
keeping the barrels exactly as they are.
And if you do that, low and behold,
those barrels are all the same size.
It's induced by virtue of the surround that
gives you a false sense of depth.
So now let me show you another picture because
of the purpose behind this.
This a picture that's in a museum
in Worcester, Massachusetts.
And it was created by a fellow called Edward Savage.
And it's a pretty unpleasant picture.
But the main reason I'm showing this to you
is that there seems to be very poor sense of depth
in this picture.
Now the reason this is interesting
is because when artists began, centuries
ago in the 13th, 12th centuries, draw things,
they did not have a concept of an understanding of how
to create depth, a third dimension, in their drawings.
So what they did eventually, they came up
with a so-called vanishing point,
and they drew very much like what we had here.
Lines that converged at a point and then
scaled the images accordingly rather
than keeping it the same size.
And that way you got a good sense of depth.
So now that has a number of interesting stories
about it that we are going to discuss
next time you talk about the perception of shapes,
OK, patterns.
But I will leave that discussion until that.
What I'm going to do next, however, I'm
going to try to give you a sense of how important stereopsis can
be for the perception of fine depths.
And so to do that, what I'm going--I'm going to show you
actually a film.
And here what we have is a so-called needle test.
What you have here is a fine needle protruding.
And here we have a bunch of different size
circular openings, a little bit like a needle, but it's round.
And the task is to take these one at a time and hang them up.
And one can time how quickly you can do that,
or you can make a film to see how well you can do it.
And then what we can do is we can
test the subject under binocular conditions,
and test them under monocular conditions.
So I'm going to show you a film of this, actually two films.
It will just take just few seconds to do it.
OK, be ready, it's going to come up in a second.
OK, here's the subject under binocular viewing conditions.
So that's the condition under binocular viewing.
And now I'm going to show it to you, same subject, same time,
but with one eye closed off.
So that then just even looking at it
without taking any careful measurements.
It's obvious that it's much, much
more difficult to thread a needle under monocular
than under binocular viewing conditions.
And so what you can do is when you go home,
and next time you want to sew something up,
try threading the needle with one eye closed and with the two
eyes open.
And you will see immediately what a huge difference it is.
And that difference, therefore, is due
to you're having the mechanism of steropsis.
Just a few seconds here.
Another test that has been used in a similar fashion which
allows you to actually calculate exactly what your error is
in reaching, you can have a subject
sit in front of one of these touch panels,
and then do this experiment either binocularly or
monocularly.
And after he presses this a dot comes up,
and then the person has to touch it.
And you have about 30 or 40 trials like that.
And then you have recorded where the person touched.
And, therefore, you can calculate the error
between where we touched and where the dot is.
And then, again, you get a huge effect
between monocular and binocular viewing conditions.
Now when you come to monocular and binocular
viewing conditions, another thing important to test
is to what degree a person who does or does not
have stereopsis is capable of integrating information
between the two eyes.
So to do that, we have here examples
of what is called binocular integration.
So what we do here, again, you can use a stereoscope.
You look at a monitor.
And this represent the left eye, this is the right eye.
And you flash these on.
If you integrate this, this is what you see.
This would be actually what you would
show in the control part of the experiment.
So you see the Star of David.
And if a subject is shown this, and they
don't see the Star of David, you worry
that their ability to integrate the information between the two
eyes is deficient.
And I would say 90% of the cases, those people who
are deficient on this also show major deficiency
in stereoscopic viewing.
Now another way to do this is an experiment
in which you can present two words here, sud and try,
so the two are separate.
And when you present them simultaneously,
you actually see the word sturdy.
So you ask the subject, please tell us what do you see.
What is it word you see.
And the subject says sud.
And the subject says try, then you
know that that subject sees, if he says try,
he sees mostly with his right and prefers it,
doesn't see too well with his left eye.
If he says sturdy, then he integrates the two.
And, therefore, you can safely say
that this guy has very good integration between the two
eyes.
So what I would like to do next then
is to provide you with any questions that you have.
This was a complicated topic, that you have,
and then we are going to summarize.
Does anybody have a question about motion parallax,
stereopsis, and so on?
Let me maybe add one more important factor.
Your eyes are separated only by so many centimeters.
Now, can you think of an animal where
there's a much larger separation?
AUDIENCE: Hammerheads.
PROFESSOR: The hammerhead shark.
Yeah, that has a separation of over a foot between the two
eyes.
And so you could ask the question, why on earth did
that animal evolve to such a huge separation between the two
eyes?
Well, that brings one to yet another interesting point.
This I think may have started during the Second World War.
It was realized that when you're flying
over some territory, where there are all kinds of weapons
and whatnot, which are well camouflaged,
that just looking down at them, you can't see them.
But obviously if you're going to have a tank
or you're going to have a gun or other that may be more
like a cannon, it sticks out of the ground.
So it was discovered that if you had in your airplane
two lenses which are far apart, that
would greatly magnify the depth.
You could defeat that camouflage.
And you could find those weapons down there
by virtue of the fact they're sticking out of the ground.
So the fact then is that the more you separate the images
from the two eyes, if you will, or your two cameras,
the more likely it is that you can calculate
the disparity of information between the two images.
So that then is probably one of the reasons, not
the sole reason, but maybe one of the reasons
why, in some animals, is an excessive separation
between the two eyes.
And that brings me to get me to yet another point, which
is that stereopsis actually works best at relatively
short distances, like threading a needle.
It doesn't work too well beyond, I don't know, 10 feet or so.
It becomes progressively less effective.
But at short distance, it's very effective.
And so I presume also many animals
that have to hunt for food are able to utilize
the mechanism of stereopsis, because everything
is at a close distance when they hunt for food on the ground.
And by contrast, when you talk about motion parallax,
that works extremely well over very long distances.
So does anybody have any questions about motion parallax
or about steropsis?
Oh, once again, I'm crystal clear, huh?
So, therefore, I think it's time for us
to summarize what we had covered today.
First of all, there are numerous mechanisms
that have emerged for analyzing depth.
And they include the ocular motor cues, which are vergence
an accommodation and then the binocular cue of stereopsis
and then the monocular cues of parallax shading
and perspective.
Then you have several cortical structures
that process stereopsis.
You don't have one specific brain area
that uniquely does this.
The number of disparities that are represented in the brain,
as studies in the area of the one by John [? Porgio, ?]
is limited.
And maybe, maybe four, but there may be six,
but certainly there are not a large number of them.
And so it's analogous to the way things
had been resolved for us to be able to process color.
Utilizing motion parallax for depth processing
necessitates neuron specific for direction, velocity,
and differential velocity.
Several areas getting V1 and MT process motion parallax,
which I did not say.
But indeed, if you make a lesion in area MT,
you go get a deficit in motion parallax, even
thought you don't get a major deficit in steropsis.
Now, area MT combines the analysis
of motion parallax, depth, and flicker.
However, these analyses are also carried out
by several other structures as I've already said.
And lastly, little is know at present
about the manner in which information about shading
and prospective are analyzed in the brain.
And hopefully, that will be one of the future tasks
by neuroscientists.
And so if any of you ever get involved in neuroscience,
this certainly is a big open area
that we hope people will start to analyze.
So that then is the essence of what I wanted to cover today.
And once again, if any of you has a question, please,
please don't hesitate to ask.
I'll be very happy to answer them.
OK, lastly then, did everybody sign the attendance sheet?
If not, please come up after the class and sign your name to it.
Very good.
So next time then, we are going to talk
about pattern perception.
And hopefully you will find that also interesting.