The following content is provided
under a Creative Commons license.
Your support will help MIT OpenCourseWare continue
to offer high quality educational resources for free.
To make a donation or view additional materials
from hundreds of MIT courses, visit MIT OpenCourseWare
at ocw.mit.edu.
PROFESSOR: We established that, essentially, what we want to do
is to describe the properties of a system that
is in equilibrium.
And a system in equilibrium is characterized
by a certain number of parameters.
We discussed displacement and forces
that are used for mechanical properties.
We described how when systems are in thermal equilibrium,
the exchange of heat requires that there is temperature
that will be the same between them.
So that was where the Zeroth Law came and told us
that there is another function of state.
Then, we saw that, from the First Law,
there was energy, which is another important function
of state.
And from the Second Law, we arrived at entropy.
And then by manipulating these, we
generated a whole set of other functions, free energy,
enthalpy, Gibbs free energy, the grand potential, and the list
goes on.
And when the system is in equilibrium,
it has a well-defined values of these quantities.
You go from one equilibrium to another equilibrium,
and these quantities change.
But of course, we saw that the number of degrees of freedom
that you have to describe the system
is indicated through looking at the changes in energy, which
if you were only doing mechanical work,
you would write as sum over all possible ways of introducing
mechanical work into the system.
Then, we saw that it was actually
useful to separate out the chemical work.
So we could also write this as a sum
of an alpha chemical potential number of particles.
But there was also ways of changing
the energy of the system through addition of heat.
And so ultimately, we saw that if there
were n ways of doing chemical and mechanical work,
and one way of introducing heat into the system, essentially
n plus 1 variables are sufficient to determine
where you are in this phase space.
Once you have n plus 1 of that list,
you can input, in principle, determine others
as long as you have not chosen things
that are really dependent on each other.
So you have to choose independent ones,
and we had some discussion of how that comes into play.
So I said that today we will briefly
conclude with the last, or the Third Law.
This is the statement about trying
to calculate the behavior of entropy
as a function of temperature.
And in principle, you can imagine
as a function of some coordinate of your system-- capital X
could indicate pressure, volume, anything.
You calculate that at some particular value
of temperature, T, T the difference in entropy
that you would have between two points
parametrized by X1 and X2.
And in principle, what you need to do
is to find some kind of a path for changing parameters
from X1 to X2 and calculate, in a reversible process, how
much heat you have to put into the system.
Let's say at this fixed temperature, T, divide by T. T
is not changing along the process from say X1 to X2.
And this would be a difference between the entropy
that you would have between these two
quantities, between these two points.
You could, in principle, then repeat this process
at some lower temperature and keep going all the way down
to 0 temperature.
What Nernst observed was that as he went through this procedure
to lower and lower temperatures, this difference-- Let's
call it delta s of T going from X1 to X2 goes to 0.
So it looks like, certainly at this temperature,
there is a change in entropy going from one to another.
There's also a change.
This change gets smaller and smaller
as if, when you get to 0 temperature,
the value of your entropy is independent of X. Whatever
X you choose, you'll have the same value of entropy.
Now, that led to, after a while, to a more ambitious version
statement of the Third Law that I will write down,
which is that the entropy of all substances at the zero
of thermodynamic temperature is the same and can be set to 0.
Same universal constant, set to 0.
It's, in principle, through these integration
from one point to another point, the only thing
that you can calculate is the difference between entropies.
And essentially, this suggests that the difference
between entropies goes to 0, but let's be more ambitious
and say that even if you look at different substances
and you go to 0 temperature, all of them have a unique value.
And so there's more evidence for being
able to do this for different substances via what
is called allotropic state.
So for example, some materials can exist potentially
in two different crystalline states that
are called allotropes, for example, sulfur
as a function of temperature.
If you lower it's temperature very slowly,
it stays in some foreign all the way down to 0 temperature.
So if you change its temperature rapidly, it stays in one form
all the way to 0 temperature in crystalline structure
that is called monoclinic.
If you cool it very, very slowly,
there is a temperature around 40 degrees Celsius
at which it makes a transition to a different crystal
structure.
That is rhombohedral.
And the thing that I am plotting here,
as a function of temperature, is the heat capacity.
And so if you are, let's say, around room temperature,
in principle you can say there's two different forms of sulfur.
One of them is truly stable, and the other is metastable.
That is, in principle, if you rate what sufficiently
is of the order of [? centuries ?],
you can get the transition from this form to the stable form.
But for our purposes, at room temperature,
you would say that at the scale of times
that I'm observing things, there are these 2 possible states
that are both equilibrium states of the same substance.
Now using these two equilibrium states,
I can start to test this Nernst theorem generalized
to different substances.
If you, again, regard these two different things
as different substances.
You could say that if I want to calculate the entropy just
slightly above the transition, I can come from two paths.
I can either come from path number one.
Along path number one, I would say
that the entropy at this Tc plus is obtained
by integrating degree heat capacity,
so integral dT Cx of T divided by T.
This combination is none other than dQ.
Basically, the combination of heat capacity dT
is the amount of heat that you have
to put the substance to change its temperature.
And you do this all the way from 0 to Tc plus.
Let's say we go along this path that
corresponds to this monoclinic way.
And I'm using this Cm that corresponds to this as opposed
to 0 that corresponds to this.
Another thing that I can do-- and I made a mistake
because what I really need to do is to, in principle,
add to this some entropy that I would
assign to this green state at 0 because this is the difference.
So this is the entropy that I would
assign to the monoclinic state at T close to 0.
Going along the orange path, I would
say that S evaluated at Tc plus is
obtained by integrating from 0.
Let's say to Tc minus dT, the heat
capacity of this rhombic phase.
But when I get to just below the transition,
I want to go to just above the transition.
I have to actually be put in certain amount of latent heat.
So here I have to add latent heat L, always
at the temperatures Tc, to gradually make the substance
transition from one to the other.
So I have to add here L of Tc.
This would be the integration of dQ,
but then I would have to add the entropy that I would assign
to the orange state at 0 temperature.
So this is something that you can do experimentally.
You can evaluate at these integrals, and what you'll find
is that these two things are the same.
So this is yet another justification
of this entropy being independent
of where you start at 0 temperature.
Again at this point, if you like,
you can by [INAUDIBLE] state that this
is 0 for everything will start with 0.
So this is a supposed new law of thermodynamics.
Is it useful?
What can we deduce from that?
So let's look at the consequences.
First thing is so what I have established
is that the limit as T goes to 0 of S,
irrespective of whatever set of parameters
I have-- so I pick T as one of my n plus one coordinates,
and I put some other bunch of coordinates here.
I take the limit of this going to 0.
This becomes 0.
So that means, almost by construction,
that if I take the derivative of S with respect to any of these
coordinates-- if I take then the limit as T goes to 0,
this would be fixed T. This is 0.
Fine.
So basically, this is another way
of stating that entropy differences go through 0.
But it does have a consequence because one thing that you will
frequently measure are quantities,
such as extensivity.
What do I mean by that?
Let's pick a displacement.
Could be the length of a wire.
Could be the volume of a gas.
And we can ask if I were to change temperature,
how does that quantity change?
So these are quantities typically called alpha.
Actually, usually you would also divide
by x to make them intensive because otherwise x
being extensive, the whole quantity
would have been extensive.
Let's say we do this at fixed corresponding displacement.
So something that is very relevant
is you take the volume of gas who changes temperature
at fixed pressure, and the volume of the gas
will shrink or expand according to this extensive.
Now, this can be related to this through Maxwell relationship.
So let's see what I have to do.
I have that dE is something like Jdx plus,
according to what I have over there, TdS.
I want to be able to write a Maxwell relation that
relates a derivative of x.
So I want to make x into a first derivative.
So I look at E minus Jx.
And this Jdx becomes minus xdJ.
But I want to take a derivative of x with respect not
s, but with respect to T. So I'll do that.
This becomes a minus SdT.
So now, I immediately see that I will have a Maxwell
relation that says dx by dT at constant J
is the same thing as dS by dJ at constant T.
So this is the same thing by the Maxwell relation as dS
by dJ at constant T. All right?
This is one of these quantities, therefore, as T goes 0,
this goes to 0.
And therefore, the expansivity should go to 0.
So any quantity that measures expansion, contraction,
or some other change as a function of temperature,
according to this law, as you go through 0 temperature,
should go to 0.
There's one other quantity that also goes to 0,
and that's the heat capacity.
So if I want to calculate the difference between entropy
at some temperature T and some temperature
at 0 along some particular path corresponding
to some constant x for example, you
would say that what I need to do is
to integrate from 0 to T the heat that I have
to put into the system at constant x.
And so if I do that slowly enough,
this heat I can write as CxdT.
Cx, potentially, is a function of T.
Actually, since I'm indicating T as the other point
of integration, let me call the variable of integration T
prime.
So I take a path in which I change temperature.
I calculate the heat capacity at constant x.
Integrate it.
Multiply by dT to convert it to T, and get the result.
So all of these results that they have been formulating
suggest that the result that you would get as a function of T,
for entropy, is something that as T goes to 0, approaches 0.
So it should be a perfectly nice, well-defined value
at any finite temperature.
Now, if you integrate a constant divided
by T, divided by dT, then essentially the constant
would give you a logarithm.
And the logarithm would blow up as we go to 0 temperature.
So the only way that this integral does not
blow up on you-- so this is finite only if the limit as T
goes to 0 of the heat capacities should also go to 0.
So any heat capacity should also essentially vanish
as you go to lower and lower temperature.
This is something that you will see many, many times when
you look at different heat capacities
in the rest of the course.
There is one other aspect of this
that I will not really explain, but you can go and look
at the notes or elsewhere, which is that another consequence is
unattainability of T equals to 0 by any finite set
of operations.
Essentially, if you want to get to 0 temperature,
you'll have to do something that cools you step by step.
And the steps become smaller and smaller,
and you have to repeat that many times.
But that is another consequence.
We'll leave that for the time being.
I would like to, however, end by discussing some distinctions
that are between these different laws.
So if you think about whatever could
be the microscopic origin, after all, I
have emphasized that thermodynamics
is a set of rules that you look at substances as black boxes
and you try to deduce a certain number of things
based on observations, such as what Nernst did over here.
But you say, these black boxes, I
know what is inside them in principle.
It's composed of atoms, molecules, light, quark,
whatever the microscope theory is
that you want to assign to the components of that box.
And I know the dynamics that governs
these microscopic degrees of freedom.
I should be able to get the laws of thermodynamics
starting from the microscopic laws.
Eventually, we will do that, and as we do that,
we will find the origin of these different laws.
Now, you won't be surprised that the First Law is intimately
connected to the fact that any microscopic set of rules
that you write down embodies the conservation of energy.
And all you have to make sure is to understand
precisely what heat is as a form of energy.
And then if we regard heat as another form of energy,
another component, it's really the conservation law
that we have.
Then, you have the Zeroth Law and the Second Law.
The Zeroth Law and Second Law have to do with equilibrium
and being able to go in some particular direction.
And that always runs a fall of the microscopic laws of motion
that are typically things that are time reversible where
as the Zeroth Law and Second Law are not.
And what we will see later on, through statistical mechanics,
is that the origin of these laws is
that we are dealing with large numbers of degrees of freedom.
And once we adapt the proper perspective
to looking at properties of large numbers of degrees
of freedom, which will be a start
to do the elements of that [? prescription ?] today,
the Zeroth Law and Second Law emerge.
Now the Third Law, you all know that once we
go through this process, eventually for example,
we get things for the description of entropy, which
is related to some number of states
that the system has indicated by g.
And if you then want to have S going through 0,
you would require that g goes to something
that is order of 1-- of 1 if you like-- as T goes to 0.
And typically, you would say that systems
adopt their ground state, lowest energy state, at 0 temperature.
And so this is somewhat a statement
about the uniqueness of the state of all possible systems
at low temperature.
Now, if you think about the gas in this room,
and let's imagine that the particles of this gas
either don't interact, which is maybe a little bit unrealistic,
but maybe repel each other.
So let's say you have a bunch of particles
that just repel each other.
Then, there is really no reason why,
as I go to lower and lower temperatures,
the number of configurations of the molecules should decrease.
All configurations that I draw that they don't overlap
have roughly the same energy.
And indeed, if I look at say any one of these properties,
like the expansivity of a gas at constant pressure
which is given in fact with a minus sign.
dV by dT at constant pressure would be the analog of one
of these extensivities.
If I use the Ideal Gas Law-- So for ideal gas,
we've seen that PV is proportional to let's
say some temperature.
Then, dV by dT at constant pressure
is none other than V over T. So this would give me
1 over V, V over T. Probably don't need it on this.
This is going to give me 1 over T.
So not only doesn't it go to 0 at 0 temperature,
if the Ideal Gas Law was satisfied,
the extensivity would actually diverge
at 0 temperature as different as you want.
So clearly the Ideal Gas Law, if it was applicable
all the way down to 0 temperature,
would violate the Third Law of thermodynamics.
Again, not surprising given that I have told you
that a gas of classical particles with repulsion
has many states.
Now, we will see later on in the course
that once we include quantum mechanics, then as you
go to 0 temperature, these particles
will have a unique state.
If they are bosons, they will be together in one wave function.
If they are fermions, they will arrange themselves
appropriately so that, because of quantum mechanics,
all of these laws would certainly breakdown at T equals
to 0.
You will get 0 entropy, and you would get consistency
with all of these things.
So somehow, the nature of the Third Law
is different from the other laws because its validity
rests on being able to be living in a world
where quantum mechanics applies.
So in principle, you could have imagined some other universe
where h-bar equals to 0, and then
the Third Law of thermodynamics would not hold there
whereas the Zeroth Law and Second Law would hold.
Yes?
AUDIENCE: Are there any known exceptions to the Third Law?
Are we going to [? account for them? ?]
PROFESSOR: For equilibrium-- So this
is actually an interesting question.
What do I know about-- classically,
I can certainly come up with lots of examples that violate.
So your question then amounts if I say that quantum mechanics is
necessary, do I know that the ground
state of a quantum mechanical system is unique.
And I don't know of a proof of that for interacting system.
I don't know of a case that's violated, but as far as I know,
there is no proof that I give you an interacting Hamiltonian
for a quantum system, and there's a unique ground state.
And I should say that there'd be no--
and I'm sure you know of cases where the ground state is not
unique like a ferromagnet.
But the point is not that g should be exactly one,
but that the limit of log g divided
by the number of degrees of freedom
that you have should go to 0 as n goes to infinity.
So something like a ferromagnet may have many ground states,
but the number of ground states is not
proportional to the number of sites, the number of spins,
and this entity will go to 0.
So all the cases that we know, the ground state
is either unique or is order of one.
But I don't know a theorem that says that should be the case.
So this is the last thing that I wanted
to say about thermodynamics.
Are there any questions in general?
So I laid out the necessity of having
some kind of a description of microscopic degrees of freedom
that ultimately will allow us to prove
the laws of thermodynamics.
And that will come through statistical mechanics, which
as the name implies, has to have certain amount
of statistic characters to it.
What does that mean?
It means that you have to abandon a description of motion
that is fully deterministic for one
that is based on probability.
Now, I could have told you first the degrees of freedom
and what is the description that we
need for them to be probabilistic,
but I find it more useful to first lay out
what the language of probability is
that we will be using and then bring
in the description of the microscopic degrees of freedom
within this language.
So if we go first with definitions--
and you could, for example, go to the branch of mathematics
that deals with probability, and you will encounter something
like this that what probability describes is a random variable.
Let's call it X, which has a number of possible outcomes,
which we put together into a set of outcomes, S.
And this set can be discrete as would be the case if you were
tossing a coin, and the outcomes would
be either a head or a tail, or we were throwing a dice,
and the outcomes would be the faces 1 through 6.
And we will encounter mostly actually cases
where S is continuous.
Like for example, if I want to describe the velocity of a gas
particle in this room, I need to specify the three components
of velocity that can be anywhere, let's say,
in the range of real numbers.
And again, mathematicians would say that to each event,
which is a subset of possible outcomes,
is assigned a value which we must satisfy the following
properties.
First thing is the probability of anything
is a positive number.
And so this is positivity.
The second thing is additivity.
That is the probability of two events, A or B,
is the sum total of the probabilities
if A and B are disjoint or distinct.
And finally, there's a normalization.
That if you're event is that something should happen
the entire set, the probability that you assign to that is 1.
So these are formal statements.
And if you are a mathematician, you start from there,
and you prove theorems.
But from our perspective, the first question to ask
is how to determine this quantity probability
that something should happen.
If it is useful and I want to do something real world about it,
I should be able to measure it or assign values to it.
And very roughly again, in theory,
we can assign probabilities two different ways.
One way is called objective.
And from the perspective of us as physicists
corresponds to what would be an experimental procedure.
And if it is assigning p of e as the frequency of outcomes
in large number of trials, i.e. you
would say that the probability that event A is obtained
is the number of times you would get outcome A divided
by the total number of trials as n goes to infinity.
So for example, if you want to assign a probability that when
you throw a dice that face 1 comes up, what you could do
is you could make a table of the number of times
1 shows up divided by the number of times you throw the dice.
Maybe you throw it 100 times, and you get 15.
You throw it 200 times, and you get--
that is probably too much.
Let's say 15-- you get 35.
And you do it 300 times, and you get something close to 48.
The ratio of these things, as the number
gets larger and larger, hopefully
will converge to something that you would call the probability.
Now, it turns out that in statistical physics,
we will assign things through a totally different procedure
which is subjective.
If you like, it's more theoretical,
which is based on uncertainty among all outcomes.
Because if I were to subjectively assign
to throwing the dice and coming up with value of 1,
I would say, well, there's six possible faces for the dice.
I don't know anything about this dice being loaded,
so I will say they are all equally alike.
Now, that may or may not be a correct assumption.
You could test it.
You could maybe throw it many times.
You will find that either the dice is loaded or not loaded
and this is correct or not.
But you begin by making this assumption.
And this is actually, we will see later on, exactly
the type of assumption that you would
be making in statistical physics.
Any question about these definitions?
So let's again proceed slowly to get
some definitions established by looking at one random variable.
So this is the next section on one random variable.
And I will assume that I'll look at the case
of the continuous random variable.
So x can be any real number minus infinity to infinity.
Now, a number of definitions.
I will use the term Cumulative-- make
sure I'll use the-- Cumulative Probability Function, CPF,
that for this one random variable,
I will indicate by capital P of x.
And the meaning of this is that capital P
of x is the probability of outcome less than x.
So generically, we say that x can
take all values along the real line.
And there is this function that I
want to plot that I will call big P of x Now big P
of x is a probability, therefore,
it has to be positive according to the first item that we
have over there.
And it will be less than 1 because the net probability
for everything toward here is equal to 1.
So asymptotically, where I go all the way
to infinity, the probability that I
will get some number along the line-- I have to get something,
so it should automatically go to 1 here.
And every element of probability is positive,
so it's a function that should gradually go down.
And presumably, it will behave something
like this generically.
Once we have the Cumulative Probability Function,
we can immediately construct the Probability Density Function,
PDF, which is the derivative of the above.
P of x is the derivative of big P of x with respect to the x.
And so if I just take here the curve that I have above
and take its derivative, the derivative
will look something like this.
Essentially, clearly by the definition of the derivative,
this quantity is therefore ability
of outcome in the interval x to x plus dx
divided by the size of the interval dx.
couple of things to remind you of, one of them
is that the Cumulative Probability is a probability.
It's a dimensionless number between 0 and 1.
Probability Density is obtained by taking a derivative,
so it has dimensions that are inverse of whatever this x is.
So if I change my variable from meters to centimeters,
let's say, the value of this function
would change by a factor of 100.
And secondly, while the Probability Density
is positive, its value is not bounded.
It can be anywhere that you like.
One other, again, minor definition
is expectation value.
So I can pick some function of x.
This could be x itself.
It could be x squared.
It could be sine x, x cubed minus x squared.
The expectation value of this is defined
by integrating the Probability Density
against the value of the function.
So essentially, what that says is
that if I pick some function of x-- function
can be positive, negative, et cetera.
So maybe I have a function such as this-- then the value
of x is random.
If x is in this interval, this would
be the corresponding contribution to f of x.
And I have to look at all possible values of x.
Question?
Now, very associated to this is a change of variables.
You would say that if x is random, then f of x is random.
So if I ask you what is the value of x squared,
and for one random variable, I get this.
The value of x squared would be this.
If I get this, the value of x squared
would be something else.
So if x is random, f of x is itself a random variable.
So f of x is a random variable, and you
can ask what is the probability, let's
say, the Probability Density Function that I would associate
with the value of this.
Let's say what's the probability that I will find it
in the interval between small f and small f plus df.
This will be Pf f of f.
You would say that the probability that I would find
the value of the function that is in this interval
corresponds to finding a value of x that is in this interval.
So what I can do, the probability
that I find the value of f in this interval,
according to what I have here, is the Probability Density
multiplied by df.
Is there a question?
No.
So the probability that I'm in this interval
translates to the probability that I'm in this interval.
So that's probability p of x dx.
But that's boring.
I want to look at the situation maybe where
the function is something like this.
Then, you say that f is in this interval provided
that x is either here or here or here.
So what I really need to do is to solve
f of x equals to f for x.
And maybe there will be solutions
that will be x1, x2, x3, et cetera.
And what I need to do is to sum over
the contributions of all of those solutions.
So here, it's three solutions.
Then, you would say the Probability Density
is the sum over i P of xi, the xi by df,
which is really the slopes.
The slopes translate the size of this interval
to the size of that interval.
You can see that here, the slope is very sharp.
The size of this interval is small.
It could be wider accordingly, so I
need to multiply by dxi by df.
So I have to multiply by dx by df evaluated at xi.
That's essentially the value of the derivative of f.
Now, sometimes, it is easy to forget these things
that I write over here.
And you would say, well obviously,
the probability of something that is positive.
But without being careful, it is easy to violate
such basic condition.
And I violated it here.
Anybody see where I violated it.
Yeah, the slope here is positive.
The slope here is positive.
The slope here is negative.
So I am subtracting a probability here.
So what I really should do-- it really
doesn't matter whether the slope is this way or that way.
I will pick up the same interval,
so make sure you don't forget the absolute values that
go accordingly.
So this is the standard way that you
would make change of variables.
Yes?
AUDIENCE: Sorry.
In the center of that board, on the second line, it says Pf.
Is that an x or a times?
PROFESSOR: In the center of this board?
This one?
AUDIENCE: Yeah.
PROFESSOR: So the value of the function
is a random variable, right?
It can come up to be here.
And so there is, as any other one parameter random variable,
a Probability Density associated with that.
That Probability Density I have called P of f
to indicate that it is the variable
f that I'm considering as opposed
to what I wrote originally that was
associated with the value of x.
AUDIENCE: But what you have written on the left-hand side,
it looks like your x [? is random. ?]
PROFESSOR: Oh, this was supposed to be a multiplication
sign, so sorry.
AUDIENCE: Thank you.
PROFESSOR: Thank you.
Yes?
AUDIENCE: CP-- that function, is this [INAUDIBLE]?
PROFESSOR: Yes.
So you're asking whether this-- so I constructed something,
and my statement is that the integral from minus infinity
to infinity df Pf of f better be one which is the normalization.
So if you're asking about this, essentially,
you would say the integral dx p of x
is the integral dx dP by dx, right?
That was the definition p of x.
And the integral of the derivative
is the value of the function evaluated at its two extremes.
And this is one minus 0.
So by construction, it is, of course, normalized
in this fashion.
Is that what you were asking?
AUDIENCE: I was asking about the first possibility
of cumulative probability function.
PROFESSOR: So the cumulative probability,
its constraint is that the limit as its variable
goes to infinity, it should go to 1.
That's the normalization.
The normalization here is that the probability
of the entire set is 1.
Cumulative adds the probabilities
to be anywhere up to point x.
So I have achieved being anywhere on the line
by going through this point.
But certainly, the integral of P of xdx is not equal to 1
if that's what you're asking.
The integral of small p of x is 1.
Yes?
AUDIENCE: Are we assuming the function is invertible?
PROFESSOR: Well, rigorously speaking, this function
is not invertible because for a value of f,
there are three possible values of x.
So it's not a function, but you can certainly
solve for f of x equals to f to find particular values.
So again, maybe it is useful to work
through one example of this.
So let's say that you have a probability that
is of the form e to the minus lambda absolute value of x.
So as a function of x, the Probability Density
falls off exponentially on both sides.
And again, I have to ensure that when I integrate this from 0
to infinity, I will get one.
The integral from 0 to infinity is
1 over lambda, from minus infinity
to zero by symmetry is 1 over lambda.
So it's really I have to divide by 2 lambda-- to lambda over 2.
Sorry.
Now, suppose I change variables to F, which is x squared.
So I want to know what the probability is
for a particular value of x squared that I will call f.
So then what I have to do is to solve this.
And this will give me x is minus plus square root of small f.
If I ask for what f of-- for what value of x, x squared
equals to f, then I have these two solutions.
So according to the formula that I have,
I have to, first of all, evaluate this
at these two possible routes.
In both cases, I will get minus lambda square root of f.
Because of the absolute value, both of them
will give you the same thing.
And then I have to look at this derivative.
So if I look at this, I can see that df by dx equals to 2x.
The locations that I have to evaluate are at plus minus
square root of f.
So the value of the slope is minus plus to square root of f.
And according to that formula, what I have to do
is to put the inverse of that.
So I have to put for one solution, 1 over 2 square root
of f.
For the other one, I have to put 1 over minus 2 square root
of f, which would be a disaster if I didn't convert this
to an absolute value.
And if I did convert that to an absolute value,
what I would get is lambda over 2 square root of f
e to the minus lambda root f.
It is important to note that this solution will only
exist only if f is positive.
And there's no solution if f is negative,
which means that if I wanted to plot a Probability
Density for this function f, which
is x squared as a function of f, it will only
have values for positive values of x squared.
There's nothing for negative values.
For positive values, I have this function
that's exponentially decays.
Yet at f equals to 0 diverges.
One reason I chose that example is
to emphasize that these Probability Density
functions can even go all the way infinity.
The requirement, however, is that you
should be able to integrate across the infinity because
integrating across the infinity should give you a finite number
less than 1.
And so the type of divergence that you could have is limited.
1 over square root of f is fine.
1/f is not accepted.
Yes?
AUDIENCE: I have a doubt about [? the postulate. ?]
It says that if you raise the value of f slowly,
you will eventually get to-- yeah, that point right there.
So if the prescription that we have of summing over
the different roots, at some point, the roots,
they converge.
PROFESSOR: Yes.
AUDIENCE: So at some point, we stop summing over 2
and we start summing over 1.
It just seems a little bit strange.
PROFESSOR: Yeah.
If you are up here, you have only one term in the sum.
If you are down here, you have three terms.
And that's really just the property of the curve
that I have drawn.
And so over here, I have only one root.
Over here, I have three roots.
And this is not surprising.
There are many situations in mathematics or physics
where you encounter situations where,
as you change some parameters, new solutions, new roots,
appear.
And so if this was really some kind of a physical system,
you would probably encounter some kind
of a singularity of phase transitions at this point.
Yes?
AUDIENCE: But how does the equation deal with that when
[INAUDIBLE]?
PROFESSOR: Let's see.
So if I am approaching that point, what I find
is that the f by the x goes to 0.
So the x by df has some kind of infinity or singularity,
so we have to deal with that.
If you want, we can choose a particular form
of that function and see what happens.
But actually, we have that already over here
because the function f that I plotted for you
as a function of x has this behavior
that, for some range of f, you have two solutions.
So for negative values of f, I have no solution.
So this curve, after having rotated,
is precisely an example of what is happening here.
And you see what the consequence of that is.
The consequence of that is that as I approach here
and the two solutions merge, I have the singularity that
is ultimately manifested in here.
So in principle, yes.
When you make these changes of variables
and you have functions that have multiple solution
behavior like that, you have to worry about this.
Let me go down here.
One other definition that, again, you've
probably seen, before we go through something
that I hope you haven't seen, moment.
A form of this expectation value-- actually, here we
did with x squared, but in general, we
can calculate the expectation value of x to the m.
And sometimes, that is called mth moment is the integral 0
to infinity dx x to the m p of x.
Now, I expect that after this point,
you would have seen everything.
But next one maybe half of you have seen.
And the next item, which we will use a lot,
is the characteristic function.
So given that I have some probability distribution
p of x, I can calculate various expectation values.
I calculate the expectation value of e to the minus ikx.
This is, by definition that you have,
I have to integrate over the domain of x--
let's say from minus infinity to infinity-- p of x against e
to the minus ikx.
And you say, well, what's special about that?
I know that to be the Fourier transform of p of x.
And it is true.
And you also know how to invert the Fourier transform.
That is if you know the characteristic function, which
is another name for the Fourier transform of a probability
distribution, you would get the p of x
back by the integral over k divided by 2pi,
the way that I chose the things, into the ikx p tilde of k.
Basically, this is the standard relationship
between these objects.
So this is just a Fourrier transform.
Now, something that appears a lot in statistical calculations
and implicit in lots of things that we
do in statistical mechanics is a generating function.
I can take the characteristic function p tilde of k.
It's a function of this Fourrier variable, k.
And I can do an expansion in that.
I can do the expansion inside the expectation value
because e to the minus ikx I can write as a sum over n running
from 0 to infinity minus ik to the power of m divided by n
factorial x to the nth.
This is the expansion of the exponential.
The variable here is x, so I can take everything else outside.
And what I see is that if I make an expansion
of the characteristic function, the coefficient
of k to the n up to some trivial factor of n factorial
will give me the nth moment.
That is once you have calculated the Fourrier
transform, or the characteristic function, you can expand it.
And you can, from out of that expansion, you
can extract all the moments essentially.
So this is expansion generates for you the moments,
hence the generating function.
You could even do something like this.
You could multiply e to the ikx0 for some x0 p tilde of k.
And that would be the expectation value of e
to the ikx minus x0.
And you can expand that, and you would
generate all of the moments not around the origin,
but around the point x0.
So simple manipulations of the characteristic function
can shift and give you other set of moments
around different points.
So the Fourier transform, or characteristic function,
is the generator of moments.
An even more important property is
possessed by the cumulant generating function.
So you have the characteristic function,
the Fourier transform.
You take its log, so another function of k.
You start expanding this function in covers of k.
Add the coefficients of that, you call cumulants.
So I essentially repeated the definition that I had up there.
I took a log, and all I did is I put this subscript c
to go from moments to cumulants.
And also, I have to start the series from 1 as opposed to 0.
And essentially, I can find the relationship between cumulants
and moments by writing this as a log
of the characteristic function, which
is 1 plus some n plus 1 to infinity
of minus ik to the n over n factorial, the nth moments.
So inside the log, I have the moments.
Outside the log, I have the cumulants.
And if I have a log of 1 plus epsilon,
I can use the expansion of this as epsilon minus
epsilon squared over 2 epsilon cubed over 3 minus epsilon
to the fourth over 4, et cetera.
And this will enable me to then match powers of minus ik
on the left and powers of minus ik on the right.
You can see that the first thing that I will find
is that the expectation value of x-- the first power,
the first term that I have here is minus ik to the mean.
Take the log, I will get that.
So essentially, what I get is that the first cumulant
on the left is the first moment that I
will get from the expansion on the right.
And this is, of course, called the mean of the distribution.
The second cumulant, I will have two contributions,
one from epsilon, the other from minus epsilon squared over 2.
And If you go through that, you will
get that it is expectation value of x squared
minus the average of x, the mean squared, which
is none other than the expectation value of x
around the mean squared, which is clearly a positive quantity.
And this is the variance.
And you can keep going.
The third cumulant is x cubed minus 3 average of x squared
average of x plus 2 average of x itself cubed.
It is called the skewness.
I don't write the formula for the next one
which is called a [? cortosis ?].
And you keep going and so forth.
So it turns out that this hierarchy of cumulants,
essentially, is a hierarchy of the most important things
that you can know about a random variable.
So if I tell you that the outcome of some experiment
is some number x, distribute it somehow-- I
guess the first thing that you would like to know
is whether the typical values that you get
are of the order of 1, are of the order of million, whatever.
So somehow, the mean is something
that tells you something that is most important is zeroth order
thing that you want to know about the variable.
But the next thing that you might want to know
is, well, what's the spread?
How far does this thing go?
And then the variance will tell you something about the spread.
So the next thing that you want to do
is maybe if given the spread, am I
more likely to get things that are on one side or things
that are on the other side.
So the measure of its asymmetry, right versus left,
is provided by the third cumulant, which is the skewness
and so forth.
So typically, the very first few members
of this hierarchy of cumulants tells you
the most important information that you
need about the probability.
Now, I will mention to you, and I
guess we probably will deal with it more next time around,
the result that is in some sense the backbone or granddaddy
of all graphical expansions that are carrying [INAUDIBLE].
And that's a relationship between the moments
and cumulants that I will express graphically.
So this is graphical representation
of moments in terms of cumulants.
Essentially, what I'm saying is that you
can go through the procedure as I outlined.
And if you want to calculate minus ik to the fifth power
so that you find the description of the fifth cumulant in terms
of the moment, you'll have to do a lot of work
in expanding the log and powers of this object and making sure
that you don't make any mistakes in the coefficient.
There is a way to circumvent that graphically
and get the relationship.
So how do we do that?
You'll represent nth cumulant as let's say a bag of endpoints.
So let's say this entity will represent the third cumulant.
It's a bag with three points.
This-- one, two, three, four, five, six--
will represent the sixth cumulant.
Then, the nth moment is some of all ways
of distributing end points amongst bags.
So what do I mean?
So I want to calculate the first moment x.
That would correspond to one point.
And really, there's only one diagram
I can put the bag around it or not
that would correspond to this.
And that corresponds to the first cumulant,
basically rewriting what I had before.
If I want to look at the second moment, the second moment
I need two points.
The two points I can either put in the same bag or I
can put into two separate bags.
And the first one corresponds to calculating
the second cumulant.
The second term corresponds to two ways
in which their first cumulant has appeared,
so I have to squared x.
if I want to calculate the third moment, I need three dots.
The three dots I can either put in one bag
or I can take one of them out and keep two of them in a bag.
And here I had the choice of three things
that I could've pulled out.
Or, I could have all of them in individual bags of their own.
And mathematically, the first term corresponds to x cubed c.
The third term corresponds to three versions
of the variance times the mean.
And the last term is just the mean cubed.
And you can massage this expression
to see that I get the expression that I have for the skewness.
I didn't offhand remember the relationship
that I have to write down for the fourth cumulant.
But I can graphically, immediately get
the relationship for the fourth moment
in terms of the fourth cumulant which is this entity.
Or, four ways that I can take one of the back
and maintain three in the bag, three ways in which I have
two bags of two, six ways in which I can have a bag of two
and two things that are individually apart, and one
way in which there are four things that
are independent of each other.
And this becomes x to the fourth cumulant, the fourth cumulant,
4 times the third cumulant times the mean,
3 times the square of the variance,
6 times the variance multiplied by the mean squared,
and the mean raised to the fourth power.
And you can keep going.
AUDIENCE: Is the variance not squared in the third term?
PROFESSOR: Did I forget that?
Yes, thank you.
All right.
So the proof of this is really just the two-line algebra
exponentiating these expressions that we have over here.
But it's much nicer to represent that graphically.
And so now you can go between things very easily.
And what I will show next time is how, using this machinery,
you can calculate any moment of a Gaussian,
for example, in just a matter of seconds as opposed
to having to do integrations and things like that.
So that's what we will do next time will
be to apply this machinery to various probability
distribution, such as a Gaussian,
that we are likely to encounter again and again.