Mechanics IV: from Hamiltonian to quantum mechanics

In our series on mechanics, we discussed several approaches to mechanics, leading up to Hamilton’s approach. Although Hamilton’s approach is abstract, it is a more symmetric approach (treating position and momentum on equal grounds), and it helps us better understand conserved quantities. However, it has another big advantage: it serves as a bridge between classical and quantum mechanics.

Recall how, in the geometric approach to Hamiltonian mechanics, we showed how symplectic geometry allows you to phrase mechanics in terms of the geometry of phase spaces -- geometric spaces which encode the possible states of some physical system. These phase spaces are abstract shapes which are hard to 'see' directly; instead, scientists access them by making measurements (see our article on Gelfand duality for an elaboration on this).

In quantum mechanics, this view of physics reigns supreme, with one change: the phase spaces involved become much stranger than any usual shapes.

What is quantum mechanics?

Schrodinger’s cat

You may have heard of the famous thought experiment of Schrodinger’s cat, that the cat is both alive and dead simultaneously. What does that mean, really?

A key principle in quantum mechanics is that particles only exist probabilistically. In other words, instead of saying a particle is at position \((1,2)\), in quantum mechanics you say that the position of a particle is a normal distribution centered around \((1,2)\) with standard deviation \(3\). Observing the particle amounts to sampling this distribution: you are likely to observe the particle close to \((1,2)\), but with a very small probability, you may observe it at \((100,-9)\). In other words, the measurements scientists make behave more like random variables than like traditional functions.

Going back to the example of Schrodinger’s cat, you take a particle which could have two states A and B, and assume it could be each state with probability \(\frac12\). Then, you make a contraption to release poison to the cat only when the particle has state B (scientists seem to have quite cruel minds... why not have the theoretical contraption release candy?). If you do this, the cat will also be dead and alive with probability \(\frac12\).

There is an important distinction between the kind of probabilities occurring here versus the kind of probability people mean when they talk about flipping a coin.

For example, when you flip a coin, it also lands heads or tails with probability \(\frac12\). Perhaps you decide you'll flip a coin, and then release the poison only when the coin flips heads. How does this differ from the quantum situation?

The key is that if you knew the speed and force at which you flipped the coin, you could in principle know whether it would land heads or tails before the experiment. In other words, we only say it lands heads or tails with probability \(\frac12\) due to our ignorance. However, in quantum mechanics, a particle can have a physical state which is state A with probability \(\frac12\) and state B with probability \(\frac12\). Even an all-knowing being could not tell you which state the particle is in before observing it. So, Schrodinger’s cat is really both physically alive and dead, in a precise physical sense.

Probability density functions

How does one talk about proabilities of something that can have infinitely many outcomes?

For example, when you choose a card at random from a standard deck, there are 52 possible outcomes, and we assign each probability \(1/52.\) But when you're talking about the position of a particle moving in 1-dimension, the possible outcome is any real number. That's infinitely many possibilities! How do we assign them probabilities?

To motivate this switch, let's think about throwing a dart on a dartboard. This is another random variable: the dart lands at a random point on the circular board. It seems to be logical to say that the dart has probability \(\frac{1}{2}\) of landing on the right side of the board, and probability \(\frac{1}{3}\) of landing in the top third. This suggests a general principle: the probability the dart lands in a certain region seems to be \[\frac{\text{area of region}}{\text{area of entire board}}.\]

This connection between probability and area led to the notion of a probability density function: when one has a random variable \(X\) whose value is a real number, we formalize it with a function \[\phi : \mathbb{R} \to [0, 1],\] and then we say that \[\text{probability \(a \leq X \leq b\)} = \int_a^b \phi(x) \, dx.\]

Wave functions

You might expect that a quantum mechanical particle is formalized then as some probability density function, describing the probability distribution of its position. And, while this is close, it turns out that to understand a quantum mechanics one is required to know a little more information; this extra information is analogous to how, in classical mechanics, one is required to not just know the position of a particle, but also its momentum, if they want to understand how the system will evolve.

It seems natural, then, to just add a second probability distribution, and store a particle as two probability distributions: one for its position, and one for its momentum. However, one of the most surprising and strange parts of quantum mechanics is that the position and momentum probability distributions are related. This relationship was discovered through experiments illustrating how quantum particles interfere with each other in strange ways, and led physicists to the idea of the wave function: a single complex-valued function \(\psi(\vec x,t)\), where \(\vec x\in\mathbb R^3\) is the position and \(t\) is the time. When a particle has wave function \(\psi(\vec x,t)\), it means at time \(t\), you have probability \(|\psi(\vec x,t)|^2\) of finding it at position \(\vec x\). In particular, since probabilities should add to 1, we want \[ \int_{\mathbb R^3}|\psi(\vec x,t)|^2d\vec x=1. \]

So, if the wave function is \(\psi,\) the particle's position has a probability density function of \(|\psi|^2.\) But what about the momentum?

Remarkably, the momentum has probability density \[\left|\frac{\hbar}{i}\frac{\partial}{\partial\vec x}\psi(\vec x, t)\right|^2,\] where \(\hbar\) is called the (reduced) Planck constant, and is given by \[ \hbar\approx1.054\times 10^{-34}\mathrm{J}\cdot\mathrm{s}. \]

The fundamental equation in quantum mechanics is Schrodinger’s equation, which states that if a particle of mass \(m\) is in a potential \(V(\vec x)\), the wave function obeys the differential equation \[ i\hbar\frac{\partial}{\partial t}\psi(\vec x,t)=\bigg(-\frac{\hbar^2}{2m}\frac{\partial^2}{\partial\vec x^2}+V(\vec x)\bigg)\psi(\vec x,t). \] This is analogous to the equation for energy \(E=\frac{p^2}{2m}+V,\) where the energy \(E\) is replaced by the operator \[ i\hbar\frac{\partial}{\partial t}, \] the momentum \(p\) is replaced by the operator \[ \frac\hbar i\frac\partial{\partial\vec x}. \]

A linear algebraic foundation to quantum mechanics

In the previous section, we saw how we can formulate quantum mechanics through wave functions \(\psi(\vec x,t)\), which describes the probability distribution of the position of a particle. However, we could equally have used wave functions \(\psi(\vec p,t)\) which describes the probability distribution of the momentum of a particle. This suggests wave functions may not be the best way to formalize quantum mechanics.

In the mathematical formalism of quantum mechanics developed by Paul Dirac, David Hilbert, John von Neumann, and Hermann Weyl, we instead think of a state of a quantum mechanical system as a vector \(|\psi\rangle\) in some vector space \(\mathcal H\). Our previous assumption that the probabilities in a wave function add to 1, \[ \int_{\mathbb R^3}|\psi(\vec x,t)|^2d\vec x=1, \] is replaced with the hypothesis that the vector \(|\psi\rangle\) has length 1. In Dirac’s notation, we write this as \[ \langle\psi|\psi\rangle=1. \] In general, given two vectors \(|\phi\rangle\) and \(|\psi\rangle\) in the vector space \(\mathcal H\), Dirac denotes their inner product as \[ \langle\phi|\psi\rangle. \] The physical state evolves with time, say as a function \(|\psi(t)\rangle\)--this is the Heisenberg picture.

How do we relate these vectors back to wave functions? First of all, in this formalism, observables, such as position, momentum, angular momentum, … are all linear operators on \(\mathcal H\). For example, position becomes some linear operator \(\hat x\). Let \(|\vec x\rangle\) be eigenvectors of this position operator (which exists for each position \(\vec x\)). Then, \[ \psi(\vec x,t)=\langle\vec x|\psi(t)\rangle. \] In other words, if we imagine expanding the vector \(|\psi(t)\rangle\) in terms of the eigenvectors \(|\vec x\rangle\), we can write it as \[ |\psi(t)\rangle=\sum_{\vec x\in\mathbb R^3}\psi(\vec x,t)|\vec x\rangle. \] Of course, this doesn’t literally make sense, since this is an infinite sum… Instead, we should write \[ |\psi(t)\rangle=\int_{\mathbb R^3}\psi(\vec x,t)|\vec x\rangle d\vec x. \] Recall that when we observe the particle, we will find it at position \(\vec x\) with probability \(|\psi(x,t)|^2\). In the language of linear algebra, we project the vector \(|\psi(t)\rangle\) to the eigenvector \(|\vec x\rangle\) with probability which is the absolute value squared of \(\langle\vec x|\psi(t)\rangle\).

We could similarly recover the wave function with respect to momentum, \(\psi(\vec p,t)\), by letting \(\hat p\) be the momentum operator, and letting \(|\vec p\rangle\) be eigenvectors. Then \[ \psi(\vec p,t)=\langle \vec p|\psi(t)\rangle, \] and we could do all the same things we did from before with position.

We can translate the Schrodinger equation, which involves the wave function \(\psi(\vec x,t)\), into linear operators. Let \(\hat H\) be the Hamiltonian operator on \(\mathcal H\). Then the Schrodinger equation for the particle \(|\psi(t)\rangle\) is \[ \boxed{i\hbar\frac d{dt}|\psi(t)\rangle=\hat H|\psi(t)\rangle}. \]

The uncertainty principle

One of the most fundamental principles in quantum mechanics is Heisenberg’s uncertainty principle. It states that you cannot know both the position and momentum of a particle precisely. This becomes particularly lucid when you think about quantum mechanics through linear operators.

Given a state \(|\psi\rangle\in\mathcal H\), observing the position will collapse the vector to some eigenvector of the position operator \(\hat x\). In other words, knowing the position of the particle exactly would mean the state \(|\psi\rangle\) is an eigenvector of \(\hat x\). Similarly, if you knew the momentum exactly, then \(|\psi\rangle\) would be an eigenvector of the momentum operator \(\hat p\).

Therefore, Heisenberg's uncertainty principle says that \(\hat x\) and \(\hat p\) have no simultaneous eigenvectors. Let's take a detour into linear algebra and think about when two matrices have simultaneous eigenvectors.

A detour: eigenvalues and eigenvectors

A matrix \(A\) has an eigenvalue \(\lambda\) if you have a non-zero vector \(\vec v\)--called an eigenvector--such that \[ A\vec v=\lambda\vec v. \] For example, the matrix \[ A=\begin{pmatrix}0&1\\ -2&3 \end{pmatrix} \] has eigenvalues \(\lambda=1,2\), since \[ A\cdot\begin{pmatrix}1\\1\end{pmatrix}=\begin{pmatrix}1\\1\end{pmatrix},\ A\cdot\begin{pmatrix}1\\2\end{pmatrix}=2\begin{pmatrix}1\\2\end{pmatrix}. \]

An important result in linear algebra is that if \(A\) and \(B\) are two matrices which commute with each other, i.e., \[ AB=BA,\] then you can find a simultaneous eigenvector: some vector \(\vec v\) such that \[ A\vec v=\lambda\vec v,\ B\vec v=\mu\vec v\] for some scalars \(\lambda\) and \(\mu\).

The basic reason is because if \(\vec v\) is an eigenvector of \(A\), then \(B\vec v\) is also an eigenvector of \(A\). Indeed, if \(A\vec v=\lambda\vec v\) for some scalar \(\lambda\), then \[ A(B\vec v)=B(A\vec v)=B(\lambda\vec v)=\lambda(B\vec v), \] since \(A\) and \(B\) commute with each other. That means you can “arrange” an eigenvector for \(B\) without breaking the condition of being an eigenvector for \(A\).

To see a more rigorous proof, you can click here.

First of all, remember that any complex matrix has at least one eigenvalue. So let \(\lambda\) be an eigenvalue for \(A\). Then the vectors with eigenvalue \(\lambda\) form a subspace \[ V=\{\vec v\in\mathbb C^n:A\vec v=\lambda\vec v\}\subset\mathbb C^n. \] The matrix \(B\) preserves this subspace, in the sense that if \(\vec v\in V\) then \(B\vec v\in V\) as well. That’s because \[ A(B\vec v)=B(A\vec v)=B(\lambda\vec v)=\lambda(B\vec v). \] Now, \(B\) is a linear operator on \(V\), hence also has an eigenvector, call it \(\vec v\in V\), with some eigenvalue \(\mu\). This is the vector we were looking for.

Back to uncertainty

Therefore, the uncertainty principle tells us that the position and momentum operators \(\hat x\) and \(\hat p\) do not commute with each other. The extent to which the operators do not commute with each other is called the commutator, \[ [\hat x,\hat p]=\hat x\hat p-\hat p\hat x. \] So \(\hat x\) and \(\hat p\) commute exactly when \([\hat x,\hat p]=0\). The uncertainty principle, in linear algebraic terms, says \[ [\hat x,\hat p]=i\hbar. \] That means the two operators \(\hat x\) and \(\hat p\) are very close to commuting, with an error of Planck’s constant \(\hbar\); this is again why we don’t see the uncertainty phenomenon at macroscopic scales.

More carefully analyzing what the equation \([\hat x,\hat p]=\hbar\) means in terms of the eigenvalues of the operators \(\hat x\) and \(\hat p\), we get a more precise statement for the uncertainty principle. It says \[ \sigma_x\cdot\sigma_p\ge\frac\hbar2, \] where \(\sigma_x\) is the standard deviation of position, and \(\sigma_p\) is the standard deviation of momentum.

Quantization of Hamiltonian mechanics

Let us summarize what we have done so far. In our formalism, the state of a physical system is a vector in some vector space \(\mathcal H\), and position and momentum are operators on this space. The position and momentum operators satisfy the equation \[ [\hat x,\hat p]=i\hbar. \] There is furthermore an operator called the Hamiltonian \(\hat H\), and the state of a particle \(|\psi(t)\rangle\) evolves according to the equation \[ i\hbar\frac d{dt}|\psi(t)\rangle=\hat H|\psi(t)\rangle. \]

How does this relate with the Hamiltonian formalism for mechanics? In Hamilton’s formalism, we have a function \(H\) of position \(\vec x\), momentum \(\vec p\), and time \(t\). It is then convenient to consider the Poisson bracket of two functions \(f\) and \(g\), defined by \[ \{f,g\}=\sum_i\bigg(\frac{\partial f}{\partial x_i}\frac{\partial g}{\partial p_i}-\frac{\partial f}{\partial p_i}\frac{\partial g}{\partial x_i}\bigg). \] Then Hamilton’s equation can be expressed as, for any function \(F\), \[ \frac{dF}{dt}=\{H,F\}. \]

To relate the two pictures, let’s take any observable \(f\) and \(g\) in classical mechanics, and consider them as operators \(\hat f\) and \(\hat g\) on \(\mathcal H\). Then the operator corresponding to the Poisson bracket \(\{f,g\}\) is the commutator \([\hat f,\hat g]/i\hbar\).

For example, the functions of position \(x_i\) and momentum \(p_i\) correspond to operators \(\hat x\) and \(\hat p\) on \(\mathcal H\), and their commutator is \[ [\hat x,\hat p]=i\hbar. \] This is what we expect, since \(\{x_i,p_i\}=1\)!

Conserved quantities in quantum mechanics

In our last article, we saw that a quantity \(A\) is conserved exactly when \(\{A,H\}=0\). In quantum mechanics, we have the same exact phenomenon.

Let \(\hat A\) be the linear operator on \(\mathcal H\) corresponding to \(A\). We view this as a function which takes a state \(|\psi\rangle\in\mathcal H\) and outputs a number \[ \langle\psi|A|\psi\rangle. \] We say \(A\) is a conserved quantity exactly when the inner product \[ \langle A\rangle_t:=\langle\psi(t)|A|\psi(t)\rangle \] does not change as we vary the time \(t\).

Now, suppose \(\hat A\) and \(\hat H\) commute with each other, and assume they are time-independent for simplicity. Then Schrodinger’s equation tells us that \[ \frac d{dt}|\psi(t)\rangle=-\frac i\hbar\hat H|\psi(t)\rangle, \] and taking adjoints, \[ \frac d{dt}\langle\psi(t)|=\frac i\hbar\langle\psi(t)|\hat H. \] Thus, by the Leibniz rule, the derivative of \(\langle A\rangle_t\) is \[ \frac d{dt}\langle A\rangle_t=\langle\dot\psi|A|\psi\rangle+\langle\psi|A|\psi\rangle. \] By applying Schrodinger’s equation, this equals \[ \frac i\hbar\langle\psi|\hat H\hat A|\psi\rangle-\frac i\hbar\langle\psi|\hat A\hat H|\psi\rangle=\frac i\hbar\langle\psi|[\hat A,\hat H]|\psi\rangle. \] When \(\hat A\) and \(\hat H\) commute with each other, this expression is just 0, so \(\langle A\rangle_t\) is independent of \(t\).

Non-commutative geometry

Now let’s relate the ideas of quantum mechanics back to Gelfand duality. There, we explored the idea that to describe a space \(X\) and to describe all the (continuous) functions on \(X\) is really the same thing. This is a “duality” between geometry (spaces) and algebra (rings).

You only get commutative rings from Gelfand duality. That’s simply because it doesn’t matter in which order you multiply functions in: for any two functions \(f\) and \(g\) on \(X\), \[ f(x)g(x)=g(x)f(x)\in\mathbb R \] for all \(x\in X\). There are, however, rings that we encounter in daily life which are not commutative. For example, the ring \(M_{2\times 2}(\mathbb R)\) of \(2\times 2\)-matrices is not commutative, since \[ \begin{pmatrix}0&1\\ 0&0\end{pmatrix}\cdot\begin{pmatrix} 0&0\\ 1&0 \end{pmatrix}=\begin{pmatrix}1&0\\0&0\end{pmatrix}, \] whereas \[ \begin{pmatrix}0&0\\ 1&0\end{pmatrix}\cdot\begin{pmatrix} 0&1\\ 0&0 \end{pmatrix}=\begin{pmatrix}0&0\\0&1\end{pmatrix}. \]

It would be fun if we could take a non-commutative ring \(\mathcal A\) such as \(M_{2\times 2}(\mathbb R)\) and imagine it is the ring of functions on some “space” \(X\). This idea is the core of the field called “non-commutative geometry.”

This process of turning our attention from a commutative situation to a non-commutative situation is exactly what happened when we went from classical to quantum mechanics. In classical mechanics we thought about observables as functions \(f(\vec x,\vec p)\) which commute with each other, whereas in quantum mechanics we consider observable as operators on a vector space \(\mathcal H\), which need not commute with each other.

More formally, in classical mechanics our observables were functions \(f(\vec x,\vec p)\) on the \(2n\)-dimensional phase space \(\mathbb R^{2n}\), which has \(n\) position coordinates and \(n\) momentum coordinates. In quantum mechanics, we instead consider a ring with variables \(\hat x_i\) and \(\hat p_i\), which satisfy the relation \[ \hat x_i\hat p_i-\hat p_i\hat x_i=i\hbar. \]

Amusingly, this relation closely resembles the relation satisfied by differential operators. The space of 1-variable functions \(f(t)\) have two interesting linear operators on it: differentiation, which takes \[ f(t)\mapsto\frac{df}{dt}(t), \] and multiplication by \(t\), which takes \[ f(t)\mapsto tf(t). \] Then, by the product rule, \[ \frac d{dt}\big(tf(t)\big)=t\frac d{dt}f(t)+f(t), \] which equivalently means \[ \big[t,\frac d{dt}\big]=1. \]

In other words, the relations satisfied by the operators \(\hat x_i\) and \(\hat p_i\) are the same as those satisfied by the operators of multiplication by \(t_i\) and differentiation \(\frac{\partial}{\partial t_i}\) on the space of all functions \(f(t_1,\dots,t_n)\) on \(n\)-dimensional space \(\mathb R^n\).

In other words, going from classical mechanics to quantum mechanics is the same as quantizing from the algebra of functions on the phase space \(\mathbb R^{2n}\) to the algebra of differential operators on \(\mathbb R^n\).