How the entropy of quantum observables relates to one of the oldest problems in statistical physics
“The house always wins”
This is the famous saying about how casinos rig the payouts of their games to ensure that, over the long run, their profits always increase. No matter how well you play the game, over a long enough timescale, you’re guaranteed to leave with a loss.
The saying also serves as a nice metaphor for the infamous second law of thermodynamics, one of the most profound concepts in all of science. This is because, like casino profits, the law also deals with a quantity that inevitably increases over time: entropy. I’ve always been fascinated by this part of physics: I think it’s amazing how the law seems to just emerge naturally, almost spontaneously, just from considering how small systems come together to form a larger collective.
Yet despite the law being both incredibly well-tested and extensively justified on theoretical grounds, it hides a paradox that’s been debated by physicists for over a century. Today, our group released a preprint with a proposal for a new perspective on the problem, especially as it relates to thermodynamics in the quantum regime.
The second law states that the ‘entropy’ property of a system always increases until it reaches its maximum possible value. But finding the words to describe what this ‘entropy’ thing even means is tough. Very loosely speaking, it quantifies disorder. A system with higher entropy is less neat, more jumbled-up, or harder to succinctly describe. And so the second law says that if a system that’s neat and orderly is left on its own, it’s guaranteed to become more disordered. It’s possible to decrease the entropy of a specific object or area (or to maintain a low-entropy state somewhere), but the cost to do so will always be an increase in the entropy of a different object or area that more than makes up for the decrease.1
Entropy increases over time because systems we see in the everyday world are made of unimaginably huge numbers of constituent parts — literally septillions of particles, and these particles are jostling around effectively randomly. In doing so, there’s more ways for them to randomly jump into disordered configurations than into ordered ones. So over time, its ‘disorder’ will increase, until it reaches the maximum possible disorder. It’s a law of statistics, not physics!
But there’s a crucial, conceptual problem at the core of this story: namely, that if you actually consider an entire system on its own in isolation, its entropy shouldn’t be able to increase like this. In fact, it should remain constant for all time. This is the ‘paradox’ we tackled in our new paper.2
The key to understanding where this paradox comes from is to realise that entropy is intimately related to information theory, an under-appreciated but vital scientific development from the 20th century. In particular, there turns out to be a deep, fundamental connection between entropy as a measure of disorder, and another definition of entropy that measures information.
To explain this “information-theory entropy” consider the following three sequences of numbers:
- A: 1,1,1,1,1,1,1,1
- B: 1,2,3,4,5,6,7,8
- C: 7,8,3,1,4,6,4,5
Instead of writing sequence A, I could tell you that the sequence is just the number 1, printed eight times. Similarly, instead of writing sequence B, I could just tell you “consider the first eight natural numbers”. But I generated Sequence C by rolling eight 8-sided dice. The information is random. There’s no way to “compress” it. No succinct summary. The only way to tell you the sequence is to just list the numbers one by one.3
Sequences A and B are “low-entropy” — in a sense they contain “less” information than Sequence C does. This manifests as them being easier to compress, in the technical sense from computing (it’s directly related to how computers “zip” files). Entropy of this sort is sometimes referred to as measuring “surprisal” — the capacity for it to surprise you. It’s not surprising that the fourth value of Sequence A is 1 — they all are! But it is surprising that the fourth value of Sequence C is 1, since it could have been any number.
The “surprisal” entropy can also be understood in terms of probability distributions. If there’s surprisal in some result, that means that it’s not guaranteed to produce one specific result: there’s a probability of getting different outcomes. If you have a set of probabilities (e.g. for a regular six-sided dice, the set is just the number 1/6 repeated six times), then that defines an entropy. The less certain your outcome is, the higher this entropy value is (since there’s more chance to be surprised by the result). This is called the Shannon entropy, after Claude Shannon, the legendary “Father of the Information Age”.
So the question becomes: is there a second law for this type of information? Does the “informational” entropy of a physical system go up over time?
This is where things get tricky, because there’s a big conceptual problem here: information-based forms of entropy are, in a very real sense, subjective. For instance, Sequence C is randomly-generated, and would be impossible to predict in advance. But suppose I give you Sequence C, and then ask you “what’s the fourth value in the sequence?” You would then be able to just look at it and say “1”. The surprisal becomes zero!4
This explains where the paradox of the entropy of an isolated system comes from. If you have a complete description of such a system, then its entropy should remain constant — there’s no surprisal. So how can there be a second law of thermodynamics that says entropy goes up?
We’re hardly the first to address this. But in our new paper, Florian Meier, Tiago Debarba, Jake Xuereb, Marcus Huber, Maximilian Lock, and I reckon we have an interesting, novel perspective on the problem specifically for quantum systems. (In particular, credit goes to Max for the main idea, and to Florian for the bulk of the work, especially mathematical heavy-lifting that made the project possible.5) Our work draws heavily from recent work on the subject of equilibration on average, or closed-system equilibration, which also informs recent work of ours on the quantum measurement problem.
The crucial insight from our work is to think not about the entropy of the physical system itself, but rather of the entropy of observations of the system. On its own, this insight isn’t entirely new, but our work is the first to rigorously apply it to isolated quantum systems where, in principle, we should have access to all the information (meaning there should be zero surprisal).
By ‘observations of the system’, we mean this: whenever we ‘measure’ some physical thing, whether it’s something in the everyday world like a car, or a quantum object like an electron, we seek to extract some specific information from it. We call the thing we’re trying to measure an observable, like the speed of the car, or the spin of the electron. Whatever it is we’re measuring, there’s a probability distribution associated with it. Before we take the measurement, there’s a certain probability of getting outcome 1, a probability of getting outcome 2, and so on. (If there was only one possible outcome for the car’s speed measurement, why would we even measure it?) This probability distribution, like with the dice roll, has an entropy associated with it — the Shannon entropy of the observable.
By looking at the system from the perspective of an observable, we’re choosing to restrict ourselves to only measuring some of the information about the system. That means there’s the chance for surprisal again, so the entropy can change over time.6 Because of this, the story from before can kick in: for an observable looking at a large system, there’s lots of ‘moving parts’ involved, so it’s more likely to go from a more certain distribution of outcomes to a less certain one. If there’s a huge collection of electrons, you’re more likely to be surprised by measurements of their spin values at later times than at earlier ones.
Our paper showed that for an observable that’s looking at an isolated quantum system, its Shannon entropy will go up, and then it will eventually settle around a maximum value. This is just like what happens with other forms of entropy, and it’s exactly what the second law of thermodynamics would predict. We demonstrated this with rigorous mathematical proofs about the dynamics of the entropy, and also illustrated the argument with a numerical simulation that showed the principle in action.
All of which is to say that, yes, the second law of thermodynamics still holds even when looking at a quantum system that’s fully isolated from any surrounding environment. You just need to find the right questions to ask about it.
In telling this story, we’re following on in the footsteps of great thinkers from the early days of both quantum mechanics and statistical physics: Boltzmann, Wigner, von Neumann, Shannon, and more. These questions were being asked as far back as the 1900s, but they took until now to be properly addressed. This is thanks to the resurgence of quantum thermodynamics in recent decades, and this is just one of many questions from the very foundations of quantum theory that we hope to address with this emerging field.
- It’s important to note that entropy is quantifiable. It can be assigned a specific numerical value. That’s what allows us to say “the entropy of object A went down by 5 units (or something), so the entropy of object B had to go up by 6 units to compensate.” If you subtract all the entropy decreases from all the entropy increases across the whole universe, the result will always be positive: more entropy is gained than lost. ↩︎
- Note that this is separate from the famous “Arrow of Time” problem (which I’ve covered for SciShow in the past). We mainly don’t address that in our work — we don’t try to explain why the past is low-entropy, we take it as a given, though we do briefly discuss the assumption. ↩︎
- The sequences “1111…” and “1234…” aren’t directly comparable due to subtleties in exactly how a source would generate them, so it’s not quite right to compare their entropies in this way, but the general point still stands. ↩︎
- Or imagine I don’t give you Sequence C in advance, but I do tell you it was generated by rolling 8d8. You’re a lot more likely to guess the next number correctly (a 1 in 8 chance) than if I didn’t tell you anything about how the sequence was generated (a 1 in ???? chance), so its surprisal is lower than it would be otherwise ↩︎
- My main contribution was some numerical simulations: writing code to illustrate our results in the form of fancy plots. By no means the main attraction but I’d like to think I also helped somewhat on the conceptual side! ↩︎
- There’s a deeper philosophical problem here, one which our work arguably leaves unanswered: how can entropy be both “subjective” in that it’s observer-dependent, and also “objective” in that it quantifies something real and physical about our universe? ↩︎