The Knowledge Machine: How Irrationality Created Modern Science

My entry in the 2026 ACX Book Review Contest.

This was my entry to the 2026 ACX Book Review Contest. Given that it was, sadly, judged inadequate to the high standards of the ACX readership, I must now inflict it on you. Hopefully, with your obviously lower standards, you’ll enjoy it. It is ridiculously long. These warnings aside, I would urge you to at least read parts one and two.

The Knowledge Machine: How Irrationality Created Modern Science

By: Michael Strevens

0. Why This Book?

Out of the tens of millions of books ever written over the centuries, why review this one?

Because science is the secret to the creation of the modern world and this book holds the secret to the creation of science.

In particular, it answers the question: why didn’t modern science develop any sooner? Why didn’t the Chinese, the Greeks, the Romans, or the Arabs develop internal combustion engines, arrive at the germ theory of disease, or send a man to the Moon?

The answer it gives is surprising, and right there in the title. Everyone thinks that the essence of science is open inquiry, but it’s really a deliberately narrow procedural straitjacket. Modern science has many irrational qualities. It requires us to wall off our thinking. Natural philosophy was never going to succeed as long as it included philosophy. Philosophy is deep. Science only works if it’s shallow.

Strevens is not the first to delve into the workings of science. Karl Popper and Thomas Kuhn trod similar ground, and they’ll make an appearance, but Strevens incorporates the best of both. He keeps Popper’s focus on empiricism, while better describing how actual scientists work. And he acknowledges Kuhn’s assertion of paradigmatic thinking, but shows how this gets channeled into productive public argument.

By showing how science works, Strevens also demonstrates how it might stop working. And once you understand the components of the “Knowledge Machine” it would appear to be in greater peril than most people realize. This is why I chose to review this book out of all the books ever written: science needs saving, and with it the rest of the world.

I. My Father, the Physics Teacher

Let’s start with a story. I’d like you to travel back with me to a sunny spring morning in 2011. On that morning, my father took some high school students to the edge of the school’s soccer field. He had taken a midlife pivot from management consultant to high school physics teacher, and quickly discovered that it took a lot to get kids to engage.¹ Fortunately, he was not without ideas.

One of these ideas was to take all the kids outside and demonstrate the difference between the speed of light and the speed of sound. He would do this in the most exciting way possible: by igniting a large quantity of Pyrodex.²

The students were positioned at the top of some stairs overlooking the soccer field, and they all had stopwatches. My father was positioned on the opposite side of the field, in the bleachers. His job was to create an explosion. The students’ job was to time the gap between seeing the smoke and hearing the boom.

The aforementioned Pyrodex rested in a steel cylinder with an open top. On top of that he tamped in some wadding in the form of paper towels. He called it a thunder pot. Online it seems to be more common to call such a device a thunder mug, or sometimes a signal cannon. Here are a couple of pictures from an eBay listing.³

​​

After filling the thunder pot with explosives, you light an inserted fuse and then quickly walk away. A few seconds later, as you might imagine from the name, it creates a powerful “Boom!”

The distance between my father and the students was approximately one thousand feet.⁴ They were supposed to look for the plume of smoke from the explosion. The instant they saw it, they started a stopwatch. Then, a little less than a second later, the instant they heard the explosion, they stopped it.⁵ By doing so they could arrive at a reasonable approximation for the speed of sound, within the limits of their reflexes. This was a demonstration of science in its truest form.

At least it was according to Michael Strevens’ book The Knowledge Machine. In Strevens’ telling, this is an example of pure empirical observation, the core of what separates real science from the broader field of philosophy.

We are always confronted with multiple theories. Perhaps one theory is that the speed of light and the speed of sound are the same. Perhaps another claims that the speed of sound is the same regardless of elevation. Opposed to these are theories which say the speed of sound and light are different, or that the speed varies based on elevation. But the presence of competing explanations is not the important part.

Strevens points out that we’ve had competing theories about the world since before Socrates. In his telling, the important thing is a methodology for deciding between these competing theories. And the only way to do that is to get both sides to agree on some empirical test which will bring them closer to choosing between the theories. Then they conduct that experiment and make their observation.

When Strevens says this is the only way, he means it. It’s important not to stray outside of the experiment—for example, with an appeal to authority, or even the argument that one explanation is more aesthetically satisfying. Strevens calls this “the iron rule of explanation”.

What the rule says is simple enough: it directs scientists to resolve their differences of opinion by conducting empirical tests rather than by shouting or fighting or philosophizing or moralizing or marrying or calling on a higher power.

What my father was doing back in 2011 was an example of this. The iron rule simplifies things down to tests and observation. In this case: thunder pots, stopwatches, and a thousand feet of athletic field.

In addition to being a great illustration of Strevens’ iron rule, the events of that morning were also notable for another reason. They led to my father getting fired.⁶ A particularly sanctimonious district official heard about the demonstration and went on a crusade against my father. (The phrase “well it’s basically a pipe bomb” was hammered into the ground.) It did not matter that he had done everything by the book. Nor did it matter that he had conducted the same demonstration in previous years without complaint. What mattered was that someone decided it was unsafe. Their righteous certainty, along with some especially gutless administrators, ensured that it was the last year he worked at that school.

Scientific inquiry has always required sacrifice. In Strevens’ account, people had to sacrifice everything that wasn’t empiricism. In my father’s case, he had to sacrifice his job. The two things are not unrelated. I’m sure the administrators at that school thought it was very important for their students to learn “science”. I’m sure they’re grateful for all of the conveniences provided by science. They would almost certainly admit that the modern world owes its existence to science. But they had lots of other things on their mind: parents to placate, careers to protect, and teachers to harass. All of these things are rational (if cowardly). And in this sense they provide an interesting analogy to Strevens’ point. Science is about stripping down to empirical evidence. You don’t bring in philosophical frameworks (or parental attitudes). You don’t tilt the scale in favor of governing bodies (or school administrators). You just look at what the evidence says. And in my father’s case, there was never any chance that the students were in the remotest danger.

Science instruction has drifted away from the pure ideal. And if Strevens is taken seriously, it’s possible science in its entirety has as well. In other words, if you can’t create an explosion to amuse some students on a beautiful spring morning, that may be a sign that modern civilization itself is in peril.⁷

II. Science Has to Be Irrational

The book’s introduction begins with these two questions:

Why is science so powerful?

Why did it take so long to arrive?

The second question is by far the more interesting one. Why didn’t the Greeks, Romans, Chinese, or Arabs create modern science? And why does Strevens claim modern science is fundamentally irrational? (An assertion he felt so strongly about that it ended up in the subtitle.) Isn’t science the height of rationality?

Apparently not, at least according to Strevens. Aristotle was plenty rational, and a keen observer of the world. But he was also a philosopher, and accordingly there was a lot of philosophy in his explanations: teleology, aesthetics, common sense, and logic. And why wouldn’t there be? If you’re trying to understand the world, why wouldn’t you bring every field of knowledge and every way of thinking to bear?

Strevens’ whole point is that the rational course of inquiry is to use all the tools at your disposal, to pull in everything you have, but that once you do, you have to make a decision on how to prioritize and integrate these various disciplines. Certainly natural philosophers observed the world around them, but they were always trying to integrate these observations into a broader philosophical framework. Or as Strevens puts it:

The natural philosophers cared a great deal about their theories’ power to explain natural phenomena. They also cared about their theories’ philosophical integrity, theological purity, and formal beauty, and they were ready and willing to make a case for their views from every one of these perspectives. The iron rule, however, permits nothing but matters of explanatory power.

Observation and measurement were just one tool among many, and certainly a tool that got a lot of use, but before modern science came along it wasn’t The Tool.

We can see this play out in Aristotle’s assertion that “superior speed in downward movement implies superior weight”⁸ or heavier things fall faster. This was wrong, but it fit in very well with a number of different frameworks. It was philosophically satisfying, commonsensical, and beautiful in its simplicity. All of the tools but one, tightly controlled observation, pointed in Aristotle’s direction. In fact, one of the biggest things that pointed the opposite way was casual observation. With so many things in favor of his explanation, there was never an incentive to make his observations rigorous.

In Strevens’ account, the transition from natural philosophy to modern science only took place when we irrationally placed limits on the kind of arguments people were allowed to make—when we restricted public arguments to empirical evidence, and only that. This was the key development.

When the story of science is told, we often highlight other things, and they were necessary but not sufficient. Aristotle, and those who followed him, engaged in empirical observation, but they mixed it in with an entire philosophical universe. They didn’t lack raw intelligence, rigor, or a willingness to experiment.⁹ They lacked a rule that said “scientific disputes can only be settled on empirical grounds”. Without this rule, they came up with lots of ingenious theories, but they lacked a reliable “Knowledge Machine” for choosing among them.

Very smart people were inquiring into the nature of things long before the Scientific Revolution. These inquiries were both sophisticated and rational, but it turned out that wasn’t the important part. The important part was creating procedural convergence, i.e., what steps can anyone take regardless of their initial opinion which will allow everyone to arrive at the same place. This convergence could only happen through empirical evidence.

Everything had to be shed but the explanation. And this explanation had to be “shallow”—an explanation of the evidence, and nothing beyond that. This is Strevens’ iron rule. Modern science could only emerge from a procedure that would have seemed absurd to premodern natural philosophers. We had to decide that people were no longer allowed to appeal to beauty, depth, metaphysics, theology, or philosophical coherence in their official scientific arguments. We had to force people to fight on one narrow strip of terrain: empirical evidence. Or to put this terrain in other terms, a thousand feet of park between a thunder pot and some student observers.

III. Francis Bacon Gets Off the Natural Philosophy Merry-Go-Round

How is it, after millennia of wandering in the rational, but unscientific, wilderness, that we eventually arrived in the promised land of pure empiricism?

In 1620, Francis Bacon, then the Lord Chancellor of England, published Novum Organum. The title was a direct reference and challenge to Aristotle and his book Organon.¹⁰ Organon means tool, or instrument, and for centuries it was what natural philosophers grabbed when they were working in the “knowledge mines”. But Bacon thought that a new tool was needed. Here’s how Strevens framed his efforts:

Bacon admired the ancient Greek natural philosophers enormously, yet they had plainly failed, he saw, in their endeavors, their inquiries going “round in circles for ever, with meager, almost negligible, progress.” Thus, “a new beginning has to be made from the most basic foundations”: the old philosophical ways would have to give way to a novel method for discerning the deep structure of the natural world.

Bacon was describing what we have already discussed. Humans had been engaged in some form of systematic inquiry for millennia. But actual progress had been “meager, almost negligible”.¹¹ Rather than converging on foundational truths they were going “round in circles for ever”. Bacon proposed to put an end to this by making a “new beginning”. But what was this new beginning?

Bacon urged people to focus squarely on the empirical observation of patterns. In particular, he said that anyone engaged in inquiry should observe three things about a subject:

1. Positive instances: Where can we see what we’re studying?

2. Negative instances: Where do we not see it?

3. Variations: In the cases where it does exist, how does it vary?

To demonstrate how his framework should be used, Bacon took heat as an example. From Strevens:

You must put aside all philosophy and therefore any prior science that is based even in part on philosophical thinking, such as the science of the ancient Greeks… These distractions and temptations Bacon called “idols”; to allow them to influence your thinking was to worship false gods rather than to revere reason.

After creating a suitably bias-free environment you should then follow these three steps:

His first step: assemble all positive instances of heat, that is, all types of circumstances in which heat is present: in fire, in the bodies of animals, when the rays of the sun are concentrated using a magnifying glass, when two solid bodies are rubbed together…

His second step: assemble all negative instances of heat, that is, all types of circumstances in which heat is not present. Of course, this list must be endless, so Bacon recommends the following alternative. For each of the positive instances, find similar circumstances in which heat is absent. For example: heat is absent in the bodies of dead animals or when two solid bodies are held together but not rubbed…

The third and final step is to assemble all the ways in which heat varies with other quantities, for example, the way that objects get hotter the closer they are to a fire or the way that metals take longer to get hot than air but retain their heat longer.

By going through this process one would converge on a solution. And because these instances and variations were available to everyone, we would all converge on the same solution. This convergence is one of the key innovations. Not only is everyone converging, but they are all converging on the true mechanisms of the universe.

Bacon used this method to eliminate heat as light and brightness, because boiling water, for instance, is not bright. And, because hot things didn’t end up weighing less as they cooled, it couldn’t be a substance. As he worked through the process in The New Organon, he demonstrated that one could only conclude “The quiddity of heat is motion and nothing else…” An early version of the kinetic theory of heat. Score one for Bacon.

This elimination looks very similar to Popperian falsification. But it’s arguably, even in 1620, describing something closer to the way scientists actually work. Bacon spends a lot of time emphasizing convergence through procedures, whereas Popper lays more emphasis on the elimination part than the convergence part. And more emphasis on the individual than on the converging collective.

Nevertheless, most of this was still just an incremental step in a journey that had been going on for a long time. The idea of controlled experiments had already been around for centuries.¹² Bacon improved on things with his proto-falsification procedures, but it still wasn’t enough to reach the promised land of pure empiricism. Much like the Children of Israel, to truly enter you had to forsake all of your previous idols, and place everything on the altar of empirical observation.

With this in mind it’s worth returning to Strevens’ question, why did it take so long? Why didn’t someone come up with the Baconian method earlier?

There are many reasons, and we’ll get to most of them by and by, but one that I want to particularly highlight is the role of the Reformation, because it laid the foundation for Bacon’s rejection of idols. It wasn’t enough to exalt empiricism—which had been around for a long time, even if it wasn’t enthroned. The hard part was removing philosophy, spirituality, and divine will from the toolkit. While Bacon’s contribution was important, the Reformation created the fertile ground in which Bacon’s exhortations could grow. Ground watered by blood.

After Luther kicked everything off in 1517, Europe went through a transformation. It was slow, and there was an enormous amount of violence. But as part of the transformation a small gap opened between the spiritual and the civic. These can be read as embryonic stand-ins for Bacon’s concept of idols and empiricism, respectively. It was a long process, and still not complete at the time of Bacon. (In 1620, the Thirty Years’ War, the bloodiest chapter of all the Wars of Religion, was just getting started.) But gradually civic institutions ceased to be synonymous with religious institutions; they were still near synonyms, but if you squinted you could start to see some daylight between the two. Public discourse was starting to develop different rules.

People had always held private beliefs that contradicted their public behavior, but the Reformation and the subsequent Wars of Religion changed that from something covert into something that was deemed necessary for a well-functioning society. The state learned to survive by narrowing its public demands, and science, led by Bacon, learned to thrive by narrowing its public arguments.

Still, things might not have exploded like they did if someone else hadn’t arrived on the scene. Of course, we’re talking about the thunderous impact of Newton.

IV. Isaac Newton, Just Weird Enough to Make It All Work

Many people point to Galileo as the first true scientist, but for Strevens, Newton was the pivotal figure.¹³ In Strevens’ telling, Newton didn’t need to be educated in Baconian ideals, he naturally embodied them. He had an innate sequestration of his intellectual pursuits that led him to automatically take on the perfect role for whatever he was engaged in:

When he entered the alchemy lab, he not only put on the alchemist’s robe; he also assumed the alchemist’s persona, taking on their allusive language, their allegorical style of thought, and their conception of matter and the principles of chemical interactions. In his beliefs, his behavior, and his words, he became the alchemist.

When he left the lab at daybreak and went back to his investigations of gravity, the robe was left hanging among the phials, funnels, and retorts. He was now wholly the physicist, intent on using the geometrical method to explain patterns of motion and concerned exclusively with the question of which trajectories can be derived from what mathematical laws.

This role-switching meant that he wasn’t necessarily a great evangelist for the iron rule, but he was a fantastic example of it. By the time Newton first published his Principia,¹⁴ many scientists were following Bacon’s methodology, but rather than quickly converging on the correct hypothesis, as Bacon predicted, the initial effect was to multiply hypotheses. But Principia was so influential, and Newton’s theories worked so well, that the power of the purely empirical approach could not be ignored.

Newton recognized the power of the approach, explaining as much in a well-known postscript to the second edition of the Principia, which was published in 1713. In this explanation, he lays out another powerful description of the iron rule.

I have not as yet been able to deduce from phenomena the reason for these properties of gravity, and I do not feign hypotheses. For whatever is not deduced from the phenomena must be called a hypothesis; and hypotheses, whether metaphysical or physical, or based on occult qualities, or mechanical, have no place in experimental philosophy. . . . It is enough that gravity really exists and acts according to the laws that we have set forth and is sufficient to explain all the motions of the heavenly bodies and of our sea.

Strevens goes on to explain that from this:

Newton…pioneers in this passage a philosophically far shallower notion of explanation, on which a phenomenon is explained simply by deriving it from the causal principles of gravitational theory—that is, from the mathematical principles laid out in the Principia. Shallow explanation does not require the explainer to grasp the implementation of the principles. More important still, it does not require the principles to pass any philosophical test or to conform to the explanatory prescriptions…

With this act of liberation, Newton escaped the endless circles of explanatory relativism and gave scientists a “timeless, ahistorical criterion” for explanatory power to serve as the raw material for an iron rule that dictates, in turn, a fixed criterion determining what counts as a legitimate empirical test.

With Newton we finally escape the “endless circles of explanatory relativism” which had so long delayed the arrival of modern science. But this escape did not come about through one innovation, in Strevens’ telling, there were actually four, and all of them had to exist in order for natural philosophy to become science. By 1713, at the time of Newton’s famous statement, they all existed in one form or another, though some were still being fleshed out:

The Four Innovations That Made Modern Science

1. A notion of explanatory power on which all scientists agree: Newton clearly laid this out in his postscript—the explanatory power of his equations was enough, no further explanation was required. This was so obviously true that it quickly became the approach everyone aspired to.

2. A distinction between public scientific argument and private scientific reasoning: This innovation was being birthed through a long-term shift in culture, the massive changes I alluded to in my discussion of the Wars of Religion. By Newton’s time, the distinction was in the water, but it hadn’t yet become the expected standard. (Newton was weird in that he innately enforced it on himself.)

3. A requirement of objectivity in scientific argument (as opposed to reasoning): Here Strevens is making a distinction between objectivity in private reasoning (which is impossible) and requiring that public debates be objective. Newton’s weirdness created this distinction in his own work, but it took a while before the dry scientific paper came to embody it.

4. A requirement that scientific argument appeal only to the outcomes of empirical tests (and not to philosophical coherence, theoretical beauty, and so on): This was Bacon’s major argument, and one of the things that got the ball rolling, but Newton really solidified it when he said, “Hypotheses, whether metaphysical or physical, or based on occult qualities, or mechanical, have no place in experimental philosophy.”

To return to the questions the book starts with, the reason true/modern science took so long to arrive is that it needed several different innovations, all of which were irrational. Why should public argument operate under different rules than private reasoning? Why should we limit our scientific investigation to pure empiricism? And why is it so important for science to converge?

In retrospect, all of these things seem obvious, particularly given how powerful they have been in application. But to understand that power, consider the example of quantum mechanics. Everyone has a favorite interpretation of quantum mechanics (Innovation 1), but no one debates the validity of the actual equations (Innovation 3). Quantum effects are used to explain all sorts of new-agey phenomena, from the law of attraction to crystal healing, but that has had no impact on the science (Innovation 4). And despite the fact that quantum experiments (for example the double-slit experiment) are profoundly counterintuitive, scientists have nevertheless refused to appeal to anything outside of the empirical results of these decidedly strange experiments (Innovation 2). All of this is crystallized in something the physicist N. David Mermin said in 1989, in a dismissal of all interpretations: “Shut up and calculate!” A phrase which perfectly embodies Strevens’ shallow conduct of science.

You’ll perhaps forgive me if I see parallels between this and my father’s experience with school administrators. Scientific education is not the same as scientific investigation, but there is still a lot of overlap. In my father’s case, a powerful and shallow (in the Strevensian sense) demonstration of science was overwhelmed by concerns about appearance, propriety, safety, and even morality. We see something similar when we consider the current, and messy, state of modern politicized science. But first, it’s worth revisiting the long-delayed arrival of science.

V. After Centuries of Farting Around, Humanity Gets a “Real Job”

Returning to Strevens’ initial questions:

Why is science so powerful?

Why did it take so long to arrive?

I’m far more interested in the second question than the first, but there’s going to be some overlap. I’m particularly interested in how modern science works to channel incentives.

According to Strevens, once everyone agrees to argue in the narrow realm of empirical evidence the only way to “win” is by collecting more evidence. As evidence accumulates, it swamps individual biases and gradually converges on the true explanation. Then, once you have that explanation—once you know how the world works—you can harness that knowledge. You can build railroads that span continents, rockets that go to the Moon, and even unlock the power of the atom.

It’s worth putting some perspective on the difference between truly modern science and what came before. From Newton’s laws of gravity to landing on the Moon was less than 300 years.¹⁵ Perhaps even more dramatically, getting from Newton to the Industrial Revolution was only around 100 years. Now consider some other societies that also had math, science, philosophy, wealth, leisure, and cultural sophistication.

Let’s begin with Athens, and the Greek city-states. Athens transformed into a democracy around 500 BC. Between then and the absorption of Greece into the Roman Republic in 146 BC, you had all the great philosophers, Archimedes, the Library of Alexandria, and the Antikythera mechanism. But in those ~350 years they didn’t even create a steam engine, to say nothing of walking on the Moon. Even though the timeframe was similar.

Rome was around for quite a bit longer, roughly twelve centuries depending on how you count. They built amazing aqueducts, durable roads, sophisticated concrete, etc. But once again, in all that time, despite having all the elements people normally associate with science they obviously weren’t able to crack the code.

Certain people I’ve talked to have pointed to the printing press as the key innovation. In that case we need to consider China. Movable type was invented in China around 1040 AD. The Chinese also invented paper, the compass, civil service examinations, and a host of other things. But modern science didn’t arrive until it was introduced from the West.

Our last example might be the most telling. The Islamic Golden Age lasted from roughly 750 AD until Baghdad was sacked by the Mongols in 1258. That’s 500 years. They were already basically starting from where the Greeks left off. As we all know, they had algebra, and optics. They even had Avicenna’s experimental rules (which covered confounders, reproducibility, and experimental controls among other things.) And yet after 500 years there were no steam engines, no railroads, and definitely no Saturn V rockets.

This is why Bacon and Newton occupy such a large role in his narrative. They’re not just two more points on the “scientific revolution” graph. They’re the ones who, respectively, stated and then demonstrated the iron rule. The thing that actually produced the modern world over the next few centuries. Rather than, as Bacon described, going “round in circles for ever, with meager, almost negligible, progress” for 500 or 1,000 years.

This is one of the weaknesses of Kuhn’s description of science. There wasn’t just a paradigmatic change in the transition from natural philosophy to modern science. There was a state change.

For most of human history the iron rule was not obvious. And keeping it alive is harder than people think. That’s why we need science teachers conducting demonstrations. Even (and especially) if those demonstrations involve large explosions.

VI. The Four Innovations vs. The Four Horsemen (Exhaustion, Morality, Subjectivity, and Schism)

No one worries about science not existing, and yet, as Strevens points out, for most of human history it didn’t. Should that worry us at all? For me this was the real value of the book. The creation of the “Knowledge Machine” was interesting, but advice on how to keep it running is where the book transitions from interesting history into “finger in the dike” of civilizational decline.

Strevens puts forth four innovations. And unfortunately each of them is either running out of steam, or under attack, or often both. Let’s take them in turn:

Innovation One–A notion of explanatory power on which all scientists agree:

The key phrase seems to be “explanatory power”. Lately we’re getting less explanatory power for our buck (running out of steam) and the space of what should count as an explanation is being distorted.

The initial discoveries unlocked by science were revolutionary because they delivered clear and actionable information. Going from the miasma theory of disease to the germ theory of disease was a huge step forward. And deciding between the two is an experimentally tractable problem. Turn the clock forward a century and things look a lot different. Picking a “safe distance” during the COVID pandemic was a very small step (literally) in the overall effort to fight the disease, and yet the experiments required to determine what even this, relatively minor piece of advice should be, were horribly complex. Science no longer has the same power it once had because it’s picked all the low-hanging fruit.

As science has expanded into more and more domains the problem has not gotten better. The “soft” sciences are called that for a reason, and experimental results in those domains have proven difficult to replicate. This has been labeled the replication crisis and it’s hard to imagine something more corrosive to the “iron rule of explanation”. And as far as the innovation in question is concerned, it means that a large percentage of this science has no explanatory power at all because it can’t be replicated!¹⁶

At this point forget about Popperian falsification, we’re having a hard time even getting to Strevensian replication/convergence!

As if all of this were not bad enough, explanatory power is now tied into whether that explanation fits preferred dogma. I don’t want to wade too deep into the culture wars, but a couple of examples might be useful to illustrate my point.

You may have heard of “stereotype threat”. It’s an explanation for why some minorities do worse on tests. They’ve been stereotyped as belonging to a group that does poorly on tests, and this expectation has a negative effect on their actual performance. People of a certain ideological bent love this result because it matches perfectly with how they think the world works. Unfortunately, as a broad explanatory framework, it fails to replicate.

On the other hand, there’s the broken windows theory of policing. This is the idea that when you don’t prosecute small crimes, it creates an atmosphere of lawlessness which leads to further, bigger crimes. This theory is beloved by people of a different ideological bent, because it’s a great fit for their biases. However, once again, as an overarching theory for the decrease of crime the evidence for its effectiveness is very mixed.

Innovation Two–A distinction between public scientific argument and private scientific reasoning:

I suspect the breakdown happening here is the one most obvious to the lay observer. As I just pointed out, science, like everything else, has become ideologically coded. Research priorities have always reflected the ideological priorities of the people doing the research or funding it. But lately the distinction between science and ideological signaling has largely collapsed. Whether you still wear a mask has less to do with scientific recommendations and more to do with showing your membership in a specific culture. A similar position has developed on the other side with respect to vaccine hesitancy. As always, this is most pronounced in online debates, but it also has real-world consequences.

One of the most dramatic examples of this collapse was during the summer of 2020. Previously numerous scientists had strongly recommended against large gatherings, but when the George Floyd protests started, many of those same scientists reversed their recommendation. This reversal has been well documented elsewhere, so I won’t dig into it again. I offer it up as a perfect example of the collapse of the distinction that forms the core of Innovation Two. With this act, morality is brought to bear on a scientific debate, which is precisely the thing forbidden by the iron rule.

This impulse to collapse the distinction is entirely understandable. It is even rational, but it doesn’t make for good science. Strevens had this to say:

The European seventeenth century excelled in producing minds ready to pull off this theatrical feat. [The “distinction” required by Innovation Two.] Deeply experienced with exacting or arbitrary rules of public engagement, such minds were able to play their scientific parts to perfection, becoming—while strutting the empirical stage—for all intents and purposes deaf to a chorus of urgent philosophical demands and numb to their most deeply held spiritual beliefs…

Oppression and bloodshed were the conditions under which these protean, multifarious minds evolved. We are—most of us—fortunate not to live in such dangerous and trying circumstances. Across the richer half of the globe, humanity enjoys a great degree of tolerance and openness in matters of religion, politics, and philosophy. Consistency between outer actions or words and inner beliefs can be attained without sanction, even without great effort.

And such consistency should be among our highest goals, we moderns tend to believe. Authenticity is a cardinal virtue of our age…

To conform to such precepts makes for a purer and more perfect realization of our ideal of what it means to be human. But at the same time it produces minds that are ill suited to the theatricality and normative compartmentalization that keep the iron rule in its proper place. The highest expression of liberal democracy undermines, in other words, the cognitive, emotional, and social skills needed to maintain a science that is both widely receptive—tuned in to the universe at every frequency—and intensely empirically focused.

The focus is essential; without it, the knowledge machine loses its traction on the world…

The situation is not hopeless. Scientists are pushing back against ideological influence regardless of the direction, but as Strevens points out, the scientific siloing required by the iron rule is not natural, nor does it fit easily with the “highest expression of liberal democracy”.

Innovation Three–A requirement of objectivity in scientific argument (as opposed to reasoning):

Innovation Two covers the what, Innovation Three covers the how. To take a couple of controversial examples. You could imagine one US administration refusing to engage in any research on possible harms from pediatric gender transition, while another administration might slow down the approval of new vaccines. Setting research priorities on the basis of ideology would be an erosion of Innovation Two. But perhaps the research that was being conducted is still held to the very highest objective standards. More likely, the ideological basis might lead to methodology that was also ideologically inflected. In the former case you could imagine that rather than studying harms, they might instead put forth an argument from someone’s lived experience. In the latter case you might expect to see arguments that newer vaccines are unnatural. In both cases, having abandoned the public/private distinction, it is that much easier to abandon objectivity.

Kuhn might frame this all as a transition to a different paradigm of science, but if it is, it’s a step backward. This shows the weakness of Kuhn’s approach. I’m pretty sure he’s wrong, but more than that I really hope he’s wrong.

This ties back into the “explanatory power” of Innovation One; if science doesn’t have explanatory power it’s lost all of its value. Consequently people are less likely to invest in it. People are happy to be objective when “objects” are distinct and easy to identify—when you can clearly see the puff of smoke and only later hear the boom. On the other hand, when evidence is harder to acquire, when subjects (like humans) are more difficult to experiment on, and when the questions themselves are more complicated, then objectivity is more difficult to maintain. And more importantly, the weight of objectivity has a harder time counterbalancing the gravitational pull of the scientist’s own biases.

When this happens, scientists can do one of three things: 1) They can work harder. 2) They can pull in things other than empirical evidence (appeals to authority, morality, etc.) 3) Or they can concoct fraudulent data. There are a lot of scientists out there doing one, and this is exactly what we would hope would happen. But we’re also seeing more and more of two, along with a depressing number of people engaged in genuine misconduct. As Strevens points out, objectivity has always been difficult, and unfortunately this difficulty has led to it being tenuous as well.

Innovation Four–A requirement that scientific argument appeal only to the outcomes of empirical tests (and not to philosophical coherence, theoretical beauty, and so on):

There’s a lot of overlap between the four innovations, but for the discussion of this innovation I want to highlight the iterative decision making process, the process of Baconian convergence. The iron rule of explanation is what allows science to converge on reality. This is the fundamental innovation, the one underlying all the others. And it’s the thing that was lacking for thousands of years as people went “round in circles for ever, with meager, almost negligible, progress”. In order to get this convergence there has to be something to converge on. As scientific arguments play out, the two sides have to be able to agree on empirical tests that will bring the argument closer to a conclusive answer.

As Strevens demonstrates, any other standard won’t bring about convergence. Whether it be philosophical coherence, theoretical beauty, lived experience, or a metaphysical quality of naturalness, none of these leads to a convergence on the underlying reality. None point us continually in the direction of the actual workings of the universe. And this is even if we could all agree on which principle to converge around. In reality there are numerous camps each with their own set of principles. Everything from “holistic science” to an instinctive libertarian distrust of all authority.

Much like my father, the four innovations seem to be under attack by misaligned culture, entrenched bureaucracy, and misguided morality. All the things that were shed in order for science to be created have returned to threaten its continued existence. So far science has mostly fought off these attacks, but will that continue to be true going forward?

VII. A Baconian Jihad Manifesto?

After reading the book, I was haunted by one very big question:

Does the recency of science imply that it might also be temporary?

Or to put it another way: If it was hard to acquire science, might it be hard to maintain as well?

After thinking deeply about the innovations described by Strevens, I think the answer to this question is clearly, “Yes!” The development of science was contingent on a great many factors, and relies on an unnatural way of approaching the world. Science is fragile. Given this, what should we do about it?

These days any discussion of the future has to incorporate the potential effects of AI. In the most extreme case, perhaps there’s nothing to be done because we’re only a few years away from the singularity, in which case science (and everything else?) will be solved.¹⁷ If, on the other hand, we imagine AI settling in short of superintelligence, might it nevertheless solve the problem?

One can imagine a few ways in which it might help, and a few ways it might harm. What it seems largely to be doing is speeding things up. The most obvious place where it might help is with the problem of explanatory power. AI should be able to gather evidence more effectively than humans. It will, in essence, be able to “work harder”.

What’s less clear is whether AI could do anything about the invasion of science by ideology. Could AI restore trust in vaccines among the hesitant or solve the debate over pediatric gender medicine? At least, could it do this in a broad enough fashion that it made a difference? 100% convergence is impossible, but if we could get to a point where educated people converge on the same assessment that would be a huge step forward.¹⁸ There are some early indications that AI is helping, but it’s definitely too soon to say. Meanwhile there are numerous reasons to be skeptical that AI will resolve the most intractable issues.

In other areas, there are even fewer reasons for hope. The core thesis of the book is that people rationally want to pull in arguments from a variety of different fields: morality, philosophy, beauty, and much else besides, whenever confronted with a mystery. AI makes this sort of intellectual integration vastly easier, and conversely makes the narrow domain of the iron rule that much harder to maintain. There is nothing to stop someone from giving the AI a prompt explicitly restricting it to offering up only empirical evidence. However, the vast majority of people love pulling in ancillary domains and are unlikely to impose such an “iron rule restriction” even in the rare circumstance where they’ve heard of such a thing.¹⁹

Certainly there are a lot of reasons to hope that AI will be good for science. But it also appears that the water has already overtopped the banks of the shallow channel in which science actually operates. With the landscape flooded as far as the eye can see, there’s no obvious way to drain the wetlands.

If AI is insufficient to recover things, what other options do we have? It’s tempting to imagine that we can implement some new policy, craft a new law, or require all new scientists to take a three credit hour class called “Introduction to the Iron Rule of Explanation”. But we’ve tried those sorts of things, and the problem continues to get worse.

As I was finishing up this review I came across an excellent Substack post by Adam Mastroianni that really distilled out the crux of the challenge:

But if you’re not actually seeking the truth, no amount of “rigor-enhancing practices” will ever cause you to find it. That’s why our revolution in scientific regulation has mostly failed. We require researchers who conduct clinical trials to post the results on a public website, but only 45% of them do. We tell people to specify their primary outcomes beforehand, but if their studies don’t work as planned, they just sneak in different analyses—one study on anesthesiology experiments found that 92%(!) of them did this. We make researchers end their papers by saying “data available on request” and then only 17% of them actually make their data available on request.

You can’t turn a cheat into a scientist by making a rule against cheating. The most important “rigor-enhancing practice” is caring about getting things right, and without that, nothing else matters.

Mastroianni gets to the heart of things. If we want to ensure the long-term viability of science, the thing that’s brought about all the conveniences we love—the methodology that allowed modern civilization to go where no previous civilization has gone before—we have to figure out how to get people to really care about getting the explanation right. People for whom the “iron rule” is really the IRON RULE.

This would appear to require a change in culture, or perhaps a reversion to a previous scientific culture which still exists in old books and the attitudes of old engineers (like my father). But this proximity should not lull us into thinking it will be easy. I believe focusing on culture will be more effective than new policies and procedures, but it will also be far more difficult. As the book points out, Bacon and later Newton were motivated by centuries of “meager, almost negligible, progress”. This lack of progress gave early scientists a strong incentive to do things differently. By contrast, rather than being desperate for progress, we have the opposite problem: we’re drowning in it. As such, all of our incentives flow in the opposite direction.

In a day and age where people seem to be fanatics about everything from racist knitting to voting systems, there’s a notable lack of fervor around the long-term health of science. This is not to say that people aren’t worried about the replication crisis or progress slowing. They are, but proportional to how consequential science actually is, the worries come across as strangely muted.

How do we wake people up from their torpor? How do we convince people of the scale of the problem? How do we ensure that people care about getting things right?

Those are all big and important questions, and neither the book nor I have any silver bullet to offer up. But certainly, those who recognize the value of science should resolve to be fanatical about getting things right. And, ideally, they should pass along that fanaticism by example to others. Perhaps, and I’m just spitballing here, by supporting high school teachers and other renegades, who know that if you want to teach kids the magic of science you have to blow something up from time to time.

—---------------------------------------------------

If you made it all the way to here, congratulations. Clearly I must be doing something right if you read all that. Perhaps a subscription would be in order, if you haven’t already?

1. Richard Feynman once opined that physics should not be taught in high school because high school teachers screwed it up so badly that kids only learned to hate physics instead of seeing its beauty. My father’s hope is that when he meets Feynman in the hereafter, that he will allow that, perhaps, my father was an exception to this rule. (My father’s memory was not 100% accurate, here’s what Feynman really said: “Anytime you try to teach the subjects without teachers who love the subject, it is doomed to failure and is a foolish thing to do.” Hopefully my father counts as someone who loves the subject.)

2. Pyrodex is a modern, safer substitute for black powder.

3. They’re often used in historical re-enactments to simulate cannon fire. My father’s thunder pot was homemade. He has his own metal lathe.

4. Note from my father: “There is no point in carefully measuring the distance because there is so much error in the student reaction time. In fact, of the three methods I use to measure the speed of sound, this is far and away the least accurate. However, it clearly illustrates that sound travels slower than light. Later, I have them measure the speed of sound using an echo method with a microphone and a Vernier Labquest. Finally, I use a golf tube in a water-filled cylinder, and a tuning fork to find the point of resonance. This simple, yet elegant measurement depends on hearing the increased volume when resonance is achieved. It not only provides very accurate measurements, but it helps reinforce the concept of resonance.”

5. At 68 °F (20 °C) the speed of sound in air is about 1,125 ft/s.

6. It was a de facto firing. In reality, it’s really difficult to fire a teacher, but you can do a lot to make it obvious that they’re no longer welcome at a school. One could write a whole piece about the failures of the public school system based on how it went down, but this is not that piece.

7. This is hyperbole, but only a slight one.

8. Aristotle. On the Heavens. Book I, Part 8. Translated by J. L. Stocks.

9. From the book: The eleventh-century Islamic philosopher Ibn Sīnā, called Avicenna in the West, laid down seven rules for medical experimentation. The English scholastic philosopher Robert Grosseteste developed the notion of a controlled experiment in the high Middle Ages, centuries before Bacon. And experiment was not only praised in principle but deployed in practice. The historian David Lindberg provides a partial list of experimenters at work before the Scientific Revolution that stretches from the ancient Roman Empire to Islamic Persia through the European Middle Ages, including, among many others, Ptolemy, Ibn al-Haytham (Alhazen), Kamāl al-Dīn, Theodoric of Freiberg, Rabbi Levi ben Gershon, Johannes de Muris, and Paul of Taranto.

10. Technically a collection of essays, not a true book.

11. We’ve all seen the graphs.

12. Refer back to footnote 9.

13. Galileo still held onto an overriding philosophy (mathematical/mechanistic framework) which kept him from being “shallow” enough to truly be a modern scientist.

14. Full title: Philosophiæ Naturalis Principia Mathematica

15. The Principia was published in 1687. Apollo 11 landed on July 20, 1969.

16. In social psychology specifically, somewhere between 55% and 75% failed to replicate.

17. Whether it’s solved for the AI overlords who have dispensed with humanity in their quest for perfect efficiency, or solved as one of the conditions of human utopia is obviously the big question, but opining on which it will be is far outside the scope of this review.

18. Think similar to the Moon landing. Sure not everyone agrees we landed on the Moon, but it’s still fair to describe it as the educated consensus.

19. I have a friend who is semi-obsessed with finding connections between the CIA Gateway program, quantum mechanics, and the New Testament. AIs have been an absolute gold mine for him.