Your otherwise excellent article doesn’t go deep enough. Confirmation bias is a recursive problem. You mention a “worldview,” which is basically our understanding of what’s happening — whether through the lens of our current understanding, or through that of an inculcated but generally agreed upon understanding. In the sciences, the scientific method is applied to “observations” that are already constrained and structured by the accepted worldview, and only when some irreconcilable phenomena are noted, is any attention paid to the current body of understanding.
Confirmation bias raises its ugly head frequently in scientific studies — which is a criticism of science as practiced, not the scientific method per se. Many studies are criticized because only data that is deemed relevant is incorporated into the final report, and this is nothing other than confirmation bias.
Thus, the scientific methodology leaves out a step, and I argue it is the most difficult step, but one required to do good science — fundamental assumptions about one’s subject that are entwined in the present worldview must be examined before launching into the iterative application of the steps you listed for the scientific method — even about the scientific methodology.
For example: there are two kinds of scientific practice — and not just one: the constructive scientific method and the contemplative method. Most scientific work today utilizes the scientific method that is constructive in nature, building up theoretical constructions from underlying constituent “facts,” i.e., observations constrained and structured by one’s controlling worldview.
The other kind of scientific work is contemplative and analytical and produces general principles. The former results in oftentimes tenuous attempts to explain phenomena, and are by their nature, and according to the rules of the scientific method, refutable and changing. The latter method establishes foundational principles that withstand the ruining of time. Take for example, Einstein and his theory of relativity, which is useful in order to see the differences in these methods:
Paul Mainwood, a Philosopher of Physics at the University of Oxford, explains that most people misunderstand and get the logic of Einstein’s Theory of Relativity entirely backwards. He points out that:
…this is unfortunate because this reversal is where Einstein’s originality really lies; the place where he decisively broke with Lorentz, Poincare, Fitzgerald, Heaviside and others who had pieces of the theory in their hands before him.
Getting the logic the wrong way around also leads people to worry why there is no mention in relativity of exactly how clocks slow, or why meter sticks shrink, and also to wonder why physicists always seem so sure about the particular theory of Special Relativity, when so many others have been superseded in the meantime.
Einstein specifically spoke about these two different methods — and why his process was different than the constructive scientific method — in a piece he wrote for The London Times:
We can distinguish various kinds of theories in physics. Most of them are constructive. They attempt to build up a picture of the more complex phenomena out of the materials of a relatively simple formal scheme from which they start out. Thus the kinetic theory of gases seeks to reduce mechanical, thermal, and diffusional processes to movements of molecules — i.e., to build them up out of the hypothesis of molecular motion. When we say that we have succeeded in understanding a group of natural processes, we invariably mean that a constructive theory has been found which covers the processes in question.
Along with this most important class of theories there exists a second, which I will call “principle-theories.” These employ the analytic, not the synthetic, method. The elements which form their basis and starting-point are not hypothetically constructed, but empirically discovered ones, general characteristics of natural processes, principles that give rise to mathematically formulated criteria which the separate processes or the theoretical representations of them have to satisfy.
Thus the science of thermodynamics seeks by analytical means to deduce necessary conditions, which separate events have to satisfy, from the universally experienced fact that perpetual motion is impossible. The advantages of the constructive theory are completeness, adaptability, and clearness, those of the principle theory are logical perfection and security of the foundations.
The theory of relativity belongs to the latter class.
Today, it is a kind of confirmation bias to believe that there is only one way to do science. It is also a misunderstanding of the scientific methods to believe that one always starts from the secure basis of one’s current understanding.
Finally, to return to that most difficult first step in scientific practice that I mentioned earlier, which Einstein alludes to, it is only by contemplatively observing our own mind at work as it notices, categorizes, and selects observations — as well as when these cognitive steps are done ineffectively — that we lessen the presence of confirmation biases. And of course, this applies in general to all of us even in our daily lives, because confirmation bias is a recursive problem.
Parts of this response are taken from “Meditation Is The Original Source of Science,” and there is a fuller treatment of the subject there.
