Induction, deduction, and the Newtonian paradigm, part 1

When I was in grad school, I had a roommate who was studying philosophy. One day he decided that he would no longer accept any conclusions that came from inductive reasoning, as deductive reasoning was superior. It was not long before he was refusing to accept premises like the sun would rise on the morrow, or  that he would have to eat the next day, saying that just because these things happened in the past, we could not conclude they would continue to happen in the future.

Most of what we imagine we know about the world was arrived at through inductive reasoning. From a scientific perspective, my ex-roommate's decision was catastrophic. In reality, however, it was probably no different from the way most people live their lives. He went out looking for food when he was hungry (I never investigated how he deduced where to find food). He avoided being run over by cars or starving to death, at least as long as I knew him. In a modern society, planning for future events like becoming hungry isn't really necessary--although whether he would have been as successful in the distant past is an open question.

Mathematics is deductive--this works because mathematics has little to do with the real world. You've never seen a triangle with internal angles adding up to 180 degrees. But mathematics was one of the first tools used to study the real world, so it is no surprise that early sciences proceeded on the basis of a modified form of deduction.

Deduction works on the basis of axioms--which are irreducible, true statements (at least within some sort of formal system)--and rules of inference, which allow us to generate more true statements from our axioms. These new true statements so generated are called theorems. An example of an axiom might be "all right angles are congruent". An example of a rule of inference might be "things that are equal to the same are equal to each other".

In the early days of geology, for instance, there were constant conflicts between different "schools"--for example, the Neptunists and the Plutonists. The schools were framed by their various axioms; which in the case of the Neptunists were that all rocks formed by the lithification of sediments in water, as compared with the Plutonists who believed that all rocks formed from crystallization of magma. These axioms shaped the thinking of geologists from the respective schools so much that geologists from different schools could look at the same rocks and interpret them differently. Discussions between geologists of the two schools must have been as exciting as those between the Montagues and  the Capulets (Romeo and Juliet notwithstanding).

The first revolution in science was to change its basis of operation from deductive to inductive. Central to this was the idea of formulating testable hypotheses. Although there has been a major improvement in our understanding due to the notion of testing hypotheses, something that can be overlooked is the general framework which limits the hypotheses we choose to test (and influences our approach to developing new hypotheses). This framework has been described elsewhere as a paradigm.

A paradigm is an overarching set of preconditions generally accepted by practitioners at a particular time. In one sense, it is not that different from having a set of axioms underlying your school--the one difference is that in principle, the paradigm is falsifiable, and so may be discarded should a better one come along. In reality, too many scientists have too much at stake to allow any paradigm to fall without a major struggle.

The next big debate in geology was between Catastrophism and Uniformitarianism, the central tenet of which is that the processes acting on the earth are consistent, for the reason that the laws of chemistry and physics do not change with time. The uniformitarian concept was of an ancient earth, which changes happening gradually over unimaginably long periods of time.

We can imagine Hutton, drawing on his father's experience as a lawyer, summarizing his arguments and presenting them logically as follows:

". . .as illustration of my views of those principles, and as evidece strengthening the system necessarily arising out of the admission of such principles, which . . . are neither more nor less than that no causes whatever have from the earliest time to which we can look back, to the present, ever acted, but those now acting; and that they never acted with different degrees of energy from that which they now exert." (15 Jan 1829, letter to Murchison).

Once the idea of uniformitarianism took hold, it proved to be nearly as restrictive as either the Plutonists' or Neptunists' schools. Even though geology advanced by testable hypotheses, the prevailing paradigm placed limits on the kind of hypotheses that could or would be tested. The last portion of his quote is the problem--that the forces of nature have never acted with degrees of energy greater than they do at present. That means that there were no storms greater than any we have witnessed, nor volcanic eruptions, nor earthquakes. The problems with this are well understood in the present day, but were the focus of intense debate in Hutton's time.

Hutton's intellectual opponents drew on different oratorial backgrounds. We can picture Adam Sedgwick, a Minister, thundering from the pulpit as follows:

"To assume, then, that . . . forces have not only been called into action at all times in the natural history of the earth, but also that in each period they have acted with equal intensity, seems to me a merely gratuitous hypothesis, unfounded on any of the great analogies of nature. . . This theory confounds the immutable and primary laws of matter with the mutable results arising from their irregular combination. It assumes, that in the laboratory of nature, no elements have ever been brought together which we ourselves have not seen combined; that no forces have been developed by their combination, of which we have not witnessed the effects. And what is this but to limit the riches of the kingdoms of nature by the poverty of our knowledge; and to surrender ourselves to a mischievous, but not uncommon philosophical scepticism, which makes us deny the reality of what we have not seen, and doubt the truth of what we do not perfectly comprehend."  (18 Feb 1831, address to the Geological Society).

The current view in geology is rather a combination of the two views, whereby catastrophes are part of the natural process of the world, and are therefore sprinkled throughout the geological record in conformance with the general principle of uniformitarianism. The debate as to whether or not catastrophic events are important in the geological record has been resolved in the affirmative.

The work I have presented over the course of the last year or so has a lot in common with the above Sedgwick quote. With the idea of innovation in earth (and other complex) systems is my attempt to not "limit the riches of nature to the poverty of our observations".

The epsilon machine approach to characterize the dynamics of the data set in a manner consistent with Occam's Razor is intended to be repeatable in a way that geological interpretation usually is not. It will still take time to establish whether this method is successful. The interpretation of the dynamics and exposition of its implications in terms of real earth dynamics may still vary from one geologist to another.

The Newtonian paradigm suggests that complex systems can be understood by detailed study of each of their components in isolation. Hypotheses are formulated and tested about each component, trusting that their combination would lead to a full understanding of nature. Testing multitudes of hypotheses, each of which leading to a small improvement in our knowledge, has built our current understanding of the world. The Newtonian approach has had the advantage over time, primarily because it is easier to formulate testable hypotheses.

Unfortunately, the Newtonian paradigm has its limits, and these limits have been visited in geological and biological systems. An understanding of complex systems still eludes us. But Newton had a rival, Leibniz, whose approach of viewing complex systems as essentially being characterized by information, or sets of rules which could in principle be determined from observation, as being complementary approaches. I will expand on this idea in a later posting.

Is complexity post-Newtonian?

There are changes coming to our approach to science. But what is behind them?

In a word, complexity. What is it? It is actually very hard to define, but is frequently used to describe systems which behave unpredictably for one reason or another. By system we usually mean some interactive group of components, which may be living or not. Thus a system may be a single organism, a group of organs within an organism, a colony of related organisms, an entire ecosystem, a planet, or some portion thereof, such as the atmosphere or hydrosphere (or both together).

I will paint this in broad strokes and hopefully fill in details later. I will also link you to much better sources of information than poor me.

Complexity is frequently described as being either organized or disorganized. Disorganized complexity is used to describe systems which have so many disparate components and so many possible interactions that our ability to describe and characterize them all defies our computational abilities. The behaviour of the system might as well be random. In some senses this type of complexity is not of great intellectual interest as it is possible that as our computational and organizational skills increase, we may be able to understand the origin of the unpredictability of such systems.

Organized complexity is much more interesting. In this case we are looking at a simpler system with only a few interactions, each of which appear to be straightforward, and yet the system surprises us with unpredictable behaviours which are sometimes called “emergent properties”. (See here for a seminal paper on complexity in which emergent properties are described).

Complexity is often described as post-Newtonian, but issue is far from settled. For instance, an earlier version of the Scholarpedia article on Complexity began with such a statement, but has since been removed.

Apart from their disputes over who had precedence in the development of the calculus, Leibniz and Newton also had different metaphysical ideas about how science should proceed. 

The mechanistic approach to science is very closely associated with Newton despite having a much earlier origin. The central logic of the mechanistic view is that knowledge about a complex system can be gained by reducing it to simpler components, each of which could be understood. The reduction could be carried out repeatedly until hopefully the components were comprehensible. This approach, known as reductionism, was formulated by Descartes. The mechanistic approach to science is commonly considered to be the only approach to science. If we recall the key approach to science is the formulation and testing of hypotheses, then it is clear that the mechanistic worldview may be described as a paradigm, in that it does not define the scientific method itself, but restricts the types of hypotheses that are formulated and tested.

The mechanistic view would consider an organism to be a divisible collection of parts which, while interrelated, could be studied and understood separately.

Leibniz’s metaphysical view was considerably different. Leibniz’s metaphysics would consider the organism to be the sum or combination of an active and a passive principle: the passive principle representing the physical manifestation of the organism while the active principle was the organizing principle which caused matter and energy in the environment to form the organism. Under this approach then, it would make no sense to study an organism one component at a time, but only somehow in its entirety. Additionally, one could argue that the essential reality of the organism (or system) was the active principle, which was not something that could be perceived directly, but which would have to be inferred on the basis of observations of the passive principle.

In order to better understand the differences between these two systems, let us consider a particular complex system and look at how we would investigate it under these two different approaches.

A nicely defined complex system.

Under the Newtonian mechanistic approach we would study the system by studying all possible parts and making every possible measurement we could think of, and . . . where was I . . . we would hope somehow to gain a complete understanding of the system at the end of this process. Even with these measurements, common experience tells us that there is a little more to this system than meets the eye. We could not determine by direct measurement many of the important parameters of this system, such as her favourite music or indeed how to get her to agree to allow us to make the measurements we alluded to above.

The Leibnizian approach would suggest that the physical form of the system before us is merely a consequence of some inner truth which can't be perceived directly, but which causes the system to organize itself out of the ambient energy and matter of the surrounding environment. The Leibnizian approach would be . . . well, it's not really clear what the Leibnizian approach would be. It seems to be the central disadvantage of Leibniz's metaphysical approach to science. What sort of hypotheses can you formulate? And how do you test them? So while Newton is busily measuring the big toe, for instance, Leibniz can only wonder.

It is very difficult for us to think about this in the same way as did Leibniz, because our view is likely to be coloured by the recent concept of information as an actually quantifiable property. It is not clear to me whether or not information was viewed as a thing that could be measured in Leibniz's day, so while it is tempting for us to say that the active principle must be information—that it could be considered to be an intangible set of rules for constructing the system of which it is the active principle; I am not sure that Leibniz would have thought about it that way.

No doubt some readers are already thinking "Aha! Genetics!" And genetics could certainly qualify as information making up Leibniz's active principle in the complex system depicted above. But I am reasonably certain that Leibniz did not have secret knowledge of genetics either. So Leibniz would not be able to apply his metaphysical approach towards understanding the complex system standing in front of him.

All of this goes to explain why the mechanistic worldview came to be looked upon as the only approach to science. Under the mechanistic approach, it is generally clear what you do. You measure, codify, observe, and you will learn something, even if it wasn't what you set out to learn. Indeed, probably 99.9% of everything we have learned in science since Newton's time has come from testing hypotheses within a reductionist, mechanistic worldview.

And still . . .

There are some problems which we have not been very successful at solving, and we are beginning to doubt whether the reductionist approach will ever work. These are problems like the workings of ecosystems, and complex systems like climate. There are too many parameters to measure, we often don't know what parameters are important to measure and which can safely be ignored, the accuracy of measurements is limited, and there is a little problem called sensitivity to initial conditions.

It is only in the past thirty years or so that methodologies for codifying the behaviour of complex systems have been developed. And testing of interesting hypotheses concerning the organizational behaviour of complex systems is even more recent. The notion of self-organized criticality has a particularly "Leibnizian" feel to it. Phase space reconstructions, computational mechanics, the idea of self-organized criticality, multifractals, . . . are all ideas that are clearly moving us away from a mechanistic reductionist world view, and towards something that is more embracing of the organization of information at the centre of complex systems. However, this is not a paradigm shift, as the Newtonian approach will not be replaced, but merely enhanced by the new approaches. And, it is not a post-Newtonian approach either, as the basic idea was around in Newton's time. The difference is that we are beginning to learn how to apply it.