'Fracking' breaks Oklahoma

Well, maybe not yet.

This story in Zerohedge has attracted the interest of the Centre for World Complexity (that's me). So I've decided to take a little break from my ongoing travelogue of China to talk about some real geology for a change (possibly the first time this year).

Our topic today is intraplate earthquakes.But rather than reiterate what Wiki has already collated, let's apply our limited understanding of earthquakes to the situation in Oklahoma, while noting that our conclusions may also be applicable in other areas where fracking is being pursued (North Dakota? Saskatchewan? Ontario?).

Most large earthquakes happen at the edges of tectonic plates where the grind slowly past one another, but there are large stresses within plates as well. For one reason, the continental plates are all composed of small bits of tectonic material that have all become stuck together due to innumerable collisions of smaller pieces of material which couldn't be subducted. Because the plates are so large, the forces that drive them are dispersed over a large area, and stresses accumulate not only near the edges, but along any fracture that may exist within the plates interior.

Sometimes these intraplate stresses cause really big earthquakes. Perhaps the most famous such earthquakes happened in New Madrid, Missouri in the early 19th century. With a magnitude up to 8, they were the largest known earthquakes not directly related to a subduction zone in America's (known) history.

Seismic hazard map for the United States, from here.

Seismic hazard can be assessed in a couple of ways. By far the most significant approach is based on a study of the historical record of earthquakes. Hence, the two big red spots in the eastern US come from the large series of earthquakes in New Madrid in 1811 to 1812, and the Charleston earthquake in 1886. Several earthquakes have also occurred in the St. Lawrence valley (NE US) as well.

The historical record in the US is short, but the geology is long. The scale invariant nature of earthquakes allows us to estimate the recurrence interval of very large earthquakes in other places of the map (devoid of historical large earthquakes). Such recurrence intervals may be greater than a thousand years In such areas, stresses do build, albeit slowly--thus the likelihood of a large earthquake may be much greater than estimated purely on the basis of the historical record because it is so short.

Pumping liquids under pressure deep into the rocks has been correlated with small earthquakes since at least the 1960s. Our understanding of why this happens is more recent. It seems the fluids act much like a lubricant, allowing stresses that are already present in the rocks to be released. As far back as the 1960s, there were proposals for using such methods to control the build-up of stresses in the rocks and so prevent large earthquakes from occurring--however the approach has not, to my knowledge, been undertaken as a deliberate policy, probably due to liability concerns.

Now we see a series of reports and presentations (all pdf) by the Oklahoma Geological Survey showing a relationship between small earthquakes and fracking activity (the most important activity appears to be disposing of waste water in deep wells). Naturally, some people are concerned about liability.

Although many small earthquakes can be tied to oil and gas activity, no one has ever tied a large earthquake to such activity. And it is likely that no one ever will. Although the O&G activity is increasing the likelihood of small earthquakes, it is difficult to say what the impact on the likelihood of a large earthquake will be. The earth, even smaller parts of it, is a complex system, and part of what makes them interesting is that their response to stimulus is at least partially a function of the entire past history of the system. Our knowledge of earth history (especially around Oklahoma) is incomplete.

Suppose that in the absence of fracking, the recurrence interval for a magnitude 7.5 earthquake in a certain part of Oklahoma is on the order of 10,000 years (I have no idea if this is reasonable). Increasing the likelihood of small earthquakes may make a large earthquake more likely. If the last major earthquake in the area occurred in the 16th century, then probably there wouldn't have been enough time for the stresses in the system to build for a large earthquake to be triggered by fracking. But if the last big earthquake was ca. 10,000 BC, then there might be a problem.

There are costs and profits to be made from all kinds of human endeavours. Drilling for oil is one of them. I don't think we can allow the risks of induced earthquakes dissuade us from searching for oil, as it is a key determinant for economic progress. My concern is--are the people making the profits from oil exploitation going to be the ones paying the inevitable costs? 

Busy lately

Writing up a formal treatment for publication of an earlier posting. The emphasis is to be on extraction of hierarchical structure from geological (primarily geochemical) time series. I've managed to find a 20 My long (inferred) record, which should be long enough to separate the climatic and biochemical hierarchical info from the tectonic info.

Just lately I have been getting lost in the details of the role that the development of hierarchy plays in distinguishing systems that are truly complex from the merely complicated. Remember that time the Tortoise and Achilles got into a bind and they escaped into a story. And in that story they got into another bind, and escaped into another story? And then in that story they got in trouble and escaped into a painting, where they once again found themselves in peril, and escaped into another story--and towards the end they stepped out of the various stories and paintings in succession, but ended up one level short of reality? Well that's where I am now.

A system doomed to fail

In the broadest sense, there are three types of systems in the world.

The first are simple systems which are characterized by only a few variables or agents, and which can be described by perhaps a handful of equations (or even one).

The second are systems which are characterized by disorganized complexity. These may consist of huge numbers of agents or variables, and their interactions cannot be described by simple equations; yet the overall system is well-described statistically through averages and can be described as being stochastic. Such systems are typically characterized by a stable equilibrium, provided there are no external shocks to the system. They are incapable of generating internal shocks or surprises. For example, you might consider the distribution of air molecules in a room. You may not be able to predict the motion of any particular air molecule, but you can be reasonable certain that the global population won't do anything unexpected (like all move into one side of the room leaving a vacuum on the other side).

The third type of system is characterized by organized complexity. As the systems above, one may consist of many variables or agents, each of which is simple, but the system's behaviour does not lend itself to statistical description because instead of the activities of each component dissolving into a background equilibrium, large-scale (even global scale) structure "emerges" instead of seething chaos. Along with these "emergent properties", common features of such a system include multiple equilibria, adaptive behaviour, and feedbacks. There is no simple way to describe its behaviour, as much of the system's history is bound up in its behaviour (what economists call "long memory").

Complex systems, for all their unpredictability are remarkably resilient. The resilience arises from the way in which this type of system interacts with its environment--through the individual actions of its simple components, the system is able to gather information about its environment and modify its operations to adapt. Yet this adaptation and evolution all occur in the absence of central control.

The above descriptions--and characterizations of three types of systems--go back to 1948. Unfortunately it appears that Dr. Weaver was too optimistic when he recommended science develop an understanding of the third type of system "over the next 50 years". Here we are 65 years later and we have made only basic improvements in our understanding of such systems.

What has gone wrong? I think it is partly due to the limitations of the Newtonian paradigm on which science has rested over the past few hundred years.

Back to Weaver. He asks,

How can currency be wisely and effectively stabilized? To what extent is it safe to depend on the free interplay of such forces as supply and demand? To what extent must systems of economic control be employed to prevent the wide swings from prosperity to depression? These are also obviously complex problems, and they too involve analyzing systems which are organic wholes, with their parts in close interrelation.

The Fed has answered.

Sixty-five years ago, economics was known to be a complex, organized system. Yet today, the Fed continues to set policy as if the economy were a stochastic system that could be sledgehammered into whatever equilibrium state is deemed politically expedient. I would further argue that the Fed has not managed to succeed even in hammering the economy into a desirable equilibrium, but rather has mastered the ability to create artificial statistics to "justify" its actions.

The system is doomed to fail, because the resilience of natural complex systems requires freedom of action for its individual components. We do not observe resilient complex systems with central control. Yet central control is the dominant ideology of our present political and economic systems. Total control, with a vanishingly thin veneer of democracy, ephemeral as the morning dew.

Complexity, bifurcations, catastrophe

What makes a system complex?

It is a perplexing problem--both its description and its quantification. One might think that the description of a system as complex would suggest it has many subsystems each acting in accordance with its own rules, and interacting with each of the other subsystems in ways that we find difficult to describe. But there are systems involving very few "parts" which exhibit the kind of behaviour we call complex.

Another possible definition may stem from the notion of the compressibility of the system's information. Is it difficult to describe the sequential outputs in a manner that is simpler than merely listing all of our observations? A good random number generator would exhibit such behaviour, but we would not describe that as complex.

John Casti has proposed that the complexity of a system is at least partially dependent on the observer. He uses an example of a rock lying on the ground. To the layperson, there are only a limited number of ways to interact with the rock (kicking it, breaking it, throwing it, etc.) . To a trained geologist, there are more (as before, as well as mass-spectral geochem, x-radiography, electron probe, etc.). So the rock seems more complex to the geologist, but that additional complexity actually stems from the observer.

Another example can be share prices, where the complexity depends on the timing of your observations. If you look at the price of, say, Anadarko Petroleum over the past year, using closing prices only. (for disclosure--no position).

Then we can look at one-minute increments on a daily chart (Nov. 6, 2013).

Note that the variability within the two charts doesn't seem all that different despite the changes of scale. Lastly we could look at how trading in Anadarko looks over one second. One particular second, that is, between 3:59:59 and 4:00:00 on May 17 of this year.

Source

You probably wouldn't expect a lot of change over 1 s, but in this case you would be wrong: the price fell from about $90 to $0.01 in less than 50 ms. That's a loss of $1 billion in market capitalization per millisecond--keep losing money at that rate and before long you're talking real money!

This all seems to trigger a philosophical debate--is the complexity present when none of the observers are capable of seeing it? In the case of Anadarko, if you were a pension fund, your losses would have been real (although the trades were all cancelled and reversed after market close).

If the complexity of a system arises from within, then what characteristics do we ascribe to complexity. One characteristic is discontinuous behaviour, particularly when the inputs to the system are continuous. For instance, tectonic processes gradually cause stresses to accumulate in an area in a fairly uniform fashion, until a critical threshold is reached and the earthquake occurs.

The branch of mathematics that investigates the sudden onset of convulsions wrought by a slow change is called catastrophe theory. Catastrophe theory is generally considered to be a branch of bifurcation theory. By bifurcation we normally mean some change in the operations of a complex system. It could represent a transition from one stable state to another. It could also represent the development of new areas of stability in phase space (or their disappearance) or simply a change in the nature of a chaotic attractor.

In particular, sometimes the sudden appearance of a new mode of stability is brought about by the changing value of a slowly migrating parameter past a critical threshold. Such behaviour is called a catastrophe, in the mathematical sense.

In the past few years we have seen a major change in the mode of operations in the markets. In particular, the rapid growth of high-frequency trading has added complexity at timescales where such behaviour did not previously exist. This is another example of a catastrophe.

Happy anniversary chaos!

Fifty years ago, Edward Lorenz published the first paper (pdf) generally recognized to discuss chaos.

Lorenz didn't call what he had discovered 'chaos'. It's not clear that he really understood the importance of what he had discovered. He knew it was interesting, and when scientists find something interesting they publish it, and worry about the ramifications later.

What Lorenz had discovered is that a deterministic system could have unpredictability. It is difficult to convey how unexpected this discovery was at the time, because the idea of what is now called chaos has disseminated (although imperfectly) through our common culture. A deterministic system is one in which the rules are well described (via equations) which operate on data to produce results. Since the time of Laplace's Dr. Manhattan quote, it had always been assumed that nothing unexpected could arise from such a system when an initial position and the rules of motion were defined to arbitrary precision. Unexpected behaviour should only result from randomness.

So when Lorenz put together a simple model for atmospheric convection in the presence of heating, he used three simple differential equations, simple boundary conditions, and an arbitrary starting point, there would have been no reason to suspect that anything unexpected might occur. After all, all the parameters in the equations were known.

In essence, what he discovered was that minute variances in starting conditions led to extremely large variations in outcome. This again was unexpected, because our knowledge was largely built on assumptions of linear behaviour, in which small variations only grow larger slowly. Lorenz's interpretation of what he had discovered was to correctly point out that long-term weather forecasting was impossible, because it was impossible to measure the present state of the system with perfect accuracy--and the range of possible differing outcomes from the measurement accuracy was essentially the range of all possible weather.

The discovery and formalization of chaos theory led to entirely new fields of study encompassing different aspects of nonlinear dynamics and complex systems. Among them is one field of endeavour which has been a point of interest on this blog--complexity.

What do we mean by complexity? Actually, I'll write about this in a future posting. For now, let's just note the relative unpredictability of complex systems and get into the whys of it all later.

- - - - - - -

This post is a bit belated, because Lorenz's publication was actually in March. But something as momentous as chaos should celebrate over the course of an entire year.

Sometimes we go all out and celebrate something over a couple of years. International Geophysical Year (1957-58) and International Heliophysical Year (2007-08) come to mind.

There have already been numerous celebratory events so far this year. But first, a word about the enablers of this year's celebrations on the markets.

High-frequency trading spams the exchanges with empty quotes destined to be cancelled--so much that it appears that many legitimate offers do not get filled at optimum pricing, as the system becomes overwhelmed with meaningless numbers.

According to the exchanges, HFT is a good thing. It increases liquidity, or at least that is the axiom that guides their acceptance. Unfortunately, observation tells us that the opposite may be the case--that HFT causes liquidity to vanish precisely at the time it is most needed.

In the last 50 years, we entered the nonlinear world. But our thinking--especially institutional thinking--is still trapped in the linear world.

In the linear world, if something is a benefit, then more of it is a greater a benefit. But in the nonlinear world, where one may be a benefit, and two may be better; three could turn out to be horrifying.

So in celebrating 50 years of chaos, the exchanges (with their sponsors, the algos) have brought you the following celebratory events.

Flash crash on the German market. Twitter feed flash crash. (Appropriately enough, both of these were in April). Thee Anadarko flash crash. Information travels faster than the speed of light! Closing of the Nasdaq options bourse. Not to mention hundreds of strange trade executions across all the exchanges.

How to lose lots of money in 45 ms by Nanex.

Most of these problems are the (un)predictable result of the interaction of numerous algorithms. Some may have been errors, or the so-called 'fat finger' trades; others may have been other form of human or algo error.

Algo error. Was that supposed to happen?

The markets are not what they used to be. The overall superstate has changed over the last ten years from one dominated by humans to one dominated by machines. The result has a been a series of entirely new phenomena, which we have earlier termed 'innovation'.

The year isn't over yet. I look forward to the next special event. I don't think I have long to wait.

- - - - - - - - - - - - - -

And then there's this. I was going to put in something by King Crimson here, but this seemed more appropriate.

Information theoretic approaches to characterizing complex systems, part 1: complexity of reconstructed epsilon machines

Introduction

In earlier posts, I opined that the behaviour of various climate subsystems showed greater complexity in the Late Pleistocene than in the early Pleistocene. This opinion is shaped by observations of the behaviour of these varying subsystems (Himalayan monsoon strength, global ice volume, and oceanographic conditions) inferred from proxy records up to about 2 million years in length.

Any such argument would be strengthened by a number. So the challenge I will address over the next few posts in this series will be--how to characterize the complexity of the output of a dynamic series by a single number. After all, in order to compare complexity between two periods, it would be helpful to have a single parameter to compare.

The principal information theoretic concept we shall use is Shannon's (1949) measurement of entropy. The trick is deciding how to apply this parameter.

Entropy of the epsilon machine for the ice volume proxy

Let's look at epsilon machine reconstructions of the ice volume proxy.

Three separate first-order epsilon machines describe portions of the Early Pleistocene variations in the ice volume proxy. A1 represents a minimum ice state, A2 is roughly what goes for an interglacial at present, and A3 is the maximum ice state of the early Pleistocene, which would pass for a minor glacial event in the late Pleistocene.

The Mid-Pleistocene epsilon machine looks more complex.

The Late Pleistocene epsilon machine reconstruction for the ice volume looks to be more complex than any of the Early Pleistocene reconstructions. But is it? How can we tell?

The approach we will try here is to characterize the complexity by the entropy of all of the state transitions. Entropy is expressed as -Σp(i)log p(i) for all values of p(i).* Entropy is considered to be a measure of the "information" in a stream of data. This expression is normally applied in systems where Σp(i) = 1, a condition not met in the figures above. The probabilities of each pathway leading from each of the predictive states adds up to 1; so that the total of all "probabilities" adds up to the number of predictive states in the epsilon machine (two or three in the Early Quaternary, four in the mid Quaternary, six in the late Quaternary.


Do we add up the probabilities as they appear? Should we divide all probabilities by the total number of predictive states so we end up with Σ p(i) = 1? Should we weight the various probabilities to reflect the relative importance of an individual predictive state?


Let's see what happens.


First off, consider a system based not on the proxy data, but on a model. Say, the Late Quaternary global ice volume model of Paillard (1998).

Quite provocative given the current state of the economy! Actually, the I stands for interglacial regime, the M for mild glacial regime, and the F for full glacial regime. The system bumps along from state to state, but there are no probabilities listed as there is only one possible successor state from each predictive state.


Is the above system more complex, less complex, or the same as one with a single state--say, "I".


From a dynamic system, which would use topological arguments--both systems would be equally complex (or  simple, in this case), as there is no choice of successor state from each predictive state. From an information sense, it is not at all clear the two systems are the same.


I M F I M F I M F I M F I M F I M F . . . 


I I I I I I I I I I I I I I I I I I I I I I . . .


It depends on whether you allow yourself to 'group' the Is, Ms, and Fs into words, which repeat. From a geological perspective, there is a difference in the complexity between the two systems--having three separate predictive states is different than having a single repeated predictive state. 


However, if we calculate entropy [-Σp(i)log p(i)] for all the states in both systems, we come up with a value of zero. This is because the probability of each transition (I → M, M→ F, F → I or I → I in the second example) is 1. If, on the other hand, we establish the probability of each of the transitions (I → M, M → F, and F → I) as 1/3, then the entropy is 1.585, as compared to zero for the system with a single predictive state.


The implied complexity is 3x greater for the I M F system as compared to I. Seems reasonable. Now let's consider the epsilon machine construction for the Early Quaternary ice volume proxy.

There are two possibilities to recalculating the probabilities of each transition for α1, for instance: we could divide each of the probabilities by by the number of predictive states (remembering that the probability on the unlabelled A1 → A3 arrow is 1), or we could multiply the probability of each transition by the probability of the originating predictive state. In the interval from 1870-1700 ka, we find p(A1) = 0.2, p(A2) = 0.4, p(A3) = 0.4.


By method 1, the entropy for α1 is 2.13. By method 2, the entropy for α1 is 2.17. Not too different.


By both methods, the entropy for both α2 and α3 is 1.


For the Mid Pleistocene, the entropy for α4 (by method 1) is 3.23. We observe p(A1) = 0.19, p(A2) = 0.125, p(A3) = 0.31, p(A4) = 0.375, so by method 2, entropy for α4 is 3.21. Again, not much different from method 1.


For the Late Pleistocene, the entropy of α5 (by method 1) is 3.42. We observe p(A1) = 0.027, p(A2) = 0.243, p(A3) = 0.243, p(A4) = 0.297, p(A5) = 0.162, p(A6) = 0.027. The entropy of α5 is 2.85, which is considerably lower than by method 1. I think this is because observations of A1 and A6 are rare, as these predictive states are only observed once each during the Late Pleistocene.


Entropy of epsilon machines for paleomonsoon strength proxy


Now consider the reconstructed epsilon machines for the paleomonsoon strength proxy. We shall only use method 2 in calculating entropy.

Three predictive states in the Early Quaternary, dominated by M1 and M2. Given p(M1) = 0.50, p(M2) = 0.41, p(M3) = 0.09, then the entropy of μ1 is 1.83.

In the Late Quaternary, there are six predictive states, with observed probabilities as follows: p(M1) = 0.4, p(M2) = 0.2, p(M3) = 0.17, p(M4) = 0.1, p(M5) = 0.1, p(M6) = 0.03. The entropy of μ2 is 3.67.


Conclusions


Method 1 is the easier calculation, however method 2 is a better calculation. However, method 1 can be used as long as the distribution of predictive states is not too far from even.


In summary


   Time                                             Entropy (ice volume)       Entropy (paleomonsoon)


Late Pleistocene                                    2.85                                    3.67


Mid-Pleistocene                                     3.21                                   1.83


Early Pleistocene                                   1-2.1                                   1.83


By this test, the behaviour of the climate system has been more complex in the Late Pleistocene than it was in the Early Pleistocene.


In our next installment, we look at how we can use the characterize the complexity for the probability density calculation for each window shown here, to give us a nice smooth graph of complexity of the climate system through time.


References



Crutchfield, J. P., 1994. The calculi of emergence: Computation, dynamics, and induction. Physica D 75: 11-54.
Gipp, M. R., 2001. Interpretation of climate dynamics from phase space portraits: Is the climate system strange or just different? Paleoceanography, 16, 335-351.
Kukla, G., Z. S. An, J. L. Melice, J. Gavin, and J. L. Xiao, 1990. Magnetic susceptibility record of Chinese loess. Trans. R. Soc. Edinburgh Earth Sci., 81: 263-288.
Paillard, D., 2001. Glacial cycles: Toward a new paradigm. Reviews of Geophysics, 3: 325-346.
Shackleton, N. J., A. Berger, and W. R. Peltier, 1990. An alternative astronomical calibration of the Lower Pleistocene timescale based on ODP site 677, Trans. R. Soc. Edinburgh, Earth Sci., 81: 251-261.
Shannon, Claude (1949). "Communication Theory of Secrecy Systems". Bell System Technical Journal 28 (4): 656–715.



* We calculate all logarithms in a base of 2, in accordance with the nerds who came up with this concept.

First steps into complexity part 1

I will try to document some of my thinking as I moved from a standard mechanistic viewpoint of science to one that was more complex.

I have been involved in Quaternary climate studies since I began my MSc in marine geology at Memorial University of Newfoundland. There I worked with Dr. Ali Aksu ostensibly on a typical marine geology study of a sedimentary basin on the continental shelf of Nova Scotia, but I also spent some time pondering Quaternary climate change--in particular, the Milankovitch theory of astronomically driven climate change.

At first the problem was a straightforward technical problem--how to tease out the appropriate signals from marine records. In the course of background reading, I encountered a relatively unknown paper (at least by geologists), by E. N. Lorenz in Quaternary Research in 1976 (far more famous were his earlier works on nondeterminism in weather prediction). The QR paper presented alternative ideas concerning the fundamental architecture of the global climate system and challenged the geological community to test them and so determine the nature of climate change on the Quaternary timescale. Most of the literature of the time considered the climate system to be deteministic, while yet acknowledging that there were nonlinearities which complicated the whole thing--but it was clear that the nonlinearities were hoped to be local in nature and that they could be dealt with through a judicious series of fudge factors. Lorenz described three possibilities for climate: 1) a straightforward "transitive" system, in which the system outputs can be linked to the system inputs by a simple set of differential equations; 2) what he called intransitive (what we would now term multistability; i.e., a system as above but with different sets of differential equations operating at different times); and 3) what he termed "almost intransitive", and called "strange attractors" in other publications, and which we now refer to as simple chaos.

I say that the paper is poorly known as I have never seen any commentary on it. Nor, for many years, did there appear to be a clear attempt to distinguish among these different modes of operation. To be sure, there have been publications advocating any one of these modes (here and here), but most of these were attempts to show observations which supported the proposed mode, rather than using observations to test between the different modes. More recently, various climate models have been proposed in which the modal operation is taken as a given.

I finished my MSc., then shifted to University of Toronto to carry out Ph.D. research with Dr. Nick Eyles. My principal thesis was again a geological one which concerned itself with tectonic influence on the development of glaciated continental margins, using Eastern Canada and the Gulf of Alaska as contrasting examples. However I also devoted a lot of time to Lorenz's proposed problem of Quaternary climate change. My approaches to this problem followed several branches. The first was improved signal processing (mainly through alterations in Fourier transform, including attempts to use maximum entropy or other methods). The second was looking at other data sets. The third involved developing entirely new techniques for processing information.

This last approach very quickly came to absorb most of my spare time.

In 1990, the concept of fractals had been around for awhile, but its application in earth sciences was still very much leading edge (I was actually thinking of the first edition of this book). The push to educate earth science professionals had only just begun. At Scarborough there was a post-doc in geography who was trying to make a name for himself by publishing paper after paper in which he reported the fractal dimension of some geographical feature. He had published something like a dozen papers in a year, each of which I must assume, was very short.

The concept of nonlinear dynamics was also very cutting edge in earth sciences. I proposed teaching a course on the topic, going so far as to propose that we teach our own mathematics to earth science students, but the idea didn't go anywhere.

I had encountered an interesting idea in a paper by Imbrie and Imbrie, in which they proposed that it was not ice volume directly that responded to solar insolation, but the rate of change of ice volume. At the time this struck as me as a brilliant insight, and I immediately constructed a figure showing the connection between insolation in the northern hemisphere and the rate of change of ice volume calculated from first differences from a deep sea O-18 record.

Plot comparing insolation at 65N and the rate of change of ice volume from a deep sea O-18 isotope record. A panel from the ill-fated Paleoceanography paper described below.



I then had the idea of constructing a figure in which I plotted the inferred global ice volume against its rate of change, once again calculated from first differences. The graph would be a curve, in which each point would represent the "state" of the system at a particular time, and when all the points were plotted in sequence, a trajectory would be traced which should reflect the dynamics of the ice volume system.

Part of my first two-dimensional phase space reconstruction of global ice volume. The small numbers represent time in thousands of years before present (ka).









Points on the graph that lie above the x-axis represent intervals where ice volume is increasing, and ice volume is decreasing over the segments of the trajectory below the x-axis. The further from the x-axis, the more rapid the growth or retreat of global ice.  The plot above shows the relatively slow advance of glaciers from the period beginning about 120 ky ago until about 20 ky ago, followed by rapid deglaciation.

What was immediately noticeable in observing the function over the past 500 thousand years was that there were particular areas on the graph to which the function seemed attracted. It moved very rapidly towards them, and tended to stay in them for long periods of time before rapidly moving to another. All of these regions plotted along the x-axis, and corresponded to particular volumes of global ice. The location made sense, because it implied that there were particular volumes of ice which were more stable than others. During the times when ice volume was stable, its rate of change must be low--hence it would be impossible to find a small region of attraction off the x-axis.

Now this is a phase space portrait, in two dimensions, using the time-derivative method (Packard et al., 1980). At the time I did this, I didn't know what to call it. I was certain it had been done before, but even in today's world of search engines, without knowing the terminology it is very difficult to find information. I knew that I was on to something, but didn't know what.

In the meantime, I had had another idea for testing climate records for multistability--at least this was a test to distinguish multistability from the transitive case using information theory (I didn't understand enough about simple chaos to devise a test for it). My approach was that if climate had one or more stable states, then there should be measurable differences in the information between the climate record (again the deep ocean O-18 isotopic record) and the driver (which was presumed to be northern hemisphere insolation). If there were multiple stable modes of climate, then the insolation would be encrypted, as if by a polyalphabetic key, and there would be a change in a particular quantity called the index of coincidenc, which is the likelihood that two randomly selected characters in a string of text are identical. There were challenges in applying this, not the least of which that it required that the data should be 'binned' and it was not at all clear how the bin size in the observed data stream should be linked to that of the northern hemisphere insolation. This work was presented at two conferences in 1991 and 1992, and was awarded a top student paper prize in 1991. But when I wrote the paper and submitted it to Paleoceanography, I overlooked one of the cardinal rules of scientific writing.

Always look like you know what you are doing.

I have always been fascinated by the intellectual process of the scientific endeavour. This fascination lead me to make a basic mistake in presenting my experiment and results. In the course of my work I had discovered what appeared to be a novel use for the process of autoencryption--by which I mean using the message as its own key in a polyalphabetic substitution cipher. The charming result is a coded stream that cannot be unambiguously decrypted even by an intended recipient who has been furnished with the key. Such a method of encryption, understandably, had no real application, and so the behaviour of the index of coincidence for this style of encryption was not well known. However I did not discover this until I was forced to come up with an explanation for a rise in the index of coincidence in the observed signals (compared with the presumed driver).

So I wrote the paper this way. Testable hypothesis with two possible outcomes, conduct experiment, find unanticipated outcome, explain why unanticipated outcome was left out of the original hypothesis, modify hypothesis, conclusion. The paper was rejected. It may have been accepted had I submitted the modified hypothesis as the original one, tested it, and reported a result. I had thought that the process of discovery would be interesting to others. In the case of peer-reviewed journals, this view was mistaken. In the course of revisions, I came to realize that the binning issues mentioned above were unresolvable, and reluctantly abandoned this approach, returning to the reconstructed phase space portrait.

Is complexity post-Newtonian?

There are changes coming to our approach to science. But what is behind them?

In a word, complexity. What is it? It is actually very hard to define, but is frequently used to describe systems which behave unpredictably for one reason or another. By system we usually mean some interactive group of components, which may be living or not. Thus a system may be a single organism, a group of organs within an organism, a colony of related organisms, an entire ecosystem, a planet, or some portion thereof, such as the atmosphere or hydrosphere (or both together).

I will paint this in broad strokes and hopefully fill in details later. I will also link you to much better sources of information than poor me.

Complexity is frequently described as being either organized or disorganized. Disorganized complexity is used to describe systems which have so many disparate components and so many possible interactions that our ability to describe and characterize them all defies our computational abilities. The behaviour of the system might as well be random. In some senses this type of complexity is not of great intellectual interest as it is possible that as our computational and organizational skills increase, we may be able to understand the origin of the unpredictability of such systems.

Organized complexity is much more interesting. In this case we are looking at a simpler system with only a few interactions, each of which appear to be straightforward, and yet the system surprises us with unpredictable behaviours which are sometimes called “emergent properties”. (See here for a seminal paper on complexity in which emergent properties are described).

Complexity is often described as post-Newtonian, but issue is far from settled. For instance, an earlier version of the Scholarpedia article on Complexity began with such a statement, but has since been removed.

Apart from their disputes over who had precedence in the development of the calculus, Leibniz and Newton also had different metaphysical ideas about how science should proceed. 

The mechanistic approach to science is very closely associated with Newton despite having a much earlier origin. The central logic of the mechanistic view is that knowledge about a complex system can be gained by reducing it to simpler components, each of which could be understood. The reduction could be carried out repeatedly until hopefully the components were comprehensible. This approach, known as reductionism, was formulated by Descartes. The mechanistic approach to science is commonly considered to be the only approach to science. If we recall the key approach to science is the formulation and testing of hypotheses, then it is clear that the mechanistic worldview may be described as a paradigm, in that it does not define the scientific method itself, but restricts the types of hypotheses that are formulated and tested.

The mechanistic view would consider an organism to be a divisible collection of parts which, while interrelated, could be studied and understood separately.

Leibniz’s metaphysical view was considerably different. Leibniz’s metaphysics would consider the organism to be the sum or combination of an active and a passive principle: the passive principle representing the physical manifestation of the organism while the active principle was the organizing principle which caused matter and energy in the environment to form the organism. Under this approach then, it would make no sense to study an organism one component at a time, but only somehow in its entirety. Additionally, one could argue that the essential reality of the organism (or system) was the active principle, which was not something that could be perceived directly, but which would have to be inferred on the basis of observations of the passive principle.

In order to better understand the differences between these two systems, let us consider a particular complex system and look at how we would investigate it under these two different approaches.

A nicely defined complex system.

Under the Newtonian mechanistic approach we would study the system by studying all possible parts and making every possible measurement we could think of, and . . . where was I . . . we would hope somehow to gain a complete understanding of the system at the end of this process. Even with these measurements, common experience tells us that there is a little more to this system than meets the eye. We could not determine by direct measurement many of the important parameters of this system, such as her favourite music or indeed how to get her to agree to allow us to make the measurements we alluded to above.

The Leibnizian approach would suggest that the physical form of the system before us is merely a consequence of some inner truth which can't be perceived directly, but which causes the system to organize itself out of the ambient energy and matter of the surrounding environment. The Leibnizian approach would be . . . well, it's not really clear what the Leibnizian approach would be. It seems to be the central disadvantage of Leibniz's metaphysical approach to science. What sort of hypotheses can you formulate? And how do you test them? So while Newton is busily measuring the big toe, for instance, Leibniz can only wonder.

It is very difficult for us to think about this in the same way as did Leibniz, because our view is likely to be coloured by the recent concept of information as an actually quantifiable property. It is not clear to me whether or not information was viewed as a thing that could be measured in Leibniz's day, so while it is tempting for us to say that the active principle must be information—that it could be considered to be an intangible set of rules for constructing the system of which it is the active principle; I am not sure that Leibniz would have thought about it that way.

No doubt some readers are already thinking "Aha! Genetics!" And genetics could certainly qualify as information making up Leibniz's active principle in the complex system depicted above. But I am reasonably certain that Leibniz did not have secret knowledge of genetics either. So Leibniz would not be able to apply his metaphysical approach towards understanding the complex system standing in front of him.

All of this goes to explain why the mechanistic worldview came to be looked upon as the only approach to science. Under the mechanistic approach, it is generally clear what you do. You measure, codify, observe, and you will learn something, even if it wasn't what you set out to learn. Indeed, probably 99.9% of everything we have learned in science since Newton's time has come from testing hypotheses within a reductionist, mechanistic worldview.

And still . . .

There are some problems which we have not been very successful at solving, and we are beginning to doubt whether the reductionist approach will ever work. These are problems like the workings of ecosystems, and complex systems like climate. There are too many parameters to measure, we often don't know what parameters are important to measure and which can safely be ignored, the accuracy of measurements is limited, and there is a little problem called sensitivity to initial conditions.

It is only in the past thirty years or so that methodologies for codifying the behaviour of complex systems have been developed. And testing of interesting hypotheses concerning the organizational behaviour of complex systems is even more recent. The notion of self-organized criticality has a particularly "Leibnizian" feel to it. Phase space reconstructions, computational mechanics, the idea of self-organized criticality, multifractals, . . . are all ideas that are clearly moving us away from a mechanistic reductionist world view, and towards something that is more embracing of the organization of information at the centre of complex systems. However, this is not a paradigm shift, as the Newtonian approach will not be replaced, but merely enhanced by the new approaches. And, it is not a post-Newtonian approach either, as the basic idea was around in Newton's time. The difference is that we are beginning to learn how to apply it.

Meanwhile in Calgary

Here is a link to an extended abstract for a paper I presented at the GAC meeting in Calgary in May of this year.