Saturday Night Science: The Case for Space

“The Case for Space” by Robert ZubrinFifty years ago, with the successful landing of Apollo 11 on the Moon, it appeared that the road to the expansion of human activity from its cradle on Earth into the immensely larger arena of the solar system was open. The infrastructure built for Project Apollo, including that in the original 1963 development plan for the Merritt Island area could support Saturn V launches every two weeks. Equipped with nuclear-powered upper stages (under active development by Project NERVA, and accommodated in plans for a Nuclear Assembly Building near the Vehicle Assembly Building), the launchers and support facilities were more than adequate to support construction of a large space station in Earth orbit, a permanently-occupied base on the Moon, exploration of near-Earth asteroids, and manned landings on Mars in the 1980s.

But this was not to be. Those envisioning this optimistic future fundamentally misunderstood the motivation for Project Apollo. It was not about, and never was about, opening the space frontier. Instead, it was a battle for prestige in the Cold War and, once won (indeed, well before the Moon landing), the budget necessary to support such an extravagant program (which threw away skyscraper-sized rockets with every launch), began to evaporate. NASA was ready to do the Buck Rogers stuff, but Washington wasn’t about to come up with the bucks to pay for it. In 1965 and 1966, the NASA budget peaked at over 4% of all federal government spending. By calendar year 1969, when Apollo 11 landed on the Moon, it had already fallen to 2.31% of the federal budget, and with relatively small year to year variations, has settled at around one half of one percent of the federal budget in recent years. Apart from a small band of space enthusiasts, there is no public clamour for increasing NASA’s budget (which is consistently over-estimated by the public as a much larger fraction of federal spending than it actually receives), and there is no prospect for a political consensus emerging to fund an increase.

Further, there is no evidence that dramatically increasing NASA’s budget would actually accomplish anything toward the goal of expanding the human presence in space. While NASA has accomplished great things in its robotic exploration of the solar system and building space-based astronomical observatories, its human space flight operations have been sclerotic, risk-averse, loath to embrace new technologies, and seemingly more oriented toward spending vast sums of money in the districts and states of powerful representatives and senators than actually flying missions.

Fortunately, NASA is no longer the only game in town (if it can even be considered to still be in the human spaceflight game, having been unable to launch its own astronauts into space without buying seats from Russia since the retirement of the Space Shuttle in 2011). In 2009, the commission headed by Norman Augustine recommended cancellation of NASA’s Constellation Program, which aimed at a crewed Moon landing in 2020, because they estimated that the heavy-lift booster it envisioned (although based largely on decades-old Space Shuttle technology) would take twelve years and US$36 billion to develop under NASA’s business-as-usual policies; Constellation was cancelled in 2010 (although its heavy-lift booster, renamed. de-scoped, re-scoped, schedule-slipped, and cost-overrun, stumbles along, zombie-like, in the guise of the Space Launch System [SLS] which has, to date, consumed around US$14 billion in development costs without producing a single flight-ready rocket, and will probably cost between one and two billion dollars for each flight, every year or two—this farce will probably continue as long as Richard Shelby, the Alabama Senator who seems to believe NASA stands for “North Alabama Spending Agency”, remains in the World’s Greatest Deliberative Body).

In February 2018, SpaceX launched its Falcon Heavy booster, which has a payload capacity to low Earth orbit comparable to the initial version of the SLS, and was developed with private funds in half the time at one thirtieth the cost (so far) of NASA’s Big Rocket to Nowhere. Further, unlike the SLS, which on each flight will consign Space Shuttle Main Engines and Solid Rocket Boosters (which were designed to be reusable and re-flown many times on the Space Shuttle) to a watery grave in the Atlantic, three of the four components of the Falcon Heavy (excluding only its upper stage, with a single engine) are reusable and can be re-flown as many as ten times. Falcon Heavy customers will pay around US$90 million for a launch on the reusable version of the rocket, less than a tenth of what NASA estimates for an SLS flight, even after writing off its enormous development costs.

On the heels of SpaceX, Jeff Bezos’s Blue Origin is developing its New Glenn orbital launcher, which will have comparable payload capacity and a fully reusable first stage. With competition on the horizon, SpaceX is developing the Super Heavy/Starship completely-reusable launcher with a payload of around 150 tonnes to low Earth orbit: more than any past or present rocket. A fully-reusable launcher with this capacity would also be capable of delivering cargo or passengers between any two points on Earth in less than an hour at a price to passengers no more than a first class ticket on a present-day subsonic airliner. The emergence of such a market could increase the demand for rocket flights from its current hundred or so per year to hundreds or thousands a day, like airline operations, with consequent price reductions due to economies of scale and moving all components of the transportation system down the technological learning curve.

Competition-driven decreases in launch cost, compounded by partially- or fully-reusable launchers, is already dramatically decreasing the cost of getting to space. A common metric of launch cost is the price to launch one kilogram into low Earth orbit. This remained stubbornly close to US$10,000/kg from the 1960s until the entry of SpaceX’s Falcon 9 into the market in 2010. Purely by the more efficient design and operations of a profit-driven private firm as opposed to a cost-plus government contractor, the first version of the Falcon 9 cut launch costs to around US$6,000/kg. By reusing the first stage of the Falcon 9 (which costs around three times as much as the expendable second stage), this was cut by another factor of two, to US$3,000/kg. The much larger fully reusable Super Heavy/Starship is projected to reduce launch cost (if its entire payload capacity can be used on every flight, which probably isn’t the way to bet) to the vicinity of US$250/kg, and if the craft can be flown frequently, say once a day, as somebody or other envisioned more than a quarter century ago, amortising fixed costs over a much larger number of launches could reduce cost per kilogram by another factor of ten, to something like US$25/kg.

Such cost reductions are an epochal change in the space business. Ever since the first Earth satellites, launch costs have dominated the industry and driven all other aspects of spacecraft design. If you’re paying US$10,000 per kilogram to put your satellite in orbit, it makes sense to spend large sums of money not only on reducing its mass, but also making it extremely reliable, since launching a replacement would be so hideously expensive (and with flight rates so low, could result in a delay of a year or more before a launch opportunity became available). But with a hundred-fold or more reduction in launch cost and flights to orbit operating weekly or daily, satellites need no longer be built like precision watches, but rather industrial gear like that installed in telecom facilities on the ground. The entire cost structure is slashed across the board, and space becomes an arena accessible for a wide variety of commercial and industrial activities where its unique characteristics, such as access to free, uninterrupted solar power, high vacuum, and weightlessness are an advantage.

But if humanity is truly to expand beyond the Earth, launching satellites that go around and around the Earth providing services to those on its surface is just the start. People must begin to homestead in space: first hundreds, then thousands, and eventually millions and more living, working, building, raising families, with no more connection to the Earth than immigrants to the New World in the 1800s had to the old country in Europe or Asia. Where will they be living, and what will they be doing?

In order to think about the human future in the solar system, the first thing you need to do is recalibrate how you think about the Earth and its neighbours orbiting the Sun. Many people think of space as something like Antarctica: barren, difficult and expensive to reach, unforgiving, and while useful for some forms of scientific research, no place you’d want to set up industry or build communities where humans would spend their entire lives. But space is nothing like that. Ninety-nine percent or more of the matter and energy resources of the solar system—the raw material for human prosperity—are found not on the Earth, but rather elsewhere in the solar system, and they are free for the taking by whoever gets there first and figures out how to exploit them. Energy costs are a major input to most economic activity on the Earth, and wars are regularly fought over access to scarce energy resources on the home planet. But in space, at the distance Earth orbits the Sun, 1.36 kilowatts of free solar power are available for every square metre of collector you set up. And, unlike on the Earth’s surface, that power is available 24 hours a day, every day of the year, and will continue to flow for billions of years into the future.

Settling space will require using the resources available in space, not just energy but material. Trying to make a space-based economy work by launching everything from Earth is futile and foredoomed. Regardless of how much you reduce launch costs (even with exotic technologies which may not even be possible given the properties of materials, such as space elevators or launch loops), the vast majority of the mass needed by a space-based civilisation will be dumb bulk materials, not high-tech products such as microchips. Water; hydrogen and oxygen for rocket fuel (which are easily made from water using electricity from solar power); aluminium, titanium, and steel for structural components; glass and silicon; rocks and minerals for agriculture and bulk mass for radiation shielding; these will account for the overwhelming majority of the mass of any settlement in space, whether in Earth orbit, on the Moon or Mars, asteroid mining camps, or habitats in orbit around the Sun. People and low-mass, high-value added material such as electronics, scientific instruments, and the like will launch from the Earth, but their destinations will be built in space from materials found there.

Why? As with most things in space, it comes down to delta-v (pronounced delta-vee), the change in velocity needed to get from one location to another. This, not distance, determines the cost of transportation in space. The Earth’s mass creates a deep gravity well which requires around 9.8 km/sec of delta-v to get from the surface to low Earth orbit. It is providing this boost which makes launching payloads from the Earth so expensive. If you want to get to geostationary Earth orbit, where most communication satellites operate, you need another 3.8 km/sec, for a total of 13.6 km/sec launching from the Earth. By comparison, delivering a payload from the surface of the Moon to geostationary Earth orbit requires only 4 km/sec, which can be provided by a simple single-stage rocket. Delivering material from lunar orbit (placed there, for example, by a solar powered electromagnetic mass driver on the lunar surface) to geostationary orbit needs just 2.4 km/sec. Given that just about all of the materials from which geostationary satellites are built are available on the Moon (if you exploit free solar power to extract and refine them), it’s clear a mature spacefaring economy will not be launching them from the Earth, and will create large numbers of jobs on the Moon, in lunar orbit, and in ferrying cargos among various destinations in Earth-Moon space.

The author surveys the resources available on the Moon, Mars, near-Earth and main belt asteroids, and, looking farther into the future, the outer solar system where, once humans have mastered controlled nuclear fusion, sufficient Helium-3 is available for the taking to power a solar system wide human civilisation of trillions of people for billions of years and, eventually, the interstellar ships they will use to expand out into the galaxy. Detailed plans are presented for near-term human missions to the Moon and Mars, both achievable within the decade of the 2020s, which will begin the process of surveying the resources available there and building the infrastructure for permanent settlement. These mission plans, unlike those of NASA, do not rely on paper rockets which have yet to fly, costly expendable boosters, or detours to “gateways” and other diversions which seem a prime example of (to paraphrase the author in chapter 14), “doing things in order to spend money as opposed to spending money in order to do things.”

This is an optimistic and hopeful view of the future, one in which the human adventure which began when our ancestors left Africa to explore and settle the far reaches of their home planet continues outward into its neighbourhood around the Sun and eventually to the stars. In contrast to the grim Malthusian vision of mountebanks selling nostrums like a “Green New Deal”, which would have humans huddled on an increasingly crowded planet, shivering in the cold and dark when the Sun and wind did not cooperate, docile and bowed to their enlightened betters who instruct them how to reduce their expectations and hopes for the future again and again as they wait for the asteroid impact to put an end to their misery, Zubrin sketches millions of diverse human (and eventually post-human, evolving in different directions) societies, exploring and filling niches on a grand scale that dwarfs that of the Earth, inventing, building, experimenting, stumbling, and then creating ever greater things just as humans have for millennia. This is a future not just worth dreaming of, but working to make a reality. We have the enormous privilege of living in the time when, with imagination, courage, the willingness to take risks and to discard the poisonous doctrines of those who preach “sustainability” but whose policies always end in resource wars and genocide, we can actually make it happen and see the first steps taken in our lifetimes.

Zubrin, Robert. The Case for Space. Amherst, NY: Prometheus Books, 2019. ISBN 978-1-63388-534-9.

Here is an interview with the author about the topics discussed in the book.

This is a one hour and forty-two minute interview (audio only) from “The Space Show” which explores the book in detail.  The audio gets much better after the pre-recorded introduction.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Einstein’s Unfinished Revolution

“Einstein's Unfinished Revolution” by Lee SmolinIn the closing years of the nineteenth century, one of those nagging little discrepancies vexing physicists was the behaviour of the photoelectric effect. Originally discovered in 1887, the phenomenon causes certain metals, when illuminated by light, to absorb the light and emit electrons. The perplexing point was that there was a minimum wavelength (colour of light) necessary for electron emission, and for longer wavelengths, no electrons would be emitted at all, regardless of the intensity of the beam of light. For example, a certain metal might emit electrons when illuminated by green, blue, violet, and ultraviolet light, with the intensity of electron emission proportional to the light intensity, but red or yellow light, regardless of how intense, would not result in a single electron being emitted.

This didn’t make any sense. According to Maxwell’s wave theory of light, which was almost universally accepted and had passed stringent experimental tests, the energy of light depended upon the amplitude of the wave (its intensity), not the wavelength (or, reciprocally, its frequency). And yet the photoelectric effect didn’t behave that way—it appeared that whatever was causing the electrons to be emitted depended on the wavelength of the light, and what’s more, there was a sharp cut-off below which no electrons would be emitted at all.

In 1905, in one of his “miracle year” papers, “On a Heuristic Viewpoint Concerning the Production and Transformation of Light”, Albert Einstein suggested a solution to the puzzle. He argued that light did not propagate as a wave at all, but rather in discrete particles, or “quanta”, later named “photons”, whose energy was proportional to the wavelength of the light. This neatly explained the behaviour of the photoelectric effect. Light with a wavelength longer than the cut-off point was transmitted by photons whose energy was too low to knock electrons out of metal they illuminated, while those above the threshold could liberate electrons. The intensity of the light was a measure of the number of photons in the beam, unrelated to the energy of the individual photons.

This paper became one of the cornerstones of the revolutionary theory of quantum mechanics, the complete working out of which occupied much of the twentieth century. Quantum mechanics underlies the standard model of particle physics, which is arguably the most thoroughly tested theory in the history of physics, with no experiment showing results which contradict its predictions since it was formulated in the 1970s. Quantum mechanics is necessary to explain the operation of the electronic and optoelectronic devices upon which our modern computing and communication infrastructure is built, and describes every aspect of physical chemistry.

But quantum mechanics is weird. Consider: if light consists of little particles, like bullets, then why when you shine a beam of light on a barrier with two slits do you get an interference pattern with bright and dark bands precisely as you get with, say, water waves? And if you send a single photon at a time and try to measure which slit it went through, you find it always went through one or the other, but then the interference pattern goes away. It seems like whether the photon behaves as a wave or a particle depends upon how you look at it. If you have an hour, here is grand master explainer Richard Feynman (who won his own Nobel Prize in 1965 for reconciling the quantum mechanical theory of light and the electron with Einstein’s special relativity) exploring how profoundly weird the double slit experiment is.

Fundamentally, quantum mechanics seems to violate the principle of realism, which the author defines as follows.

The belief that there is an objective physical world whose properties are independent of what human beings know or which experiments we choose to do. Realists also believe that there is no obstacle in principle to our obtaining complete knowledge of this world.

This has been part of the scientific worldview since antiquity and yet quantum mechanics, confirmed by innumerable experiments, appears to indicate we must abandon it. Quantum mechanics says that what you observe depends on what you choose to measure; that there is an absolute limit upon the precision with which you can measure pairs of properties (for example position and momentum) set by the uncertainty principle; that it isn’t possible to predict the outcome of experiments but only the probability among a variety of outcomes; and that particles which are widely separated in space and time but which have interacted in the past are entangled and display correlations which no classical mechanistic theory can explain—Einstein called the latter “spooky action at a distance”. Once again, all of these effects have been confirmed by precision experiments and are not fairy castles erected by theorists.

From the formulation of the modern quantum theory in the 1920s, often called the Copenhagen interpretation after the location of the institute where one of its architects, Neils Bohr, worked, a number of eminent physicists including Einstein and Louis de Broglie were deeply disturbed by its apparent jettisoning of the principle of realism in favour of what they considered a quasi-mystical view in which the act of “measurement” (whatever that means) caused a physical change (wave function collapse) in the state of a system. This seemed to imply that the photon, or electron, or anything else, did not have a physical position until it interacted with something else: until then it was just an immaterial wave function which filled all of space and (when squared) gave the probability of finding it at that location.

In 1927, de Broglie proposed a pilot wave theory as a realist alternative to the Copenhagen interpretation. In the pilot wave theory there is a real particle, which has a definite position and momentum at all times. It is guided in its motion by a pilot wave which fills all of space and is defined by the medium through which it propagates. We cannot predict the exact outcome of measuring the particle because we cannot have infinitely precise knowledge of its initial position and momentum, but in principle these quantities exist and are real. There is no “measurement problem” because we always detect the particle, not the pilot wave which guides it. In its original formulation, the pilot wave theory exactly reproduced the predictions of the Copenhagen formulation, and hence was not a competing theory but rather an alternative interpretation of the equations of quantum mechanics. Many physicists who preferred to “shut up and calculate” considered interpretations a pointless exercise in phil-oss-o-phy, but de Broglie and Einstein placed great value on retaining the principle of realism as a cornerstone of theoretical physics. Lee Smolin sketches an alternative reality in which “all the bright, ambitious students flocked to Paris in the 1930s to follow de Broglie, and wrote textbooks on pilot wave theory, while Bohr became a footnote, disparaged for the obscurity of his unnecessary philosophy”. But that wasn’t what happened: among those few physicists who pondered what the equations meant about how the world really works, the Copenhagen view remained dominant.

In the 1950s, independently, David Bohm invented a pilot wave theory which he developed into a complete theory of nonrelativistic quantum mechanics. To this day, a small community of “Bohmians” continue to explore the implications of his theory, working on extending it to be compatible with special relativity. From a philosophical standpoint the de Broglie-Bohm theory is unsatisfying in that it involves a pilot wave which guides a particle, but upon which the particle does not act. This is an “unmoved mover”, which all of our experience of physics argues does not exist. For example, Newton’s third law of motion holds that every action has an equal and opposite reaction, and in Einstein’s general relativity, spacetime tells mass-energy how to move while mass-energy tells spacetime how to curve. It seems odd that the pilot wave could be immune from influence of the particle it guides. A few physicists, such as Jack Sarfatti, have proposed “post-quantum” extensions to Bohm’s theory in which there is back-reaction from the particle on the pilot wave, and argue that this phenomenon might be accessible to experimental tests which would distinguish post-quantum phenomena from the predictions of orthodox quantum mechanics. A few non-physicist crackpots have suggested these phenomena might even explain flying saucers.

Moving on from pilot wave theory, the author explores other attempts to create a realist interpretation of quantum mechanics: objective collapse of the wave function, as in the Penrose interpretation; the many worlds interpretation (which Smolin calls “magical realism”); and decoherence of the wavefunction due to interaction with the environment. He rejects all of them as unsatisfying, because they fail to address glaring lacunæ in quantum theory which are apparent from its very equations.

The twentieth century gave us two pillars of theoretical physics: quantum mechanics and general relativity—Einstein’s geometric theory of gravitation. Both have been tested to great precision, but they are fundamentally incompatible with one another. Quantum mechanics describes the very small: elementary particles, atoms, and molecules. General relativity describes the very large: stars, planets, galaxies, black holes, and the universe as a whole. In the middle, where we live our lives, neither much affects the things we observe, which is why their predictions seem counter-intuitive to us. But when you try to put the two theories together, to create a theory of quantum gravity, the pieces don’t fit. Quantum mechanics assumes there is a universal clock which ticks at the same rate everywhere in the universe. But general relativity tells us this isn’t so: a simple experiment shows that a clock runs slower when it’s in a gravitational field. Quantum mechanics says that it isn’t possible to determine the position of a particle without its interacting with another particle, but general relativity requires the knowledge of precise positions of particles to determine how spacetime curves and governs the trajectories of other particles. There are a multitude of more gnarly and technical problems in what Stephen Hawking called “consummating the fiery marriage between quantum mechanics and general relativity”. In particular, the equations of quantum mechanics are linear, which means you can add together two valid solutions and get another valid solution, while general relativity is nonlinear, where trying to disentangle the relationships of parts of the systems quickly goes pear-shaped and many of the mathematical tools physicists use to understand systems (in particular, perturbation theory) blow up in their faces.

Ultimately, Smolin argues, giving up realism means abandoning what science is all about: figuring out what is really going on. The incompatibility of quantum mechanics and general relativity provides clues that there may be a deeper theory to which both are approximations that work in certain domains (just as Newtonian mechanics is an approximation of special relativity which works when velocities are much less than the speed of light). Many people have tried and failed to “quantise general relativity”. Smolin suggests the problem is that quantum theory itself is incomplete: there is a deeper theory, a realistic one, to which our existing theory is only an approximation which works in the present universe where spacetime is nearly flat. He suggests that candidate theories must contain a number of fundamental principles. They must be background independent, like general relativity, and discard such concepts as fixed space and a universal clock, making both dynamic and defined based upon the components of a system. Everything must be relational: there is no absolute space or time; everything is defined in relation to something else. Everything must have a cause, and there must be a chain of causation for every event which traces back to its causes; these causes flow only in one direction. There is reciprocity: any object which acts upon another object is acted upon by that object. Finally, there is the “identity of indescernibles”: two objects which have exactly the same properties are the same object (this is a little tricky, but the idea is that if you cannot in some way distinguish two objects [for example, by their having different causes in their history], then they are the same object).

This argues that what we perceive, at the human scale and even in our particle physics experiments, as space and time are actually emergent properties of something deeper which was manifest in the early universe and in extreme conditions such as gravitational collapse to black holes, but hidden in the bland conditions which permit us to exist. Further, what we believe to be “laws” and “constants” may simply be precedents established by the universe as it tries to figure out how to handle novel circumstances. Just as complex systems like markets and evolution in ecosystems have rules that change based upon events within them, maybe the universe is “making it up as it goes along”, and in the early universe, far from today’s near-equilibrium, wild and crazy things happened which may explain some of the puzzling properties of the universe we observe today.

This needn’t forever remain in the realm of speculation. It is easy, for example, to synthesise a protein which has never existed before in the universe (it’s an example of a combinatorial explosion). You might try, for example, to crystallise this novel protein and see how difficult it is, then try again later and see if the universe has learned how to do it. To be extra careful, do it first on the International Space Station and then in a lab on the Earth. I suggested this almost twenty years ago as a test of Rupert Sheldrake’s theory of morphic resonance, but (although doubtless Smolin would shun me for associating his theory with that one), it might produce interesting results.

The book concludes with a very personal look at the challenges facing a working scientist who has concluded the paradigm accepted by the overwhelming majority of his or her peers is incomplete and cannot be remedied by incremental changes based upon the existing foundation. He notes:

There is no more reasonable bet than that our current knowledge is incomplete. In every era of the past our knowledge was incomplete; why should our period be any different? Certainly the puzzles we face are at least as formidable as any in the past. But almost nobody bets this way. This puzzles me.

Well, it doesn’t puzzle me. Ever since I learned classical economics, I’ve always learned to look at the incentives in a system. When you regard academia today, there is huge risk and little reward to get out a new notebook, look at the first blank page, and strike out in an entirely new direction. Maybe if you were a twenty-something patent examiner in a small city in Switzerland in 1905 with no academic career or reputation at risk you might go back to first principles and overturn space, time, and the wave theory of light all in one year, but today’s institutional structure makes it almost impossible for a young researcher (and revolutionary ideas usually come from the young) to strike out in a new direction. It is a blessing that we have deep thinkers such as Lee Smolin setting aside the easy path to retirement to ask these deep questions today.

Smolin, Lee. Einstein’s Unfinished Revolution. New York: Penguin Press, 2019. ISBN 978-1-59420-619-1.

Here is a lecture by the author at the Perimeter Institute about the topics discussed in the book. He concentrates mostly on the problems with quantum theory and not the speculative solutions discussed in the latter part of the book.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Connected: The Emergence of Global Consciousness

“Connected: The Emergence of Global Consciousness”, by Roger D. NelsonIn the first half of the twentieth century Pierre Teilhard de Chardin developed the idea that the process of evolution which had produced complex life and eventually human intelligence on Earth was continuing and destined to eventually reach an Omega Point in which, just as individual neurons self-organise to produce the unified consciousness and intelligence of the human brain, eventually individual human minds would coalesce (he was thinking mostly of institutions and technology, not a mystical global mind) into what he called the noosphere—a sphere of unified thought surrounding the globe just like the atmosphere. Could this be possible? Might the Internet be the baby picture of the noosphere? And if a global mind was beginning to emerge, might we be able to detect it with the tools of science? That is the subject of this book about the Global Consciousness Project, which has now been operating for more than two decades, collecting an immense data set which has been, from inception, completely transparent and accessible to anyone inclined to analyse it in any way they can imagine. Written by the founder of the project and operator of the network over its entire history, the book presents the history, technical details, experimental design, formal results, exploratory investigations from the data set, and thoughts about what it all might mean.

Over millennia, many esoteric traditions have held that “all is one”—that all humans and, in some systems of belief, all living things or all of nature are connected in some way and can interact in ways other than physical (ultimately mediated by the electromagnetic force). A common aspect of these philosophies and religions is that individual consciousness is independent of the physical being and may in some way be part of a larger, shared consciousness which we may be able to access through techniques such as meditation and prayer. In this view, consciousness may be thought of as a kind of “field” with the brain acting as a receiver in the same sense that a radio is a receiver of structured information transmitted via the electromagnetic field. Belief in reincarnation, for example, is often based upon the view that death of the brain (the receiver) does not destroy the coherent information in the consciousness field which may later be instantiated in another living brain which may, under some circumstances, access memories and information from previous hosts.

Such beliefs have been common over much of human history and in a wide variety of very diverse cultures around the globe, but in recent centuries these beliefs have been displaced by the view of mechanistic, reductionist science, which argues that the brain is just a kind of (phenomenally complicated) biological computer and that consciousness can be thought of as an emergent phenomenon which arises when the brain computer’s software becomes sufficiently complex to be able to examine its own operation. From this perspective, consciousness is confined within the brain, cannot affect the outside world or the consciousness of others except by physical interactions initiated by motor neurons, and perceives the world only through sensory neurons. There is no “consciousness field”, and individual consciousness dies when the brain does.

But while this view is more in tune with the scientific outlook which spawned the technological revolution that has transformed the world and continues to accelerate, it has, so far, made essentially zero progress in understanding consciousness. Although we have built electronic computers which can perform mathematical calculations trillions of times faster than the human brain, and are on track to equal the storage capacity of that brain some time in the next decade or so, we still don’t have the slightest idea how to program a computer to be conscious: to be self-aware and act out of a sense of free will (if free will, however defined, actually exists). So, if we adopt a properly scientific and sceptical view, we must conclude that the jury is still out on the question of consciousness. If we don’t understand enough about it to program it into a computer, then we can’t be entirely confident that it is something we could program into a computer, or that it is just some kind of software running on our brain-computer.

It looks like humans are, dare I say, programmed to believe in consciousness as a force not confined to the brain. Many cultures have developed shamanism, religions, philosophies, and practices which presume the existence of the following kinds of what Dean Radin calls Real Magic, and which I quote from my review of his book with that title.

  • Force of will: mental influence on the physical world, traditionally associated with spell-casting and other forms of “mind over matter”.
  • Divination: perceiving objects or events distant in time and space, traditionally involving such practices as reading the Tarot or projecting consciousness to other places.
  • Theurgy: communicating with non-material consciousness: mediums channelling spirits or communicating with the dead, summoning demons.

Starting in the 19th century, a small number of scientists undertook to investigate whether these phenomena could possibly be real, whether they could be demonstrated under controlled conditions, and what mechanism might explain these kinds of links between consciousness and will and the physical world. In 1882 the Society for Psychical Research was founded in London and continues to operate today, publishing three journals. Psychic research, now more commonly called parapsychology, continues to investigate the interaction of consciousness with the outside world through (unspecified) means other than the known senses, usually in laboratory settings where great care is taken to ensure no conventional transfer of information occurs and with elaborate safeguards against fraud, either by experimenters or test subjects. For a recent review of the state of parapsychology research, I recommend Dean Radin’s excellent 2006 book, Entangled Minds.

Parapsychologists such as Radin argue that while phenomena such as telepathy, precognition, and psychokinesis are very weak effects, elusive, and impossible to produce reliably on demand, the statistical evidence for their existence from large numbers of laboratory experiments is overwhelming, with a vanishingly small probability that the observed results are due to chance. Indeed, the measured confidence levels and effect sizes of some categories of parapsychological experiments exceed those of medical clinical trials such as those which resulted in the recommendation of routine aspirin administration to reduce the risk of heart disease in older males.

For more than a quarter of a century, an important centre of parapsychology research was the Princeton Engineering Anomalies Research (PEAR) laboratory, established in 1979 by Princeton University’s Dean of Engineering, Robert G. Jahn. (The lab closed in 2007 with Prof. Jahn’s retirement, and has now been incorporated into the International Consciousness Research Laboratories, which is the publisher of the present book.) An important part of PEAR’s research was with electronic random event generators (REGs) connected to computers in experiments where a subject (or “operator”, in PEAR terminology) would try to influence the generator to produce an excess of one or zero bits.  In a large series of experiments [PDF] run over a period of twelve years with multiple operators, it was reported that an influence in the direction of the operator’s intention was seen with a highly significant probability of chance of one in a trillion. The effect size was minuscule, with around one bit in ten thousand flipping in the direction of the operator’s stated goal.

If one operator can produce a tiny effect on the random data, what if many people were acting together, not necessarily with active intention, but with their consciousnesses focused on a single thing, for example at a sporting event, musical concert, or religious ceremony? The miniaturisation of electronics and computers eventually made it possible to build a portable REG and computer which could be taken into the field. This led to the FieldREG experiments in which this portable unit was taken to a variety of places and events to monitor its behaviour. The results were suggestive of an effect, but the data set was far too small to be conclusive.

Mindsong Random Event GeneratorIn 1998, Roger D. Nelson, the author of this book, realised that the rapid development and worldwide deployment of the Internet made it possible to expand the FieldREG concept to a global scale. Random event generators based upon quantum effects (usually shot noise from tunnelling across a back-biased Zener diode or a resistor) had been scaled down to small, inexpensive devices which could be attached to personal computers via an RS-232 serial port. With more and more people gaining access to the Internet (originally mostly via dial-up to commercial Internet Service Providers, then increasingly via persistent broadband connections such as ADSL service over telephone wires or a cable television connection), it might be possible to deploy a network of random event generators at locations all around the world, each of which would constantly collect timestamped data which would be transmitted to a central server, collected there, and made available to researchers for analysis by whatever means they chose to apply.

As Roger Nelson discussed the project with his son Greg (who would go on to be the principal software developer for the project), Greg suggested that what was proposed was essentially an electroencephalogram (EEG) for the hypothetical emerging global mind, an “ElectroGaiaGram” or EGG. Thus was born the “EGG Project” or, as it is now formally called, the Global Consciousness Project. Just as the many probes of an EEG provide a (crude) view into the operation of a single brain, perhaps the wide-flung, always-on network of REGs would pick up evidence of coherence when a large number of the world’s minds were focused on a single event or idea. Once the EGG project was named, terminology followed naturally: the individual hosts running the random event generators would be “eggs” and the central data archiving server the “basket”.

In April 1998, Roger Nelson released the original proposal for the project and shortly thereafter Greg Nelson began development of the egg and basket software. I became involved in the project in mid-summer 1998 and contributed code to the egg and basket software, principally to allow it to be portable to other variants of Unix systems (it was originally developed on Linux) and machines with different byte order than the Intel processors on which it ran, and also to reduce the resource requirements on the egg host, making it easier to run on a non-dedicated machine. I also contributed programs for the basket server to assemble daily data summaries from the raw data collected by the basket and to produce a real-time network status report. Evolved versions of these programs remain in use today, more than two decades later. On August 2nd, 1998, I began to run the second egg in the network, originally on a Sun workstation running Solaris; this was the first non-Linux, non-Intel, big-endian egg host in the network. A few days later, I brought up the fourth egg, running on a Sun server in the Hall of the Servers one floor below the second egg; this used a different kind of REG, but was otherwise identical. Both of these eggs have been in continuous operation from 1998 to the present (albeit with brief outages due to power failures, machine crashes, and other assorted disasters over the years), and have migrated from machine to machine over time. The second egg is now connected to Raspberry Pi running Linux, while the fourth is now hosted on a Dell Intel-based server also running Linux, which was the first egg host to run on a 64-bit machine in native mode.

Here is precisely how the network measures deviation from the expectation for genuinely random data. The egg hosts all run a Network Time Protocol (NTP) client to provide accurate synchronisation with Internet time server hosts which are ultimately synchronised to atomic clocks or GPS. At the start of every second a total of 200 bits are read from the random event generator. Since all the existing generators provide eight bits of random data transmitted as bytes on a 9600 baud serial port, this involves waiting until the start of the second, reading 25 bytes from the serial port (first flushing any potentially buffered data), then breaking the eight bits out of each byte of data. A precision timing loop guarantees that the sampling starts at the beginning of the second-long interval to the accuracy of the computer’s clock.

This process produces 200 random bits. These bits, one or zero, are summed to produce a “sample” which counts the number of one bits for that second. This sample is stored in a buffer on the egg host, along with a timestamp (in Unix time() format), which indicates when it was taken.

Buffers of completed samples are archived in files on the egg host’s file system. Periodically, the basket host will contact the egg host over the Internet and request any samples collected after the last packet it received from the egg host. The egg will then transmit any newer buffers it has filled to the basket. All communications are performed over the stateless UDP Internet protocol, and the design of the basket request and egg reply protocol is robust against loss of packets or packets being received out of order.

(This data transfer protocol may seem odd, but recall that the network was designed more than twenty years ago when many people, especially those outside large universities and companies, had dial-up Internet access. The architecture would allow a dial-up egg to collect data continuously and then, when it happened to be connected to the Internet, respond to a poll from the basket and transmit its accumulated data during the time it was connected. It also makes the network immune to random outages in Internet connectivity. Over two decades of operation, we have had exactly zero problems with Internet outages causing loss of data.)

When a buffer from an egg host is received by the basket, it is stored in a database directory for that egg. The buffer contains a time stamp identifying the second at which each sample within it was collected. All times are stored in Universal Time (UTC), so no correction for time zones or summer and winter time is required.

This is the entire collection process of the network. The basket host, which was originally located at Princeton University and now is on a server at, only stores buffers in the database. Buffers, once stored, are never modified by any other program. Bad data, usually long strings of zeroes or ones produced when a hardware random event generator fails electrically, are identified by a “sanity check” program and then manually added to a “rotten egg” database which causes these sequences to be ignored by analysis programs. The random event generators are very simple and rarely fail, so this is a very unusual circumstance.

The raw database format is difficult for analysis programs to process, so every day an automated program (which I wrote) is run which reads the basket database, extracts every sample collected for the previous 24 hour period (or any desired 24 hour window in the history of the project), and creates a day summary file with a record for every second in the day with a column for the samples from each egg which reported that day. Missing data (eggs which did not report for that second) is indicated by a blank in that column. The data are encoded in CSV format which is easy to load into a spreadsheet or read with a program. Because some eggs may not report immediately due to Internet outages or other problems, the summary data report is re-generated two days later to capture late-arriving data. You can request custom data reports for your own analysis from the Custom Data Request page. If you are interested in doing your own exploratory analysis of the Global Consciousness Project data set, you may find my EGGSHELL C++ libraries useful.

The analysis performed by the Project proceeds from these summary files as follows.

First, we observe than each sample (xi) from egg i consists of 200 bits with an expected equal probability of being zero or one. Thus each sample has a mean expectation value (μ) of 100 and a standard deviation (σ) of 7.071 (which is just the square root of half the mean value in the case of events with probability 0.5).

Then, for each sample, we can compute its Stouffer Z-score as Zi = (xi −μ) / σ. From the Z-score, it is possible to directly compute the probability that the observed deviation from the expected mean value (μ) was due to chance.

It is now possible to compute a network-wide Z-score for all eggs reporting samples in that second using Stouffer’s formula:

Summing Stouffer Z-scores

over all k eggs reporting. From this, one can compute the probability that the result from all k eggs reporting in that second was due to chance.

Squaring this composite Z-score over all k eggs gives a chi-squared distributed value we shall call V, V = Z² which has one degree of freedom. These values may be summed, yielding a chi-squared distributed number with degrees of freedom equal to the number of values summed. From the chi-squared sum and number of degrees of freedom, the probability of the result over an entire period may be computed. This gives the probability that the deviation observed by all the eggs (the number of which may vary from second to second) over the selected window was due to chance. In most of the analyses of Global Consciousness Project data an analysis window of one second is used, which avoids the need for the chi-squared summing of Z-scores across multiple seconds.

The most common way to visualise these data is a “cumulative deviation plot” in which the squared Z-scores are summed to show the cumulative deviation from chance expectation over time. These plots are usually accompanied by a curve which shows the boundary for a chance probability of 0.05, or one in twenty, which is often used a criterion for significance. Here is such a plot for U.S. president Obama’s 2012 State of the Union address, an event of ephemeral significance which few people anticipated and even fewer remember.

Cumulative deviation: State of the Union 2012

What we see here is precisely what you’d expect for purely random data without any divergence from random expectation. The cumulative deviation wanders around the expectation value of zero in a “random walk” without any obvious trend and never approaches the threshold of significance. So do all of our plots look like this (which is what you’d expect)?

Well, not exactly. Now let’s look at an event which was unexpected and garnered much more worldwide attention: the death of Muammar Gadaffi (or however you choose to spell it) on 2011-10-20.

Cumulative deviation: Gadaffi killed, 2011-10-20

Now we see the cumulative deviation taking off, blowing right through the criterion of significance, and ending twelve hours later with a Z-score of 2.38 and a probability of the result being due to chance of one in 111.

What’s going on here? How could an event which engages the minds of billions of slightly-evolved apes affect the output of random event generators driven by quantum processes believed to be inherently random? Hypotheses non fingo. All, right, I’ll fingo just a little bit, suggesting that my crackpot theory of paranormal phenomena might be in play here. But the real test is not in potentially cherry-picked events such as I’ve shown you here, but the accumulation of evidence over almost two decades. Each event has been the subject of a formal prediction, recorded in a Hypothesis Registry before the data were examined. (Some of these events were predicted well in advance [for example, New Year’s Day celebrations or solar eclipses], while others could be defined only after the fact, such as terrorist attacks or earthquakes).

The significance of the entire ensemble of tests can be computed from the network results from the 500 formal predictions in the Hypothesis Registry and the network results for the periods where a non-random effect was predicted. To compute this effect, we take the formal predictions and compute a cumulative Z-score across the events. Here’s what you get.

Cumulative deviation: GCP 1998 through 2015

Now this is…interesting. Here, summing over 500 formal predictions, we have a Z-score of 7.31, which implies that the results observed were due to chance with a probability of less than one in a trillion. This is far beyond the criterion usually considered for a discovery in physics. And yet, what we have here is a tiny effect. But could it be expected in truly random data? To check this, we compare the results from the network for the events in the Hypothesis Registry with 500 simulated runs using data from a pseudorandom normal distribution.

Cumulative deviation: GCP results versus pseudorandom simulations

Since the network has been up and running continually since 1998, it was in operation on September 11, 2001, when a mass casualty terrorist attack occurred in the United States. The formally recorded prediction for this event was an elevated network variance in the period starting 10 minutes before the first plane crashed into the World Trade Center and extending for over four hours afterward (from 08:35 through 12:45 Eastern Daylight Time). There were 37 eggs reporting that day (around half the size of the fully built-out network at its largest). Here is a chart of the cumulative deviation of chi-square for that period.

Cumulative deviation of chi-square: terrorist attacks 2001-09-11

The final probability was 0.028, which is equivalent to an odds ratio of 35 to one against chance. This is not a particularly significant result, but it met the pre-specified criterion of significance of probability less than 0.05. An alternative way of looking at the data is to plot the cumulative Z-score, which shows both the direction of the deviations from expectation for randomness as well as their magnitude, and can serve as a measure of correlation among the eggs (which should not exist in genuinely random data). This and subsequent analyses did not contribute to the formal database of results from which the overall significance figures were calculated, but are rather exploratory analyses at the data to see if other interesting patterns might be present.

Cumulative deviation of Z-score: terrorist attacks 2001-09-11

Had this form of analysis and time window been chosen a priori, it would have been calculated to have a chance probability of 0.000075, or less than one in ten thousand. Now let’s look at a week-long window of time between September 7 and 13. The time of the September 11 attacks is marked by the black box. We use the cumulative deviation of chi-square from the formal analysis and start the plot of the P=0.05 envelope at that time.

Cumulative deviation of chi-square: seven day window around 2001-09-11

Another analysis looks at a 20 hour period centred on the attacks and smooths the Z-scores by averaging them within a one hour sliding window, then squares the average and converts to odds against chance.

Odds: twenty hour window around 2001-09-11, one hour smoothing

Dean Radin performed an independent analysis of the day’s data binning Z-score data into five minute intervals over the period from September 6 to 13, then calculating the odds against the result being a random fluctuation. This is plotted on a logarithmic scale of odds against chance, with each 0 on the X axis denoting midnight of each day.

Binned odds: 2001-09-06 to 2001-09-13

The following is the result when the actual GCP data from September 2001 is replaced with pseudorandom data for the same period.

Binned odds: pseudorandom data 2001-09-06 to 2001-09-13

So, what are we to make of all this? That depends upon what you, and I, and everybody else make of this large body of publicly-available, transparently-collected data assembled over more than twenty years from dozens of independently-operated sites all over the world. I don’t know about you, but I find it darned intriguing. Having been involved in the project since its very early days and seen all of the software used in data collection and archiving with my own eyes, I have complete confidence in the integrity of the data and the people involved with the project. The individual random event generators pass exhaustive randomness tests. When control runs are made by substituting data for the periods predicted in the formal tests with data collected at other randomly selected intervals from the actual physical network, the observed deviations from randomness go away, and the same happens when network data are replaced by computer-generated pseudorandom data. The statistics used in the formal analysis are all simple matters you’ll learn in an introductory stat class and are explained in my “Introduction to Probability and Statistics”.

If you’re interested in exploring further, Roger Nelson’s book is an excellent introduction to the rationale and history of the project, how it works, and a look at the principal results and what they might mean. There is also non-formal exploration of other possible effects, such as attenuation by distance, day and night sleep cycles, and effect sizes for different categories of events. There’s also quite a bit of New Age stuff which makes my engineer’s eyes glaze over, but it doesn’t detract from the rigorous information elsewhere.

The ultimate resource is the Global Consciousness Project’s sprawling and detailed Web site. Although well-designed, the site can be somewhat intimidating due to its sheer size. You can find historical documents, complete access to the full database, analyses of events, and even the complete source code for the egg and basket programs.

A Kindle edition is available.

All graphs in this article are as posted on the Global Consciousness Project Web site.

Nelson, Roger D. Connected: The Emergence of Global Consciousness. Princeton: ICRL Press, 2019. ISBN 978-1-936033-35-5.

Here is a one hour interview of author Roger D. Nelson by Jeffrey Mishlove.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Ultima Thule Encounter

New Horizon and 2014 MU69 (artists's conception)(Saturday Night Science usually appears on the first Saturday of the month.  I have moved up the January 2019 edition one week to discuss the New Horizons spacecraft fly-by of Kuiper belt object 2014 MU69, “Ultima Thule”, on New Year’s Day, January 1st, 2019.)

In January 2006 the New Horizons spacecraft was launched to explore Pluto and its moons and, if all went well, proceed onward to another object in the Kuiper Belt of the outer solar system, Pluto being one of the largest, closest, and best known members.  New Horizons was the first spacecraft launched from Earth directly on a solar system escape (interstellar) trajectory (the Pioneer and Voyager probes had earlier escaped the solar system, but only with the help of gravity assists from Jupiter and Saturn).  It was launched from Earth with such velocity (16.26 km/sec) that it passed the Moon’s orbit in just nine hours, a distance that took the Apollo missions three days to traverse.

In February 2007, New Horizons flew by Jupiter at a distance of 2.3 million km, using the planet’s gravity to increase its speed to 23 km/sec, thereby knocking three years off its transit time to Pluto.  While passing through the Jupiter system, it used its instruments to photograph the planet and its moons.  There were no further encounters with solar system objects until arrival at Pluto in 2015, and the spacecraft spent most of its time in hibernation, with most systems powered down to extend their lives, reduce staffing requirements for the support team on Earth, and free up the NASA Deep Space Network to support other missions.

As New Horizons approached Pluto, selection of possible targets for a post-Pluto extended mission became a priority.  In orbital mechanics, what matters isn’t so much distance and speed but rather “delta-v”: the change in velocity needed to divert the trajectory of a spacecraft from where it is currently headed to where you want it to go.  For chemical rockets, like the thrusters on New Horizons, this depends entirely on how much propellant is on board, and this resource would be scarce after expending what was required for the Pluto mission.  New Horizons was launched with propellant to provide 290 metres/sec delta-v, but most of this would be used in course corrections en route to Pluto and maneuvers during the Pluto encounter (the scientific instruments are fixed to the spacecraft structure, which must be turned by firing the thrusters to aim them at their targets.)  Starting in 2011, an observing campaign using large Earth-based telescopes began searching for objects in the Kuiper belt which might be suitable targets for New Horizons after Pluto.  These objects are extraordinarily difficult to observe: they are more than four billion kilometres from Earth, small, and mostly very dark, and thus visible only with the largest telescopes with long exposure times under perfectly clear and dark skies.  To make things worse, as it happens, during this time Pluto’s orbit took it past some of the densest star fields of the Milky Way, near the centre of the galaxy in the constellation of Sagittarius, so the search was cluttered with myriad background stars.  A total of 143 new Kuiper belt objects were discovered by this search, but none was reachable with the 33 kg of hydrazine monopropellant expected to remain after the Pluto encounter.

It was time to bring a bigger hammer to the job, and in June 2014, time on the Hubble Space Telescope was assigned to the search.  By October of that year three potential targets, all too faint to spot with ground-based telescopes, had been identified and called, imaginatively, potential targets PT1, PT2, and PT3.  The course change to get to PT1 would use only around 35% of New Horizons‘ remaining fuel, while the others were more difficult to reach (and thus less probable to result in a successful mission).  PT1 was chosen, and subsequently re-named “2014 MU69”, along with its minor planet number of 486958.  Subsequently, a “public outreach” effort by NASA chose the nickname “Ultima Thule”, which means a distant place beyond the known world.  A recommendation for an official name will not be made until New Horizons reveals its properties.

The fly-by of Pluto in July 2015 was a tremendous success, fulfilling all of its scientific objectives, and in October 2015 New Horizons fired its thrusters for sixteen minutes to change its velocity by 10 metres per second (equivalent to accelerating your car to 22 miles per hour), setting it on course for Ultima Thule.  Three subsequent burns would further refine the trajectory and adjust the circumstances of the fly-by.  This was the first time in history that a spacecraft was targeted to explore an object which had not been discovered when launched from Earth.  After transmitting all the data collected in the Pluto encounter to Earth, which took until October 2016, New Horizons went back into hibernation.

In June 2018, the spacecraft was awakened and in August 2018 it observed its target with its own instruments for the first time.  Measurement of its position against the background star field allowed precise determination of the inbound trajectory, which was used in final course correction maneuvers.  At the same time, the spacecraft joined Earth-based telescopes and the Hubble in a search for possible moons, rings, or dust around Ultima Thule which might damage the spacecraft on a close approach.  Had such hazards been found, the fly-by would have been re-targeted to be at a safer distance, but none was found and the original plan for a fly-by at 3500 km was selected.

Although New Horizons is bearing down on its target at a velocity of 14.4 km/sec, it will remain just a faint dot until hours before closest approach at 05:33 UTC on New Year’s Day, January 1st, 2019.  Other than its position, brightness, and colour (reddish), little or nothing is known about the properties of Ultima Thule.  We don’t know its size, shape, composition, temperature, rate of rotation, albedo (reflectivity), whether it is one object or two or more in close orbit or in contact, or anything about its history.  What is almost certain, however, is that it is nothing like anything in the solar system we’ve explored close-up so far.

Its orbit, unlike that of Pluto, is that of a conventional, well-behaved member of the Sun’s extended family.  The orbit, which takes Ultima Thule around the Sun every 296 years, is almost perfectly circular (eccentricity 0.045) and close to the ecliptic (2.45°).  (By contrast, Pluto’s orbit has an eccentricity of 0.25 and an inclination to the ecliptic of 17°.)  This makes it probable that Ultima Thule has avoided the cosmic billiards game which has perturbed the orbits of so many distant objects in the solar system, making it a “cold classical Kuiper belt object” (the “cold” refers not to temperature but its analogue in dispersion of velocity).  What this means is that it is highly probable that this body, unlike the planets and moons of the inner solar system, which have been extensively reprocessed from their original constituents, has been undisturbed since the formation of the solar system 4.5 billion years ago and is a time capsule preserving the raw materials from which the inner planets were assembled.

In 2017, predictions of Ultima Thule’s orbit indicated that it would pass in front of, or occult, a distant star, with the shadow passing through southern Argentina.  Since the distance to the object and its speed in orbit are known reasonably well, simply by measuring the duration of the star’s occultation, it is possible to compute the length of the chord of the object’s path in front of the star.  Multiple observing stations and precise timings allow estimating an object’s size and shape.  A network of twenty-four small telescopes was set up along the expected path (there is substantial uncertainty in the orbit, so not all were expected to see the occultation, but five succeeded in observing it).  Combining their results  yielded this estimation of Ultima Thule’s size and shape.

2014 MU69 Occultation resultsThe best fit was to a close binary or “contact binary”: two lobes, probably originally separate objects, in contact with one another.  What does it actually look like?  We’ll have to wait and see.  The occultation observations found no evidence for rings, moons, or a dust halo, increasing confidence in the planned close fly-by.

Another mystery which will have to await close-up observation is the absence of a pronounced light curve.  An irregularly-shaped object like Ultima Thule would be expected to vary dramatically in brightness as it rotates, but extended observations by Hubble failed to find any variation at all.  The best guess is that we’re observing it close to the pole of rotation, but again it’s anybody’s guess until we get there and take a look.

Are we there yet?  No, but it won’t be long now.  As I noted, the closest fly-by will be at 05:33 UTC on 2019-01-01.  Most of the scientific data will be collected in the day before and after the moment of closest approach.  Coverage of this event will not be like what you’ve become accustomed to from other space missions.  New Horizons will be 6.6 billion kilometres from the Earth at the time of the fly-by, more than 43 times the distance of the Earth from the Sun.  It takes light (and radio waves) six hours to travel that distance, so anything transmitted to Earth will take that long to arrive.  Further, since the high-gain antenna used to send data back to Earth is fixed to the same spacecraft structure as the scientific instruments, while they are collecting data during the fly-by, the antenna won’t be pointed in the correct direction to send it back to the distant home planet.

After the scientific observations are complete, the antenna will be pointed at the Earth to send “quick look” data, spacecraft health information, and the first images.  These are expected later on the first of January and over the next few days.  To those accustomed to broadband Internet, these data arrive excruciatingly slowly.

Deep Space Network: New Horizons data downlink

Even with a 70 metre Deep Space Network antenna, the downlink rate is 501 bits per second.  If you have a 50 megabit per second broadband Internet connection, this is one hundred thousand times slower: comparable to the dial-up computer terminal (300 bits per second) I used in 1968.  It takes around an hour to return a single image, even in the compressed formats used for quick-look data.  Downloading all of the science data collected during the fly-by will begin on the 9th of January, when New Horizons returns to spin-stabilised mode (which requires no maneuvering fuel) with its antenna pointed at Earth, and is expected to take twenty months.  When the data download is complete, the spacecraft will be placed back into hibernation mode.  If another Kuiper belt target is identified which can be reached with the remaining maneuvering fuel before its nuclear power source decays or its distance to Earth becomes too great to return fly-by data (expected in the 2030s), it may be re-targeted for another fly-by.

Coverage of the New Horizons fly-by of Ultima Thule will be broadcast on the Johns Hopkins University Applied Physics Laboratory (who built the spacecraft and manages the mission) YouTube channel.  Here is a schedule of mission-related programming.  This is the mission Web site, with links to resources for the spacecraft and its destination.  This article by Emily Lakdawalla of the Planetary Society gives more detail about the encounter, when data and images will be returned, and what we can expect to see when.

I will post news and data as they arrive in the comments to this post.  If you wish to be notified when new comments are posted but don’t have a comment to add at the moment, simply post a comment consisting of the single word “follow” and you’ll receive notifications without your comment appearing.

Here is a Science Chat from September 2018 with New Horizons principal investigator Alan Stern looking ahead to the encounter with Ultima Thule.

This is a panel discussion at the American Geophysical Union meeting in December 2017 describing the preparations for the encounter with Ultima Thule and what may be learned from the fly-by.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Apollo 8 Fifty Years Ago

Apollo 8 EarthriseAs the tumultuous year 1968 drew to a close, NASA faced a serious problem with the Apollo project. The Apollo missions had been carefully planned to test the Saturn V booster rocket and spacecraft (Command/Service Module [CSM] and Lunar Module [LM]) in a series of increasingly ambitious missions, first in low Earth orbit (where an immediate return to Earth was possible in case of problems), then in an elliptical Earth orbit which would exercise the on-board guidance and navigation systems, followed by lunar orbit, and finally proceeding to the first manned lunar landing. The Saturn V had been tested in two unmanned “A” missions: Apollo 4 in November 1967 and Apollo 6 in April 1968. Apollo 5 was a “B” mission, launched on a smaller Saturn 1B booster in January 1968, to test an unmanned early model of the Lunar Module in low Earth orbit, primarily to verify the operation of its engines and separation of the descent and ascent stages. Apollo 7, launched in October 1968 on a Saturn 1B, was the first manned flight of the Command and Service modules and tested them in low Earth orbit for almost 11 days in a “C” mission.

Apollo 8 was planned to be the “D” mission, in which the Saturn V, in its first manned flight, would launch the Command/Service and Lunar modules into low Earth orbit, where the crew, commanded by Gemini veteran James McDivitt, would simulate the maneuvers of a lunar landing mission closer to home. McDivitt’s crew was trained and ready to go in December 1968. Unfortunately, the lunar module wasn’t. The lunar module scheduled for Apollo 8, LM-3, had been delivered to the Kennedy Space Center in June of 1968, but was, to put things mildly, a mess. Testing at the Cape discovered more than a hundred serious defects, and by August it was clear that there was no way LM-3 would be ready for a flight in 1968. In fact, it would probably slip to February or March 1969. This, in turn, would push the planned “E” mission, for which the crew of commander Frank Borman, command module pilot James Lovell, and lunar module pilot William Anders were training, aimed at testing the Command/Service and Lunar modules in an elliptical Earth orbit venturing as far as 7400 km from the planet and originally planned for March 1969, three months later, to June, delaying all subsequent planned missions and placing the goal of landing before the end of 1969 at risk.

But NASA were not just racing the clock—they were also racing the Soviet Union. Unlike Apollo, the Soviet space program was highly secretive and NASA had to go on whatever scraps of information they could glean from Soviet publications, the intelligence community, and independent tracking of Soviet launches and spacecraft in flight. There were, in fact, two Soviet manned lunar programmes running in parallel. The first, internally called the Soyuz 7K-L1 but dubbed “Zond” for public consumption, used a modified version of the Soyuz spacecraft launched on a Proton booster and was intended to carry two cosmonauts on a fly-by mission around the Moon. The craft would fly out to the Moon, use its gravity to swing around the far side, and return to Earth. The Zond lacked the propulsion capability to enter lunar orbit. Still, success would allow the Soviets to claim the milestone of first manned mission to the Moon. In September 1968 Zond 5 successfully followed this mission profile and safely returned a crew cabin containing tortoises, mealworms, flies, and plants to Earth after their loop around the Moon. A U.S. Navy destroyer observed recovery of the re-entry capsule in the Indian Ocean. Clearly, this was preparation for a manned mission which might occur on any lunar launch window.

(The Soviet manned lunar landing project was actually far behind Apollo, and would not launch its N1 booster on that first, disastrous, test flight until February 1969. But NASA did not know this in 1968.) Every slip in the Apollo program increased the probability of its being scooped so close to the finish line by a successful Zond flyby mission.

These were the circumstances in August 1968 when what amounted to a cabal of senior NASA managers including George Low, Chris Kraft, Bob Gilruth, and later joined by Wernher von Braun and chief astronaut Deke Slayton, began working on an alternative. They plotted in secret, beneath the radar and unbeknownst to NASA administrator Jim Webb and his deputy for manned space flight, George Mueller, who were both out of the country, attending an international conference in Vienna. What they were proposing was breathtaking in its ambition and risk. They envisioned taking Frank Borman’s crew, originally scheduled for Apollo 9, and putting them into an accelerated training program to launch on the Saturn V and Apollo spacecraft currently scheduled for Apollo 8. They would launch without a Lunar Module, and hence be unable to land on the Moon or test that spacecraft. The original idea was to perform a Zond-like flyby, but this was quickly revised to include going into orbit around the Moon, just as a landing mission would do. This would allow retiring the risk of many aspects of the full landing mission much earlier in the program than originally scheduled, and would also allow collection of precision data on the lunar gravitational field and high resolution photography of candidate landing sites to aid in planning subsequent missions. The lunar orbital mission would accomplish all the goals of the originally planned “E” mission and more, allowing that mission to be cancelled and therefore not requiring an additional booster and spacecraft.

But could it be done? There were a multitude of requirements, all daunting. Borman’s crew, training toward a launch in early 1969 on an Earth orbit mission, would have to complete training for the first lunar mission in just sixteen weeks. The Saturn V booster, which suffered multiple near-catastrophic engine failures in its second flight on Apollo 6, would have to be cleared for its first manned flight. Software for the on-board guidance computer and for Mission Control would have to be written, tested, debugged, and certified for a lunar mission many months earlier than previously scheduled. A flight plan for the lunar orbital mission would have to be written from scratch and then tested and trained in simulations with Mission Control and the astronauts in the loop. The decision to fly Borman’s crew instead of McDivitt’s was to avoid wasting the extensive training the latter crew had undergone in LM systems and operations by assigning them to a mission without an LM. McDivitt concurred with this choice: while it might be nice to be among the first humans to see the far side of the Moon with his own eyes, for a test pilot the highest responsibility and honour is to command the first flight of a new vehicle (the LM), and he would rather skip the Moon mission and fly later than lose that opportunity. If the plan were approved, Apollo 8 would become the lunar orbit mission and the Earth orbit test of the LM would be re-designated Apollo 9 and fly whenever the LM was ready.

While a successful lunar orbital mission on Apollo 8 would demonstrate many aspects of a full lunar landing mission, it would also involve formidable risks. The Saturn V, making only its third flight, was coming off a very bad outing in Apollo 6 whose failures might have injured the crew, damaged the spacecraft hardware, and precluded a successful mission to the Moon. While fixes for each of these problems had been implemented, they had never been tested in flight, and there was always the possibility of new problems not previously seen.

The Apollo Command and Service modules, which would take them to the Moon, had not yet flown a manned mission and would not until Apollo 7, scheduled for October 1968. Even if Apollo 7 were a complete success (which was considered a prerequisite for proceeding), Apollo 8 would be only the second manned flight of the Apollo spacecraft, and the crew would have to rely upon the functioning of its power generation, propulsion, and life support systems for a mission lasting six days. Unlike an Earth orbit mission, if something goes wrong en route to or returning from the Moon, you can’t just come home immediately. The Service Propulsion System on the Service Module would have to work perfectly when leaving lunar orbit or the crew would be marooned forever or crash on the Moon. It would only have been tested previously in one manned mission and there was no backup (although the single engine did incorporate substantial redundancy in its design).

The spacecraft guidance, navigation, and control system and its Apollo Guidance Computer hardware and software, upon which the crew would have to rely to navigate to and from the Moon, including the critical engine burns to enter and leave lunar orbit while behind the Moon and out of touch with Mission Control, had never been tested beyond Earth orbit.

The mission would go to the Moon without a Lunar Module. If a problem developed en route to the Moon which disabled the Service Module (as would happen to Apollo 13 in April 1970), there would be no LM to serve as a lifeboat and the crew would be doomed.

When the high-ranking conspirators presented their audacious plan to their bosses, the reaction was immediate. Manned spaceflight chief Mueller immediately said, “Can’t do that! That’s craziness!” His boss, administrator James Webb, said “You try to change the entire direction of the program while I’m out of the country?” Mutiny is a strong word, but this seemed to verge upon it. Still, Webb and Mueller agreed to meet with the lunar cabal in Houston on August 22. After a contentious meeting, Webb agreed to proceed with the plan and to present it to President Johnson, who was almost certain to approve it, having great confidence in Webb’s management of NASA. The mission was on.

It was only then that Borman and his crewmembers Lovell and Anders learned of their reassignment. While Anders was disappointed at the prospect of being the Lunar Module Pilot on a mission with no Lunar Module, the prospect of being on the first flight to the Moon and entrusted with observation and photography of lunar landing sites more than made up for it. They plunged into an accelerated training program to get ready for the mission.

NASA approached the mission with its usual “can-do” approach and public confidence, but everybody involved was acutely aware of the risks that were being taken. Susan Borman, Frank’s wife, privately asked Chris Kraft, director of Flight Operations and part of the group who advocated sending Apollo 8 to the Moon, with a reputation as a plain-talking straight shooter, “I really want to know what you think their chances are of coming home.” Kraft responded, “You really mean that, don’t you?” “Yes,” she replied, “and you know I do.” Kraft answered, “Okay. How’s fifty-fifty?” Those within the circle, including the crew, knew what they were biting off.

The launch was scheduled for December 21, 1968. Everybody would be working through Christmas, including the twelve ships and thousands of sailors in the recovery fleet, but lunar launch windows are set by the constraints of celestial mechanics, not human holidays. In November, the Soviets had flown Zond 6, and it had demonstrated the “double dip” re-entry trajectory required for human lunar missions. There were two system failures which killed the animal test subjects on board, but these were covered up and the mission heralded as a great success. From what NASA knew, it was entirely possible the next launch would be with cosmonauts bound for the Moon.

Space launches were exceptional public events in the 1960s, and the first flight of men to the Moon, just about a hundred years after Jules Verne envisioned three men setting out for the Moon from central Florida in a “cylindro-conical projectile” in De la terre à la lune (From the Earth to the Moon), similarly engaging the world, the launch of Apollo 8 attracted around a quarter of a million people to watch the spectacle in person and hundreds of millions watching on television both in North America and around the globe, thanks to the newfangled technology of communication satellites.  Let’s tune in to CBS television and relive this singular event with Walter Cronkite.  (For one of those incomprehensible reasons in the Internet of Trash, this video, for which YouTube will happily generate an embed code, fails to embed in WordPress.  You’ll have to click the link below to view it.)

CBS coverage of the Apollo 8 launch

Now we step inside Mission Control and listen in on the Flight Director’s audio loop during the launch, illustrated with imagery and simulations.

The Saturn V performed almost flawlessly. During the second stage burn mild pogo oscillations began but, rather than progressing to the point where they almost tore the rocket apart as had happened on the previous Saturn V launch, von Braun’s team’s fixes kicked in and seconds later Borman reported, “Pogo’s damping out.” A few minutes later Apollo 8 was in Earth orbit.

Jim Lovell had sixteen days of spaceflight experience across two Gemini missions, one of them Gemini 7 where he endured almost two weeks in orbit with Frank Borman. Bill Anders was a rookie, on his first space flight. Now weightless, all three were experiencing a spacecraft nothing like the cramped Mercury and Gemini capsules which you put on as much as boarded. The Apollo command module had an interior volume of six cubic metres (218 cubic feet, in the quaint way NASA reckons things) which may not seem like much for a crew of three, but in weightlessness, with every bit of space accessible and usable, felt quite roomy. There were five real windows, not the tiny portholes of Gemini, and plenty of space to move from one to another.

With all this roominess and mobility came potential hazards, some verging on slapstick, but, in space, serious nonetheless. NASA safety personnel had required the astronauts to wear life vests over their space suits during the launch just in case the Saturn V malfunctioned and they ended up in the ocean. While moving around the cabin to get to the navigation station after reaching orbit, Lovell, who like the others hadn’t yet removed his life vest, snagged its activation tab on a strut within the cabin and it instantly inflated. Lovell looked ridiculous and the situation comical, but it was no laughing matter. The life vests were inflated with carbon dioxide which, if released in the cabin, would pollute their breathing air and removal would use up part of a CO₂ scrubber cartridge, of which they had a limited supply on board. Lovell finally figured out what to do. After being helped out of the vest, he took it down to the urine dump station in the lower equipment bay and vented it into a reservoir which could be dumped out into space. One problem solved, but in space you never know what the next surprise might be.

The astronauts wouldn’t have much time to admire the Earth through those big windows. Over Australia, just short of three hours after launch, they would re-light the engine on the third stage of the Saturn V for the “trans-lunar injection” (TLI) burn of 318 seconds, which would accelerate the spacecraft to just slightly less than escape velocity, raising its apogee so it would be captured by the Moon’s gravity. After housekeeping (presumably including the rest of the crew taking off those pesky life jackets, since there weren’t any wet oceans where they were going) and reconfiguring the spacecraft and its computer for the maneuver, they got the call from Houston, “You are go for TLI.” They were bound for the Moon.

The third stage, which had failed to re-light on its last outing, worked as advertised this time, with a flawless burn. Its job was done; from here on the astronauts and spacecraft were on their own. The booster had placed them on a free-return trajectory. If they did nothing (apart from minor “trajectory correction maneuvers” easily accomplished by the spacecraft’s thrusters) they would fly out to the Moon, swing around its far side, and use its gravity to slingshot back to the Earth (as Lovell would do two years later when he commanded Apollo 13, although there the crew had to use the engine of the LM to get back onto a free-return trajectory after the accident).

Apollo 8 rapidly climbed out of the Earth’s gravity well, trading speed for altitude, and before long the astronauts beheld a spectacle no human eyes had glimpsed before: an entire hemisphere of Earth at once, floating in the inky black void. On board, there were other concerns: Frank Borman was puking his guts out and having difficulties with the other end of the tubing as well. Borman had logged more than six thousand flight hours in his career as a fighter and test pilot, most of it in high-performance jet aircraft, and fourteen days in space on Gemini 7 without any motion sickness. Many people feel queasy when they experience weightlessness the first time, but this was something entirely different and new in the American space program. And it was very worrisome. The astronauts discussed the problem on private tapes they could downlink to Mission Control without broadcasting to the public, and when NASA got around to playing the tapes, the chief flight surgeon, Dr. Charles Berry, became alarmed.

As he saw it, there were three possibilities: motion sickness, a virus of some kind, or radiation sickness. On its way to the Moon, Apollo 8 passed directly through the Van Allen radiation belts, spending two hours in this high radiation environment, the first humans to do so. The total radiation dose was estimated as roughly the same as one would receive from a chest X-ray, but the composition of the radiation was different and the exposure was over an extended time, so nobody could be sure it was safe. The fact that Lovell and Anders had experienced no symptoms argued against the radiation explanation. Berry concluded that a virus was the most probable cause and, based upon the mission rules said, “I’m recommending that we consider canceling the mission.” The risk of proceeding with the commander unable to keep food down and possibly carrying a virus which the other astronauts might contract was too great in his opinion. This recommendation was passed up to the crew. Borman, usually calm and collected even by astronaut standards, exclaimed, “What? That is pure, unadulterated horseshit.” The mission would proceed, and within a day his stomach had settled.

This was the first case of space adaptation syndrome to afflict an American astronaut. (Apparently some Soviet cosmonauts had been affected, but this was covered up to preserve their image as invincible exemplars of the New Soviet Man.) It is now known to affect around a third of people experiencing weightlessness in environments large enough to move around, and spontaneously clears up in two to four (miserable) days.

The two most dramatic and critical events in Apollo 8’s voyage would occur on the far side of the Moon, with 3500 km of rock between the spacecraft and the Earth totally cutting off all communications. The crew would be on their own, aided by the computer and guidance system and calculations performed on the Earth and sent up before passing behind the Moon. The first would be lunar orbit insertion (LOI), scheduled for 69 hours and 8 minutes after launch. The big Service Propulsion System (SPS) engine (it was so big—twice as large as required for Apollo missions as flown—because it was designed to be able to launch the entire Apollo spacecraft from the Moon if a “direct ascent” mission mode had been selected) would burn for exactly four minutes and seven seconds to bend the spacecraft’s trajectory around the Moon into a closed orbit around that world.

If the SPS failed to fire for the LOI burn, it would be a huge disappointment but survivable. Apollo 8 would simply continue on its free-return trajectory, swing around the Moon, and fall back to Earth where it would perform a normal re-entry and splashdown. But if the engine fired and cut off too soon, the spacecraft would be placed into an orbit which would not return them to Earth, marooning the crew in space to die when their supplies ran out. If it burned just a little too long, the spacecraft’s trajectory would intersect the surface of the Moon—lithobraking is no way to land on the Moon.

When the SPS engine shut down precisely on time and the computer confirmed the velocity change of the burn and orbital parameters, the three astronauts were elated, but they were the only people in the solar system aware of the success. Apollo 8 was still behind the Moon, cut off from communications. The first clue Mission Control would have of the success or failure of the burn would be when Apollo 8’s telemetry signal was reacquired as it swung around the limb of the Moon. If too early, it meant the burn had failed and the spacecraft was coming back to Earth; that moment passed with no signal. Now tension mounted as the clock ticked off the seconds to the time expected for a successful burn. If that time came and went with no word from Apollo 8, it would be a really bad day. Just on time, the telemetry signal locked up and Jim Lovell reported, “Go ahead, Houston, this is Apollo 8. Burn complete. Our orbit 160.9 by 60.5.” (Lovell was using NASA’s preferred measure of nautical miles; in proper units it was 311 by 112 km. The orbit would subsequently be circularised by another SPS burn to 112.7 by 114.7 km.) The Mission Control room erupted into an un-NASA-like pandemonium of cheering.

Apollo 8 would orbit the Moon ten times, spending twenty hours in a retrograde orbit with an inclination of 12 degrees to the lunar equator, which would allow it to perform high-resolution photography of candidate sites for early landing missions under lighting conditions similar to those expected at the time of landing. In addition, precision tracking of the spacecraft’s trajectory in lunar orbit would allow mapping of the Moon’s gravitational field, including the “mascons” which perturb the orbits of objects in low lunar orbits and would be important for longer duration Apollo orbital missions in the future.

During the mission, the crew were treated to amazing sights and, in particular, the dramatic difference between the near side, with its many flat “seas”, and the rugged highlands of the far side. Coming around the Moon they saw the spectacle of earthrise for the first time and, hastily grabbing a magazine of colour film and setting aside the planned photography schedule, Bill Anders snapped the photo of the Earth rising above the lunar horizon which became one of the most iconic photographs of the twentieth century. Here is a reconstruction of the moment that photo was taken.

On the ninth and next-to-last orbit, the crew conducted a second television transmission which was broadcast worldwide. It was Christmas Eve on much of the Earth, and, coming at the end of the chaotic, turbulent, and often tragic year of 1968, it was a magical event, remembered fondly by almost everybody who witnessed it and felt pride for what the human species had just accomplished.

You have probably heard this broadcast from the Moon, often with the audio overlaid on imagery of the Moon from later missions, with much higher resolution than was actually seen in that broadcast. Here, in three parts, is what people, including this scrivener, actually saw on their televisions that enchanted night. The famous reading from Genesis is in the third part. This description is eerily similar to that in Jules Verne’s 1870 Autour de la lune.

After the end of the broadcast, it was time to prepare for the next and absolutely crucial maneuver, also performed on the far side of the Moon: trans-Earth injection, or TEI. This would boost the spacecraft out of lunar orbit and send it back on a trajectory to Earth. This time the SPS engine had to work, and perfectly. If it failed to fire, the crew would be trapped in orbit around the Moon with no hope of rescue. If it cut off too soon or burned too long, or the spacecraft was pointed in the wrong direction when it fired, Apollo 8 would miss the Earth and orbit forever far from its home planet or come in too steep and burn up when it hit the atmosphere. Once again the tension rose to a high pitch in Mission Control as the clock counted down to the two fateful times: this time they’d hear from the spacecraft earlier if it was on its way home and later or not at all if things had gone tragically awry. Exactly when expected, the telemetry screens came to life and a second later Jim Lovell called, “Houston, Apollo 8. Please be informed there is a Santa Claus.”

Now it was just a matter of falling the 375,000 kilometres from the Moon, hitting the precise re-entry corridor in the Earth’s atmosphere, executing the intricate “double dip” re-entry trajectory, and splashing down near the aircraft carrier which would retrieve the Command Module and crew. Earlier unmanned tests gave confidence it would all work, but this was the first time men would be trying it.

There was some unexpected and embarrassing excitement on the way home. Mission Control had called up a new set of co-ordinates for the “barbecue roll” which the spacecraft executed to even out temperature. Lovell was asked to enter “verb 3723, noun 501” into the computer. But, weary and short on sleep, he fat-fingered the commands and entered “verb 37, noun 01”. This told the computer the spacecraft was back on the launch pad, pointing straight up, and it immediately slewed to what it thought was that orientation. Lovell quickly figured out what he’d done, “It was my goof”, but by this time he’d “lost the platform”: the stable reference the guidance system used to determine in which direction the spacecraft was pointing in space. He had to perform a manual alignment, taking sightings on a number of stars, to recover the correct orientation of the stable platform. This was completely unplanned but, as it happens, in doing so Lovell acquired experience that would prove valuable when he had to perform the same operation in much more dire circumstances on Apollo 13 after an explosion disabled the computer and guidance system in the Command Module. Here is the author of the book, Jeffrey Kluger, discussing Jim Lovell’s goof.

The re-entry went completely as planned, flown entirely under computer control, with the spacecraft splashing into the Pacific Ocean just 6 km from the aircraft carrier Yorktown. But because the splashdown occurred before dawn, it was decided to wait until the sky brightened to recover the crew and spacecraft. Forty-three minutes after splashdown, divers from the Yorktown arrived at the scene, and forty-five minutes after that the crew was back on the ship. Apollo 8 was over, a total success. This milestone in the space race had been won definitively by the U.S., and shortly thereafter the Soviets abandoned their Zond circumlunar project, judging it an anticlimax and admission of defeat to fly by the Moon after the Americans had already successfully orbited it.

This is the official NASA contemporary documentary about Apollo 8.

Here is an evening with the Apollo 8 astronauts recorded at the National Air and Space Museum on 2008-11-13 to commemorate the fortieth anniversary of the flight.

This is a reunion of the Apollo 8 astronauts on 2009-04-23.

As of this writing, all of the crew of Apollo 8 are alive, and, in a business where divorce was common, remain married to the women they wed as young military officers.

Kluger, Jeffrey. Apollo 8. New York: Picador, 2017. ISBN 978-1-250-18251-7.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: The Forgotten Genius of Oliver Heaviside

“The Forgotten Genius of Oliver Heaviside” by Basil MahonAt age eleven, in 1861, young Oliver Heaviside’s family, supported by his father’s irregular income as an engraver of woodblock illustrations for publications (an art beginning to be threatened by the advent of photography) and a day school for girls operated by his mother in the family’s house, received a small legacy which allowed them to move to a better part of London and enroll Oliver in the prestigious Camden House School, where he ranked among the top of his class, taking thirteen subjects including Latin, English, mathematics, French, physics, and chemistry. His independent nature and iconoclastic views had already begun to manifest themselves: despite being an excellent student he dismissed the teaching of Euclid’s geometry in mathematics and English rules of grammar as worthless. He believed that both mathematics and language were best learned, as he wrote decades later, “observationally, descriptively, and experimentally.” These principles would guide his career throughout his life.

At age fifteen he took the College of Perceptors examination, the equivalent of today’s A Levels. He was the youngest of the 538 candidates to take the examination and scored fifth overall and first in the natural sciences. This would easily have qualified him for admission to university, but family finances ruled that out. He decided to study on his own at home for two years and then seek a job, perhaps in the burgeoning telegraph industry. He would receive no further formal education after the age of fifteen.

His mother’s elder sister had married Charles Wheatstone, a successful and wealthy scientist, inventor, and entrepreneur whose inventions include the concertina, the stereoscope, and the Playfair encryption cipher, and who made major contributions to the development of telegraphy. Wheatstone took an interest in his bright nephew, and guided his self-studies after leaving school, encouraging him to master the Morse code and the German and Danish languages. Oliver’s favourite destination was the library, which he later described as “a journey into strange lands to go a book-tasting”. He read the original works of Newton, Laplace, and other “stupendous names” and discovered that with sufficient diligence he could figure them out on his own.

At age eighteen, he took a job as an assistant to his older brother Arthur, well-established as a telegraph engineer in Newcastle. Shortly thereafter, probably on the recommendation of Wheatstone, he was hired by the just-formed Danish-Norwegian-English Telegraph Company as a telegraph operator at a salary of £150 per year (around £12000 in today’s money). The company was about to inaugurate a cable under the North Sea between England and Denmark, and Oliver set off to Jutland to take up his new post. Long distance telegraphy via undersea cables was the technological frontier at the time—the first successful transatlantic cable had only gone into service two years earlier, and connecting the continents into a world-wide web of rapid information transfer was the booming high-technology industry of the age. While the job of telegraph operator might seem a routine clerical task, the élite who operated the undersea cables worked in an environment akin to an electrical research laboratory, trying to wring the best performance (words per minute) from the finicky and unreliable technology.

Heaviside prospered in the new job, and after a merger was promoted to chief operator at a salary of £175 per year and transferred back to England, at Newcastle. At the time, undersea cables were unreliable. It was not uncommon for the signal on a cable to fade and then die completely, most often due to a short circuit caused by failure of the gutta-percha insulation between the copper conductor and the iron sheath surrounding it. When a cable failed, there was no alternative but to send out a ship which would find the cable with a grappling hook, haul it up to the surface, cut it, and test whether the short was to the east or west of the ship’s position (the cable would work in the good direction but fail in that containing the short. Then the cable would be re-spliced, dropped back to the bottom, and the ship would set off in the direction of the short to repeat the exercise over and over until, by a process similar to binary search, the location of the fault was narrowed down and that section of the cable replaced. This was time consuming and potentially hazardous given the North Sea’s propensity for storms, and while the cable remained out of service it made no money for the telegraph company.

Heaviside, who continued his self-study and frequented the library when not at work, realised that knowing the resistance and length of the functioning cable, which could be easily measured, it would be possible to estimate the location of the short simply by measuring the resistance of the cable from each end after the short appeared. He was able to cancel out the resistance of the fault, creating a quadratic equation which could be solved for its location. The first time he applied this technique his bosses were sceptical, but when the ship was sent out to the location he predicted, 114 miles from the English coast, they quickly found the short circuit.

At the time, most workers in electricity had little use for mathematics: their trade journal, The Electrician (which would later publish much of Heaviside’s work) wrote in 1861, “In electricity there is seldom any need of mathematical or other abstractions; and although the use of formulæ may in some instances be a convenience, they may for all practical purpose be dispensed with.” Heaviside demurred: while sharing disdain for abstraction for its own sake, he valued mathematics as a powerful tool to understand the behaviour of electricity and attack problems of great practical importance, such as the ability to send multiple messages at once on the same telegraphic line and increase the transmission speed on long undersea cable links (while a skilled telegraph operator could send traffic at thirty words per minute on intercity land lines, the transatlantic cable could run no faster than eight words per minute). He plunged into calculus and differential equations, adding them to his intellectual armamentarium.

He began his own investigations and experiments and began to publish his results, first in English Mechanic, and then, in 1873, the prestigious Philosophical Magazine, where his work drew the attention of two of the most eminent workers in electricity: William Thomson (later Lord Kelvin) and James Clerk Maxwell. Maxwell would go on to cite Heaviside’s paper on the Wheatstone Bridge in the second edition of his Treatise on Electricity and Magnetism, the foundation of the classical theory of electromagnetism, considered by many the greatest work of science since Newton’s Principia, and still in print today. Heady stuff, indeed, for a twenty-two year old telegraph operator who had never set foot inside an institution of higher education.

Heaviside regarded Maxwell’s Treatise as the path to understanding the mysteries of electricity he encountered in his practical work and vowed to master it. It would take him nine years and change his life. He would become one of the first and foremost of the “Maxwellians”, a small group including Heaviside, George FitzGerald, Heinrich Hertz, and Oliver Lodge, who fully grasped Maxwell’s abstract and highly mathematical theory (which, like many subsequent milestones in theoretical physics, predicted the results of experiments without providing a mechanism to explain them, such as earlier concepts like an “electric fluid” or William Thomson’s intricate mechanical models of the “luminiferous ether”) and built upon its foundations to discover and explain phenomena unknown to Maxwell (who would die in 1879 at the age of just 48).

While pursuing his theoretical explorations and publishing papers, Heaviside tackled some of the main practical problems in telegraphy. Foremost among these was “duplex telegraphy”: sending messages in each direction simultaneously on a single telegraph wire. He invented a new technique and was even able to send two messages at the same time in both directions as fast as the operators could send them. This had the potential to boost the revenue from a single installed line by a factor of four. Oliver published his invention, and in doing so made an enemy of William Preece, a senior engineer at the Post Office telegraph department, who had invented and previously published his own duplex system (which would not work), that was not acknowledged in Heaviside’s paper. This would start a feud between Heaviside and Preece which would last the rest of their lives and, on several occasions, thwart Heaviside’s ambition to have his work accepted by mainstream researchers. When he applied to join the Society of Telegraph Engineers, he was rejected on the grounds that membership was not open to “clerks”. He saw the hand of Preece and his cronies at the Post Office behind this and eventually turned to William Thomson to back his membership, which was finally granted.

By 1874, telegraphy had become a big business and the work was increasingly routine. In 1870, the Post Office had taken over all domestic telegraph service in Britain and, as government is wont to do, largely stifled innovation and experimentation. Even at privately-owned international carriers like Oliver’s employer, operators were no longer concerned with the technical aspects of the work but rather tending automated sending and receiving equipment. There was little interest in the kind of work Oliver wanted to do: exploring the new horizons opened up by Maxwell’s work. He decided it was time to move on. So, he quit his job, moved back in with his parents in London, and opted for a life as an independent, unaffiliated researcher, supporting himself purely by payments for his publications.

With the duplex problem solved, the largest problem that remained for telegraphy was the slow transmission speed on long lines, especially submarine cables. The advent of the telephone in the 1870s would increase the need to address this problem. While telegraphic transmission on a long line slowed down the speed at which a message could be sent, with the telephone voice became increasingly distorted the longer the line, to the point where, after around 100 miles, it was incomprehensible. Until this was understood and a solution found, telephone service would be restricted to local areas.

Many of the early workers in electricity thought of it as something like a fluid, where current flowed through a wire like water through a pipe. This approximation is more or less correct when current flow is constant, as in a direct current generator powering electric lights, but when current is varying a much more complex set of phenomena become manifest which require Maxwell’s theory to fully describe. Pioneers of telegraphy thought of their wires as sending direct current which was simply switched off and on by the sender’s key, but of course the transmission as a whole was a varying current, jumping back and forth between zero and full current at each make or break of the key contacts. When these transitions are modelled in Maxwell’s theory, one finds that, depending upon the physical properties of the transmission line (its resistance, inductance, capacitance, and leakage between the conductors) different frequencies propagate along the line at different speeds. The sharp on/off transitions in telegraphy can be thought of, by Fourier transform, as the sum of a wide band of frequencies, with the result that, when each propagates at a different speed, a short, sharp pulse sent by the key will, at the other end of the long line, be “smeared out” into an extended bump with a slow rise to a peak and then decay back to zero. Above a certain speed, adjacent dots and dashes will run into one another and the message will be undecipherable at the receiving end. This is why operators on the transatlantic cables had to send at the painfully slow speed of eight words per minute.

In telephony, it’s much worse because human speech is composed of a broad band of frequencies, and the frequencies involved (typically up to around 3400 cycles per second) are much higher than the off/on speeds in telegraphy. The smearing out or dispersion as frequencies are transmitted at different speeds results in distortion which renders the voice signal incomprehensible beyond a certain distance.

In the mid-1850s, during development of the first transatlantic cable, William Thomson had developed a theory called the “KR law” which predicted the transmission speed along a cable based upon its resistance and capacitance. Thomson was aware that other effects existed, but without Maxwell’s theory (which would not be published in its final form until 1873), he lacked the mathematical tools to analyse them. The KR theory, which produced results that predicted the behaviour of the transatlantic cable reasonably well, held out little hope for improvement: decreasing the resistance and capacitance of the cable would dramatically increase its cost per unit length.

Heaviside undertook to analyse what is now called the transmission line problem using the full Maxwell theory and, in 1878, published the general theory of propagation of alternating current through transmission lines, what are now called the telegrapher’s equations. Because he took resistance, capacitance, inductance, and leakage all into account and thus modelled both the electric and magnetic field created around the wire by the changing current, he showed that by balancing these four properties it was possible to design a transmission line which would transmit all frequencies at the same speed. In other words, this balanced transmission line would behave for alternating current (including the range of frequencies in a voice signal) just like a simple wire did for direct current: the signal would be attenuated (reduced in amplitude) with distance but not distorted.

In an 1887 paper, he further showed that existing telegraph and telephone lines could be made nearly distortionless by adding loading coils to increase the inductance at points along the line (as long as the distance between adjacent coils is small compared to the wavelength of the highest frequency carried by the line). This got him into another battle with William Preece, whose incorrect theory attributed distortion to inductance and advocated minimising self-inductance in long lines. Preece moved to block publication of Heaviside’s work, with the result that the paper on distortionless telephony, published in The Electrician, was largely ignored. It was not until 1897 that AT&T in the United States commissioned a study of Heaviside’s work, leading to patents eventually worth millions. The credit, and financial reward, went to Professor Michael Pupin of Columbia University, who became another of Heaviside’s life-long enemies.

You might wonder why what seems such a simple result (which can be written in modern notation as the equation L/R = C/G) which had such immediate technological utlilty eluded so many people for so long (recall that the problem with slow transmission on the transatlantic cable had been observed since the 1850s). The reason is the complexity of Maxwell’s theory and the formidably difficult notation in which it was expressed. Oliver Heaviside spent nine years fully internalising the theory and its implications, and he was one of only a handful of people who had done so and, perhaps, the only one grounded in practical applications such as telegraphy and telephony. Concurrent with his work on transmission line theory, he invented the mathematical field of vector calculus and, in 1884, reformulated Maxwell’s original theory which, written in modern notation less cumbersome than that employed by Maxwell, looks like:

Maxwell's original 20 equations

into the four famous vector equations we today think of as Maxwell’s.

Maxwell's equations: modern vector calculus representation

These are not only simpler, condensing twenty equations to just four, but provide (once you learn the notation and meanings of the variables) an intuitive sense for what is going on. This made, for the first time, Maxwell’s theory accessible to working physicists and engineers interested in getting the answer out rather than spending years studying an arcane theory. (Vector calculus was independently invented at the same time by the American J. Willard Gibbs. Heaviside and Gibbs both acknowledged the work of the other and there was no priority dispute. The notation we use today is that of Gibbs, but the mathematical content of the two formulations is essentially identical.)

And, during the same decade of the 1880s, Heaviside invented the operational calculus, a method of calculation which reduces the solution of complicated problems involving differential equations to simple algebra. Heaviside was able to solve so many problems which others couldn’t because he was using powerful computational tools they had not yet adopted. The situation was similar to that of Isaac Newton who was effortlessly solving problems such as the brachistochrone using the calculus he’d invented while his contemporaries struggled with more cumbersome methods. Some of the things Heaviside did in the operational calculus, such as cancel derivative signs in equations and take the square root of a derivative sign made rigorous mathematicians shudder but, hey, it worked and that was good enough for Heaviside and the many engineers and applied mathematicians who adopted his methods. (In the 1920s, pure mathematicians used the theory of Laplace transforms to reformulate the operational calculus in a rigorous manner, but this was decades after Heaviside’s work and long after engineers were routinely using it in their calculations.)

Heaviside’s intuitive grasp of electromagnetism and powerful computational techniques placed him in the forefront of exploration of the field. He calculated the electric field of a moving charged particle and found it contracted in the direction of motion, foreshadowing the Lorentz-FitzGerald contraction which would figure in Einstein’s special relativity. In 1889 he computed the force on a point charge moving in an electromagnetic field, which is now called the Lorentz force after Hendrik Lorentz who independently discovered it six years later. He predicted that a charge moving faster than the speed of light in a medium (for example, glass or water) would emit a shock wave of electromagnetic radiation; in 1934 Pavel Cherenkov experimentally discovered the phenomenon, now called Cherenkov radiation, for which he won the Nobel Prize in 1958. In 1902, Heaviside applied his theory of transmission lines to the Earth as a whole and explained the propagation of radio waves over intercontinental distances as due to a transmission line formed by conductive seawater and a hypothetical conductive layer in the upper atmosphere dubbed the Heaviside layer. In 1924 Edward V. Appleton confirmed the existence of such a layer, the ionosphere, and won the Nobel prize in 1947 for the discovery.

Oliver Heaviside never won a Nobel Price, although he was nominated for the physics prize in 1912. He shouldn’t have felt too bad, though, as other nominees passed over for the prize that year included Hendrik Lorentz, Ernst Mach, Max Planck, and Albert Einstein. (The winner that year was Gustaf Dalén, “for his invention of automatic regulators for use in conjunction with gas accumulators for illuminating lighthouses and buoys”—oh well.) He did receive Britain’s highest recognition for scientific achievement, being named a Fellow of the Royal Society in 1891. In 1921 he was the first recipient of the Faraday Medal from the Institution of Electrical Engineers.

Having never held a job between 1874 and his death in 1925, Heaviside lived on his irregular income from writing, the generosity of his family, and, from 1896 onward a pension of £120 per year (less than his starting salary as a telegraph operator in 1868) from the Royal Society. He was a proud man and refused several other offers of money which he perceived as charity. He turned down an offer of compensation for his invention of loading coils from AT&T when they refused to acknowledge his sole responsibility for the invention. He never married, and in his elder years became somewhat of a recluse and, although he welcomed visits from other scientists, hardly ever left his home in Torquay in Devon.

His impact on the physics of electromagnetism and the craft of electrical engineering can be seen in the list of terms he coined which are in everyday use: “admittance”, “conductance”, “electret”, “impedance”, “inductance”, “permeability”, “permittance”, “reluctance”, and “susceptance”. His work has never been out of print, and sparkles with his intuition, mathematical prowess, and wicked wit directed at those he considered pompous or lost in needless abstraction and rigor. He never sought the limelight and among those upon whose work much of our present-day technology is founded, he is among the least known. But as long as electronic technology persists, it is a monument to the life and work of Oliver Heaviside.

Mahon, Basil. The Forgotten Genius of Oliver Heaviside. Amherst, NY: Prometheus Books, 2017. ISBN 978-1-63388-331-4.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Life After Google

“Life after Google” by George GilderIn his 1990 book Life after Television, George Gilder predicted that the personal computer, then mostly boxes that sat on desktops and worked in isolation from one another, would become more personal, mobile, and be used more to communicate than to compute. In the 1994 revised edition of the book, he wrote. “The most common personal computer of the next decade will be a digital cellular phone with an IP address … connecting to thousands of databases of all kinds.” In contemporary speeches he expanded on the idea, saying, “it will be as portable as your watch and as personal as your wallet; it will recognize speech and navigate streets; it will collect your mail, your news, and your paycheck.” In 2000, he published Telecosm, where he forecast that the building out of a fibre optic communication infrastructure and the development of successive generations of spread spectrum digital mobile communication technologies would effectively cause the cost of communication bandwidth (the quantity of data which can be transmitted in a given time) to asymptotically approach zero, just as the ability to pack more and more transistors on microprocessor and memory chips was doing for computing.

Clearly, when George Gilder forecasts the future of computing, communication, and the industries and social phenomena that spring from them, it’s wise to pay attention. He’s not infallible: in 1990 he predicted that “in the world of networked computers, no one would have to see an advertisement he didn’t want to see”. Oh, well. The very difference between that happy vision and the advertisement-cluttered world we inhabit today, rife with bots, malware, scams, and serial large-scale security breaches which compromise the personal data of millions of people and expose them to identity theft and other forms of fraud is the subject of this book: how we got here, and how technology is opening a path to move on to a better place.

The Internet was born with decentralisation as a central concept. Its U.S. government-funded precursor, ARPANET, was intended to research and demonstrate the technology of packet switching, in which dedicated communication lines from point to point (as in the telephone network) were replaced by switching packets, which can represent all kinds of data—text, voice, video, mail, cat pictures—from source to destination over shared high-speed data links. If the network had multiple paths from source to destination, failure of one data link would simply cause the network to reroute traffic onto a working path, and communication protocols would cause any packets lost in the failure to be automatically re-sent, preventing loss of data. The network might degrade and deliver data more slowly if links or switching hubs went down, but everything would still get through.

This was very attractive to military planners in the Cold War, who worried about a nuclear attack decapitating their command and control network by striking one or a few locations through which their communications funnelled. A distributed network, of which ARPANET was the prototype, would be immune to this kind of top-down attack because there was no top: it was made up of peers, spread all over the landscape, all able to switch data among themselves through a mesh of interconnecting links.

As the ARPANET grew into the Internet and expanded from a small community of military, government, university, and large company users into a mass audience in the 1990s, this fundamental architecture was preserved, but in practice the network bifurcated into a two tier structure. The top tier consisted of the original ARPANET-like users, plus “Internet Service Providers” (ISPs), who had top-tier (“backbone”) connectivity, and then resold Internet access to their customers, who mostly initially connected via dial-up modems. Over time, these customers obtained higher bandwidth via cable television connections, satellite dishes, digital subscriber lines (DSL) over the wired telephone network, and, more recently, mobile devices such as cellular telephones and tablets.

The architecture of the Internet remained the same, but this evolution resulted in a weakening of its peer-to-peer structure. The approaching exhaustion of 32 bit Internet addresses (IPv4) and the slow deployment of its successor (IPv6) meant most small-scale Internet users did not have a permanent address where others could contact them. In an attempt to shield users from the flawed security model and implementation of the software they ran, their Internet connections were increasingly placed behind firewalls and subjected to Network Address Translation (NAT), which made it impossible to establish peer to peer connections without a third party intermediary (which, of course, subverts the design goal of decentralisation). While on the ARPANET and the original Internet every site was a peer of every other (subject only to the speed of their network connections and computer power available to handle network traffic), the network population now became increasingly divided into producers or publishers (who made information available), and consumers (who used the network to access the publishers’ sites but did not publish themselves).

While in the mid-1990s it was easy (or as easy as anything was in that era) to set up your own Web server and publish anything you wished, now most small-scale users were forced to employ hosting services operated by the publishers to make their content available. Services such as AOL, Myspace, Blogger, Facebook, and YouTube were widely used by individuals and companies to host their content, while those wishing their own apparently independent Web presence moved to hosting providers who supplied, for a fee, the servers, storage, and Internet access used by the site.

All of this led to a centralisation of data on the Web, which was accelerated by the emergence of the high speed fibre optic links and massive computing power upon which Gilder had based his 1990 and 2000 forecasts. Both of these came with great economies of scale: it cost a company like Google or Amazon much less per unit of computing power or network bandwidth to build a large, industrial-scale data centre located where electrical power and cooling were inexpensive and linked to the Internet backbone by multiple fibre optic channels, than it cost an individual Internet user or small company with their own server on premises and a modest speed link to an ISP. Thus it became practical for these Goliaths of the Internet to suck up everybody’s data and resell their computing power and access at attractive prices.

As a example of the magnitude of the economies of scale we’re talking about, when I migrated the hosting of my site from my own on-site servers and Internet connection to an Amazon Web Services data centre, my monthly bill for hosting the site dropped by a factor of fifty—not fifty percent, one fiftieth the cost, and you can bet Amazon’s making money on the deal.

This tremendous centralisation is the antithesis of the concept of ARPANET. Instead of a worldwide grid of redundant data links and data distributed everywhere, we have a modest number of huge data centres linked by fibre optic cables carrying traffic for millions of individuals and enterprises. A couple of submarines full of Trident D5s would probably suffice to reset the world, computer network-wise, to 1970.

As this concentration was occurring, the same companies who were building the data centres were offering more and more services to users of the Internet: search engines; hosting of blogs, images, audio, and video; E-mail services; social networks of all kinds; storage and collaborative working tools; high-resolution maps and imagery of the world; archives of data and research material; and a host of others. How was all of this to be paid for? Those giant data centres, after all, represent a capital investment of tens of billions of dollars, and their electricity bills are comparable to those of an aluminium smelter. Due to the architecture of the Internet or, more precisely, missing pieces of the puzzle, a fateful choice was made in the early days of the build-out of these services which now pervade our lives, and we’re all paying the price for it. So far, it has allowed the few companies in this data oligopoly to join the ranks of the largest, most profitable, and most highly valued enterprises in human history, but they may be built on a flawed business model and foundation vulnerable to disruption by software and hardware technologies presently emerging.

The basic business model of what we might call the “consumer Internet” (as opposed to businesses who pay to host their Web presence, on-line stores, etc.) has, with few exceptions, evolved to be what the author calls the “Google model” (although it predates Google): give the product away and make money by afflicting its users with advertisements (which are increasingly targeted to them through information collected from the user’s behaviour on the network through intrusive tracking mechanisms). The fundamental flaws of this are apparent to anybody who uses the Internet: the constant clutter of advertisements, with pop-ups, pop-overs, auto-play video and audio, flashing banners, incessant requests to allow tracking “cookies” or irritating notifications, and the consequent arms race between ad blockers and means to circumvent them, with browser developers (at least those not employed by those paid by the advertisers, directly or indirectly) caught in the middle. There are even absurd Web sites which charge a subscription fee for “membership” and then bombard these paying customers with advertisements that insult their intelligence. But there is a fundamental problem with “free”—it destroys the most important channel of communication between the vendor of a product or service and the customer: the price the customer is willing to pay. Deprived of this information, the vendor is in the same position as a factory manager in a centrally planned economy who has no idea how many of each item to make because his orders are handed down by a planning bureau equally clueless about what is needed in the absence of a price signal. In the end, you have freight cars of typewriter ribbons lined up on sidings while customers wait in line for hours in the hope of buying a new pair of shoes. Further, when the user is not the customer (the one who pays), and especially when a “free” service verges on monopoly status like Google search, Gmail, Facebook, and Twitter, there is little incentive for providers to improve the user experience or be responsive to user requests and needs. Users are subjected to the endless torment of buggy “beta” releases, capricious change for the sake of change, and compromises in the user experience on behalf of the real customers—the advertisers. Once again, this mirrors the experience of centrally-planned economies where the market feedback from price is absent: to appreciate this, you need only compare consumer products from the 1970s and 1980s manufactured in the Soviet Union with those from Japan.

The fundamental flaw in Karl Marx’s economics was his belief that the industrial revolution of his time would produce such abundance of goods that the problem would shift from “production amid scarcity” to “redistribution of abundance”. In the author’s view, the neo-Marxists of Silicon Valley see the exponentially growing technologies of computing and communication providing such abundance that they can give away its fruits in return for collecting and monetising information collected about their users (note, not “customers”: customers are those who pay for the information so collected). Once you grasp this, it’s easier to understand the politics of the barons of Silicon Valley.

The centralisation of data and information flow in these vast data silos creates another threat to which a distributed system is immune: censorship or manipulation of information flow, whether by a coercive government or ideologically-motivated management of the companies who provide these “free” services. We may never know who first said “The Internet treats censorship as damage and routes around it” (the quote has been attributed to numerous people, including two personal friends, so I’m not going there), but it’s profound: the original decentralised structure of the ARPANET/Internet is as robust against censorship as it is in the face of nuclear war. If one or more nodes on the network start to censor information or refuse to forward it on communication links it controls, the network routing protocols simply assume that node is down and send data around it through other nodes and paths which do not censor it. On a network with a multitude of nodes and paths among them, owned by a large and diverse population of operators, it is extraordinarily difficult to shut down the flow of information from a given source or viewpoint; there will almost always be an alternative route that gets it there. (Cryptographic protocols and secure and verified identities can similarly avoid the alteration of information in transit or forging information and attributing it to a different originator; I’ll discuss that later.) As with physical damage, top-down censorship does not work because there’s no top.

But with the current centralised Internet, the owners and operators of these data silos have enormous power to put their thumbs on the scale, tilting opinion in their favour and blocking speech they oppose. Google can push down the page rank of information sources of which they disapprove, so few users will find them. YouTube can “demonetise” videos because they dislike their content, cutting off their creators’ revenue stream overnight with no means of appeal, or they can outright ban creators from the platform and remove their existing content. Twitter routinely “shadow-bans” those with whom they disagree, causing their tweets to disappear into the void, and outright banishes those more vocal. Internet payment processors and crowd funding sites enforce explicit ideological litmus tests on their users, and revoke long-standing commercial relationships over legal speech. One might restate the original observation about the Internet as “The centralised Internet treats censorship as an opportunity and says, ‘Isn’t it great!’ ” Today there’s a top, and those on top control the speech of everything that flows through their data silos.

This pernicious centralisation and “free” funding by advertisement (which is fundamentally plundering users’ most precious possessions: their time and attention) were in large part the consequence of the Internet’s lacking three fundamental architectural layers: security, trust, and transactions. Let’s explore them.

Security. Essential to any useful communication system, security simply means that communications between parties on the network cannot be intercepted by third parties, modified en route, or otherwise manipulated (for example, by changing the order in which messages are received). The communication protocols of the Internet, based on the OSI model, had no explicit security layer. It was expected to be implemented outside the model, across the layers of protocol. On today’s Internet, security has been bolted-on, largely through the Transport Layer Security (TLS) protocols (which, due to history, have a number of other commonly used names, and are most often encountered in the “https:” URLs by which users access Web sites). But because it’s bolted on, not designed in from the bottom-up, and because it “just grew” rather than having been designed in, TLS has been the locus of numerous security flaws which put software that employs it at risk. Further, TLS is a tool which must be used by application designers with extreme care in order to deliver security to their users. Even if TLS were completely flawless, it is very easy to misuse it in an application and compromise users’ security.

Trust. As indispensable as security is knowing to whom you’re talking. For example, when you connect to your bank’s Web site, how do you know you’re actually talking to their server and not some criminal whose computer has spoofed your computer’s domain name system server to intercept your communications and who, the moment you enter your password, will be off and running to empty your bank accounts and make your life a living Hell? Once again, trust has been bolted on to the existing Internet through a rickety system of “certificates” issued mostly by large companies for outrageous fees. And, as with anything centralised, it’s vulnerable: in 2016, one of the top-line certificate vendors was compromised, requiring myriad Web sites (including this one) to re-issue their security certificates.

Transactions. Business is all about transactions; if you aren’t doing transactions, you aren’t in business or, as Gilder puts it, “In business, the ability to conduct transactions is not optional. It is the way all economic learning and growth occur. If your product is ‘free,’ it is not a product, and you are not in business, even if you can extort money from so-called advertisers to fund it.” The present-day Internet has no transaction layer, even bolted on. Instead, we have more silos and bags hanging off the side of the Internet called PayPal, credit card processing companies, and the like, which try to put a Band-Aid over the suppurating wound which is the absence of a way to send money over the Internet in a secure, trusted, quick, efficient, and low-overhead manner. The need for this was perceived long before ARPANET. In Project Xanadu, founded by Ted Nelson in 1960, rule 9 of the “original 17 rules” was, “Every document can contain a royalty mechanism at any desired degree of granularity to ensure payment on any portion accessed, including virtual copies (‘transclusions’) of all or part of the document.” While defined in terms of documents and quoting, this implied the existence of a micropayment system which would allow compensating authors and publishers for copies and quotations of their work with a granularity as small as one character, and could easily be extended to cover payments for products and services. A micropayment system must be able to handle very small payments without crushing overhead, extremely quickly, and transparently (without the Japanese tea ceremony that buying something on-line involves today). As originally envisioned by Ted Nelson, as you read documents, their authors and publishers would be automatically paid for their content, including payments to the originators of material from others embedded within them. As long as the total price for the document was less than what I termed the user’s “threshold of paying”, this would be completely transparent (a user would set the threshold in the browser: if zero, they’d have to approve all payments). There would be no need for advertisements to support publication on a public hypertext network (although publishers would, of course, be free to adopt that model if they wished). If implemented in a decentralised way, like the ARPANET, there would be no central strangle point where censorship could be applied by cutting off the ability to receive payments.

So, is it possible to remake the Internet, building in security, trust, and transactions as the foundation, and replace what the author calls the “Google system of the world” with one in which the data silos are seen as obsolete, control of users’ personal data and work returns to their hands, privacy is respected and the panopticon snooping of today is seen as a dark time we’ve put behind us, and the pervasive and growing censorship by plutocrat ideologues and slaver governments becomes impotent and obsolete? George Gilder responds “yes”, and in this book identifies technologies already existing and being deployed which can bring about this transformation.

At the heart of many of these technologies is the concept of a blockchain, an open, distributed ledger which records transactions or any other form of information in a permanent, public, and verifiable manner. Originally conceived as the transaction ledger for the Bitcoin cryptocurrency, it provided the first means of solving the double-spending problem (how do you keep people from spending a unit of electronic currency twice) without the need for a central server or trusted authority, and hence without a potential choke-point or vulnerability to attack or failure. Since the launch of Bitcoin in 2009, blockchain technology has become a major area of research, with banks and other large financial institutions, companies such as IBM, and major university research groups exploring applications with the goals of drastically reducing transaction costs, improving security, and hardening systems against single-point failure risks.

Applied to the Internet, blockchain technology can provide security and trust (through the permanent publication of public keys which identify actors on the network), and a transaction layer able to efficiently and quickly execute micropayments without the overhead, clutter, friction, and security risks of existing payment systems. By necessity, present-day blockchain implementations are add-ons to the existing Internet, but as the technology matures and is verified and tested, it can move into the foundations of a successor system, based on the same lower-level protocols (and hence compatible with the installed base), but eventually supplanting the patched-together architecture of the Domain Name System, certificate authorities, and payment processors, all of which represent vulnerabilities of the present-day Internet and points at which censorship and control can be imposed. Technologies to watch in these areas are:

As the bandwidth available to users on the edge of the network increases through the deployment of fibre to the home and enterprise and via 5G mobile technology, the data transfer economy of scale of the great data silos will begin to erode. Early in the Roaring Twenties, the aggregate computing power and communication bandwidth on the edge of the network will equal and eventually dwarf that of the legacy data smelters of Google, Facebook, Twitter, and the rest. There will no longer be any need for users to entrust their data to these overbearing anachronisms and consent to multi-dozen page “terms of service” or endure advertising just to see their own content or share it with others. You will be in possession of your own data, on your own server or on space for which you freely contract with others, with backup and other services contracted with any other provider on the network. If your server has extra capacity, you can turn it into money by joining the market for computing and storage capacity, just as you take advantage of these resources when required. All of this will be built on the new secure foundation, so you will retain complete control over who can see your data, no longer trusting weasel-worded promises made by amorphous entities with whom you have no real contract to guard your privacy and intellectual property rights. If you wish, you can be paid for your content, with remittances made automatically as people access it. More and more, you’ll make tiny payments for content which is no longer obstructed by advertising and chopped up to accommodate more clutter. And when outrage mobs of pink hairs and soybeards (each with their own pronoun) come howling to ban you from the Internet, they’ll find nobody to shriek at and the kill switch rusting away in a derelict data centre: your data will be in your own hands with access through myriad routes. Technologies moving in this direction include:

This book provides a breezy look at the present state of the Internet, how we got here (versus where we thought we were going in the 1990s), and how we might transcend the present-day mess into something better if not blocked by the heavy hand of government regulation (the risk of freezing the present-day architecture in place by unleashing agencies like the U.S. Federal Communications Commission, which stifled innovation in broadcasting for six decades, to do the same to the Internet is discussed in detail). Although it’s way too early to see which of the many contending technologies will win out (and recall that the technically superior technology doesn’t always prevail), a survey of work in progress provides a sense for what they have in common and what the eventual result might look like.

There are many things to quibble about here. Gilder goes on at some length about how he believes artificial intelligence is all nonsense, that computers can never truly think or be conscious, and that creativity (new information in the Shannon sense) can only come from the human mind, with a lot of confused arguments from Gödel incompleteness, the Turing halting problem, and even the uncertainty principle of quantum mechanics. He really seems to believe in vitalism, that there is an élan vital which somehow infuses the biological substrate which no machine can embody. This strikes me as superstitious nonsense: a human brain is a structure composed of quarks and electrons arranged in a certain way which processes information, interacts with its environment, and is able to observe its own operation as well as external phenomena (which is all consciousness is about). Now, it may be that somehow quantum mechanics is involved in all of this, and that our existing computers, which are entirely deterministic and classical in their operation, cannot replicate this functionality, but if that’s so it simply means we’ll have to wait until quantum computing, which is already working in a rudimentary form in the laboratory, and is just a different way of arranging the quarks and electrons in a system, develops further.

He argues that while Bitcoin can be an efficient and secure means of processing transactions, it is unsuitable as a replacement for volatile fiat money because, unlike gold, the quantity of Bitcoin has an absolute limit, after which the supply will be capped. I don’t get it. It seems to me that this is a feature, not a bug. The supply of gold increases slowly as new gold is mined, and by pure coincidence the rate of increase in its supply has happened to approximate that of global economic growth. But still, the existing inventory of gold dwarfs new supply, so there isn’t much difference between a very slowly increasing supply and a static one. If you’re on a pure gold standard and economic growth is faster than the increase in the supply of gold, there will be gradual deflation because a given quantity of gold will buy more in the future. But so what? In a deflationary environment, interest rates will be low and it will be easy to fund new investment, since investors will receive money back which will be more valuable. With Bitcoin, once the entire supply is mined, supply will be static (actually, very slowly shrinking, as private keys are eventually lost, which is precisely like gold being consumed by industrial uses from which it is not reclaimed), but Bitcoin can be divided without limit (with minor and upward-compatible changes to the existing protocol). So, it really doesn’t matter if, in the greater solar system economy of the year 8537, a single Bitcoin is sufficient to buy Jupiter: transactions will simply be done in yocto-satoshis or whatever. In fact, Bitcoin is better in this regard than gold, which cannot be subdivided below the unit of one atom.

Gilder further argues, as he did in The Scandal of Money, that the proper dimensional unit for money is time, since that is the measure of what is required to create true wealth (as opposed to funny money created by governments or fantasy money “earned” in zero-sum speculation such as currency trading), and that existing cryptocurrencies do not meet this definition. I’ll take his word on the latter point; it’s his definition, after all, but his time theory of money is way too close to the Marxist labour theory of value to persuade me. That theory is trivially falsified by its prediction that more value is created in labour-intensive production of the same goods than by producing them in a more efficient manner. In fact, value, measured as profit, dramatically increases as the labour input to production is reduced. Over forty centuries of human history, the one thing in common among almost everything used for money (at least until our post-reality era) is scarcity: the supply is limited and it is difficult to increase it. The genius of Bitcoin and its underlying blockchain technology is that it solved the problem of how to make a digital good, which can be copied at zero cost, scarce, without requiring a central authority. That seems to meet the essential requirement to serve as money, regardless of how you define that term.

Gilder’s books have a good record for sketching the future of technology and identifying the trends which are contributing to it. He has been less successful picking winners and losers; I wouldn’t make investment decisions based on his evaluation of products and companies, but rather wait until the market sorts out those which will endure.

Gilder, George. Life after Google. Washington: Regnery Publishing, 2018. ISBN 978-1-62157-576-4.

Here is a talk by the author at the Blockstack Berlin 2018 conference which summarises the essentials of his thesis in just eleven minutes and ends with an exhortation to designers and builders of the new Internet to “tear down these walls” around the data centres which imprison our personal information.

This Uncommon Knowledge interview provides, in 48 minutes, a calmer and more in-depth exploration of why the Google world system must fail and what may replace it.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: The Taking of K-129


“The Taking of K-129” by Josh DeanOn February 24, 1968, Soviet Golf class submarine K-129 sailed from its base in Petropavlovsk for a routine patrol in the Pacific Ocean. These ballistic missile submarines were, at the time, a key part of the Soviet nuclear deterrent. Each carried three SS-N-5 missiles armed with one 800 kiloton nuclear warhead per missile. This was an intermediate range missile which could hit targets inside an enemy country if the submarine approached sufficiently close to the coast. For defence and attacking other ships, Golf class submarines carried two torpedoes with nuclear warheads as well as conventional high explosive warhead torpedoes.

Unlike the U.S. nuclear powered Polaris submarines, the Golf class had conventional diesel-electric propulsion. When submerged, the submarine was powered by batteries which provided limited speed and range and required surfacing or running at shallow snorkel depth for regular recharging by the diesel engines. They would be the last generation of Soviet diesel-electric ballistic missile submarines: the Hotel class and subsequent boats would be nuclear powered.

K-129’s mission was to proceed stealthily to a region of open ocean north of Midway Atoll and patrol there, ready to launch its missiles at U.S. assets in the Pacific in case of war. Submarines on patrol would send coded burst transmissions on a prearranged schedule to indicate that their mission was proceeding as planned.

On March 8, a scheduled transmission from K-129 failed to arrive. This wasn’t immediately cause for concern, since equipment failure was not uncommon, and a submarine commander might choose not to transmit if worried that surfacing and sending the message might disclose his position to U.S. surveillance vessels and aircraft. But when K-129 remained silent for a second day, the level of worry escalated rapidly. Losing a submarine armed with nuclear weapons was a worst-case scenario, and one which had never happened in Soviet naval operations.

A large-scale search and rescue fleet of 24 vessels, including four submarines, set sail from the base in Kamchatka, all communicating in the open on radio and pinging away with active sonar. They were heard to repeatedly call a ship named Red Star with no reply. The search widened, and eventually included thirty-six vessels and fifty-three aircraft, continuing over a period of seventy-three days. Nothing was found, and six months after the disappearance, the Soviet Navy issued a statement that K-129 had been lost while on duty in the Pacific with all on board presumed dead. This was not only a wrenching emotional blow to the families of the crew, but also a financial gut-shot, depriving them of the pension due families of men lost in the line of duty and paying only the one-time accidental death payment and partial pension for industrial accidents.

But if the Soviets had no idea where their submarine was, this was not the case for the U.S. Navy. Sound travels huge distances through the oceans, and starting in the 1950s, the U.S. began to install arrays of hydrophones (undersea sound detectors) on the floors of the oceans around the world. By the 1960s, these arrays, called SOSUS (SOund SUrveillance System) were deployed and operational in both the Atlantic and Pacific and used to track the movements of Soviet submarines. When K-129 went missing, SOSUS analysts went back over their archived data and found a sharp pulse just a few seconds after midnight local time on March 11 around 180° West and 40° North: 2500 km northeast of Hawaii. Not only did the pulse appear nothing like the natural sounds often picked up by SOSUS, events like undersea earthquakes don’t tend to happen at socially constructed round number times and locations like this one. The pulse was picked up by multiple sensors, allowing its position to be determined accurately. The U.S. knew where the K-129 lay on the ocean floor. But what to do with that knowledge?

One thing was immediately clear. If the submarine was in reasonably intact condition, it would be an intelligence treasure unparalleled in the postwar era. Although it did not represent the latest Soviet technology, it would provide analysts their first hands-on examination of Soviet ballistic missile, nuclear weapon, and submarine construction technologies. Further, the boat would certainly be equipped with cryptographic and secure radio communications gear which might provide an insight into penetrating the secret communications to and from submarines on patrol. (Recall that British breaking of the codes used to communicate with German submarines in World War II played a major part in winning the Battle of the Atlantic.) But a glance at a marine chart showed how daunting it would be to reach the site of the wreck. The ocean in the vicinity of the co-ordinates identified by SOSUS was around 5000 metres deep. Only a very few special-purpose research vessels can operate at such a depth, where the water pressure is around 490 times that of the atmosphere at sea level.

The U.S. intelligence community wanted that sub. The first step was to make sure they’d found it. The USS Halibut, a nuclear-powered Regulus cruise missile launching submarine converted for special operations missions, was dispatched to the area where the K-129 was thought to lie. Halibut could not dive anywhere near as deep as the ocean floor, but was equipped with a remote-controlled, wire-tethered “fish”, which could be lowered near the bottom and then directed around the search area, observing with side-looking sonar and taking pictures. After seven weeks searching in vain, with fresh food long exhausted and crew patience wearing thin, the search was abandoned and course set back to Pearl Harbor.

But the prize was too great to pass up. So Halibut set out again, and after another month of operating the fish, developing thousands of pictures, and fraying tempers, there it was! Broken into two parts, but with both apparently largely intact, lying on the ocean bottom. Now what?

While there were deep sea research vessels able to descend to such depths, they were completely inadequate to exploit the intelligence haul that K-129 promised. That would require going inside the structure, dismantling the missiles and warheads, examining and testing the materials, and searching for communications and cryptographic gear. The only way to do this was to raise the submarine. To say that this was a challenge is to understate its difficulty—adjectives fail. The greatest mass which had ever been raised from such a depth was around 50 tonnes and K-129 had a mass of 1,500 tonnes—thirty times greater. But hey, why not? We’re Americans! We’ve landed on the Moon! (By then it was November, 1969, four months after that “one small step”.) And so, Project Azorian was born.

When it comes to doing industrial-scale things in the deep ocean, all roads (or sea lanes) lead to Global Marine. A publicly-traded company little known to those outside the offshore oil exploration industry, this company and its genius naval architect John Graham had pioneered deep-sea oil drilling. While most offshore oil rigs, like those on terra firma, were firmly anchored to the land around the drill hole, Global Marine had pioneered the technology which allowed a ship, with a derrick mounted amidships, to precisely station-keep above the bore-hole on the ocean floor far beneath the ship. The required dropping sonar markers on the ocean floor which the ship used to precisely maintain its position with respect to them. This was just one part of the puzzle.

To recover the submarine, the ship would need to lower what amounted to a giant claw (“That’s claw, not craw!”, you “Get Smart” fans) to the abyssal plain, grab the sub, and lift its 1500 tonne mass to the surface. During the lift, the pipe string which connected the ship to the claw would be under such stress that, should it break, it would release energy comparable to an eight kiloton nuclear explosion, which would be bad.

This would have been absurdly ambitious if conducted in the open, like the Apollo Project, but in this case it also had to be done covertly, since the slightest hint that the U.S. was attempting to raise K-129 would almost certainly provoke a Soviet response ranging from diplomatic protests to a naval patrol around the site of the sinking aimed at harassing the recovery ships. The project needed a cover story and a cut-out to hide the funding to Global Marine which, as a public company, had to disclose its financials quarterly and, unlike minions of the federal government funded by taxes collected from hairdressers and cab drivers through implicit threat of violence, could not hide its activities in a “black budget”.

This was seriously weird and, as a contemporary philosopher said, “When the going gets weird, the weird turn pro.” At the time, nobody was more professionally weird than Howard Hughes. He had taken reclusion to a new level, utterly withdrawing from contact with the public after revulsion from dealing with the Washington swamp and the media. His company still received royalties from every oil well drilled using his drill bits, and his aerospace and technology companies were plugged into the most secret ventures of the U.S. government. Simply saying, “It’s a Hughes project” was sufficient to squelch most questions. This meant it had unlimited funds, the sanction of the U.S. government (including three-letter agencies whose names must not be spoken [brrrr!]), and told pesky journalists they’d encounter a stone wall from the centre of the Earth to the edge of the universe if they tried to dig into details.

Hughes Glomar Explorer

But covert as the project might be, aspects of its construction and operation would unavoidably be in the public eye. You can’t build a 189 metre long, 51,000 tonne ship, the Hughes Glomar Explorer, with an 80 metre tall derrick sticking up amidships, at a shipyard on the east coast of the U.S., send it around Cape Horn to its base on the west coast (the ship was too wide to pass through the Panama Canal), without people noticing. A cover story was needed, and the CIA and their contractors cooked up a doozy.

Large areas of the deep sea floor are covered by manganese nodules, concretions which form around a seed and grow extremely slowly, but eventually reach the size of potatoes or larger. Nodules are composed of around 30% manganese, plus other valuable metals such as nickel, copper, and cobalt. There are estimated to be more than 21 billion tonnes of manganese nodules on the deep ocean floor (depths of 4000 to 6000 metres), and their composition is richer than many of the ores from which the metals they contain are usually extracted. Further, they’re just lying on the seabed. If you could figure out how to go down there and scoop them up, you wouldn’t have to dig mines and process huge amounts of rock. Finally, they were in international waters, and despite attempts by kleptocratic dictators (some in landlocked countries) and the international institutions who support them to enact a “Law of the Sea” treaty to pick the pockets of those who created the means to use this resource, at the time the nodules were just there for the taking—you didn’t have to pay kleptocratic dictators for mining rights or have your profits skimmed by ever-so-enlightened democratic politicians in developed countries.

So, the story was put out that Howard Hughes was setting out to mine the nodules on the Pacific Ocean floor, and that Glomar Explorer, built by Global Marine under contract for Hughes (operating, of course, as a cut-out for the CIA), would deploy a robotic mining barge called the Hughes Mining Barge 1 (HMB-1) which, lowered to the ocean floor, would collect nodules, crush them, and send the slurry to the surface for processing on the mother ship.

This solved a great number of potential problems. Global Marine, as a public company, could simply (and truthfully) report that it was building Glomar Explorer under contract to Hughes, and had no participation in the speculative and risky mining venture, which would have invited scrutiny by Wall Street analysts and investors. Hughes, operating as a proprietorship, was not required to disclose the source of the funds it was paying Global Marine. Everybody assumed the money was coming from Howard Hughes’ personal fortune, which he had invested, over his career, in numerous risky ventures, when in fact, he was simply passing through money from a CIA black budget account. The HMB-1 was built by Lockheed Missiles and Space Company under contract from Hughes. Lockheed was involved in numerous classified U.S. government programs, so operating in the same manner for the famously secretive Hughes raised few eyebrows.

The barge, 99 metres in length, was built in a giant enclosed hangar in the port of Redwood City, California, which shielded it from the eyes of curious onlookers and Soviet reconnaissance satellites passing overhead. This was essential, because a glance at what was being built would have revealed that it looked nothing like a mining barge but rather a giant craw—sorry—claw! To install the claw on the ship, it was towed, enclosed in its covered barge, to a location near Catalina Island in southern California, where deeper water allowed it to be sunk beneath the surface, and then lifted into the well (“moon pool”) of Glomar Explorer, all out of sight to onlookers.

So far, the project had located the target on the ocean floor, designed and built a special ship and retrieval claw to seize it, fabricated a cover story of a mining venture so persuasive other mining companies were beginning to explore launching their own seabed mining projects, and evaded scrutiny by the press, Congress, and Soviet intelligence assets. But these are pussycats compared to the California Tax Nazis! After the first test of mating the claw to the ship, Glomar Explorer took to the ocean to, it was said, test the stabilisation system which would keep the derrick vertical as the ship pitched and rolled in the sea. Actually, the purpose of the voyage was to get the ship out of U.S. territorial waters on March 1st, the day California assessed a special inventory tax on all commercial vessels in state waters. This would not only cost a lot of money, it would force disclosure of the value of the ship, which could be difficult to reconcile with its cover mission. Similar fast footwork was required when Hughes took official ownership of the vessel from Global Marine after acceptance. A trip outside U.S. territorial waters was also required to get off the hook for the 7% sales tax California would otherwise charge on the transfer of ownership.

Finally, in June 1974, all was ready, and Glomar Explorer with HMB-1 attached set sail from Long Beach, California to the site of K-129’s wreck, arriving on site on the Fourth of July, only to encounter foul weather. Opening the sea doors in the well in the centre of the ship and undocking the claw required calm seas, and it wasn’t until July 18th that they were ready to begin the main mission. Just at that moment, what should show up but a Soviet missile tracking ship. After sending its helicopter to inspect Explorer, it eventually departed. This wasn’t the last of the troubles with pesky Soviets.

On July 21, the recovery operation began, slowly lowering the claw on its string of pipes. Just at this moment, another Soviet ship arrived, a 47 metre ocean-going tug called SB-10. This tug would continue to harass the recovery operation for days, approaching on an apparent collision course and then veering off. (Glomar Explorer could not move during the retrieval operation, being required to use its thrusters to maintain its position directly above the wrecked submarine on the bottom.)

On August 3, the claw reached the bottom and its television cameras revealed it was precisely on target—there was the submarine, just as it had been photographed by the Halibut six years earlier. The claw gripped the larger part of the wreck, its tines closed under it, and a combination of pistons driving against the ocean bottom and the lift system pulling on the pipe from the ship freed the submarine from the bottom. Now the long lift could begin.

Everything had worked. The claw had been lowered, found its target on the first try, successfully seized it despite the ocean bottom’s being much harder than expected, freed it from the bottom, and the ship had then successfully begun to lift the 6.4 million kg of pipe, claw, and submarine back toward the surface. Within the first day of the lift, more than a third of the way to the surface, with the load on the heavy lift equipment diminishing by 15 tonnes as each segment of lift pipe was removed from the string, a shudder went through the ship and the heavy lift equipment lurched violently. Something had gone wrong, seriously wrong. Examination of television images from the claw revealed that several of the tines gripping the hull of the submarine had failed and part of the sub, maybe more than half, had broken off and fallen back toward the abyss. (It was later decided that the cause of the failure was that the tines had been fabricated from maraging steel, which is very strong but brittle, rather than a more ductile alloy which would bend under stress but not break.)

After consultation with CIA headquarters, it was decided to continue the lift and recover whatever was left in the claw. (With some of the tines broken and the mechanism used to break the load free of the ocean floor left on the bottom, it would have been impossible to return and recover the lost part of the sub on this mission.) On August 6th, the claw and its precious payload reached the ship and entered the moon pool in its centre. Coincidentally, the Soviet tug departed the scene the same day. Now it was possible to assess what had been recovered, and the news was not good: two thirds of the sub had been lost, including the ballistic missile tubes and the code room. Only the front third was in the claw. Further, radiation five times greater than background was detected even outside the hull—those exploring it would have to proceed carefully.

An “exploitation team” composed of CIA specialists and volunteers from the ship’s crew began to explore the wreckage, photographing and documenting every part recovered. They found the bodies of six Soviet sailors and assorted human remains which could not be identified; all went to the ship’s morgue. Given that the bow portion of the submarine had been recovered, it is likely that one or more of its torpedoes equipped with nuclear warheads were recovered, but to this day the details of what was found in the wreck remain secret. By early September, the exploitation was complete and the bulk of the recovered hull, less what had been removed and sent for analysis, was dumped in the deep ocean 160 km south of Hawaii.

One somber task remained. On September 4, 1974, the remains of the six recovered crewmen and the unidentified human remains were buried at sea in accordance with Soviet Navy tradition. A video tape of this ceremony was made and, in 1992, a copy was presented to Russian President Boris Yeltsin by then CIA director Robert Gates.

The partial success encouraged some in the CIA to mount a follow-up mission to recover the rest of the sub, including the missiles and code room. After all, they knew precisely where it was, had a ship in hand, fully paid for, which had successfully lowered the claw to the bottom and returned to the surface with part of the sub, and they knew what had gone wrong with the claw and how to fix it. The effort was even given a name, Project Matador. But it was not to be.

Over the five years of the project there had been leaks to the press and reporters sniffing on the trail of the story but the CIA had been able to avert disclosure by contacting the reporters directly, explaining the importance of the mission and need for secrecy, and offering them an exclusive of full disclosure and permission to publish it before the project was officially declassified for the general public. This had kept a lid on the secret throughout the entire development process and the retrieval and analysis, but this all came to an end in March 1975 when Jack Anderson got wind of the story. There was no love lost between Anderson and what we now call the Deep State. Anderson believed the First Amendment was divinely inspired and absolute, while J. Edgar Hoover had called Anderson “lower than the regurgitated filth of vultures”. Further, this was a quintessential Jack Anderson story—based upon his sources, he presented Project Azorian as a US$ 350 million failure which had produced no useful intelligence information and was being kept secret only to cover up the squandering of taxpayers’ money.

New York Times Azorian story by Seymour HershCIA Director William Colby offered Anderson the same deal other journalists had accepted, but was flatly turned down. Five minutes before Anderson went on the radio to break the story, Colby was still pleading with him to remain silent. On March 18, 1975, Anderson broke the story on his Mutual Radio Network show and, the next day, published additional details in his nationally syndicated newspaper column. Realising the cover had been blown, Colby called all of the reporters who had agreed to hold the story to give them the green light to publish. Seymour Hersh of the New York Times had his story ready to go, and it ran on the front page of the next day’s paper, providing far more detail (albeit along with a few errors) than Anderson’s disclosure. Hersh revealed that he had been aware of the project since 1973 but had agreed to withhold publication in the interest of national security.

The story led newspaper and broadcast news around the country and effectively drove a stake through any plans to mount a follow-up retrieval mission. On June 16, 1975, Secretary of State Henry Kissinger made a formal recommendation to president Gerald Ford to terminate the project and that was the end of it. The Soviets had communicated through a back channel that they had no intention of permitting a second retrieval attempt and they had maintained an ocean-going tug on site to monitor any activity since shortly after the story broke in the U.S.

The CIA’s official reaction to all the publicity was what has come to be called the “Glomar Response”: “We can neither confirm nor can we deny.” And that is where things stand more that four decades after the retrieval attempt. Although many of those involved in the project have spoken informally about aspects of it, there has never been an official report on precisely what was recovered or what was learned from it. Some CIA veterans have said, off the record, that much more was learned from the recovered material than has been suggested in press reports, with a few arguing that the entire large portion of the sub was recovered and the story about losing much of it was a cover story. (But if this was the case, the whole plan to mount a second retrieval mission and the substantial expense repairing and upgrading the claw for the attempt, which is well documented, would also have to have been a costly cover story.)

What is certain is that Project Azorian was one of the most daring intelligence exploits in history, carried out in total secrecy under the eyes of the Soviets, and kept secret from an inquiring press for five years by a cover story so persuasive other mining companies bought it hook, line, and sinker. We may never know all the details of the project, but from what we do know it is a real-world thriller which equals or exceeds those imagined by masters of the fictional genre.

Dean, Josh. The Taking of K-129. New York: Dutton, 2012. ISBN 978-1-101-98443-7.

Here is a Discovery Channel documentary (complete with cheesy period commercials) about Project Azorian and the recovery of K-129.  A portion of the burial at sea of the Soviet crew is shown at the end.

John Evans was Vice President of Global Marine during the project.  In this talk he gives his personal reminiscences of the project, declining to comment on information which has not become public from other sources.

This educational film was produced while the K-129 recovery project was underway.  Purporting to be about deep sea mining, it presents the cover story for Glomar Explorer, including bogus film of a supposed mining barge which was never used with the ship.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Losing the Nobel Prize

“Losing the Nobel Prize” by Brian KeatingEver since the time of Galileo, the history of astronomy has been punctuated by a series of “great debates”—disputes between competing theories of the organisation of the universe which observation and experiment using available technology are not yet able to resolve one way or another. In Galileo’s time, the great debate was between the Ptolemaic model, which placed the Earth at the centre of the solar system (and universe) and the competing Copernican model which had the planets all revolving around the Sun. Both models worked about as well in predicting astronomical phenomena such as eclipses and the motion of planets, and no observation made so far had been able to distinguish them.

Then, in 1610, Galileo turned his primitive telescope to the sky and observed the bright planets Venus and Jupiter. He found Venus to exhibit phases, just like the Moon, which changed over time. This would not happen in the Ptolemaic system, but is precisely what would be expected in the Copernican model—where Venus circled the Sun in an orbit inside that of Earth. Turning to Jupiter, he found it to be surrounded by four bright satellites (now called the Galilean moons) which orbited the giant planet. This further falsified Ptolemy’s model, in which the Earth was the sole source of attraction around which all celestial bodies revolved. Since anybody could build their own telescope and confirm these observations, this effectively resolved the first great debate in favour of the Copernican heliocentric model, although some hold-outs in positions of authority resisted its dethroning of the Earth as the centre of the universe.

This dethroning came to be called the “Copernican principle”, that Earth occupies no special place in the universe: it is one of a number of planets orbiting an ordinary star in a universe filled with a multitude of other stars. Indeed, when Galileo observed the star cluster we call the Pleiades, he saw myriad stars too dim to be visible to the unaided eye. Further, the bright stars were surrounded by a diffuse bluish glow. Applying the Copernican principle again, he argued that the glow was due to innumerably more stars too remote and dim for his telescope to resolve, and then generalised that the glow of the Milky Way was also composed of uncountably many stars. Not only had the Earth been demoted from the centre of the solar system, so had the Sun been dethroned to being just one of a host of stars possibly stretching to infinity.

But Galileo’s inference from observing the Pleiades was wrong. The glow that surrounds the bright stars is due to interstellar dust and gas which reflect light from the stars toward Earth. No matter how large or powerful the telescope you point toward such a reflection nebula, all you’ll ever see is a smooth glow. Driven by the desire to confirm his Copernican convictions, Galileo had been fooled by dust. He would not be the last.

William Herschel was an eminent musician and composer, but his passion was astronomy. He pioneered the large reflecting telescope, building more than sixty telescopes. In 1789, funded by a grant from King George III, Herschel completed a reflector with a mirror 1.26 metres in diameter, which remained the largest aperture telescope in existence for the next fifty years. In Herschel’s day, the great debate was about the Sun’s position among the surrounding stars. At the time, there was no way to determine the distance or absolute brightness of stars, but Herschel decided that he could compile a map of the galaxy (then considered to be the entire universe) by surveying the number of stars in different directions. Only if the Sun was at the centre of the galaxy would the counts be equal in all directions.

Aided by his sister Caroline, a talented astronomer herself, he eventually compiled a map which indicated the galaxy was in the shape of a disc, with the Sun at the centre. This seemed to refute the Copernican view that there was nothing special about the Sun’s position. Such was Herschel’s reputation that this finding, however puzzling, remained unchallenged until 1847 when Wilhelm Struve discovered that Herschel’s results had been rendered invalid by his failing to take into account the absorption and scattering of starlight by interstellar dust. Just as you can only see the same distance in all directions while within a patch of fog, regardless of the shape of the patch, Herschel’s survey could only see so far before extinction of light by dust cut off his view of stars. Later it was discovered that the Sun is far from the centre of the galaxy. Herschel had been fooled by dust.

In the 1920s, another great debate consumed astronomy. Was the Milky Way the entire universe, or were the “spiral nebulæ” other “island universes”, galaxies in their own right, peers of the Milky Way? With no way to measure distance or telescopes able to resolve them into stars, many astronomers believed spiral neublæ were nearby objects, perhaps other solar systems in the process of formation. The discovery of a Cepheid variable star in the nearby Andromeda “nebula” by Edwin Hubble in 1923 allowed settling this debate. Andromeda was much farther away than the most distant stars found in the Milky Way. It must, then be a separate galaxy. Once again, demotion: the Milky Way was not the entire universe, but just one galaxy among a multitude.

But how far away were the galaxies? Hubble continued his search and measurements and found that the more distant the galaxy, the more rapidly it was receding from us. This meant the universe was expanding. Hubble was then able to calculate the age of the universe—the time when all of the galaxies must have been squeezed together into a single point. From his observations, he computed this age at two billion years. This was a major embarrassment: astrophysicists and geologists were confident in dating the Sun and Earth at around five billion years. It didn’t make any sense for them to be more than twice as old as the universe of which they were a part. Some years later, it was discovered that Hubble’s distance estimates were far understated because he failed to account for extinction of light from the stars he measured due to dust. The universe is now known to be seven times the age Hubble estimated. Hubble had been fooled by dust.

By the 1950s, the expanding universe was generally accepted and the great debate was whether it had come into being in some cataclysmic event in the past (the “Big Bang”) or was eternal, with new matter spontaneously appearing to form new galaxies and stars as the existing ones receded from one another (the “Steady State” theory). Once again, there were no observational data to falsify either theory. The Steady State theory was attractive to many astronomers because it was the more “Copernican”—the universe would appear overall the same at any time in an infinite past and future, so our position in time is not privileged in any way, while in the Big Bang the distant past and future are very different than the conditions we observe today. (The rate of matter creation required by the Steady State theory was so low that no plausible laboratory experiment could detect it.)

The discovery of the cosmic background radiation in 1965 definitively settled the debate in favour of the Big Bang. It was precisely what was expected if the early universe were much denser and hotter than conditions today, as predicted by the Big Bang. The Steady State theory made no such prediction and was, despite rear-guard actions by some of its defenders (invoking dust to explain the detected radiation!), was considered falsified by most researchers.

But the Big Bang was not without its own problems. In particular, in order to end up with anything like the universe we observe today, the initial conditions at the time of the Big Bang seemed to have been fantastically fine-tuned (for example, an infinitesimal change in the balance between the density and rate of expansion in the early universe would have caused the universe to quickly collapse into a black hole or disperse into the void without forming stars and galaxies). There was no physical reason to explain these fine-tuned values; you had to assume that’s just the way things happened to be, or that a Creator had set the dial with a precision of dozens of decimal places.

In 1979, the theory of inflation was proposed. Inflation held that in an instant after the Big Bang the size of the universe blew up exponentially so that all the observable universe today was, before inflation, the size of an elementary particle today. Thus, it’s no surprise that the universe we now observe appears so uniform. Inflation so neatly resolved the tensions between the Big Bang theory and observation that it (and refinements over the years) became widely accepted. But could inflation be observed? That is the ultimate test of a scientific theory.

There have been numerous cases in science where many years elapsed between a theory being proposed and definitive experimental evidence for it being found. After Galileo’s observations, the Copernican theory that the Earth orbits the Sun became widely accepted, but there was no direct evidence for the Earth’s motion with respect to the distant stars until the discovery of the aberration of light in 1727. Einstein’s theory of general relativity predicted gravitational radiation in 1915, but the phenomenon was not directly detected by experiment until a century later. Would inflation have to wait as long or longer?

Things didn’t look promising. Almost everything we know about the universe comes from observations of electromagnetic radiation: light, radio waves, X-rays, etc., with a little bit more from particles (cosmic rays and neutrinos). But the cosmic background radiation forms an impenetrable curtain behind which we cannot observe anything via the electromagnetic spectrum, and it dates from around 380,000 years after the Big Bang. The era of inflation was believed to have ended 10−32 seconds after the Bang; considerably earlier. The only “messenger” which could possibly have reached us from that era is gravitational radiation. We’ve just recently become able to detect gravitational radiation from the most violent events in the universe, but no conceivable experiment would be able to detect this signal from the baby universe.

So is it hopeless? Well, not necessarily…. The cosmic background radiation is a snapshot of the universe as it existed 380,000 years after the Big Bang, and only a few years after it was first detected, it was realised that gravitational waves from the very early universe might have left subtle imprints upon the radiation we observe today. In particular, gravitational radiation creates a form of polarisation called B-modes which most other sources cannot create.

If it were possible to detect B-mode polarisation in the cosmic background radiation, it would be a direct detection of inflation. While the experiment would be demanding and eventually result in literally going to the end of the Earth, it would be strong evidence for the process which shaped the universe we inhabit and, in all likelihood, a ticket to Stockholm for those who made the discovery.

This was the quest on which the author embarked in the year 2000, resulting in the deployment of an instrument called BICEP1 (Background Imaging of Cosmic Extragalactic Polarization) in the Dark Sector Laboratory at the South Pole. Here is my picture of that laboratory in January 2013. The BICEP telescope is located in the foreground inside a conical shield which protects it against thermal radiation from the surrounding ice. In the background is the South Pole Telescope, a millimetre wave antenna which was not involved in this research.

BICEP2 and South Pole Telescope, 2013-01-09

BICEP1 was a prototype, intended to test the technologies to be used in the experiment. These included cooling the entire telescope (which was a modest aperture [26 cm] refractor, not unlike Galileo’s, but operating at millimetre wavelengths instead of visible light) to the temperature of interstellar space, with its detector cooled to just ¼ degree above absolute zero. In 2010 its successor, BICEP2, began observation at the South Pole, and continued its run into 2012. When I took the photo above, BICEP2 had recently concluded its observations.

On March 17th, 2014, the BICEP2 collaboration announced, at a press conference, the detection of B-mode polarisation in the region of the southern sky they had monitored. Note the swirling pattern of polarisation which is the signature of B-modes, as opposed to the starburst pattern of other kinds of polarisation.

Cosmic background radiation B-modes from BICEP2

But, not so fast, other researchers cautioned. The risk in doing “science by press release” is that the research is not subjected to peer review—criticism by other researchers in the field—before publication and further criticism in subsequent publications. The BICEP2 results went immediately to the front pages of major newspapers. Here was direct evidence of the birth cry of the universe and confirmation of a theory which some argued implied the existence of a multiverse—the latest Copernican demotion—the idea that our universe was just one of an ensemble, possibly infinite, of parallel universes in which every possibility was instantiated somewhere. Amid the frenzy, a few specialists in the field, including researchers on competing projects, raised the question, “What about the dust?” Dust again! As it happens, while gravitational radiation can induce B-mode polarisation, it isn’t the only thing which can do so. Our galaxy is filled with dust and magnetic fields which can cause those dust particles to align with them. Aligned dust particles cause polarised reflections which can mimic the B-mode signature of the gravitational radiation sought by BICEP2.

The BICEP2 team was well aware of this potential contamination problem. Unfortunately, their telescope was sensitive only to one wavelength, chosen to be the most sensitive to B-modes due to primordial gravitational radiation. It could not, however, distinguish a signal from that cause from one due to foreground dust. At the same time, however, the European Space Agency Planck spacecraft was collecting precision data on the cosmic background radiation in a variety of wavelengths, including one sensitive primarily to dust. Those data would have allowed the BICEP2 investigators to quantify the degree their signal was due to dust. But there was a problem: BICEP2 and Planck were direct competitors.

Planck had the data, but had not released them to other researchers. However, the BICEP2 team discovered that a member of the Planck collaboration had shown a slide at a conference of unpublished Planck observations of dust. A member of the BICEP2 team digitised an image of the slide, created a model from it, and concluded that dust contamination of the BICEP2 data would not be significant. This was a highly dubious, if not explicitly unethical move. It confirmed measurements from earlier experiments and provided confidence in the results.

In September 2014, a preprint from the Planck collaboration (eventually published in 2016) showed that B-modes from foreground dust could account for all of the signal detected by BICEP2. In January 2015, the European Space Agency published an analysis of the Planck and BICEP2 observations which showed the entire BICEP2 detection was consistent with dust in the Milky Way. The epochal detection of inflation had been deflated. The BICEP2 researchers had been deceived by dust.

The author, a founder of the original BICEP project, was so close to a Nobel prize he was already trying to read the minds of the Nobel committee to divine who among the many members of the collaboration they would reward with the gold medal. Then it all went away, seemingly overnight, turned to dust. Some said that the entire episode had injured the public’s perception of science, but to me it seems an excellent example of science working precisely as intended. A result is placed before the public; others, with access to the same raw data are given an opportunity to critique them, setting forth their own raw data; and eventually researchers in the field decide whether the original results are correct. Yes, it would probably be better if all of this happened in musty library stacks of journals almost nobody reads before bursting out of the chest of mass media, but in an age where scientific research is funded by agencies spending money taken from hairdressers and cab drivers by coercive governments under implicit threat of violence, it is inevitable they will force researchers into the public arena to trumpet their “achievements”.

In parallel with the saga of BICEP2, the author discusses the Nobel Prizes and what he considers to be their dysfunction in today’s scientific research environment. I was surprised to learn that many of the curious restrictions on awards of the Nobel Prize were not, as I had heard and many believe, conditions of Alfred Nobel’s will. In fact, the conditions that the prize be shared no more than three ways, not be awarded posthumously, and not awarded to a group (with the exception of the Peace prize) appear nowhere in Nobel’s will, but were imposed later by the Nobel Foundation. Further, Nobel’s will explicitly states that the prizes shall be awarded to “those who, during the preceding year, shall have conferred the greatest benefit to mankind”. This constraint (emphasis mine) has been ignored since the inception of the prizes.

He decries the lack of “diversity” in Nobel laureates (by which he means, almost entirely, how few women have won prizes). While there have certainly been women who deserved prizes and didn’t win (Lise Meitner, Jocelyn Bell Burnell, and Vera Rubin are prime examples), there are many more men who didn’t make the three laureates cut-off (Freeman Dyson an obvious example for the 1965 Physics Nobel for quantum electrodynamics). The whole Nobel prize concept is capricious, and rewards only those who happen to be in the right place at the right time in the right field that the committee has decided deserves an award this year and are lucky enough not to die before the prize is awarded. To imagine it to be “fair” or representative of scientific merit is, in the estimation of this scribbler, in flying unicorn territory.

In all, this is a candid view of how science is done at the top of the field today, with all of the budget squabbles, maneuvering for recognition, rivalry among competing groups of researchers, balancing the desire to get things right with the compulsion to get there first, and the eye on that prize, given only to a few in a generation, which can change one’s life forever.

Personally, I can’t imagine being so fixated on winning a prize one has so little chance of gaining. It’s like being obsessed with winning the lottery—and about as likely.

In parallel with all of this is an autobiographical account of the career of a scientist with its ups and downs, which is both a cautionary tale and an inspiration to those who choose to pursue that difficult and intensely meritocratic career path.

I recommend this book on all three tracks: a story of scientific discovery, mis-interpretation, and self-correction, the dysfunction of the Nobel Prizes and how they might be remedied, and the candid story of a working scientist in today’s deeply corrupt coercively-funded research environment.

Keating, Brian. Losing the Nobel Prize. New York: W. W. Norton, 2018. ISBN 978-1-324-00091-4.

Here is a one hour talk by the author about the BICEP2 experience and the Nobel Prize.

This is the BICEP2 press conference on March 17, 2014, announcing the discovery of B-mode polarisation in the cosmic microwave background radiation.

What do you do after losing the Nobel prize?  In this April, 2016 (much) more technical talk at the SETI Institute, Brian Keating describes post-BICEP2 research aimed at using the cosmic background radiation to explore other aspects of the early universe including whether the universe has an inherent chirality (left- or right-handedness).  (The preview image for this video looks like it’s broken, but if you click the play button it plays correctly, at least for me.)


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Bad Blood

“Bad Blood” by John CarreyrouThe drawing of blood for laboratory tests is one of my least favourite parts of a routine visit to the doctor’s office. Now, I have no fear of needles and hardly notice the stick, but frequently the doctor’s assistant who draws the blood (whom I’ve nicknamed Vampira) has difficulty finding the vein to get a good flow and has to try several times. On one occasion she made an internal puncture which resulted in a huge, ugly bruise that looked like I’d slammed a car door on my arm. I wondered why they need so much blood, and why draw it into so many different containers? (Eventually, I researched this, having been intrigued by the issue during the O. J. Simpson trial; if you’re curious, here is the information.) Then, after the blood is drawn, it has to be sent off to the laboratory, which sends back the results days later. If something pops up in the test results, you have to go back for a second visit with the doctor to discuss it.

Wouldn’t it be great if they could just stick a fingertip and draw a drop or two of blood, as is done by diabetics to test blood sugar, then run all the tests on it? Further, imagine if, after taking the drop of blood, it could be put into a desktop machine right in the doctor’s office which would, in a matter of minutes, produce test results you could discuss immediately with the doctor. And if such a technology existed and followed the history of decline in price with increase in volume which has characterised other high technology products since the 1970s, it might be possible to deploy the machines into the homes of patients being treated with medications so their effects could be monitored and relayed directly to their physicians in case an anomaly was detected. It wouldn’t quite be a Star Trek medical tricorder, but it would be one step closer. With the cost of medical care rising steeply, automating diagnostic blood tests and bringing them to the mass market seemed an excellent candidate as the “next big thing” for Silicon Valley to revolutionise.

This was the vision that came to 19 year old Elizabeth Holmes after completing a summer internship at the Genome Institute of Singapore after her freshman year as a chemical engineering major at Stanford. Holmes had decided on a career in entrepreneurship from an early age and, after her first semester told her father, “No, Dad, I’m, not interested in getting a Ph.D. I want to make money.” And Stanford, in the heart of Silicon Valley, was surrounded by companies started by professors and graduates who had turned inventions into vast fortunes. With only one year of college behind her, she was sure she’d found her opportunity. She showed the patent application she’d drafted for an arm patch that would diagnose medical conditions to Channing Robertson, professor of chemical engineering at Stanford, and Shaunak Roy, the Ph.D. student in whose lab she had worked as an assistant during her freshman year. Robertson was enthusiastic, and when Holmes said she intended to leave Stanford and start a company to commercialise the idea, he encouraged her. When the company was incorporated in 2004, Roy, then a newly-minted Ph.D., became its first employee and Robertson joined the board.

From the outset, the company was funded by other people’s money. Holmes persuaded a family friend, Tim Draper, a second-generation venture capitalist who had backed, among other companies, Hotmail, to invest US$ 1 million in first round funding. Draper was soon joined by Victor Palmieri, a corporate turnaround artist and friend of Holmes’ father. The company was named Theranos, from “therapy” and “diagnosis”. Elizabeth, unlike this scribbler, had a lifelong aversion to needles, and the invention she described in the business plan pitched to investors was informed by this. A skin patch would draw tiny quantities of blood without pain by means of “micro-needles”, the blood would be analysed by micro-miniaturised sensors in the patch and, if needed, medication could be injected. A wireless data link would send results to the doctor.

This concept, and Elizabeth’s enthusiasm and high-energy pitch allowed her to recruit additional investors, raising almost US$ 6 million in 2004. But there were some who failed to be persuaded: MedVentures Associates, a firm that specialised in medical technology, turned her down after discovering she had no answers for the technical questions raised in a meeting with the partners, who had in-depth experience with diagnostic technology. This would be a harbinger of the company’s fund-raising in the future: in its entire history, not a single venture fund or investor with experience in medical or diagnostic technology would put money into the company.

Shaunak Roy, who, unlike Holmes, actually knew something about chemistry, quickly realised that Elizabeth’s concept, while appealing to the uninformed, was science fiction, not science, and no amount of arm-waving about nanotechnology, microfluidics, or laboratories on a chip would suffice to build something which was far beyond the state of the art. This led to a “de-scoping” of the company’s ambition—the first of many which would happen over succeeding years. Instead of Elizabeth’s magical patch, a small quantity of blood would be drawn from a finger stick and placed into a cartridge around the size of a credit card. The disposable cartridge would then be placed into a desktop “reader” machine, which would, using the blood and reagents stored in the cartridge, perform a series of analyses and report the results. This was originally called Theranos 1.0, but after a series of painful redesigns, was dubbed the “Edison”. This was the prototype Theranos ultimately showed to potential customers and prospective investors.

This was a far cry from the original ambitious concept. The hundreds of laboratory tests doctors can order are divided into four major categories: immunoassays, general chemistry, hæmatology, and DNA amplification. In immunoassay tests, blood plasma is exposed to an antibody that detects the presence of a substance in the plasma. The antibody contains a marker which can be detected by its effect on light passed through the sample. Immunoassays are used in a number of common blood tests, such the 25(OH)D assay used to test for vitamin D deficiency, but cannot perform other frequently ordered tests such as blood sugar and red and white blood cell counts. Edison could only perform what is called “chemiluminescent immunoassays”, and thus could only perform a fraction of the tests regularly ordered. The rationale for installing an Edison in the doctor’s office was dramatically reduced if it could only do some tests but still required a venous blood draw be sent off to the laboratory for the balance.

This didn’t deter Elizabeth, who combined her formidable salesmanship with arm-waving about the capabilities of the company’s products. She was working on a deal to sell four hundred Edisons to the Mexican government to cope with an outbreak of swine flu, which would generate immediate revenue. Money was much on the minds of Theranos’ senior management. By the end of 2009, the company had burned through the US$ 47 million raised in its first three rounds of funding and, without a viable product or prospects for sales, would have difficulty keeping the lights on.

But the real bonanza loomed on the horizon in 2010. Drugstore giant Walgreens was interested in expanding their retail business into the “wellness market”: providing in-store health services to their mass market clientèle. Theranos pitched them on offering in-store blood testing. Doctors could send their patients to the local Walgreens to have their blood tested from a simple finger stick and eliminate the need to draw blood in the office or deal with laboratories. With more than 8,000 locations in the U.S., if each were to be equipped with one Edison, the revenue to Theranos (including the single-use testing cartridges) would put them on the map as another Silicon Valley disruptor that went from zero to hundreds of millions in revenue overnight. But here, as well, the Elizabeth effect was in evidence. Of the 192 tests she told Walgreens Theranos could perform, fewer than half were immunoassays the Edisons could run. The rest could be done only on conventional laboratory equipment, and certainly not on a while-you-wait basis.

Walgreens wasn’t the only potential saviour on the horizon. Grocery godzilla Safeway, struggling with sales and earnings which seemed to have reached a peak, saw in-store blood testing with Theranos machines as a high-margin profit centre. They loaned Theranos US$ 30 million and began to plan for installation of blood testing clinics in their stores.

But there was a problem, and as the months wore on, this became increasingly apparent to people at both Walgreens and Safeway, although dismissed by those in senior management under the spell of Elizabeth’s reality distortion field. Deadlines were missed. Simple requests, such as A/B comparison tests run on the Theranos hardware and at conventional labs were first refused, then postponed, then run but results not disclosed. The list of tests which could be run, how blood for them would be drawn, and how they would be processed seemed to dissolve into fog whenever specific requests were made for this information, which was essential for planning the in-store clinics.

There was, indeed, a problem, and it was pretty severe, especially for a start-up which had burned through US$ 50 million and sold nothing. The product didn’t work. Not only could the Edison only run a fraction of the tests its prospective customers had been led by Theranos to believe it could, for those it did run the results were wildly unreliable. The small quantity of blood used in the test introduced random errors due to dilution of the sample; the small tubes in the cartridge were prone to clogging; and capillary blood collected from a finger stick was prone to errors due to “hemolysis”, the rupture of red blood cells, which is minimal in a venous blood draw but so prevalent in finger stick blood it could lead to some tests producing values which indicated the patient was dead.

Meanwhile, people who came to work at Theranos quickly became aware that it was not a normal company, even by the eccentric standards of Silicon Valley. There was an obsession with security, with doors opened by badge readers; logging of employee movement; information restricted to narrow silos prohibiting collaboration between, say, engineering and marketing which is the norm in technological start-ups; monitoring of employee Internet access, E-mail, and social media presence; a security detail of menacing-looking people in black suits and earpieces (which eventually reached a total of twenty); a propensity of people, even senior executives, to “vanish”, Stalin-era purge-like, overnight; and a climate of fear that anybody, employee or former employee, who spoke about the company or its products to an outsider, especially the media, would be pursued, harassed, and bankrupted by lawsuits. There aren’t many start-ups whose senior scientists are summarily demoted and subsequently commit suicide. That happened at Theranos. The company held no memorial for him.

Throughout all of this, a curious presence in the company was Ramesh (“Sunny”) Balwani, a Pakistani-born software engineer who had made a fortune of more than US$ 40 million in the dot-com boom and cashed out before the bust. He joined Theranos in late 2009 as Elizabeth’s second in command and rapidly became known as a hatchet man, domineering boss, and clueless when it came to the company’s key technologies (on one occasion, an engineer mentioned a robotic arm’s “end effector”, after which Sunny would frequently speak of its “endofactor”). Unbeknownst to employees and investors, Elizabeth and Sunny had been living together since 2005. Such an arrangement would be a major scandal in a public company, but even in a private firm, concealing such information from the board and investors is a serious breach of trust.

Let’s talk about the board, shall we? Elizabeth was not only persuasive, but well-connected. She would parley one connection into another, and before long had recruited many prominent figures including:

  • George Schultz (former U.S. Secretary of State)
  • Henry Kissinger (former U.S. Secretary of State)
  • Bill Frist (former U.S. Senator and medical doctor)
  • James Mattis (General, U.S. Marine Corps)
  • Riley Bechtel (Chairman and former CEO, Bechtel Group)
  • Sam Nunn (former U.S. Senator)
  • Richard Kobacevich (former Wells Fargo chairman and CEO)

Later, super-lawyer David Boies would join the board, and lead its attacks against the company’s detractors. It is notable that, as with its investors, not a single board member had experience in medical or diagnostic technology. Bill Frist was an M.D., but his speciality was heart and lung transplants, not laboratory tests.

By 2014, Elizabeth Holmes had come onto the media radar. Photogenic, articulate, and with a story of high-tech disruption of an industry much in the news, she began to be featured as the “female Steve Jobs”, which must have pleased her, since she affected black turtlenecks, kale shakes, and even a car with no license plates to emulate her role model. She appeared on the cover of Fortune in January 2014, made the Forbes list of 400 most wealthy shortly thereafter, was featured in puff pieces in business and general market media, and was named by Time as one of the hundred most influential people in the world. The year 2014 closed with another glowing profile in the New Yorker. This would be the beginning of the end, as it happened to be read by somebody who actually knew something about blood testing.

Adam Clapper, a pathologist in Missouri, spent his spare time writing Pathology Blawg, with a readership of practising pathologists. Clapper read what Elizabeth was claiming to do with a couple of drops of blood from a finger stick and it didn’t pass the sniff test. He wrote a sceptical piece on his blog and, as it passed from hand to hand, he became a lightning rod for others dubious of Theranos’ claims, including those with direct or indirect experience with the company. Earlier, he had helped a Wall Street Journal reporter comprehend the tangled web of medical laboratory billing, and he decided to pass on the tip to the author of this book.

Thus began the unravelling of one of the greatest scams and scandals in the history of high technology, Silicon Valley, and venture investing. At the peak, privately-held Theranos was valued at around US$ 9 billion, with Elizabeth Holmes holding around half of its common stock, and with one of those innovative capital structures of which Silicon Valley is so fond, 99.7% of the voting rights. Altogether, over its history, the company raised around US$ 900 million from investors (including US$ 125 million from Rupert Murdoch in the US$ 430 million final round of funding). Most of the investors’ money was ultimately spent on legal fees as the whole fairy castle crumbled.

The story of the decline and fall is gripping, involving the grandson of a Secretary of State, gumshoes following whistleblowers and reporters, what amounts to legal terrorism by the ever-slimy David Boies, courageous people who stood their ground in the interest of scientific integrity against enormous personal and financial pressure, and the saga of one of the most cunning and naturally talented confidence women ever, equipped with only two semesters of freshman chemical engineering, who managed to raise and blow through almost a billion dollars of other people’s money without checking off the first box on the conventional start-up check list: “Build the product”.

I have, in my career, met three world-class con men. Three times, I (just barely) managed to pick up the warning signs and beg my associates to walk away. Each time I was ignored. After reading this book, I am absolutely sure that had Elizabeth Holmes pitched me on Theranos (about which I never heard before the fraud began to be exposed), I would have been taken in. Walker’s law is “Absent evidence to the contrary, assume everything is a scam”. A corollary is “No matter how cautious you are, there’s always a confidence man (or woman) who can scam you if you don’t do your homework.”

Carreyrou, John. Bad Blood. New York: Alfred A. Knopf, 2018. ISBN 978-1-984833-63-1.

Here is Elizabeth Holmes at Stanford in 2013, when Theranos was riding high and she was doing her “female Steve Jobs” act.

This is a CNN piece, filmed after the Theranos scam had begun to collapse, in which you can still glimpse the Elizabeth Holmes reality distortion field at full intensity directed at CNN medical correspondent Sanjay Gupta. There are several curious things about this video. The machine that Gupta is shown is the “miniLab”, a prototype second-generation machine which never worked acceptably, not the Edison, which was actually used in the Walgreens and Safeway tests. Gupta’s blood is drawn and tested, but the process used to perform the test is never shown. The result reported is a cholesterol test, but the Edison cannot perform such tests. In the plans for the Walgreens and Safeway roll-outs, such tests were performed on purchased Siemens analysers which had been secretly hacked by Theranos to work with blood diluted well below their regulatory-approved specifications (the dilution was required due to the small volume of blood from the finger stick). Since the miniLab never really worked, the odds are that Gupta’s blood was tested on one of the Siemens machines, not a Theranos product at all.

In a June 2018 interview, author John Carreyrou recounts the story of Theranos and his part in revealing the truth.  There is substantial background information in the question and answer period which does not appear in the book.

If you are a connoisseur of the art of the con, here is a masterpiece. After the Wall Street Journal exposé had broken, after retracting tens of thousands of blood tests, and after Theranos had been banned from running a clinical laboratory by its regulators, Holmes got up before an audience of 2500 people at the meeting of the American Association of Clinical Chemistry and turned up the reality distortion field to eleven. Watch a master at work. She comes on the stage at the six minute mark.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Project Cyclops

Project Cyclops: Full arrayThere are few questions in science as simple to state and profound in their implications as “are we alone?”—are humans the only species with a technological civilisation in the galaxy, or in the universe?  This has been a matter of speculation by philosophers, theologians, authors of fiction, and innumerable people gazing at the stars since antiquity, but it was only in the years after World War II, which had seen the development of high-power microwave transmitters and low-noise receivers for radar, that it dawned upon a few visionaries that this had now become a question which could be scientifically investigated.

The propagation of radio waves through the atmosphere and the interstellar medium is governed by basic laws of physics, and the advent of radio astronomy demonstrated that many objects in the sky, some very distant, could be detected in the microwave spectrum.  But if we were able to detect these natural sources, suppose we connected a powerful transmitter to our radio telescope and sent a signal to a nearby star?  It was easy to calculate that, given the technology of the time (around 1960), existing microwave transmitters and radio telescopes could transmit messages across interstellar distances.

But, it’s one thing to calculate that intelligent aliens with access to microwave communication technology equal or better than our own could communicate over the void between the stars, and entirely another to listen for those communications.  The problems are simple to understand but forbidding to face: where do you point your antenna, and where do you tune your dial?  There are on the order of a hundred billion stars in our galaxy.  We now know, as early researchers suspected without evidence, that most of these stars have planets, some of which may have conditions suitable for the evolution of intelligent life.  Suppose aliens on one of these planets reach a level of technological development where they decide to join the “Galactic Club” and transmit a beacon which simply says “Yo!  Anybody out there?”  (The beacon would probably announce a signal with more information which would be easy to detect once you knew where to look.)  But for the beacon to work, it would have to be aimed at candidate stars where others might be listening (a beacon which broadcasted in all directions—an “omnidirectional beacon”—would require so much energy or be limited to such a short range as to be impractical for civilisations with technology comparable to our own).

Then there’s the question of how many technological communicating civilisations there are in the galaxy.  Note that it isn’t enough that a civilisation have the technology which enables it to establish a beacon: it has to do so.  And it is a sobering thought that more than six decades after we had the ability to send such a signal, we haven’t yet done so.  The galaxy may be full of civilisations with our level of technology and above which have the same funding priorities we do and choose to spend their research budget on intersectional autoethnography of transgender marine frobdobs rather than communicating with nerdy pocket-protector types around other stars who tediously ask Big Questions.

And suppose a civilisation decides it can find the spare change to set up and operate a beacon, inviting others to contact it.  How long will it continue to transmit, especially since it’s unlikely, given the finite speed of light and the vast distances between the stars, there will be a response in the near term?  Before long, scruffy professors will be marching in the streets wearing frobdob hats and rainbow tentacle capes, and funding will be called into question.  This is termed the “lifetime” of a communicating civilisation, or L, which is how long that civilisation transmits and listens to establish contact with others.  If you make plausible assumptions for the other parameters in the Drake equation (which estimates how many communicating civilisations there are in the galaxy), a numerical coincidence results in the estimate of the number of communicating civilisations in the galaxy being roughly equal to their communicating life in years, L.  So, if a typical civilisation is open to communication for, say, 10,000 years before it gives up and diverts its funds to frobdob research, there will be around 10,000 such civilisations in the galaxy.  With 100 billion stars (and around as many planets which may be hosts to life), that’s a 0.00001% chance that any given star where you point your antenna may be transmitting, and that has to be multiplied by the same probability they are transmitting their beacon in your direction while you happen to be listening.  It gets worse.  The galaxy is huge—around 150 million light years in diameter, and our technology can only communicate with comparable civilisations out to a tiny fraction of this, say 1000 light years for high-power omnidirectional beacons, maybe ten to a hundred times that for directed beacons, but then you have the constraint that you have to be listening in their direction when they happen to be sending.

It seems hopeless.  It may be.  But the 1960s were a time very different from our constrained age.  Back then, if you had a problem, like going to the Moon in eight years, you said, “Wow!  That’s a really big nail.  How big a hammer do I need to get the job done?”  Toward the end of that era when everything seemed possible, NASA convened a summer seminar at Stanford University to investigate what it would take to seriously investigate the question of whether we are alone.  The result was Project Cyclops: A Design Study of a System for Detecting Extraterrestrial Intelligent Life, prepared in 1971 and issued as a NASA report (no Library of Congress catalogue number or ISBN was assigned) in 1973; the link will take you to a NASA PDF scan of the original document, which is in the public domain.  The project assembled leading experts in all aspects of the technologies involved: antennas, receivers, signal processing and analysis, transmission and control, and system design and costing.

Project Cyclops: Ground level viewThey approached the problem from what might be called the “Apollo perspective”: what will it cost, given the technology we have in hand right now, to address this question and get an answer within a reasonable time?  What they came up with was breathtaking, although no more so than Apollo.  If you want to listen for beacons from communicating civilisations as distant as 1000 light years and incidental transmissions (“leakage”, like our own television and radar emissions) within 100 light years, you’re going to need a really big bucket to collect the signal, so they settled on 1000 dishes, each 100 metres in diameter.  Putting this into perspective, 100 metres is about the largest steerable dish anybody envisioned at the time, and they wanted to build a thousand of them, densely packed.

But wait, there’s more.  These 1000 dishes were not just a huge bucket for radio waves, but a phased array, where signals from all of the dishes (or a subset, used to observe multiple targets) were combined to provide the angular resolution of a single dish the size of the entire array.  This required breathtaking precision of electronic design at the time which is commonplace today (although an array of 1000 dishes spread over 16 km would still give most designers pause).  The signals that might be received would not be fixed in frequency, but would drift due to Doppler shifts resulting from relative motion of the transmitter and receiver.  With today’s computing hardware, digging such a signal out of the raw data is something you can do on a laptop or mobile phone, but in 1971 the best solution was an optical data processor involving exposing, developing, and scanning film.  It was exquisitely clever, although obsolete only a few years later, but recall the team had agreed to use only technologies which existed at the time of their design.  Even more amazing (and today, almost bizarre) was the scheme to use the array as an imaging telescope.  Again, with modern computers, this is a simple matter of programming, but in 1971 the designers envisioned a vast hall in which the signals from the antennas would be re-emitted by radio transmitters which would interfere in free space and produce an intensity image on an image surface where it would be measured by an array of receiver antennæ.

What would all of this cost?  Lots—depending upon the assumptions used in the design (the cost was mostly driven by the antenna specifications, where extending the search to shorter wavelengths could double the cost, since antennas had to be built to greater precision) total system capital cost was estimated as between 6 and 10 billion dollars (1971).  Converting this cost into 2018 dollars gives a cost between 37 and 61 billion dollars.  (By comparison, the Apollo project cost around 110 billion 2018 dollars.)  But since the search for a signal may “almost certainly take years, perhaps decades and possibly centuries”, that initial investment must be backed by a long-term funding commitment to continue the search, maintain the capital equipment, and upgrade it as technology matures.  Given governments’ record in sustaining long-term efforts in projects which do not line politicians’ or donors’ pockets with taxpayer funds, such perseverance is not the way to bet.  Perhaps participants in the study should have pondered how to incorporate sufficient opportunities for graft into the project, but even the early 1970s were still an idealistic time when we didn’t yet think that way.

This study is the founding document of much of the work in the Search for Extraterrestrial Intelligence (SETI) conducted in subsequent decades.  Many researchers first realised that answering this question, “Are we alone?”, was within our technological grasp when chewing through this difficult but inspiring document.  (If you have an equation or chart phobia, it’s not for you; they figure on the majority of pages.)  The study has held up very well over the decades.  There are a number of assumptions we might wish to revise today (for example, higher frequencies may be better for interstellar communication than were assumed at the time, and spread spectrum transmissions may be more energy efficient than the extreme narrowband beacons assumed in the Cyclops study).

Despite disposing of wealth, technological capability, and computing power of which authors of the Project Cyclops report never dreamed, we only make little plans today.  Most readers of this post, in their lifetimes, have experienced the expansion of their access to knowledge in the transition from being isolated to gaining connectivity to a global, high-bandwidth network.  Imagine what it means to make the step from being confined to our single planet of origin to being plugged in to the Galactic Web, exchanging what we’ve learned with a multitude of others looking at things from entirely different perspectives.  Heck, you could retire the entire capital and operating cost of Project Cyclops in the first three years just from advertising revenue on frobdob videos!  (Did I mention they have very large eyes which are almost all pupil?  Never mind the tentacles.)

Oliver, Bernard M., John Billingham, et al.  Project Cyclops [PDF].  Stanford, CA: Stanford/NASA Ames Research Center, 1971.  NASA-CR-114445 N73-18822.

This document has been subjected to intense scrutiny over the years.  The SETI League maintains a comprehensive errata list for the publication.

Here is a recent conversation among SETI researchers on the state of the art and future prospects for SETI with ground-based telescopes.

This is a two part lecture on the philosophy of the existence and search for extraterrestrial beings from antiquity to the present day.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Orbits in Strongly Curved Spacetime

Your browser does not support HTML5 canvas.
Angular momentum: Mass: Max radius:
    Click the title of this post to see the interactive simulation.


The display above shows, from three different physical perspectives, the orbit of a low-mass test particle, the small red circle, around a non-rotating black hole (represented by a grey circle in the panel at the right), where the radius of the circle is the black hole’s gravitational radius, or event horizon. Kepler’s laws of planetary motion, grounded in Newton’s theory of gravity, state that the orbit of a test particle around a massive object is an ellipse with one focus at the centre of the massive object. But when gravitational fields are strong, as is the case for collapsed objects like neutron stars and black holes, Newton’s theory is inaccurate; calculations must be done using Einstein’s theory of General Relativity.

In Newtonian gravitation, an orbit is always an ellipse. As the gravitating body becomes more massive and the test particle orbits it more closely, the speed of the particle in its orbit increases without bound, always balancing the gravitational force. For a black hole, Newton’s theory predicts orbital velocities greater than the speed of light, but according to Einstein’s Special Theory of Relativity, no material object can achieve or exceed the speed of light. In strong gravitational fields, General Relativity predicts orbits drastically different from the ellipses of Kepler’s laws. This article allows you to explore them.

The Orbit Plot

Orbit PlotThe panel at the right of the animation shows the test mass orbiting the black hole, viewed perpendicular to the plane of its orbit. The path of the orbit is traced by the green line. After a large number of orbits the display will get cluttered; just click the mouse anywhere in the right panel to erase the path and start over. When the test mass reaches its greatest distance from the black hole, a yellow line is plotted from the centre of the black hole to that point, the apastron of the orbit. In Newtonian gravity, the apastron remains fixed in space. The effects of General Relativity cause it to precess. You can see the degree of precession in the displacement of successive yellow lines (precession can be more than 360°; the yellow line only shows precession modulo one revolution).

The Gravitational Effective-Potential

Effective potentialThe two panels at the left of the animation display the orbit in more abstract ways. The Effective Potential plot at the top shows the position of the test mass on the gravitational energy curve as it orbits in and out. The summit on the left side of the curve is unique to General Relativity—in Newtonian gravitation the curve rises without bound as the radius decreases, approaching infinity at zero. In Einstein’s theory, the inability of the particle to orbit at or above the speed of light creates a “pit in the potential” near the black hole. As the test mass approaches this summit, falling in from larger radii with greater and greater velocity, it will linger near the energy peak for an increasingly long time, while its continued angular motion will result in more and more precession. If the particle passes the energy peak and continues to lesser radii, toward the left, its fate is sealed—it will fall into the black hole and be captured.

The Gravity Well

Gravity wellSpacetime around an isolated spherical non-rotating uncharged gravitating body is described by Schwarzschild Geometry, in which spacetime can be thought of as being bent by the presence of mass. This creates a gravity well which extends to the surface of the body or, in the case of a black hole, to oblivion. The gravity well has the shape of a four-dimensional paraboloid of revolution, symmetrical about the central mass. Since few Web browsers are presently equipped with four-dimensional display capability, I’ve presented a two-dimensional slice through the gravity well in the panel at the bottom left of the animation. Like the energy plot above, the left side of the panel represents the centre of the black hole and the radius increases to the right. Notice that the test mass radius moves in lockstep on the Effective-Potential and Gravity Well charts, as the radius varies on the orbit plot to their right.

The gravity well of a Schwarzschild black hole has a throat at a radius determined solely by its mass—that is the location of the hole’s event horizon; any matter or energy which crosses the horizon is captured. The throat is the leftmost point on the gravity well curve, where the slope of the paraboloidal geometry becomes infinite (vertical). With sufficient angular momentum, a particle can approach the event horizon as closely as it wishes (assuming it is small enough so it isn’t torn apart by tidal forces), but it can never cross the event horizon and return.

Hands On

Orbits in Strongly Curved Spacetime control panel

By clicking in the various windows and changing values in the controls at the bottom of the window you can explore different scenarios. To pause the simulation, press the Pause button; pressing it again resumes the simulation. Click anywhere in the orbit plot at the right to clear the orbital trail and apastron markers when the screen becomes too cluttered. You can re-launch the test particle at any given radius from the black hole (with the same angular momentum) by clicking at the desired radius in either the Effective Potential or Gravity Well windows. The green line in the Effective Potential plot indicates the energy minimum at which a stable circular orbit exists for a particle of the given angular momentum.

The angular momentum is specified by the box at left in terms of the angular momentum per unit mass of the black hole, all in geometric units—all of this is explained in detail below. What’s important to note is that for orbits like those of planets in the Solar System, this number is huge; only in strong gravitational fields does it approach small values. If the angular momentum is smaller than a critical value (\(2\sqrt 3\), about 3.464 for a black hole of mass 1, measured in the same units), no stable orbits exist; the particle lacks the angular momentum to avoid being swallowed. When you enter a value smaller than this, notice how the trough in the energy curve and the green line marking the stable circular orbit disappear. Regardless of the radius, any particle you launch is doomed to fall into the hole.

The Mass box allows you to change the mass of the black hole, increasing the radius of its event horizon. Since the shape of the orbit is determined by the ratio of the angular momentum to the mass, it’s just as easy to leave the mass as 1 and change the angular momentum. You can change the scale of all the panels by entering a new value for the maximum radius; this value becomes the rightmost point in the effective potential and gravity well plots and the distance from the centre of the black hole to the edge of the orbit plot. When you change the angular momentum or mass, the radius scale is automatically adjusted so the stable circular orbit (if any) is on screen.

Kepler, Newton, and Beyond

In the early 17th century, after years of tedious calculation and false starts, Johannes Kepler published his three laws of planetary motion:

  • First law (1605): A planet’s orbit about the Sun is an ellipse, with the Sun at one focus.
  • Second law (1604): A line from the Sun to a planet sweeps out equal areas in equal times.
  • Third law (1618): The square of the orbital period of a planet is proportional to the cube of the major axis of the orbit.

Kepler’s discoveries about the behaviour of planets in their orbits played an essential rôle in Isaac Newton’s formulation of the law of universal gravitation in 1687. Newton’s theory showed the celestial bodies were governed by the same laws as objects on Earth. The philosophical implications of this played as key a part in the Enlightenment as did the theory itself in the subsequent development of physics and astronomy.

While Kepler’s laws applied only to the Sun and planets, Newton’s universal theory allowed one to calculate the gravitational force and motion of any bodies whatsoever. To be sure, when many bodies were involved and great accuracy was required, the calculations were horrifically complicated and tedious—so much so that those reared in the computer age may find it difficult to imagine embarking upon them armed with nothing but a table of logarithms, pencil and paper, and the human mind. But performed they were, with ever greater precision as astronomers made increasingly accurate observations. And those observations agreed perfectly with the predictions of Newton’s theory.

Well,… almost perfectly. After painstaking observations of the planets and extensive calculation, astronomer Simon Newcomb concluded in 1898 that the orbit of Mercury was precessing 43 arc-seconds per century more than could be explained by the influence of the other planets. This is a tiny discrepancy, but further observations and calculations confirmed Newcomb’s—the discrepancy was real. Some suggested a still undiscovered planet closer to the Sun than Mercury (and went so far as to name it, sight unseen, “Vulcan”), but no such planet was ever found, nor any other plausible explanation advanced. For nearly twenty years Mercury’s precession or “perihelion advance” remained one of those nagging anomalies in the body of scientific data that’s trying to tell us something, if only we knew what.

In 1915, Albert Einstein’s General Theory of Relativity extended Newtonian gravitation theory, revealing previously unanticipated subtleties of nature. And Einstein’s theory explained the perihelion advance of Mercury. That tiny discrepancy in the orbit of Mercury was actually the first evidence for what lay beyond Newtonian gravitation, the first step down a road that would lead to understanding black holes, gravitational radiation, and the source of inertia, which remains a fertile ground for theoretical and experimental physics a century thereafter.

If we’re interested in the domain where general relativistic effects are substantial, we’re better off calculating with units scaled to the problem. A particularly convenient and elegant choice is the system of geometric units, obtained by setting Newton’s gravitational constant G, the speed of light c, and Boltzmann’s constant k all equal to 1. We can then express any of the following units as a length in centimetres by multiplying by the following conversion factors.

Geometric units

The enormous exponents make it evident that these units are far removed from our everyday experience. It would be absurd to tell somebody, “I’ll call you back in \(1.08\times 10^{14}\) centimetres”, but it is a perfectly valid way of saying “one hour”. The discussion that follows uses geometric units throughout, allowing us to treat mass, time, length, and energy without conversion factors. To express a value calculated in geometric units back to conventional units, just divide by the value in the table above.

The Gravitational Effective-Potential

Effective potential

The gravitational effective-potential for a test particle orbiting in a Schwarzschild geometry is:


where \(\tilde{L}\) is the angular momentum per unit rest mass expressed in geometric units, M is the mass of the gravitating body, and r is the radius of the test particle from the centre of the body.

The radius of a particle from the centre of attraction evolves in proper time τ (time measured by a clock moving along with the particle) according to:

(dr/dTAU)² + V(L,r) = E²

where \(\tilde{E}\) is the potential energy of the test mass at infinity per rest mass.

Angular motion about the centre of attraction is then:

dPHI/dTAU = L/r²

while time, as measured by a distant observer advances according to:

dt/dTAU = E / (1 - 2M/r)

and can be seen to slow down as the event horizon at the gravitational radius is approached. At the gravitational radius of 2M time, as measured from far away, stops entirely so the particle never seems to reach the event horizon. Proper time on the particle continues to advance unabated; an observer on-board sails through the event horizon without a bump (or maybe not) and continues toward the doom which awaits at the central singularity.

Circular Orbits

Circular orbits are possible at maxima and minima of the effective-potential. Orbits at minima are stable, since a small displacement increases the energy and thus creates a restoring force in the opposite direction. Orbits at maxima are unstable; the slightest displacement causes the particle to either be sucked into the black hole or enter a highly elliptical orbit around it.

To find the radius of possible circular orbits, differentiate the gravitational effective-potential with respect to the radius r:

DV²/dr = (2 (3L²M - L²r + Mr²)) / (r^4)

The minima and maxima of a function are at the zero crossings of its derivative, so a little algebra gives the radii of possible circular orbits as:

(L(L ± sqrt(L² - 12M²)) / 2M

The larger of these solutions is the innermost stable circular orbit, while the smaller is the unstable orbit at the maximum. For a black hole, this radius will be outside the gravitational radius at 2M, while for any other object the radius will be less than the diameter of the body, indicating no such orbit exists. If the angular momentum L² is less than 12M², no stable orbit exists; the object will impact the surface or, in the case of a black hole, fall past the event horizon and be swallowed.


Gallmeier, Jonathan, Mark Loewe, and Donald W. Olson. “Precession and the Pulsar.” Sky & Telescope (September 1995): 86–88.
A BASIC program which plots orbital paths in Schwarzschild geometry. The program uses different parameters to describe the orbit than those used here, and the program does not simulate orbits which result in capture or escape. This program can be downloaded from the Sky & Telescope Web site.
Misner, Charles W., Kip S. Thorne, and John Archibald Wheeler. Gravitation. San Francisco: W. H. Freeman, 1973. ISBN 978-0-7167-0334-1.
Chapter 25 thoroughly covers all aspects of motion in Schwarzschild geometry, both for test particles with mass and massless particles such as photons.
Wheeler, John Archibald. A Journey into Gravity and Spacetime. New York: W. H. Freeman, 1990. ISBN 978-0-7167-5016-1.
This book, part of the Scientific American Library series (but available separately), devotes chapter 10 to a less technical discussion of orbits in Schwarzschild spacetime. The “energy hill” on page 173 and the orbits plotted on page 176 provided the inspiration for this page.

Here is a short video about orbiting a black hole:

This is a 45 minute lecture on black holes and the effects they produce.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Antifragile

“Antifragile” by Nassim Nicholas TalebThis book is volume three in the author’s Incerto series, following Fooled by Randomness  and The Black Swan. It continues to explore the themes of randomness, risk, and the design of systems: physical, economic, financial, and social, which perform well in the face of uncertainty and infrequent events with large consequences. He begins by posing the deceptively simple question, “What is the antonym of ‘fragile’?”

After thinking for a few moments, most people will answer with “robust” or one of its synonyms such as “sturdy”, “tough”, or “rugged”. But think about it a bit more: does a robust object or system actually behave in the opposite way to a fragile one? Consider a teacup made of fine china. It is fragile—if subjected to more than a very limited amount of force or acceleration, it will smash into bits. It is fragile because application of such an external stimulus, for example by dropping it on the floor, will dramatically degrade its value for the purposes for which it was created (you can’t drink tea from a handful of sherds, and they don’t look good sitting on the shelf). Now consider a teacup made of stainless steel. It is far more robust: you can drop it from ten kilometres onto a concrete slab and, while it may be slightly dented, it will still work fine and look OK, maybe even acquiring a little character from the adventure. But is this really the opposite of fragility? The china teacup was degraded by the impact, while the stainless steel one was not. But are there objects and systems which improve as a result of random events: uncertainty, risk, stressors, volatility, adventure, and the slings and arrows of existence in the real world? Such a system would not be robust, but would be genuinely “anti-fragile” (which I will subsequently write without the hyphen, as does the author): it welcomes these perturbations, and may even require them in order to function well or at all.

Antifragility seems an odd concept at first. Our experience is that unexpected events usually make things worse, and that the inexorable increase in entropy causes things to degrade with time: plants and animals age and eventually die; machines wear out and break; cultures and societies become decadent, corrupt, and eventually collapse. And yet if you look at nature, antifragility is everywhere—it is the mechanism which drives biological evolution, technological progress, the unreasonable effectiveness of free market systems in efficiently meeting the needs of their participants, and just about everything else that changes over time, from trends in art, literature, and music, to political systems, and human cultures. In fact, antifragility is a property of most natural, organic systems, while fragility (or at best, some degree of robustness) tends to characterise those which were designed from the top down by humans. And one of the paradoxical characteristics of antifragile systems is that they tend to be made up of fragile components.

How does this work? We’ll get to physical systems and finance in a while, but let’s start out with restaurants. Any reasonably large city in the developed world will have a wide variety of restaurants serving food from numerous cultures, at different price points, and with ambience catering to the preferences of their individual clientèles. The restaurant business is notoriously fragile: the culinary preferences of people are fickle and unpredictable, and restaurants who are behind the times frequently go under. And yet, among the population of restaurants in a given area at a given time, customers can usually find what they’re looking for. The restaurant population or industry is antifragile, even though it is composed of fragile individual restaurants which come and go with the whims of diners, which will be catered to by one or more among the current, but ever-changing population of restaurants.

Now, suppose instead that some Food Commissar in the All-Union Ministry of Nutrition carefully studied the preferences of people and established a highly-optimised and uniform menu for the monopoly State Feeding Centres, then set up a central purchasing, processing, and distribution infrastructure to optimise the efficient delivery of these items to patrons. This system would be highly fragile, since while it would deliver food, there would be no feedback based upon customer preferences, and no competition to respond to shifts in taste. The result would be a mediocre product which, over time, was less and less aligned with what people wanted, and hence would have a declining number of customers. The messy and chaotic market of independent restaurants, constantly popping into existence and disappearing like virtual particles, exploring the culinary state space almost at random, does, at any given moment, satisfy the needs of its customers, and it responds to unexpected changes by adapting to them: it is antifragile.

Now let’s consider an example from metallurgy. If you pour molten metal from a furnace into a cold mould, its molecules, which were originally jostling around at random at the high temperature of the liquid metal, will rapidly freeze into a structure with small crystals randomly oriented. The solidified metal will contain dislocations wherever two crystals meet, with each forming a weak spot where the metal can potentially fracture under stress. The metal is hard, but brittle: if you try to bend it, it’s likely to snap. It is fragile.

To render it more flexible, it can be subjected to the process of annealing, where it is heated to a high temperature (but below melting), which allows the molecules to migrate within the bulk of the material. Existing grains will tend to grow, align, and merge, resulting in a ductile, workable metal. But critically, once heated, the metal must be cooled on a schedule which provides sufficient randomness (molecular motion from heat) to allow the process of alignment to continue, but not to disrupt already-aligned crystals. Here is a video from Cellular Automata Laboratory which demonstrates annealing. Note how sustained randomness is necessary to keep the process from quickly “freezing up” into a disordered state.

As it happens, last month’s Saturday Night Science discussed solving the travelling salesman problem through the technique of simulated annealing, which is analogous to annealing metal, and like it, is a manifestation of antifragility—it doesn’t work without randomness.

When you observe a system which adapts and prospers in the face of unpredictable changes, it will almost always do so because it is antifragile. This is a large part of how nature works: evolution isn’t able to predict the future and it doesn’t even try. Instead, it performs a massively parallel, planetary-scale search, where organisms, species, and entire categories of life appear and disappear continuously, but with the ecosystem as a whole constantly adapting itself to whatever inputs may perturb it, be they a wholesale change in the composition of the atmosphere (the oxygen catastrophe at the beginning of the Proterozoic eon around 2.45 billion years ago), asteroid and comet impacts, and ice ages.

Most human-designed systems, whether machines, buildings, political institutions, or financial instruments, are the antithesis of those found in nature. They tend to be highly-optimised to accomplish their goals with the minimum resources, and to be sufficiently robust to cope with any stresses they may be expected to encounter over their design life. These systems are not antifragile: while they may be designed not to break in the face of unexpected events, they will, at best, survive, but not, like nature, often benefit from them.

The devil’s in the details, and if you reread the last paragraph carefully, you may be able to see the horns and pointed tail peeking out from behind the phrase “be expected to”. The problem with the future is that it is full of all kinds of events, some of which are un-expected, and whose consequences cannot be calculated in advance and aren’t known until they happen. Further, there’s usually no way to estimate their probability. It doesn’t even make any sense to talk about the probability of something you haven’t imagined could happen. And yet such things happen all the time.

Today, we are plagued, in many parts of society, with “experts” the author dubs fragilistas. Often equipped with impeccable academic credentials and with powerful mathematical methods at their fingertips, afflicted by the “Soviet-Harvard delusion” (overestimating the scope of scientific knowledge and the applicability of their modelling tools to the real world), they are blind to the unknown and unpredictable, and they design and build systems which are highly fragile in the face of such events. A characteristic of fragilista-designed systems is that they produce small, visible, and apparently predictable benefits, while incurring invisible risks which may be catastrophic and occur at any time.

Let’s consider an example from finance. Suppose you’re a conservative investor interested in generating income from your lifetime’s savings, while preserving capital to pass on to your children. You might choose to invest, say, in a diversified portfolio of stocks of long-established companies in stable industries which have paid dividends for 50 years or more, never skipping or reducing a dividend payment. Since you’ve split your investment across multiple companies, industry sectors, and geographical regions, your risk from an event affecting one of them is reduced. For years, this strategy produces a reliable and slowly growing income stream, while appreciation of the stock portfolio (albeit less than high flyers and growth stocks, which have greater risk and pay small dividends or none at all) keeps you ahead of inflation. You sleep well at night.

Then 2008 rolls around. You didn’t do anything wrong. The companies in which you invested didn’t do anything wrong. But the fragilistas had been quietly building enormous cross-coupled risk into the foundations of the financial system (while pocketing huge salaries and bonuses, while bearing none of the risk themselves), and when it all blows up, in one sickening swoon, you find the value of your portfolio has been cut by 50%. In a couple of months, you have lost half of what you worked for all of your life. Your “safe, conservative, and boring” stock portfolio happened to be correlated with all of the other assets, and when the foundation of the system started to crumble, suffered along with them. The black swan landed on your placid little pond.

What would an antifragile investment portfolio look like, and how would it behave in such circumstances? First, let’s briefly consider a financial option. An option is a financial derivative contract which gives the purchaser the right, but not the obligation, to buy (“call option”) or sell (”put option”) an underlying security (stock, bond, market index, etc.) at a specified price, called the “strike price” (or just “strike”). If the a call option has a strike above, or a put option a strike below, the current price of the security, it is called “out of the money”, otherwise it is “in the money”. The option has an expiration date, after which, if not “exercised” (the buyer asserts his right to buy or sell), the contract expires and the option becomes worthless.

Let’s consider a simple case. Suppose Consolidated Engine Sludge (SLUJ) is trading for US$10 per share on June 1, and I buy a call option to buy 100 shares at US$15/share at any time until August 31. For this right, I might pay a premium of, say, US$7. (The premium depends upon sellers’ perception of the volatility of the stock, the term of the option, and the difference between the current price and the strike price.) Now, suppose that sometime in August, SLUJ announces a breakthrough that allows them to convert engine sludge into fructose sweetener, and their stock price soars on the news to US$19/share. I might then decide to sell on the news, exercise the option, paying US$1500 for the 100 shares, and then immediately sell them at US$19, realising a profit of US$400 on the shares or, subtracting the cost of the option, US$393 on the trade. Since my original investment was just US$7, this represents a return of 5614% on the original investment, or 22457% annualised. If SLUJ never touches US$15/share, come August 31, the option will expire unexercised, and I’m out the seven bucks. (Since options can be bought and sold at any time and prices are set by the market, it’s actually a bit more complicated than that, but this will do for understanding what follows.)

You might ask yourself what would motivate somebody to sell such an option. In many cases, it’s an attractive proposition. If I’m a long-term shareholder of SLUJ and have found it to be a solid but non-volatile stock that pays a reasonable dividend of, say, two cents per share every quarter, by selling the call option with a strike of 15, I pocket an immediate premium of seven cents per share, increasing my income from owning the stock by a factor of 4.5. For this, I give up the right to any appreciation should the stock rise above 15, but that seems to be a worthwhile trade-off for a stock as boring as SLUJ (at least prior to the news flash).

A put option is the mirror image: if I bought a put on SLUJ with a strike of 5, I’ll only make money if the stock falls below 5 before the option expires.

Now we’re ready to construct a genuinely antifragile investment. Suppose I simultaneously buy out of the money put and call options on the same security, a so-called “long straddle”. Now, as long as the price remains within the strike prices of the put and call, both options will expire worthless, but if the price either rises above the call strike or falls below the put strike, that option will be in the money and pay off the further the underlying price veers from the band defined by the two strikes. This is, then, a pure bet on volatility: it loses a small amount of money as long as nothing unexpected happens, but when a shock occurs, it pays off handsomely.

Now, the premiums on deep out of the money options are usually very modest, so an investor with a portfolio like the one I described who was clobbered in 2008 could have, for a small sum every quarter, purchased put and call options on, say, the Standard & Poor’s 500 stock index, expecting to usually have them expire worthless, but under the circumstance which halved the value of his portfolio, would pay off enough to compensate for the shock. (If worried only about a plunge he could, of course, have bought just the put option and saved money on premiums, but here I’m describing a pure example of antifragility being used to cancel fragility.)

I have only described a small fraction of the many topics covered in this masterpiece, and described none of the mathematical foundations it presents (which can be skipped by readers intimidated by equations and graphs). Fragility and antifragility is one of those concepts, simple once understood, which profoundly change the way you look at a multitude of things in the world. When a politician, economist, business leader, cultural critic, or any other supposed thinker or expert advocates a policy, you’ll learn to ask yourself, “Does this increase fragility?” and have the tools to answer the question. Further, it provides an intellectual framework to support many of the ideas and policies which libertarians and advocates of individual liberty and free markets instinctively endorse, founded in the way natural systems work. It is particularly useful in demolishing “green” schemes which aim at replacing the organic, distributed, adaptive, and antifragile mechanisms of the market with coercive, top-down, and highly fragile central planning which cannot possibly have sufficient information to work even in the absence of unknowns in the future.

There is much to digest here, and the ramifications of some of the clearly-stated principles take some time to work out and fully appreciate. Indeed, I spent more than five years reading this book, a little bit at a time. It’s worth taking the time and making the effort to let the message sink in and figure out how what you’ve learned applies to your own life and act accordingly. As Fat Tony says, “Suckers try to win arguments; nonsuckers try to win.”

Taleb, Nassim Nicholas. Antifragile. New York: Random House, 2012. ISBN 978-0-8129-7968-8.

Here is a lecture by the author about the principles discussed in the book.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Simulated Annealing

Your browser does not support HTML5 canvas.
path length   River cost

Travelling Salesman

(Click Read More to view the interactive simulation in this post.)
The travelling salesman problem is one of the most-studied problems in combinatorial optimisation. It couldn’t be easier to state:

Given a list of cities and their locations (usually specified as Cartesian co-ordinates on a plane), what is the shortest itinerary which will visit every city exactly once and return to the point of origin?

Easy to ask, but devilishly difficult to answer…. The obvious way to solve the travelling salesman problem would be to write down all of the possible sequences in which the cities could be visited, compute the distance of each path, and then choose the smallest. But the number of possible itineraries for visiting n cities grows as the factorial of n, which is written, appropriately as “n!”. The factorial of a positive integer is the product of that number and all smaller numbers down to one. Hence 2!=2, 3!=6, 6!=720, and 10!=3,628,800. As you can see, these numbers grow very rapidly, so as you increase the number of cities, the number of paths you have to compare blows up in a combinatorial explosion which makes finding the optimal path by brute force computation a hopeless undertaking.

“But”, you ask, “computers are getting faster every year. Why not just be patient and wait a few years?” Neither you, nor I, nor the universe has sufficient patience. The box at the top of this page contains thirty cities represented by red balls placed at random in the grey square, connected by a path drawn in blue lines in the order in which they were placed. Every time you press the “Place” button, thirty new randomly-placed cities are generated; you can change the number by setting the box to the right of the button. But let’s stick with thirty cities for the nonce.

The number of possible paths along which we can visit the thirty cities is equal to the number of permutations of a set of thirty distinct members, which is equal to the factorial of the number of members, or 30!. This is a breathtakingly large number.

30! = 265,252,859,812,191,058,636,308,480,000,000 ≈ 2.6525×1032

Now, let’s assume you had a supercomputer which was able to compute the value of a billion (109) paths per second. Chugging away at this task around the clock, without a day of rest, it would take 2.65×1023 seconds to get through the list. How long is that? About 8.4 quadrillion (1015) years, or about 600,000 times the present age of the universe. And if you modestly increased the number of cities to fifty? Prepare to wait eight thousand billion billion times the age of the universe for the answer.

Solution for 30 citiesNow scroll back up to the top of the page and click the “Solve” button Almost instantaneously, you’ll see a near-optimal path to tour the thirty cities with the least distance of travel. Try clicking “Place” and then “Solve” several times to create and solve new problems, then increase the number of cities to 50 and then 100 and try solving those problems. In each case, the solution appears in a fraction of a second. Now, these solutions are not guaranteed to be absolutely optimal; they may be a percent or two longer than the absolute best path (if you click “Solve” multiple times, you may see several different solutions, all of which are close in total path length). They’re not perfect, but then you don’t have to wait huge multiples of the age of the universe for the result. How did we do it?

Simulated Annealing

This page attacks the travelling salesman problem through a technique of combinatorial optimisation called simulated annealing. By analogy with the process of annealing a material such as metal or glass by raising it to a high temperature and then gradually reducing the temperature, allowing local regions of order to grow outward, increasing ductility and reducing stresses in the material, the algorithm randomly perturbs the original path to a decreasing extent according to a gradually decreasing logical “temperature”.

In simulated annealing, the equivalent of temperature is a measure of the randomness by which changes are made to the path, seeking to minimise it. When the temperature is high, larger random changes are made, avoiding the risk of becoming trapped in a local minimum (of which there are usually many in a typical travelling salesman problem), then homing in on a near-optimal minimum as the temperature falls. The temperature falls in a series of steps on an exponential decay schedule where, on each step, the temperature is 0.9 times that of the previous step.

The process of annealing starts with a path which simply lists all of the cities in the order their positions were randomly selected (this is the path you’ll see after pressing the “Place” button). On each temperature step, a number of random transformations of the path are made. First of all, a segment of the path is selected, with its start and end cities chosen at random. Then, a software coin is flipped to decide which kind of transformation to try: reverse or transport.

If reverse comes up, an alternative path is generated in which the cities in the chosen segment are reversed in order of visit. If transport, the segment is clipped out of its current position in the path and spliced in at a randomly chosen point in the remainder of the path. The length of the modified path is then calculated and compared to the path before modification, producing a quantity called the cost difference. If negative, the modified path is shorter than the original path and always replaces it. If there is an increase in cost, however, the exponential of its negative magnitude divided by the current temperature is compared to a uniformly distributed random number between 0 and 1 and, if greater, the modified path will be used even though it increased the cost. Note that initially, when the temperature is high, there will be a greater probability of making such changes, but that as the temperature falls, only smaller increases in cost will be accepted. The total number of changes tested at each temperature level is arbitrarily set to 100 times the number of cities in the path, and after ten times the number of changes which decrease the path length as the number of cities are found, the temperature is decreased and the search continued. If, after trying all of the potential changes at a given temperature level, no changes are found which reduce the path length, the solution is considered “good enough” and the resulting path is displayed.

Watching it Happen

To watch the optimisation process as it unfolds, instead of pressing the “Solve” button, press the “Step” button to see the path evolve at each level of decreasing temperature. The “Animate” button will automatically show the path evolving at one second per temperature level. Check the “Trace solution” box to display the temperature, cost (path length), and number of changes made to the path at each step in the optimisation. After a solution is found, the chosen itinerary will be shown listing the cities in order, their co-ordinates, and the cost of the path from each city to the next (wrapping around at the bottom) and, if the path crosses the river (see below), an “R” to indicate that it does.

Instead of using the “Place” button to randomly place cities, you can place them manually by pressing “New” to clear the map and then click the mouse in the map to indicate the city locations. They will initially be connected by paths in the order you placed the cities. You can also add cities to maps created by the “Place” button by clicking in the map.

Minimise or Maximise?

30 cities, maximise path lengthThe travelling salesman problem is usually formulated in terms of minimising the path length to visit all of the cities, but the process of simulated annealing works just as well with a goal of maximising the length of the itinerary. If you change the goal in the drop-down list from “Minimise” to “Maximise”, the cost function being optimised will be the negative of the path length, resulting in a search for the longest path. Try it, and see how the annealing process finds a star-like pattern that chooses the longest inter-city paths.

A River Runs through It

30 cities, river cost 50%We can add another wrinkle to the cost function by adding a “river” that runs through the map from top to bottom halfway across it. If you set the “River cost” nonzero, the river will be drawn as a dark blue line, and any path from one city to another which crosses it is assessed a penalty given by the river cost as a percentage of the size of the map. If you set the river cost high, say to 50%, you’ll find a strong preference for paths which only cross the river twice, finding a near-minimum path length independently on each side of the river. (This may be more apparent if you place a large number of cities, say 100 or 250.)

100 cities, river cost −25%You can also set the cost of crossing the river negative, which turns the travelling salesman into a peripatetic smuggler who profits from carrying goods between Freedonia and Grand Fenwick. Try placing 100 cities and setting the river cost to −25: the smuggler will settle on an efficient path on each side of the river, but prefer river crossings between cities close to the river where the benefit of the crossing is significant compared to the distance between them.

100 cities, river cost −100%, maximise costFinally, try setting the goal to Maximise path length, the river crossing cost to −100 (benefit from crossing the river), and place 100 cities. When you solve, you’ll find the solution produces two star-like patterns on each side of the river which maximises the travel distance on each side, but avoids river crossings at all costs.

Other Optimisation Techniques

Here we’ve explored one technique of combinatorial optimisation: simulated annealing. This is only one of the many approaches which have been taken to problems of this kind. These include exact approaches such as branch and boundlinear programming, and cutting-plane algorithms.

There are many approximation techniques which find near-optimal solutions, of which simulated annealing is but one. One algorithm even models how ants find the shortest path between their anthill and a source of food.


Press, William H., Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. Numerical Recipes in C, 2nd ed. Cambridge: Cambridge University Press, [1988] 1992. ISBN 978-0-521-43108-8. Section 10.9, pp. 444–451. (A third edition of this book with algorithms in C++ is available.)


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Hot Shots: A Radioactive Lens

Leica M6 with Summicron 50 mm f/2 lens

Between the 1940s and 1970s, a number of camera manufacturers designed lenses employing thoriated glass in one or more elements. Incorporating as much as 40% thorium dioxide (ThO2) in the glass mixture increases the index of refraction of the glass while maintaining low dispersion. Thoriated glass elements allowed lenses to deliver low levels of aberration and distortion with relatively simple and easy to manufacture designs.

As with everything in engineering, there are trade-offs. Thorium is a radioactive element; it has no stable isotopes. Natural thorium consists of 99.98% thorium-232, which has a half-life of 1.4×1010 years. While this is a long half-life, more than three times that of uranium-238, it is still substantially radioactive and easily detected with a Geiger-Müller counter. Thorium decays by alpha emission into radium-228, which continues to decay through the thorium series into various nuclides, eventually arriving at stable lead-208.

Leica Summicron 50 mm f/2 lensAttached to my Leica M6 film camera above is a Leica Summicron 50 mm f/2 lens which contains thoriated glass. Its serial number, 1041925, indicates its year of manufacture as 1952. This lens was a screw mount design, but can be used on more recent bayonet mount Leica cameras with a simple adapter. Like many early Leica lenses, it is collapsible: you can rotate the front element and push the barrel back into the camera body when not in use, making the camera more compact to pack and carry. Although 66 years old at this writing, the lens performs superbly, although not as well as current Leica lenses which are, however, more than an order of magnitude more expensive.

To measure the radiation emitted by this thoriated glass lens I used a QuartaRAD RADEX RD1706 Geiger-Müller counter and began by measuring the background radiation in my office.

Radiation monitor: 0.12 μSv/h

This came in (averaged over several measurement periods) as 0.12 microsieverts (μSv) per hour, what I typically see. Background radiation varies slightly over the day (I know not why), and this was near the low point of the cycle.

I then placed the detector directly before the front element of the lens, still mounted on the camera. The RADEX RD1706 has two Geiger tubes, one on each side of the meter. I positioned the meter so its left tube would be as close as possible to the front element.

Radiation monitor: 1.14 μSv/h

After allowing the reading to stabilise and time average, I measured radiation flux around 1.14 μSv/h, nearly ten times background radiation. Many lenses using thoriated glass employed it only for the front element(s), with regular crown or flint glass at the rear. This limits radiation which might, over time, fog the film in the camera. With such lenses, you can easily detect the radiation from the front element, but little is emitted backward in the direction of the film (and the photographer). This is not the case with this lens, however. I removed the lens from the camera, collapsed it so the back element would be closer to the detector (about as far as the front element was in the previous measurement) and repeated the test.

Radiation monitor: 1.51 μSv/h

This time I saw 1.51 μSv/h, more than twelve times background radiation. What were they thinking? First of all the most commonly used films in the early 1950s were slower (less sensitive) than modern emulsions, and consequently less prone to fogging due to radiation. Second, all Leica rangefinder cameras use a focal-plane shutter, which means the film behind the lens is shielded from the radiation it emits except for the instant the shutter is open when making an exposure, which would produce negligible fogging. Since the decay chain of thorium consists exclusively of alpha and beta particle emission, neither of which is very penetrating, the closed shutter protects the film from the radiation from the rear of the lens.

Many camera manufacturers used thoriated lenses. Kodak even used thoriated glass in its top of the line 800 series Instamatic cameras, and Kodak Aero-Ektar lenses, manufactured in great quantity during World War II for aerial reconnaissance, are famously radioactive. After 1970, thoriated glass ceased to be used in optics, both out of concern over radiation, but also due to a phenomenon which caused the thoriated glass to deteriorate over time. Decaying thorium atoms create defects in the glass called F-centres which, as they accumulated, would cause the glass to acquire a yellowish or brownish tint. This wasn’t much of a problem with black and white film, but it would cause a shift in the colour balance which was particularly serious for the colour reversal (transparency) film favoured by professional photographers in many markets. (My 1952 vintage lens has a slight uniform yellow cast to it—much lighter than a yellow filter. It’s easy to correct for in digital image post-processing.) Annealing the glass by exposing it to intense ultraviolet light (I’ve heard that several days in direct sunlight will do the job) can reduce or eliminate the yellowing.

Thorium glass was replaced by glass containing lanthanum oxide (La2O3), which has similar optical properties. Amusingly, lanthanum is itself very slightly radioactive: while the most common isotope, lanthanum-139, which makes up 99.911% of natural lanthanum, is stable, 0.089% is the lanthanum-138 isotope, which has a half-life of 1011 years, about ten times that of thorium. Given the tiny fraction of the radioisotope and its long half-life, the radiation from lanthanum glass (about 1/10000 that of thorium glass), while detectable with a sensitive counter, is negligible compared to background radiation.

If you have one of these lenses, should you be worried? In a word, no. The radiation from the lens is absorbed by the air, so that just a few centimetres away you’ll measure nothing much above background radiation. To receive a significant dose of radiation, you’d have to hold the front element of the lens up against your skin for an extended period of time, and why would you do that? Even if you did, the predominantly alpha radiation is blocked by human skin, and the dose you received on that small patch of skin would be no more than you receive on your whole body for an extended period on an airline flight due to cosmic rays. The only danger from thorium glass would be if you had a telescope or microscope eyepiece containing it, and looked through it with the naked eye. Alpha radiation can damage the cornea of the eye. Fortunately, most manufacturers were wise enough to avoid thoriated glass for such applications, and radioactive eyepieces are very rare. (Still, if you buy a vintage telescope or microscope, you might want to test the eyepieces, especially if the glass appears yellowed.)



Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar