Saturday Night Science: Life After Google

“Life after Google” by George GilderIn his 1990 book Life after Television, George Gilder predicted that the personal computer, then mostly boxes that sat on desktops and worked in isolation from one another, would become more personal, mobile, and be used more to communicate than to compute. In the 1994 revised edition of the book, he wrote. “The most common personal computer of the next decade will be a digital cellular phone with an IP address … connecting to thousands of databases of all kinds.” In contemporary speeches he expanded on the idea, saying, “it will be as portable as your watch and as personal as your wallet; it will recognize speech and navigate streets; it will collect your mail, your news, and your paycheck.” In 2000, he published Telecosm, where he forecast that the building out of a fibre optic communication infrastructure and the development of successive generations of spread spectrum digital mobile communication technologies would effectively cause the cost of communication bandwidth (the quantity of data which can be transmitted in a given time) to asymptotically approach zero, just as the ability to pack more and more transistors on microprocessor and memory chips was doing for computing.

Clearly, when George Gilder forecasts the future of computing, communication, and the industries and social phenomena that spring from them, it’s wise to pay attention. He’s not infallible: in 1990 he predicted that “in the world of networked computers, no one would have to see an advertisement he didn’t want to see”. Oh, well. The very difference between that happy vision and the advertisement-cluttered world we inhabit today, rife with bots, malware, scams, and serial large-scale security breaches which compromise the personal data of millions of people and expose them to identity theft and other forms of fraud is the subject of this book: how we got here, and how technology is opening a path to move on to a better place.

The Internet was born with decentralisation as a central concept. Its U.S. government-funded precursor, ARPANET, was intended to research and demonstrate the technology of packet switching, in which dedicated communication lines from point to point (as in the telephone network) were replaced by switching packets, which can represent all kinds of data—text, voice, video, mail, cat pictures—from source to destination over shared high-speed data links. If the network had multiple paths from source to destination, failure of one data link would simply cause the network to reroute traffic onto a working path, and communication protocols would cause any packets lost in the failure to be automatically re-sent, preventing loss of data. The network might degrade and deliver data more slowly if links or switching hubs went down, but everything would still get through.

This was very attractive to military planners in the Cold War, who worried about a nuclear attack decapitating their command and control network by striking one or a few locations through which their communications funnelled. A distributed network, of which ARPANET was the prototype, would be immune to this kind of top-down attack because there was no top: it was made up of peers, spread all over the landscape, all able to switch data among themselves through a mesh of interconnecting links.

As the ARPANET grew into the Internet and expanded from a small community of military, government, university, and large company users into a mass audience in the 1990s, this fundamental architecture was preserved, but in practice the network bifurcated into a two tier structure. The top tier consisted of the original ARPANET-like users, plus “Internet Service Providers” (ISPs), who had top-tier (“backbone”) connectivity, and then resold Internet access to their customers, who mostly initially connected via dial-up modems. Over time, these customers obtained higher bandwidth via cable television connections, satellite dishes, digital subscriber lines (DSL) over the wired telephone network, and, more recently, mobile devices such as cellular telephones and tablets.

The architecture of the Internet remained the same, but this evolution resulted in a weakening of its peer-to-peer structure. The approaching exhaustion of 32 bit Internet addresses (IPv4) and the slow deployment of its successor (IPv6) meant most small-scale Internet users did not have a permanent address where others could contact them. In an attempt to shield users from the flawed security model and implementation of the software they ran, their Internet connections were increasingly placed behind firewalls and subjected to Network Address Translation (NAT), which made it impossible to establish peer to peer connections without a third party intermediary (which, of course, subverts the design goal of decentralisation). While on the ARPANET and the original Internet every site was a peer of every other (subject only to the speed of their network connections and computer power available to handle network traffic), the network population now became increasingly divided into producers or publishers (who made information available), and consumers (who used the network to access the publishers’ sites but did not publish themselves).

While in the mid-1990s it was easy (or as easy as anything was in that era) to set up your own Web server and publish anything you wished, now most small-scale users were forced to employ hosting services operated by the publishers to make their content available. Services such as AOL, Myspace, Blogger, Facebook, and YouTube were widely used by individuals and companies to host their content, while those wishing their own apparently independent Web presence moved to hosting providers who supplied, for a fee, the servers, storage, and Internet access used by the site.

All of this led to a centralisation of data on the Web, which was accelerated by the emergence of the high speed fibre optic links and massive computing power upon which Gilder had based his 1990 and 2000 forecasts. Both of these came with great economies of scale: it cost a company like Google or Amazon much less per unit of computing power or network bandwidth to build a large, industrial-scale data centre located where electrical power and cooling were inexpensive and linked to the Internet backbone by multiple fibre optic channels, than it cost an individual Internet user or small company with their own server on premises and a modest speed link to an ISP. Thus it became practical for these Goliaths of the Internet to suck up everybody’s data and resell their computing power and access at attractive prices.

As a example of the magnitude of the economies of scale we’re talking about, when I migrated the hosting of my Fourmilab.ch site from my own on-site servers and Internet connection to an Amazon Web Services data centre, my monthly bill for hosting the site dropped by a factor of fifty—not fifty percent, one fiftieth the cost, and you can bet Amazon’s making money on the deal.

This tremendous centralisation is the antithesis of the concept of ARPANET. Instead of a worldwide grid of redundant data links and data distributed everywhere, we have a modest number of huge data centres linked by fibre optic cables carrying traffic for millions of individuals and enterprises. A couple of submarines full of Trident D5s would probably suffice to reset the world, computer network-wise, to 1970.

As this concentration was occurring, the same companies who were building the data centres were offering more and more services to users of the Internet: search engines; hosting of blogs, images, audio, and video; E-mail services; social networks of all kinds; storage and collaborative working tools; high-resolution maps and imagery of the world; archives of data and research material; and a host of others. How was all of this to be paid for? Those giant data centres, after all, represent a capital investment of tens of billions of dollars, and their electricity bills are comparable to those of an aluminium smelter. Due to the architecture of the Internet or, more precisely, missing pieces of the puzzle, a fateful choice was made in the early days of the build-out of these services which now pervade our lives, and we’re all paying the price for it. So far, it has allowed the few companies in this data oligopoly to join the ranks of the largest, most profitable, and most highly valued enterprises in human history, but they may be built on a flawed business model and foundation vulnerable to disruption by software and hardware technologies presently emerging.

The basic business model of what we might call the “consumer Internet” (as opposed to businesses who pay to host their Web presence, on-line stores, etc.) has, with few exceptions, evolved to be what the author calls the “Google model” (although it predates Google): give the product away and make money by afflicting its users with advertisements (which are increasingly targeted to them through information collected from the user’s behaviour on the network through intrusive tracking mechanisms). The fundamental flaws of this are apparent to anybody who uses the Internet: the constant clutter of advertisements, with pop-ups, pop-overs, auto-play video and audio, flashing banners, incessant requests to allow tracking “cookies” or irritating notifications, and the consequent arms race between ad blockers and means to circumvent them, with browser developers (at least those not employed by those paid by the advertisers, directly or indirectly) caught in the middle. There are even absurd Web sites which charge a subscription fee for “membership” and then bombard these paying customers with advertisements that insult their intelligence. But there is a fundamental problem with “free”—it destroys the most important channel of communication between the vendor of a product or service and the customer: the price the customer is willing to pay. Deprived of this information, the vendor is in the same position as a factory manager in a centrally planned economy who has no idea how many of each item to make because his orders are handed down by a planning bureau equally clueless about what is needed in the absence of a price signal. In the end, you have freight cars of typewriter ribbons lined up on sidings while customers wait in line for hours in the hope of buying a new pair of shoes. Further, when the user is not the customer (the one who pays), and especially when a “free” service verges on monopoly status like Google search, Gmail, Facebook, and Twitter, there is little incentive for providers to improve the user experience or be responsive to user requests and needs. Users are subjected to the endless torment of buggy “beta” releases, capricious change for the sake of change, and compromises in the user experience on behalf of the real customers—the advertisers. Once again, this mirrors the experience of centrally-planned economies where the market feedback from price is absent: to appreciate this, you need only compare consumer products from the 1970s and 1980s manufactured in the Soviet Union with those from Japan.

The fundamental flaw in Karl Marx’s economics was his belief that the industrial revolution of his time would produce such abundance of goods that the problem would shift from “production amid scarcity” to “redistribution of abundance”. In the author’s view, the neo-Marxists of Silicon Valley see the exponentially growing technologies of computing and communication providing such abundance that they can give away its fruits in return for collecting and monetising information collected about their users (note, not “customers”: customers are those who pay for the information so collected). Once you grasp this, it’s easier to understand the politics of the barons of Silicon Valley.

The centralisation of data and information flow in these vast data silos creates another threat to which a distributed system is immune: censorship or manipulation of information flow, whether by a coercive government or ideologically-motivated management of the companies who provide these “free” services. We may never know who first said “The Internet treats censorship as damage and routes around it” (the quote has been attributed to numerous people, including two personal friends, so I’m not going there), but it’s profound: the original decentralised structure of the ARPANET/Internet is as robust against censorship as it is in the face of nuclear war. If one or more nodes on the network start to censor information or refuse to forward it on communication links it controls, the network routing protocols simply assume that node is down and send data around it through other nodes and paths which do not censor it. On a network with a multitude of nodes and paths among them, owned by a large and diverse population of operators, it is extraordinarily difficult to shut down the flow of information from a given source or viewpoint; there will almost always be an alternative route that gets it there. (Cryptographic protocols and secure and verified identities can similarly avoid the alteration of information in transit or forging information and attributing it to a different originator; I’ll discuss that later.) As with physical damage, top-down censorship does not work because there’s no top.

But with the current centralised Internet, the owners and operators of these data silos have enormous power to put their thumbs on the scale, tilting opinion in their favour and blocking speech they oppose. Google can push down the page rank of information sources of which they disapprove, so few users will find them. YouTube can “demonetise” videos because they dislike their content, cutting off their creators’ revenue stream overnight with no means of appeal, or they can outright ban creators from the platform and remove their existing content. Twitter routinely “shadow-bans” those with whom they disagree, causing their tweets to disappear into the void, and outright banishes those more vocal. Internet payment processors and crowd funding sites enforce explicit ideological litmus tests on their users, and revoke long-standing commercial relationships over legal speech. One might restate the original observation about the Internet as “The centralised Internet treats censorship as an opportunity and says, ‘Isn’t it great!’ ” Today there’s a top, and those on top control the speech of everything that flows through their data silos.

This pernicious centralisation and “free” funding by advertisement (which is fundamentally plundering users’ most precious possessions: their time and attention) were in large part the consequence of the Internet’s lacking three fundamental architectural layers: security, trust, and transactions. Let’s explore them.

Security. Essential to any useful communication system, security simply means that communications between parties on the network cannot be intercepted by third parties, modified en route, or otherwise manipulated (for example, by changing the order in which messages are received). The communication protocols of the Internet, based on the OSI model, had no explicit security layer. It was expected to be implemented outside the model, across the layers of protocol. On today’s Internet, security has been bolted-on, largely through the Transport Layer Security (TLS) protocols (which, due to history, have a number of other commonly used names, and are most often encountered in the “https:” URLs by which users access Web sites). But because it’s bolted on, not designed in from the bottom-up, and because it “just grew” rather than having been designed in, TLS has been the locus of numerous security flaws which put software that employs it at risk. Further, TLS is a tool which must be used by application designers with extreme care in order to deliver security to their users. Even if TLS were completely flawless, it is very easy to misuse it in an application and compromise users’ security.

Trust. As indispensable as security is knowing to whom you’re talking. For example, when you connect to your bank’s Web site, how do you know you’re actually talking to their server and not some criminal whose computer has spoofed your computer’s domain name system server to intercept your communications and who, the moment you enter your password, will be off and running to empty your bank accounts and make your life a living Hell? Once again, trust has been bolted on to the existing Internet through a rickety system of “certificates” issued mostly by large companies for outrageous fees. And, as with anything centralised, it’s vulnerable: in 2016, one of the top-line certificate vendors was compromised, requiring myriad Web sites (including this one) to re-issue their security certificates.

Transactions. Business is all about transactions; if you aren’t doing transactions, you aren’t in business or, as Gilder puts it, “In business, the ability to conduct transactions is not optional. It is the way all economic learning and growth occur. If your product is ‘free,’ it is not a product, and you are not in business, even if you can extort money from so-called advertisers to fund it.” The present-day Internet has no transaction layer, even bolted on. Instead, we have more silos and bags hanging off the side of the Internet called PayPal, credit card processing companies, and the like, which try to put a Band-Aid over the suppurating wound which is the absence of a way to send money over the Internet in a secure, trusted, quick, efficient, and low-overhead manner. The need for this was perceived long before ARPANET. In Project Xanadu, founded by Ted Nelson in 1960, rule 9 of the “original 17 rules” was, “Every document can contain a royalty mechanism at any desired degree of granularity to ensure payment on any portion accessed, including virtual copies (‘transclusions’) of all or part of the document.” While defined in terms of documents and quoting, this implied the existence of a micropayment system which would allow compensating authors and publishers for copies and quotations of their work with a granularity as small as one character, and could easily be extended to cover payments for products and services. A micropayment system must be able to handle very small payments without crushing overhead, extremely quickly, and transparently (without the Japanese tea ceremony that buying something on-line involves today). As originally envisioned by Ted Nelson, as you read documents, their authors and publishers would be automatically paid for their content, including payments to the originators of material from others embedded within them. As long as the total price for the document was less than what I termed the user’s “threshold of paying”, this would be completely transparent (a user would set the threshold in the browser: if zero, they’d have to approve all payments). There would be no need for advertisements to support publication on a public hypertext network (although publishers would, of course, be free to adopt that model if they wished). If implemented in a decentralised way, like the ARPANET, there would be no central strangle point where censorship could be applied by cutting off the ability to receive payments.

So, is it possible to remake the Internet, building in security, trust, and transactions as the foundation, and replace what the author calls the “Google system of the world” with one in which the data silos are seen as obsolete, control of users’ personal data and work returns to their hands, privacy is respected and the panopticon snooping of today is seen as a dark time we’ve put behind us, and the pervasive and growing censorship by plutocrat ideologues and slaver governments becomes impotent and obsolete? George Gilder responds “yes”, and in this book identifies technologies already existing and being deployed which can bring about this transformation.

At the heart of many of these technologies is the concept of a blockchain, an open, distributed ledger which records transactions or any other form of information in a permanent, public, and verifiable manner. Originally conceived as the transaction ledger for the Bitcoin cryptocurrency, it provided the first means of solving the double-spending problem (how do you keep people from spending a unit of electronic currency twice) without the need for a central server or trusted authority, and hence without a potential choke-point or vulnerability to attack or failure. Since the launch of Bitcoin in 2009, blockchain technology has become a major area of research, with banks and other large financial institutions, companies such as IBM, and major university research groups exploring applications with the goals of drastically reducing transaction costs, improving security, and hardening systems against single-point failure risks.

Applied to the Internet, blockchain technology can provide security and trust (through the permanent publication of public keys which identify actors on the network), and a transaction layer able to efficiently and quickly execute micropayments without the overhead, clutter, friction, and security risks of existing payment systems. By necessity, present-day blockchain implementations are add-ons to the existing Internet, but as the technology matures and is verified and tested, it can move into the foundations of a successor system, based on the same lower-level protocols (and hence compatible with the installed base), but eventually supplanting the patched-together architecture of the Domain Name System, certificate authorities, and payment processors, all of which represent vulnerabilities of the present-day Internet and points at which censorship and control can be imposed. Technologies to watch in these areas are:

As the bandwidth available to users on the edge of the network increases through the deployment of fibre to the home and enterprise and via 5G mobile technology, the data transfer economy of scale of the great data silos will begin to erode. Early in the Roaring Twenties, the aggregate computing power and communication bandwidth on the edge of the network will equal and eventually dwarf that of the legacy data smelters of Google, Facebook, Twitter, and the rest. There will no longer be any need for users to entrust their data to these overbearing anachronisms and consent to multi-dozen page “terms of service” or endure advertising just to see their own content or share it with others. You will be in possession of your own data, on your own server or on space for which you freely contract with others, with backup and other services contracted with any other provider on the network. If your server has extra capacity, you can turn it into money by joining the market for computing and storage capacity, just as you take advantage of these resources when required. All of this will be built on the new secure foundation, so you will retain complete control over who can see your data, no longer trusting weasel-worded promises made by amorphous entities with whom you have no real contract to guard your privacy and intellectual property rights. If you wish, you can be paid for your content, with remittances made automatically as people access it. More and more, you’ll make tiny payments for content which is no longer obstructed by advertising and chopped up to accommodate more clutter. And when outrage mobs of pink hairs and soybeards (each with their own pronoun) come howling to ban you from the Internet, they’ll find nobody to shriek at and the kill switch rusting away in a derelict data centre: your data will be in your own hands with access through myriad routes. Technologies moving in this direction include:

This book provides a breezy look at the present state of the Internet, how we got here (versus where we thought we were going in the 1990s), and how we might transcend the present-day mess into something better if not blocked by the heavy hand of government regulation (the risk of freezing the present-day architecture in place by unleashing agencies like the U.S. Federal Communications Commission, which stifled innovation in broadcasting for six decades, to do the same to the Internet is discussed in detail). Although it’s way too early to see which of the many contending technologies will win out (and recall that the technically superior technology doesn’t always prevail), a survey of work in progress provides a sense for what they have in common and what the eventual result might look like.

There are many things to quibble about here. Gilder goes on at some length about how he believes artificial intelligence is all nonsense, that computers can never truly think or be conscious, and that creativity (new information in the Shannon sense) can only come from the human mind, with a lot of confused arguments from Gödel incompleteness, the Turing halting problem, and even the uncertainty principle of quantum mechanics. He really seems to believe in vitalism, that there is an élan vital which somehow infuses the biological substrate which no machine can embody. This strikes me as superstitious nonsense: a human brain is a structure composed of quarks and electrons arranged in a certain way which processes information, interacts with its environment, and is able to observe its own operation as well as external phenomena (which is all consciousness is about). Now, it may be that somehow quantum mechanics is involved in all of this, and that our existing computers, which are entirely deterministic and classical in their operation, cannot replicate this functionality, but if that’s so it simply means we’ll have to wait until quantum computing, which is already working in a rudimentary form in the laboratory, and is just a different way of arranging the quarks and electrons in a system, develops further.

He argues that while Bitcoin can be an efficient and secure means of processing transactions, it is unsuitable as a replacement for volatile fiat money because, unlike gold, the quantity of Bitcoin has an absolute limit, after which the supply will be capped. I don’t get it. It seems to me that this is a feature, not a bug. The supply of gold increases slowly as new gold is mined, and by pure coincidence the rate of increase in its supply has happened to approximate that of global economic growth. But still, the existing inventory of gold dwarfs new supply, so there isn’t much difference between a very slowly increasing supply and a static one. If you’re on a pure gold standard and economic growth is faster than the increase in the supply of gold, there will be gradual deflation because a given quantity of gold will buy more in the future. But so what? In a deflationary environment, interest rates will be low and it will be easy to fund new investment, since investors will receive money back which will be more valuable. With Bitcoin, once the entire supply is mined, supply will be static (actually, very slowly shrinking, as private keys are eventually lost, which is precisely like gold being consumed by industrial uses from which it is not reclaimed), but Bitcoin can be divided without limit (with minor and upward-compatible changes to the existing protocol). So, it really doesn’t matter if, in the greater solar system economy of the year 8537, a single Bitcoin is sufficient to buy Jupiter: transactions will simply be done in yocto-satoshis or whatever. In fact, Bitcoin is better in this regard than gold, which cannot be subdivided below the unit of one atom.

Gilder further argues, as he did in The Scandal of Money, that the proper dimensional unit for money is time, since that is the measure of what is required to create true wealth (as opposed to funny money created by governments or fantasy money “earned” in zero-sum speculation such as currency trading), and that existing cryptocurrencies do not meet this definition. I’ll take his word on the latter point; it’s his definition, after all, but his time theory of money is way too close to the Marxist labour theory of value to persuade me. That theory is trivially falsified by its prediction that more value is created in labour-intensive production of the same goods than by producing them in a more efficient manner. In fact, value, measured as profit, dramatically increases as the labour input to production is reduced. Over forty centuries of human history, the one thing in common among almost everything used for money (at least until our post-reality era) is scarcity: the supply is limited and it is difficult to increase it. The genius of Bitcoin and its underlying blockchain technology is that it solved the problem of how to make a digital good, which can be copied at zero cost, scarce, without requiring a central authority. That seems to meet the essential requirement to serve as money, regardless of how you define that term.

Gilder’s books have a good record for sketching the future of technology and identifying the trends which are contributing to it. He has been less successful picking winners and losers; I wouldn’t make investment decisions based on his evaluation of products and companies, but rather wait until the market sorts out those which will endure.

Gilder, George. Life after Google. Washington: Regnery Publishing, 2018. ISBN 978-1-62157-576-4.

Here is a talk by the author at the Blockstack Berlin 2018 conference which summarises the essentials of his thesis in just eleven minutes and ends with an exhortation to designers and builders of the new Internet to “tear down these walls” around the data centres which imprison our personal information.

This Uncommon Knowledge interview provides, in 48 minutes, a calmer and more in-depth exploration of why the Google world system must fail and what may replace it.

Like 12+

Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: The Taking of K-129

 

“The Taking of K-129” by Josh DeanOn February 24, 1968, Soviet Golf class submarine K-129 sailed from its base in Petropavlovsk for a routine patrol in the Pacific Ocean. These ballistic missile submarines were, at the time, a key part of the Soviet nuclear deterrent. Each carried three SS-N-5 missiles armed with one 800 kiloton nuclear warhead per missile. This was an intermediate range missile which could hit targets inside an enemy country if the submarine approached sufficiently close to the coast. For defence and attacking other ships, Golf class submarines carried two torpedoes with nuclear warheads as well as conventional high explosive warhead torpedoes.

Unlike the U.S. nuclear powered Polaris submarines, the Golf class had conventional diesel-electric propulsion. When submerged, the submarine was powered by batteries which provided limited speed and range and required surfacing or running at shallow snorkel depth for regular recharging by the diesel engines. They would be the last generation of Soviet diesel-electric ballistic missile submarines: the Hotel class and subsequent boats would be nuclear powered.

K-129’s mission was to proceed stealthily to a region of open ocean north of Midway Atoll and patrol there, ready to launch its missiles at U.S. assets in the Pacific in case of war. Submarines on patrol would send coded burst transmissions on a prearranged schedule to indicate that their mission was proceeding as planned.

On March 8, a scheduled transmission from K-129 failed to arrive. This wasn’t immediately cause for concern, since equipment failure was not uncommon, and a submarine commander might choose not to transmit if worried that surfacing and sending the message might disclose his position to U.S. surveillance vessels and aircraft. But when K-129 remained silent for a second day, the level of worry escalated rapidly. Losing a submarine armed with nuclear weapons was a worst-case scenario, and one which had never happened in Soviet naval operations.

A large-scale search and rescue fleet of 24 vessels, including four submarines, set sail from the base in Kamchatka, all communicating in the open on radio and pinging away with active sonar. They were heard to repeatedly call a ship named Red Star with no reply. The search widened, and eventually included thirty-six vessels and fifty-three aircraft, continuing over a period of seventy-three days. Nothing was found, and six months after the disappearance, the Soviet Navy issued a statement that K-129 had been lost while on duty in the Pacific with all on board presumed dead. This was not only a wrenching emotional blow to the families of the crew, but also a financial gut-shot, depriving them of the pension due families of men lost in the line of duty and paying only the one-time accidental death payment and partial pension for industrial accidents.

But if the Soviets had no idea where their submarine was, this was not the case for the U.S. Navy. Sound travels huge distances through the oceans, and starting in the 1950s, the U.S. began to install arrays of hydrophones (undersea sound detectors) on the floors of the oceans around the world. By the 1960s, these arrays, called SOSUS (SOund SUrveillance System) were deployed and operational in both the Atlantic and Pacific and used to track the movements of Soviet submarines. When K-129 went missing, SOSUS analysts went back over their archived data and found a sharp pulse just a few seconds after midnight local time on March 11 around 180° West and 40° North: 2500 km northeast of Hawaii. Not only did the pulse appear nothing like the natural sounds often picked up by SOSUS, events like undersea earthquakes don’t tend to happen at socially constructed round number times and locations like this one. The pulse was picked up by multiple sensors, allowing its position to be determined accurately. The U.S. knew where the K-129 lay on the ocean floor. But what to do with that knowledge?

One thing was immediately clear. If the submarine was in reasonably intact condition, it would be an intelligence treasure unparalleled in the postwar era. Although it did not represent the latest Soviet technology, it would provide analysts their first hands-on examination of Soviet ballistic missile, nuclear weapon, and submarine construction technologies. Further, the boat would certainly be equipped with cryptographic and secure radio communications gear which might provide an insight into penetrating the secret communications to and from submarines on patrol. (Recall that British breaking of the codes used to communicate with German submarines in World War II played a major part in winning the Battle of the Atlantic.) But a glance at a marine chart showed how daunting it would be to reach the site of the wreck. The ocean in the vicinity of the co-ordinates identified by SOSUS was around 5000 metres deep. Only a very few special-purpose research vessels can operate at such a depth, where the water pressure is around 490 times that of the atmosphere at sea level.

The U.S. intelligence community wanted that sub. The first step was to make sure they’d found it. The USS Halibut, a nuclear-powered Regulus cruise missile launching submarine converted for special operations missions, was dispatched to the area where the K-129 was thought to lie. Halibut could not dive anywhere near as deep as the ocean floor, but was equipped with a remote-controlled, wire-tethered “fish”, which could be lowered near the bottom and then directed around the search area, observing with side-looking sonar and taking pictures. After seven weeks searching in vain, with fresh food long exhausted and crew patience wearing thin, the search was abandoned and course set back to Pearl Harbor.

But the prize was too great to pass up. So Halibut set out again, and after another month of operating the fish, developing thousands of pictures, and fraying tempers, there it was! Broken into two parts, but with both apparently largely intact, lying on the ocean bottom. Now what?

While there were deep sea research vessels able to descend to such depths, they were completely inadequate to exploit the intelligence haul that K-129 promised. That would require going inside the structure, dismantling the missiles and warheads, examining and testing the materials, and searching for communications and cryptographic gear. The only way to do this was to raise the submarine. To say that this was a challenge is to understate its difficulty—adjectives fail. The greatest mass which had ever been raised from such a depth was around 50 tonnes and K-129 had a mass of 1,500 tonnes—thirty times greater. But hey, why not? We’re Americans! We’ve landed on the Moon! (By then it was November, 1969, four months after that “one small step”.) And so, Project Azorian was born.

When it comes to doing industrial-scale things in the deep ocean, all roads (or sea lanes) lead to Global Marine. A publicly-traded company little known to those outside the offshore oil exploration industry, this company and its genius naval architect John Graham had pioneered deep-sea oil drilling. While most offshore oil rigs, like those on terra firma, were firmly anchored to the land around the drill hole, Global Marine had pioneered the technology which allowed a ship, with a derrick mounted amidships, to precisely station-keep above the bore-hole on the ocean floor far beneath the ship. The required dropping sonar markers on the ocean floor which the ship used to precisely maintain its position with respect to them. This was just one part of the puzzle.

To recover the submarine, the ship would need to lower what amounted to a giant claw (“That’s claw, not craw!”, you “Get Smart” fans) to the abyssal plain, grab the sub, and lift its 1500 tonne mass to the surface. During the lift, the pipe string which connected the ship to the claw would be under such stress that, should it break, it would release energy comparable to an eight kiloton nuclear explosion, which would be bad.

This would have been absurdly ambitious if conducted in the open, like the Apollo Project, but in this case it also had to be done covertly, since the slightest hint that the U.S. was attempting to raise K-129 would almost certainly provoke a Soviet response ranging from diplomatic protests to a naval patrol around the site of the sinking aimed at harassing the recovery ships. The project needed a cover story and a cut-out to hide the funding to Global Marine which, as a public company, had to disclose its financials quarterly and, unlike minions of the federal government funded by taxes collected from hairdressers and cab drivers through implicit threat of violence, could not hide its activities in a “black budget”.

This was seriously weird and, as a contemporary philosopher said, “When the going gets weird, the weird turn pro.” At the time, nobody was more professionally weird than Howard Hughes. He had taken reclusion to a new level, utterly withdrawing from contact with the public after revulsion from dealing with the Washington swamp and the media. His company still received royalties from every oil well drilled using his drill bits, and his aerospace and technology companies were plugged into the most secret ventures of the U.S. government. Simply saying, “It’s a Hughes project” was sufficient to squelch most questions. This meant it had unlimited funds, the sanction of the U.S. government (including three-letter agencies whose names must not be spoken [brrrr!]), and told pesky journalists they’d encounter a stone wall from the centre of the Earth to the edge of the universe if they tried to dig into details.

Hughes Glomar Explorer

But covert as the project might be, aspects of its construction and operation would unavoidably be in the public eye. You can’t build a 189 metre long, 51,000 tonne ship, the Hughes Glomar Explorer, with an 80 metre tall derrick sticking up amidships, at a shipyard on the east coast of the U.S., send it around Cape Horn to its base on the west coast (the ship was too wide to pass through the Panama Canal), without people noticing. A cover story was needed, and the CIA and their contractors cooked up a doozy.

Large areas of the deep sea floor are covered by manganese nodules, concretions which form around a seed and grow extremely slowly, but eventually reach the size of potatoes or larger. Nodules are composed of around 30% manganese, plus other valuable metals such as nickel, copper, and cobalt. There are estimated to be more than 21 billion tonnes of manganese nodules on the deep ocean floor (depths of 4000 to 6000 metres), and their composition is richer than many of the ores from which the metals they contain are usually extracted. Further, they’re just lying on the seabed. If you could figure out how to go down there and scoop them up, you wouldn’t have to dig mines and process huge amounts of rock. Finally, they were in international waters, and despite attempts by kleptocratic dictators (some in landlocked countries) and the international institutions who support them to enact a “Law of the Sea” treaty to pick the pockets of those who created the means to use this resource, at the time the nodules were just there for the taking—you didn’t have to pay kleptocratic dictators for mining rights or have your profits skimmed by ever-so-enlightened democratic politicians in developed countries.

So, the story was put out that Howard Hughes was setting out to mine the nodules on the Pacific Ocean floor, and that Glomar Explorer, built by Global Marine under contract for Hughes (operating, of course, as a cut-out for the CIA), would deploy a robotic mining barge called the Hughes Mining Barge 1 (HMB-1) which, lowered to the ocean floor, would collect nodules, crush them, and send the slurry to the surface for processing on the mother ship.

This solved a great number of potential problems. Global Marine, as a public company, could simply (and truthfully) report that it was building Glomar Explorer under contract to Hughes, and had no participation in the speculative and risky mining venture, which would have invited scrutiny by Wall Street analysts and investors. Hughes, operating as a proprietorship, was not required to disclose the source of the funds it was paying Global Marine. Everybody assumed the money was coming from Howard Hughes’ personal fortune, which he had invested, over his career, in numerous risky ventures, when in fact, he was simply passing through money from a CIA black budget account. The HMB-1 was built by Lockheed Missiles and Space Company under contract from Hughes. Lockheed was involved in numerous classified U.S. government programs, so operating in the same manner for the famously secretive Hughes raised few eyebrows.

The barge, 99 metres in length, was built in a giant enclosed hangar in the port of Redwood City, California, which shielded it from the eyes of curious onlookers and Soviet reconnaissance satellites passing overhead. This was essential, because a glance at what was being built would have revealed that it looked nothing like a mining barge but rather a giant craw—sorry—claw! To install the claw on the ship, it was towed, enclosed in its covered barge, to a location near Catalina Island in southern California, where deeper water allowed it to be sunk beneath the surface, and then lifted into the well (“moon pool”) of Glomar Explorer, all out of sight to onlookers.

So far, the project had located the target on the ocean floor, designed and built a special ship and retrieval claw to seize it, fabricated a cover story of a mining venture so persuasive other mining companies were beginning to explore launching their own seabed mining projects, and evaded scrutiny by the press, Congress, and Soviet intelligence assets. But these are pussycats compared to the California Tax Nazis! After the first test of mating the claw to the ship, Glomar Explorer took to the ocean to, it was said, test the stabilisation system which would keep the derrick vertical as the ship pitched and rolled in the sea. Actually, the purpose of the voyage was to get the ship out of U.S. territorial waters on March 1st, the day California assessed a special inventory tax on all commercial vessels in state waters. This would not only cost a lot of money, it would force disclosure of the value of the ship, which could be difficult to reconcile with its cover mission. Similar fast footwork was required when Hughes took official ownership of the vessel from Global Marine after acceptance. A trip outside U.S. territorial waters was also required to get off the hook for the 7% sales tax California would otherwise charge on the transfer of ownership.

Finally, in June 1974, all was ready, and Glomar Explorer with HMB-1 attached set sail from Long Beach, California to the site of K-129’s wreck, arriving on site on the Fourth of July, only to encounter foul weather. Opening the sea doors in the well in the centre of the ship and undocking the claw required calm seas, and it wasn’t until July 18th that they were ready to begin the main mission. Just at that moment, what should show up but a Soviet missile tracking ship. After sending its helicopter to inspect Explorer, it eventually departed. This wasn’t the last of the troubles with pesky Soviets.

On July 21, the recovery operation began, slowly lowering the claw on its string of pipes. Just at this moment, another Soviet ship arrived, a 47 metre ocean-going tug called SB-10. This tug would continue to harass the recovery operation for days, approaching on an apparent collision course and then veering off. (Glomar Explorer could not move during the retrieval operation, being required to use its thrusters to maintain its position directly above the wrecked submarine on the bottom.)

On August 3, the claw reached the bottom and its television cameras revealed it was precisely on target—there was the submarine, just as it had been photographed by the Halibut six years earlier. The claw gripped the larger part of the wreck, its tines closed under it, and a combination of pistons driving against the ocean bottom and the lift system pulling on the pipe from the ship freed the submarine from the bottom. Now the long lift could begin.

Everything had worked. The claw had been lowered, found its target on the first try, successfully seized it despite the ocean bottom’s being much harder than expected, freed it from the bottom, and the ship had then successfully begun to lift the 6.4 million kg of pipe, claw, and submarine back toward the surface. Within the first day of the lift, more than a third of the way to the surface, with the load on the heavy lift equipment diminishing by 15 tonnes as each segment of lift pipe was removed from the string, a shudder went through the ship and the heavy lift equipment lurched violently. Something had gone wrong, seriously wrong. Examination of television images from the claw revealed that several of the tines gripping the hull of the submarine had failed and part of the sub, maybe more than half, had broken off and fallen back toward the abyss. (It was later decided that the cause of the failure was that the tines had been fabricated from maraging steel, which is very strong but brittle, rather than a more ductile alloy which would bend under stress but not break.)

After consultation with CIA headquarters, it was decided to continue the lift and recover whatever was left in the claw. (With some of the tines broken and the mechanism used to break the load free of the ocean floor left on the bottom, it would have been impossible to return and recover the lost part of the sub on this mission.) On August 6th, the claw and its precious payload reached the ship and entered the moon pool in its centre. Coincidentally, the Soviet tug departed the scene the same day. Now it was possible to assess what had been recovered, and the news was not good: two thirds of the sub had been lost, including the ballistic missile tubes and the code room. Only the front third was in the claw. Further, radiation five times greater than background was detected even outside the hull—those exploring it would have to proceed carefully.

An “exploitation team” composed of CIA specialists and volunteers from the ship’s crew began to explore the wreckage, photographing and documenting every part recovered. They found the bodies of six Soviet sailors and assorted human remains which could not be identified; all went to the ship’s morgue. Given that the bow portion of the submarine had been recovered, it is likely that one or more of its torpedoes equipped with nuclear warheads were recovered, but to this day the details of what was found in the wreck remain secret. By early September, the exploitation was complete and the bulk of the recovered hull, less what had been removed and sent for analysis, was dumped in the deep ocean 160 km south of Hawaii.

One somber task remained. On September 4, 1974, the remains of the six recovered crewmen and the unidentified human remains were buried at sea in accordance with Soviet Navy tradition. A video tape of this ceremony was made and, in 1992, a copy was presented to Russian President Boris Yeltsin by then CIA director Robert Gates.

The partial success encouraged some in the CIA to mount a follow-up mission to recover the rest of the sub, including the missiles and code room. After all, they knew precisely where it was, had a ship in hand, fully paid for, which had successfully lowered the claw to the bottom and returned to the surface with part of the sub, and they knew what had gone wrong with the claw and how to fix it. The effort was even given a name, Project Matador. But it was not to be.

Over the five years of the project there had been leaks to the press and reporters sniffing on the trail of the story but the CIA had been able to avert disclosure by contacting the reporters directly, explaining the importance of the mission and need for secrecy, and offering them an exclusive of full disclosure and permission to publish it before the project was officially declassified for the general public. This had kept a lid on the secret throughout the entire development process and the retrieval and analysis, but this all came to an end in March 1975 when Jack Anderson got wind of the story. There was no love lost between Anderson and what we now call the Deep State. Anderson believed the First Amendment was divinely inspired and absolute, while J. Edgar Hoover had called Anderson “lower than the regurgitated filth of vultures”. Further, this was a quintessential Jack Anderson story—based upon his sources, he presented Project Azorian as a US$ 350 million failure which had produced no useful intelligence information and was being kept secret only to cover up the squandering of taxpayers’ money.

New York Times Azorian story by Seymour HershCIA Director William Colby offered Anderson the same deal other journalists had accepted, but was flatly turned down. Five minutes before Anderson went on the radio to break the story, Colby was still pleading with him to remain silent. On March 18, 1975, Anderson broke the story on his Mutual Radio Network show and, the next day, published additional details in his nationally syndicated newspaper column. Realising the cover had been blown, Colby called all of the reporters who had agreed to hold the story to give them the green light to publish. Seymour Hersh of the New York Times had his story ready to go, and it ran on the front page of the next day’s paper, providing far more detail (albeit along with a few errors) than Anderson’s disclosure. Hersh revealed that he had been aware of the project since 1973 but had agreed to withhold publication in the interest of national security.

The story led newspaper and broadcast news around the country and effectively drove a stake through any plans to mount a follow-up retrieval mission. On June 16, 1975, Secretary of State Henry Kissinger made a formal recommendation to president Gerald Ford to terminate the project and that was the end of it. The Soviets had communicated through a back channel that they had no intention of permitting a second retrieval attempt and they had maintained an ocean-going tug on site to monitor any activity since shortly after the story broke in the U.S.

The CIA’s official reaction to all the publicity was what has come to be called the “Glomar Response”: “We can neither confirm nor can we deny.” And that is where things stand more that four decades after the retrieval attempt. Although many of those involved in the project have spoken informally about aspects of it, there has never been an official report on precisely what was recovered or what was learned from it. Some CIA veterans have said, off the record, that much more was learned from the recovered material than has been suggested in press reports, with a few arguing that the entire large portion of the sub was recovered and the story about losing much of it was a cover story. (But if this was the case, the whole plan to mount a second retrieval mission and the substantial expense repairing and upgrading the claw for the attempt, which is well documented, would also have to have been a costly cover story.)

What is certain is that Project Azorian was one of the most daring intelligence exploits in history, carried out in total secrecy under the eyes of the Soviets, and kept secret from an inquiring press for five years by a cover story so persuasive other mining companies bought it hook, line, and sinker. We may never know all the details of the project, but from what we do know it is a real-world thriller which equals or exceeds those imagined by masters of the fictional genre.

Dean, Josh. The Taking of K-129. New York: Dutton, 2012. ISBN 978-1-101-98443-7.

Here is a Discovery Channel documentary (complete with cheesy period commercials) about Project Azorian and the recovery of K-129.  A portion of the burial at sea of the Soviet crew is shown at the end.

John Evans was Vice President of Global Marine during the project.  In this talk he gives his personal reminiscences of the project, declining to comment on information which has not become public from other sources.

This educational film was produced while the K-129 recovery project was underway.  Purporting to be about deep sea mining, it presents the cover story for Glomar Explorer, including bogus film of a supposed mining barge which was never used with the ship.

Like 12+

Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Losing the Nobel Prize

“Losing the Nobel Prize” by Brian KeatingEver since the time of Galileo, the history of astronomy has been punctuated by a series of “great debates”—disputes between competing theories of the organisation of the universe which observation and experiment using available technology are not yet able to resolve one way or another. In Galileo’s time, the great debate was between the Ptolemaic model, which placed the Earth at the centre of the solar system (and universe) and the competing Copernican model which had the planets all revolving around the Sun. Both models worked about as well in predicting astronomical phenomena such as eclipses and the motion of planets, and no observation made so far had been able to distinguish them.

Then, in 1610, Galileo turned his primitive telescope to the sky and observed the bright planets Venus and Jupiter. He found Venus to exhibit phases, just like the Moon, which changed over time. This would not happen in the Ptolemaic system, but is precisely what would be expected in the Copernican model—where Venus circled the Sun in an orbit inside that of Earth. Turning to Jupiter, he found it to be surrounded by four bright satellites (now called the Galilean moons) which orbited the giant planet. This further falsified Ptolemy’s model, in which the Earth was the sole source of attraction around which all celestial bodies revolved. Since anybody could build their own telescope and confirm these observations, this effectively resolved the first great debate in favour of the Copernican heliocentric model, although some hold-outs in positions of authority resisted its dethroning of the Earth as the centre of the universe.

This dethroning came to be called the “Copernican principle”, that Earth occupies no special place in the universe: it is one of a number of planets orbiting an ordinary star in a universe filled with a multitude of other stars. Indeed, when Galileo observed the star cluster we call the Pleiades, he saw myriad stars too dim to be visible to the unaided eye. Further, the bright stars were surrounded by a diffuse bluish glow. Applying the Copernican principle again, he argued that the glow was due to innumerably more stars too remote and dim for his telescope to resolve, and then generalised that the glow of the Milky Way was also composed of uncountably many stars. Not only had the Earth been demoted from the centre of the solar system, so had the Sun been dethroned to being just one of a host of stars possibly stretching to infinity.

But Galileo’s inference from observing the Pleiades was wrong. The glow that surrounds the bright stars is due to interstellar dust and gas which reflect light from the stars toward Earth. No matter how large or powerful the telescope you point toward such a reflection nebula, all you’ll ever see is a smooth glow. Driven by the desire to confirm his Copernican convictions, Galileo had been fooled by dust. He would not be the last.

William Herschel was an eminent musician and composer, but his passion was astronomy. He pioneered the large reflecting telescope, building more than sixty telescopes. In 1789, funded by a grant from King George III, Herschel completed a reflector with a mirror 1.26 metres in diameter, which remained the largest aperture telescope in existence for the next fifty years. In Herschel’s day, the great debate was about the Sun’s position among the surrounding stars. At the time, there was no way to determine the distance or absolute brightness of stars, but Herschel decided that he could compile a map of the galaxy (then considered to be the entire universe) by surveying the number of stars in different directions. Only if the Sun was at the centre of the galaxy would the counts be equal in all directions.

Aided by his sister Caroline, a talented astronomer herself, he eventually compiled a map which indicated the galaxy was in the shape of a disc, with the Sun at the centre. This seemed to refute the Copernican view that there was nothing special about the Sun’s position. Such was Herschel’s reputation that this finding, however puzzling, remained unchallenged until 1847 when Wilhelm Struve discovered that Herschel’s results had been rendered invalid by his failing to take into account the absorption and scattering of starlight by interstellar dust. Just as you can only see the same distance in all directions while within a patch of fog, regardless of the shape of the patch, Herschel’s survey could only see so far before extinction of light by dust cut off his view of stars. Later it was discovered that the Sun is far from the centre of the galaxy. Herschel had been fooled by dust.

In the 1920s, another great debate consumed astronomy. Was the Milky Way the entire universe, or were the “spiral nebulæ” other “island universes”, galaxies in their own right, peers of the Milky Way? With no way to measure distance or telescopes able to resolve them into stars, many astronomers believed spiral neublæ were nearby objects, perhaps other solar systems in the process of formation. The discovery of a Cepheid variable star in the nearby Andromeda “nebula” by Edwin Hubble in 1923 allowed settling this debate. Andromeda was much farther away than the most distant stars found in the Milky Way. It must, then be a separate galaxy. Once again, demotion: the Milky Way was not the entire universe, but just one galaxy among a multitude.

But how far away were the galaxies? Hubble continued his search and measurements and found that the more distant the galaxy, the more rapidly it was receding from us. This meant the universe was expanding. Hubble was then able to calculate the age of the universe—the time when all of the galaxies must have been squeezed together into a single point. From his observations, he computed this age at two billion years. This was a major embarrassment: astrophysicists and geologists were confident in dating the Sun and Earth at around five billion years. It didn’t make any sense for them to be more than twice as old as the universe of which they were a part. Some years later, it was discovered that Hubble’s distance estimates were far understated because he failed to account for extinction of light from the stars he measured due to dust. The universe is now known to be seven times the age Hubble estimated. Hubble had been fooled by dust.

By the 1950s, the expanding universe was generally accepted and the great debate was whether it had come into being in some cataclysmic event in the past (the “Big Bang”) or was eternal, with new matter spontaneously appearing to form new galaxies and stars as the existing ones receded from one another (the “Steady State” theory). Once again, there were no observational data to falsify either theory. The Steady State theory was attractive to many astronomers because it was the more “Copernican”—the universe would appear overall the same at any time in an infinite past and future, so our position in time is not privileged in any way, while in the Big Bang the distant past and future are very different than the conditions we observe today. (The rate of matter creation required by the Steady State theory was so low that no plausible laboratory experiment could detect it.)

The discovery of the cosmic background radiation in 1965 definitively settled the debate in favour of the Big Bang. It was precisely what was expected if the early universe were much denser and hotter than conditions today, as predicted by the Big Bang. The Steady State theory made no such prediction and was, despite rear-guard actions by some of its defenders (invoking dust to explain the detected radiation!), was considered falsified by most researchers.

But the Big Bang was not without its own problems. In particular, in order to end up with anything like the universe we observe today, the initial conditions at the time of the Big Bang seemed to have been fantastically fine-tuned (for example, an infinitesimal change in the balance between the density and rate of expansion in the early universe would have caused the universe to quickly collapse into a black hole or disperse into the void without forming stars and galaxies). There was no physical reason to explain these fine-tuned values; you had to assume that’s just the way things happened to be, or that a Creator had set the dial with a precision of dozens of decimal places.

In 1979, the theory of inflation was proposed. Inflation held that in an instant after the Big Bang the size of the universe blew up exponentially so that all the observable universe today was, before inflation, the size of an elementary particle today. Thus, it’s no surprise that the universe we now observe appears so uniform. Inflation so neatly resolved the tensions between the Big Bang theory and observation that it (and refinements over the years) became widely accepted. But could inflation be observed? That is the ultimate test of a scientific theory.

There have been numerous cases in science where many years elapsed between a theory being proposed and definitive experimental evidence for it being found. After Galileo’s observations, the Copernican theory that the Earth orbits the Sun became widely accepted, but there was no direct evidence for the Earth’s motion with respect to the distant stars until the discovery of the aberration of light in 1727. Einstein’s theory of general relativity predicted gravitational radiation in 1915, but the phenomenon was not directly detected by experiment until a century later. Would inflation have to wait as long or longer?

Things didn’t look promising. Almost everything we know about the universe comes from observations of electromagnetic radiation: light, radio waves, X-rays, etc., with a little bit more from particles (cosmic rays and neutrinos). But the cosmic background radiation forms an impenetrable curtain behind which we cannot observe anything via the electromagnetic spectrum, and it dates from around 380,000 years after the Big Bang. The era of inflation was believed to have ended 10−32 seconds after the Bang; considerably earlier. The only “messenger” which could possibly have reached us from that era is gravitational radiation. We’ve just recently become able to detect gravitational radiation from the most violent events in the universe, but no conceivable experiment would be able to detect this signal from the baby universe.

So is it hopeless? Well, not necessarily…. The cosmic background radiation is a snapshot of the universe as it existed 380,000 years after the Big Bang, and only a few years after it was first detected, it was realised that gravitational waves from the very early universe might have left subtle imprints upon the radiation we observe today. In particular, gravitational radiation creates a form of polarisation called B-modes which most other sources cannot create.

If it were possible to detect B-mode polarisation in the cosmic background radiation, it would be a direct detection of inflation. While the experiment would be demanding and eventually result in literally going to the end of the Earth, it would be strong evidence for the process which shaped the universe we inhabit and, in all likelihood, a ticket to Stockholm for those who made the discovery.

This was the quest on which the author embarked in the year 2000, resulting in the deployment of an instrument called BICEP1 (Background Imaging of Cosmic Extragalactic Polarization) in the Dark Sector Laboratory at the South Pole. Here is my picture of that laboratory in January 2013. The BICEP telescope is located in the foreground inside a conical shield which protects it against thermal radiation from the surrounding ice. In the background is the South Pole Telescope, a millimetre wave antenna which was not involved in this research.

BICEP2 and South Pole Telescope, 2013-01-09

BICEP1 was a prototype, intended to test the technologies to be used in the experiment. These included cooling the entire telescope (which was a modest aperture [26 cm] refractor, not unlike Galileo’s, but operating at millimetre wavelengths instead of visible light) to the temperature of interstellar space, with its detector cooled to just ¼ degree above absolute zero. In 2010 its successor, BICEP2, began observation at the South Pole, and continued its run into 2012. When I took the photo above, BICEP2 had recently concluded its observations.

On March 17th, 2014, the BICEP2 collaboration announced, at a press conference, the detection of B-mode polarisation in the region of the southern sky they had monitored. Note the swirling pattern of polarisation which is the signature of B-modes, as opposed to the starburst pattern of other kinds of polarisation.

Cosmic background radiation B-modes from BICEP2

But, not so fast, other researchers cautioned. The risk in doing “science by press release” is that the research is not subjected to peer review—criticism by other researchers in the field—before publication and further criticism in subsequent publications. The BICEP2 results went immediately to the front pages of major newspapers. Here was direct evidence of the birth cry of the universe and confirmation of a theory which some argued implied the existence of a multiverse—the latest Copernican demotion—the idea that our universe was just one of an ensemble, possibly infinite, of parallel universes in which every possibility was instantiated somewhere. Amid the frenzy, a few specialists in the field, including researchers on competing projects, raised the question, “What about the dust?” Dust again! As it happens, while gravitational radiation can induce B-mode polarisation, it isn’t the only thing which can do so. Our galaxy is filled with dust and magnetic fields which can cause those dust particles to align with them. Aligned dust particles cause polarised reflections which can mimic the B-mode signature of the gravitational radiation sought by BICEP2.

The BICEP2 team was well aware of this potential contamination problem. Unfortunately, their telescope was sensitive only to one wavelength, chosen to be the most sensitive to B-modes due to primordial gravitational radiation. It could not, however, distinguish a signal from that cause from one due to foreground dust. At the same time, however, the European Space Agency Planck spacecraft was collecting precision data on the cosmic background radiation in a variety of wavelengths, including one sensitive primarily to dust. Those data would have allowed the BICEP2 investigators to quantify the degree their signal was due to dust. But there was a problem: BICEP2 and Planck were direct competitors.

Planck had the data, but had not released them to other researchers. However, the BICEP2 team discovered that a member of the Planck collaboration had shown a slide at a conference of unpublished Planck observations of dust. A member of the BICEP2 team digitised an image of the slide, created a model from it, and concluded that dust contamination of the BICEP2 data would not be significant. This was a highly dubious, if not explicitly unethical move. It confirmed measurements from earlier experiments and provided confidence in the results.

In September 2014, a preprint from the Planck collaboration (eventually published in 2016) showed that B-modes from foreground dust could account for all of the signal detected by BICEP2. In January 2015, the European Space Agency published an analysis of the Planck and BICEP2 observations which showed the entire BICEP2 detection was consistent with dust in the Milky Way. The epochal detection of inflation had been deflated. The BICEP2 researchers had been deceived by dust.

The author, a founder of the original BICEP project, was so close to a Nobel prize he was already trying to read the minds of the Nobel committee to divine who among the many members of the collaboration they would reward with the gold medal. Then it all went away, seemingly overnight, turned to dust. Some said that the entire episode had injured the public’s perception of science, but to me it seems an excellent example of science working precisely as intended. A result is placed before the public; others, with access to the same raw data are given an opportunity to critique them, setting forth their own raw data; and eventually researchers in the field decide whether the original results are correct. Yes, it would probably be better if all of this happened in musty library stacks of journals almost nobody reads before bursting out of the chest of mass media, but in an age where scientific research is funded by agencies spending money taken from hairdressers and cab drivers by coercive governments under implicit threat of violence, it is inevitable they will force researchers into the public arena to trumpet their “achievements”.

In parallel with the saga of BICEP2, the author discusses the Nobel Prizes and what he considers to be their dysfunction in today’s scientific research environment. I was surprised to learn that many of the curious restrictions on awards of the Nobel Prize were not, as I had heard and many believe, conditions of Alfred Nobel’s will. In fact, the conditions that the prize be shared no more than three ways, not be awarded posthumously, and not awarded to a group (with the exception of the Peace prize) appear nowhere in Nobel’s will, but were imposed later by the Nobel Foundation. Further, Nobel’s will explicitly states that the prizes shall be awarded to “those who, during the preceding year, shall have conferred the greatest benefit to mankind”. This constraint (emphasis mine) has been ignored since the inception of the prizes.

He decries the lack of “diversity” in Nobel laureates (by which he means, almost entirely, how few women have won prizes). While there have certainly been women who deserved prizes and didn’t win (Lise Meitner, Jocelyn Bell Burnell, and Vera Rubin are prime examples), there are many more men who didn’t make the three laureates cut-off (Freeman Dyson an obvious example for the 1965 Physics Nobel for quantum electrodynamics). The whole Nobel prize concept is capricious, and rewards only those who happen to be in the right place at the right time in the right field that the committee has decided deserves an award this year and are lucky enough not to die before the prize is awarded. To imagine it to be “fair” or representative of scientific merit is, in the estimation of this scribbler, in flying unicorn territory.

In all, this is a candid view of how science is done at the top of the field today, with all of the budget squabbles, maneuvering for recognition, rivalry among competing groups of researchers, balancing the desire to get things right with the compulsion to get there first, and the eye on that prize, given only to a few in a generation, which can change one’s life forever.

Personally, I can’t imagine being so fixated on winning a prize one has so little chance of gaining. It’s like being obsessed with winning the lottery—and about as likely.

In parallel with all of this is an autobiographical account of the career of a scientist with its ups and downs, which is both a cautionary tale and an inspiration to those who choose to pursue that difficult and intensely meritocratic career path.

I recommend this book on all three tracks: a story of scientific discovery, mis-interpretation, and self-correction, the dysfunction of the Nobel Prizes and how they might be remedied, and the candid story of a working scientist in today’s deeply corrupt coercively-funded research environment.

Keating, Brian. Losing the Nobel Prize. New York: W. W. Norton, 2018. ISBN 978-1-324-00091-4.

Here is a one hour talk by the author about the BICEP2 experience and the Nobel Prize.

This is the BICEP2 press conference on March 17, 2014, announcing the discovery of B-mode polarisation in the cosmic microwave background radiation.

What do you do after losing the Nobel prize?  In this April, 2016 (much) more technical talk at the SETI Institute, Brian Keating describes post-BICEP2 research aimed at using the cosmic background radiation to explore other aspects of the early universe including whether the universe has an inherent chirality (left- or right-handedness).  (The preview image for this video looks like it’s broken, but if you click the play button it plays correctly, at least for me.)

Like 12+

Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Bad Blood

“Bad Blood” by John CarreyrouThe drawing of blood for laboratory tests is one of my least favourite parts of a routine visit to the doctor’s office. Now, I have no fear of needles and hardly notice the stick, but frequently the doctor’s assistant who draws the blood (whom I’ve nicknamed Vampira) has difficulty finding the vein to get a good flow and has to try several times. On one occasion she made an internal puncture which resulted in a huge, ugly bruise that looked like I’d slammed a car door on my arm. I wondered why they need so much blood, and why draw it into so many different containers? (Eventually, I researched this, having been intrigued by the issue during the O. J. Simpson trial; if you’re curious, here is the information.) Then, after the blood is drawn, it has to be sent off to the laboratory, which sends back the results days later. If something pops up in the test results, you have to go back for a second visit with the doctor to discuss it.

Wouldn’t it be great if they could just stick a fingertip and draw a drop or two of blood, as is done by diabetics to test blood sugar, then run all the tests on it? Further, imagine if, after taking the drop of blood, it could be put into a desktop machine right in the doctor’s office which would, in a matter of minutes, produce test results you could discuss immediately with the doctor. And if such a technology existed and followed the history of decline in price with increase in volume which has characterised other high technology products since the 1970s, it might be possible to deploy the machines into the homes of patients being treated with medications so their effects could be monitored and relayed directly to their physicians in case an anomaly was detected. It wouldn’t quite be a Star Trek medical tricorder, but it would be one step closer. With the cost of medical care rising steeply, automating diagnostic blood tests and bringing them to the mass market seemed an excellent candidate as the “next big thing” for Silicon Valley to revolutionise.

This was the vision that came to 19 year old Elizabeth Holmes after completing a summer internship at the Genome Institute of Singapore after her freshman year as a chemical engineering major at Stanford. Holmes had decided on a career in entrepreneurship from an early age and, after her first semester told her father, “No, Dad, I’m, not interested in getting a Ph.D. I want to make money.” And Stanford, in the heart of Silicon Valley, was surrounded by companies started by professors and graduates who had turned inventions into vast fortunes. With only one year of college behind her, she was sure she’d found her opportunity. She showed the patent application she’d drafted for an arm patch that would diagnose medical conditions to Channing Robertson, professor of chemical engineering at Stanford, and Shaunak Roy, the Ph.D. student in whose lab she had worked as an assistant during her freshman year. Robertson was enthusiastic, and when Holmes said she intended to leave Stanford and start a company to commercialise the idea, he encouraged her. When the company was incorporated in 2004, Roy, then a newly-minted Ph.D., became its first employee and Robertson joined the board.

From the outset, the company was funded by other people’s money. Holmes persuaded a family friend, Tim Draper, a second-generation venture capitalist who had backed, among other companies, Hotmail, to invest US$ 1 million in first round funding. Draper was soon joined by Victor Palmieri, a corporate turnaround artist and friend of Holmes’ father. The company was named Theranos, from “therapy” and “diagnosis”. Elizabeth, unlike this scribbler, had a lifelong aversion to needles, and the invention she described in the business plan pitched to investors was informed by this. A skin patch would draw tiny quantities of blood without pain by means of “micro-needles”, the blood would be analysed by micro-miniaturised sensors in the patch and, if needed, medication could be injected. A wireless data link would send results to the doctor.

This concept, and Elizabeth’s enthusiasm and high-energy pitch allowed her to recruit additional investors, raising almost US$ 6 million in 2004. But there were some who failed to be persuaded: MedVentures Associates, a firm that specialised in medical technology, turned her down after discovering she had no answers for the technical questions raised in a meeting with the partners, who had in-depth experience with diagnostic technology. This would be a harbinger of the company’s fund-raising in the future: in its entire history, not a single venture fund or investor with experience in medical or diagnostic technology would put money into the company.

Shaunak Roy, who, unlike Holmes, actually knew something about chemistry, quickly realised that Elizabeth’s concept, while appealing to the uninformed, was science fiction, not science, and no amount of arm-waving about nanotechnology, microfluidics, or laboratories on a chip would suffice to build something which was far beyond the state of the art. This led to a “de-scoping” of the company’s ambition—the first of many which would happen over succeeding years. Instead of Elizabeth’s magical patch, a small quantity of blood would be drawn from a finger stick and placed into a cartridge around the size of a credit card. The disposable cartridge would then be placed into a desktop “reader” machine, which would, using the blood and reagents stored in the cartridge, perform a series of analyses and report the results. This was originally called Theranos 1.0, but after a series of painful redesigns, was dubbed the “Edison”. This was the prototype Theranos ultimately showed to potential customers and prospective investors.

This was a far cry from the original ambitious concept. The hundreds of laboratory tests doctors can order are divided into four major categories: immunoassays, general chemistry, hæmatology, and DNA amplification. In immunoassay tests, blood plasma is exposed to an antibody that detects the presence of a substance in the plasma. The antibody contains a marker which can be detected by its effect on light passed through the sample. Immunoassays are used in a number of common blood tests, such the 25(OH)D assay used to test for vitamin D deficiency, but cannot perform other frequently ordered tests such as blood sugar and red and white blood cell counts. Edison could only perform what is called “chemiluminescent immunoassays”, and thus could only perform a fraction of the tests regularly ordered. The rationale for installing an Edison in the doctor’s office was dramatically reduced if it could only do some tests but still required a venous blood draw be sent off to the laboratory for the balance.

This didn’t deter Elizabeth, who combined her formidable salesmanship with arm-waving about the capabilities of the company’s products. She was working on a deal to sell four hundred Edisons to the Mexican government to cope with an outbreak of swine flu, which would generate immediate revenue. Money was much on the minds of Theranos’ senior management. By the end of 2009, the company had burned through the US$ 47 million raised in its first three rounds of funding and, without a viable product or prospects for sales, would have difficulty keeping the lights on.

But the real bonanza loomed on the horizon in 2010. Drugstore giant Walgreens was interested in expanding their retail business into the “wellness market”: providing in-store health services to their mass market clientèle. Theranos pitched them on offering in-store blood testing. Doctors could send their patients to the local Walgreens to have their blood tested from a simple finger stick and eliminate the need to draw blood in the office or deal with laboratories. With more than 8,000 locations in the U.S., if each were to be equipped with one Edison, the revenue to Theranos (including the single-use testing cartridges) would put them on the map as another Silicon Valley disruptor that went from zero to hundreds of millions in revenue overnight. But here, as well, the Elizabeth effect was in evidence. Of the 192 tests she told Walgreens Theranos could perform, fewer than half were immunoassays the Edisons could run. The rest could be done only on conventional laboratory equipment, and certainly not on a while-you-wait basis.

Walgreens wasn’t the only potential saviour on the horizon. Grocery godzilla Safeway, struggling with sales and earnings which seemed to have reached a peak, saw in-store blood testing with Theranos machines as a high-margin profit centre. They loaned Theranos US$ 30 million and began to plan for installation of blood testing clinics in their stores.

But there was a problem, and as the months wore on, this became increasingly apparent to people at both Walgreens and Safeway, although dismissed by those in senior management under the spell of Elizabeth’s reality distortion field. Deadlines were missed. Simple requests, such as A/B comparison tests run on the Theranos hardware and at conventional labs were first refused, then postponed, then run but results not disclosed. The list of tests which could be run, how blood for them would be drawn, and how they would be processed seemed to dissolve into fog whenever specific requests were made for this information, which was essential for planning the in-store clinics.

There was, indeed, a problem, and it was pretty severe, especially for a start-up which had burned through US$ 50 million and sold nothing. The product didn’t work. Not only could the Edison only run a fraction of the tests its prospective customers had been led by Theranos to believe it could, for those it did run the results were wildly unreliable. The small quantity of blood used in the test introduced random errors due to dilution of the sample; the small tubes in the cartridge were prone to clogging; and capillary blood collected from a finger stick was prone to errors due to “hemolysis”, the rupture of red blood cells, which is minimal in a venous blood draw but so prevalent in finger stick blood it could lead to some tests producing values which indicated the patient was dead.

Meanwhile, people who came to work at Theranos quickly became aware that it was not a normal company, even by the eccentric standards of Silicon Valley. There was an obsession with security, with doors opened by badge readers; logging of employee movement; information restricted to narrow silos prohibiting collaboration between, say, engineering and marketing which is the norm in technological start-ups; monitoring of employee Internet access, E-mail, and social media presence; a security detail of menacing-looking people in black suits and earpieces (which eventually reached a total of twenty); a propensity of people, even senior executives, to “vanish”, Stalin-era purge-like, overnight; and a climate of fear that anybody, employee or former employee, who spoke about the company or its products to an outsider, especially the media, would be pursued, harassed, and bankrupted by lawsuits. There aren’t many start-ups whose senior scientists are summarily demoted and subsequently commit suicide. That happened at Theranos. The company held no memorial for him.

Throughout all of this, a curious presence in the company was Ramesh (“Sunny”) Balwani, a Pakistani-born software engineer who had made a fortune of more than US$ 40 million in the dot-com boom and cashed out before the bust. He joined Theranos in late 2009 as Elizabeth’s second in command and rapidly became known as a hatchet man, domineering boss, and clueless when it came to the company’s key technologies (on one occasion, an engineer mentioned a robotic arm’s “end effector”, after which Sunny would frequently speak of its “endofactor”). Unbeknownst to employees and investors, Elizabeth and Sunny had been living together since 2005. Such an arrangement would be a major scandal in a public company, but even in a private firm, concealing such information from the board and investors is a serious breach of trust.

Let’s talk about the board, shall we? Elizabeth was not only persuasive, but well-connected. She would parley one connection into another, and before long had recruited many prominent figures including:

  • George Schultz (former U.S. Secretary of State)
  • Henry Kissinger (former U.S. Secretary of State)
  • Bill Frist (former U.S. Senator and medical doctor)
  • James Mattis (General, U.S. Marine Corps)
  • Riley Bechtel (Chairman and former CEO, Bechtel Group)
  • Sam Nunn (former U.S. Senator)
  • Richard Kobacevich (former Wells Fargo chairman and CEO)

Later, super-lawyer David Boies would join the board, and lead its attacks against the company’s detractors. It is notable that, as with its investors, not a single board member had experience in medical or diagnostic technology. Bill Frist was an M.D., but his speciality was heart and lung transplants, not laboratory tests.

By 2014, Elizabeth Holmes had come onto the media radar. Photogenic, articulate, and with a story of high-tech disruption of an industry much in the news, she began to be featured as the “female Steve Jobs”, which must have pleased her, since she affected black turtlenecks, kale shakes, and even a car with no license plates to emulate her role model. She appeared on the cover of Fortune in January 2014, made the Forbes list of 400 most wealthy shortly thereafter, was featured in puff pieces in business and general market media, and was named by Time as one of the hundred most influential people in the world. The year 2014 closed with another glowing profile in the New Yorker. This would be the beginning of the end, as it happened to be read by somebody who actually knew something about blood testing.

Adam Clapper, a pathologist in Missouri, spent his spare time writing Pathology Blawg, with a readership of practising pathologists. Clapper read what Elizabeth was claiming to do with a couple of drops of blood from a finger stick and it didn’t pass the sniff test. He wrote a sceptical piece on his blog and, as it passed from hand to hand, he became a lightning rod for others dubious of Theranos’ claims, including those with direct or indirect experience with the company. Earlier, he had helped a Wall Street Journal reporter comprehend the tangled web of medical laboratory billing, and he decided to pass on the tip to the author of this book.

Thus began the unravelling of one of the greatest scams and scandals in the history of high technology, Silicon Valley, and venture investing. At the peak, privately-held Theranos was valued at around US$ 9 billion, with Elizabeth Holmes holding around half of its common stock, and with one of those innovative capital structures of which Silicon Valley is so fond, 99.7% of the voting rights. Altogether, over its history, the company raised around US$ 900 million from investors (including US$ 125 million from Rupert Murdoch in the US$ 430 million final round of funding). Most of the investors’ money was ultimately spent on legal fees as the whole fairy castle crumbled.

The story of the decline and fall is gripping, involving the grandson of a Secretary of State, gumshoes following whistleblowers and reporters, what amounts to legal terrorism by the ever-slimy David Boies, courageous people who stood their ground in the interest of scientific integrity against enormous personal and financial pressure, and the saga of one of the most cunning and naturally talented confidence women ever, equipped with only two semesters of freshman chemical engineering, who managed to raise and blow through almost a billion dollars of other people’s money without checking off the first box on the conventional start-up check list: “Build the product”.

I have, in my career, met three world-class con men. Three times, I (just barely) managed to pick up the warning signs and beg my associates to walk away. Each time I was ignored. After reading this book, I am absolutely sure that had Elizabeth Holmes pitched me on Theranos (about which I never heard before the fraud began to be exposed), I would have been taken in. Walker’s law is “Absent evidence to the contrary, assume everything is a scam”. A corollary is “No matter how cautious you are, there’s always a confidence man (or woman) who can scam you if you don’t do your homework.”

Carreyrou, John. Bad Blood. New York: Alfred A. Knopf, 2018. ISBN 978-1-984833-63-1.

Here is Elizabeth Holmes at Stanford in 2013, when Theranos was riding high and she was doing her “female Steve Jobs” act.

This is a CNN piece, filmed after the Theranos scam had begun to collapse, in which you can still glimpse the Elizabeth Holmes reality distortion field at full intensity directed at CNN medical correspondent Sanjay Gupta. There are several curious things about this video. The machine that Gupta is shown is the “miniLab”, a prototype second-generation machine which never worked acceptably, not the Edison, which was actually used in the Walgreens and Safeway tests. Gupta’s blood is drawn and tested, but the process used to perform the test is never shown. The result reported is a cholesterol test, but the Edison cannot perform such tests. In the plans for the Walgreens and Safeway roll-outs, such tests were performed on purchased Siemens analysers which had been secretly hacked by Theranos to work with blood diluted well below their regulatory-approved specifications (the dilution was required due to the small volume of blood from the finger stick). Since the miniLab never really worked, the odds are that Gupta’s blood was tested on one of the Siemens machines, not a Theranos product at all.

In a June 2018 interview, author John Carreyrou recounts the story of Theranos and his part in revealing the truth.  There is substantial background information in the question and answer period which does not appear in the book.

If you are a connoisseur of the art of the con, here is a masterpiece. After the Wall Street Journal exposé had broken, after retracting tens of thousands of blood tests, and after Theranos had been banned from running a clinical laboratory by its regulators, Holmes got up before an audience of 2500 people at the meeting of the American Association of Clinical Chemistry and turned up the reality distortion field to eleven. Watch a master at work. She comes on the stage at the six minute mark.

Like 12+

Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Project Cyclops

Project Cyclops: Full arrayThere are few questions in science as simple to state and profound in their implications as “are we alone?”—are humans the only species with a technological civilisation in the galaxy, or in the universe?  This has been a matter of speculation by philosophers, theologians, authors of fiction, and innumerable people gazing at the stars since antiquity, but it was only in the years after World War II, which had seen the development of high-power microwave transmitters and low-noise receivers for radar, that it dawned upon a few visionaries that this had now become a question which could be scientifically investigated.

The propagation of radio waves through the atmosphere and the interstellar medium is governed by basic laws of physics, and the advent of radio astronomy demonstrated that many objects in the sky, some very distant, could be detected in the microwave spectrum.  But if we were able to detect these natural sources, suppose we connected a powerful transmitter to our radio telescope and sent a signal to a nearby star?  It was easy to calculate that, given the technology of the time (around 1960), existing microwave transmitters and radio telescopes could transmit messages across interstellar distances.

But, it’s one thing to calculate that intelligent aliens with access to microwave communication technology equal or better than our own could communicate over the void between the stars, and entirely another to listen for those communications.  The problems are simple to understand but forbidding to face: where do you point your antenna, and where do you tune your dial?  There are on the order of a hundred billion stars in our galaxy.  We now know, as early researchers suspected without evidence, that most of these stars have planets, some of which may have conditions suitable for the evolution of intelligent life.  Suppose aliens on one of these planets reach a level of technological development where they decide to join the “Galactic Club” and transmit a beacon which simply says “Yo!  Anybody out there?”  (The beacon would probably announce a signal with more information which would be easy to detect once you knew where to look.)  But for the beacon to work, it would have to be aimed at candidate stars where others might be listening (a beacon which broadcasted in all directions—an “omnidirectional beacon”—would require so much energy or be limited to such a short range as to be impractical for civilisations with technology comparable to our own).

Then there’s the question of how many technological communicating civilisations there are in the galaxy.  Note that it isn’t enough that a civilisation have the technology which enables it to establish a beacon: it has to do so.  And it is a sobering thought that more than six decades after we had the ability to send such a signal, we haven’t yet done so.  The galaxy may be full of civilisations with our level of technology and above which have the same funding priorities we do and choose to spend their research budget on intersectional autoethnography of transgender marine frobdobs rather than communicating with nerdy pocket-protector types around other stars who tediously ask Big Questions.

And suppose a civilisation decides it can find the spare change to set up and operate a beacon, inviting others to contact it.  How long will it continue to transmit, especially since it’s unlikely, given the finite speed of light and the vast distances between the stars, there will be a response in the near term?  Before long, scruffy professors will be marching in the streets wearing frobdob hats and rainbow tentacle capes, and funding will be called into question.  This is termed the “lifetime” of a communicating civilisation, or L, which is how long that civilisation transmits and listens to establish contact with others.  If you make plausible assumptions for the other parameters in the Drake equation (which estimates how many communicating civilisations there are in the galaxy), a numerical coincidence results in the estimate of the number of communicating civilisations in the galaxy being roughly equal to their communicating life in years, L.  So, if a typical civilisation is open to communication for, say, 10,000 years before it gives up and diverts its funds to frobdob research, there will be around 10,000 such civilisations in the galaxy.  With 100 billion stars (and around as many planets which may be hosts to life), that’s a 0.00001% chance that any given star where you point your antenna may be transmitting, and that has to be multiplied by the same probability they are transmitting their beacon in your direction while you happen to be listening.  It gets worse.  The galaxy is huge—around 150 million light years in diameter, and our technology can only communicate with comparable civilisations out to a tiny fraction of this, say 1000 light years for high-power omnidirectional beacons, maybe ten to a hundred times that for directed beacons, but then you have the constraint that you have to be listening in their direction when they happen to be sending.

It seems hopeless.  It may be.  But the 1960s were a time very different from our constrained age.  Back then, if you had a problem, like going to the Moon in eight years, you said, “Wow!  That’s a really big nail.  How big a hammer do I need to get the job done?”  Toward the end of that era when everything seemed possible, NASA convened a summer seminar at Stanford University to investigate what it would take to seriously investigate the question of whether we are alone.  The result was Project Cyclops: A Design Study of a System for Detecting Extraterrestrial Intelligent Life, prepared in 1971 and issued as a NASA report (no Library of Congress catalogue number or ISBN was assigned) in 1973; the link will take you to a NASA PDF scan of the original document, which is in the public domain.  The project assembled leading experts in all aspects of the technologies involved: antennas, receivers, signal processing and analysis, transmission and control, and system design and costing.

Project Cyclops: Ground level viewThey approached the problem from what might be called the “Apollo perspective”: what will it cost, given the technology we have in hand right now, to address this question and get an answer within a reasonable time?  What they came up with was breathtaking, although no more so than Apollo.  If you want to listen for beacons from communicating civilisations as distant as 1000 light years and incidental transmissions (“leakage”, like our own television and radar emissions) within 100 light years, you’re going to need a really big bucket to collect the signal, so they settled on 1000 dishes, each 100 metres in diameter.  Putting this into perspective, 100 metres is about the largest steerable dish anybody envisioned at the time, and they wanted to build a thousand of them, densely packed.

But wait, there’s more.  These 1000 dishes were not just a huge bucket for radio waves, but a phased array, where signals from all of the dishes (or a subset, used to observe multiple targets) were combined to provide the angular resolution of a single dish the size of the entire array.  This required breathtaking precision of electronic design at the time which is commonplace today (although an array of 1000 dishes spread over 16 km would still give most designers pause).  The signals that might be received would not be fixed in frequency, but would drift due to Doppler shifts resulting from relative motion of the transmitter and receiver.  With today’s computing hardware, digging such a signal out of the raw data is something you can do on a laptop or mobile phone, but in 1971 the best solution was an optical data processor involving exposing, developing, and scanning film.  It was exquisitely clever, although obsolete only a few years later, but recall the team had agreed to use only technologies which existed at the time of their design.  Even more amazing (and today, almost bizarre) was the scheme to use the array as an imaging telescope.  Again, with modern computers, this is a simple matter of programming, but in 1971 the designers envisioned a vast hall in which the signals from the antennas would be re-emitted by radio transmitters which would interfere in free space and produce an intensity image on an image surface where it would be measured by an array of receiver antennæ.

What would all of this cost?  Lots—depending upon the assumptions used in the design (the cost was mostly driven by the antenna specifications, where extending the search to shorter wavelengths could double the cost, since antennas had to be built to greater precision) total system capital cost was estimated as between 6 and 10 billion dollars (1971).  Converting this cost into 2018 dollars gives a cost between 37 and 61 billion dollars.  (By comparison, the Apollo project cost around 110 billion 2018 dollars.)  But since the search for a signal may “almost certainly take years, perhaps decades and possibly centuries”, that initial investment must be backed by a long-term funding commitment to continue the search, maintain the capital equipment, and upgrade it as technology matures.  Given governments’ record in sustaining long-term efforts in projects which do not line politicians’ or donors’ pockets with taxpayer funds, such perseverance is not the way to bet.  Perhaps participants in the study should have pondered how to incorporate sufficient opportunities for graft into the project, but even the early 1970s were still an idealistic time when we didn’t yet think that way.

This study is the founding document of much of the work in the Search for Extraterrestrial Intelligence (SETI) conducted in subsequent decades.  Many researchers first realised that answering this question, “Are we alone?”, was within our technological grasp when chewing through this difficult but inspiring document.  (If you have an equation or chart phobia, it’s not for you; they figure on the majority of pages.)  The study has held up very well over the decades.  There are a number of assumptions we might wish to revise today (for example, higher frequencies may be better for interstellar communication than were assumed at the time, and spread spectrum transmissions may be more energy efficient than the extreme narrowband beacons assumed in the Cyclops study).

Despite disposing of wealth, technological capability, and computing power of which authors of the Project Cyclops report never dreamed, we only make little plans today.  Most readers of this post, in their lifetimes, have experienced the expansion of their access to knowledge in the transition from being isolated to gaining connectivity to a global, high-bandwidth network.  Imagine what it means to make the step from being confined to our single planet of origin to being plugged in to the Galactic Web, exchanging what we’ve learned with a multitude of others looking at things from entirely different perspectives.  Heck, you could retire the entire capital and operating cost of Project Cyclops in the first three years just from advertising revenue on frobdob videos!  (Did I mention they have very large eyes which are almost all pupil?  Never mind the tentacles.)

Oliver, Bernard M., John Billingham, et al.  Project Cyclops [PDF].  Stanford, CA: Stanford/NASA Ames Research Center, 1971.  NASA-CR-114445 N73-18822.

This document has been subjected to intense scrutiny over the years.  The SETI League maintains a comprehensive errata list for the publication.

Here is a recent conversation among SETI researchers on the state of the art and future prospects for SETI with ground-based telescopes.

This is a two part lecture on the philosophy of the existence and search for extraterrestrial beings from antiquity to the present day.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Orbits in Strongly Curved Spacetime

Your browser does not support HTML5 canvas.
Angular momentum: Mass: Max radius:
    Click the title of this post to see the interactive simulation.

Introduction

The display above shows, from three different physical perspectives, the orbit of a low-mass test particle, the small red circle, around a non-rotating black hole (represented by a grey circle in the panel at the right), where the radius of the circle is the black hole’s gravitational radius, or event horizon. Kepler’s laws of planetary motion, grounded in Newton’s theory of gravity, state that the orbit of a test particle around a massive object is an ellipse with one focus at the centre of the massive object. But when gravitational fields are strong, as is the case for collapsed objects like neutron stars and black holes, Newton’s theory is inaccurate; calculations must be done using Einstein’s theory of General Relativity.

In Newtonian gravitation, an orbit is always an ellipse. As the gravitating body becomes more massive and the test particle orbits it more closely, the speed of the particle in its orbit increases without bound, always balancing the gravitational force. For a black hole, Newton’s theory predicts orbital velocities greater than the speed of light, but according to Einstein’s Special Theory of Relativity, no material object can achieve or exceed the speed of light. In strong gravitational fields, General Relativity predicts orbits drastically different from the ellipses of Kepler’s laws. This article allows you to explore them.

The Orbit Plot

Orbit PlotThe panel at the right of the animation shows the test mass orbiting the black hole, viewed perpendicular to the plane of its orbit. The path of the orbit is traced by the green line. After a large number of orbits the display will get cluttered; just click the mouse anywhere in the right panel to erase the path and start over. When the test mass reaches its greatest distance from the black hole, a yellow line is plotted from the centre of the black hole to that point, the apastron of the orbit. In Newtonian gravity, the apastron remains fixed in space. The effects of General Relativity cause it to precess. You can see the degree of precession in the displacement of successive yellow lines (precession can be more than 360°; the yellow line only shows precession modulo one revolution).

The Gravitational Effective-Potential

Effective potentialThe two panels at the left of the animation display the orbit in more abstract ways. The Effective Potential plot at the top shows the position of the test mass on the gravitational energy curve as it orbits in and out. The summit on the left side of the curve is unique to General Relativity—in Newtonian gravitation the curve rises without bound as the radius decreases, approaching infinity at zero. In Einstein’s theory, the inability of the particle to orbit at or above the speed of light creates a “pit in the potential” near the black hole. As the test mass approaches this summit, falling in from larger radii with greater and greater velocity, it will linger near the energy peak for an increasingly long time, while its continued angular motion will result in more and more precession. If the particle passes the energy peak and continues to lesser radii, toward the left, its fate is sealed—it will fall into the black hole and be captured.

The Gravity Well

Gravity wellSpacetime around an isolated spherical non-rotating uncharged gravitating body is described by Schwarzschild Geometry, in which spacetime can be thought of as being bent by the presence of mass. This creates a gravity well which extends to the surface of the body or, in the case of a black hole, to oblivion. The gravity well has the shape of a four-dimensional paraboloid of revolution, symmetrical about the central mass. Since few Web browsers are presently equipped with four-dimensional display capability, I’ve presented a two-dimensional slice through the gravity well in the panel at the bottom left of the animation. Like the energy plot above, the left side of the panel represents the centre of the black hole and the radius increases to the right. Notice that the test mass radius moves in lockstep on the Effective-Potential and Gravity Well charts, as the radius varies on the orbit plot to their right.

The gravity well of a Schwarzschild black hole has a throat at a radius determined solely by its mass—that is the location of the hole’s event horizon; any matter or energy which crosses the horizon is captured. The throat is the leftmost point on the gravity well curve, where the slope of the paraboloidal geometry becomes infinite (vertical). With sufficient angular momentum, a particle can approach the event horizon as closely as it wishes (assuming it is small enough so it isn’t torn apart by tidal forces), but it can never cross the event horizon and return.

Hands On

Orbits in Strongly Curved Spacetime control panel

By clicking in the various windows and changing values in the controls at the bottom of the window you can explore different scenarios. To pause the simulation, press the Pause button; pressing it again resumes the simulation. Click anywhere in the orbit plot at the right to clear the orbital trail and apastron markers when the screen becomes too cluttered. You can re-launch the test particle at any given radius from the black hole (with the same angular momentum) by clicking at the desired radius in either the Effective Potential or Gravity Well windows. The green line in the Effective Potential plot indicates the energy minimum at which a stable circular orbit exists for a particle of the given angular momentum.

The angular momentum is specified by the box at left in terms of the angular momentum per unit mass of the black hole, all in geometric units—all of this is explained in detail below. What’s important to note is that for orbits like those of planets in the Solar System, this number is huge; only in strong gravitational fields does it approach small values. If the angular momentum is smaller than a critical value (\(2\sqrt 3\), about 3.464 for a black hole of mass 1, measured in the same units), no stable orbits exist; the particle lacks the angular momentum to avoid being swallowed. When you enter a value smaller than this, notice how the trough in the energy curve and the green line marking the stable circular orbit disappear. Regardless of the radius, any particle you launch is doomed to fall into the hole.

The Mass box allows you to change the mass of the black hole, increasing the radius of its event horizon. Since the shape of the orbit is determined by the ratio of the angular momentum to the mass, it’s just as easy to leave the mass as 1 and change the angular momentum. You can change the scale of all the panels by entering a new value for the maximum radius; this value becomes the rightmost point in the effective potential and gravity well plots and the distance from the centre of the black hole to the edge of the orbit plot. When you change the angular momentum or mass, the radius scale is automatically adjusted so the stable circular orbit (if any) is on screen.

Kepler, Newton, and Beyond

In the early 17th century, after years of tedious calculation and false starts, Johannes Kepler published his three laws of planetary motion:

  • First law (1605): A planet’s orbit about the Sun is an ellipse, with the Sun at one focus.
  • Second law (1604): A line from the Sun to a planet sweeps out equal areas in equal times.
  • Third law (1618): The square of the orbital period of a planet is proportional to the cube of the major axis of the orbit.

Kepler’s discoveries about the behaviour of planets in their orbits played an essential rôle in Isaac Newton’s formulation of the law of universal gravitation in 1687. Newton’s theory showed the celestial bodies were governed by the same laws as objects on Earth. The philosophical implications of this played as key a part in the Enlightenment as did the theory itself in the subsequent development of physics and astronomy.

While Kepler’s laws applied only to the Sun and planets, Newton’s universal theory allowed one to calculate the gravitational force and motion of any bodies whatsoever. To be sure, when many bodies were involved and great accuracy was required, the calculations were horrifically complicated and tedious—so much so that those reared in the computer age may find it difficult to imagine embarking upon them armed with nothing but a table of logarithms, pencil and paper, and the human mind. But performed they were, with ever greater precision as astronomers made increasingly accurate observations. And those observations agreed perfectly with the predictions of Newton’s theory.

Well,… almost perfectly. After painstaking observations of the planets and extensive calculation, astronomer Simon Newcomb concluded in 1898 that the orbit of Mercury was precessing 43 arc-seconds per century more than could be explained by the influence of the other planets. This is a tiny discrepancy, but further observations and calculations confirmed Newcomb’s—the discrepancy was real. Some suggested a still undiscovered planet closer to the Sun than Mercury (and went so far as to name it, sight unseen, “Vulcan”), but no such planet was ever found, nor any other plausible explanation advanced. For nearly twenty years Mercury’s precession or “perihelion advance” remained one of those nagging anomalies in the body of scientific data that’s trying to tell us something, if only we knew what.

In 1915, Albert Einstein’s General Theory of Relativity extended Newtonian gravitation theory, revealing previously unanticipated subtleties of nature. And Einstein’s theory explained the perihelion advance of Mercury. That tiny discrepancy in the orbit of Mercury was actually the first evidence for what lay beyond Newtonian gravitation, the first step down a road that would lead to understanding black holes, gravitational radiation, and the source of inertia, which remains a fertile ground for theoretical and experimental physics a century thereafter.

If we’re interested in the domain where general relativistic effects are substantial, we’re better off calculating with units scaled to the problem. A particularly convenient and elegant choice is the system of geometric units, obtained by setting Newton’s gravitational constant G, the speed of light c, and Boltzmann’s constant k all equal to 1. We can then express any of the following units as a length in centimetres by multiplying by the following conversion factors.

Geometric units

The enormous exponents make it evident that these units are far removed from our everyday experience. It would be absurd to tell somebody, “I’ll call you back in \(1.08\times 10^{14}\) centimetres”, but it is a perfectly valid way of saying “one hour”. The discussion that follows uses geometric units throughout, allowing us to treat mass, time, length, and energy without conversion factors. To express a value calculated in geometric units back to conventional units, just divide by the value in the table above.

The Gravitational Effective-Potential

Effective potential

The gravitational effective-potential for a test particle orbiting in a Schwarzschild geometry is:

V²(L,r)=1-2M/r)(1+L²/r²)

where \(\tilde{L}\) is the angular momentum per unit rest mass expressed in geometric units, M is the mass of the gravitating body, and r is the radius of the test particle from the centre of the body.

The radius of a particle from the centre of attraction evolves in proper time τ (time measured by a clock moving along with the particle) according to:

(dr/dTAU)² + V(L,r) = E²

where \(\tilde{E}\) is the potential energy of the test mass at infinity per rest mass.

Angular motion about the centre of attraction is then:

dPHI/dTAU = L/r²

while time, as measured by a distant observer advances according to:

dt/dTAU = E / (1 - 2M/r)

and can be seen to slow down as the event horizon at the gravitational radius is approached. At the gravitational radius of 2M time, as measured from far away, stops entirely so the particle never seems to reach the event horizon. Proper time on the particle continues to advance unabated; an observer on-board sails through the event horizon without a bump (or maybe not) and continues toward the doom which awaits at the central singularity.

Circular Orbits

Circular orbits are possible at maxima and minima of the effective-potential. Orbits at minima are stable, since a small displacement increases the energy and thus creates a restoring force in the opposite direction. Orbits at maxima are unstable; the slightest displacement causes the particle to either be sucked into the black hole or enter a highly elliptical orbit around it.

To find the radius of possible circular orbits, differentiate the gravitational effective-potential with respect to the radius r:

DV²/dr = (2 (3L²M - L²r + Mr²)) / (r^4)

The minima and maxima of a function are at the zero crossings of its derivative, so a little algebra gives the radii of possible circular orbits as:

(L(L ± sqrt(L² - 12M²)) / 2M

The larger of these solutions is the innermost stable circular orbit, while the smaller is the unstable orbit at the maximum. For a black hole, this radius will be outside the gravitational radius at 2M, while for any other object the radius will be less than the diameter of the body, indicating no such orbit exists. If the angular momentum L² is less than 12M², no stable orbit exists; the object will impact the surface or, in the case of a black hole, fall past the event horizon and be swallowed.

References

Gallmeier, Jonathan, Mark Loewe, and Donald W. Olson. “Precession and the Pulsar.” Sky & Telescope (September 1995): 86–88.
A BASIC program which plots orbital paths in Schwarzschild geometry. The program uses different parameters to describe the orbit than those used here, and the program does not simulate orbits which result in capture or escape. This program can be downloaded from the Sky & Telescope Web site.
Misner, Charles W., Kip S. Thorne, and John Archibald Wheeler. Gravitation. San Francisco: W. H. Freeman, 1973. ISBN 978-0-7167-0334-1.
Chapter 25 thoroughly covers all aspects of motion in Schwarzschild geometry, both for test particles with mass and massless particles such as photons.
Wheeler, John Archibald. A Journey into Gravity and Spacetime. New York: W. H. Freeman, 1990. ISBN 978-0-7167-5016-1.
This book, part of the Scientific American Library series (but available separately), devotes chapter 10 to a less technical discussion of orbits in Schwarzschild spacetime. The “energy hill” on page 173 and the orbits plotted on page 176 provided the inspiration for this page.

Here is a short video about orbiting a black hole:

This is a 45 minute lecture on black holes and the effects they produce.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Antifragile

“Antifragile” by Nassim Nicholas TalebThis book is volume three in the author’s Incerto series, following Fooled by Randomness  and The Black Swan. It continues to explore the themes of randomness, risk, and the design of systems: physical, economic, financial, and social, which perform well in the face of uncertainty and infrequent events with large consequences. He begins by posing the deceptively simple question, “What is the antonym of ‘fragile’?”

After thinking for a few moments, most people will answer with “robust” or one of its synonyms such as “sturdy”, “tough”, or “rugged”. But think about it a bit more: does a robust object or system actually behave in the opposite way to a fragile one? Consider a teacup made of fine china. It is fragile—if subjected to more than a very limited amount of force or acceleration, it will smash into bits. It is fragile because application of such an external stimulus, for example by dropping it on the floor, will dramatically degrade its value for the purposes for which it was created (you can’t drink tea from a handful of sherds, and they don’t look good sitting on the shelf). Now consider a teacup made of stainless steel. It is far more robust: you can drop it from ten kilometres onto a concrete slab and, while it may be slightly dented, it will still work fine and look OK, maybe even acquiring a little character from the adventure. But is this really the opposite of fragility? The china teacup was degraded by the impact, while the stainless steel one was not. But are there objects and systems which improve as a result of random events: uncertainty, risk, stressors, volatility, adventure, and the slings and arrows of existence in the real world? Such a system would not be robust, but would be genuinely “anti-fragile” (which I will subsequently write without the hyphen, as does the author): it welcomes these perturbations, and may even require them in order to function well or at all.

Antifragility seems an odd concept at first. Our experience is that unexpected events usually make things worse, and that the inexorable increase in entropy causes things to degrade with time: plants and animals age and eventually die; machines wear out and break; cultures and societies become decadent, corrupt, and eventually collapse. And yet if you look at nature, antifragility is everywhere—it is the mechanism which drives biological evolution, technological progress, the unreasonable effectiveness of free market systems in efficiently meeting the needs of their participants, and just about everything else that changes over time, from trends in art, literature, and music, to political systems, and human cultures. In fact, antifragility is a property of most natural, organic systems, while fragility (or at best, some degree of robustness) tends to characterise those which were designed from the top down by humans. And one of the paradoxical characteristics of antifragile systems is that they tend to be made up of fragile components.

How does this work? We’ll get to physical systems and finance in a while, but let’s start out with restaurants. Any reasonably large city in the developed world will have a wide variety of restaurants serving food from numerous cultures, at different price points, and with ambience catering to the preferences of their individual clientèles. The restaurant business is notoriously fragile: the culinary preferences of people are fickle and unpredictable, and restaurants who are behind the times frequently go under. And yet, among the population of restaurants in a given area at a given time, customers can usually find what they’re looking for. The restaurant population or industry is antifragile, even though it is composed of fragile individual restaurants which come and go with the whims of diners, which will be catered to by one or more among the current, but ever-changing population of restaurants.

Now, suppose instead that some Food Commissar in the All-Union Ministry of Nutrition carefully studied the preferences of people and established a highly-optimised and uniform menu for the monopoly State Feeding Centres, then set up a central purchasing, processing, and distribution infrastructure to optimise the efficient delivery of these items to patrons. This system would be highly fragile, since while it would deliver food, there would no feedback based upon customer preferences, and no competition to respond to shifts in taste. The result would be a mediocre product which, over time, was less and less aligned with what people wanted, and hence would have a declining number of customers. The messy and chaotic market of independent restaurants, constantly popping into existence and disappearing like virtual particles, exploring the culinary state space almost at random, does, at any given moment, satisfy the needs of its customers, and it responds to unexpected changes by adapting to them: it is antifragile.

Now let’s consider an example from metallurgy. If you pour molten metal from a furnace into a cold mould, its molecules, which were originally jostling around at random at the high temperature of the liquid metal, will rapidly freeze into a structure with small crystals randomly oriented. The solidified metal will contain dislocations wherever two crystals meet, with each forming a weak spot where the metal can potentially fracture under stress. The metal is hard, but brittle: if you try to bend it, it’s likely to snap. It is fragile.

To render it more flexible, it can be subjected to the process of annealing, where it is heated to a high temperature (but below melting), which allows the molecules to migrate within the bulk of the material. Existing grains will tend to grow, align, and merge, resulting in a ductile, workable metal. But critically, once heated, the metal must be cooled on a schedule which provides sufficient randomness (molecular motion from heat) to allow the process of alignment to continue, but not to disrupt already-aligned crystals. Here is a video from Cellular Automata Laboratory which demonstrates annealing. Note how sustained randomness is necessary to keep the process from quickly “freezing up” into a disordered state.

As it happens, last month’s Saturday Night Science discussed solving the travelling salesman problem through the technique of simulated annealing, which is analogous to annealing metal, and like it, is a manifestation of antifragility—it doesn’t work without randomness.

When you observe a system which adapts and prospers in the face of unpredictable changes, it will almost always do so because it is antifragile. This is a large part of how nature works: evolution isn’t able to predict the future and it doesn’t even try. Instead, it performs a massively parallel, planetary-scale search, where organisms, species, and entire categories of life appear and disappear continuously, but with the ecosystem as a whole constantly adapting itself to whatever inputs may perturb it, be they a wholesale change in the composition of the atmosphere (the oxygen catastrophe at the beginning of the Proterozoic eon around 2.45 billion years ago), asteroid and comet impacts, and ice ages.

Most human-designed systems, whether machines, buildings, political institutions, or financial instruments, are the antithesis of those found in nature. They tend to be highly-optimised to accomplish their goals with the minimum resources, and to be sufficiently robust to cope with any stresses they may be expected to encounter over their design life. These systems are not antifragile: while they may be designed not to break in the face of unexpected events, they will, at best, survive, but not, like nature, often benefit from them.

The devil’s in the details, and if you reread the last paragraph carefully, you may be able to see the horns and pointed tail peeking out from behind the phrase “be expected to”. The problem with the future is that it is full of all kinds of events, some of which are un-expected, and whose consequences cannot be calculated in advance and aren’t known until they happen. Further, there’s usually no way to estimate their probability. It doesn’t even make any sense to talk about the probability of something you haven’t imagined could happen. And yet such things happen all the time.

Today, we are plagued, in many parts of society, with “experts” the author dubs fragilistas. Often equipped with impeccable academic credentials and with powerful mathematical methods at their fingertips, afflicted by the “Soviet-Harvard delusion” (overestimating the scope of scientific knowledge and the applicability of their modelling tools to the real world), they are blind to the unknown and unpredictable, and they design and build systems which are highly fragile in the face of such events. A characteristic of fragilista-designed systems is that they produce small, visible, and apparently predictable benefits, while incurring invisible risks which may be catastrophic and occur at any time.

Let’s consider an example from finance. Suppose you’re a conservative investor interested in generating income from your lifetime’s savings, while preserving capital to pass on to your children. You might choose to invest, say, in a diversified portfolio of stocks of long-established companies in stable industries which have paid dividends for 50 years or more, never skipping or reducing a dividend payment. Since you’ve split your investment across multiple companies, industry sectors, and geographical regions, your risk from an event affecting one of them is reduced. For years, this strategy produces a reliable and slowly growing income stream, while appreciation of the stock portfolio (albeit less than high flyers and growth stocks, which have greater risk and pay small dividends or none at all) keeps you ahead of inflation. You sleep well at night.

Then 2008 rolls around. You didn’t do anything wrong. The companies in which you invested didn’t do anything wrong. But the fragilistas had been quietly building enormous cross-coupled risk into the foundations of the financial system (while pocketing huge salaries and bonuses, while bearing none of the risk themselves), and when it all blows up, in one sickening swoon, you find the value of your portfolio has been cut by 50%. In a couple of months, you have lost half of what you worked for all of your life. Your “safe, conservative, and boring” stock portfolio happened to be correlated with all of the other assets, and when the foundation of the system started to crumble, suffered along with them. The black swan landed on your placid little pond.

What would an antifragile investment portfolio look like, and how would it behave in such circumstances? First, let’s briefly consider a financial option. An option is a financial derivative contract which gives the purchaser the right, but not the obligation, to buy (“call option”) or sell (”put option”) an underlying security (stock, bond, market index, etc.) at a specified price, called the “strike price” (or just “strike”). If the a call option has a strike above, or a put option a strike below, the current price of the security, it is called “out of the money”, otherwise it is “in the money”. The option has an expiration date, after which, if not “exercised” (the buyer asserts his right to buy or sell), the contract expires and the option becomes worthless.

Let’s consider a simple case. Suppose Consolidated Engine Sludge (SLUJ) is trading for US$10 per share on June 1, and I buy a call option to buy 100 shares at US$15/share at any time until August 31. For this right, I might pay a premium of, say, US$7. (The premium depends upon sellers’ perception of the volatility of the stock, the term of the option, and the difference between the current price and the strike price.) Now, suppose that sometime in August, SLUJ announces a breakthrough that allows them to convert engine sludge into fructose sweetener, and their stock price soars on the news to US$19/share. I might then decide to sell on the news, exercise the option, paying US$1500 for the 100 shares, and then immediately sell them at US$19, realising a profit of US$400 on the shares or, subtracting the cost of the option, US$393 on the trade. Since my original investment was just US$7, this represents a return of 5614% on the original investment, or 22457% annualised. If SLUJ never touches US$15/share, come August 31, the option will expire unexercised, and I’m out the seven bucks. (Since options can be bought and sold at any time and prices are set by the market, it’s actually a bit more complicated than that, but this will do for understanding what follows.)

You might ask yourself what would motivate somebody to sell such an option. In many cases, it’s an attractive proposition. If I’m a long-term shareholder of SLUJ and have found it to be a solid but non-volatile stock that pays a reasonable dividend of, say, two cents per share every quarter, by selling the call option with a strike of 15, I pocket an immediate premium of seven cents per share, increasing my income from owning the stock by a factor of 4.5. For this, I give up the right to any appreciation should the stock rise above 15, but that seems to be a worthwhile trade-off for a stock as boring as SLUJ (at least prior to the news flash).

A put option is the mirror image: if I bought a put on SLUJ with a strike of 5, I’ll only make money if the stock falls below 5 before the option expires.

Now we’re ready to construct a genuinely antifragile investment. Suppose I simultaneously buy out of the money put and call options on the same security, a so-called “long straddle”. Now, as long as the price remains within the strike prices of the put and call, both options will expire worthless, but if the price either rises above the call strike or falls below the put strike, that option will be in the money and pay off the further the underlying price veers from the band defined by the two strikes. This is, then, a pure bet on volatility: it loses a small amount of money as long as nothing unexpected happens, but when a shock occurs, it pays off handsomely.

Now, the premiums on deep out of the money options are usually very modest, so an investor with a portfolio like the one I described who was clobbered in 2008 could have, for a small sum every quarter, purchased put and call options on, say, the Standard & Poor’s 500 stock index, expecting to usually have them expire worthless, but under the circumstance which halved the value of his portfolio, would pay off enough to compensate for the shock. (If worried only about a plunge he could, of course, have bought just the put option and saved money on premiums, but here I’m describing a pure example of antifragility being used to cancel fragility.)

I have only described a small fraction of the many topics covered in this masterpiece, and described none of the mathematical foundations it presents (which can be skipped by readers intimidated by equations and graphs). Fragility and antifragility is one of those concepts, simple once understood, which profoundly change the way you look at a multitude of things in the world. When a politician, economist, business leader, cultural critic, or any other supposed thinker or expert advocates a policy, you’ll learn to ask yourself, “Does this increase fragility?” and have the tools to answer the question. Further, it provides an intellectual framework to support many of the ideas and policies which libertarians and advocates of individual liberty and free markets instinctively endorse, founded in the way natural systems work. It is particularly useful in demolishing “green” schemes which aim at replacing the organic, distributed, adaptive, and antifragile mechanisms of the market with coercive, top-down, and highly fragile central planning which cannot possibly have sufficient information to work even in the absence of unknowns in the future.

There is much to digest here, and the ramifications of some of the clearly-stated principles take some time to work out and fully appreciate. Indeed, I spent more than five years reading this book, a little bit at a time. It’s worth taking the time and making the effort to let the message sink in and figure out how what you’ve learned applies to your own life and act accordingly. As Fat Tony says, “Suckers try to win arguments; nonsuckers try to win.”

Taleb, Nassim Nicholas. Antifragile. New York: Random House, 2012. ISBN 978-0-8129-7968-8.

Here is a lecture by the author about the principles discussed in the book.

Like 14+

Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Simulated Annealing

Your browser does not support HTML5 canvas.
cities
path length   River cost

Travelling Salesman

(Click Read More to view the interactive simulation in this post.)
The travelling salesman problem is one of the most-studied problems in combinatorial optimisation. It couldn’t be easier to state:

Given a list of cities and their locations (usually specified as Cartesian co-ordinates on a plane), what is the shortest itinerary which will visit every city exactly once and return to the point of origin?

Easy to ask, but devilishly difficult to answer…. The obvious way to solve the travelling salesman problem would be to write down all of the possible sequences in which the cities could be visited, compute the distance of each path, and then choose the smallest. But the number of possible itineraries for visiting n cities grows as the factorial of n, which is written, appropriately as “n!”. The factorial of a positive integer is the product of that number and all smaller numbers down to one. Hence 2!=2, 3!=6, 6!=720, and 10!=3,628,800. As you can see, these numbers grow very rapidly, so as you increase the number of cities, the number of paths you have to compare blows up in a combinatorial explosion which makes finding the optimal path by brute force computation a hopeless undertaking.

“But”, you ask, “computers are getting faster every year. Why not just be patient and wait a few years?” Neither you, nor I, nor the universe has sufficient patience. The box at the top of this page contains thirty cities represented by red balls placed at random in the grey square, connected by a path drawn in blue lines in the order in which they were placed. Every time you press the “Place” button, thirty new randomly-placed cities are generated; you can change the number by setting the box to the right of the button. But let’s stick with thirty cities for the nonce.

The number of possible paths along which we can visit the thirty cities is equal to the number of permutations of a set of thirty distinct members, which is equal to the factorial of the number of members, or 30!. This is a breathtakingly large number.

30! = 265,252,859,812,191,058,636,308,480,000,000 ≈ 2.6525×1032

Now, let’s assume you had a supercomputer which was able to compute the value of a billion (109) paths per second. Chugging away at this task around the clock, without a day of rest, it would take 2.65×1023 seconds to get through the list. How long is that? About 8.4 quadrillion (1015) years, or about 600,000 times the present age of the universe. And if you modestly increased the number of cities to fifty? Prepare to wait eight thousand billion billion times the age of the universe for the answer.

Solution for 30 citiesNow scroll back up to the top of the page and click the “Solve” button Almost instantaneously, you’ll see a near-optimal path to tour the thirty cities with the least distance of travel. Try clicking “Place” and then “Solve” several times to create and solve new problems, then increase the number of cities to 50 and then 100 and try solving those problems. In each case, the solution appears in a fraction of a second. Now, these solutions are not guaranteed to be absolutely optimal; they may be a percent or two longer than the absolute best path (if you click “Solve” multiple times, you may see several different solutions, all of which are close in total path length). They’re not perfect, but then you don’t have to wait huge multiples of the age of the universe for the result. How did we do it?

Simulated Annealing

This page attacks the travelling salesman problem through a technique of combinatorial optimisation called simulated annealing. By analogy with the process of annealing a material such as metal or glass by raising it to a high temperature and then gradually reducing the temperature, allowing local regions of order to grow outward, increasing ductility and reducing stresses in the material, the algorithm randomly perturbs the original path to a decreasing extent according to a gradually decreasing logical “temperature”.

In simulated annealing, the equivalent of temperature is a measure of the randomness by which changes are made to the path, seeking to minimise it. When the temperature is high, larger random changes are made, avoiding the risk of becoming trapped in a local minimum (of which there are usually many in a typical travelling salesman problem), then homing in on a near-optimal minimum as the temperature falls. The temperature falls in a series of steps on an exponential decay schedule where, on each step, the temperature is 0.9 times that of the previous step.

The process of annealing starts with a path which simply lists all of the cities in the order their positions were randomly selected (this is the path you’ll see after pressing the “Place” button). On each temperature step, a number of random transformations of the path are made. First of all, a segment of the path is selected, with its start and end cities chosen at random. Then, a software coin is flipped to decide which kind of transformation to try: reverse or transport.

If reverse comes up, an alternative path is generated in which the cities in the chosen segment are reversed in order of visit. If transport, the segment is clipped out of its current position in the path and spliced in at a randomly chosen point in the remainder of the path. The length of the modified path is then calculated and compared to the path before modification, producing a quantity called the cost difference. If negative, the modified path is shorter than the original path and always replaces it. If there is an increase in cost, however, the exponential of its negative magnitude divided by the current temperature is compared to a uniformly distributed random number between 0 and 1 and, if greater, the modified path will be used even though it increased the cost. Note that initially, when the temperature is high, there will be a greater probability of making such changes, but that as the temperature falls, only smaller increases in cost will be accepted. The total number of changes tested at each temperature level is arbitrarily set to 100 times the number of cities in the path, and after ten times the number of changes which decrease the path length as the number of cities are found, the temperature is decreased and the search continued. If, after trying all of the potential changes at a given temperature level, no changes are found which reduce the path length, the solution is considered “good enough” and the resulting path is displayed.

Watching it Happen

To watch the optimisation process as it unfolds, instead of pressing the “Solve” button, press the “Step” button to see the path evolve at each level of decreasing temperature. The “Animate” button will automatically show the path evolving at one second per temperature level. Check the “Trace solution” box to display the temperature, cost (path length), and number of changes made to the path at each step in the optimisation. After a solution is found, the chosen itinerary will be shown listing the cities in order, their co-ordinates, and the cost of the path from each city to the next (wrapping around at the bottom) and, if the path crosses the river (see below), an “R” to indicate that it does.

Instead of using the “Place” button to randomly place cities, you can place them manually by pressing “New” to clear the map and then click the mouse in the map to indicate the city locations. They will initially be connected by paths in the order you placed the cities. You can also add cities to maps created by the “Place” button by clicking in the map.

Minimise or Maximise?

30 cities, maximise path lengthThe travelling salesman problem is usually formulated in terms of minimising the path length to visit all of the cities, but the process of simulated annealing works just as well with a goal of maximising the length of the itinerary. If you change the goal in the drop-down list from “Minimise” to “Maximise”, the cost function being optimised will be the negative of the path length, resulting in a search for the longest path. Try it, and see how the annealing process finds a star-like pattern that chooses the longest inter-city paths.

A River Runs through It

30 cities, river cost 50%We can add another wrinkle to the cost function by adding a “river” that runs through the map from top to bottom halfway across it. If you set the “River cost” nonzero, the river will be drawn as a dark blue line, and any path from one city to another which crosses it is assessed a penalty given by the river cost as a percentage of the size of the map. If you set the river cost high, say to 50%, you’ll find a strong preference for paths which only cross the river twice, finding a near-minimum path length independently on each side of the river. (This may be more apparent if you place a large number of cities, say 100 or 250.)

100 cities, river cost −25%You can also set the cost of crossing the river negative, which turns the travelling salesman into a peripatetic smuggler who profits from carrying goods between Freedonia and Grand Fenwick. Try placing 100 cities and setting the river cost to −25: the smuggler will settle on an efficient path on each side of the river, but prefer river crossings between cities close to the river where the benefit of the crossing is significant compared to the distance between them.

100 cities, river cost −100%, maximise costFinally, try setting the goal to Maximise path length, the river crossing cost to −100 (benefit from crossing the river), and place 100 cities. When you solve, you’ll find the solution produces two star-like patterns on each side of the river which maximises the travel distance on each side, but avoids river crossings at all costs.

Other Optimisation Techniques

Here we’ve explored one technique of combinatorial optimisation: simulated annealing. This is only one of the many approaches which have been taken to problems of this kind. These include exact approaches such as branch and boundlinear programming, and cutting-plane algorithms.

There are many approximation techniques which find near-optimal solutions, of which simulated annealing is but one. One algorithm even models how ants find the shortest path between their anthill and a source of food.

Reference

Press, William H., Saul A. Teukolsky, William T. Vetterling, and Brian P. Flannery. Numerical Recipes in C, 2nd ed. Cambridge: Cambridge University Press, [1988] 1992. ISBN 978-0-521-43108-8. Section 10.9, pp. 444–451. (A third edition of this book with algorithms in C++ is available.)

Like 11+

Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Hot Shots: A Radioactive Lens

Leica M6 with Summicron 50 mm f/2 lens

Between the 1940s and 1970s, a number of camera manufacturers designed lenses employing thoriated glass in one or more elements. Incorporating as much as 40% thorium dioxide (ThO2) in the glass mixture increases the index of refraction of the glass while maintaining low dispersion. Thoriated glass elements allowed lenses to deliver low levels of aberration and distortion with relatively simple and easy to manufacture designs.

As with everything in engineering, there are trade-offs. Thorium is a radioactive element; it has no stable isotopes. Natural thorium consists of 99.98% thorium-232, which has a half-life of 1.4×1010 years. While this is a long half-life, more than three times that of uranium-238, it is still substantially radioactive and easily detected with a Geiger-Müller counter. Thorium decays by alpha emission into radium-228, which continues to decay through the thorium series into various nuclides, eventually arriving at stable lead-208.

Leica Summicron 50 mm f/2 lensAttached to my Leica M6 film camera above is a Leica Summicron 50 mm f/2 lens which contains thoriated glass. Its serial number, 1041925, indicates its year of manufacture as 1952. This lens was a screw mount design, but can be used on more recent bayonet mount Leica cameras with a simple adapter. Like many early Leica lenses, it is collapsible: you can rotate the front element and push the barrel back into the camera body when not in use, making the camera more compact to pack and carry. Although 66 years old at this writing, the lens performs superbly, although not as well as current Leica lenses which are, however, more than an order of magnitude more expensive.

To measure the radiation emitted by this thoriated glass lens I used a QuartaRAD RADEX RD1706 Geiger-Müller counter and began by measuring the background radiation in my office.

Radiation monitor: 0.12 μSv/h

This came in (averaged over several measurement periods) as 0.12 microsieverts (μSv) per hour, what I typically see. Background radiation varies slightly over the day (I know not why), and this was near the low point of the cycle.

I then placed the detector directly before the front element of the lens, still mounted on the camera. The RADEX RD1706 has two Geiger tubes, one on each side of the meter. I positioned the meter so its left tube would be as close as possible to the front element.

Radiation monitor: 1.14 μSv/h

After allowing the reading to stabilise and time average, I measured radiation flux around 1.14 μSv/h, nearly ten times background radiation. Many lenses using thoriated glass employed it only for the front element(s), with regular crown or flint glass at the rear. This limits radiation which might, over time, fog the film in the camera. With such lenses, you can easily detect the radiation from the front element, but little is emitted backward in the direction of the film (and the photographer). This is not the case with this lens, however. I removed the lens from the camera, collapsed it so the back element would be closer to the detector (about as far as the front element was in the previous measurement) and repeated the test.

Radiation monitor: 1.51 μSv/h

This time I saw 1.51 μSv/h, more than twelve times background radiation. What were they thinking? First of all the most commonly used films in the early 1950s were slower (less sensitive) than modern emulsions, and consequently less prone to fogging due to radiation. Second, all Leica rangefinder cameras use a focal-plane shutter, which means the film behind the lens is shielded from the radiation it emits except for the instant the shutter is open when making an exposure, which would produce negligible fogging. Since the decay chain of thorium consists exclusively of alpha and beta particle emission, neither of which is very penetrating, the closed shutter protects the film from the radiation from the rear of the lens.

Many camera manufacturers used thoriated lenses. Kodak even used thoriated glass in its top of the line 800 series Instamatic cameras, and Kodak Aero-Ektar lenses, manufactured in great quantity during World War II for aerial reconnaissance, are famously radioactive. After 1970, thoriated glass ceased to be used in optics, both out of concern over radiation, but also due to a phenomenon which caused the thoriated glass to deteriorate over time. Decaying thorium atoms create defects in the glass called F-centres which, as they accumulated, would cause the glass to acquire a yellowish or brownish tint. This wasn’t much of a problem with black and white film, but it would cause a shift in the colour balance which was particularly serious for the colour reversal (transparency) film favoured by professional photographers in many markets. (My 1952 vintage lens has a slight uniform yellow cast to it—much lighter than a yellow filter. It’s easy to correct for in digital image post-processing.) Annealing the glass by exposing it to intense ultraviolet light (I’ve heard that several days in direct sunlight will do the job) can reduce or eliminate the yellowing.

Thorium glass was replaced by glass containing lanthanum oxide (La2O3), which has similar optical properties. Amusingly, lanthanum is itself very slightly radioactive: while the most common isotope, lanthanum-139, which makes up 99.911% of natural lanthanum, is stable, 0.089% is the lanthanum-138 isotope, which has a half-life of 1011 years, about ten times that of thorium. Given the tiny fraction of the radioisotope and its long half-life, the radiation from lanthanum glass (about 1/10000 that of thorium glass), while detectable with a sensitive counter, is negligible compared to background radiation.

If you have one of these lenses, should you be worried? In a word, no. The radiation from the lens is absorbed by the air, so that just a few centimetres away you’ll measure nothing much above background radiation. To receive a significant dose of radiation, you’d have to hold the front element of the lens up against your skin for an extended period of time, and why would you do that? Even if you did, the predominantly alpha radiation is blocked by human skin, and the dose you received on that small patch of skin would be no more than you receive on your whole body for an extended period on an airline flight due to cosmic rays. The only danger from thorium glass would be if you had a telescope or microscope eyepiece containing it, and looked through it with the naked eye. Alpha radiation can damage the cornea of the eye. Fortunately, most manufacturers were wise enough to avoid thoriated glass for such applications, and radioactive eyepieces are very rare. (Still, if you buy a vintage telescope or microscope, you might want to test the eyepieces, especially if the glass appears yellowed.)

Related:


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Life 3.0

“Life 3.0” by Max TegmarkThe Earth formed from the protoplanetary disc surrounding the young Sun around 4.6 billion years ago. Around one hundred million years later, the nascent planet, beginning to solidify, was clobbered by a giant impactor which ejected the mass that made the Moon. This impact completely re-liquefied the Earth and Moon. Around 4.4 billion years ago, liquid water appeared on the Earth’s surface (evidence for this comes from Hadean zircons which date from this era). And, some time thereafter, just about as soon as the Earth became environmentally hospitable to life (lack of disruption due to bombardment by comets and asteroids, and a temperature range in which the chemical reactions of life can proceed), life appeared. In speaking of the origin of life, the evidence is subtle and it’s hard to be precise. There is completely unambiguous evidence of life on Earth 3.8 billion years ago, and more subtle clues that life may have existed as early as 4.28 billion years before the present. In any case, the Earth has been home to life for most of its existence as a planet.

This was what the author calls “Life 1.0”. Initially composed of single-celled organisms (which, nonetheless, dwarf in complexity of internal structure and chemistry anything produced by other natural processes or human technology to this day), life slowly diversified and organised into colonies of identical cells, evidence for which can be seen in rocks today.

About half a billion years ago, taking advantage of the far more efficient metabolism permitted by the oxygen-rich atmosphere produced by the simple organisms which preceded them, complex multi-cellular creatures sprang into existence in the “Cambrian explosion”. These critters manifested all the body forms found today, and every living being traces its lineage back to them. But they were still Life 1.0.

What is Life 1.0? Its key characteristics are that it can metabolise and reproduce, but that it can learn only through evolution. Life 1.0, from bacteria through insects, exhibits behaviour which can be quite complex, but that behaviour can be altered only by the random variation of mutations in the genetic code and natural selection of those variants which survive best in their environment. This process is necessarily slow, but given the vast expanses of geological time, has sufficed to produce myriad species, all exquisitely adapted to their ecological niches.

To put this in present-day computer jargon, Life 1.0 is “hard-wired”: its hardware (body plan and metabolic pathways) and software (behaviour in response to stimuli) are completely determined by its genetic code, and can be altered only through the process of evolution. Nothing an organism experiences or does can change its genetic programming: the programming of its descendants depends solely upon its success or lack thereof in producing viable offspring and the luck of mutation and recombination in altering the genome they inherit.

Much more recently, Life 2.0 developed. When? If you want to set a bunch of paleontologists squabbling, simply ask them when learned behaviour first appeared, but some time between the appearance of the first mammals and the ancestors of humans, beings developed the ability to learn from experience and alter their behaviour accordingly. Although some would argue simpler creatures (particularly birds) may do this, the fundamental hardware which seems to enable learning is the neocortex, which only mammalian brains possess. Modern humans are the quintessential exemplars of Life 2.0; they not only learn from experience, they’ve figured out how to pass what they’ve learned to other humans via speech, writing, and more recently, YouTube comments.

While Life 1.0 has hard-wired hardware and software, Life 2.0 is able to alter its own software. This is done by training the brain to respond in novel ways to stimuli. For example, you’re born knowing no human language. In childhood, your brain automatically acquires the language(s) you hear from those around you. In adulthood you may, for example, choose to learn a new language by (tediously) training your brain to understand, speak, read, and write that language. You have deliberately altered your own software by reprogramming your brain, just as you can cause your mobile phone to behave in new ways by downloading a new application. But your ability to change yourself is limited to software. You have to work with the neurons and structure of your brain. You might wish to have more or better memory, the ability to see more colours (as some insects do), or run a sprint as fast as the current Olympic champion, but there is nothing you can do to alter those biological (hardware) constraints other than hope, over many generations, that your descendants might evolve those capabilities. Life 2.0 can design (within limits) its software, but not its hardware.

The emergence of a new major revision of life is a big thing. In 4.5 billion years, it has only happened twice, and each time it has remade the Earth. Many technologists believe that some time in the next century (and possibly within the lives of many reading this review) we may see the emergence of Life 3.0. Life 3.0, or Artificial General Intelligence (AGI), is machine intelligence, on whatever technological substrate, which can perform as well as or better than human beings, all of the intellectual tasks which they can do. A Life 3.0 AGI will be better at driving cars, doing scientific research, composing and performing music, painting pictures, writing fiction, persuading humans and other AGIs to adopt its opinions, and every other task including, most importantly, designing and building ever more capable AGIs. Life 1.0 was hard-wired; Life 2.0 could alter its software, but not its hardware; Life 3.0 can alter both its software and hardware. This may set off an “intelligence explosion” of recursive improvement, since each successive generation of AGIs will be even better at designing more capable successors, and this cycle of refinement will not be limited to the glacial timescale of random evolutionary change, but rather an engineering cycle which will run at electronic speed. Once the AGI train pulls out of the station, it may develop from the level of human intelligence to something as far beyond human cognition as humans are compared to ants in one human sleep cycle. Here is a summary of Life 1.0, 2.0, and 3.0.

Life 1.0, 2.0, 3.0

The emergence of Life 3.0 is something about which we, exemplars of Life 2.0, should be concerned. After all, when we build a skyscraper or hydroelectric dam, we don’t worry about, or rarely even consider, the multitude of Life 1.0 organisms, from bacteria through ants, which may perish as the result of our actions. Might mature Life 3.0, our descendants just as much as we are descended from Life 1.0, be similarly oblivious to our fate and concerns as it unfolds its incomprehensible plans? As artificial intelligence researcher Eliezer Yudkowsky puts it, “The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.” Or, as Max Tegmark observes here, “[t]he real worry isn’t malevolence, but competence”. It’s unlikely a super-intelligent AGI would care enough about humans to actively exterminate them, but if its goals don’t align with those of humans, it may incidentally wipe them out as it, for example, disassembles the Earth to use its core for other purposes.

But isn’t this all just science fiction—scary fairy tales by nerds ungrounded in reality? Well, maybe. What is beyond dispute is that for the last century the computing power available at constant cost has doubled about every two years, and this trend shows no evidence of abating in the near future. Well, that’s interesting, because depending upon how you estimate the computational capacity of the human brain (a contentious question), most researchers expect digital computers to achieve that capacity within this century, with most estimates falling within the years from 2030 to 2070, assuming the exponential growth in computing power continues (and there is no physical law which appears to prevent it from doing so).

My own view of the development of machine intelligence is that of the author in this “intelligence landscape”.

The landscape of intelligence

Altitude on the map represents the difficulty of a cognitive task. Some tasks, for example management, may be relatively simple in and of themselves, but founded on prerequisites which are difficult. When I wrote my first computer program half a century ago, this map was almost entirely dry, with the water just beginning to lap into rote memorisation and arithmetic. Now many of the lowlands which people confidently said (often not long ago), “a computer will never…”, are submerged, and the ever-rising waters are reaching the foothills of cognitive tasks which employ many “knowledge workers” who considered themselves safe from the peril of “automation”. On the slope of Mount Science is the base camp of AI Design, which is shown in red since when the water surges into it, it’s game over: machines will now be better than humans at improving themselves and designing their more intelligent and capable successors. Will this be game over for humans and, for that matter, biological life on Earth? That depends, and it depends upon decisions we may be making today.

Assuming we can create these super-intelligent machines, what will be their goals, and how can we ensure that our machines embody them? Will the machines discard our goals for their own as they become more intelligent and capable? How would bacteria have solved this problem contemplating their distant human descendants?

First of all, let’s assume we can somehow design our future and constrain the AGIs to implement it. What kind of future will we choose? That’s complicated. Here are the alternatives discussed by the author. I’ve deliberately given just the titles without summaries to stimulate your imagination about their consequences.

  • Libertarian utopia
  • Benevolent dictator
  • Egalitarian utopia
  • Gatekeeper
  • Protector god
  • Enslaved god
  • Conquerors
  • Descendants
  • Zookeeper
  • 1984
  • Reversion
  • Self-destruction

Choose wisely: whichever you choose may be the one your descendants (if any exist) may be stuck with for eternity. Interestingly, when these alternatives are discussed in chapter 5, none appears to be without serious downsides, and that’s assuming we’ll have the power to guide our future toward one of these outcomes. Or maybe we should just hope the AGIs come up with something better than we could think of. Hey, it worked for the bacteria and ants, both of which are prospering despite the occasional setback due to medical interventions or kids with magnifying glasses.

Let’s assume progress toward AGI continues over the next few decades. I believe that what I’ve been calling the “Roaring Twenties” will be a phase transition in the structure of human societies and economies. Continued exponential growth in computing power will, without any fundamental breakthroughs in our understanding of problems and how to solve them, allow us to “brute force” previously intractable problems such as driving and flying in unprepared environments, understanding and speaking natural languages, language translation, much of general practice medical diagnosis and routine legal work, interaction with customers in retail environments, and many jobs in service industries, allowing them to be automated. The cost to replace a human worker will be comparable to a year’s wages, and the automated replacement will work around the clock with only routine maintenance and never vote for a union.

This is nothing new: automation has been replacing manual labour since the 1950s, but as the intelligence landscape continues to flood, not just blue collar jobs, which have already been replaced by robots in automobile plants and electronics assembly lines, but white collar clerical and professional jobs people went into thinking them immune from automation. How will the economy cope with this? In societies with consensual government, those displaced vote; the computers who replace them don’t (at least for the moment). Will there be a “robot tax” which funds a basic income for those made redundant? What are the consequences for a society where a majority of people have no job? Will voters at some point say “enough” and put an end to development of artificial intelligence (but note that this would have to be global and enforced by an intrusive and draconian regime; otherwise it would confer a huge first mover advantage on an actor who achieved AGI in a covert program)?

The following chart is presented to illustrate stagnation of income of lower-income households since around 1970.

Income per U.S. household, 1920–2015

I’m not sure this chart supports the argument that technology has been the principal cause for the stagnation of income among the bottom 90% of households since around 1970. There wasn’t any major technological innovation which affected employment that occurred around that time: widespread use of microprocessors and personal computers did not happen until the 1980s when the flattening of the trend was already well underway. However, two public policy innovations in the United States which occurred in the years immediately before 1970 (1, 2) come to mind. You don’t have to be an MIT cosmologist to figure out how they torpedoed the rising trend of prosperity for those aspiring to better themselves which had characterised the U.S. since 1940.

Nonetheless, what is coming down the track is something far more disruptive than the transition from an agricultural society to industrial production, and it may happen far more rapidly, allowing less time to adapt. We need to really get this right, because everything depends on it.

Observation and our understanding of the chemistry underlying the origin of life is compatible with Earth being the only host to life in our galaxy and, possibly, the visible universe. We have no idea whatsoever how our form of life emerged from non-living matter, and it’s entirely possible it may have been an event so improbable we’ll never understand it and which occurred only once. If this be the case, then what we do in the next few decades matters even more, because everything depends upon us, and what we choose. Will the universe remain dead, or will life burst forth from this most improbable seed to carry the spark born here to ignite life and intelligence throughout the universe? It could go either way. If we do nothing, life on Earth will surely be extinguished: the death of the Sun is certain, and long before that the Earth will be uninhabitable. We may be wiped out by an asteroid or comet strike, by a dictator with his fat finger on a button, or by accident (as Nathaniel Borenstein said, “The most likely way for the world to be destroyed, most experts agree, is by accident. That’s where we come in; we’re computer professionals. We cause accidents.”).

But if we survive these near-term risks, the future is essentially unbounded. Life will spread outward from this spark on Earth, from star to star, galaxy to galaxy, and eventually bring all the visible universe to life. It will be an explosion which dwarfs both its predecessors, the Cambrian and technological. Those who create it will not be like us, but they will be our descendants, and what they achieve will be our destiny. Perhaps they will remember us, and think kindly of those who imagined such things while confined to one little world. It doesn’t matter; like the bacteria and ants, we will have done our part.

The author is co-founder of the Future of Life Institute which promotes and funds research into artificial intelligence safeguards. He guided the development of the Asilomar AI Principles, which have been endorsed to date by 1273 artificial intelligence and robotics researchers. In the last few years, discussion of the advent of AGI and the existential risks it may pose and potential ways to mitigate them has moved from a fringe topic into the mainstream of those engaged in developing the technologies moving toward that goal. This book is an excellent introduction to the risks and benefits of this possible future for a general audience, and encourages readers to ask themselves the difficult questions about what future they want and how to get there.

In the Kindle edition, everything is properly linked. Citations of documents on the Web are live links which may be clicked to display them. There is no index.

Tegmark, Max. Life 3.0. New York: Alfred A. Knopf, 2017. ISBN 978-1-101-94659-6.

This is a one hour talk by Max Tegmark at Google in December 2017 about the book and the issues discussed in it.

Watch the Google DeepMind artificial intelligence learn to play and beat Atari Breakout knowing nothing about the game other than observing the pixels on the screen and the score.

In this July 2017 video, DeepMind develops legged locomotion strategies by training in rich environments.  Its only reward was forward progress, and nothing about physics or the mechanics of locomotion was pre-programmed: it learned to walk just like a human toddler.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Apparent Diurnal Variation in Background Radiation

Aware Electronics RM-80 sensorSince October 16th, 1998, I’ve run an Aware Electronics RM-80 radiation monitor pretty much continuously, connected to a serial port on a 1992 vintage 486/50 machine. The RM-80 uses a 7313 pancake Geiger-Müller tube. The tube is halogen quenched and has a minimum dead time of 30 µS, with a mica window which allows alpha particles to pass. The computer interface generates high voltage for the tube from the Data Terminal Ready and Request to Send pins of the serial port and toggles the state of the Ring Indicator line whenever a count is detected. The serial port can be programmed to interrupt on changes in the state of this signal, making it straightforward to implement radiation monitoring in software. Tube sensitivity, calibrated with Cesium-137 (137Cs), is 3.54 µR/hour per count per minute.

The second generation HotBits generator uses an RM-80 detector illuminated by a 5 microcurie 137Cs check source. I decided to attach the HotBits spare detector to a PC and let it run as a background radiation monitor, as much as anything to let the detector run for a while to guard against “infant mortality” in any of its components, should it have to take over for the in-service detector. Aware Electronics supplies the detector with a DOS driver program called AW-SRAD, which I used to log the number of counts per minute, logging the collected data to files in CSV format.

Database Coverage

Here’s a plot that shows the extent of coverage by month over the period I monitored background radiation. The month of May, for example, has just about one year’s complete data (this doesn’t necessarily mean all of May for one year—it might be half of May in 1999 and half in 2000, for example). November has the greatest coverage, in excess of 2.5 years of data. The summer months have the least coverage due to vacations and other circumstances which caused me to shut down the machine connected to the detector. Since we’re examining only diurnal variations—change in flux within a day—the uneven coverage over the months shouldn’t be a problem. If we wanted to explore, for example, whether any diurnal variation we detected varied from month to month, it would be best to extract a subset of the data weighted equally by month, ideally with full coverage of each day of the solar year even though the days may be taken from different years.

Database Coverage

Counts per Minute Histogram

The first obvious step in reducing the data is to plot a histogram showing the distribution of counts per minute; the vertical axis is the number of minutes in the database in which the number of counts on the horizontal axis were recorded. The histogram table is plotted below.

Counts per minute histogram

At first glance, this looks like the Gaussian “bell curve” you’d expect for a random process. At second glance, however, it doesn’t…note that the “tail” on the right hand side, corresponding to larger numbers of counts per minutes appears distinctly longer and “fatter” than the tail on the left side. Still, let’s proceed for the moment on the assumption that we do have a Gaussian distribution and calculate the mean and standard deviation from the data set. Crunching the numbers, we find a mean value of 56.65 counts per minute with a standard deviation of 9.31. We can then plot this Gaussian (normal) distribution as a red line superimposed on the histogram of experimental results.

Histogram vs. normal distribution

This makes it even more apparent that there’s something happening which isn’t “perfectly normal”. Note how the excess on the high end of the histogram pulls the best-fit normal distribution to the right of the actual data distribution, with the histogram bars to the left of the mean consistently exceeding the value of the normal curve and those to the right falling below.

You might be inclined to wonder just how closely we should expect experimental results like these to approximate a normal distribution. Could the observed deviation be nothing more than a statistical fluke? No…compare the fit of the background radiation histogram above with the the plot below of a data set of comparable size collected using the same detector but, instead of monitoring background radiation in counts per minute, measuring counts per second with the detector illuminated by the HotBits Cesium-137 source. Although this data set necessarily includes background radiation as well as counts due to the radiation source, with about 2000 counts per second from the 137Cs source, the roughly one count per second from background radiation has a negligible influence on the results.

Detector illuminated by Cs-137 source

In the radiation source data set we see an essentially perfect fit of the expected normal distribution to the experimental data. This makes it quite clear that there’s something curious going on with the background radiation data. But what?

The Two Population Hypothesis

One way we might end up with the kind of asymmetrical distribution we observe for the background radiation is if the radiation flux we’re measuring is composed of two separate components, or populations, each individually normally distributed, but with different peak value, mean, and standard deviation. Suppose, for example, that the background radiation the detector measures is the sum of that due to decay of heavy elements (principally thorium and daughter nuclides) in the immediate vicinity (the earth beneath the detector and the structure of the building in which it is located) and a less intense cosmic ray flux which occurs in sporadic bursts with a greater mean value than the terrestrial flux. (At this point I’m simply suggesting these sources to illustrate how the background radiation might be composed of two (or more) separate populations with different statistics; I’m not identifying them as the actual components.)

Here’s a “toy model” which illustrates how this might work.

Two population model

Let the brown curve labeled “Population 1” represent the terrestrial component of background radiation. It has a mean value of around 58 counts per minute and exhibits a precisely Gaussian distribution with total flux equal to the area under the brown Population 1 curve. We take the red “Population 2” curve to represent the supposed cosmic ray flux. This also has a normal distribution, but with a mean value of about 70 counts per minute, a greater standard deviation (resulting in a broader distribution curve), and a total flux only about one tenth that of Population 1.

Since counts resulting from the two populations cannot be distinguished by our simple detector, what we observe is the sum of the two populations, shown as the orange “Combined Flux” curve. Note the strong resemblance between this curve and the histogram plot from the detector; while Population 1 dominates the result, the contribution of Population 2 lifts and extends the high end of the combined distribution, precisely as we observed in the experimental data set.

Radiation by Local Solar Time

Next I tried binning the results hourly by local solar time. The following chart shows the results, plotted in terms of average background radiation flux in micro-Roentgens per hour. (The average background radiation of 16.2 µR/hr—142 mR per year—may seem high, but my detector is located at an altitude of 800 metres above sea level. Both the soft and hard [primarily muon] components of cosmic rays are absorbed by the atmosphere, so at a higher altitude more they are more intense. At sea level, cosmic rays contribute about 30 mR/year, but at the 10 km altitude commercial jet aircraft fly, cosmic radiation accounts for about 2000 mR/year; more than 60 times as intense.) When I plotted the hourly local time averages, I obtained the following result.

Radiation by local solar time

I’ve read about variations in cosmic ray flux varying with latitude, in Easterly and Westerly incidence, the solar cycle, and changes in the geomagnetic field, without a mention of a diurnal cycle, yet this plot appears to show a sinusoidal variation, with a magnitude variation between the highest three-hour period and the lowest of almost 6% of the mean value and, further, the trough in the curve seems to be about 12 hours from the peak.

To explore whether this might be nothing but an artifact or statistical fluctuation, I then re-binned the same data minute by minute, resulting in the following plot, in which the blue curve is the raw minute-binned data and the red curve is the same data filtered by an exponentially smoothed moving average with a smoothing factor of 0.9.

Radiation minute-by-minute, smoothed

Randomly selected data subsetWell, it still looks credibly sinusoidal, with the maximum and minimum at about the same point. As we all know, the human eye and brain are extraordinarily adept at seeing patterns in random data. So let’s try another test frequently applied as a reality check when apparently significant results appear in a data set. The chart at the left was created by randomly selecting 25% of the points appearing in the complete data set and plotting them hour by hour. We find that the selection has little effect on the shape of the curve or the location of its maximum and minimum.

Outliers removedNext, I decided to explore whether the apparent sinusoidal variation might disappear if I discarded outlying values, which might conceivably vary differently in time than those which make up the bulk of the database. I pruned the bell curve at one standard deviation, then used the remaining data to prepare the plot at the left. As you can see, the case for a sinusoidal variation is eroded somewhat, but the general shape, magnitude, and location of extrema is conserved.

Outliers and Populations

The fact that removing the outlying values reduced the diurnal variation in the above plot suggests that we may indeed have two populations contributing to the observed flux, with the population responsible for the outlying values containing more diurnal variation than that near the mean. To investigate this further, I passed the data set through a variety of filters and prepared the following plot.

Outliers and populations

In the Stars?

Finally, I decided to plot the average radiation flux against local sidereal time. Sidereal time tracks the position of the distant stars as viewed from a given point on the Earth. At the same sidereal time, the same celestial objects (external to the solar system) will cross the celestial meridian above a given place on the Earth. Because the viewpoint of the Earth shifts as it orbits the Sun, the sidereal day (time between successive meridian crossings of a given star) is about 4 minutes shorter than the solar day (mean time between solar meridian crossings). Correlation with the sidereal period is powerful evidence for a distant source as the cause of a given effect. For example, it was correlation with the sidereal period which provided early radio astronomers evidence the centre of the galaxy and Crab Nebula were celestial sources of the noise they were monitoring. Here’s a plot of average radiation flux by sidereal time.  There is no significant evidence for a correlation of flux with sidereal time.

Background radiation by sidereal time

 

What’s Going On Here?

Darned if I know! The floor is open to inference and speculation.

First of all, I think it’s reasonable to assume that any diurnal variation, should such exist, is due to cosmic rays. The balance of background radiation is primarily due to thorium, radon, and daughter nuclides in the local environment. Where I live, in the Jura mountains of Switzerland, subterranean rocks are almost entirely limestone, which has little or no direct radioactivity (as opposed to, for example, granite), nor radon precursors. In such an environment, it’s hard to imagine a background radiation component other than cosmic rays which would vary on a daily basis. (This would not be the case, for example, in a house with a radon problem, where you would expect to see a decrease when doors and windows were opened during the day.)

If the effect is genuine, and the cause is cosmic ray flux, what are possible causes? The two which pop to mind are atmospheric density and the geomagnetic field. During the day, as the Sun heats the atmosphere, it expands. If you’re at sea level, the total absorption cross section remains the same, but the altitude at which the primary cosmic ray first interacts with an atmospheric atom may increase. Further, an increase in atmospheric temperature may change the scale height of of the atmosphere, which would perturb values measured at various altitudes above sea level. We could explore temperature dependence by comparing average background radiation in summer and winter months.

Let’s move on to the geomagnetic field. It’s well documented that the Earth’s magnetic field and its interaction with the Sun’s create measurable changes in cosmic ray incidence, since the proton and heavy ion component of primary particles is charged and follows magnetic field lines. As any radio amateur or listener to AM radio in the 1950s knows, the ionosphere changes dramatically at night, allowing “skip propagation” of medium- and high-frequency signals far beyond the horizon. Perhaps this effect also modifies the geomagnetic field, affecting the number of charged cosmic rays incident at a given location.

If there is a diurnal effect, why on Earth should it peak around 07:00 local time? Beats me.

References

  1. Clay, Roger, and Bruce Dawson. Cosmic Bullets. Reading, MA: Addison-Wesley, 1997. ISBN 978-0-7382-0139-9.
  2. Wheeler, John Archibald, and Kenneth Ford. Geons, Black Holes, and Quantum Foam: A Life in Physics. New York: W.W. Norton, 1998. ISBN 978-0-393-31991-0.

Download Raw Data and Analysis Programs

If you’d like to perform your own investigations of this data set, you can download the data and programs used in preparing this page. The 2.8 Mb Zipped archive contains the raw data in rad5.csv and a variety of Perl programs which were used to process it in various ways. There is no documentation and these programs are utterly unsupported: you’re entirely on your own.


Users who have liked this post:

  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar
  • avatar

Saturday Night Science: Fun with Cosmic Rays

I took an international flight today, and did something I’ve intended to do for some time: monitor the background radiation flux as the plane changed altitudes.  I brought along a QuartaRAD RADEX RD1706 Geiger-Müller counter which detects beta particles (high energy electrons) and photons in the x-ray and gamma ray spectra and displays a smoothed moving average of the radiation dose in microsieverts (μSv) per hour.  The background radiation depends upon your local environment: areas with rocks such as granite which are rich in mildly radioactive thorium will have more background radiation than those with rocks such as limestone.

One important component of background radiation is cosmic rays caused by high energy particles striking the Earth’s atmosphere.  The atmosphere is an effective radiation shield and absorbs many of these particles before they reach sea level, but as you go to higher altitudes, fewer particles are absorbed and you experience a higher background radiation dose from cosmic rays.  Background radiation at sea level is usually around 0.10 to 0.13 μSv/h.  At Fourmilab, at an altitude of 806 metres above mean sea level, it usually runs around 0.16 μSv/h.

I waited until the flight was at cruising altitude before turning on the detector and placing it on my tray table near the window of my window seat.  This was not a high-flyer: the plane was a Bombardier Q400 Dash 8 regional turboprop on a medium-range flight within Europe, with a cruising altitude of 7000 metres  (the plane’s service ceiling is 8229 metres, modest compared to the Boeing 747-8’s ceiling of 13,000 m).  My first reading was:

Radiation monitor: 1.24 μSv/h

Wow!  1.24 microsieverts per hour is almost ten times the usual reading near sea level.  And this was inside the fuselage of an airplane cruising at a modest altitude.

About half way through the flight, we encountered moderately high turbulence (enough to turn on the seat belts sign, but nothing really scary), and the pilot in command requested a lower altitude to try to escape it.  Air traffic control approved a descent to 6000 metres.  During the descent, the background radiation level smoothly decreased.  Here is part way down the slope.

Radiation monitor: 0.86 μSv/h

And now we’re at at the new cruising altitude of 6000 m.

Radiation monitor: 0.67 μSv/h

Finally the plane began its descent for landing.  Here are readings on the way down, with the last one on final approach over water shortly before touchdown on the runway on the coast.

Radiation monitor: 0.20 μSv/h

Radiation monitor: 0.13 μSv/h

Now the radiation level has fallen to that around sea level.  But wait, there’s more!

Radiation monitor: 0.07 μSv/h

This is at an altitude of just dozens of metres, still over water, seconds before touchdown.  Background radiation is now around half the usual at sea level.  (This wasn’t a fluke—I got this reading on several consecutive measurement cycles.)  But think about it: the contribution to background radiation from terrestrial sources (such as thorium and uranium in rocks) and cosmic rays are about the same.  But in an airplane flying low over water, the terrestrial component is very small (since the sea has very few radioactive nuclides), so it’s plausible that we’ll see around half the background radiation in such a situation as on terra firma.  Indeed, after landing, the background radiation while taxiing to the terminal went back up to around 0.13 μSv/h.

It would be interesting to repeat this experiment on an intercontinental flight at higher altitude and through higher latitudes, where the Earth’s magnetic field provides less shielding against cosmic rays.  But the unpleasantness of such journeys deters me from making them in anything less that the most exigent circumstances.  There is no original science to be done here: extensive monitoring and analysis of the radiation dose experienced by airline passengers and crews has been done.  This is a Fourmilab “basement science” experiment (well, not in the basement, but in a shrieking aluminium death tube) you can do yourself for amusement.  If you do this on a crowded flight, your seatmate may inquire what’re you’re up to.  “Measuring the cosmic radiation dose we’re receiving on this flight.”  This can either lead to a long and interesting conversation about atmospheric absorption of cosmic rays, background radiation, and radiation hormesis or, more likely, your having an empty seat next to you for the remainder of the flight.  Think of it as win-win.  There were only seven passengers on this flight (I don’t go to places that are too crowded—nobody goes there), so this didn’t come up during this experiment.

Return Flight

A couple of weeks later, the return flight was on an Embraer E190 regional turbofan airliner. The altitude of the flight was never announced en route, but this aircraft has a service ceiling of 12,000 m and usually cruises around 10,000 m, substantially higher than the turboprop I took on the outbound flight. I expected to see a higher radiation level on this flight, and I did.

Return flight: 5.07 μSv/hour

Did I ever! Most of the readings I obtained during cruise were around 3.8 μSv/h, more than thirty times typical sea level background radiation. (I’d show you one of these readings, but there was substantial turbulence on the flight and all of my attempts to photograph the reading are blurred.) During the cruise, I got several substantially higher values such as the 5.07 μSv/h shown above—more than forty times sea level.

Why was there such variation in background radiation during the cruise? I have no idea. If I had to guess, it would be that at the higher altitude there is more exposure to air showers, which might account for the greater variance than observed at sea level or lower altitude in flight. Or, maybe the gremlin on the wing was wearing a radioactive bracelet.


Users who have liked this post:

  • avatar
  • avatar
  • avatar