Summary of two sessions I had the privilege of chairing  at the AI in Sci-Fi Film and Literature  conference, 15–16 March 2018 Jesus College, Cambridge. The conference was part of the Science & Human Dimension Project.

Friday, 16 March 2018
13.30-14.25. Session 8 – AI in Sci-Fi Author Session
Chair: Prof Thore Husfeldt
Justina Robson
Dr Paul J. McAuley A brief history of encounters with things that think
Lavie Tidhar, Greek Gods, Potemkin AI and Alien Intelligence
Ian McDonald, The Quickness of Hand Deceives the AI

14.30-15.25 Session 9 – AGI Scenarios in Sci-Fi
Workshop lead by Prof Thore Husfeldt

The workshop consisted of me giving a brief introduction to taxonomies for superintelligence scenarios, adapted from Barrett and Baum (2017), Sotala (2018), Bostrom (2014), and Tegmark (2018). I then distributed the conference participants into 4 groups, led by authors Robson, McAuley, Tidhar, and McDonald. The groups were tasked with quickly filling each of these scenarios with as many fictionalisations as they could.

(Technical detail: Each group had access to a laptop and the workshop participants collaboratively edited a single on-line document, to prevent redundancy and avoid the usual round-robin group feedback part of such a workshop. This took some preparation but worked surprisingly well.)

This summary collates these suggestions, completed with hyperlinks to the relevant works, but otherwise unedited. I made no judgement calls about germaneness or artistic quality of the suggestions.

Overview

In a superintelligence scenario, our environment contains nonhuman agent exceeding human cognitive capabilities, including intelligence, reasoning, empathy, social skills, agency, etc. Not only does this agent exist (typically as a result of human engineering), it is unfettered and controls a significant part of the infrastucture, such as communication, production, or warfare.

The summary has three parts:

1. Pathways: How did the Superintelligence come about?
2. Containment failure: Given that the Superintelligence was constructed with some safety mechanisms in mind, how did it break free?
3. Aftermaths: How does the world with Superintelligence look?

---

Part I: Pathways to Superintelligence

Most of the scenarios below describe speculative developments in which some other entity (or entities) than modern humans acquire the capability to think faster or better (or simply more) than us.

Network

In the first scenario, the Superintelligence emerges from networking a large number of electronic computers (which individually need not exhibit Superintelligence). This network can possibly include humans and entire organisations as its nodes.

• Ian McDonald, River of Gods (novel 2014)
• Matthew De Abaitua, The Destructives (novel 2016)
• The Borg from the Stark Trek TV series.
• Her (film 2013)
• Orson Scott Card, Speaker for the Dead (novel 1986)
• The Construct Council of New Crobuzon in China Mieville’s novels Perdido Street Station (2000) and Iron Council (2004).
• The Matrix (film 1999).
• Ann Leckie, Ancillary trilogy (novels 2013–2015).

Augmented human brains

Individual human have their brains are augmented, for instance by interfacing with an electronic computer. The result far exceeds the cognitive capabilities of a single human.

• Charles Stross, Accelerando (novel, 2005)
• Daniel Keyes, Flowers for Algernon (short story 1959, novel 1966)
•  Samuel R., Delany, Babel-17 (novel 1966)
• Frederik Pohl, Man Plus (novel 1976)
• Transcendance (film 2014)
• Forbidden Planet (film 1956)
• Peter F. Hamilton, Greg Mandel / Event Horizon trilogy (novels 1993–1995)
• Ramez Naam, Nexus trilogy (novels 2012–2015)
• Robert A. Heinlein, Waldo (short story 1942)
• William Gibson, Johnny Mnemonic (short story 1986, eponymous film 1995)
• Stephen King, Lawnmower Man (short story 1975, eponymous film 1992)

Better biological cognition

The genotype of some or all humans have has been changed, using eugenics or deliberate genome editing, selecting for higher intelligence that far surpasses modern Humans.

• Paul J. McAuly, Fairyland (novel 1996)
• Lucy (film 2014)
• Khan Noonien Singh and Julian Bashir in Star Trek (tv series and films).
• Alan Glynn, The Dark Fields (novel 2001, Limitless film 2011).
• Planet of the Apes reboot (films 20011–)

Brain emulation

The brains of individual humans are digitized and their neurological processes emulated on hardware that allows for higher processing speed, duplication, and better networking that biological brain tissue. Also called whole brain emulation, mind copying, just  uploading.

• Robin Hanson, The Age of Em (nonfiction 2016)
• Richard K. Morgan, Altered Carbon (novel 2002, eponymous tv series 2018–)
• Matthew De Abaitua, The Red Men (novel 2009).
• Greg Egan, Permutation City (novel 1994)
• Pat Cadigan, Dervish is Digital (novel 2002)
• Cory Doctorow, Walkaway (novel 2017)
• M.T. Anderson, Feed (novel 2002)
• GLaDOS in Portal (video game series 2007–2011).

See also Mind Uploading in Fiction at Wikipedia.

Algorithms

Thanks to breakthroughs in symbolic artificial intelligence, machine learning, or artificial life, cognition (including agency, volition, explanation) has been algorithmicised and optimised, typically in an electronic computer.

• Colossus: The Forbin Project (film 1970)

• Matthew De Abaitua, If Then (novel 2015)
• Ultron in the Marvel Comics universe (e.g., Avengers: Age of Ultron, film 2015).
• Gus Gorman’s supercomputer in Superman III (film 1983).
• Janet from The Good Place (tv series 2016–).
• William Gibson and Bruce Sterling,  The Difference Engine (novel 1990)
• Vernor Vinge, The Peace War  (novel 1984) and  Marooned in Realtime (novel 1986).

Other

For most purposes, the arrival of alien intelligences has the same effect as the construction of a Superintelligence. Various other scenarios (mythological beings, magic) are operationally similar and have been fictionalised many times.

Continues in Part II: Failures.  Part III is forthcoming.

---

References

• Anthony Michael Barrett, Seth D. Baum, A Model of Pathways to Artificial Superintelligence Catastrophe for Risk and Decision Analysis, 2016, http://arxiv.org/abs/1607.07730
• Sotala, Kaj (2018). Disjunctive Scenarios of Catastrophic AI Risk. AI Safety and Security (Roman Yampolskiy, ed.), CRC Press. Forthcoming.
• Nick Bostrom, Superintelligence: Paths, Dangers, Strategies, Oxford University Press, 2014.
• Max Tegmark, Life 3.0, Knopf, 2018.