Asimov for Lawmakers

Journal of Business & Technology Law Journal of Business & Technology Law

Volume 18 Issue 1 Article 2

Asimov for Lawmakers Asimov for Lawmakers

Jerome De Cooman

Nicolas Petit

Follow this and additional works at: https://digitalcommons.law.umaryland.edu/jbtl

Recommended Citation Recommended Citation

Jerome D. Cooman, & Nicolas Petit,

Asimov for Lawmakers

, 18 J. Bus. & Tech. L. 1 (2022)

Available at: https://digitalcommons.law.umaryland.edu/jbtl/vol18/iss1/2

This Article is brought to you for free and open access by the Academic Journals at DigitalCommons@UM Carey

Law. It has been accepted for inclusion in Journal of Business & Technology Law by an authorized editor of

DigitalCommons@UM Carey Law. For more information, please contact smccarty@law.umaryland.edu.

Petit Page Proof (Do Not Delete)  2/1/2023  5:10 PM 
 
Journal of Business & Technology Law  1 
Articles 
 
Asimov for Lawmakers 
 
JEROME DE COOMAN
*
 AND NICOLAS PETIT
**
 
ABSTRACT 
Are  Isaac  Asimov’s  books  on  lawmakers’  bedside  tables?  Many 
emerging laws on Artificial Intelligence (“AI”) mention Asimov’s Three 
Laws of Robotics. But should they? Asimov’s stories describe failures 
of the Three Laws, not successes. This paper attempts to address this 
question by diving into Asimov’s works of and on science fiction. The 
paper  shows  that  the  wisdom  that  lawmakers  can  derive  from 
Asimov’s writing is different from the regulation by design approach 
embodied  in  the  Three  Laws.  Seven  nuanced  lessons  about 
technological  change,  and  the  way  societies  respond  to  it,  emerge 
from Asimov’s works.  
 
Introduction ........................................................................................ 2 
I.  Why Fiction, Why Science Fiction, Why Asimov? ........................... 4 
II. Technological Insights and the Closeness to Science Rule ............. 7 
III.  Why and How Science Fiction’s Social Insights Matter .............. 12 
IV.  Social Hostility, Then Affinity, to Technological Change ............ 14 
V.  The Inevitability of Regulation by Design .................................... 18 
VI.  The Fallibility of Regulation by Design ........................................ 22 
VII.  The Necessity of Expert Human Agency .................................... 26 
VIII.  The False Analogy Between Law and Technology’s Mental 
Models .................................................................................... 30 
Conclusion ......................................................................................... 34 
 
 
*
  ©  Jerome  De  Cooman,  Ph.D  Candidate,  Research  and  Teaching  Assistant, 
University  of  Liege  (ULiege),  Liege  Competition  and  Innovation  Institute  (LCII), 
jerome.decooman@uliege.be.  
**
 © Nicolas Petit, Professor of Law, European University Institute (EUI); Visiting 
Professor,  College  of  Europe,  Nicolas[email protected].  The  authors  want  to 
acknowledge the excellent feedback received from EUI colleagues and researchers 
during the Law and Technology seminar in 2021 and 2022. 

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

2 Journal of Business & Technology Law

INTRODUCTION

Lawmakers read Isaac Asimov’s robot stories. Justifications for

lawmaking initiatives towards Artificial Intelligence (“AI”) often use

Asimov’s Three Laws of robotics (“the Three Laws”).

This is strange.

Asimov’s robot stories describe failures of the Three Laws. The robot

stories embody no suggestion that law is good, or even needed.

What Asimov said about law is this: science fiction embodies useful

insights for lawmaking.

But what insights?

This is the question that this paper tries to answer. Like fiction,

science fiction embodies insights useful to the resolution of social

conflicts, which is what law is about. The insights are thought

experiments, scenarios, and hypothetical cases.

Compared to fiction,

two types of insights from science fiction are relevant for lawmakers.

The first concern technological or scientific change. The second

. In 2017, the European Parliament referred to the Three Laws in its Resolution

of 16 February 2017 with recommendations to the Commission on Civil Law Rules

on Robotics. 2018 O.J. (C 252) 25. In 2020, a French congressman introduced a draft

bill seeking to codify the Three Laws of robotics in the Preamble of the French

Constitution. Proposition de loi Constitutioneelle relative à la Charte, Assemblée

Nationale, 2020, No. 2585 (Fr.).

. Besides, a careful reader will know that Asimov hardly ever talked of AI, but

of robotics and positronic brains.

. Asimov wrote about 500 books during his career. STANLEY ASIMOV, YOURS, ISAAC

ASIMOV, x (1996). See also David Leslie, Isaac Asimov: centenary of the great

explainer, 577 NATURE 614 (2020). It is worth noting that Isaac Asimov published

three collections to celebrate the publication of his one hundredth (ISAAC ASIMOV,

OPUS 100 (1969)), two hundredth (ISAAC ASIMOV, OPUS 200 (1979)), and three

hundredth book (ISAAC ASIMOV, OPUS 300 (1984)). The first short story that Asimov

published was “The Callistan Menace” (initially named “stowaway”), in 1939. ISAAC

ASIMOV, THE EARLY ASIMOV 13 (1972) [hereinafter ASIMOV, THE EARLY ASIMOV]. For the

record, it is the second short story written by Asimov (or the third if we take into

account “Little Brothers”, a short text written by Asimov and published in the

literary review of Brooklyn Boys’ High School), the first being “Cosmic Corkscrow”

(never published). His first book was Pebble in the Sky (initially named Grow Old

with Me) in January 19

, 1950. ISAAC ASIMOV, PEBBLE IN THE SKY (1950). See ASIMOV, THE

EARLY ASIMOV. It therefore took twenty years for Asimov to publish one hundred

books (1950-1969), ten years to publish one hundred more (1969-1979), and five

more years to publish his three hundredth book (1979-1984).

. Giovanni Sartor, Human Rights in the Information Society: Utopias, Dystopias

and Human Values, in PHILOSOPHICAL DIMENSIONS OF HUMAN RIGHTS, 293 (Claudio

Corradetti ed., 2012). See also Kieran Tranter, LIVING IN TECHNICAL LEGALITY: SCIENCE

FICTION AND LAW AS TECHNOLOGY (2018).

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 3

concern social responses to changes in levels of science and

technology. Both the technological and social insights of science

fiction can help lawmakers overcome some of the world’s

uncertainty.

This paper attempts to derive lessons from Asimov’s works. Seven

lessons emerge. The first lesson concerns technological insights. The

others relate to social insights. The first two lessons are drawn from

Asimov’s writings on science fiction. The rest of the lessons stem from

Asimov’s works of science fiction.

The first lesson is that some technological insights of science fiction

are more ‘predictive’ than others.

The second lesson is that science

fiction contains useful conjectures about society.

The third lesson is

that societies initially object to technological change, but never end

up discarding it.

The fourth lesson is that regulation by design tends

to be societies’ initial response to technological uncertainty.

The fifth

lesson is that regulation by design is fallible.

The sixth lesson is that

human agency, common sense, and ingenuity play a key role in

solving problems of regulation by design.

The last lesson is that

regulation by design never obviates the need for (real) law.

This paper is divided in nine parts. The first part shows the

relevance of science fiction in general, and of Asimov’s writings, in

particular, in lawmaking contexts. The seven following parts describe

the lessons that can be drawn from Asimov’s works. The last part

shows how the lessons suggest that Asimov was neither a techno-

determinist, nor a techno-solutionist, but a techno-institutionalist.

. Asimov, himself, wrote an entire series about psychohistory, or the science

of predicting the ultra-long term future. ISAAC ASIMOV, FOUNDATION AND THE EMPIRE

(1952).

. See infra note 33 seq. and accompanying text.

. See infra note 65 seq. and accompanying text.

. See infra note 78 seq. and accompanying text.

. See infra note 97 seq. and accompanying text.

. See infra note 113 seq. and accompanying text.

. See infra note 124 seq. and accompanying text.

. See infra note 134 seq. and accompanying text.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

4 Journal of Business & Technology Law

Note that the paper takes a broad view of the concept lawmaking that

covers legislation, judicial decision, and administrative action.

I. WHY FICTION, WHY SCIENCE FICTION, WHY ASIMOV?

A critical eye will quickly pick up the implicit assumption of this paper.

The assumption is that useful lawmaking knowledge resides in fiction

literature (“fiction”), science fiction, and Asimov’s work.

The assumption is not the product of the writers’ imagination. Law

as a social science consists of the study of conflicts between

individuals, and how society solves them. Through this lens, fiction

constitutes a relevant source of knowledge for lawmakers.

Fiction

develops one’s “thinking about imaginary cases [that] can help us

learn new things about the world.”

In more specific terms, fiction

allows lawmakers to conduct thought experiments,

through “a

process of reasoning carried out within the context of a well-

articulated imaginary scenario in order to answer a specific question

about a non-imaginary situation.”

Now why science fiction? As its name suggests, science fiction

blends fiction, and science.

And science fiction draws heavily from

. MICHAEL SALER, AS IF: MODERN ENCHANTMENT AND THE LITERARY PRE-HISTORY OF

VIRTUAL REALITY 30 (2012) [hereinafter SALER, AS IF].

. TAMAR SZABÓ GENDLER, THOUGHT EXPERIMENT: ON THE POWERS AND LIMITS OF

IMAGINARY CASES 1 (2000) [hereinafter, GENDLER, THOUGHT EXPERIMENT].

. See Tamar Szabó Gendler, Galileo and the Indispensability of Scientific

Thought Experiment, 49, BRIT. J. PHIL. SCI., 397, 397-424 (1998); GENDLER, THOUGHT

EXPERIMENT, supra note 14, 18; S. . .ren Höggqvist, A Model for Thought Experiments,

39 CAN. J. PHIL., 55, 56 (2009); Kimberley Brownlee & Zofia Stemplowka, Thought

Experiments, in METHODS IN ANALYTICAL POLITICAL THEORY 21, 24 (2017); THE ROUTLEDGE

COMPANION TO THOUGHT EXPERIMENTS (Michael T. Stuart et. al. eds., 2017).

. GENDLER, THOUGHT EXPERIMENT, supra note 14, x.

. The writers do not aim to debate the definition of science fiction. Others have

dedicated entire book to this question. See THE CAMBRIDGE COMPANION TO SCIENCE

FICTION 1 (Edward James & Farah Mendlesohn eds., 2003); THE ROUTLEDGE COMPANION

TO SCIENCE FICTION 3 (Mark Bould, Andrex M. Butler, Adam Roberts and Sherryl Vint

eds., 2009); DAVID SEED, SCIENCE FICTION: A VERY SHORT INTRODUCTION 1 (2011); THE

OXFORD HANDBOOK OF SCIENCE FICTION (Robert Latham eds., 2014); ADAM ROBERTS, THE

HISTORY OF SCIENCE FICTION 51 (2016); SCIENCE FICTION CRITICISM: AN ANTHOLOGY OF

ESSENTIAL WRITINGS 1 (Robert Latham eds., 2017).

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 5

natural science.

The natural science dimension of science fiction is

interesting in a lawmaking context. Tapping into science fiction can

give lawmakers confidence. The knowledge mobilized is based largely

on facts, logic, and, analytics.

Of course, the point here is not that science fiction provides

accurate predictions of what technology will come next, or how

technology will reshape society. The science involved in science

fiction is neither “true” nor “false”.

Science fiction constitutes

“might-be-true” science relevant in contexts of uncertainty.

enables scenario planning, strategic foresight, and technological

forecasting.

In addition, science fiction can drive lawmakers to gain interest in

science, and search for better facts to be translated in law.

Asimov’s own words, science fiction is a “learning device” that

“stimulates curiosity and the desire to know.”

Science fiction, simply

put, might incentivize lawmakers to search for more or better facts.

Now, why cherry pick Asimov? Put differently, if science fiction is

so important, why does this paper focus on Asimov’s works, instead

of science fiction as a whole?

Leaving aside the writers’ own

preferences, there is an objective reason to focus on Asimov’s work.

Asimov was not just one of the “Big Three” authors of science

. Isaac Asimov, By No Means Vulgar, ISAAC ASIMOV’S SCIENCE FICTION MAGAZINE,

Sept.-Oct. 1978, reprinted in ISAAC ASIMOV, ASIMOV ON SCIENCE FICTION 46 (1981)

[hereinafter ASIMOV, ASIMOV ON SCIENCE FICTION].

. Eur. Parl. Doc. (COM 125) 2 (2015) (noting at 2 that “the principles of better

regulation will ensure that measures are evidence-based”).

. Isaac Asimov, The Name of Our Field, ISAAC ASIMOV’S SCIENCE FICTION MAGAZINE,

May-Jun. 1978, reprinted in ASIMOV, ASIMOV ON SCIENCE FICTION, supra note 18, at 25

[hereinafter Asimov, The Name of Our Field].

. Id.

. See infra note 65 seq. and accompanying text.

. More generally, Asimov himself wrote that science fiction constitutes “a way

of arousing people’s interest in science.” See Isaac Asimov, Learning Device, ISAAC

ASIMOV’S SCIENCE FICTION MAGAZINE, Aug. 1979, reprinted in ASIMOV, ASIMOV ON SCIENCE

FICTION, supra note 18, at 51 [hereinafter Asimov, Learning Device].

. Id.

. For such an analysis dedicated to media law through science fiction, see for

instance DAXTON R. STEWART, MEDIA LAW THROUGH SCIENCE FICTION: DO ANDROIDS DREAM

OF ELECTRIC FREE SPEECH? (2020).

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

6 Journal of Business & Technology Law

fiction.

He was also a scientist who earned a doctorate in chemistry

in 1948 and taught biochemistry at Boston University since 1949.

Asimov had been exposed to sociology in his first professional

endeavors.

At this stage, one question remains. What Asimov stories should

lawmakers read? Asimov was “a prolific writer.”

Many of his works

allow a law and policy discussion.

No clear test exists to determine

what is relevant, and what is not, in his abundant prose. The present

paper focuses on two strands of Asimov’s works.

. See Carl Freedman, Science Fiction and Utopia: A Historico-Philosophical

Overview, in LEARNING FROM OTHER WORLDS: ESTRANGEMENT, COGNITION, AND THE POLITICS

OF SCIENCE FICTION AND UTOPIA 72, 81 (Patrick Parrinder, ed., 2001). See also Isaac

Asimov, Call It SF or Sci-Fi, It’s Big!, in THE 1980 WORLD YEAR BOOK, WORLD BOOK

ENCYCLOPEDIA (1980), reprinted as How Science Fiction Came to Be Big Business in

ASIMOV, ASIMOV ON SCIENCE FICTION, supra note 18, at 121, 123-4 [hereinafter Asimov,

How Science Fiction Came to Be Big Business].

. ISAAC ASIMOV, IN MEMORY YET GREEN: THE AUTOBIOGRAPHY OF ISAAC ASIMOV, 1920-

1954, at 552-55 (1979).

. He worked as a typist for a sociologist while he was a student. ASIMOV, THE

EARLY ASIMOV, supra note 3, at 189.

. Isaac Asimov, The Prolific Writer, THE WRITER, Oct. 1979, reprinted in ASIMOV,

ASIMOV ON SCIENCE FICTION, supra note 18, at 315.

. For instance, an analysis of legal personhood with The Bicentennial Man,

Isaac Asimov, The Bicentennial Man, STELLAR SCIENCE FICTION, Feb. 1976 [hereinafter

Asimov, The Bicentennial Man], reprinted in ISAAC ASIMOV, THE COMPLETE ROBOT, 519

(1982) [hereinafter ASIMOV, THE COMPLETE ROBOT], romantic application of robots

with Satisfaction Guaranteed, Isaac Asimov, Satisfaction Guaranteed, AMAZING

STORIES, Apr. 1951, reprinted in ASIMOV, THE COMPLETE ROBOT at 285 [hereinafter

Asimov, Satisfaction Guaranteed], the legal consequences of the destruction of a

robot with The Robots of Dawn, ISAAC ASIMOV, THE ROBOTS OF DAWN (1983)

[hereinafter ASIMOV, THE ROBOTS OF DAWN], the paradox of deontology with Liar!,

Isaac Asimov, Liar!, ASTOUNDING SCIENCE FICTION, May 1941, reprinted in ASIMOV, THE

COMPLETE ROBOT at 267 [hereinafter Asimov, Liar!], Hans Kelsen’s PURE THEORY OF LAW

with Runaround, Isaac Asimov, Runaround, ASTOUNDING SCIENCE FICTION, Mar. 1942

[hereinafter Asimov, Runaround], reprinted in ASIMOV, THE COMPLETE ROBOT at 209,

and The Evitable Conflict, Isaac Asimov, The Evitable Conflict, ASTOUNDING SCIENCE

FICTION, June 1950 [hereinafter Asimov, The Evitable Conflict], reprinted in ASIMOV,

THE COMPLETE ROBOT at 447, the automated law enforcement with Evidence, Isaac

Asimov, Evidence, ASTOUNDING SCIENCE FICTION, Sep. 1946, reprinted in ASIMOV, THE

COMPLETE ROBOT at 425, and the Bentham’s utilitarianism especially with Robots and

Empire, ISAAC ASIMOV, ROBOTS AND EMPIRE (1985) [hereinafter ASIMOV, ROBOTS AND

EMPIRE], Prelude to Foundation, ISAAC ASIMOV, PRELUDE TO FOUNDATION (1988)

[hereinafter, ASIMOV, PRELUDE TO FOUNDATION], Forward the Foundation, ISAAC ASIMOV,

FORWARD THE FOUNDATION (1993) and Foundation and Earth, ISAAC ASIMOV, FOUNDATION

AND EARTH (1986).

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 7

First, the prevalence of concerns towards AI in the law and policy

conversation leads us to focus on the robot and supercomputer

(“Multivac”) stories. Second, the paper relies on Asimov’s writings on

science fiction, and in particular his 1981 volume “Asimov on Science

Fiction.”

The selection has shortcomings. Asimov’s robot and

Multivac stories represent no more than 15% of his science fiction

writings, and a total of approximately fifty short chapters and six

novels.

That said, the selection is a good proxy. Asimov robot stories

occupied a special place in his heart. They might be representative of

what he considered science fiction with a big S and F.

II. TECHNOLOGICAL INSIGHTS AND THE CLOSENESS TO SCIENCE RULE

Some technological insights of science fiction are prescient.

In 1865,

Jules Verne foresaw the first flight to the moon in “From the Earth to

the Moon”.

Star Trek prefigured intelligent user interfaces like

Amazon Alexa.

Asimov also successfully predicted the mass

diffusion of self-driving cars in society between 2015 and 2045 in the

short story Sally.

As a side note, Asimov correctly speculated that

the main driver of social adoption would be absolute safety,

compared to cars driven by human hand.

. ASIMOV, ASIMOV ON SCIENCE FICTION, supra note 18.

. Jacques Goimard, Asimov et les robots, in ISAAC ASIMOV, LE GRAND LIVRE DES

ROBOTS, 2 LA GLOIRE DE TRANTOR (1991).

. See Chip Stewart, Do Androids Dream of Electric Free Speech? Visions of the

Future of Copyright, Privacy and the First Amendment in Science Fiction 19 COMMC’N

L. & POL’Y 433, 436 (2014) (“[W]hile science fiction may not exist primarily to be

predictive, it is difficult to ignore the effectiveness of science fiction writers in

revealing technologies years if not decades before they become reality.”).

. JULES VERNE, DE LA TERRE A LA LUNE, TRAJET DIRECT EN 97 HEURES ET 20 MINUTES X, X

(Pierre-Jules Hetzel, 1865).

. Jeffrey P. Bezos, 2018 Letter to Shareholders, AMAZON (Apr. 11, 2019),

https://s2.q4cdn.com/299287126/files/doc_financials/annual/2018-Letter-to-

Shareholders.pdf, cited in NICOLAS PETIT, BIG TECH AND THE DIGITAL ECONOMY: THE

MOLIGOPOLY SCENARIO 142 note 236 (2020).

. Isaac Asimov, Sally, FANTASTIC, May-June 1953, reprinted in ASIMOV, THE

COMPLETE ROBOT, supra note 30, at 7 [hereinafter Asimov, Sally].

. Id. at 9. Asimov noted: “Every year machines like that [contemporary cars]

used to kill tens of thousands or people. The automatics [autonomous cars] fixed

that. A positronic brain can react much faster than a human, of course, and it paid

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

8 Journal of Business & Technology Law

Now, other technological insights of science fiction are very under-

predictive. Philip K. Dick’s Do Androids Dream of Electric Sheep?,

Robert Zemeckis’ Back to the Future,

and many other works from

the 1980s imagined flying cars as the future of technology for

mobility.

So how far should we trust the technological predictions of science

fiction? Asimov held nuanced views about the predictive force of

science fiction. On the one hand, he considered that “there is very

little in the vast output of science fiction […] which come true, or

which is ever likely to come true.”

On the other hand, he conceded

that “successful prediction [could] take place.”

In these cases,

science fiction resembles “futurism”, that is the “respectable

specialty thought of by those, in government and industry, who must,

every day, make decision by guessing the future”.

A close reading hints at how the technological insights of science

fiction should be used in a lawmaking context. In particular, Asimov

suggests that a test of closeness to science is what allows “a glimpse

of things that later turn out to be near the truth.”

Asimov wrote that

predictive science fiction tracks “trends in science and technology.”

In his writings on science fiction, Asimov exemplifies the point by

reference to two other science fiction works, Deadline and Solution

people to keep hands off the control. You got in, punched your destination, and let

it go its own way.”

. PHILIP K. DICK, DO ANDROIDS DREAM OF ELECTRIC SHEEP? (1968).

. See generally BACK TO THE FUTURE (Amblin Entertainment 1985).

. Lawmakers seeking technological facts in science fiction may develop wrong

hypotheses and create irrelevant laws. Nicolas Petit & Jerome De Cooman, Models

of Law and Regulation for AI, in THE ROUTLEDGE SOCIAL SCIENCE HANDBOOK OF AI 199,

204 (Anthony Elliott ed., 2021) [hereinafter Petit and De Cooman, Models of Law

and Regulation].

. Isaac Asimov, How Easy to See the Future!, NATURAL HISTORY, Apr. 1975, at 92,

reprinted in ASIMOV, ASIMOV ON SCIENCE FICTION, supra note 30, at 75 [hereinafter

Asimov, How Easy to See the Future].

. Id.

. Isaac Asimov, Science Fiction: Real Life Mirror of Social Change, PRISM, Jan.

1974, reprinted as Science Fiction and Society in ASIMOV, ASIMOV ON SCIENCE FICTION,

supra note 18, at 103 [hereinafter Asimov, Science Fiction and Society].

. Asimov, How Easy to See the Future, supra note 41, at 75.

. Id.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 9

Unsatisfactory by Cleve Cartmill and Robert A. Heinlein, respectively.

Both rightly predicted military applications of the nuclear bomb years

before Hiroshima in 1945.

Asimov wrote that “once uranium fission

was discovered, a nuclear bomb was an easy extrapolation.”

Asimov’s works do not give a test to separate serious technological

insights from junk ones.

But they describe science fiction unworthy

of consideration. Asimov was critical of science fiction weak in

empirical content.

He particularly loathed a generation of 1960s’

young writers who wrote fictional stories notwithstanding their lack

of “knowledge of science, and even sympathy for science.”

Asimov

spoke plainly of “trashy material.”

That insight embodies a clear test of distance versus closeness to

science. The test draws a line between what Asimov called

“entertainment stories,” “escape literature” and “anticipation

stories” on the one hand, and “realistic science fiction” on the other.

Table 1 applies this distance v closeness to science test to the twenty-

eight science fiction themes that Asimov once described as “a good

summary of the futuristic (and, possibly, predictive) aspects of science

fiction.”

. Cleve Cartmill, Deadline, Astounding Science Fiction (1944); Anson

MacDonald, Solution Unsatisfactory, Astounding Science Fiction (1941). Note that

Anson MacDonald was Robert A. Heinlein’s pseudonym.

. Asimov, How Easy to See the Future, supra note 41, at 78. “Even before the

bomb fell, science fiction stories were dealing not only with the bomb itself, but

with the nuclear stalemate, with peaceful uses of nuclear fission, and with the

possible radiation dangers of nuclear fission.” Asimov, How Science Fiction Came to

Be Big Business, supra note 26, at 115.

. Sure, Asimov wrote about facts that constitute “an extrapolation of the

present […] that is so clear and obvious as to forecast something is inevitable”.

Asimov, How Easy to See the Future, supra note 41, at 76. A textbook example of a

serious technological insight is the depletion of fossil fuel predicted in The Man Who

Awoke. Laurence Manning, The Man Who Awoke, WONDER STORIES, Mar. 1933, at

757, reprinted in LAURENCE MANNING, THE MAN WHO AWOKE (1975).

. See Asimov, Science Fiction and Society, supra note 43, at 103-104.

. Id., at 104.

. Asimov, The Name of Our Field, supra note 20, at 27.

. Asimov, How Easy to See the Future, supra note 41, at 78.

. Asimov, The Dreams of Science Fiction, in ASIMOV, ASIMOV ON SCIENCE FICTION,

supra note 30, at 81.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

10 Journal of Business & Technology Law

Technological Insights of Science Fiction

Distance to Science

Closeness to Science

Weather Control

World Government

Mass Transference

Immortality

Telepathy

Interspecies Communication

Space Settlements

Terraforming

Gravitational Control

Interstellar Communication

Interstellar Travel

Black Holes

Galactic Empires

Time Travel

Alternate Time Paths

Population Control

Permanent Energy Sources

Robots

Computers

Computerized Education

Global Village

Cloning

Bionic Human Beings

Genetic Engineering

Control of Evolution

Exploitation of Near Space

Low-gravity Flying

Interplanetary Travel

Table 1 – Distance v closeness to science test applied to Asimov’s

technological insights of science fiction (Source: Isaac Asimov, The

Dreams of Science Fiction, in Isaac Asimov, Asimov on Science Fiction

(Doubleday 1981) 81-89.

The left column of the table shows technological insights that

Asimov deemed distant to science.

By distant to science, Asimov

meant works far from applied, practical, and empirical science.

Despite some theoretical grounding, for example, the science of time

. Id. Asimov explained how “[s]cience fiction can have its fantasy aspects. I

have written stories about galactic empires, about faster-than-light speeds, about

intelligent robots which eventually became God, about time travel. I don’t consider

that any of these have predictive value, they weren’t intended for that. I was just

trying to write entertaining stories.” See Asimov, How to Easy See the Future, supra

note 41, at 78.

. Asimov argued that science fiction came as a literary response to noticeable

technological change. He explained that science fiction writers write stories dealing

with “reasonable advances in technology” and targets “what such advances might

mean to society.” Asimov, How Science Fiction Came to Be Big Business, supra note

26, at 116.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 11

travel or mass transference has kept hitting walls in applied

contexts.

The right column, by contrast, shows technological

insights that are close to science. With the benefit of hindsight, all or

most appear within our reach today.

Now, not all technological insights that are distant to science are

irrelevant. If science fiction does not predict the future, it might well

construct it. The view that space settlements (left column) are a far

away technology frontier has motivated gigantic investments and

research in space technology.

The implication is that some distant

to science technological insights might actually play a key role in

constructing the future, thus they should not be underestimated.

Conversely, as science advances, technological insights that are

close to science can become “hopelessly wrong.”

For example, using

state of the art science, Asimov incorrectly described Venus as a

worldwide ocean.

Similarly, the galaxy in the Foundation series

exists without quasars, pulsars, or black holes.

Overall, Asimov’s closeness to science rule is thus far from perfect.

The rule produces false positives (and negatives). Many technological

insights that pass the closeness to science test are not more worthy

than astrology. Besides, when technological insights are strongly

predictive, the rule gives little details. Asimov correctly predicted the

first flight to the moon. But he got a lot of things wrong including the

timing.

. David Deutsch & Michael Lockwood, The Quantum Physics of Time Travel, in

SCIENCE FICTION AND PHILOSOPHY: FROM TIME TRAVEL TO SUPERINTELLIGENCE (Susan

Schneider ed., 2nd ed., 2016); Tongcang Li & Zhang-Qi Yin, Quantum superposition,

entanglement, and state teleportation of a microorganism on an electromechanical

oscillator, 61, SCI. BULL., 163 (2016).

. People like Elon Musk or Jeff Bezos certainly consider settlement to be

realistic. Mars & Beyond: The Road to Making Humanity Multiplanetary, SPACEX,

https://www.spacex.com/human-spaceflight/mars/ (last visited Sep. 30, 2022).

. See Asimov, Learning Device, supra note 23, at 48.

. See PAUL FRENCH, LUCKY STARR AND THE OCEANS OF VENUS (1954). Paul French is

the pseudonym sometimes used by Isaac Asimov.

. See Asimov, Learning Device, supra note 23, at 50.

. See Isaac Asimov, Trends, ASTOUNDING SCIENCE FICTION, Jul. 1939, reprinted in

Asimov, THE EARLY ASIMOV, supra note 3, at 76 [hereinafter Asimov, Trends].

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

12 Journal of Business & Technology Law

The main merit of the closeness to science rule lies elsewhere. The

test reduces the risk of succumbing to what we called the flying car

fallacy.

Lawmakers mitigate the risk of adoption of irrelevant law by

avoiding reliance on entertainment, escape, and anticipation science

fiction.

To take a concrete example, lawmakers today concerned

with Artificial General Intelligence (“AGI”) should think twice. AI’s

science struggles to make autonomous machines cooperate. A

“treacherous turn” of robotic insurgence as in Michael Crichton’s

Westworld, Alex Garland’s Ex Machina, or even Nick Bostrom’s

Superintelligence is not for tomorrow.

III. WHY AND HOW SCIENCE FICTION’S SOCIAL INSIGHTS MATTER

Someone once said that “a good science-fiction story should be able

to predict not the automobile but the traffic jam”.

The point is

apparent. Asimov wrote that the social impacts of technology

constitute the “the core of science fiction – its essence.”

Contrary

to a popular perception, science fiction does not simply concern itself

with stories about technology.

Science fiction is about “human

responses to changes in the level of science and technology.”

The

social changes triggered by technology matter as much as

technological changes.

In Asimov’s mind, the implication was clear.

Lawmakers cannot

ignore science fiction. He wrote:

. See Petit & De Cooman, Models of Law and Regulation for AI, supra note 40.

. Id.

. See NICK BOSTROM, SUPERINTELLIGENCE: PATHS, DANGERS, STRATEGIES (2014).

. Frederick Pohl, The Great Invention, GALAXY MAGAZINE SCIENCE FICTION, Dec.

1968, at 6.

. Isaac Asimov, My Own View, in THE ENCYCLOPEDIA OF SCIENCE FICTION (Robert

Holdstock ed., 1978) [hereinafter Asimov, My Own View], reprinted in ASIMOV,

ASIMOV ON SCIENCE FICTION, supra note 18, at 19.

. Id. “It is not that science fiction predicts this particular change or that that

makes it important, it is that it predicts change” (we do not emphasize).

. Id.

. Asimov repeated this idea time and time again. See Asimov, My Own View,

supra note 66, at 17; See also Asimov, How Easy to See the Future, supra note 41,

at 75; Isaac Asimov, The Prescientific Universe, ASIMOV’S SCIENCE FICTION MAGAZINE,

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 13

No sensible decision can be made any longer without taking into account

not only the world as it is, but the world as it will be – and naturally this

means that there must be an accurate perception of the world as it will

be. This, in turn, means that our statesmen, our businessmen, our

everyman must take on a science fictional way of thinking, whether he

likes it or not, or even whether he knows it or not. Only so can the deadly

problems of today be solved.

But why? Science fiction enables lawmakers to understand social

responses to technological change, to discuss them, and to act upon

them.

In July 1939, Asimov wrote Trends, a short story that dealt

with the first flight to the moon.

Asimov astutely forecasted

ideological opposition to space flight that arose in the late 1960s.

Now should lawmakers be as cautious with science fiction’s social

insights as they are with technological facts? Probably not. The

reason is that science fiction often draws from history to conjecture

social responses to technological change. Star Wars, Battlestar

Galactica, The Expanse, or even Foundation, are revisitations of the

Cold War, the Age of Discovery, and the fall of the Roman Empire.

Asimov gave great consideration to history.

Understanding the past

Summer 1979, reprinted in ASIMOV, ASIMOV ON SCIENCE FICTION, supra note 18, at 93;

Asimov, Science Fiction and Society, supra note 43, at 97.

. See Asimov, My Own View, supra note 66 at 19.

. See SALER, AS IF, supra note 13.

. See Asimov, Trends, supra note 61 at 301.

. See Asimov, How Easy to See the Future, supra note 41, at 78.

. See respectively POLI SCI FI: AN INTRODUCTION TO POLITICAL SCIENCE THOUGH SCIENCE

FICTION (Michael A. Allen & Justin S. Vaughn eds., Routledge 2016); BATTLESTAR

GALACTICA AND INTERNATIONAL RELATIONS (Nicholas J. Kiersey & Iver B. Neumann eds.,

Routledge 2014). See also ASIMOV, THE EARLY ASIMOV, supra note 3, at 385 (explaining

that Foundation was a “story against the background of the slow fall of the Galactic

Empire (something I [Asimov] intended to model quite frankly on the fall of the

Roman Empire”)).

. See Asimov, THE EARLY ASIMOV, supra note 3, at 143 (explaining “In both the

situation I pictured on Earth was inspired by that of Judea under the Romans. The

climactic battle in “Black Friar of the Flame,” however, was inspired by that of the

Battle of Salamis, the great victory of the Greeks over the Persians. In telling future-

history I always felt it wisest to be guided by past-history. This was true in the

“Foundation” series, too.”).

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

14 Journal of Business & Technology Law

is the best way to learn about the future.

Moreover, history being

contingent and subject to constant revision, no true or false

proposition can be made about science fiction’s social insights. Unlike

with technological insights, this property should give more confidence

to lawmakers interested in understanding the repercussions of

technology on the evolution of society.

Besides, the fact that the

social insights are found in escape, entertainment, or anticipation

literature is less consequential.

IV. SOCIAL HOSTILITY, THEN AFFINITY, TO TECHNOLOGICAL CHANGE

Asimov’s works call our attention to one key social insight. Societies

initially object to technological change but never end up discarding it.

Societies tend to discount the present and idealize the past. In his

words, “the happy pastoral world […] never existed except in the

mind of Nostalgia.”

It is a popular bias to view the past as a steady

state. Because humans are on average averse to change, the past

tends to be reified. Logically, when a technological discontinuity

occurs, this is contrasted with the “good old days.”

But an initial attitude of knee jerk hostility is often followed by one of

technological lucidity. Even when a technology is dangerous, humans

are reluctant to discard it. Malicious robots, for example, populate

Asimov’s stories.

Yet, they are not banned. Robots are introduced

. Jacques Goimard, Asimov et nous, in ISAAC ASIMOV, LE GRAND LIVRE DES ROBOTS,

1 PRÉLUDE À TRANTOR, xix (1990) [hereinafter ASIMOV, LE GRAND LIVRE DES ROBOTS 1].

. Isaac Asimov, Social Science Fiction, in MODERN SCIENCE FICTION: ITS MEANING AND

ITS FUTURE (Regina Bretnor ed., 1953).

. Isaac Asimov, The One Ring is What We Make It, in PANORAMA 43, 1980,

reprinted as The Ring of Evil in ASIMOV, ASIMOV ON SCIENCE FICTION, supra note 18, at

279 [hereinafter Asimov, The Ring of Evil].

. Isaac Asimov, How Easy to See the Future!, supra note 41, at 76.

. See Reason, a robot concludes human beings are inferior beings and locks

them up. Isaac Asimov, Reason, ASTOUNDING SCIENCE FICTION, Apr. 1941, at 33,

reprinted in ASIMOV, THE COMPLETE ROBOT, supra note 30, at 227 [hereinafter Asimov,

Reason]; in Liar!, a telepathic robot chooses to lie even if it means causing great

psychological trouble once the truth is discovered. Isaac Asimov, Liar, supra note

30, at 267; in Little Lost Robot, a robot develops a sense of superiority and is ready

to hurt human beings to remain hidden. Isaac Asimov, Little Lost Robot, ASTOUNDING

SCIENCE FICTION, Mar. 1947, at 111, reprinted in ASIMOV, THE COMPLETE ROBOT, supra

note 30, at 349 [hereinafter Asimov, Little Lost Robot]; In Someday, a robot Bard

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 15

on other planets where they have limited interactions with humans.

This allows Asimov to write that “at no time in the history of mankind

has any culture voluntarily given up significant technological advances

because of the inconvenience of harm or side effects.”

Many novels of Asimov go even further, painting a picture of long-

technological affinity. In a story called Someday, giant computers

manage the human population. Asimov writes:

Some [computers] ran factories, and some ran farms. Some organized

population and some analyzed all kinds of data. Many were very

threatens that one day, computers will take over the world. Isaac Asimov, Someday,

INFINITY SCIENCE FICTION, Aug. 1956, reprinted in ASIMOV, THE COMPLETE ROBOTS, supra

note 30 at 35 [hereinafter Asimov, Someday]; in True Love, a computer arranges

evidence to ensure its designer was convicted for malfeasance. Isaac Asimov, True

love, AMERICAN WAY, Feb. 1977, reprinted in ASIMOV, THE COMPLETE ROBOT, supra note

30, at 51.

. See Asimov, Reason, supra note 80, in which the robot is operating on a space

station. Id.; in Runaround, the robot is operating on Mercury. Asimov, Runaround,

supra note 30, at 94; in Robot Al-76 Goes, a robot is designed for lunar mining (but

is accidentally released on Earth). Isaac Asimov, Robot Al-76 Goes Astray, AMAZING

STORIES, Feb. 1942, at 218, reprinted in ASIMOV, THE COMPLETE ROBOT, supra note 30,

at 59 [hereinafter Asimov, Robot Al-76 Goes Astray]. See also Asimov, Little Lost

Robot, supra note 80; in Risk, the robot is designed to pilot a spaceship. Isaac

Asimov, Risk, ASTOUNDING SCIENCE FICTION, May 1955, at 60, reprinted in ASIMOV, THE

COMPLETE ROBOT, supra note 30, at 375 [hereinafter Asimov, Risk]; in First Law, the

robot is operating on Saturn’s moon Titan. Isaac Asimov, First Law, FANTASTIC

UNIVERSE, Oct. 1956, at 29, reprinted in ASIMOV, THE COMPLETE ROBOT, supra note 30,

at 205. Much more interesting, Asimov imagined Earth authorities vote a ban on

robots in Satisfaction Guaranteed. Asimov, Satisfaction Guaranteed, supra note 30,

at 285. The same idea is found in …That Thou Art Mindful of Him. Isaac Asimov,

…That Thou Art Mindful of Him, FANTASY AND SCIENCE FICTION, March 1974, reprinted

in ASIMOV, THE COMPLETE ROBOT, supra note 30. The idea of (partial) robot ban on

Earth and full acceptance in space is also found in Asimov’s novels. See ISAAC ASIMOV,

THE CAVES OF STEEL (1954) [hereinafter ASIMOV, THE CAVES OF STEEL]; ISAAC ASIMOV, THE

NAKED SUN (1957) [hereinafter ASIMOV, THE NAKED SUN]; ASIMOV, THE ROBOTS OF DAWN,

supra note 30.

. See ISAAC ASIMOV, The Myth of the Machine, in SCIENCE FICTION: CONTEMPORARY

MYTHOLOGY 244 (Patricia S. Warrick et al. ed. 1978) [hereinafter Asimov, The Myth

of the Machine], reprinted in ASIMOV, ASIMOV ON SCIENCE FICTION, supra note 30 at 148.

Note though that Asimov’s statement deserves to be nuanced. At times, some

technologies have been abandoned. An example is the ban on chemicals such as

chlorofluorocarbons.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

16 Journal of Business & Technology Law

powerful and very wise, much more powerful and wise than the step-

people who were so cruel to the little computer.

One young boy in the story cannot believe that a past existed in

which “farmers grew things with their hands and people had to do all

the work in the factories and run all the machines.”

In other stories

about Multivac, when some people started questioning computers’

dominion, Asimov wrote “Have you forgotten? Have you all

forgotten? Do you remember how it once was? Do you remember the

century? We live long now; we live securely now; we live happily

now.”

Technological affinity is very Asimovian. Technology in general, and

computers in particular, are useful. They solve “all the world’s

problem.”

In Franchise, the computer dispenses with organizing

presidential elections by asking a few questions to an individual

designated as the most representative of the entire population.

All the Troubles of the World, Multivac improves law enforcement by

predicting crimes before they happen.

And in The Evitable Conflict,

supercomputers end up optimizing the global economy by nudging

humankind towards what machines consider the right direction.

Susan Calvin, one of the most important protagonists of Asimov’s

stories, qualifies this as “wonderful.”

But in Asimov’s world, technological affinity is not just about

convenience. It is also about trust. Asimov’s stories feature countless

. See Asimov, Someday, supra note 80, at 35.

. Id. at 32.

. Isaac Asimov, The Life and Times of Multivac, N. Y. TIMES, Jan. 5, 1975, at 166,

168.

. Isaac Asimov, Point of View, BOYS LIFE, July 1975, at 34, 34, reprinted in

ASIMOV, THE COMPLETE ROBOT, supra note 30, at 37 (explaining how “the world’s

problems have become so serious and the questions we are asked are so

complicated that it takes all [the robot’s] smartness to answer them.” Id.).

. Isaac Asimov, Franchise, IF: WORLDS OF SCIENCE FICTION, Aug. 1955, at 2,

reprinted in ISAAC ASIMOV, ROBOT DREAMS 193 (1986) [hereinafter ASIMOV, ROBOT

DREAMS].

. Isaac Asimov, All the Troubles of the World, SUPER-SCIENCE FICTION, Apr. 1958,

reprinted in ISAAC ASIMOV, NINE TOMORROWS 137, 137-153 (1959).

. See Asimov, The Evitable Conflict, supra note 30.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 17

examples of strong bonds between robots and humans. We have the

theme of the robot-pet preferred to animals in A Boy’s Best Friend,

the robot-butler in Light Verse,

the robot-babysitter in Robbie,

even the baby-robot in Lenny.

Far from a solutionist, Asimov however considered the need for

balanced complementarity between technology and humanity. The

Caves of Steel provides a good illustration.

On the one hand, Asimov

depicts Earth’s society as one using very few robots and banning them

in cities. The upshot is stagnation. On the other hand, robots are

widely used in Spacers (first Earth settlement wave) colonies. Too

much perhaps, as full reliance on robots leads to a decline of the

Spacers’ civilization which becomes complacent, lazy, and self-

satisfied.

For Asimov, the Settlers’ (second Earth settlement wave)

. Isaac Asimov, A Boy’s Best Friend, BOYS LIFE, Mar. 1975, at 26, reprinted in

ASIMOV, THE COMPLETE ROBOT, supra note 30, at 3.

. Isaac Asimov, Light verse, in ASIMOV, THE COMPLETE ROBOT, supra note 30, at

121, 123. Asimov wrote, explaining why one of his characters “always addresses her

robots with the most formal courtesy”:

“‘I do not ask for speed and efficiency,’ she said. ‘I ask goodwill. My robots love

me.’…’Once a robot is in my house,’ she said, ‘and has performed his duties, any

minor eccentricities must be borne with. I will not have him manhandled.’…

‘Nothing that is as intelligent as a robot can ever be but a machine. I treat them as

people.’” Id.

. Isaac Asimov, Robbie, SUPER SCIENCE FICTION, Sept. 1940, reprinted in ASIMOV,

THE COMPLETE ROBOT, supra note 30, at 133, 138-39. Asimov wrote:

“A robot is infinitely more to be trusted than a human nursemaid. Robbie was

constructed for only one purpose really – to be the companion of a little child. His

entire ‘mentality’ has been created for the purpose. He just can’t help being faithful

and loving and kind. He’s a machine-made so. That’s more than you can say for

humans.”

. Isaac Asimov, Lenny, INFINITY SCIENCE FICTION, Jan. 1958, reprinted in ASIMOV,

THE COMPLETE ROBOT, supra note 30, at 301.

. ASIMOV, THE CAVES OF STEEL, supra note 81.

. ASIMOV, THE NAKED SUN, supra note 81. Asimov compares them with the

Ancient city of Sparta, wherein Spartan citizens were outnumbered by their helots.

The overreliance on technology is a theme that was developed by Asimov in The

Feeling of Power. In this short story, he warns against the side effect of excessive

use of the calculating machine. The Feeling of Power, IF: WORLDS OF SCIENCE FICTION,

Feb. 1958, reprinted in ASIMOV, ROBOT DREAMS, supra note 87 at 301. Asimov also

analyzes this in “The Computerized World.” Isaac Asimov, The Computerized World,

[hereinafter Asimov, The Computerized World] reprinted in ISAAC ASIMOV, THE ROVING

MIND, 214-27 (2nd ed., 1997) [hereinafter ASIMOV, THE ROVING MIND].

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

18 Journal of Business & Technology Law

middle-ground approach is better.

A well-balanced human-robot

combination leads to an effective expansion of humanity.

V. THE INEVITABILITY OF REGULATION BY DESIGN

Another social insight from Asimov is that solution to technological

risks does not lie in the “abandonment of technology, but [in]

additional technology.”

Drawing from history, Asimov wrote that

“the danger of the spear was countered by the shield.”

The robot

stories embody a clear view that societies rely on technology to

reduce the dangers of scientific discoveries. In contrast to science

fiction that posits “robot-as-menace” or “robot-as-pathos,” Asimov’s

stories envision robots as tools.

100

No more, no less. Now, Asimov’s

functional perspective on robots has a key implication. Considered as

machines built by problem solving-minded engineers, it is safe to

assume that robots will be secured. Asimov wrote:

Knives are manufactured with hilts so that they may be grasped safely,

stairs possess banisters, electric wiring is insulated, pressure cookers

have safety valves—in every artifact, thought is put into minimizing

danger. (…)

Consider a robot, then, as simply another artifact. It is not a sacrilegious

invasion of the domain of the Almighty, any more (or any less) than any

. ASIMOV, ROBOTS AND EMPIRE, supra note 30.

. Asimov reaches a similar conclusion in his writings on science fiction. For

Asimov, “the two intelligences, human and computers, may supplement far more

than compete and, in cooperation, may do far more than either separately could.”

Asimov, The Computerized World, supra note 95 at 226. He explained that human-

machine relation is “a matter of complementation” and wrote, “It could be that a

human and computer might form a symbiotic intelligence that would be far greater

than either could develop alone, a symbiotic intelligence that would open new

horizons and make it possible to achieve new heights. Isaac Asimov, Homo

Obsoletus?, reprinted in ASIMOV, THE ROVING MIND, supra note 95 at 305.

. Asimov, The Myth of the Machine, supra note 82 at 148. See also the nuance

we made supra note 82, acknowledging some technologies were actually

abandoned.

. Id.

100

. ASIMOV, THE COMPLETE ROBOT, supra note 30, at xi.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 19

other artifact is. As a machine, a robot will surely be designed for safety,

as far as possible.

101

From this functional perspective, Asimov derives a fundamental

idea. There is an inevitability of regulation by design. The Three Laws

are just an emendation of that predicate.

102

In a 1942 short story

called Runaround, Asimov’s Three Laws are introduced to elaborate

the types of safety safeguards that can be built by design in a robot.

103

The Three Laws are:

First Law: A robot may not injure a human being or, through inaction,

allow a human being to come to harm.

Second law: A robot must obey the orders given it by human beings

except where such orders would conflict with the First Law.

Third Law: A robot must protect its own existence as long as such

protection does not conflict with the First or Second Laws.

A related insight from Asimov is that because solution for

technological risks consists in adding technology, regulation by design

is an iterative response to technological problems. The novel The

Naked Sun provides an example.

104

The story is about a poisoning

crime. A human orders a robot to pour poison in a glass for an

experiment. The robot has to obey in compliance with the Second

Law. Another robot is subsequently told to give the glass to its master.

The second robot, which hands over the glass, does not know about

the poison, and does not consider it possible to breach the First

101

. ISAAC ASIMOV, THE REST OF THE ROBOTS, xiii (1965). For an explanation of the

Asimov’s conception of robot as an artifact, see Gorman Beauchamp, The

Frankenstein Complex and Asimov’s Robots, 13, No. 3/4 MOSAIC: AN INTERDISCIPLINARY

CRITICAL JOURNAL 85 (1980) [hereinafter Beauchamp, Frankenstein Complex].

102

. ASIMOV, THE COMPLETE ROBOT, supra note 30, at xi.

103

. Asimov, Runaround, supra note 30. However, we have to note that Asimov

explained it is John W. Campbell, Jr. who coined the famous Three Laws although

the latter assures it is Asimov who drafted them. ASIMOV, THE EARLY ASIMOV, supra

note 3, at 309. Asimov later wrote a fourth law. ASIMOV, PRELUDE TO FOUNDATION,

supra note 30, at 397. This law prevails over the First and is therefore called the

Zeroth Law “since zero comes before one.” Id. The Zeroth Law is, “A robot may not

harm humanity, or, by inaction, allow humanity to come to harm.” Id.

104

. ASIMOV, THE NAKED SUN, supra note 80.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

20 Journal of Business & Technology Law

Law.

105

A robot can therefore kill a human by ignorance, in spite of

the First Law. In following works, Asimov subsequently upgrades the

First Law as follows:

(Modified) First Law: A robot may do nothing that, to its knowledge,

will harm a human being; nor, through inaction, knowingly allow a

human being to come to harm.

106

Now, Asimov has not been always consistent in his functional

perspective on robots. Some of his stories feature robots as living

beings. In Little Lost Robot, Susan Calvin explains:

All normal life, (…) consciously or otherwise, resents domination. If the

domination is by an inferior, or by a supposed inferior, the resentment

becomes stronger. Physically, and, to an extent, mentally, a robot – any

robot – is superior to human beings. What makes him slavish, then? Only

the First Law! Why, without it, the first order you tried to give a robot

would result in your death.

107

105

. Some have argued that Asimov, while fighting against the fear of robot

(Asimov called “the Frankenstein Complex”), actually reinforced it by proposing

subtler situations that give readers new reasons to fear despite the Three Laws. See

Beauchamp, Frankenstein Complex. Despite Asimov’s works, the Frankenstein

Complex is alive and well. From Stanley Kubrick 2001: A Space Odyssey to the recent

Westworld TV Serie, robots are seen as dangerous things. Hollywoodian

productions of these seventy last years are full of examples of this pitch. May it

suffice to name, without intending to make an exhaustive list, Terminator, the

Matrix, Blade Runner, Ex Machina and I Am Mother. Video games follow a similar

trend. Again, not exhaustively, see Mass Effect, Detroit: Become Human and

Horizon: Zero Dawn. This fear is, however, not solely a fictional theme. Scientists

also warn about the rapid pace of technological change. This is the case of Nick

Bostrom in Superintelligence. Stephen Hawking, though not an AI expert, once

declared that he feared artificial intelligence could be the greatest achievements of

humanity, but also its last one. Rory Cellan-Jones, British Broadcasting Corporation,

Stephen Hawking warns artificial intelligence could end mankind, (2 December

2014), https://www.bbc.com/news/technology-30290540. And the controversy

between the optimist Mark Zuckerberg and the pessimist Elon Musk over the

dangerousness of AI is well known. Cade Metz, Mark Zuckerberg, Elon Musk and

the Feud Over Killer Robots, N.Y. Times June 9, 2018

https://www.nytimes.com/2018/06/09/technology/elon-musk-mark-zuckerberg-

artificial-intelligence.html.

106

. ASIMOV, THE NAKED SUN, supra note 80.

107

. Asimov, Little Lost Robot, supra note 80.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 21

Additionally in Robot Dreams

108

, a robot named Elvex starts

dreaming after receiving a new brain programmed using a non-usual

method.

I saw that all the robots were bowed down with toil and affliction, that

all were weary of responsibility and care, and I wished them to rest (…).

In my dream, (…) it seemed to me there was neither First nor Second Law,

but the only the Third, and the Third Law was ‘A robot must protect its

own existence.’ That was the whole of the Law.

109

Susan Calvin’s explanations cast doubt on the exact nature of the

robot, either conscious being or simple machine. She explains the

dreamed robot’s whish of freedom: “As we would say of a human

being, not consciously. But who would have thought there was an

unconscious layer beneath the obvious positronic brain paths, a layer

that was not necessarily under the control of the Three Laws?”

110

What should we think about this? In a straight application of the

closeness to science rule, Asimov would not draw much from his own

description of robots as living beings.

111

The science of artificial

consciousness has historically been mired with controversies. On the

contrary, Asimov astonishingly correctly forecasted the current

discussions on regulation by design laid up in the recent proposal for

an EU Regulation of artificial intelligence.

112

108

. Isaac Asimov, Robot Dreams, in ASIMOV, ROBOT DREAMS, supra note 87.

109

. For the record, this short story ends with the destruction of the robot, when

he finally reveals that in his dreams, he is a man – not a robot – who claims “Let my

people go!”. Id.

110

. Id. at 27.

111

. Asimov, The Computerized World, supra note 95 at 214.

112

. Commission Regulation 2021/206 final, REGULATION OF THE EUR. PARL. AND

OF THE COUNCIL LAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE

(ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION LEGISLATIVE

ACTS (EC). Article 9 for instance requires that risk raised by AI systems have to be

eliminated or reduced “as far as possible through adequate design and

development.” Id.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

22 Journal of Business & Technology Law

VI. THE FALLIBILITY OF REGULATION BY DESIGN

Even more than the inevitability of regulation by design, Asimov’s

Three Laws show its insufficiency. The key message of Asimov’s robot

stories is that regulation by design is set to fail. The story Runaround,

where the Three Laws are first introduced, provides a good

illustration.

113

Two scientists of US Robotics, Gregory Powell and Michael

Donovan are sent to Mercury.

114

Life support systems on Mercury

require selenium to produce oxygen. The scientists send a new robot

named Speedy on a selenium extraction mission. Speedy does not

come back. Stocks of oxygen are running low. Powell and Donovan go

on a search for the robot. They find Speedy running in circle around a

selenium pool.

Speedy’s odd behavior can be explained as follow. The robot

received the order to extract selenium. According to the Second Law,

it must obey. But Speedy is a very expensive robot. To limit potential

damages, developers reinforced the importance of the Third Law. In

short, Speedy is risk-averse by-design.

115

Now, the problem is that the

selenium pool where the robot must go is located on a high volcanic

intensity area. Due to the reinforced Third Law, Speedy cannot extract

selenium. As Powell and Donovan forgot about the reinforcement of

the Third Law, they gave the order without stressing the life-

threatening emergency of the mission. The result is that the Third Law

partially bypassed the Second Law. Speedy faced an order forcing it

113

. Asimov, Runaround, supra note 30.

114

. In 1942, when Asimov wrote Runaround, scientists believed Mercury always

show the same face to the sun due to a synchronous rotation (tidal locking).

Scientists have proved later it is not true. This is another illustration of what we

explained above with the intertwining of science and science fiction See supra notes

33-64 and accompanying text.

115

. Asimov, Runaround, supra note 30. The author wrote:

“The conflict between the various rules is ironed out by the different positronic

potentials in the brain. . .. Speedy is one of the latest models, extremely specialized

. . . So Rule 3 has been strengthened . . . so that his allergy to danger is unusually

high. At the same time, when you sent him out after the selenium, you gave him his

order casually and without special emphasis, so that the Rule 2 potential set-up was

rather weak.” .

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 23

to go to the pool under the Second Law, and an equivalent order to

stay away from the pool under the Third Law. The robot ended up

running around the pool in circles, each point of the circle

corresponding to an equilibrium position between the Second and

Third Laws. The conflict of instructions made Speedy acts like it was

“drunk.”

116

Runaround is an important story in Asimov’s works. For the first

time that Asimov displays the Three Laws, he chooses a context of

failure. Asimov knew the Three Laws were too simple to solve every

imaginable problem. He explains that the Three Laws cannot walk

away from the constraints of nature:

Sometimes the safety achieved is insufficient because of limitations

imposed by the nature of the universe or the nature of the human mind.

However, the effort is there. (…) The safety may not be perfect (what is?),

but it will be as complete as men can make it.

117

Humans, who write the laws, have cognitive limitations. And

robotic laws, like rules of law, run into friction with the natural world

when applied in reality.

The natural fallibility of law is a feature of Asimov’s stories, not a

bug. In Little Lost Robot, Asimov writes about law’s unintended

consequences.

118

The story features scientists who work with

radiation that are lethal when exposure lasts a long time. Robots are

in a worse predicament. Radiations destroy robots’ circuits

immediately. Now robots systematically try to save scientists from

the future danger of radiation (under the First Law) with the result

that they destroy themselves (infringing the Third Law). Scientists

116

. Asimov wrote: There’s some sort of danger centering at the selenium pool. It

increases as he approaches, and at a certain distance from it the Rule 3 potential,

unusually high to start with, exactly balances the Rule 2 potential, unusually low to

start with. . . .So he follows a circle around the selenium pool, staying on the locus

of all points of potential equilibrium. . . . And that, by the way, is what makes him

drunk. At potential equilibrium, half the positronic paths of his brain are out of

kilter. Id. at 127.

117

. ISAAC ASIMOV, I, ROBOT (1950).

118

. Asimov, Little Lost Robot, supra note 80 at 351.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

24 Journal of Business & Technology Law

order robots to stay out of the radiation field (under the Second Law).

But the instruction remains ignored because the First Law takes

precedence over the Second. So, scientists decide to modify the First

Law that says “no robot may injure a human”, by removing the

segment “or through inaction, allow a human to come to harm.” With

this, robots can watch scientists work under radiation, and stay put.

One day, however, an upset scientist tells Nestor 10, a modified

robot, “go lose yourself” while insulting him generously. Under the

Second Law, the modified robot obeys. It runs away and hides

amongst sixty-two non modified robots freshly arrived. The modified

robot must be found. Humans would never accept letting modified

robots operate without tracking. At the same time, US Robotics would

never want to destroy all sixty-three expensive robots. Dr Calvin runs

experiments to find the lost robots. Many attempts fail. At some

point, the modified robot lies to protect itself under the Third Law. Dr

Calvin ends up tricking the modified robot.

119

Other examples of fallibility of regulation by design are abound in

Asimov’s stories. Hereafter is a short but representative sample:

• In Risk,

120

Asimov illustrates the difficulty of formulating

law. The story features a robot used as a test pilot for the

first hyper spatial flight. Engineers have instructed the

robot to firmly pull back the control bar. The order is quite

vague. The robot executes the instruction under the

Second Law. However, being much stronger than a

human, the bar bends and the test fails;

• In Robot Al-76 Goes Astray, Asimov shows the context

dependence limitations of regulation by design. The pitch

is simple. A lunar mining robot is mistakenly released on

Earth. Asimov wrote “Its positronic brain was equipped for

a lunar environment, and only a lunar environment. On

Earth it’s going to receive seventy-five umptillion sense

119

. See infra note 125 and accompanying text.

120

. Asimov, Risk, supra note 81 at 396.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 25

impressions for which it was never prepared. There’s no

telling what its reaction will be. No telling!;”

121

• In “…That Thou Art Mindful of Him,

122

Asimov writes on

interpretive problems. Here, US Robots tries to introduce

robots on planet Earth where they had been forbidden. US

Robots must solve an intractable problem. The Second

Law orders robots to obey human instructions. But should

robots follow childish, silly or criminal orders? In Asimov’s

history of robotics, this problem was never solved,

precisely because the robots did not evolve on Earth.

123

What is the bottom line? In law and technology scholarship, a lot

of hype surrounds regulation by design as a substitute to traditional

legislation. Asimov warns that ordinary lawmaking and robotic

regulation by design share similar challenges. Both activities are

fraught with uncertainties. The hype might not be justified.

VII. THE NECESSITY OF EXPERT HUMAN AGENCY

Asimov predicted the fallibility of law by-design. But he did not stop

here. Distinct from the dystopian genre, Asimov’s works stress how

human agency excels at problem solving.

124

Most robot stories

describe how human reason brings solutions to puzzles arising from

the Three Laws’ inconsistency. Two stories drive this point home.

121

. Asimov, Robot AL-76 Goes Astray, supra note 81 at 60.

122

. Asimov, …That Thou Art Mindful of Him, supra note 81.

123

. See id. Asimov wrote: “We have been hampered by the very fact that our

robots have been used only in specialized environments out in space, where the

men who dealt with them were experts in their field. There were no children, no

idiots, no criminals, no well-meaning ignoramuses present. Even so, there were

occasions when damage was done by foolish or merely unthinking orders.” Id. at

498.

124

. Some stories though had a dystopian spin. For example, Sally, supra note 36,

features a conjecture about robot cars conspiring to kill humans. Someday, supra

note 80, and …That Thou Art Mindful of Him, supra note 81, display similar pitches.

See supra note 80 and accompanying text.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

26 Journal of Business & Technology Law

Dr. Susan Calvin’s intervention in Little Lost Robot provides a first

good illustration.

125

To identify Nestor 10, Dr. Calvin first sets up a

trap. A human is about to be hit by a heavy weight. The sixty-three

robots are unaware that the human is in fact protected. All robots set

out to protect the human before the charge hits. Nestor 10, which

could have abstained given the modified First Law, chooses to act to

maintain its anonymity.

Dr. Calvin decides then to put high-tension cables between the

falsely endangered human and the robots. Dr. Calvin believes Nestor

10 will be the only robot that will not act. Nestor 10 has no obligation

to save a human due to the modified First Law and it has the

obligation to protect itself under the Third Law. Again, however, the

test fails. Not a single robot, in this setting, tries to save the human.

All robots justify inaction by saying the human was doomed anyway

because the robot would have been destroyed before he could save

the human, and another human could have been in danger in a near

future. The useless destruction of all robots would lead to a possible

future harm, which is not allowed under the First Law.

Dr. Calvin ultimately conceives another test. She tells the sixty-

three robots that a man will be endangered by radiations. The

radiations will kill any robot that attempts to save the human. In view

of the previous experiment, Calvin advises robots not to act if they

detect radiation between them and the endangered human. When

the experiment starts, only Nestor 10 rescues the human. Calvin has

replaced the lethal radiation with inoffensive infrared rays. Nestor 10

is the only robot able to differentiate lethal from non-lethal rays.

Nestor 10 learned this during while working in the space station.

Other robots, without that specific knowledge, only detected rays and

decided to stay put. Nestor 10 forgot that fact and, believing others

would follow suit, found itself to be the only one exposed.

126

Catch that Rabbit is the second story displaying the power of

human agency. Powell and Donovan must evaluate a robotic

125

. Asimov, Little Lost Robot, supra note 80.

126

. See id.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 27

supervisor, Dave. The robot’s task is to coordinate six subsidiary

mining robots.

127

Powell and Donovan find that Dave acts strangely

during emergencies. The Three Laws have not been manipulated.

Powell and Donovan conclude that the robot performs well in routine

situation that requires no special monitoring. From there, they ask

themselves, what is different under emergency? It does not take long

to Powell and Donovan to realize that emergency changes the game

to the extent that “all six subsidiaries must be mobilized immediately

and simultaneously.”

128

The problem is thus similar to a computer

lagging because it runs too many programs at the same time. The

solution is obvious and, above all, non-technical. Reducing the

number of subsidiaries that Dave has to concurrently coordinate

should allow the maintenance of a consistent level of performance in

both emergency and non-emergency contexts.

The two stories show the need for human agency.

129

However, in

Asimov’s eyes, human agency is not equivalent to layman agency.

Through the character of Dr Calvin, Asimov stresses specific

properties of human agency required to solve the problems raised by

the Three Laws. The properties are common sense, intuition, logic,

reason, and experience. Or put differently, the properties correspond

127

. Isaac Asimov, Catch That Rabbit, ASTOUNDING SCIENCE FICTION, Feb. 1944,

reprinted in ASIMOV, THE COMPLETE ROBOT, supra note 30, at 245.

128

. Id. at 263 (emphasis added).

129

. Other stories illustrate this too. This is clearly explained in Asimov, Risk, supra

note 81. The vague order given to the robot leads to the flight-test failure. See supra

note 120 and accompanying text. Knowing that pitfall, “the inadequacy of a robot

must be made up for by the ingenuity and intelligence of a man.” Asimov, Risk, supra

note 81, at 397 (emphasis added). Asimov wrote in Risk: “Robots have no ingenuity.

Their minds are finite and can be calculated to the last decimal. . . . Now if a robot

is given an order, a precise order, he can follow it. If the order is not precise, he

cannot correct his own mistake without further orders. . . . How . . . can we send a

robot to find a flaw in a mechanism when we cannot possibly give precise orders,

since we know nothing about the flaw ourselves? ‘Find out what’s wrong’ is not an

order you can give to a robot; only to a man. The human brain, so far at least, is

beyond calculation.” Id. at 398. In ASIMOV, THE NAKED SUN, supra note 80, it is a

human detective that identifies a gap in the First Law. A robot might well hurt a

human being if he is not aware his action will result in a damage. See supra notes

104-106 and accompanying text. And it is the same human that gives the first clue

that a zeroth law was needed in ASIMOV, ROBOTS AND EMPIRE, supra note 30. See supra

note 103 and accompanying text.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

28 Journal of Business & Technology Law

in popular form to the hallmarks of the scientific method, that is

observation, hypothesis, prediction, experiment, and solution.

In a story called Someday, Asimov dwells on the requested abilities.

The story features a future in which two young boys unsuccessfully

try to upgrade an old story telling machine. One of them reminds a

teacher’s advice:

It gets harder all the time to find people who can really run [giant]

computers. [A]nyone can keep an eye on the controls and check off

answers and put through routine problems. [T]he trick is to expand

research and figure out ways to ask the right questions, and that’s

hard.

130

The two boys realize they need to learn coding and programming.

The broader point is this. Societies should not pull solutions to

technological problems “out of thin and implausible air”.

131

concretely, Asimov’s science fiction calls into question whether we

can rely on the ‘wisdom of crowds’ to solve hard technological

problems.

In addition, integrity is another property that Dr Calvin

impersonates. At some point in Little Lost Robots, Dr Calvin threatens

US Robots colleagues to blow the whistle if she is not allowed to test

her theories. The threat of public backlash against US robots is

enough to induce the company’s cooperation. If humans ever knew

that modified robots were in operation, this would spell the end of

the robotic industry. Clearly, for Asimov, good science meant

unrestricted science.

132

130

. Asimov, Someday, supra note 80, at 29-30.

131

. Asimov, How Easy to See the Future, supra note 41, at 80.

132

. In a short non-fictional essay discussing the opportunity to regulate science,

Asimov explained neither the side effect (e.g.: nuclear waste) nor the evil use (e.g.:

atomic bomb) of technology “can or should imply that the acquisition of knowledge

itself must be regulated, directed, or stopped.” Isaac Asimov, Do We Regulate

Science?, reprinted in ASIMOV, THE ROVING MIND 104, 105 (emphasis added). He

emphasized: “Knowledge increases options, offering us additional opportunities to

manipulate the universe for good or for evil, and, if we choose wisely, we end with

more opportunity for good. . . . Even where new knowledge offers little good and

much evil, might we not select the little good and discard the much evil? Or is

humanity so certain to choose the evil out of some kind of malevolent stupidity that

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 29

In a word, Asimov forecasted a world of complementarity between

machines and humans.

133

Because ex ante safety regulation is both

inevitable and fallible, they require corrections by human

intervention. And the best guarantee of effective problem solving is

expert human agency, not popular human agency.

VIII. THE FALSE ANALOGY BETWEEN LAW AND TECHNOLOGY’S MENTAL

MODELS

Asimov’s Three Laws share significant analogies with ordinary rules

of law. They are ordered like a hierarchical pyramid. They are both

proscriptive and prescriptive. And they are definite, constant, public,

and systematic.

At least formally, Asimov’s Three Laws hold the attributes of law.

But are they really law? Or to put the question more elaborately, are

engineering instructions like the Three Laws – Professor Lessig calls

them “architecture”

134

– formally equivalent to law so that Asimov’s

Three Laws can serve as inspiration for a model of social control of

technology? The short answer is no.

The long answer requires going back to a distinction introduced by

Hans Kelsen in 1934 in Pure Theory of Laws.

135

In his seminal work,

Kelsen drew a distinction between laws of nature and legal laws. Laws

ignorance is the only way out? . . . If, however, we do have the faculty of intelligent

choice, then let us make that choice as effective as possible by constantly increasing

knowledge of the potential dangers to be avoided as well as of the usefulness to be

chosen.” Id. at 105-06. Asimov was not, however, a naïve techno-optimist. He

explained that he views “technology and science (wisely used – an enormously

important condition) as beneficent and as the key to human progress.” Isaac

Asimov, The Scientist as Villain, ISAAC ASIMOV’S SCIENCE FICTION MAGAZINE, Oct. 1979,

reprinted in ASIMOV, ASIMOV ON SCIENCE FICTION, supra note 18 at 61 (emphasis added).

For Asimov, the technology is “not wholly [e]vil”, but “what we make it, and we

must rescue and extend those parts of it that are Good.” Asimov, The Ring of Evil,

supra note 78, at 280 (emphasis added).

133

. Furthermore, Asimov himself rejected the dichotomy between endearing

and threatening robots. Instead, he chose to see them as tools. Thinking of

industrial robots, the future has proved he was right. See ASIMOV, THE COMPLETE

ROBOT, supra note 30.

134

. See LAWRENCE LESSIG, CODE: VERSION 2.0 124-25 (2006).

135

. For this paper, we will rely on the following version of Kelsen’s book: HANS

KELSEN, PURE THEORY OF LAW (Max Knight trans., 1967) [hereinafter KELSEN, PURE THEORY

OF LAW].

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

30 Journal of Business & Technology Law

of nature are characterized by “causality.”

136

Legal laws are

characterized by “imputation.” While both causality and imputation

establish a relationship between two events – a trigger A and a

consequence B – the occurrence of the consequence of causality B(c)

is certain while it is not for imputation B(i).

For any A, probability of B(c) > B(i)

As Kelsen wrote “the rule of law does not say, as the law of nature

does: when A is, “is” B; but when A is, B “ought” to be, even though B

perhaps actually is not.”

137

The occurrence of the consequence under

a legal law is dependent on human agency, which is not the case for

a law of nature.

138

With this, the Three Laws do not constitute formal law. Human

agency entails a possibility of choosing to violate the law. But robots

have no free will.

The Three Laws are metaphorically closer to the genome than to

rules of law adopted by a legislator or an administrator.

139

As much as

136

. Either prescriptive or proscriptive. Prescriptive laws are constructed on

imputation: if A happens, then follow instruction B. Concerning proscriptive laws,

they simply forbid a behaviour – thou shall not kill –, eventually with condition – car

drivers must stop when traffic light turns red – or exception – car drivers must stop

when traffic light turns red except if a police officer tells them to go anyway. See

KELSEN, PURE THEORY OF LAW supra note 135, at 27.

137

. Id., at 77.

138

. Similarly, the syllogism at stake in legal proceedings is not duplicable for

algorithm, precisely because it is impossible to reduce the legal reasoning to a

syllogism. The syllogism is more a presentation of the legal reasoning than a true

description. If there is indeed an application of the legislation at stake to the specific

facts of the case – which implies the establishment of the facts and the

identification of the norm – rarely the judge will find a unique solution. The

inexorability of the syllogism lacks in the legal reasoning. For a discussion on this

topic, see Pierre Moreau, L’intelligence artificielle au service du droit et de la justice,

CHRONIQUE DE DROIT A L’USAGE DES JUGES DE PAIX ET DE POLICE, 314 (2019).

139

. This, however, is directly challenged by Asimov in The Bicentennial Man. In

this Novelette, a robot named Andrew requests and obtains from the US Supreme

Court his freedom. But the Court makes clear the former human owner is still

responsible for the actions of the robot. Asimov wrote:

“– The responsibility is no great chore. You know you won’t have to do a thing. The

Three Laws still hold.”

“– Then how is he [Andrew] free?”

“– Are not human beings bound by their laws, Sir?”

Asimov, The Bicentennial Man, supra note 30, at 529.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 31

humans cannot evade biological instructions embodied in their DNA,

robots cannot bypass the Three Laws. Strikingly, Asimov wrote a story

titled Christmas Without Rodney in which a robot that must obey a

rude young boy dreams of a world in which the restrictive Three Laws

would not exist.

140

Technically, the Three laws, more broadly code, and generally

“architecture,” work in a deterministic way.

141

Causality is the

deterministic functional relationship that governs the execution of a

computer program. Once written, its operation no longer requires

human intervention. Code self-executes. Each input leads to a given

output.

142

The deterministic nature of code does not mean that

everything is predictable.

143

Informational limits prevent us from

forecasting every possible input-output causal relation. Besides,

errors in programming can occur. And computer code can be altered

by external elements called viruses that modify the code to reproduce

140

. Isaac Asimov, Christmas Without Rodney, ISAAC ASIMOV’S SCIENCE FICTION

MAGAZINE (Dec. 1988), at 18, reprinted in French in ASIMOV, LE GRAND LIVRE DES ROBOTS

141

. On the functioning of DNA which “encodes information through the order, or

sequence, of the nucleotides along each strand,” see BRUCE ALBERTS ET AL. MOLECULAR

BIOLOGY OF THE CELL (4th ed. 2002); See also id. at 192–97. On the functioning of

computer code, see HAROLD ABELSON & GERALD J. SUSSMAN, STRUCTURE AND

INTERPRETATION OF COMPUTER PROGRAMS (2nd ed. 1996). C. GORDON BELL & ALLEN NEWELL,

COMPUTER STRUCTURES: READINGS AND EXAMPLES (1971).

142

. Andreas Blass and Yuri Gurevich, Algorithms: A Quest for Absolute

Definitions, 81 BULL. OF EUR. ASS’N FOR THEORETICAL COMPUT. SCI. (2003); See DONALD E.

KNUTH, ART OF COMPUTER PROGRAMMING (3rd ed. 1997); See also David Danks,

Learning, in THE CAMBRIDGE HANDBOOK OF ARTIFICIAL INTELLIGENCE 154 (Keith Frankish &

William M. Ramsey eds., 2014); David Lehr and Paul Ohm, Playing with the Data:

What Legal Scholars Should Learn About Machine Learning, 51 U.C. DAVIS L. REV. 653

(2017); John Zerilli & Adrian Weller, The Technology, in THE LAW OF ARTIFICIAL

INTELLIGENCE (Matt Hervey & Matthew Lavy eds. 2021).

143

. Writers emphasize even without engineers’ mistakes in coding, problems can

happen if users are not able to properly use robots. See supra notes 124 seq. and

accompanying text.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

32 Journal of Business & Technology Law

themselves.

144

A programming error can be compared to an inherited

genetic disorder.

145

Where does this lead us? The two take away from the lack of

formal analogy between law and computer code are this. First,

lawmakers should not take for granted the idea that regulation by

design can be a surrogate to traditional rules of law. In a story called

Think!, a scientist discovers a laser protocol allowing human-

computer telepathy. As she dwells on the applications in psychiatry,

the treatment of mental diseases, education, legal investigations, and

criminal trials, one of her colleagues warns: “Frankly, the social

implications are staggering. I don’t know if something like this should

be allowed.”

146

The warning is important. The lure of code as a

solution to the social control of technology is an illusion. At a time

where lawmakers place great hopes in using computer scientists to

assist them, they should not wash their hands of the hard moral

questions that ordinary lawmaking towards technology requires

facing.

Second, in spite of causality, informational limitations maintain a

possibility of emergent behavior that cannot be perfectly predicted.

Preventive, or even precautionary, lawmaking approaches to

technology remain valid, even in the face of low probability events.

Lawmakers should not be fooled by terminology. Asimov decided

to call “laws” the code of his robots in the scientific sense, not in the

socio-political sense.

147

Just as much as the world operates under a

law of gravity, or transistors evolve alongside Moore’s law, Asimov’s

144

. On the use of metaphors from the medical world for computer science, see

John Humbley, La traduction des métaphores dans les langues de spécialité : le cas

des virus informatiques, 52 REVUE DES LINGUISTES DE L’UNIVERSITE PARIS X NANTERRE, 49

(2005).

145

. Each ex post modification of the computer code shares analogies with genetic

manipulation including, of course, the intrinsic uncertainty it implies. Id. at 55–56.

146

. Isaac Asimov, Think!, ISAAC ASIMOV’S SCIENCE FICTION MAGAZINE, 1977, at 40,

reprinted in ASIMOV, THE COMPLETE ROBOT, supra note 30, at 47.

147

. In Robot AL-76 Goes Astray, Asimov explained legal laws may be bypassed

(“laws could always be squared”). Asimov, Robot AL-76 Goes Astray, supra note 81

at 59-60. In Robbie, he qualified such unfeasible violation as “a mathematical

impossibility.” Isaac Asimov, Robbie, supra note 92 at 139.

Petit Page Proof (Do Not Delete) 2/1/2023 5:10 PM

JEROME DE COOMAN & NICOLAS PETIT

Vol. 18 No. 1 2022 33

laws were thought of as code, design, and architectural instructions.

No more, no less. Tellingly, the Three Laws appeared in the 58

edition of the Textbook on Robotics, 2058 bc. Asimov did not appear

to entertain an ambition to have the Three Laws preempt real rules

of law.

Even more generally, Asimov did not even seem to hope that his

Three Laws would have impact in the technological world. Asimov

was very happy to learn AI experts found his Three Laws a “good

guide” for their work.

148

But his work concentrated on showing the

latent inconsistencies and ambiguity inherently enshrined in code,

and how human ingenuity solves them.

CONCLUSION

This paper draws seven lessons from Isaac Asimov’s writings of and

on science fiction. First, lawmakers should read science fiction, but

not all science fiction facts are reliable. One should distinguish

realistic science fiction from escape literature. Second, science fiction

social facts are a powerful tool that enable lawmakers to understand

social responses to technological change. Third, Asimov calls

attention to one key social insight, namely the trend of human initial

hostility and then affinity to technologically-driven change. Fourth,

Asimov forecasted the inevitability of law by design. All machines

were, are, and will be designed for safety. Fifth, despite its

inevitability, regulation by design is also fallible. This is the core idea

of the Three Laws of Robotics. Code, architecture, and technical

instructions are inevitable, but also insufficient and imperfect. Sixth,

human agency is essential to solve the fallibility of regulation by

design. By human agency, Asimov had in mind expert, rational, and

scientific agency. Last, having controls on technology in code is hardly

ever a fully effective safe harbor. The adoption of ordinary rules of

law is required to deals with complex moral tradeoffs and emergent

behavior.

148

. ASIMOV, THE COMPLETE ROBOT, supra note 30 at xii.

PETIT Page Proof (DO NOT DELETE) 2/1/2023 5:10 PM

Asimov for Lawmakers

34 Journal of Business & Technology Law

With this, Asimov’s philosophy of law and technology is nuanced.

Neither techno determinist, nor techno solutionist, Asimov believed

that technology and humans are both part of the problem and of the

solution. Science and technology are good under the condition of

being wisely used. Humans are good under the condition of being

sufficiently rational, logical, and virtuous. In a certain way, Asimov

was a techno-institutionalist. New technology creates risks that

cannot be solved by adding new units of technology. Human inputs

are required. Machines cannot be left on their own. Cooperation, not

competition between human and machines, holds the promise of a

better future.

* * *