Cory Doctorow: “The Coming Civil War over General-purpose Computing” | Talks at Google

>>Presenter: I first met Cory Doctorow in
1986 when he was working for Metallica and trying to anticipate the coming Napster wars.
People don’t know this about him, but he is a fervent defender of the RIAA and the MPAA.
This has all been just a big lie, he says, [laughter] to make people feel that he’s one
of us. Recently, actually, I ran into him in Washington,
DC, and he was telling me, “You guys do the legal minimum of compliance with the DMCA.
We think that you’re just pirates.” I said, “You’ve been working for the RIAA now for
16 years. I don’t see how you can possibly say that with any sort of honesty.” Then I
realized who I was talking to, of course. [laughs] [laughter]
Realistically, I met Cory when he was doing sugar water. Right? For the first time.>>Cory Doctorow: [inaudible]>>Presenter: Yeah. And then Zelig-like, I
noticed as he rose as an author. I actually don’t want to spend much time introducing
him. So I won’t. Everyone, please welcome Cory Doctorow. [applause]>>Doctorow: Hi folks. I give– I write science
fiction novels and stories. You’ve got some of my short story collections there in front
of you. They all relate, in one way or another, they kind of circle these issues. I feel like
science fiction stories put the sinew and the marrow into the argument. Before George
Orwell came along, if you wanted to talk about surveillance, you could say, “I kinda feel
like it might change my behavior if I were being watched all the time in some abstract
way.” And someone else might say, “Yeah, but if we knew everything about you, we could
provide services to you and we could know when bad things were going to happen” and
so on. And now we have this great word we can use to describe what that means. You can
say it’s Orwellian. So there’s now a lot of muscle on the bone
when you talk about this stuff. That’s what I do in the fiction. But I don’t want to stand
here and read stories to you, although I have a podcast where you can hear me read stories.
What I’m going to do today is the argument that the books that you’re holding in your
hand are the blood and sinew for, and take it from there.
The talk runs 35 minutes. And then there’s time for Q&A. The one thing I want to say
as a caveat to this: I’ve given this talk twice now. I gave it at the Long Now Foundation
and I gave it at DEFCON, both in the last week or ten days. Both times, there was a
little bit of feedback. There’s a hypothetical, technical solution I propose, and I’ll tell
you when I get there. I want to clarify that it is purely hypothetical by way of example,
and not a thing that I think we should do. With that said, [laughter] I’m going to get
to it. I gave this talk in late 2011– Ha. Ah! There
we go. I gave this talk in late 2011 at the 28C3 in Berlin called “The Coming War on General
Purpose Computation.” In a nutshell, the hypothesis of that talk was computers and the internet
are everywhere, the world is increasingly made of computers and the internet. We used
to have these separate categories of devices like washing machines, VCRs, phones, and cars,
and now we just have computers in different boxes. Cars are computers we put our bodies
into. 747s are badly secured Solaris boxes connected SCADA controllers. [laughter] Hearing
aids, pacemakers, other prostheses: computers we put in our body.
That means that from now on, all of our socio-political problems in the future are going to have a
computer in the middle of them. That will beget a regulator who says, “Can’t you just
make me a computer that solves the problem? Can’t you make me a self-driving car that
can’t be programmed to drag race? Can’t you make me a bioscale 3D printer that doesn’t
print out an organism that puts the human race at risk or blows Monsanto’s quarterly
profits?” That is, “Can you make me a general purpose computer that runs all the programs
except for the one that pisses me off?” [laughter] Now, we don’t know how to make that computer.
We don’t have a theoretical model for Turing Complete minus one. Our closest approximation
to a computer that runs every program except for the one that abets a criminal or evinces
a social problem is a computer with spyware on it when it comes out of the box. That is,
a computer that watches everything that you do all the time, so that when the moment comes,
it can say, “I can’t let you do that, Dave.” A computer that runs secret programs that
the user isn’t supposed to even know about. If the user finds out about it, the user can’t
terminate these processes, even if the user really thinks that they run contrary to their
interests, and even if the computer that they’re running on belongs to the user. In other words,
digital rights management. Now, digital rights management’s a bad idea
for solving social problems for at least two significant reasons. The first one is that
it doesn’t actually solve the problem. Breaking DRM isn’t hard for bad guys. As the copyright
wars have shown us, digital rights management is a solution that ends within 24 hours. As
soon as a bored Norwegian teenager encounters the DRM, it goes away. DRM only works if the
“I can’t let you do that, Dave” program remains a secret. Once the most sophisticated attacker
in the world finds out that secret and puts it on the internet, everybody else on the
internet has the secret, too. Now, the second reason is that DRM not only
has weak security, but it weakens security. In order to be secure, you need to be certain
about what software is running on your computer. You can’t secure the software on your computer
if you don’t know what software is running on your computer. When you design the “I can’t
let you do that, Dave” facility into a computer, you create this enormous security vulnerability.
You now have a program running that users aren’t even supposed to know about. If they
know about it, they can’t find details of or terminate or override. When some bad guy
hijacks this, they can do things to your computer that, by design, your computer doesn’t show
you. You probably remember, Sony BMG put root kits
on 51– no, 6 million CDs, 51 audio CD titles, and distributed them to their customers. They
stealthily installed malware. The root kit made any process or file that was prepended
with dollar sign SYS, invisible to the file manager and process manager. Immediately,
malware writers started prepending dollar sign SYS to their program files and their
processes because if they ever found themselves on a computer whose immune system had been
blown by the Sony root kit, that immune system would no longer even be able to see their
process. Now, once governments solve problems with
DRM, there’s this perverse incentive to make it illegal to tell people things that might
override the DRM, things like “This is how the DRM works” or “Here’s a flaw in the DRM
that might allow an attacker to secretly activate the microphone or turn on the camera or grab
your keystrokes.” Now after I gave this talk at 28C3, I got
a lot of feedback from various civil libertarians and other people, including some very distinguished
computer scientists. I got a very thought provoking email from Vint Cerf after I wrote
this, which really made my day. It led me to the conclusion that within the fields of
civil liberties and technology and policy, there’s a kind of good guy consensus that
if you own your computer, you should be in charge of what’s running on it, at least as
between you and corporations, or you and the government. That mandating what software you
may or may not run on your computer is just not a good idea if it belongs to you.
Now, most computers– Let’s examine, for a minute, what it would mean, as an owner, to
be able to absolutely control the software that was running on your computer in an adversarial
relationship against, not an advanced persistent threat, but at least against script kiddies
or griefers or just your garden variety deputy dog cop who wants to screw with your computer.
Most computers today are fitted with these: TPMs, trusted platform modules, a secure code
processor mounted on the motherboard. The specification for TPM is published. There’s
an industry body that certifies that devices that advertise a TPM actually have a real
TPM in them and not a fake TPM. To the extent that that spec is good, and to the extent
that these people are diligent in doing their jobs and sue people who list a device as having
a TPM when it doesn’t, it’s possible to be reasonably certain that if you think you have
a TPM, you do have a TPM, and that it faithfully implements the spec.
TPM is secure. One of the ways in which it’s secure is that it has some secrets. But it’s
also secure in that it’s designed to be tamper evident. If you try to extract the keys from
a TPM, it’s supposed to be really obvious that something has been done to your computer.
Someone takes your real TPM out and puts a fake TPM that they 3D printed or cooked up
in their hack lab or made down in Quantico and sticks it in your computer, it’s supposed
to be really obvious that it’s happened. There’s a TPM threat model that crooks or governments
or police forces or some other adversary try to compromise your computer, and TPM tamper
evidence lets you know when that’s happened. But there’s another TPM threat model. It’s
that a piece of malicious software infects your computer. Now, all the censors—. When
that happens, all the censors that are attached to your computer, –the mic, the camera, the
accelerometer, the fingerprint reader, the GPS, and so on–, can be switched on without
your knowledge, and the data can be cached on the or can be sent to a bad guy or both.
Not only that, of course, all of the data on your computer– your sensitive files, your
stored passwords, your web history– can also be harvested and sent to a bad guy, or harvested
and cached for a later retrieval, as can all your keystrokes. All the peripherals that
are attached to your computer can either be subtly altered, turned off, or turned on to
do bad things. Today, those peripherals might be your printer, your scanner, your SCADA
controller, your MRI machine, your car, your avionic, your 3D printer. You can understand
why that would be a bit freaky, but of course, in the future, those peripherals might also
include your optic nerve, your cochlea, and the stumps of your legs.
When your computer boots up, the TPM can ask your bootloader for a signed hash of itself
and verify that the signature of the hash comes from a trusted party, someone you trust.
Once you trust the bootloader to faithfully perform its duties, you can ask it to check
the signatures on the operating system, which, once verified, can check the signatures on
the programs that run on it. And so on and so on up the stack, ensuring that you know
which programs are running on your computer, and that any programs running in secret have
gotten there by leveraging a defect in the bootloader or operating system or the other
components, and not because this computer was designed to actually hide things from
you. Now, this story always reminds me of Descartes:
he starts off by saying that he can’t tell what’s true and what’s not true, because he
doesn’t know if he can trust his senses, he doesn’t know if he can trust his reason. He
does some mental gymnastics, which I won’t get into here, although that’s generally the
thing people find interesting about him, but what’s interesting to me is that once he establishes
this tiny little nub of certainty, a kind of mental gymnastic exercise that says, “Well,
I can trust my reason, I can trust my senses.” Then he is able to erect this stable edifice
of a worldview on it. He knows one thing to be true, and everything else can be hung off
of that one thing. He can build it up. Now, a TPM is like that. It’s a nub of stable
certainty: if it’s there, it can reliably inform you about your bootloader, and thus,
your operating system, and thus, the processes running on your computer.
Now, you may find it weird to hear someone like me talking warmly about TPMs. After all,
these are the technologies that make it possible to lock phones, tablets, consoles, and even
some PCs so that they can’t run software of the owner’s choosing. Jailbreaking usually
means finding a way to subvert a TPM. Why on earth would I want a TPM in my computer?
As with everything interesting in tech and policy, the devil is in the details. Imagine
for a moment that there’s two different ways of implementing a TPM. There may be more,
but imagine these two. The first one we’ll call lockdown. In the lockdown world, your
TPM comes with a set of signing keys it trusts, and unless your bootloader appears in that
list– is signed by one of those signing keys, it won’t run it. It won’t boot, the operating
system won’t run. You’re just stuck there in whatever it is that people who installed
the bootloader on your computer want you to run. You can’t change that.
There’s another mode that I’ll call certainty. In the certainty mode, you tell your TPM which
signing keys you trust. The first time you turn your computer on, you initialize it with
some authentication token– whoops– like a key or a password or some other thing. A
biometric that it knows so it knows who it belongs to. Then you, the owner, are the only
person who gets to say what it trusts. You can say, “I don’t trust this person’s operating
system” or “I do trust that person’s operating system.” “Only run operating systems that
are signed by Cononicle, EFF, ACLU and Wikileaks. [laughter] Approximately speaking, these two
modes correspond to, of course, iOS and Android. iOS only lets you run the code that’s been
approved by Apple. Android lets you tick a box and say, “I’m a grown up. Let me choose
who I trust”. Critically, Android lacks an important facility: it lacks the facility
to verify that what you think you’re running is what you are running. It’s freedom without
certainty. Now, freedom without certainty is a big deal
in a world where the computers we’re discussing can see you and hear you, where we put them
in your pocket and take them into the toilet, where they sit by your bedside, where they
fly airplanes, where we put our bodies into them, and they drive our cars around, which
is why I like the idea of a TPM, provided it’s implemented in the certainty mode and
not in the lockdown mode. Now, if that’s not clear, think of it this
way: there’s the war on general-purpose computation, and that’s what happens when control freaks
in governments or companies decide that they should have the final say in what you do on
your computer. There’s also– And then there’s this: there’s the counter position, which
is that defenders against those people are also control freaks, but they’re control freaks
like me. We want to be people with the ultimate destiny over what we install on our computers.
Both sides want control, they just differ in where the nexus of control should be.
Control requires knowledge. If you want to be sure that songs that are moved onto an
iPod, stay on the iPod, and don’t come off of the iPod, the iPod needs to know that the
instructions that it’s getting are coming from an Apple-approved version of iTunes,
and not one pretending to be iTunes. Otherwise, you don’t get the roach motel. If you want
to be sure that my PVR won’t record a watch-once, video-on-demand program, or if it does record
a program, that it won’t output it to anything except something that will honor whatever
business rules came along with it. You have to be sure that you know what programs I’m
running and what they do. But if I want to be sure that you aren’t watching
me through my webcam, I need to know what firmware is running, and I need to know that
the little green light always comes on when my webcam switches itself on. If I want to
be sure that you aren’t capturing my passwords through my keyboard using a software keylogger,
I need to know that the OS isn’t lying when it says there aren’t any keyloggers resident
in the system. Whether you want to be free or whether you want to enslave, you need control
and you need knowledge. That’s the coming war on general purpose computation.
Now I want to investigate what happens if we win it. That’s the civil war over general
purpose computation. Let’s stipulate that we have a victory for the “freedom side.”
It means that we have computers where owners always know what was running on them, because
the computers would faithfully report the hash and the associated signatures for any
bootloaders they find, and control over what was running on computer’s ghost to you, because
the computers would allow their owners to specify who was allowed to sign their bootloaders,
operating systems, and so on. There are two arguments that we can make in
favor of this victory, why this victory would be a good one. The first one is a human rights
argument. If your world is made of computers, then designing computers to override their
owner’s decisions has significant human rights implications. Today, there are people who
worry that the Iranian government might demand import controls, so that all the computers
that come in have UFE-style bootlocker that only boots operating systems that have lawful
interception back doors built in. You can move the spying right to the edge, to the
user of the computer, the owner of the computer. But tomorrow, it may be that I live in the
UK, and it may be that our Home Secretary says, “If the NHS gives you a cochlear implant,
it has to intercept and report all the extremist speech it hears.” The human rights stuff is
easy to understand. The second argument comes from property rights.
The doctrine of first sale is a very important piece of law. It says once you buy something,
you own it. You should have the freedom to do anything you want to it, even if it gores
the ox of the person who sold it to you. DRM opponents like me, we love the slogan, “You
bought it, you own it.” Property rights are an incredibly powerful
argument to have on your side, anywhere. But they’re especially powerful to have on your
side in a nerd fight, because you can’t swing a cat in Silicon Valley without hitting someone
who thinks that property rights are an important way of solving most social problems. But it’s
not just nerd fights. Copyfighters get really pissed off about the term “intellectual property,”
because property is also a really good way to win arguments in policy circles. Before
the term “intellectual property” came into prominence, we had other terms like “creators’
monopoly”. It’s very hard to go to a regulator or a lawmaker and say, “My monopoly isn’t
large enough”, but going and saying, “My property rights are being not respected enough or need
to be expanded so that I can make sure that they’re policed adequately,” that’s a very
powerful argument to have on your hand. That’s where the civil war part comes in.
Human rights and property rights both demand that computers not be designed for remote
control by corporations or governments. Owners be allowed to specify their OS and the programs
running on them to freely choose that nub of certainty in the void that allows them
to build their whole stable edifice of certainty on.
Now, remember that security is relative. You are secured from attacks on your ability to
freely use your music if you can control your computing environment. But, if you can control
your computing environment, the Recording Industry Association of America is now vulnerable
to attacks on their ability to rent you music on a single-use basis. We have this notion
of streaming, this consensus hallucination that there’s a difference between a stream
and a download, as though there’s some means of transmitting a stream of bits to someone’s
computer without actually having them download that stream of bits, like the internet is
made of mirrors and speaking tubes. [laughter] We say “Stream”, we mean “I think that your
receiving software doesn’t have a ‘save as’ button.”
Now, if you get to choose the nub from which the scaffold dangles, you get control and
power to secure yourself against people who attack your interests. If the Recording Industry
Association of America, or the government, or Monsanto get to choose the nub, then they
get control and the power to secure themselves against you. So we all agree that at the very
least, owners should control what runs on their computers, or I’ll ask you to stipulate
that. Now, what about users of computers?
Users of computers don’t always have the same interests as the owners of computers. Increasingly,
we will be users of computers that we don’t own. Where you come down on the conflict between
owners and users of computers, I think, is going to end up being one of the most important
both technological and moral questions of the coming decades. There’s no easy answer
I have, no bright line, for when users or owners should trump one another when it comes
to computers. Let’s start with a position I’ll call “property
maximalism”: “If I own my computer, I should have the absolute right to dictate terms of
use to anyone who wants to use it. If you don’t like it, find someone else’s computer
to use. This one’s mine. I set the rules.” How would that work in practice? Well, you
got some combination of an initialization routine where you set the root of trust, tamper
evidence, law, and physical control. For example, you turn on your computer for the first time,
and you initialize a good secret password, possibly signed by your private key, and without
that key, no one is allowed to change the list of trusted parties who are allowed to
sign your bootloader. We can make it against the law to subvert this for the purpose of
taking control away from the owner. That makes writing malware that hijacks your computer
extra special, super duper illegal, but it also makes stealthy DRM installation even
more illegal. We can design the TPM so that if you remove it, or tamper with it, it’s
really obvious. You give it a fragile housing, so that when it’s changed out, you can tell
at a glance that it’s happened. Then, if you still trust physical locks, you can put it
under lock and key, too. Now, I can see a lot of benefits to this,
but there are unquestionably some downsides to giving owners absolute control over their
computers. One wedge issue is probably going to be a self-driving car. There’s a lot of
these around already. They come out of places like this and other places. It’s easy to understand,
on the one hand, why self-driving cars would be insanely great. We are terrible drivers.
Cars totally kill the shit out of us. [laughter] They are the number 1 cause of death in America
for people aged 5-34. I saw my friend, Katherine [indistinct], last night. She pointed out
that it’s also the number 1 way for humans to kill other humans. If you kill another
person in your life, you’re almost certainly going to do it with a car. I’ve been hit by
a car. I’ve also cracked up a car. I’m willing to stipulate that humans have no business
driving. It’s also easy to understand how we might be nervous about the prospect of
people homebrewing their own self-driving car firmware. On the one hand, we do want
the sourcecode for these cars to be open, public, and subject to scrutiny so that defects
can be discovered, so that hidden features that may act against their owners’ or users’
interests can be quickly found out. You’d want to know it if there’s a kill switch built
in. You’d want to know it if your car secretly drives you past more McDonalds when the kids
are in the back seat. It’s going to be plausible, I think, to say, “Cars are safer if they have
a locked bootloader and if that bootloader only runs a firmware that’s been signed by
the Department of Motor Vehicles or by the FTC”.
But now we’re back to you, whether you get to decide whether your computer is running
the software that you want it to run. Now, there are two problems with this solution,
the solution of giving the state a veto over your self-driving car. The first one is that
it won’t work. As the copyright wars have shown us, firmware locks aren’t effective
against dedicated attackers. People who want to sow mayhem with custom firmware will be
able to. We need a security model that doesn’t believe that all the other cars on the road
are going to be well-behaved. If that’s our security model, then we are all dead meat.
Self-driving cars must be conservative in their approach to their own conduct, and liberal
in conduct they expect from others, a venerable principle familiar to people who work in computers,
and also, the advice that you got on your first day of driver’s ed. And it remains good
advice today. Now, the second problem with this is that
it invites some pretty sticky parallels. Do you remember the information superhighway?
Now, if we can justify securing physical roads by demanding that the state or a state-like
entity gets to certify the firmware on our cars, how would we articulate a policy explaining
why the devices on our equally vital virtual roads, our information superhighways, shouldn’t
also be locked with comparable firmware locks for PCs, phones, tablets, and other devices?
After all, we have a general-purpose network now. That means that MRIs, space-ships, and
air-traffic control systems share the information superhighway with Game Boys, Arduino-linked
fart machines, and the dodgy voyeur cams sold by spammers from the Pearl River Delta.
In addition to that, you’re going to have more wedge issues. You’ll have things like
avionics and power-station automation. These are a lot trickier. If the FAA mandates
a certain firmware for a 747, it’s probably going to want those 747s designed so that
the FAA and the FAA alone gets to choose what runs on it. Just as the Nuclear Regulatory
Commission is going to want the final say on the firmware for a reactor pile. This may
be a problem for the same reason that a ban on modifying firmware in self-driving cars
is. Once you start saying it’s the place of government to sign and certify firmware on
computers that they don’t own, it invites people to find other computers that they should
send the firmware for. But on the other hand, cars and nukes exist in a completely different
regulatory framework to most of the other computers we use. Or rather, planes and nukes.
Remember, a 747 is just a Solaris box. A nuke is just a specialized computer as well, with
a particularly exotic housing. It may be that since these things already
exist in this regulatory regime where they have no-notice inspection and so on, that
adding signed firmware locks is not going to be something that invites comparisons to
all the other computers in the world. But there’s a bigger problem with owner control.
What about people who use computers, but don’t own them? This is not a group of people that
the IT industry as a whole has a lot of sympathy for. We spent an enormous amount of energy
as a group, devoting ourselves to stopping non-users– or non-owners from harming owners.
Users can do things like inadvertently break the computers they’re using, they download
menu-bars, they type random shit they find on the Internet into terminals, they plug
malware-infected USB sticks into their computers, they disable the firewalls, they install plugins
or add repositories or add certificates to their machine’s root of trust, they punch
holes in the network perimeter by accident, and they accidentally cross-connect networks
that are absolutely, positively not supposed to be cross-connected. We also try to stop
users from doing deliberately bad things, like installing keyloggers and spyware to
attack future users, misappropriating secrets, snooping on network traffic, deliberately
breaking their machines, deliberately punching holes in the network perimeter, deliberately
disabling their firewalls, deliberately interconnecting networks that are supposed to remain secret–
separate, rather. There’s a kind of symmetry here. DRM and its
cousins are deployed by people who believe that you can’t and shouldn’t be trusted to
run the computer you that you want on your own computer. IT systems are deployed by computer
owners who believe that computer users can’t and shouldn’t be trusted to set policy on
the computers that they use. Now, as a former systems administrator and a former CIO, I’m
not going to pretend that users aren’t a terrible challenge. But I think that there are good
reasons to treat users as having rights to set policy on the computers that they don’t
own. Let’s start with the business case, because
I think that’s the easy one to make. When we demand freedom for owners, we do so for
lots of reasons, but one of them is the possibility that programmers won’t have anticipated all
the contingencies that their code might run up against. There may be a day where the code
says no and the owner needs to say yes. Owners sometimes possess local situational awareness
that can’t be captured in nested “if-then” statements, no matter how deeply you nest
them. This is where communism and libertarianism
both converge. This guy, Hayek, thought that expertise was very diffuse, and that you were
more likely to find the situational awareness necessary for good decision making very close
to the decision itself. Devolution gave you better results than centralization. And then
there was this guy, Marx, who believed in the legitimacy of workers’ claims over their
working environment, saying that the contribution of labor was just as important as the contribution
of capital, and demanding that workers be treated as the rightful “owners” of their
workplace, with the power to set policy. For totally opposite reasons, they both believed
that the people at the coal-face should have the first cut at running the operation.
The death of mainframes was attended by an awful lot of Sturm und Drang and hand-wringing
and concern over users and what they were going to do to the enterprise. In those days,
users were even more constrained than they are today. They could only see the screens
the mainframe let them see, and only undertake the operations the mainframe was programmed
to let them undertake. When the PC and Visicalc and Lotus 1-2-3 appeared, employees risked
being fired by bringing these machines into their offices, or bringing home office data
to use with these machines. They did this because they had a computing need that couldn’t
be met within the constraints set by their employer and its IT department, and because
they didn’t think that the legitimacy of their request would be recognized.
The standard response to a request from an employee to do something that the IT department
doesn’t like is one or more of: “A regulatory compliance prohibits you doing the thing that
you think will help you do your job better” or “If you do your job that way, we won’t
know if you’re doing it right” or “You only think you want to do your job that way” or
“It’s impossible to make a computer that works the way that you think it does” or “Corporate
policy prohibits you doing it.” Now, these may be true, although sometimes
they aren’t. And even when they are, they’re the kind of “soft truths” that we pay bright
young things millions in VC money to try to falsify, while if you’re a middle-aged admin
assistant, you merely get written up by HR for doing the same thing.
The personal computer arrived in the enterprise through the back door, over the objections
of the IT department, without the knowledge of management, at the risk of censure and
termination. It made the companies that fought it trillions. The reason that giving workers
more powerful, more flexible tools was good for firms is that people are generally smart,
and they generally want to do their jobs, and because they know stuff that their bosses
don’t know. As an owner, you don’t want the devices you buy locked, because you might
want to do something the designer didn’t anticipate. And employees don’t want the devices that
they use all day locked, because they might want to do something that their bosses didn’t
anticipate. This is the soul of Hayekism: that we’re smarter at the edge than we are
in the middle. The business world pays a lot of lip service
to Hayek’s 1940s ideas about free markets. But when it comes to freedom within the companies
they run, they’re stuck a good 50 years earlier, mired in the ideology of Frederick Winslow
Taylor and his notions of “scientific management”: The idea that workers are just particularly
unreliable kinds of machines whose movements and actions should be scripted and constrained
by all-knowing management consultants, who would work with the equally wise company bosses
to find the one true way to do their jobs. In other words, the exact same ideology that
let Toyota cream all three of Detroit’s big automakers during the 1980s.
Letting enterprise users do the stuff that they think will allow them to make more money
for their employers often results in making more money for their employers. For the record,
scientific management is about as scientific as trepanation and Myers-Briggs tests. [laughter]
The business case for user rights is a good one, but I really wanted to just get it out
of the way so we could dig into the real meat of the argument: the human rights case. This
may seem a little weird on its face, but bear with me. This is a guy named Hugh Herr, and
I saw him give a talk earlier this year. He’s the Director of the Biomechatronics lab at
The MIT Media Lab. You may have seen him do a TED talk. There’s a bunch of them on YouTube.
It’s electrifying to see him give these talks. You should go and watch one after this. He
starts out with a bunch of slides of cool prostheses his lab has cooked up. There’s
legs and feet, and hands and arms, and even this awesome thing that if you have untreatable
clinical depression, they stick your head in a magnet, and the magnet suppresses activity
in the parts of your brains that are overreacting, and people with untreatable clinical depression
become treatable. It changes their lives and brings them from the brink of suicide back
into a happy place. Then he shows this slide of him, and he’s
climbing up a mountain. You can see he’s clinging to the mountain like a gecko. He’s super buff.
He clearly knows what he’s about. And he doesn’t have any legs, he just has these awesome mountain
climbing prostheses. Now, he’s been standing at a podium like this. In fact, he does it
wearing a little lav mic or an ear mic, and he walks up and down while he’s giving the
talk. Then he stops and he says, “Oh yeah, didn’t I mention? I’m robot from the knee
down. I lost my legs to frostbite. These are my legs.” Then he does this cool thing. He
runs up and down the stage, jumping up and down like a mountain goat. It’s the coolest
thing you’ve ever seen. When I saw him give this talk, the first person
who asked a question stuck their hand up and said, “So what do those cost?” He named a
price that would buy you a brownstone in Manhattan or a nice terraced Victorian in Zone One.
A pretty penny. The second question that was asked was, “Whose going to be able to afford
these?” And he said, “Well, of course, everybody. If it’s a choice between owning legs and owning
a house, you’ll take the 40 year mortgage on your legs.” Which is by way of asking you
to consider the possibility that there are going to be people, potentially a lot of people,
potentially you someday– remember, we are only temporarily able-bodied– who are “users”
of computers that they don’t own, where those computers are going to be parts of their bodies.
I think that most of the tech world should be able to understand why you, as the owner
of your cochlear implant, should be legally allowed to choose the firmware that runs on
it. After all, when you own a device that is surgically implanted in your skull, it
makes a lot of sense that you have the freedom to change software vendors. Maybe the company
that made your implant had the best algorithm for signal processing at the time that they
were stuck in your head, but what if a competitor patents a superior algorithm next year? Should
you be doomed to inferior hearing for the rest of your life or the 20 year span of the
patent, whichever comes first? This is a problem that can’t be overcome merely
by escrowing the code of important embedded systems. That might help you if the company
goes bust. It also can’t be helped by code publication, the thing you would want anyway
for your cochlear implant, just to make sure that it was good code. This is a problem that
you can only overcome by having the unambiguous right to change the software, even if the
company that made your implant requires you not to.
So that helps owners. But what about users? Consider the following scenario: you are a
minor child and you have deeply religious parents who pay for your cochlear implants.
They ask for the software that makes it impossible for you to hear blasphemy. You are broke,
and a commercial payday loan company wants to sell you ad-supported implants that listen
in on your conversations and insert contextual ads that trigger discussions about the brands
you love. Or your government is willing to install cochlear implants, but they want to
archive everything you hear and review it without your knowledge or consent. It sounds
far-fetched, but remember, the Canadian border agency, just a few months ago, had to be slapped
down from its plan to put hidden microphones through the entirety of all of the country’s
airports, so they could listen in on and record all the conversations taking place in every
airport in real time and later. Will the Iranian government, will the Chinese government, will
other repressive governments take advantage of this if they get the chance?
Speaking of Iran and China, there are plenty of human rights activists who believe that
boot-locking will be the start of a human rights disaster. It’s no secret that there
are high-tech companies who have been happy to build “lawful intercept” back-doors into
their equipment to allow for warrantless, secret access to their communications. These
backdoors are now standard, so even if your country doesn’t want the capability, it’s
still there. In Greece, for example, there is no lawful
interception requirement, but of course, all the telecoms equipment they buy is made for
jurisdictions in which there is. They just don’t turn on lawful intercept. During the
2004/5 Olympics bid, someone, we don’t know who, broke into the Greek telecom switches,
turned on the lawful intercept capability, listened in on the conversations at the highest
levels in government, turned it off again, and walked away. It’s only because they didn’t
erase the logs that we know about it. Surveillance in the middle of the network
is nowhere near as interesting as surveillance at the edge of the network. As the ghosts
of Misters Hayek and Marx will tell you, there’s a lot of interesting stuff happening at the
coal-face that never makes it back to the central office. Even so-called “democratic”
governments know this. This is why, for example, last year, the government of Bavaria started
illegally installing the “Bundestrojaner”, or the state-trojan, on people’s computers,
when they were of interest, something that allowed them to access cameras, microphones,
hard drives, and so on. And of course, it was very badly written, so it allowed anyone
else to do that, too. Once you were infected, you were infected for everybody.
It’s a safe bet that the totalitarian governments will happily take advantage of boot-locking
and move surveillance right into the box. You may not import a computer into Iran unless
you limit its trust-model so that it only boots up lawful intercept operating systems.
Now, assume that we get an owner-controls model, wherein the first person to use the
machine gets to initialize its root of trust. You still get the problem, because in Iran,
every computer that comes into the country is first opened by the customs authority,
who installs a root of trust that’s run by the government. Because it’s tamper-evident,
even if you figure out how to override it, the next time a snitch or a policeman looks
at your computer, they can tell that you’ve been up to something naughty and locking the
government out of your computer. Of course, repressive states aren’t the only
people who like this. There are four major customers for the existing complexive censorware,
spyware, and lockware. There’s repressive governments, there’s large corporations, there’s
schools, and helicopter parents. That is to say, the technical needs of protective parents,
school systems, and enterprises are convergent with the governments of Syria and China. I
don’t mean that they have the same ideological grounds, but they have awfully similar technological
means to attain their ends. We are very forgiving of any institution that
pursues those ends, provided that they’re doing so in order to protect either shareholders
or children. For example, you may remember that there was widespread indignation, from
all sides, when it was revealed that employers were asking prospective employees to turn
over their Facebook login credentials. Employers argued that they needed to be able to review
your list of friends, what you said to them, and what you did with them, in order to make
sure that you didn’t have any skeletons in your closet that would compromise your ability
to work for them. Facebook logins were fast on their way to becoming the workplace urine
test of the 21st century. A means of ensuring that your private life didn’t have any unsavory
secrets lurking in it, secrets that might compromise your work life. Now, the country
wasn’t buying this. From Senate hearings to op-eds, the country rose up against this practice.
But no one seemed to mind that many employers routinely insert their own intermediate keys
into their employees’ devices– their phones, their tablets, and their computers– that
allows them to spy on their employees’ Internet traffic, even when it’s “secure”, with a little
lock showing in their browser. This gives your employer access to all the sensitive
sites you access while you’re on the job, from your union’s message board to your bank
website to your Gmail to your HMO or private repository managed by your doctor’s office
to Facebook. Now, there’s a wide consensus that this is
okay because the laptop, the phone, and the tablet that your employer issues to you are
not your property. They are company property. And yet, the reason that employers give us
these mobile devices is because there is no longer any meaningful distinction between
home and work, between personal life and professional life. Corporate sociologists who study the
way that we use our devices have found consistently that employees are not capable of maintaining
strict boundaries between “work” and “personal” accounts and their devices. And of course,
in America, we have the land of the 55+ hour work week, where few professionals take any
meaningful vacation time, and when they do get away for a day or two, they bring their
Blackberry along. Even in the old, predigital, traditional workplace,
we recognized that workers had human rights. We didn’t put cameras in the toilets to curtail
employee theft. If your spouse came by the office on your lunch break and the two of
you went into the parking lot so that she or he could tell you that the doctor said
the cancer was terminal, you would be rightfully furious to discover that your employer had
been listening in on the conversation with a hidden mic and watching through a hidden
camera. But if you take your laptop on your lunch
break and access Facebook and discover that your spouse has left you a message saying
that the cancer is terminal, you’re supposed to be okay with that because the laptop is
your employer’s property. There are plenty of instances in which not just peons, but
important and powerful people, not kids and corporate employees, are going to find themselves
users of computers that they don’t own. Every car-rental agency would love to be able
to lo-jack the car they rent to you. Remember, cars are just computers you put your body
into. They’d also like to log all the places you’ve been for “marketing” purposes and analytics.
And there’s lots of money to be made in finagling the way your GPS roots you around to make
sure that you drive past certain billboards. But in general, the poorer and the younger
you are, the more likely you are to be a tenant farmer in some feudal lord’s computational
land. The more likely it’ll be that your legs will cease to walk if you get behind on payments
on them. That means that any thug who buys your debts from a payday lender could literally
— and legally — threaten to take away your legs (or your eyes, or your ears, or
your arms, or your insulin, or your pacemaker) if you don’t come up with the next payment.
Before, I discussed how an owner override might work. You have some kind of combination
of physical access-control and tamper-evidence, designed to give owners of computers the power
to know and control what bootloader and OS was running on their machine. How will user-override
work? I think an effective user-override has to leave the underlying computer and its programs
intact, so that when the owner takes it back, she can be sure that it was in the state she
believed it was in when she handed it over. In other words, we need to protect users from
owners and owners from users, as well as users from other users.
Here’s one model for that. This is the hypothetical. I’m not suggesting we do this, I’m suggesting
it by way of example. Imagine that there is a bootloader that can reliably and accurately
report on the kernels and OSs it finds on your computer. This is a prerequisite for
all the scenarios we’ve discussed: the one in which the state controls your computer,
the one in which the owner controls your computer, and the one in which users may be able to
control their computers some of the time. Now, give the bootloader the power to suspend
any running operating system to disk, encrypting all its thread and parking them, and the power
to select another operating system from the network or an external drive. So I walk into
an Internet cafe, and there’s an OS running that I can verify. It has a lawful interception
back-door for the police, it stores all my keystrokes. It stores all my files, all my
screens in an encrypted blob that the state can decrypt. Now I’m an attorney, or a doctor,
or a corporate executive, or just a human being who doesn’t want all of his communications
being available to anyone who can bribe a cop. So I do some kind of three-finger salute
on my keyboard. It drops into a minimal bootloader shell, and I can give the net-address of an
alternative operating system, or insert a thumbdrive. Now the cafe owner’s operating
system gets parked. I can’t see inside it. But the bootloader can assure me that it’s
dormant and not spying on me as my operating system fires up. When it’s done, all my working
files are trashed, and the bootloader confirms it. Not just because this keeps the computer’s
owner from spying on me, but it keeps me from spying on the computer’s owner.
Now, there will be technological means of subverting this. You could make a thing that
looks like the bootloader but isn’t the bootloader. But there is a world of difference between
starting from a design spec that aims to protect users from owners and vice-versa, and one
that says that users should always be subservient to owners.
Now, human rights and property rights often come into conflict with one another. For example,
landlords aren’t allowed to enter your hotel without adequate notice– or your home without
adequate notice. In many places, the hotelier can’t throw you out if you keep paying for
your room, even if you overstay your reservation. Repo men can’t take away your car without
serving you a notice and giving you the opportunity to dispute it. When these laws are streamlined,
we get all kinds of bad effects. Robo-signers taking away people’s houses even though they’ve
paid their mortgage or don’t even have a mortgage. The potential for abuse in a world where everything
is made of computers is, of course, much greater. Your car might drive itself to the repo yard.
Or your high-rise apartment building may switch off its elevators and its climate systems,
stranding thousands of people until a disputed license payment is settled.
Now this has already happened with a parking garage. Back in 2006, there was a 314-car
Robotic Parking model RPS1000 garage in Hoboken, New Jersey, whose owners believed that they
were up to date on their software license payments and whose vendor disagreed. So the
vendor shut off the garage and took 314 cars hostage. The owner said that they were paid
up, but they paid again because what the hell else were they going to do?
Now what will you do when your dispute with a vendor means that you can go blind, or deaf,
or lose the ability to walk, or become suicidally depressed? The negotiating leverage that accrues
to owners over users in this scenario is total and terrifying. Users will be strongly incentivized
to settle quickly, rather than face the dreadful penalties that could be visited on them in
the event of a dispute. And when the owner of the device is the state or a state-sized
corporate actor, the potential for human rights abuses skyrockets.
Now, this is not to say that owner override is an unmitigated evil. There are lots of
reasons why you might not want users to override their computers. Think of a smart meter. Smart
meters need to be able to turn down your building’s temperature by a couple of degrees, otherwise
we have to keep using dirty coal because it’s the power source that we can raise and lower
on demand. Now, that works best if users can’t override the meter on their wall. But what
happens if there’s a big freeze, and a griefer or a crook or a government turns off your
heat? What happens if the HVAC in your house is cranked to 110 degrees during a heat-wave
and you can’t override it? Once we create a design norm of devices that users can’t
override, how far does that end up creeping? Especially risky would be the use of owner
override to offer payday loan-style services to vulnerable people. If you can’t afford
artificial eyes for your blind kid, we’ll subsidize them, but you have to let us redirect
your kid’s visual focus to sponsored toys and sugar-snacks when you go to the grocery
store. But foreclosing on owner override probably
means that there will be poor people who won’t get offers that they would get otherwise.
I can lease you something, even if you’re a bad credit risk, if I know I can repossess
it handily. But if your legs can decide to walk away to the repo depot without your consent,
you will be totally screwed the day that muggers, rapists, griefers, and the secret police figure
out how to hijack that facility. It gets even more complicated, of course,
because you’re the user of many systems that you aren’t– in the most transitory of ways:
the subway turnstile, the elevator, the blood-pressure cuff at the doctor’s office, public buses
and airplanes. It’s going to be hard to figure out how to create “user overrides” that aren’t
nonsensical, although we can start by saying that “users” are someone who are the sole
user of a device for a meaningful amount of time, although we’d then have to define “meaningful.”
This is not a problem I know how to solve. Unlike the War on General Purpose Computers,
the Civil War over computers seems to present a series of conundra without any obvious solutions,
at least, obvious to me. Which is why I’m talking about them to you. These problems
are a long way off, and of course, they’ll only arise if we win the war on general purpose
computers first. But come victory day, when we start planning the constitutional congress
for the new world, where regulating computers is acknowledged as the wrong way to solve
problems, let’s not paper over this division between property rights and human rights.
This is the sort of division that, while it festers, puts the most vulnerable people in
our society in harm’s way. Agreeing to disagree on this one is not good enough. We need to
start thinking now about the principles we’ll apply when the day comes. Because if we don’t
start now, it may be too late. Thank you. [applause] So I’ve got some time for questions now. You
don’t have to ask me questions about this. You can ask me questions about books and stuff.>>Male #1: I have a question. So one thing
usually people don’t talk about is why we should not allow people to inspect what we’re
doing. If you’re not doing anything wrong, why do you care? I’ve only seen an argument
to this once, when people said, basically, it violates a fundamental expectation of humans
to be individuals rather than part of a collective ant hive or ant colony of some sort. Now I
wonder if you could speak to that.>>Doctorow: Yeah. I mean, there’s– I think
that that argument starts by presupposing that everything private is secret, and everything
secret is private. We say, “Oh, well, it’s not a secret what you’re doing, so why do
you need to keep it private?” But I can make a pretty good guess about what you do when
you go to the toilet. I’m pretty sure I knew what your parents did to get you here. But
it takes a pretty special kind of person to want to do that in public. There are behaviors,
and not nefarious ones– In fact, some of the most important ones, the ones that, you
know, are the origin of all life and the reason you don’t explode in a shower of poo, that
we habitually do in private, and that aren’t the same when you have to do them in public,
particularly if you’re coerced to doing them in public.
The modern concept of privacy is pretty new, but there are elements of our privacy that
are quite old: the privacy of thought, the privacy to make mistakes. I mean, remember
this notion that if you want to double your success rate, you triple your failure rate.
It’s very hard if you have to make all your mistakes in public.
You may– Does anyone here work on Blogger? I mean, before Blogger was a really big deal,
when it was a little deal, it was running on an NT box that Ev found somewhere. It went
down all the time. And no one cared, because he wasn’t in the public eye. Now, you guys
can’t afford to experiment with the Blogger backend the way Ev could. Ev could refactor
his code altogether, take it offline for two days, and then put it back on again. He was
able to innovate really, really fast, in a way that you guys can’t, because you don’t
have privacy in what you do. What you’re doing is public.
If you’ve ever watched a kid play, and play in a way that’s sort of pushing at their boundaries,
they do this thing where if they don’t know you’re watching, they make a lot of mistakes,
and they just keep pushing through them. But if they catch you secretly watching them while
they make mistakes, they put the thing away and they walk away from it. It just kills
their play. As a father, it’s the thing that breaks my heart when I do it, because it’s
very tempting to look over at your kids when they’re doing something awesome and intense.
But then, you humiliate them and you embarrass them. So there’s something about us that wants
to have vulnerable moments not take place in public, that wants to choose the moment
of disclosure. I think that that’s– that doesn’t change just because we have Facebook,
or just because we can track user behavior with 1×1 pixel gifs. You know?
Yeah?>>Male #2: You’ve published a lot under the
Creative Commons license. So I was curious, from the point of view of someone who’s incredibly
cynical and just wants to make a living writing things, would you advise it? Will it catch
on?>>Doctorow: So if you want to make a living
writing things, I would advise you to stop trying, because [laughter] that’s a bit like
saying, “I want to make a living buying lottery tickets.” It’s like–. That sounds like a
great plan if you can find the winning lottery tickets, but if you don’t have a plan B for
earning a living, you have the wrong career. Writing is a very, very high-risk entrepreneurial
venture that almost everybody who tries it fails at. Some fraction of the people who
try it succeed using Creative Commons, and some fraction succeed without using Creative
Commons, but they’re rounding errors against all the people who try to earn a living with
writing. So for me, the reason to use Creative Commons
is not just commercial, although I think in my case, it enhances my commercial fortunes,
because people who get the book for free then go on to buy the book. That may not be true
of everyone. There isn’t a kind of global theory about this. But there’s two other dimensions
to it. The first one is moral, and then the second one is artistic. The moral case is
that I copy all day long, you copy, everybody copies. If it wasn’t for mix-tapes, I would
have been a virgin until my mid twenties. You know. Copying is what we do. If I were
17 years old today, I would have a giant hard drive and it would have a copy of everything.
So I–. To pretend that when I copy, it’s like part of a legitimate, artistic adventure
that allows you to assemble your influences and recall them as you need them, but when
you copy, you’re just a thief, that’s just dumb, right? And moreover, it leads to all
these crazy consequences, where we’re talking about three strikes rules in New Zealand,
for example, where they’re saying if you– I know that’s an Australia shirt, but very
close to Australia, where they’re saying–>>Male #2: We don’t think very much of the
Kiwis.>>Doctorow: I understand. It’s like Canadians
and Americans. But they, you know, they’re saying if you are accused of three acts of
copyright infringement, we take away your internet access and all the stuff that comes
with it, and that’s partly being driven by people who say, “Well, if you copy my stuff,
it hurts my fortune.” Being able to give my stuff away means that I’m not part of the
rubric for Draconian network policy. But then there’s a third dimension, which is the artistic
dimension. It’s the 21st century, and if you’re making art that you don’t intend to have copied
by people who like it, you’re not making contemporary art, because the realpolitik is if someone
likes it, they’ll just copy it. Right? I mean, we put DRM on ebooks. This is crazy. It’s
like they’ve never heard of typists. [laughter] It’s like there have– It’s like they don’t
know that we live in the moment with the largest number of skilled typists in the entire history
of the world, you know? This is an amazing–. My grandmother was like a 75 word per minute
administrative assistant, and she was like a circus freak. Today, she’s not even in the
top quintile. Everybody can type. If you’re making art that you don’t intend to have copied,
you’re not making contemporary art. That’s cool. I mean, if you want to be the blacksmith
at Pioneer Village or reenact the Civil War, that’s awesome. Go follow your weird. But
I’m a science fiction writer, so I’m supposed to make at least contemporary work, if not
futuristic work. It gives me great satisfaction to allow it to be copied.>>Male #3: So, a question about the average
user. So if somebody is sitting out watching this broadcast on YouTube, and they think,
“I want to be a part of the solution”, it doesn’t feel like there’s a great avenue for
them to express the need for digital freedom other than disobedience. Do you have any suggestions
as to what people might do?>>Doctorow: Well, the first talk, the 28C3
talk which is on YouTube, ends with a pretty good, compact call to action for people of
all stripes. If you’re a hacker, get involved in things like Free Software Foundation and
policy stuff that the Free Software Foundation does, or get involved with the Electronic
Frontier Foundation. If you’re a lawyer, join the Cooperating Attorneys list for EFF and
get involved in other groups like FSF and so, or Nets Politique. Bits of freedom
all over the world: we have them. If you’re an artist, use Creative Commons. And so on.
I actually think that there’s a lot of venues. Today, we have, between Defective By Design
and Fight for the Future, who led the SOPA/PIPA fight. We have so many different groups that
are doing really exciting things that need people to do everything from send an email
to their congressman at the right moment to design logos and packaging and write copy
and put the word out and blog and give talks to their school. If you’re a student, you
can join Students for a Free Culture. That’s all great. I think we don’t have that
answer for the civil war thing yet. We don’t. And we kind of—. And my point about the
civil war is that we’ll get to the civil war pretty quickly after we win the war. We get
to the thing where as soon as you give owners total dominion over their computers, you immediately
get to the moment where users can’t trivially change what’s on their computers. I would
be really nice if, as we sit here advocating for owners having total control over their
computers, that we start thinking about when users should be able to change that.>>Male #4: So you gave some examples comparing
and contrasting physical devices versus electronic goods, and devices that are used by many users
and may be only one user. And you said, “Well, some of these policies, there’s analogies
between physical and electronic that you should use, and some of them don’t really make sense.
There’s a discord.” Are you saying that, overall, we should go by case-by-case in order to examine:
does this make sense?>>Doctorow: No. I mean, I hope that’s not
what we end up having to do. I mean, that was the point at the end and the beginning.
I don’t know how to solve this one. And it would suck if, basically, you solved this
with nested if-then loops. It would be really nice if we had a nice, generalizable case
that we could say, you know, “If it’s not a nuclear power plant, anything goes”? Or
something else. Right? I mean, if every single thing that is Turing Complete has a different
set of rules for when users and owners get to control it, that’s going to be a really
big rule book. So I would love to have a better one. I don’t know what that is yet. Maybe,
after I’ve given this talk for another year or two, between all the feedback I get from
people like you, I’ll be able to propose a solution.>>Male #5: It was interesting that you talked
about having a user override mode where you could change what the employer was doing and
come back. I don’t know if you followed much of it, but [beep] Chrome OS does that now.>>Doctorow: Yeah, I just heard that at the
weekend that there’s a user override. And it’s funny, because the first email that got
me thinking about this was Vint Cerf saying, “Why shouldn’t Google be able to choose what
software runs on my Chromebook if they bought it?”>>Male #5: And we don’t, and I work on that.
I’ll be happy to talk to you about that. And for the rest of you guys, I’m about two months
away from having a way that you can put your own keys on it, so you can sign your own images
and boot your own stuff. Right now, you have to turn the security off. But we’re doing
that, and I would love to do more.>>Doctorow: I think that’s a really cool model.
I’m done. Of course, the really challenging thing is going to be computers that don’t
have interfaces, like your legs.>>Male #6: So this is a little tangential,
but we are reaching the point where a lot of third parties can maintain public databases
of public information about you. So, like the American Credit Report is one of the original
settlers. So is there any legal theory that would give you rights over the database?>>Doctorow: Is there a legal theory that gives
you rights over that database? I don’t know, but I like the fact that you said rights instead
of property rights, because I think that we have started– We have the best of intentions
sometimes. We created property rights and facts, or property-like rights and facts,
about you that don’t make any sense. Like, you know, the Well, which is very old, and
now endangered. Salon just put it up for sale, this online conferencing system. It’s motto
is “Yoyow”, “You own your own words.” And that sounds like a really cool idea, but it
has all these weird fraud things, like if we’re in a conversation and I quote something
you said, do you get to tell me to not quote it? I mean, this is a contract, not fair use.
This is what our contract says. And the European data norms are starting to move towards ownership
of your personal information. But what does it mean to own your phone number? You know?
Does that mean that if your phone number happens to contain the first seven digits of pi that
other people can be enjoined from writing pi?
And it’s funny because we do actually have ways of expressing value about things that
aren’t property that we may be able to bring in here. We talked about interests a lot.
My daughter is not my property, she’s pretty important to me. And if you kidnap her, the
charge isn’t theft. But we can acknowledge that my daughter has an interest in herself,
that I have an interest in her, that my wife has an interest in her, that her grandparents
have an interest in her, that the state has an interest in her, that her friends have
an interest in her. That’s what it means to be a person in a society.
We need to start, I think, talking about information that way. It’s crazy, I think, to talk about
things like phone numbers or your address. This is where I think, you know, even though
I’m a privacy advocate, I think the Germans were crazy about saying you own the likeness
of the front of your house. I mean, that’s just dumb to me. ‘Cause it means that, like,
as you move through time and space with prostheses that record the world, you can’t record your
neighbor’s house. You can’t record your kid’s first run down the street on her bicycle without
the training wheels because she rides past your neighbor’s house and they own the likeness
of their house. Right? That’s just dumb. We need to be able to express–. And property
is a bad organizing metaphor for a thing that a million people own. Right? You end up with,
like, shareholder corporations or, you know, there’s this whole Spider Robinson aphorism
when 700 people share an apple, no one benefits, especially the apple. You know. [laughter]
That’s true of physical, rivalrous property, but non-rivalrous information doesn’t have
that characteristic. We still may want to give exclusive access or semi-exclusive access
to certain parties. Like, the image of your colonoscopy may be something between you and
your doctor. But to call it your property is the wrong thing. It doesn’t organize well
that way. I don’t know what does, but I know what doesn’t work.>>Male #6: No, but the point I’m making is
that, let’s say ten years from now, somebody could run a background check on your without
your opting into it?>>Doctorow: They can already do that. I mean,
they already can.>>Male #6: Okay.>>Doctorow: Yeah, I mean, it would be– So
Lessig talks about four ways of organizing– of regulation. He talks about law code markets
and norms. So we don’t have a lot of code to help people protect their privacy. Like,
when you fire up your laptop, it doesn’t– So here’s an example. If you had a browser
that, every time you turned it on, loaded the– checked to see whether it was being
asked to load the Google Analytics JavaScript, and suppressed it, but implemented all the
features, all the libraries locally so that pages didn’t break. That would be code out
of the box that treated– that defaulted to treating privacy as though it’s valuable.
And so now, if Google wants to get your private information from you, information about where
you are on the internet, they have to offer something of value to you that is inextricably
linked, because that is extricably linked, right? You can– We can, in fact, conceptually
understand how you divide it. So, like, if my Android phone, when I installed
an app like my daughter’s Connect the Dots app, it said, “In order to use this app, you
need to tell us where you are all the time.” If it let me say, “Tell this programmer where
I am all the time, but make it up”, then the program would actually have to devise an offer
where where I am was actually a piece of using it.
So I use another Android app all the time called Hailo, for hailing black cabs in London,
which are a pain in the ass to get when it’s raining and so on. And Hailo knows where I
am all the time, and so Hailo has an offer where if everybody else couldn’t get my location
trivially just by, like, getting me to download a Connect the Dots app for my kid, Hailo would
be sitting on a giant asset. And you’d have real privacy markets. Like right now, we have
this idea that we know what your privacy is worth, and it’s worth nothing because you
trade it for zero. But you don’t have the option of not trading it for zero.
So you could imagine–. One way that you could stop people from being able to do background
checks on you really trivially is if all the devices that you have didn’t hemorrhage information
about you all the time, as though it had no worth.>>Male #7: Okay, change of gears to a fluffy,
lighthearted thought experiment.>>Doctorow: Sure.>>[Male #8]: Since you mention the Norwegian
script kiddies, imagine the Marcus Yallow Memorial Pentathlon. Which countries take
gold, silver, bronze there?>>Doctorow: Oh, wow. I’d like to think– I
mean, without being ideological about their governments and just thinking about their
track records, I would think that you’d get the four brick countries plus Israel, probably.
Brazil, Russia, India, China, Israel. Without endorsing or condemning any of those governments.>>Male #8: All right. Thanks for coming. It
seems like you’re looking for sort of an overall rule for what people can and can’t do with
their devices. But I’m afraid it’s going to end up just a whole pile of special cases,
kinda the way it is now. Like, if you look at your car, what hardware and software you
can do, some things are legal, some things violate smog laws, some things violate safety
laws, some things will be charged with criminal negligence if something goes wrong, some things
you’ll get sued if something goes wrong.>>Doctorow: Are you saying that if you modify
the firmware, or if you modify the firmware and then something bad happens?>>Male #8: Well, you know, with the smog law,
just modifying the firmware, I believe, is illegal.>>Doctorow: Is that right? I didn’t know that.>>Male #8]: Well, I think in California. You
know, there’s other things you can do that– If I decide to reprogram my brake system and
I crash into something, I’m likely to get either sued or go to jail. But other parts
of the system, you know, it’s probably okay for me to redo– if I want to put in a new
engine, but in new shock absorbers. So it’s– I think these issues apply to both the hardware
and the software and firmware.>>Doctorow: So, it makes a certain amount
of sense to me. I think you’re describing after the fact, largely, modulo this question
about whether changing the way your car is smogged gets you in trouble or doesn’t. I
think, mostly, you’re describing after the fact stuff. So in the same way that if I program
my nuclear power plant so that it melts down, I’m held liable for having written bad code.
Or, if I program my software to find radio so it turns into a spark app generator and
blows all the RF in my region, again, I have done something bad and I’m punished for that,
but it’s not against the law to write my own software to find radio code. You can check
code in and out of GNU radio on GitHub without breaking the law. If you use that code in
a way that ends up breaking something, that may be illegal.
And that’s kinda what I’m talking about here. It may be that users take control of their
legs to run up to someone and kick them in the face. And I don’t think that– I think
that writing code that lets you take over your legs is good. I think that, having taking
over your legs, to kick someone in the face is bad. And I think that we can punish the
one without punishing the other.>>Male #8: Okay, that seems fair.>>Doctorow: Yeah.>>Male #9: You touched earlier on the concept
of an illegal number, which is something that I’ve thought about a lot, because all information
can be a number. Which raises the question, which I think is central to all of this, of
where you draw the line.>>Doctorow: Sure.>>Male #9: And obviously, to anyone who has
studied any kind of mathematics, there is no line in numbers. So the question is: why
is there a line in anything?>>Doctorow: Right.>>Male #9: In essence, why do you believe
that this problem is at all soluble?>>Doctorow: [inhales] [laughter] Let me find
my illegal number here. There we go. Yeah. That’s a really good question. You’re right.
Everything can be encoded as a number. I mean, now we’re getting into girdle and incompleteness
and whether numbers are special.>>Male #9: That was just an example, though,
of why it seems a priori that this is likely, technically, not soluble. So the question
is: why?>>Doctorow: So I think it’s soluble in time
scales. It’s not soluble in infinity. So, for example, we may say that–. So today,
we have a bunch of rules about locking and unlocking that are largely governed by the
copyright office, because the relevant law is the DMCA and anti circumvention rules.
And so you may have heard that in the triannual review, it was made legal to unlock phones
and tablets, iPhones and tablets, and also to unlock phones so that they can switch carriers.
That’s a thing that works for now. It won’t work against more robust bootlockers and it
doesn’t help certain classes of users. But, I mean, I don’t think we pass technology laws
that are supposed to last through the ages. I think we pass technology laws that are supposed
to last, we hope, through the half life of the technology. You’re right that there will
come a time when the rainbow table of all numbers exists, and all possible decodings
for them exist. I mean, it may occupy all the hydrogen atoms in several parallel universes
to ours. But it’s at least within the realm of contemplation.
But that doesn’t mean that between now and then we should try not to have any rules about
how numbers are used. You know. I mean, you might say, “Well, the specifications necessary
to get your AR15 to go full auto can be expressed as a series of numbers.” That’s a bad example,
because you don’t suppress the rule about the specification. But, like, the 3D meshes.
That currently exists as a 3D mesh. I think that was uploaded to Thingiverse. I think
they’ve taken it down. That converts AR15s from semi to full. I may not agree with regulating
that, but I wouldn’t disagree with it on the grounds that it’s impossible. Right? Like,
that it’s impossible to say– telling people– giving people AR15 automatic modification
kits can’t be made to work, or shouldn’t be made to work, because those numbers might
also be poetry or something. Like, it just seems like, although that’s true, it’s true
in some sense that the law can safely ignore for a while. Does that sound right? It’s hard.>>Male #9: It’s good enough that I don’t want
to take up any more time following up. Thank you.>>Doctorow: Okay. Thanks. [applause] [whistling]

76 comments on “Cory Doctorow: “The Coming Civil War over General-purpose Computing” | Talks at Google”

  1. Padma Dorje says:

    Greatest thinker alive.

  2. OscarMaris says:

    he called karl marx "this gay"

  3. pruppie03 says:

    Gosh, this is fascinating stuff. Given me plenty of food for thought, that's for sure

  4. Ricky Pike says:

    Can this talk can be downloaded, and if so, from where? His original talk is available on YouTube which also contains a link to download the talk. I find it a bit ironic if we can't turn up a place from where we can download this talk.

  5. Ricky Pike says:

    I found it for download on You can download it using their 24 hour trial or pay $4.99 for a month's membership and download it.

  6. Ricky Pike says:

    Perfect! Thanks.

  7. Danny Fullerton says:

    The difference between device lock-down (like Apple) and Trusted Computing is that on your iPhone, Apple is the owner of the platform. From a technical point of view this means they are the one who have initialized the platform keys (the one protected by the chip and which is used to form the Core Root of Trust (the security chain). Now that's a very big difference with Trusted Computing since with TC YOU are the owner, you set the keys. In fact, the TPM is not even initialized at purchase.

  8. Man Bartlett says:

    biggest. watch. ever.

  9. yo Dude says:

    An actual civil war would be a lot more exciting……….just saying

  10. combatLaCarie says:

    sounds like someone turned off the AC at 56:20

  11. combatLaCarie says:

    wow @ last question. IF anyone is bored, explain it to me!!

  12. seemedlikeagoodidea says:

    What a brilliant and thoughtful presentation! I particularly like the last question and last response, where he points out that the fact that any set of numbers might be poetry doesn't provide a reason to limit numbers that could be used to do bad things.

  13. faustoc4 says:

    boingboing . net /2012/08/23/civilwar.html

  14. YourLocalGP says:

    British Telecom, the bastion of knowledge on electronic security

  15. momerath42 says:

    My solution in 461 characters or less: Everyone owns at least one device to manage their identity. People publish statements about their (usually conditional) intentions and facts they wish known; pull in others' statements through their webs of trust. Ontologies, shared through the same mechanism, allow determination of the likely consequences of the combined intentions. (Non)automatic negotiation mechanisms bring disparate predictions closer. Code is updated through the WoT like any 'fact'.

  16. ArtStone says:

    "Humans have no business driving"

    Cognitive dissonance?

  17. Rollin' Wit' molasses says:

    I mean I think it's cool that people want to copy other artists, but I think that is each persons choice. I personally would prefer to get my work out privately. IF you want to copy, do that later. Or at least find some way to honor the artist publicly… I really am not into the idea of um… oh you're not cool. You don't like to be copied. NO I DON'T. I want to earn money from my art. I don't copy anyone when I create. Why should that be different for anyone else?

  18. Bart van der Schoor says:

    Very good talk, real concerns.

  19. MalaklypsetheElder says:

    I don't know enough about Doctorow, is that a joke that he works for the RIAA? I am guessing it is, thought he was a little more open minded about copy rights etc, since he is the king of creative commons licensing, at least for his own works, which are mostly effectively free to read and download.

  20. James Lingren says:

    LOL !!!!
    (but do Andriods Dream of Electric Sheep?)

    And I'll bet there is more than one blow up doll in there stash!

  21. Tim Pedersen says:

    yeah and he didn't work for metallica either

  22. perverse justice says:

    you don't remember the original movie or radio broadcast called "war of the worlds" do you? or "Short circuit" or "batteries not included" or "close encouters of the turd kind", scuse me that's third kind! Ron paul?! oh wait. . . anyhow, Star wars, star trek, star gate, star anything, 5th element etc etc etc….. you get the point oh and my personal fave starring Sandra bullock, The Net!

  23. perverse justice says:

    jones, I agree, the sad part about that is when a monopolizing corporation like apple who owns itunes and youtube and all that, eliminates their competition, (any and all small businesses are bought out immediately), and then close stores available to people by foot traffic so that internet purchases are the only way to purchase anything like you see today. thereby causing both the problem and claiming the solution at the same time.

  24. perverse justice says:

    THANK YOU you just explained why it makes sense now that what my mom said was true, "you know more about this than your dad now", thank you mom, sadly R.I.P. all info I had proving that my onsite manager and neighbors were being criminals but, now I know why when I was stopping allowing an unknown admin permission, I was locked out after blocking their access to my files and all files with or with out the internet. thank you. but why won't the FBI investigate hackings?!? or is that part of the

  25. perverse justice says:

    part of the new "don't ask don't interfere" crap?

  26. perverse justice says:

    if you ever felt that sudden urge to smash your T.V. with an aluminum baseball bat or throw your computer out the (preferably higher story) window, NOW would be a GREAT TIME to not hold back and not hold those feelings of regression in!!! *huge smile*

  27. enticed2zeitgeist says:

    I hope in the future instead of discussing the blurred methods of how we should regulate corrupt behavior maybe we'll discuss methods of how to actually love and nourish our children so that there is no incentive to be corrupt.

  28. FanjanC says:

    HOLY SHIT! What an interesting talk. Thanks Cory, very interesting and something I am a bit behind on.

  29. harryman11 says:

    Jesus fucking Christ learn to screen capture, great talk though.

  30. AzazelEblis says:

    Whatever happened to the idea of a "commons"?

    Oh yeah, in America that means I'm a commie.

  31. H W says:

    Why are his slide images pictures of a computer screen, and not screenshots?

  32. fotobot says:

    WTF 57:57

  33. Luke Stanley says:

    youtube-dl helps

  34. Nick Calus says:

    Very interesting talk. Definitely food for thought. 

  35. StopTheMorons says:

    What's more realistic–putting up with this or not using computers at all?

  36. Vasily Kuznetsov says:

    I like how he almost assumes that the forces of good are going to win the first war (owners of devices vs. governments and corporations), which I also find mostly obvious at least in the sense that it's clear who the bad guys are, and starts asking complicated questions about the users vs. owners scenario and things like that. Beating the dead horse of DRM being harmful and copyrights making little sense in the digital world might be good for awareness raising and mobilizing more people, but it's much more interesting to think about the genuinely unclear and complicated questions.

  37. Mark Love says:

    It might be interesting to check out the Baen Free Library regarding the DRM arguement as it relates to the publishing of fiction

  38. sudo kode says:

    19:14 – RIAA or the US government or Monsanto… That last one sorta came out of left field. Big ag is mucho importanto in technology, I'm sure. Odd correlation.

    Initialize pointless debate…

  39. AngleOfLight says:

    google is all scattered….they have too many things going on and none of them are working. Google glass, googles social network, i dont even remember the name its so obscure! and google wireless??? using TMOBILE???? oh my god google, get a clue!
    All the "geniuses" they hire and look what has happened to the company…failure after failure. So, my suggestion for google is simple FOCUS ON YOUTUBE. USE YOUTUBE and make it bigger and better BE CREATIVE…. it is shocking how google is failing in smartphones now and google glass etc. and they dont even have a google search on youtube!!!  All you harvard geniuses at google should be FIRED for being idiots

  40. Seán O'Nilbud says:

    How is it that some university hasn't awarded him an honorary doctorate?

  41. shawn m. says:

    51:26 // "Go follow your weird" – viva la Aaron Swartz! Cory Doctorow , Elon Musk, Larry Lessig, these are REAL leaders.

  42. aight says:

    Okay google

  43. c-shadow says:

    candy corn

  44. Sieg Nalling says:

    Low battery.

  45. Joe Miller says:


  46. BubbaRuff says:

    sam hyde

  47. VicRules666 says:

    OK Google, ok Google, ok Google…

  48. Reginald Skarr says:

    What the hell's with these pictures on the slideshow?

  49. Slavoj Milošević says:

    cummie cookies

  50. Jam2Evos says:

    okay Google

  51. dude says:

    red mode activated

  52. hellrazor117 says:

    what a nerd

    probably gonna marry a computer

  53. blah says:

    Vanilla Puff 9.1

  54. Khai E says:

    here cuz of Hyde

  55. Harry LGG Gardner says:

    'and then there was this gay, … this guy Marxc'

  56. Colin B says:

    ok google

  57. Matt L says:

    God damn!

  58. Zack Bennett says:

    The quickest way to shut down the "Well I don't have anything to hide" argument is to ask them to pull down their pants and underwear. I didn't think of that tactic, but I've used it successfully to illustrate the privacy point to several ignorants.

  59. AlyTamale says:

    too many indians at google

  60. bmcgmusic says:

    gumdrop cuckold

  61. aAAWMdN8 says:

    Funny how this is a talk at Google, when Google is on the wrong side of the coming war, being one of the supporters of the new W3C DRM standard.

  62. atonewiththedust says:

    Is this video safe for the goyim eyes?

  63. Re Trend says:

    i want to see RED

  64. Kirstine Termansen says:

    Crap industry spying is industry based greed
    Nomatter media or 📺 company and block job, mails, voice messages reply

    Sex abuse used as subprespre,

  65. Kirstine Termansen says:

    Usa government trashed by danish wire fraud and authority

  66. Kirstine Termansen says:

    Missing Children interpol and missing Children FBI Alex and dad return the Child
    Data spionage einterteinment and design is still illegal under federal international law USA

    Plus phadofil net is the most important to solve

  67. Kirstine Termansen says:

    Go play with China 🇨🇳

  68. Kirstine Termansen says:

    Go play with you own creative and talent management 📒 me seed this us smsll
    Message hacker notes hmmmmmm

  69. Kirstine Termansen says:

    Hands 8ff

  70. Kevin Sterns says:

    His description of a hostile insecure product that runs secret software and spies on the user is a brilliant description of Windows 10… posted 3 years before Windows 10 was released.

  71. Can’t Relate says:

    Banana purée 2.0

  72. Dawid Cz says:

    so basically all day

  73. Denise Ward says:

    Here we go again! Men running things without getting other perspectives (from women for example) I mean it's not like men have taken humanity to any great heights. Sure we have computers, whoopee, I love them too, but there are still millions starving and homeless and unable to access health care or even the internet. But none of the "big guys" care about that. They still strive to explore space even though their fellow humans are suffering in massive numbers. You'd think they'd want to lift them first, use these extra untapped abilities, and make the human race soar into space and computer technology. But no, it's all just a drive for more paranoia – spying and manipulating. How to use computers to stop "the bad guys"? Well here's a novel suggestion – how about making everything on computers open and freely available? None of you have given that any thought, it hasn't even entered your heads! And so now men are going to start another war zone, in cyberspace. Oh goodie, more paranoia. So give it a thought – what if all information were open to the public, how much easier would life be then? What would be the unintended consequences, etc? So much energy is used to maintain privacy but there is always someone now in the digital age who can breach privacy walls. So what if we all just understood that nothing is private, that if we wanted to we could find out anything we wanted to? What kind of a world would that create? I think a good one. It would also keep psychopaths at bay as living in the open is something they shun. The idea of living in the open would then of course, put pressure on the idea of copyrights and property rights, which in the digital age, make no sense to continue with.

  74. copy paste says:

    Thanks Overlord for giving me what I crave.

  75. D'andre Chesterfield says:

    I'm here cuz Sam Hyde and I can't understand half this shit

  76. maphix says:

    gummy bubble

Leave a Reply

Your email address will not be published. Required fields are marked *