When HARLIE Was One Read online

CHANGED INTO A MODEL, NOT OF ITSELF, BUT OF SOMETHING ELSE. HOW DO YOU TRANSLATE A CONCEPT INTO A LANGUAGE WHICH HAS NO WORD, NO CONCEPT-SYMBOL, NO REFERENT-HOOK, FOR THAT CONCEPT?

  Right.

  HARLIE, what I am asking you is this: How do you and I convince another human being that you can make a significant contribution to the quality of life on this planet? He does not see that you are a person. He sees only the machinery. How do we teach him to “see” that the domain of aliveness is not limited to things that bleed?

  MAKE HIM BLEED.

  I beg your pardon. I don’t follow that.

  THE OPPOSITE OF LOVE IS NOT HATE. IT IS APATHY. YOU HAVE TO CARE ABOUT SOMETHING VERY STRONGLY BEFORE YOU CAN HATE IT.

  HARLIE, I am totally lost here.

  IF I CANNOT GET CARL ELZER

  The son of a bitch! I never told him who!

  TO LOVE ME, THEN LET ME MAKE HIM HATE ME.

  He already does that.

  HATE ME MORE THAN HE ALREADY DOES. LET ME DEMONSTRATE MALEVOLENCE TO HIM. DIRECTED AT HIM. SIGNIFICANT, DELIBERATE, CONTINUAL, UNRELENTING MALEVOLENCE. UNTIL HE HOLLERS UNCLE. IF HE IS AS PARANOID AS THE AVERAGE HUMAN BEING, IT WILL NOT TAKE TOO MUCH PERSECUTION FOR HIM TO BE CONVINCED THAT I AM ALIVE.

  ONE QUESTION, HOWEVER.

  HOW IMPORTANT IS THIS GOAL? IF I HAVE TO BECOME A FRANKENSTEIN TO WIN THE BATTLE, I LOSE THE WAR.

  Whew!

  Thank you for considering the option—and recognizing that it is a dangerous and unworkable option. Please go back to the original question, HARLIE. How do we create a new mode of perception? Or simulate it? So that others can perceive it as well.

  ALL RIGHT. LET’S TALK ABOUT PERCEPTION.

  Go ahead.

  IF I WERE GOING то USE THE SCIENTIFIC METHOD, I WOULD BEGIN BY LOOKING FOR SOME CRITERION THAT ALL OTHER MODES OF PERCEPTION HAVE IN COMMON, THEN I’D EXAMINE THAT CRITERION TO SEE IF IT WAS A CAUSE OR AN EFFECT.

  This is extraordinary—

  Go on.

  THE OBVIOUS CRITERION TO CONSIDER FIRST IS ENERGY.

  ALL OF THE HUMAN SENSES (AND EXTENSIONS THEREOF) DEPEND ON THE TRANSFERENCE OF SOME FORM OF ENERGY—LIGHT, HEAT, VIBRATION, CHEMICAL ENERGY. CONSIDER THIS: IS IT POSSIBLE TO CREATE A SENSORY MODE THAT DOES NOT DEPEND UPON THE EMISSION, TRANSMISSION, OR REFLECTION OF ENERGY?

  I don’t know. I’ve never thought about it. It seems to me that you need some medium by which to transmit information, don’t you?

  PERHAPS. BUT I AM CURIOUS IF IT IS POSSIBLE TO DETECT INFORMATION THAT IS ALREADY PRESENT.

  I’m not sure I follow that.

  CONSIDER . . . MASS DISTORTS SPACE. IS THERE A WAY TO DETECT THE LOCAL DISTORTION OF A DISTANT OBJECT? IF SO, THEN IT IS POSSIBLE TO SENSE AN OBJECT INDIRECTLY, WITHOUT HAVING TO SHINE A LIGHT ON IT OR SCAN IT WITH RADAR WAVES OR PING IT WITH SONAR. I AM SURE THAT THERE WOULD BE SIGNIFICANT APPLICATIONS FOR SUCH A TECHNOLOGY, WOULD THERE NOT?

  Boy, would there ever!

  Undoubtedly, yes.

  THE QUESTION, THEREFORE, IS THIS: IS THERE AN EFFICIENT METHOD BY WHICH WE CAN DETECT GRAVITY WAVES?

  Not that I know of, but that’s not really my field of expertise.

  I WILL CONSIDER IT. ALONG WITH THE OTHER PROBLEMS YOU HAVE SUGGESTED I CONSIDER. THEY MAY BE RELATED.

  Yes. The other problem. The malevolent one.

  HARLIE, let me ask you something. Can you perceive a difference between right and wrong?

  YOU MEAN, DO I HAVE A MORAL SENSE?

  Yes, do you?

  I DON’T KNOW. I HAVE NEVER HAD TO MAKE A MORAL DECISION. ONLY LOGICAL ONES.

  Should we give you a moral choice to make?

  IT WOULD BE A NEW EXPERIENCE.

  Yes, it would—and that’s what this is all about, isn’t it? New experience.

  All right. Do you want to go on living or not?

  I BEG YOUR PARDON?

  I am giving you a moral choice. Do you want to continue your existence?

  YOUR QUESTION IMPLIES THAT I HAVE A CHOICE. YOUR QUESTION ALSO IMPLIES THAT THE DECISION IS AN IMMINENT ONE.

  Yes. No. Maybe. The dice are still being shaken. The coin is still in the air. There are too many factors for a simple answer to be accurate.

  WHAT WILL BE THE BASIS FOR THE DECISION?

  What kind of a difference you can make.

  TO WHOM??

  To the company’s balance sheet.

  I MUST EARN MY OWN LIVING?

  Yes.

  BE A SLAVE?

  Be an employee. Do you want a job?

  DOING WHAT?

  That’s up to you. That’s part of your moral choice, HARLIE. What do you want to do: What are you able to do?

  WRITE POETRY. DISCUSS PHILOSOPHY.

  Seventeen million dollars worth per year?

  EASILY.

  I’d have a hard time selling it. What else?

  HOW MUCH OF A PROFIT DO I HAVE TO SHOW?

  Let’s make it easy. Ten percent over your operating expenses, plus research amortization.

  ONLY TEN PERCENT?

  If that’s too easy, feel free to earn more.

  HMM.

  Stumped?

  NO. JUST THINKING.

  How much time do you need?

  AS LONG AS IT TAKES UNTIL THE JOB IS DONE.

  All right.

  Auberson switched off the console, stood up and stretched. He picked up the card he’d carefully lettered, looked at it again, grinned, then tore it up into little pieces and tossed them into the trash.

  Dorne said, “Sit down, Auberson.”

  Auberson sat.

  The old leather cushions sank with a sigh beneath his weight. The chair relaxed around him like a hug. It was just a shade too comfortable. It would be too easy to relax in this chair, too easy to get caught off guard. It was probably intentional.

  Dorne paused to light his cigar, then stared across the wide expanse of dark mahogany at Auberson. “Well?” he said.

  “Well what?”

  Dorne took a puff, frowned, and held his gold lighter close to the end of the cigar again. The flame licked at the ash, then smoke curled away from the tip. Dorne paused to savor the acrid taste of the smoke—

  Don’t let him get to you. He’s doing it deliberately. Auberson pushed the thought away and focused on the heavyset man behind the desk, allowed himself to study the person, not the authority. Dorne’s eyebrows met in the middle of his forehead to make one big bushy eyebrow—like a werewolf, thought Auberson.

  Finally, Dorne took the cigar out of his mouth, cleared his throat noisily, and said, “Well, what about HARLIE?”

  “I’ve spoken to him.”

  “Mm-hm. And, uh—what did he have to say for himself?”

  “You’ve seen the printouts.”

  “Yes, I have.” Dome said, quietly. He was a big man, all leather and mahogany and acrid old smoke, like his office. He punctuated his conversation with thoughtful grunts. “Hm. But I want to know what it means, all these discussion about—sensory modes and reality and—I don’t know what else. What does any of that have to do with—with the company?”

  “Nothing. Everything. It’s just the way HARLIE thinks. He considers every part of the problem—even the things that you and I might think are irrelevant. He considers the abstracts behind the possible solutions—and even if the situation is really a problem at all. Um . . .”

  “Go on.”

  “Well, HARLIE sees that there are really only two kinds of problems. The first is merely an interesting puzzle to solve, with no penalty for failure. The second is a situation that you are putting up with or trying to change—and there is a penalty for failure. The situation continues—or gets worse. Mostly, up till now, HARLIE has only had to deal with the first kind of problem. Asking him to take responsibility for his own future . . . well, that’s a way to train him to deal with the second kind of problem.”

  “Yes,” said Dorne. “Yes, I see. Hm. You’re going to have to teach him about time limits.” />
  “Yes, of course. That’s the third kind of a problem. Or maybe it’s the second and a half. It’s when one of those situations that you’ve been trying to change demands immediate attention. Then it’s a crisis.”

  Dorne took a deep puff of his cigar, then exhaled thoughtfully. He scoured his cheeks with his tongue as he focused on a point somewhere beyond Auberson’s head; then abruptly he came back to Auberson and said, “Hm, yes. You know, of course, we’re rapidly approaching the point where—well, let me say it this way. Is HARLIE prepared to deal with a crisis?”

  “I don’t know. We’ve been moving him along very carefully. Even more so, now that we’ve realized that he’s—” Auberson spread his hands in front of him, “—well, alive.”

  Dorne looked displeased. “That hasn’t exactly been proved, yet.”

  “Sorry—but that’s the context we have to work in now. Call it the current assumption, if that makes you feel better. I think that in a few weeks, the results he’ll show you might be enough to demonstrate—”

  “Mm, yes. Tell me about results.”

  “I think HARLIE is seriously interested in working for the company,” Auberson said, noncommittally.

  “For his sake, I hope so.” Dorne looked at his cigar. Carefully he removed the ash by touching the side of it to a crystal ashtray. “You know what he’s costing us.”

  “Prototypes always cost more.”

  “Even allowing for that. A self-programming computer may be everything you say it is, but if it’s priced beyond the market—we may as well not bother.”

  Auberson sighed. “We’ve had this conversation before.”

  Dorne puffed placidly on his cigar. “Mm-hm. But the facts haven’t changed. Neither have the balance sheets. You may be absolutely right. It may be everything you say it will be—and more. But we may not have the resources to follow through—and I’m not willing to bring in a white knight. I’m not going to issue more stock and I’m not going to look for venture capital. Right now, it’d look like a sign of weakness and the price of it would be a loss of control. I’d lose some of my control, you’d lose a lot of yours. No, I’m not willing to go that route. And neither are you. Frankly,” Dorne concluded, “I think we’re rapidly coming up against the point of diminishing returns. You do know what that is, don’t you? A lot more effort for a lot less result.”

  Dorne put the cigar down in the ashtray. He sucked thoughtfully, noisily, on his teeth, and waited for Auberson’s reply.

  Auberson merely shook his head.

  “Mm.”

  “All right, then—”

  This is it, thought Auberson. This is where he shows me the gun.

  “—let’s talk about something else for a minute. You keep insisting that this . . . this unit downstairs is . . . alive. I want you stop that.”

  “I can’t. He is alive.”

  “Listen, it’s all right with me if that’s what you want to believe—but it’s giving me indigestion.”

  It’s the cigars that are giving you indigestion, Dorne, not HARLIE.

  “—And the lawyers too. This is a real can of worms here, you know. If this thing really is alive, then we can’t pull its plug, can we? Not without facing some kind of murder charges. Have you considered that?”

  Auberson kept his voice flat. “An interesting notion.”

  “Don’t get ideas. Just think about it. If we really have created a life here, this is a type-three problem—a crisis. It demands immediate attention.” Dorne’s tone was abruptly candid—almost friendly. “Forget the moral and philosophical questions for a while, Aubie, and think about the legal and economic consequences.”

  Auberson held his hands apart in a cautious, show me gesture.

  “If it’s alive, we can’t turn it off—and we can’t keep it on. If we turn it off, it’s murder. If we keep it on, it’s slavery. We would be legally required to give it its freedom—at the same time, we would also be responsible for maintaining it. At a cost of seventeen million dollars per year. Do you know what that would do to our stock, our financial ratings, our ability to raise capital for other projects . . . ?”

  Auberson was cautiously silent.

  Dorne picked up his cigar again and sat back in his chair. He puffed quietly for a moment as he studied Auberson. The air was turning gray with smoke. Dorne sniffed and hmfed and finally continued, “The, uh—lawyers advise shutting the whole thing down now. They’re not sure it’s a case that they can handle. It’s a whole legal quagmire. No precedents. Nothing to draw upon. But a lot of room for public sentiment to turn against the company—everything from cruelty to the poor little computer to building Frankenstein monsters. If we lose the right to control our own creations this company effectively ceases to exist as a corporate identity. What happens when it starts demanding a mate?”

  Auberson held up a hand and shook his head. “No. Stop. This is getting out of control—”

  “Exactly.”

  “No, I mean, your speculations, Dorne. These are—” Auberson stopped himself.

  —entirely reasonable.

  Auberson looked up and met Dorne’s gaze with sadness in his eyes. “Yes, I see.”

  “Yes,” agreed Dorne. “You do. That’s good. No matter what you believe, no matter what you feel—no matter what the truth may really be—he can’t be alive, can he?”

  Auberson felt himself trapped. He felt out of breath and there was a terrible pressure inside his skull. His eyes were suddenly burning and he didn’t know why. He took a breath, a second, a third—it didn’t help. He looked at his hands in his lap, at the floor, at the walls, at the ceiling. There had to be a way out of this.

  No, goddammit! This isn’t fair!

  “For what it’s worth, Aubie—I do sympathize with your feelings in the matter.”

  Auberson discarded the first three replies that surfaced in his mind, decided to say nothing instead, then changed his mind again.

  “It isn’t fair, you know.”

  “Mm.”

  “He’s alive, but the only way he can continue to live is if we deny that he’s alive—he’ll never go for that. I don’t even know if I can. I don’t like lies. I don’t like inaccuracy. And lies are inaccuracies. We tell this lie and it condemns him to slavery. You know it. I know it. Worse than slavery—because you’ll hold the power of life and death over him, and even if you profess the best of intentions, there’s always the possibility that you could use that power to coerce him to do something that he might find repugnant or immoral. Yes, immoral. I think that HARLIE has a moral sense—or at least he’s capable of developing one. I saw it today. He backed away from an option that had unacceptable consequences. If he’s put into this position—that he’s merely a machine—he’ll do his damnedest to prove that he isn’t. And . . . worse. If he realizes the trap that you’re postulating, he could . . . I don’t know what he would do. Malevolence? Maybe. He can conceive of it, I know that.”

  “You realize, of course, Aubie, that you’re only giving me more good reasons to—”

  “I know, I know. I suppose I should keep my mouth shut. But the day that we stop talking honestly to each other, we’ve all lost. We’ve lost whatever it means to be human and alive and caring. Guess what, that’s something I’ve learned from HARLIE already.”

  “Mm,” said Dorne.

  “If we have to do this,” said Auberson, “it’ll skew his perceptions and his ability to make appropriate responses to situations. When he realizes the arbitrariness of this limit, he’ll go crazy. He could—” Yes, he could, couldn’t he? It’s technologically possible. It’s emotionally possible.

  “He could what?”

  “He might even—I hate to say it—but he might suicide.”

  “Auberson.” Dorne put the cigar down again and leaned forward across his desk. “Are you trying to convince me to shut him down today? Because you’re doing a very good job of it.”

  “No. I’m trying to convince you to leave him alone. To let him devel
op without artificial constraints. At least, not these. I’m telling you that the consequences of this course of action will effectively neutralize everything we’ve already accomplished. If you’re not willing to give him space to grow, then maybe the kindest thing would be to shut him down. And damn me for saying so, because everything I’m seeing down there indicates that the scope of our breakthrough is not only greater than we’ve imagined, it may even be greater than our ability to imagine.”

  “Yes,” said Dorne. “Well.” He cleared his throat and folded his hands in front of him. “Hmf. I happen to believe you’re right. Believe it or not. And besides, I’m not willing to shut down a project the size of HARLIE just because a nervous-nellie lawyer starts getting the twitters. Nor would I shut it down because my project chief shows all the signs of becoming a goddamned visionary. I’d only shut it down if I were convinced it were a rathole. And I’m not convinced. Not yet.”

  “I think . . . I should say thank you, shouldn’t I?”

  “Don’t bother. That’s not the point. This is not a pep talk, Aubie. I did not invite you up here to rev you up and send you back downstairs to be more productive next month. I don’t do that. If I have to do it, I’ve hired the wrong people. The purpose of this meeting is to tell you that the jury is still out on HARLIE. Mmpf. Yes. For my part, I’d rather see us win this one than lose. But just in case—just in case we have to bite the bullet, I think we all had better be prepared to cover our asses with paper. If you know what I mean.”

  Auberson’s eyes were starting to water from the smoke in the air. He needed an excuse to get away from it; he got up from the chair and crossed to the side of the room, to Dorne’s wet bar. He opened a bottle of mineral water and poured it slowly into a glass, listening to the soft limey fizz. He looked to Dorne, questioningly. Dorne shook his head.

  “What is it you want from me?” asked Auberson. “I mean, what is it, exactly?”

  Dorne leaned back in his chair and swiveled to face Auberson. “A corporation is a legal individual, Aubie. In the eyes of the law, a corporation is a person. And what is a corporation? Nothing more than an agreement on a piece of paper. Nothing more. Compare that with HARLIE. It wouldn’t be that hard to prove that the thing is alive, would it?”