MJD’s “Turing Test”

One of my first websites was a collection of stories from around what was then the much-smaller World Wide Web. Some of my favorites were by future Perl guru and Dischordian Mark Jason Dominus. I’m reprinting a couple of my favorites (by permission of the author).


Sandy Miller was very pleased with himself and with Hubie. Not that the 
whole team wasn’t, but Sandy is the kind of person who gets very puffed up 
and pleased about anything he does, and tends to blow it out of proportion. 
After four years of hard work, he had his baby Hubie, a machine and a 
program that he thought might pass the Turing test. I won’t bore you with 
long-winded explanations of the knowledge-representation schemes that 
Siobhan Murphy had developed, or descriptions of the complex, dynamically 
self-reconfiguring hardware that Max Jameson and Hsien-Hui Wu put together. 
I’ll just say that Sandy thought we had a machine that had a good shot at 
winning the twelve million dollar prize offered by the Turing Institute, and 
that I thought we had a machine that ran the world’s most sophisticated 
version of ‘eliza’, and leave it at that.



Siobhan and I were in the lab, late evening, and we’d been ‘talking’ to 
Hubie, trying to pound on him linguistically and see what parts weren’t 
working. We’d hit him with some classic examples:



> Ok, Hubie, we’re going to tell you a story.



ALL OF YOU AT ONCE? 


> 



That was rather startling; I was expecting bad mistakes, but none quite as 
stupid as that; we weren’t even trying to confuse him yet. It wasn’t a good 
start for the evening. I called to Siobhan, who was watching the diagnostic 
screens—Hubie’s unconscious mind, maybe it helps to think of it that way. 
”Siobhan, was that a joke, or what?” 



Siobhan studied the screens and tapped some commands. She peered into the 
nasty twiny details of Hubie’s parsing process. “Nope,” she said at last. 
”He took ‘we’ too literally.” 



Damn, I thought. I’ll have to track that one down and fix it. I reached over 
to the hardcopy and made a red mark in the margin. 



> I mean that I am going to tell you a story.



OK. FIRE AWAY.



> A goat wandered into the yard where Jack was 
painting. The goat got paint all over himself. 
When Mother saw the goat, she asked, “Jack, did 
you do that?” 



> 



There was a pause. 



> Okay? 



YEAH? SO WHAT? 



>



Siobhan chuckled. I shrugged. That kind of crap is just parlor tricks. 



> So what I want you to tell me is what the 
antecedent of the word “that” is, that’s so 
what. 



I’M NOT SURE.



> 

Goddamn pox on a black rotting corpse of a shipwrecked ship rat! 



> Please try to guess. It is very important to 
us. 



There was a long pause. Too long. If Hubie had to think so hard about it, 
that meant he was constructing some wildly complex and absurd answer. Sure 
enough, when Hubie finally made up his mind, he said: 



MAYBE THE GOAT MADE A MESS ON THE FLOOR?



>



I must have looked pretty mad, because Siobhan got up and came over to me.


”What went wrong?” she asked. 

”Lookit,” I said. “I mean, it’s a pretty tricky problem—‘that’ doesn’t 
refer to anything in particular, it sort of refers to the action of Jack 
making the goat all sticky. But Hubie didn’t know. And he made this absurd 
guess. You can tell exactly what was going on, too. He didn’t know, so he 
looked up his ‘goat’ frame and hunted for a thing a goat might do that would 
get Mom to utter an exclamation, and decided that maybe the goddamn goat 
shat on the goddamn floor. No human in the world would make an absurd random 
guess like that.” 



Siobhan chewed her lip, then went back to the diagnostic outputs. I glared 
at the screen. We sat there a while in silence, while Siobhan bit her nails 
and studied Hubie’s reasoning process. 

Finally she said, “It wasn’t a big disaster. You were right: Hubie looked 
in his ‘goat’ frame and traced cross-references until he found the chain 
’goat’, then ‘partly-domestic’, so then ‘partly-housebroken’, then that the 
presence of excrement is undesireable, then that authorities blame their 
inferiors for undesirable conditions, and a few other things. It wasn’t a 
bad job, all in all.” 



“Yeah,” I said. “Great. Max sure did a wonderful job. Hubie would never 
have found that absurd explanation if not for his brilliant hardware. He is 
thinking very fast indeed. And very badly. Can’t you just see it? Goat shits 
on floor, mom sees shit, sees goat, says ‘Jack, did you shit on the floor?’ 
Damn, pox, tripe, scum, nonsense.” I glared back at the terminal. “And 
look at that! We haven’t told Hubie anything for twenty minutes, and he’s 
just sitting there, happy as a clam. Why the hell doesn’t he ask us if he 
got the right answer or not?” 



“Quit steaming,” said Siobhan. “We’re not finished with him, that’s all. 
Look, he wasn’t confused by superfluous ‘so whats’ in your question.” 



“Yeah,” I said. “Sorry. I’m too on edge. I better—” 

Then Sandy rushed in. He was grinning and his tie was all loose.

“Guess 
what, folks?” he cried. Nobody said anything. “Guess what?” he repeated, 
not noticing. 



“What?” asked Siobhan at last. 

”I’ve finished talking to the Turing Institute. They’re going to send an 
examiner around a week from tomorrow to give Hubie the B test. ” 



I didn’t say anything. Siobhan bit her lip, and chuckled nervously. 



“Well?” asked Sandy. “What’s the matter?” 



“You didn’t come at a very good time, Sandy.” explained Siobhan. Sandy 
looked at me.



For the last twenty years, Turing Institute has been examining AI claims and 
offering the big prize to the first machine that can fool their examiner.



I guess I should describe the Test, because not everyone knows how it works. 
Alan Turing was maybe the first person to recognize the centrality of 
communication in intelligence, and he proposed the Test, way back in the 
1930’s. You play a game; there are two rooms, one with a man and one with a 
woman, and a terminal in each room. The man and the woman talk through the 
terminals to an examiner in a third room; each is trying to convince the 
examiner that they’re the woman. Whoever does, wins. 



Now if you want to see if a machine is intelligent, you replace the man with 
a machine, and you see if the machine can win as often as the man did. If it 
can, it’s probably intelligent—it can communicate in the same way people 
can, it can understand complex and subtle points about the way language and 
information and the real world work, and so on. There’s a whole lot of 
philosophy you can learn that underlies the Test, and it seems to do fairly 
well in practice. If you want to see whether the machine really understands 
the kind of things that people understand, what better way than by just 
giving it an oral quiz? 



Turing’s Test that I just described is what Turing Institute calls the ‘A 
Test’; it’s the hard one. The simpler version, the B test, involves just one 
run, with an examiner, a human, and a machine, and the machine tries to 
confuse the examiner about who’s human and who’s the machine. The B test is 
quicker and simpler, and so you usually want to try to pass that one first; 
certainly no machine can pass the A test without being able to pass the B 
test. Of course, the A prize is bigger.



”I don’t know why you’re bothering with the Test, Sandy,” I said. 
”There’s not a prayer that Hubie can pass. It’s a big waste of time.” 



“I think you underestimate our chances, Karl.”



”I don’t.”



”We’ve spent four years and hundreds of thousands of dollars on Hubie.”



I was trying hard not to lose my temper. Sandy is a nice guy to go to a 
restaurant with, and he’s a cognitive sciences god and all, but his 
unrestrained enthusiasm and naivete were the most irritating thing I could 
have imagined at that particular moment. “So?” I snapped. “You think the 
universe owes you intelligence or double your money back? ‘If not completely 
satisfied, your wasted time will be refunded?’ ” 



“He may not be intelligent, but I think he can fool an examiner.” 



“It can’t fool jackshit! It takes intelligence to convince a human into 
thinking you’re another human; that’s the whole point of the Test. You can’t 
fake intelligence. Hubie certainly doesn’t fool me.” 



“Yes, but you know all of Hubie’s weak points. And examiner might not know 
where to look.” 



“Look,” I said. “Look. Look here, at this.” I showed him what Hubie had 
said about the goat.



”I think Hubie did very well,” protested Sandy. “Maybe he shouldn’t have 
made the mistake in the first place, but I do think he recovered very well. 
That was quite plausible.” 



“Yes,” I said, in my most nasal and sarcastic voice. “It certainly was 
more plausible than guessing that perhaps a piece of green cheese had fallen 
off the moon and crashed through the ceiling, or that the goat had a 
pustulent goiter because Jack forgot to feed it its iodine ration, or 
something, but people do not spout non-sequiturs when they don’t know the 
answer.” 



“I don’t think there’s any reason to be nasty,” said Sandy. “You did tell 
him that it was very important, so naturally he said something, even if he 
thought it wasn’t very plausible.” 



That was true, but then this was hardly the first blunder Hubie had made. I 
tried again. “You’re wasting your time,” I repeated. “The hardware is 
worthwhile, even for plain computing. The side-effect resolution routines in 
the situation frames are very clever; Siobhan did a brilliant job on them, 
and that made it really easy for me to dump a whole lot of miscellaneous 
information into the machine. Hubie is making good use of at least some of 
the right information at the right time. But it’s not enough. The spark, or 
the quantum whatever, just isn’t there, and I know it and you know it.” 



“Maybe the examiner won’t know it,” grinned Sandy. 



I shrugged. “All right,” I sighed. “It pisses me off that you’re tossing 
our test chance down the toilet, but I see I can’t stop you. Just, if I were 
you, I wouldn’t publicize the examination too widely.”



”There’s something I didn’t tell you,” said Sandy. “I’m going to be the 
control.” He meant that he would be the person in the other room, trying to 
prove that Hubie was the machine and that he wasn’t. Of course, he didn’t 
have to try very hard, and the lazier he was, the better Hubie’s chances of 
winning the prize.



I nearly dropped my teeth. “What? How did you get them to agree to that ?” 

What had happened was this: Old man Johann Arndtfeld, chairman of Turing’s 
prize committee, (he wouldn’t use the word ‘chair’) didn’t like Sandy, and 
this wasn’t the first time that Sandy had come in with wild and fantastic 
claims about his new fabulous AI. So he cut Sandy a deal: Since Sandy was so 
sure that Hubie would win this time, Arndtfeld would send his most astute 
examiner and would let Sandy be the control, on condition that if Sandy 
lost, he’d never bother Turing Institute again. It was a pretty stupid thing 
for either of them to agree to, because even if Sandy did win, it wouldn’t 
be a real victory for him, because of the rule change. But sometimes we get 
so wrapped up in the path to the goal that we forget what the goal actually 
is; I think that was what happened to Sandy the day he met with Arndtfeld. I 
wasn’t at the meeting, but I suspect that tempers were rather high all 
around. 



With Sandy as the control, it seemed like we might have a chance after all, 
because it’s much easier for a human to act like a dumb computer and fool an 
examiner than it is for a dumb computer to act like a human and fool an 
examiner. It certainly would have been nice to get the prize money, so I 
shut up and started working around the clock to iron as many bugs out of 
Hubie as I could before the big day. 



Test Day came, and Sandy and I ran Hubie through a few conversation tests to 
warm him up. 



> Hello, Hubie. This is Sandy. 



GOOD MORNING, DR. MILLER.



> How are you feeling today, Hubie? 



VERY WELL, THANK YOU. HOW ARE YOU?



> I am very excited. Do you know what today is?



AUGUST 27TH.



>



I rolled my eyes. Sandy looked dismayed for a few moments, and then said 
”Well, no, that was reasonable.” 



“If you feel the need to defend it, Sandy, then it wasn’t reasonable.” 

Sandy tried again. 




> Today is the day the examiner from the Turing 
Institute is coming to see if you are 
intelligent, Hubie. 



WHY, THAT’S SILLY. OF COURSE I’M INTELLIGENT. 



> Of course you are. We want you to try as hard 
as you can to prove it.



RIGHT. WILL DO.



> 



“You’ve described the test to him?” asked Sandy. 

I nodded.



> The examiner will administer Turing’s Test to 
you, Hubie. Do you know what that is?



TURING’S TEST IS THE SAME AS THE “TURING TEST?” 



> Yes, Hubie. 



YES, I KNOW THAT TEST. WHO WILL BE IN THE OTHER ROOM?



>



That was pretty good. “Hey, Siobhan,” I said. “Hubie wants to know who 
will be in the other room.” 



“Yes, I see,” she said. “Not bad.” 



> I will be in the other room, Hubie. 



DOESN’T THAT BIAS THE TEST IN YOUR FAVOR?



> 



Better and better. 



> Yes, but Turing agreed to that condition last 
week. Are you ready, Hubie?



”Hubie correctly identified ‘Turing’ as being the Turing Institute, not 
Alan Turing,” called Siobhan, from across the room. 



“Cool,” I said. “Can you trace the reasoning process?” 

”Just a second, let me look…um…’Alan Turing died in 1954′, ‘dead people 
stay dead’, specification to ‘Turing stayed dead after 1954’, last week is 
after 1954, causality to ‘Turing was still dead as of last week’, then 
’agree’ isn’t permissible for dead people, so by specification ‘Turing 
instantiation of last week didn’t agree to anything’, equivalence to ‘Turing 
didn’t agree to anything last week’, specification to ‘Turing didn’t agree 
to rule changes last week’, so contradiction, so by Gricean inference, it 
must have been the other one.” 



“Right answer, but the wrong reason, dammit. I’d really like to back that 
up and try it again without including the ‘last week’ part—I bet he screws 
it up. Well, maybe tomorrow” 

Sandy and I looked at Hubie’s response. 



YES, I AM READY. WHY DO YOU KEEP CALLING ME ‘HUBIE’? I 
KNOW WHAT MY NAME IS ALREADY. 



> That is just a convention of the way we speak.



NO REASON AT ALL THEN. 



“You better tell him to shut up. It’s good that he’s asking questions like 
that, but they’re a sure giveaway to the examiner.”



> That’s right. Hubie, for the next four hours, 
please do not speak except to answer a direct 
question. Do you understand?



YES, DR. MILLER.



We left Hubie to wait, idle, until the Turing examiner arrived. I never know 
what to expect from Turing. I wish I knew where they found the pool of 
mutant weirdoes that they use for examiners. 



We waited outside and a cab pulled up. Arndtfeld and the examiner stepped 
out onto the curb. Arndtfeld was tall and lean, with a creased yellow face 
and a bowler hat. 



“Well, Dr. Miller, I suppose this is your big day. How do you feel?” He 
smiled down at us; it wasn’t a very nice smile, and his teeth were yellow 
too. 



“Confident, Dr. Arndtfeld,” said Sandy. “Very confident.”



”Let’s hope you’re not as overconfident as you were the other times, then, 
eh? It would be a shame if we never heard from you again.” 

Sandy bristled, but didn’t say anything. 



“This is Hosai,” continued Dr. Arndtfeld. “He is one of our very best 
examiners. An excellent judge of humanity.”



Hosai was wearing a long robe, and his head was shaved. He carried a strange 
kind of bag or rucksack in one hand, and an ugly wooden staff in the other. 
We said hello, and followed Arndtfeld and Hosai inside.

“He looks like a 
Buddhist monk,” I whispered to Sandy. “Why would a Buddhist monk be 
examining machine intelligences for Turing?” 



“Zen Buddhist, actually,” said Sandy, ignoring my question. He didn’t know 
either. 

We’d prepared two rooms, one for Sandy and one for the examiner. I felt 
uncomfortable as I took Hosai to the little bare room with the two 
terminals. 

Hosai smiled at me.

“May I ask that you put the terminals on the floor?” 

I nodded and moved the terminals onto the floor, carried the furniture out 
of the room. Hosai was unrolling a bamboo mat that he took from his bag.



”May I stay and watch?” I inquired. It’s always more interesting to watch 
the examiners than it is to watch Sandy, and I can watch Hubie just as well 
next week or next month by reading through the postmortem of the Test run. 
Examiners are such a queer bunch, you learn a lot about the fringes of human 
communication and mental behavior by watching them closely and thinking 
hard. 



“Why certainly,” replied Hosai, kneeling on the mat and putting on a pair 
of delicate steel-rimmed spectacles. “But I must ask that you remain 
completely silent at all times. It would not do to break my concentration.” 



I agreed. Just then Hsien-Hui, one of our hardware wizards, appeared at the 
door. 

”Sandy is ready,” he announced. “Everything is ready.” 



“Very well,” said Hosai. “You may tell Hubie and Dr. Miller that we are 
beginning.” Hsien-Hui vanished again. Hosai shifted a little on his mat, 
and crossed his hands and half-closed his eyes. I stood in the corner, 
waiting for him to begin. Several minutes went by. 



“Um,” I said, “I think you can begin now.” 



Hosai turned his head and fixed his eyes on me. “Did you not promise to 
remain completely silent?” 

I sat down, abashed. Hosai turned back to the terminals and half-closed his 
eyes again. He sat motionless for several more minutes. I tried not to 
fidget. 

After several more minutes, something happened. 

On Sandy’s terminal appeared the words 



IS SOMETHING WRONG? I’M NOT GETTING ANYTHING. 



> 



Hosai didn’t seem to notice. A few more minutes went by.



HELLO? HELLO? WHAT IS GOING ON?



> 



Still Hosai remained silent and impassive.



WHAT THE HELL ARE YOU WAITING FOR? START THE TEST!



>



My mouth was dry. Hosai didn’t move. Suddenly the door flew open and Sandy 
burst in. “What is this?” he cried. “Some kind of a joke? Begin the 
questioning!”



Hosai didn’t look around. He reached out and placed his left hand on Sandy’s 
terminal. “Man,” he said. Then he placed his right hand on Hubie’s 
terminal. “Machine,” he said. He stood up, rolled up his bamboo mat, 
picked up his staff. 



Sandy’s face was lead-colored and his jaw was slack. He stammered and when 
the blood finally came back into his face and he could speak again, he still 
wasn’t coherent: he called Hosai a cheater and a fraud; he threatened to sue 
Turing Institute; he foamed and babbled. Hosai looked at him with contempt, 
and then raised his staff and stamped the end on Sandy’s foot. Sandy cried 
out in pain and was silent. He stared at Hosai.



Hosai raised his finger to his lips and said nothing. In that moment, Sandy 
was enlightened.


mjd@plover.com

Leave a Reply

Your email address will not be published. Required fields are marked *