In the 1930, people like Kurt Goedel, Alonzo Church, and Alan Turing had sketched out mechanisms that could automatically synthesize logical formulas that were implied by existing formulas. Such machinery is the ultimate fake - there is no mystery in it at all. Already in the 1930s these pioneers of mathematical logic had explored both the power of such automated inference mechanisms and discovered some limitations. But even human intelligence is evidently limited! Could a machine be clever enough to deserve to be called "intelligent?"
It could be a bit like asking whether synthetic diamonds are really diamonds - maybe they're just too perfect! We might want to distinguish between natural and artificial intelligence, but still accept that they share the essence of intelligence, at least if we can find a way to detect it.
Turing proposed his famous test to answer this question. Nowadays "teletype" conversation has become pervasive - text messaging by email, cell phone, or whatever other channel. What if one enagaged in a text dialogue with someone, only to discover that they were in fact a machine? That's the Turing test - if a machine could carry on a text dialog in a fashion indistinguishable from a human, then the machine deserves to be called "intelligent".
One curious feature of our world today is that much of our text dialog is in fact carried on by machines - spambots, etc. The need for an effective way to distinguish between real people and machines has driven the construction of automated tests that require users to recognize squiggly letters. Turing's test involved a human judge, rather than an automated test.
But is simulated intelligence really the same as the genuine article? John Searle argued against this. The central processing unit, or CPU, inside a computer performs complex computations by reading a sequence of instructions from the computer's memory and executing the operations specified by those instructions, one by one. Each individual computation is very simple. The results of each operation become the inputs to other operations, and can also cause the reading of the instruction sequence to jump to a different place in memory - it's this pattern of interaction between operations that lets computers perform such sophisticated computations.
The CPU itself, though, is really blind to all that sophistication - it just keeps reading, interpreting, and performing simple operations one by one. A human being could just as easily perform those operations, albeit much more slowly. A human could simulate a computer! The simulated computer could be carrying on a text dialogue in the Chinese language, but the person who is simulating the CPU might well not know any Chinese at all. If a human can simulate intelligence with no comprehension at all of what is being simulated, Searle argues that simulated intelligence must therefore be different than the real thing.
Is Searle right? The question is in large part semantic. Useful distinctions make important difference. What difference might it make whether something is intelligent or not? Suppose someone destroys that possibly intelligent something - what sorts of legal penalties might be appropriate? Destruction of a mere machine, that would qualify as a misdemeanor like vandalism and call for a fine or maybe a night or two in jail. Destruction of an intelligent being, that would be murder and require severe penalities.
Suppose someone destroys the CPU in my computer. Generally a CPU in a computer can easily be replaced. These are generic components. Destruction of a CPU is not a serious matter.
Suppose instead someone destroys the hard disk in my computer! That's a whole different affair! With luck, maybe I have my most crucial data backed up to CD-ROM or some such media. Disks are so big, practically nobody can afford to maintain up-to-date full disk image back-ups. Loss of a hard disk is really a major catastrophe.
Whatever intelligence is in a computer is clearly not in the CPU. Searle's argument fails, because he is simulating the wrong component. He should be simulating the hard drive - as if such a thing were possible! It's a bit like saying there might be intelligence in a large library filled with rare and unique books. Not so strange, after all!