More (from Wiki) on how the Turing test is unsuitable for where we are with AI now
The Turing test does not directly test whether the computer behaves intelligently. It tests only whether the computer behaves like a human being. Since human behaviour and intelligent behaviour are not exactly the same thing, the test can fail to accurately measure intelligence in two ways:
Some human behaviour is unintelligent
The Turing test requires that the machine be able to execute allhuman behaviours, regardless of whether they are intelligent. It even tests for behaviours that may not be considered intelligent at all, such as the susceptibility to insults,[75] the temptation to lie or, simply, a high frequency of typing mistakes. If a machine cannot imitate these unintelligent behaviours in detail it fails the test.
This objection was raised by The Economist, in an article entitled “artificial stupidity” published shortly after the first Loebner Prize competition in 1992. The article noted that the first Loebner winner’s victory was due, at least in part, to its ability to “imitate human typing errors.”[51] Turing himself had suggested that programs add errors into their output, so as to be better “players” of the game.[76]
Some intelligent behaviour is inhuman
The Turing test does not test for highly intelligent behaviours, such as the ability to solve difficult problems or come up with original insights. In fact, it specifically requires deception on the part of the machine: if the machine is more intelligent than a human being it must deliberately avoid appearing too intelligent. If it were to solve a computational problem that is practically impossible for a human to solve, then the interrogator would know the program is not human, and the machine would fail the test.
Because it cannot measure intelligence that is beyond the ability of humans, the test cannot be used to build or evaluate systems that are more intelligent than humans. Because of this, several test alternatives that would be able to evaluate super-intelligent systems have been proposed.[77]
The Language-centric Objection(Editing Turing test - Wikipedia)
Another well known objection raised towards the Turing Test concerns its exclusive focus on the linguistic behaviour (i.e. it is only a “language-based” experiment, while all the other cognitive faculties are not tested). This drawback downsizes the role of other modality-specific “intelligent abilities” concerning human beings that the psychologist Howard Gardner, in his “multiple intelligence theory”, proposes to consider (verbal-linguistic abilities are only one of those). [78].
Consciousness vs. the simulation of consciousnessEdit
Main article: Chinese room
See also: Synthetic intelligence
The Turing test is concerned strictly with how the subject acts – the external behaviour of the machine. In this regard, it takes a behaviouristor functionalist approach to the study of the mind. The example of ELIZA suggests that a machine passing the test may be able to simulate human conversational behaviour by following a simple (but large) list of mechanical rules, without thinking or having a mind at all.
John Searle has argued that external behaviour cannot be used to determine if a machine is “actually” thinking or merely “simulating thinking.”[45] His Chinese roomargument is intended to show that, even if the Turing test is a good operational definition of intelligence, it may not indicate that the machine has a mind, consciousness, or intentionality. (Intentionality is a philosophical term for the power of thoughts to be “about” something.)
Turing anticipated this line of criticism in his original paper,[79]writing:
I do not wish to give the impression that I think there is no mystery about consciousness. There is, for instance, something of a paradox connected with any attempt to localise it. But I do not think these mysteries necessarily need to be solved before we can answer the question with which we are concerned in this paper.[80]
Naïveté of interrogators(Editing Turing test - Wikipedia)
In practice, the test’s results can easily be dominated not by the computer’s intelligence, but by the attitudes, skill, or naïveté of the questioner.
Turing does not specify the precise skills and knowledge required by the interrogator in his description of the test, but he did use the term “average interrogator”: “[the] average interrogator would not have more than 70 per cent chance of making the right identification after five minutes of questioning”.[81]
Chatterbot programs such as ELIZA have repeatedly fooled unsuspecting people into believing that they are communicating with human beings. In these cases, the “interrogators” are not even aware of the possibility that they are interacting with computers. To successfully appear human, there is no need for the machine to have any intelligence whatsoever and only a superficial resemblance to human behaviour is required.
Early Loebner Prize competitions used “unsophisticated” interrogators who were easily fooled by the machines.[52] Since 2004, the Loebner Prize organisers have deployed philosophers, computer scientists, and journalists among the interrogators. Nonetheless, some of these experts have been deceived by the machines.[82]
One interesting feature of the Turing test is the frequency of the confederate effect, when the confederate (tested) humans are misidentified by the interrogators as machines. It has been suggested that what interrogators expect as human responses is not necessarily typical of humans. As a result, some individuals can be categorised as machines. This can therefore work in favour of a competing machine. The humans are instructed to “act themselves”, but sometimes their answers are more like what the interrogator expects a machine to say.[83] This raises the question of how to ensure that the humans are motivated to “act human”.
Silence(Editing Turing test - Wikipedia)
A critical aspect of the Turing test is that a machine must give itself away as being a machine by its utterances. An interrogator must then make the “right identification” by correctly identifying the machine as being just that. If however a machine remains silent during a conversation, then it is not possible for an interrogator to accurately identify the machine other than by means of a calculated guess.[84]Even taking into account a parallel/hidden human as part of the test may not help the situation as humans can often be misidentified as being a machine.[85]
Impracticality and irrelevance: the Turing test and AI research
Mainstream AI researchers argue that trying to pass the Turing test is merely a distraction from more fruitful research.[53] Indeed, the Turing test is not an active focus of much academic or commercial effort—as Stuart Russell and Peter Norvigwrite: “AI researchers have devoted little attention to passing the Turing test.”[86] There are several reasons.
First, there are easier ways to test their programs. Most current research in AI-related fields is aimed at modest and specific goals, such as object recognition or logistics. To test the intelligence of the programs that solve these problems, AI researchers simply give them the task directly. Stuart Russell and Peter Norvig suggest an analogy with the history of flight: Planes are tested by how well they fly, not by comparing them to birds. “Aeronautical engineering texts,” they write, “do not define the goal of their field as ‘making machines that fly so exactly like pigeons that they can fool other pigeons.’”[86]
Second, creating lifelike simulations of human beings is a difficult problem on its own that does not need to be solved to achieve the basic goals of AI research. Believable human characters may be interesting in a work of art, a game, or a sophisticated user interface, but they are not part of the science of creating intelligent machines, that is, machines that solve problems using intelligence.
Turing did not intend for his idea to be used to test the intelligence of programs — he wanted to provide a clear and understandable example to aid in the discussion of the philosophy of artificial intelligence.[87] John McCarthy argues that we should not be surprised that a philosophical idea turns out to be useless for practical applications. He observes that the philosophy of AI is “unlikely to have any more effect on the practice of AI research than philosophy of science generally has on the practice of science.”[88]
