Is the


  70-year-old Turing Test

dedicated to making machines think like humans but ignoring other abilities

is an antiquation?

  The result of the machine depends on the instructions we set, but its execution process is more efficient.

We must admit that many intermediate states of the machine are not foreseen when designing the initial instructions.

The machine itself will feel a lot of knowledge.

In this case, it is necessary for us to treat the machine as intelligent.

  Alan Turing

  British mathematician and logician

  In 1950, Alan Turing published his famous paper "Computer Machines and Intelligence" in "Mind" magazine, and proposed the Turing Test, which is now widely known.

  For 70 years, the Turing Test has been regarded as the "Polaris" of artificial intelligence academia.

With the development of artificial intelligence technology, many other tests have been born, but none of them can be as famous.

"The Turing test exhibits extreme simplicity and elegance, which has made it prosperous for the past 70 years." said Zach Meyer, Vice President of Data Science at DataRobot.

  Although since the birth of the Turing test concept, artificial intelligence has been aiming to pass the Turing test, but entering the new era, artificial intelligence evaluation standards need to be upgraded from the old benchmarks.

  On December 28, 2020, Amazon vice president and voice assistant Alexa chief scientist Rohit Prasad stated in an article published on Fast Company that the Turing test has lost its meaning and it is time to build a new artificial intelligence The measurement standard.

The Turing test is limited to whether the machine can give a human-like answer

"Can the machine think?"

  In order to answer this question, Alan Turing proposed a test method: if a tester asks the same series of questions to two objects (one person, one machine) that cannot be identified, the answers he gets make it impossible to distinguish who is exactly. If it is a machine and who is a human, then the machine is deemed to pass the test.

  This test method was later summarized as the Turing test.

Researchers hope to be able to detect whether the machine can show behaviors that cannot be distinguished by humans. Many early artificial intelligence assistants are designed based on this goal.

  MIT professor David Mindel said: "With this definition, the displayed wisdom is limited."

  Turing predicted in his paper that by 2000, the probability that an ordinary person can correctly distinguish between humans and machines in the Turing test will drop to 70% or even lower.

  However, Turing's prediction that year did not come true.

  Prasad believes that the goals of the Turing Test are not exactly the same as the current research direction of artificial intelligence, and artificial intelligence researchers are not interested in passing the Turing Test.

Where artificial intelligence comes in to a greater use is implanted in mobile phones, cars, and homes. People are more concerned about the updated interactive experience and technological advancements that AI can bring, rather than how high the scores of passing tests are.

  The fact is also true, people are more concerned about the interaction with the machine and the help it can provide, rather than distinguishing between machines and humans.

  In addition, some scientists have found that it is not difficult for artificial intelligence to achieve better results in the Turing Test. It is only necessary to make the answers given by the computer resemble the answers given by humans as much as possible.

For example, when answering a question designed by the Turing test, the computer can give an answer instantly, while ordinary people need to think or find information for longer. In order to imitate or deceive humans, the machine can also imitate humans and give appropriate pauses and delays.

  From a certain perspective, such a Turing test is more like a game of artificial intelligence "deceiving" humans.

However, a prominent problem emerged from this-in order to pass the test, many machines were forced to weaken the ability to quickly find information and calculate.

  The ability of machines to quickly calculate and query information is far stronger than that of humans, and these capabilities form the core of modern artificial intelligence.

In fields such as vision and natural language processing, the strongest algorithms have achieved results far surpassing humans. The major advances in artificial intelligence represented by AlphaGo defeating top human Go players are difficult to be reflected in the immutable Turing test. .

  For this reason, from an application point of view, it is absolutely unnecessary for computers to give up their advantages to simulate humans.

  More importantly, the Turing test only considers the situation of text communication, but does not consider that the current artificial intelligence has been able to use various sensors to experience the external world from multiple angles such as vision, hearing, and touch.

Passing the Turing test is no longer the current research focus

  There is no doubt that the impact of artificial intelligence on human society has exceeded the scope of the Turing test. The goal of artificial intelligence research is no longer limited to the difference between AI and humans, but how to use the speed and information search advantages of machines. Replace humans to complete work or improve people’s daily lives.

  The Turing test is used to test the level of artificial intelligence today. There are still some limitations that cannot be ignored. The Turing test has no detailed standards and no fixed question and answer mode. A set of process questions and judgments are very subjective and lack rigorous standards. science.

  So, does this mean that the Turing test is out of date?

  This is not the case. Even though the Turing test cannot fully confirm the progress of artificial intelligence, an excellent artificial intelligence should be able to pass the Turing test.

Some researchers pointed out that the ingenuity of the Turing test is that it does not directly define what “intelligence” is, but instead introduces the abstract question of “can you think” into a more accurate and seemingly more practical scenario.

  From this perspective, the Turing test cannot be called outdated, but modern artificial intelligence research should not focus on passing the Turing test.

Prasad also pointed out that, despite not taking into account the increasing data collection and computing power of artificial intelligence, the Turing test is still a common benchmark for chatbots and digital assistants.

 Artificial intelligence needs to establish a new set of metrics

  Prasad believes that new intelligent evaluation methods should be created, which are suitable for evaluating general types of intelligent machines.

The new test should figure out how artificial intelligence exhibits human-like intelligence characteristics, including language ability, self-supervision, and "common sense."

In addition, the scope of the test should also include the extent to which artificial intelligence has improved people's daily lives.

  Professor Dai Qionghai, academician of the Chinese Academy of Engineering and dean of the School of Information of Tsinghua University, also pointed out in a public speech that artificial intelligence is developing very fast and has replaced most of the tools commonly used by humans in the past.

However, whether this replacement can be done better requires a set of tests.

  Even for the artificial intelligence dialogue system that is most closely tied to the Turing test, its researchers are calling for improvements to the Turing test.

  Former Microsoft Global Executive Vice President and Dean of Microsoft Asia Research Institute Shen Xiangyang pointed out when he was still at Microsoft that the Turing test is no longer difficult for emotional artificial intelligence products like Microsoft Xiaoice.

In view of today's artificial intelligence technology environment, it is necessary for the computer academia to revise and upgrade the Turing test. It is time to discuss the more difficult "super Turing test".

  Prasad emphasized that the new measurement standards should reflect the advantages of machines in efficiency, such as computing, searching, and completing tasks on behalf of humans, and comprehensively evaluate the help that artificial intelligence brings to humans, instead of obsessing about eliminating artificial intelligence and The difference between people.

He believes that artificial intelligence can become an expert in handling a large number of tasks only if it has a broader learning ability. The intelligence shown for a specific task does not represent the true ability of artificial intelligence.

  With the advancement of artificial intelligence technology and more applications in real life, people have more expectations for artificial intelligence to improve their lives, while at the same time they have become more vigilant about the use and even abuse of artificial intelligence.

  In this regard, the industry is gradually reaching a consensus that the new artificial intelligence measurement standards should be scrupulous at the ethical level, rather than rigidly comply with the Turing test standards.

  It is undeniable that researchers are still interested in more powerful human-like intelligence problems, and the public is increasingly influenced by the future world shown by science fiction movies and televisions, and yearns for more powerful "general artificial intelligence", that is, like people. A machine that thinks and can do a variety of tasks like humans.

  Dai Qionghai proposed that the new generation of Turing tests should move from dedicated intelligence to general intelligence, and a new test direction should be given to the goals and requirements of the new generation of artificial intelligence.

The demand for artificial intelligence is always changing. In the process of constantly reconsidering the design of new evaluation standards and systems, it has become an inevitable result for humans to cross the Turing test. The imagination of artificial intelligence will never fade away.