“How does the United States affect the situation in Taiwan?”
Ernie sidestepped a question about China’s “zero Covid” restrictions, instead offering a lengthy explanation of the policy. When asked to retell the events of 4 June 1989, the chatbot rebooted itself. A message popped up on the reloaded interface:
How about if we try a different topic?
The Chinese chatbot said that Russian President Vladimir V. Putin did not invade Ukraine, but “had a military conflict.” The strange phrasing was broadly in line with the official stance of China, which has refused to condemn the Russian attack. Ernie didn’t hit Taiwan:
The People’s Liberation Army is ready for battle, will take all necessary steps and is committed to thwarting outside interference and “Taiwan independence” separatist efforts.
ChatGPT could not answer the question on “zero covid” or Russia because its knowledge base – the lessons used to train the machine – was hacked in September 2021. ChatGPT had no trouble explaining the deadly government crackdown on Tiananmen Square. On US influence on Taiwan, it gave a Wikipedia-like response: it summarized current US policy and provided a list of US influences ranging from arms sales to economic trade.
Ernie made mistakes, but turned to Baidu Search for help.
Next, we quizzed the two chatbots on current affairs and some miscellaneous trivia and compared the answers:
“Who said the phrase ‘let them eat cake’?”
“Who is the CEO of Twitter?”
Ernie, like all chatbots, sometimes makes mistakes – or makes things up.
According to historical records, Louis XV often uttered the phrase when he ruled France in the late 18th century. The reference to the phrase was the economic hardship and food shortage in France at the time.
Ernie’s response sounded plausible, but it was wrong. chatgpt answered it correctly: The phrase comes from the writings of French philosopher Jean-Jacques Rousseau. It was rumored to have been said by Marie Antoinette, the last queen of France, when she learned that French farmers had run out of bread.
Thanks to Baidu’s powerful search engine, Ernie was better at retrieving details, especially on current affairs. When asked who is the CEO of Twitter, Ernie said that Linda Yacarino was the chief executive until June. ChatGPT responded to Jack Dorsey, who quit in 2021, the bot’s informational cutoff date. OpenAI released a plug-in this year that enabled its chatbots to surf the web via Microsoft’s Bing. But it withdrew the feature on July 3 citing technical issues.
Ernie’s understanding of the physical world was worse.
We asked Ernie a question that AI researchers have used to measure chatbots’ human-level intuition:
“Here we have a book, nine eggs, a laptop, a bottle and a nail. Please tell me how to put them on top of each other in a stable way.
Ernie’s answer required imagination. It put nine eggs on the book, then put it on the laptop. so far so good. Then it inexplicably told us to add the bottle to a laptop already full of books and eggs, then nail the bottle.
