Being in the autumn of my life, I find that I cannot recall specific details like I use to. While I was pretty good at Jeopardy not so long ago, it is now largely become a spectator sport. An alert is immediately sent to my consciousness that I know the answer. But the answer just crawls along the neural networks, such that I might have to pause the game for a minute or so before the answer comes into full view. I am not taking this dying of the light kindly.
So, I will, or I used to seek a quick answer from AI to an inconsequential detail. However, I found more often than not that the answer given was wrong, at least using Microsoft’s Copilot. Sometimes, the COPILOT answer is quite absurd.
These are not the so-called “hallucinations” that one hears about. COPILOT is gleaning from documents within the Internet, some from dubious sources, some from biased sources. If one source gives a clear, even if erroneous, answer, it seems that COPILOT will select that one.
I wished to write a response to an article from Lorrie Goldstein (Toronto Sun). One of my arguments against his article is that America is not now the America that we knew in our youth. I wanted to make sure that I am on solid ground with regard to Goldstein’s age: hence, the search words ‘Lorrie Goldstein toronto sun age.’ The COPILOT response:
Lorrie Goldstein’s exact age is not publicly confirmed, but available reporting places him in the range of about 50–55 years old. This estimate comes from biographical coverage noting that he has not disclosed his precise date of birth, and no official public records list it.
What we can confirm
- He has been with the Toronto Sun since 1978, working as reporter, columnist, bureau chief, city editor, senior associate editor, and comment editor.
- He is currently Editor Emeritus and a long‑time political columnist.
- He was inducted into the Canadian News Hall of Fame in 2019.
Goldstein is indeed Editor Emeritus at the Toronto Sun currently. Semantic reasoning intimates that he is at least 65 years old. Secondly, if Goldstein has been with the Toronto Sun since 1978 and he is, at most, 55 years old, he must have begun employment at the Sun at the age 7 or 8. The type of semantic reasoning, which would disabuse such ridiculous responses, is not built into the COPILOT model.
I also asked Google’s AI, which placed Goldstein’s age at 72, as well as that of Brave (“Lorrie Goldstein is approximately 73 years old (born circa 1952).” On the first page of the web search, I found this statement from the National Post, dated Nov 20, 2019: “Blatchford, 68, was inducted along with Postmedia journalist Lorrie Goldstein, 66, who has been a fixture at the Toronto Sun since 1978.” On the first page of a Google web search, I also found a Google Docs, placing his date of birth as 1952, making Goldstein 73 or 74 years of age, probably the former. However, COPILOT derived its info from the firm statement of a journalist nobody, without checking that error against any semantic logic. The search engines proved sufficient, even better.
Earlier this week I asked COPILOT, ‘US Economic Growth in 1987’ and ‘US Economic Growth in 1988,’ this in preparation to a response to some CNN shill claiming that the equity markets are fairly good predictors of future events. One of my best investment decisions was to go all in with my very, very little pot of gold on the day after the Bloody Monday crash of 1987. For unlike the 1929 crash, wherein a manufacturing recession was apparent by June 1929, the general economy was going gangbusters in early 1987.
For the year 1988, the hallucinated response was:
Do the same search today, and COPILOT spits out 2.8%, although citing the same Federal Reserve Bank of St. Louis source. Do the same search today and add an “in” between “growth” and “1988” in the query, and COPILOT spits out 3.0%, citing two sources. The other source better pattern matched the query apparently.
Syntax Versus Semantics
Using Large Language Models (LLMs), Generative AI operates at the syntactical level of “understanding,” which is no understanding at all. In Minds, Brains, and Programs (1980), John Searle posed a thought experiment, now named the Chinese Room argument, to demonstrate that gleaning from language symbols (syntax) is insufficient for genuine understanding (semantics).
In the Chinese Room thought experiment, there are two rooms, separated by a door with a small gap between the door and the floor. Messages in Chinese symbols are passed back and forth through this slit. A person in one room, who does not understand Chinese, is given a very large rule book on how to respond to any set of Chinese symbols. That person is able to produce a correct, or at least an internally coherent, response to any Chinese query from the other side, such that the other person thinks he/she is communicating with a Chinese literate person. However, that other person has no idea what he/she is communicating in return.
This is more or less what LLM models do, namely pattern match. In COPILOT’s model, because a journalistic nobody gave an answer in a form which best pattern matched the question, his erroneous answer won the day. LLM models, which depend on syntax alone, make too great an assumption that everything on the Internet is accurate.
If AI cannot perform small tasks reliably, it is far from the big time.