Like most people, I find that artificial intelligence is creeping into my life in many ways. Increasingly, people I deal with disagree with me because their favorite AI chatbot tells them I am mistaken. AI can easily demean your self-confidence if you let it.
Some clients tell me that their chatbot questions my tax advice. I typically find that, when this happens, my client did not give the chatbot all of the relevant information. Sometimes, the chatbot hallucinates.
AI hallucinations occur when it generates false, misleading, or completely fabricated information, but presents it with total confidence as if it were a true fact.
I frequently drive through the intersection of Valencia Boulevard and McBean Parkway. Last week, when I drove by the Shell gas station located on the northwest corner of that intersection, I noticed that it was not selling gas. The electric price sign was turned off and there was yellow tape on each of the gas pumps.
I wondered why, so when I got home, I decided to ask Google’s chatbot for an answer.
My specific request was, “Why is the Shell gas station located at the corner of Valencia Boulevard and McBean Parkway in Santa Clarita, California, not selling gas today?”
The response was that the gas station was not selling gas on that day because, in August 2024, the Santa Clarita City Council ordered the gas station be shut down and the gas station was subsequently demolished.
When I responded that I did not believe that was the correct answer because the gas station was still standing, the chatbot apologized and “corrected” itself by telling me the station was closed because an oak tree on the premises was being removed. Again, I questioned whether that was a correct answer.
After repeating this process several additional times, the chatbot’s final response was that the gas station was not selling gas because of ongoing road construction work at the intersection and the city determined that gas station traffic would disrupt the road construction.
A few hours later, I drove by the Shell station and it was open for business as usual.
I think you get the point — artificial intelligence sometimes stubbornly hallucinates.
I use artificial intelligence professionally to assist in researching tax law and drafting memos documenting such research. I treat the AI-generated product as if it was produced by a first-year associate and thoroughly review the workproduct.
Although AI does a better job than some of the first-year associates who previously worked for me, it only gets about 85%-90% correct. Hallucinations cause a disturbingly high component of the errors — just like the responses concerning the Shell gas station.
Did the chatbot tell me what it thought I wanted to hear?
Ostensibly, generative AI is supposed to have guardrails to mitigate user biases when responding to inquiries. However, I am not certain about the guardrails’ effectiveness. I have experienced situations where my wife and I made identical inquiries of the same chatbot using different computers — each getting different results.
According to a study conducted by the University of Kansas, “AI chatbots are trained on an enormous amount of publicly available online text. As a result, they have the same biases as those sources and the society that produced them.”
For example, in my professional tax research, I have encountered situations where the chatbot cited an article written by another tax professional and based its answer on that professional’s conclusions.
Upon closer review, I discovered that professional’s interpretation of tax law was inconsistent with the underlying primary legal sources. When that happens, the chatbot’s response is inherently incorrect.
This problem is likely to compound as time passes when chatbots increasingly rely on the output of other chatbots — especially if the other chatbots are designed to manipulate information to coincide with the original user’s preconceived biases.
If so, it will become increasingly difficult to establish and maintain sufficient bias mitigation guardrails.
As a corollary, it is increasingly plausible that generative AI will be used to make propaganda appear credible.
When I watch YouTube, I am constantly directed to podcasts and other materials that appear believable, but when I step back and question what was said, I realize that the espoused position is not credible.
But, as the presentations become more sophisticated, it may become increasingly difficult to discern their level of credibility.
The moral of the story is not to blindly accept chatbots’ accuracy. Be skeptical while keeping an open mind. Remember, artificial intelligence is not the ultimate authority.
Jim de Bree is a Valencia resident.
