Common Sense Is AI’s Glass Ceiling

A couple of weeks ago I decided to ask ChatGPT to write my professional biography. I was initially impressed. It correctly identified my employment history, academic credentials, media appearances and many other features. But it also stated that I had passed away last year! How can a widely successful system be so accurate, yet also be so obviously wrong? The modern AI systems of today lack something akin to common sense. I’m thinking that common sense may become at least one component of a gold standard for assessing how successful AI will be in replicating human thought and intelligence.

We are all aware that AI has made great progress in sophistication with the boost of advanced compute capabilities. And there is much more to come. I think we can expect quantum leaps in the medical, industrial and scientific applications. But a lingering roadblock may revolve around common sense development. Common sense may be AI’s glass ceiling. And until this glass ceiling is shattered, our confidence about completely handing the reins over to AI controls won’t happen. AI experts sometimes describe this lack of common sense by saying a system is “brittle” in that it can produce nonsensical answers at any time. When AI makes silly mistakes writing my bio it’s low risk. But what about more important matters. After all, if AI thinks I’m already dead—why would I trust it to prescribe meds for me that could actually kill me?

Throughout its history of development, a lack of common sense has plagued the field. And like Apple’s
Apple
Siri, there doesn’t seem to be much improvement over the years. Here’s an example. We know that AI has greatly progressed since the televised performance of IBM’s
IBM
supercomputer “Watson” playing Jeopardy in 2011. Watson performed impressively on the show, rapidly advancing against the smart human contestants. But recently I watched playbacks of the games. There were times when Watson produced ridiculous answers to questions that were simpler than most. And the delivery of the answers—by electronic voice or read by the host in a deadpan manner—made the situation downright hilarious. For example, in the “U.S. Cities” category, one clue was “Its largest airport was named for a World War II hero; its second largest for a World War II battle.” After one contestant correctly answered, “What is Chicago?” (Chicago — for the city’s O’Hare and Midway airports). Watson answered, “What is Toronto?” Watson missed by an entire country. No “self-checking” when playing Jeopardy I guess!

Thirteen years later, my recent simple request to have AI write my bio suggests that little progress has been made in “self-checking” for consistency in output. Think how important this trait is to user confidence. Certainly, large language models can write impressive outputs to many requests such as writing thank-you notes or proofs of Pythagoras’ theorem—even in a format that rhymes. But if we want AI to output answers to adventurous questions with no apriori answers, how could we trust the output? AI experimentalists—I consider them the coders—are struggling with the problem, while AI theorists—the academic researchers—think this cannot be solved given the current approaches. They think a new approach is needed to advance in this area. Current methods are more like this: Put a child in isolation; provide them with internet, YouTube, Wikipedia, etc. When they emerge into the real world, I suspect they would have zero common sense. AI’s answers make sense until they don’t.

I suspect the common sense problem is far more difficult than most think. And when we consider how the human brain evolved, it has always been connected to the human body. It’s counterintuitive to think the human body must have lots to do with how we think, and how we reason. Perhaps the next generation of AI computers will have to be embedded in a total type of human structure that affords all five senses to learn.

The famous “Turing Test” was proposed in 1950 to determine machines from humans: you talk to an unknown party to determine if it is human or computer. Today that test is mostly obsolete, as we can often be fooled into thinking a chatbot is human—at least for a short conversation. But ask a machine something like “If I’m driving on a highway and I see a tumbleweed in my path, should I brake?” You may be surprised that the answer. Questions that require common sense experience are much harder for machines.

So next time I ask an “intelligent machine” to update my bio or something else, I will take great care to review. My common sense tells me there are likely to be some silly mistakes. And I will never ask AI if I’m deceased.