[caption id="attachment_142250" align="aligncenter" width="3029"] A.I. assistants are actually pretty dumb.[/caption] Big tech companies want us all to embrace digital assistants in a big way. While users know this, it's rare that you find someone enthusing about the capabilities of Cortana, Siri, or whatever synthetic being lives on their smartphone (or home appliance). A new study shows us why. A research paper published by the Cornell University Library tells us our ‘smart’ assistants are actually pretty stupid. Conducted by Feng Liu, Yong Shi and Ying Liu, the study positions Google Assistant as the clear winner in the "smartness" competition, but even the search engine giant's take on A.I. still wasn't more intelligent than the average six-year-old human. The researchers developed a test in 2014 to determine the intelligence of a digital assistant, assigning it an "IQ score." Originally designed to measure the relative dangerousness of artificial intelligence (A.I.), the study “proposes a standard intelligence model that unifies AI and human characteristics in terms of four aspects of knowledge, i.e., input, output, mastery, and creation.” From the paper’s abstract:

Using this model, we observe three challenges, namely, expanding of the von Neumann architecture; testing and ranking the intelligence quotient of naturally and artificially intelligent systems, including humans, Google, Bing, Baidu, and Siri; and finally, the dividing of artificially intelligent systems into seven grades from robots to Google Brain. Based on this, we conclude that AlphaGo belongs to the third grade.

The von Neumann model is an architecture that essentially defines all hardware: It takes input, figures out what to do with that input, then returns some value. The Liu/Shi/Liu paper suggests the average IQ of a six-year-old is 55.5. Google’s assistant measured 47.28. That seems pretty dumb, but it blew away the competition. Cortana scored 31.98. Baidu’s assistant was on par with Cortana, scoring a 32.92. Siri’s IQ in this study is 23.9. For context, an 18-year-old’s IQ is roughly 97, according to the paper. Testing began in February 2016, using the current iterations of the various assistants. The paper's authors admit that current A.I. systems are a product of their human creators, making them literally only as good their makers. They also suggest that humans are a “special artificial intelligence system made by nature,” due to our ability to create and be creative. The study shows A.I. learns differently than humans; it either learns quickly and keeps learning, or learns quickly and tapers off. Humans tend to learn at a more gradual pace, but start on a higher level than A.I. The takeaway is that A.I. assistants are stupid, so don’t expect too much from Siri just yet. What can’t be foreseen is the impact of Google, Apple and others making big plays (such as CoreML) for more data, which may crowdsource how A.I. learns in coming years (accelerating its collective "growth"). Sometimes, it really does take a village to raise an A.I. assistant.