Watson, the IBM supercomputer that won $1 million playing Jeopardy in 2011, is now becoming available to app developers. The system that was built for Jeopardy was not your everyday PC. With 2,880 cores and 16 TB of RAM running SUSE Linux Enterprise Server 11, it cost roughly $3 million. Its processing speed can read a million books a second, approximately 500 GB of data. On the day of the TV show, it had 200 million pages of structured and unstructured content, including a complete edition of Wikipedia (~30 GB), but was not connected to the Internet. It deduced answers from 4 TB of data held in RAM. Watson is significant in showing that it can “understand” natural language. Using software developed in Java, C++ and Prolog, it employs over a 100 different techniques to analyze natural language, devise answers and calculate how confident it is of its answer. In many ways, this is analogous to chess-playing computers that analyze millions or billions of positions before choosing the best moves. Neither Watson nor the chess-playing computers understand in the same way that a human does -- they are more mechanical. Understanding questions and identifying the context in which they are asked is a very difficult challenge. Consider the phrase “Time Flies.” Does it mean that the passage of time goes quickly, or refer to using a stopwatch to time a fly's flight? It's all about the context in which a question is framed. Since its performance on Jeopardy, Watson has found employment in different domains: helping medical practitioners diagnose disease, handling online technical questions and parsing large volumes of legal documents. These have been big-company applications, but soon the power of Watson will be coming to smartphones and tablets. There are several systems in play. Deep QA turns questions hundreds of queries across multiple data sources. The results are collated to give a ranking strength. This powerful data mining identifies patterns that an individual would be unlikely to pick up. Then there’s a system with 41 different subsystems that assess questions, information and answers from multiple sources.

Roll Your Own Watson

The latest announcement from IBM is of the Watson Developer Cloud. This will include a developer tool kit, educational materials and access to Watson's application programming interface. Companies like Fluid Retail will be launching apps that use Watson as a personal shopping assistant to pull data from multiple sources. The dream of having your own online personal shopping assistant that can understand what you really want could be a lot closer than you think. Google and Siri on iOS may come under serious threat in the next few years, though Google has the resources with its Knowledge Graph to fight back. The Knowledge Graph is a database of 500 million objects and 18 billion facts about relationships between them. So queries about, say, “orange” are interpreted to understand when they mean color and when they mean fruit. The power of Watson to go beyond simple search will make possible what are called “cognitive apps.” How about an interactive travel guide that learns what you like and offers a tailored tour?  Or an investment advisor that can understand which stocks are rising? The next killer app could well be a something like this, powered by Watson.