Business Differentiation through Machine Learning
I look at my child and marvel as he embraces the ever fast moving world around him, adapting to new experiences, grasping technology, absorbing a bombardment of information from so many sources. It’s staggering to watch their progress from basic learning of just accepting facts they are taught, to augmenting those facts with their own knowledge, to asking questions, using their knowledge to express their opinions and values to others, then challenging facts and hypotheses that they once accepted to adapting their knowledge, understanding, decisions and value systems. And then they start reasoning with their parents! Their brains are like sponges using their senses of seeing, hearing, touch, smell and taste, absorbing information, experiences, emotions, constantly connecting new events and situations with those from the past – pushing boundaries and testing us.
Watching my son learn was fascinating but quite frankly businesses can’t wait years to learn about their data and how best to act on it.
From Business Intelligence to the Intelligent Business
The business world is complex with many fast moving dynamics. We only see the tip of the iceberg when it comes to the total information that exists at any point in time as it grows exponentially. Nor are we able to fully harness the collective thoughts or full knowledge of the people around us. This is where cognitive technologies such as the many forms of machine learning (ML) can really be a huge advantage to business people.
Simply put machine learning is the capability of computers to learn without being explicitly programmed.
Imagine systems that have total information awareness. “Nodes” that connect with each other on premise and in the cloud that learn about each other like synapses in the brain. Harnessing the collective consciousness of these “machines” (don’t restrict your thoughts to hardware here) and applying intelligence to advise on the “optimal” decisions resulting in the “optimal” outcome is what every organization would want. I think I just defined the killer-app for business.
What does it look like? Well I envisage it as the graphic below – a cycle or workflow of constant learning.
Figure 1 : Machine Learning – a cycle of constant learning
What’s out there today?
The good news is that much of these capabilities exist in IBM solutions today and much more in our research labs yet to emerge and potentially change the industry.
So how do you get started? There are four main states when considering knowledge: Know what you know, Know what you don’t know, Don’t know what you know, Don’t know what you don’t know. And all this starts with your data. We have solutions that can catalogue what information you have, discover information that you didn’t know you had and help identify information that is missing or untrustworthy. These can help reduce not knowing what you don’t have.
Next you need capabilities that can ingest information where ever it is without having to move it. From this data we can apply machine learning (ML) that can discover relationships, rationalize, predict what might happen next and actually become intelligent (all knowing) about the data and all previous outcomes. This can only be achieved through continuous learning and adapting. There are also other steps around optimization of that learning that can be read here. There are many vendors that claim to provide one form of ML capability or another and while that may be interesting in its own right, on its own it has minimal value. Combining all forms or learning and analytics can take you closer to the bigger picture of “cognitive” in which IBM strives to be a leader. I’ll expand a little more on cognitive later in this blog.
From Tic-Tac-Toe , Jeopardy, Crime Prevention, Cancer Treatments and More
In my early student days I learned how to code simple recursive algorithms using rote learning to produce a game of Tic-Tac-Toe that could not be beaten after five or six games. It did this by minimizing its losses by avoiding outcomes that would lead to failure. In 2011 “Watson” competed on Jeopardy and beat the top contestants. It had access to 200 million pages of structured and unstructured content consuming terabytes of disk storage including the full text of Wikipedia but was not connected to the Internet during the game. For each clue, Watson’s three most probable responses were displayed on the television screen. Watson consistently outperformed its human opponents on the game’s signaling device, but had trouble in a few categories, notably those having short clues containing only a few words – kind of similar to human language ambiguity. Its natural language processing, predictive scoring and models were key to its success.
In February 2013, IBM, Wellpoint and Memorial Sloan-Kettering Cancer Center announced the first commercially developed Watson based cognitive computing technology to be implemented for utilization management recommendations to physicians in lung cancer treatment at Memorial Sloan Kettering Cancer Center in conjunction with health insurance company WellPoint Inc. (now Anthem Inc). At the time of writing this blog, nearly 43,000 organizations have registered to use IBM Watson Healthcare Analytics Platform (1).
Machine Learning 101
I am often asked “How confident are you in that decision?” I used to base my answer on the strength of the information I had – and some of it was intuition, gut feel, life experiences. But now I can scientifically put a figure on it – as precise as the underlying information allows of course. In fact, well established IBM analytics products have been using these predictive models and scoring algorithms (included in Watson Analytics) across many industries for years to better manage risk, and identify potential fraud (I’ll tell you about my credit card experience in a later blog while attempting to rent a vehicle). In 2015 IBM donated its SystemML to the Apache™ Software Foundation. SystemML is a flexible machine learning system designed to auto-scale to Spark and Hadoop® clusters and extends the core machine learning in the Apache Spark™MLlib libraries. We also have machine learning in many other forms available as-a-service including but not limited to Natural Language Classifier, Retrieve and Rank, AlchemyVision –and many others that we will describe in more detail later. Below is a diagram of how machine learning is implemented as a generic model.
Figure 2 – A generic machine learning model
Beyond Traditional ML – Learning through Senses
As mentioned earlier IBM has many other forms of machine learning technologies that help differentiate our cognitive capabilities from other vendors. Using vision, language and other ML technologies begins to more closely simulate human behavior of understanding, reasoning and learning. Below are some of the key ML technologies that are being used by many organizations around the world.
AlchemyVision is an API that can analyze an image and return the objects, people, and text found within the image. AlchemyVision can enhance the way businesses make decisions by integrating image cognition. Organizations across a variety of industries ranging from publishing and advertising to eCommerce and enterprise search can effectively integrate images as part of big data analytics being used to make critical business decisions by better targeting ads, organizing image libraries, improve consumer experience, monitor your brand, profile target markets, improve researching. The Tabelog case study is particularly interesting. Over 40,000 foodies visit the Tabelog site, confident that Tabelog will provide accurate and reliable recommendations. Additionally, more than 200,000 registered restaurants use the site to help brand and promote their establishment. Try it now.
Natural Language Classifier is a service that enables developers without a background in machine learning or statistical algorithms to create natural language interfaces for their applications. The service interprets the intent behind text and returns a corresponding classification with associated confidence levels. The return value can then be used to trigger a corresponding action, such as redirecting the request or answering a question. The Natural Language Classifier is tuned and tailored to short text (1000 characters or less) and can be trained to function in any domain or application. Typical usage scenarios are:
Tackle common questions from your users that are typically handled by a live agent.
Classify SMS texts as personal, work, or promotional
Classify tweets into a set of classes, such as events, news, or opinions.
Based on the response from the service, an application can control the outcome to the user. For example, you can start another application, respond with an answer, begin a dialog, or any number of other possible outcomes.
You can try it out by clicking here.
Retrieve and Rank is a service helping users find the most relevant information for their query by using a combination of search and machine learning algorithms to detect “signals” in the data. Built on top of Apache Solr, developers load their data into the service, train a machine learning model based on known relevant results, then leverage this model to provide improved results to their end users based on their question or query. The Retrieve and Rank Service can be applied to a number of information retrieval scenarios. For example, an experienced technician who is going onsite and requires help troubleshooting a problem, or a contact center agent who needs assistance in dealing with an incoming customer issue, or a project manager finding domain experts from a professional services organization to build out a project team. You can try it out here.
From Crystal Ball Predictions to Prescriptive Actions
So predicting a hurricane or an outcome or recognizing images and video or understanding the importance of a particular piece of information is just a first step. Now you need to take prescriptive actions on that understanding to, for example, survive the hurricane or achieve business advantage from a business opportunity that may only last a short time.
IBM has the capabilities not only to help provide an advanced and wide range of machine learning capabilities and to act on them – but to do it in the “right-time” – for example being able to predict whether a trade or bank transaction is legitimate or fraudulent 15,000 times a second. In business, speed is of the essence.
Cognitive + Cloud = Optimal Business Outcomes
So it seems that machine learning can be advantageous (even smarter) and faster than humans in certain situations – given complete and accurate data. Combining the wide range of advanced machine learning capabilities described above with the ability to act prescriptively on what has been learned with IBM’s full cognitive analytics capabilities can yield many opportunities to potentially out-maneuver your competition, act with confidence and help your organization become the optimal Intelligent Business. Cloud is key here because it has the potential for everyone and everything (including data) to be interconnected.
Machine learning can help enable cognitive systems to learn, reason and engage with us in a more natural and personalized way. These systems will get smarter and more customized through interactions with data, devices and people. They will help us take on what may have been seen as unsolvable problems by using all the information that surrounds us and bringing the right insight or suggestion to our fingertips right when it’s most needed. Over the next five years, machine learning applications could lead to new breakthroughs to help amplify human abilities, assist us in making good choices, look out for us and help us navigate our world in powerful new ways.
In summary machine learning in all its forms has the potential to bring the collective knowledge and consciousness of humans and machines together to help make the world a better, safe place.
In the following months the team and I will be taking you on a journey exploring many more aspects of IBM’s cognitive and learning capabilities.
Of course you won’t be able to predict where I’m taking you next.
For more information on IBM’s cognitive and machine learning capabilities click this link ibm.com/outthink
Dinesh Nirmal – Vice President, IBM Analytics Development
Follow me on Twitter: @dineshknirmal
Jean Francois Puget (PhD) - Distinguished Engineer, Chief Architect IBM Analytics Solutions
Follow JFP on Twitter @JFPuget
Foot notes
(1) IBM Watson Analytics Team based on number of registrations as at June 27 2016
TRADEMARK DISCLAIMER: Apache, Apache Hadoop, Hadoop, Apache Spark, Spark are trademarks of The Apache Software Foundation.