IBM Machine Learning for z/OS – Like No Other
Like no other Private Cloud
With many of the top banks, retailers, and insurance organizations using IBM® z Systems® , combined with tried and tested virtualization capabilities, EAL5+ security rating and the ability to handle billions of transactions a day[1], the platform becomes attractive as a private cloud for running advanced analytics as well as cloud managed services.
Those organizations are in an enviable position, with volumes of new and historical business-critical data available on such powerful and reliable systems. The sheer volume and velocity of the transactions, the richness of data in each transaction, coupled with data in log files, is a potential gold mine for machine learning applications to exploit cognitive capabilities and do smarter work, more intelligently — and more securely.
Leveraging Machine Learning on z Systems
Set against an asymptotic curve of information growth, Chief Information Officers and data scientists constantly battle to gain deeper insights from the volumes of transactions and log data on the platform (and many other platforms) and turn those insights into concrete gains. In most cases, the CIOs already have astute teams of data scientists and data engineers combing through this data — and yet they see their teams struggle to make enough time for the deep work they’re trained to do.
“Enterprises are well aware of the tremendous potential and value of the transactional and operational data on their z Systems. Yet most of them struggle with how to expose the data within the enterprise in a secure and controlled way that’s still flexible enough to foster innovation and support efficient access for a variety of roles — data scientists, operations, and application developers. Not an easy task, but organizations that can do so potentially obtain an edge over the competition.”
—ANDREI LURIE, DB2 FOR Z/OS ARCHITECT, IBM
Machine learning has the potential to be the perfect intelligent app — to hike efficiency, create and cement deep personal relationships with customers, push into new lines of business and new markets while helping to minimize financial risk and fraud.
I have heard customers say that the mainframe has never been hacked. But it doesn’t mean cyber criminals aren’t trying, nor that unscrupulous people aren’t attempting to commit fraud. Having applications that embed predictive models that can analyze, sense, react and become smarter with every transaction and interaction in such a business critical environment brings us a long way toward identifying and preventing potential fraud.
But z Systems is not just about transactions. It is already considered to be a hybrid transaction and analytics processing (HTAP) environment with a complete set of the analytics capabilities and acceleration technologies available today. IBM has also added full exploitation of Apache Spark™ on both z/OS and Linux® on z Systems – a solid base for building, testing and deploying high performance machine learning applications.
“By running advanced Apache Spark™ analytics directly on their production systems, enterprises can improve both the efficiency and timeliness of their insights. Moving Spark inside the mainframe also simplifies and can help reduce security risks as there is only one copy of the data to protect, and that copy resides inside z/OS’s security rich environment.”
— FRED REISS, CHIEF ARCHITECT, IBM SPARK TECHNOLOGY CENTER
For all these reasons and more, we are delivering the full range of our machine learning capabilities to z/OS essentially bringing advanced ML to the world’s most valued data.
Machine Learning without Compromise.
When asked to describe machine learning I break it down into three perspectives: Freedom, Productivity and Trust. I find these resonate well with customers’ needs.
Freedom. Think of freedom as a set of unified but powerful capabilities such as the flexibility of the interfaces that can be used to interact with machine learning — whether a Jupyter notebook or intuitive graphical interfaces catering to the needs of various personas from beginners to expert data scientists. With support for Python™, Java™, and Scala, different organizations can leverage their preferred programming language and skills when building machine learning applications. Machine learning from IBM can be developed on and deployed across different computing environments such as private cloud and public cloud – including IBM z Systems z/OS with a choice of frameworks such as SparkML, TensorFlow™ and H20.
With the data available to machine learning solutions, users can create advanced algorithms or choose from a set of predefined powerful algorithms and models without requiring advanced data science expertise.
Think of all this capability running on one of the highest performing platforms available: IBM z Systems. It means machine learning can be brought to bear many thousands of times per second [2] — which can help reduce costs and risks, finding and leveraging new opportunities at every transaction and interaction.
Productivity. To make machine learning consumable it has to be easy and intuitive for end users. To this end, IBM machine learning was built around three core principles of simplicity, collaboration (across multiple personas) and convergence of a wide range of technologies from our analytics portfolios and our research laboratories. The user experience is key, whether the user is a data scientist – advanced or beginner — or a computing generalist. Across personas, IBM Machine Learning lets users engage and collaborate on machine learning projects– leveraging the combined skills of the team. Wizards within the tools provide step-by-step processes and workflows that automate many aspects of building, testing and deploying learning machines. As part of the process the IBM Cognitive Assistance for Data Scientists (CADS) automates the selection of the best algorithm given a training data set. It starts by allocating a small part of the data set to each candidate algorithm, then estimates performance on a full data set. It uses the estimate to rank algorithms, and allocates more data to the best ones. It iterates until the best algorithms get all of the data set.
Trust. Once a model is built and tested, it needs to be deployed. A model – in fact the entire machine learning application (learning machine) — is similar to a living organism, evolving and adapting over time as it encounters new data with each transaction and interaction. This is achieved through a continuous feedback loop that enables the model to adapt and change, altering specific parameters within the model itself to become smarter and more consistent over time – while avoiding overfitting. This auto-tuning is key to reducing manual intervention. Of course some human intervention or model adaptation may be necessary where a human judgement or decision is required. Therefore, keeping track of the version of the models over the lifecycle of the learning machine is important for audit purposes or to fall back to a previous version. Another aspect of trust is of course the governance and security (the who, how, when, where) of the data, the models, and the many machine learning artifacts. IBM z Systems is recognized in the industry when it comes to security [3]– and a key reason why some of the biggest names and well known organizations across many industries run their business critical applications and data on the platform.
These three perspectives are summarized in the Figure #1 below.
Figure #1 – IBM Machine Learning – the complete picture.
From a technology point of view, our aim is to free up data science teams to do the deep work that’s being asked of them — work that gets harder and harder as the world moves faster and with less certainty. Ultimately, the gains that CIOs are seeking will come from a collaboration between smart data systems and smart data scientists. Machine learning on z/OS will help enable and encourage that collaboration.
IBM Machine Learning “Hub” – Beyond the Technology
While the technology aspects may deliver very advanced machine learning capabilities, IBM recognizes the need to nurture and partner with organizations as they embrace and fully exploit its machine learning technologies. The first IBM Machine Learning Hub will provide the means to achieve this, with the aim to accelerate and enrich organizations’ machine earning knowledge and expertise.
The “hub” will allow organizations to access IBM world-class data science professionals who can provide education and training, expert advice on all aspects of machine learning – as well as lead and deliver proofs-of-concept and full client engagements. They focus on delivering tailored machine learning knowledge and skills transfer built around the needs and wants of customers. This combination of both the technology aspects and the knowledge / skills base is an opportunity to provide a unique machine learning experience what I consider to be the machine learning ecosystem.
Let me close this blog post by inviting you to take a look at a short video on machine learning here and reading the recent announcement of IBM Brings Machine Learning to the Private Cloud.
Dinesh Nirmal – Vice President, IBM Analytics Development
Follow me on Twitter: @dineshknirmal
[1] http://www-03.ibm.com/press/us/en/pressrelease/51623.wss
[2] based on IBM SPSS Modeler Scoring Adapter for DB2 for z/OS performance
[3] EAL5 + Security Rating
TRADEMARK DISCLAIMER: Apache Spark, Spark and the Spark logo are trademarks of the Apache Software Foundation (ASF). Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Python is a registered trademark of the PSF. Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates. TENSORFLOW is a trademark of Google Inc. IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both.