Analytics + AI = Analytics as a Conversation

The pendulum is swinging in the business intelligence and analytics world. The on going technology evolution driven in part by the adoption of Big Data, machine learning and other advancements in cloud computing have made the storing, modeling and analyzing of huge volumes and velocities of data possible. The tools and IT skills needed to turn this data into rich visual information is now more possible than ever before.

Products like Tableau, Splunk, Qlik, Birst, among others, have brought rich visualization and actionable-minded analytics (actionable analytics still not that common :) to the masses. It is now easier more than ever to build rich visualizations, reports and dashboards. Building BI solutions to tackle all that data percolating around us and across social networks, IoT and within the enterprise is available to the IT masses to build compelling visual user experiences.

Visualization Overload

But there is trouble brewing on the horizon. Is there such a thing as too much data? Too much information? Too much visualization? I have built my share of BI and I have seen many amazing and compelling visualizations and dashboards using powerful solutions like Tableau and many home grown SaaS BI platforms. But I think it is time to step out of the forest and look at how humans effectively interact with information.

While we rely heavily on our visual sense, even the most well intention and minimalistic BI dashboard (and its supporting drill-down reports) might not be the best solution all the time at getting to the information you want or need. Humans have another ability for consuming information, the conversation (question and answer).

There are many technologies now converging and making it possible for us to evolve our BI stack beyond purely visualization based analytics. Analytics-as-a-Conversation (A3C) in my mind is the next frontier for BI. It does not necessary replace today's rich visualization based BI, but augments it.

Living in the Matrix

What is A3C? Well, in movie terms, it is sort of the Matrix. It is about having a conversation with your BI and getting at what you need (the what) through normal human-like conversation (think texting, hashtags, tweets and even emojis). Also, this conversational form of BI is a much more natural way of interacting with complex information and can more naturally lead to asking not just the "what" questions but the "why" questions to your BI Matrix. And this form of information interrogation lends itself to setting a more clear context to the information exchange, as the BI conversation progresses from one question-answer to the next question-answer. For example, perhaps you ask your A3C system the value of a particular KPI or which KPI is the most off its norm this quarter. And then this can naturally lead to such questions as to "why is this KPI higher this quarter?"

Obviously we are not Neo and we are not talking to the Matrix, so the system has to be taught (or programmed to learn) how to converse with a human-like grammar and has to programmed to extract what it needs from the grammar/questions using NLP and then translate that into queries against the target data and metadata system. There would have to be bounds on the grammar and enough knowledge of the system's metadata to compose the proper answers. No small engineering effort, to say the least, but from where we are today with AI, bots, machine learning, NLP and general computing stacks, the technology is there to accomplish this.

All the Right Ingredients Coming Together for AI

Why now? Because the technologies needed to construct the BI Matrix I am describing is largely here and the data volumes are now, in my mind, overwhelming even for the best BI visualizations. With a bit of creativity (and sweat), and with current availability and advancements in Machine Learning, AI and general computing power, it is possible today to begin to build such intelligent conversational analytics systems and user experiences. Don't forget this a about changing how the user "experiences" data.

It is not just about data volumes and technology capabilities, human interaction has itself evolved in the past decade. We have seen with the recent explosion of mobile and social communication that humans are using texting and short messages for communication more than ever and with no sign of ebbing. In fact, texting is quickly becoming the dominant form of communication and the main form of information exchange across the globe and across all demographics.

How is this better than the visualization based BI we have today? Well, I would say it is not necessarily a replacement for the BI we have today, but is instead complementary and can lead to BI answering questions of "what" and "why" that the original BI developer/modeler could not necessarily anticipate out of the box. And as artificial intelligence and machine learning systems continue to evolve and improve the potential is virtually limitless and no longer bounded by what can be rendered on a 2D display or a click of the mouse.

Back to the Future

The revenge of the CLI (the command line interface) is upon us :) But don't underestimate the conversational CLI, it will prove to be orders of magnitude more powerful than any visualization a human can conjure up.

Stay tuned, Analytics-as-a-Conversation is coming and we will all be talking about it (or talking with it).