Natural Language Processing (NLP) has advanced to the point where it can allow humans to interact with machines in non-technical terms. This opens the door to deriving value very quickly from the most complex enterprise data management systems.
Natural language processingOpens a new window (NLP) is a branch of AI focused on enabling humans to communicate with computers in human languages vs. computer languages. When humans speak to each other, the conversations are flexible and fluid. Most discussions don’t break down because of a single error in grammar.
Computer languages, however, do. One typo in code syntax, and the entire function totally breaks down. This distinct quality of computer languages has numerous impacts, not the least of which is preventing businesses from getting value from their data.
Currently, the only people who can talk to databases, data warehouses and data lakes are those with programming skills. We communicate with computers on their terms, in code that they can decipher, but which takes us years to learn. The focus of NLP is on reversing this pain-in-the-neck process and getting computers to communicate with humans on our terms, in our languages.
NLP Becomes Ready for the Big Time
The seeds of NLP were sewn at the beginning of computer science by Alan Turing and have since been slowly developed by a series of innovations that have taken place from the 1950s up till today. The biggest change was the advent of machine learning, which marked a shift from complicated hard-coded rules to algorithms that can generalize. Where traditional computing provided an overly complicated list of precise instructions, machine learning offered the ability for programs to learn from data and base decisions on probabilities.
For instance, with hard-coding, an extremely code-heavy and resource intensive algorithm might be able to identify a photo of a specific cat, provided that all the conditions (lighting, angle, etc.) were exactly right. With infinitely less code, machine learning models could, within seconds, learn how to identify not just a cat but any cat under a wide range of conditions. This same ability left the technology primed for deciphering one of the most wily, unpredictable and downright sloppy forms of communication–human speech.
Machine Learning Takes NLP to New Heights
Whereas improving hard-coded recognition models required layering on more complex code (which meant that the algorithms were slower and much less practical), machine learning models can be improved by merely providing more data. Just like children, they become more accurate with more exposure to human speech.
Taking things further, deep learningOpens a new window –a branch of AI which mimics the functioning of a human brain by layering ‘artificial neural networks’–has produced significant advances. NLP software can now can overlook a multitude of errors and nuances in human speech and provide the answers people seek. That’s why you can type a poorly thought-out question into a Google browser and get a cogent answer or ask Alexa to play songs like The Rolling Stones and get blues-based rock music.
How NLP Will Take Enterprise Data Management to the Next Level
At this point, most of us are thoroughly familiar with NLP in consumer tech, and it’s quickly becoming a part of life that we take for granted. This has caused a large number of people to wonder why we can’t get the same value on the business side of things. If I can ask Alexa to tell me how many people there are in Bolivia, why can’t I ask my company’s data warehousing system how many customers we have in Bolivia? While the former question will be answered in seconds, the latter might take weeks for a team of data scientists to retrieve, depending on how fractured and siloed their enterprise data management system might be.
This very scenario, however, illustrates the potential value that NLP has for business systems. And in fact, it is just a matter of taking the breakthroughs that have given us Alexa and Cortana and applying them to business scenarios. Currently many systems allow you to do text or keyword-based search for database tables, but NLP opens the door to EDM systems that actually have the ability to understand the context and intent of the question. Based on understanding of previous interactions, a natural language query such as “Tell me how many customers we have in Bolivia that spent more than $300 in 2018†could produce an assembly of tables joined together across multiple data systems that included the exact information required to answer this question.
What’s more, it could do it in seconds, whereas it might take data scientists, DBAs and engineers days to hunt it down. With these kinds of improvements, these same teams could be putting their efforts toward doing what they were trained, hired and paid the big bucks to do–discovering insights that will improve your business, get your products to market faster and allow you to innovate vs. hunting down elusive datasets.