white chat bubble on a light blue background

Why Companies Are Investing in Natural Language Processing

May 19, 2021 Information Management
David Roe
By David Roe

Almost one-third of IT professionals are currently using artificial intelligence in their business and many others are joining the race.

According to IBM’s Global AI Adoption Index, nearly one in three IT pros say their business is now using artificial intelligence, with 43 percent reporting their company has accelerated their rollout of AI because of the COVID-19 pandemic. While recent advances in the technology are making AI more accessible than ever, the annual survey, which included 5,501 companies, also found a lack of AI skills and increasing data complexity are continuing challenges.

The survey also showed that COVID-19 accelerated how businesses are using automation today: 80 percent of companies are already using automation software or plan to use this technology in the next 12 months, and more than a third said the pandemic influenced their decision to use automation to bolster employee productivity. Others found new applications of this technology to make themselves more resilient, such as helping to automate the resolution of IT incidents.

One finding that stands out, probably not surprising given the circumstances of remote work, is that natural language processing is at the forefront of recent adoption.

Natural Language Processing Gains Traction

Natural language processing, or NLP for short, is the automatic manipulation of natural language like speech and text by software. The IBM research showed that almost half of businesses are using applications powered by NLP and one in four businesses plan to begin using NLP technology over the next 12 months. 

Customer service is the top use case, with 52 percent of global IT professionals reporting their company is using or considering using NLP to improve customer experience. It was also the area where IT professionals reported they were most likely to increase their focus on due to COVID-19.

So where does NLP fit into the organization’s tech stack? NLP is a branch of artificial intelligence that deals with the interaction between computers and humans using natural language, said Hima Pujara, of Emeryville, Calif.-based Bug Raptors, a software testing and QA company. Its use has now gone beyond virtual assistants such as Siri or Alexa and has grown to natural language understanding (NLU).

NLU, for its part, goes beyond specific words and language structure to precisely discern intent, context and ambiguity. Moreover, NLU algorithms can handle all the inferences, suggestions, idioms, and subtleties that humans employ in written text and speech.

“NLP is now present in sentiment analysis, where texts surrounding social gestures or comments may give a clue as to whether such gestures or comments are positive or negative,” Pujara said. “With further improvements in speech recognition technology, NLP expands the scope of traditional BI into every aspect of the business."

Citing the work of Dina Demner-Fushman, a researcher in NLP, he pointed out that much of the clinical decision-making process at the National Institute of Health is guided by text-based evidence, made possible due to NLP.

NLP-fueled smart assistants offer significant rewards to businesses. They help businesses increase customer satisfaction rates and custom loyalty. Furthermore, NLP is a good choice for businesses in the future due to its role in sentiment analysis, market intelligence, customer-centric services, target advertising funnels and software testing.

Related Article: Machine Learning Simplified: Building an Understanding

NLP Automates Mundane Tasks

While NLP systems have been around for years, completing simple tasks like spell check, autocomplete or establishing spam filters, it is growing in adoption thanks to significantly more sophisticated models which draw from larger datasets and machine learning techniques, said Will Robinson, chief technology officer at New York City-based ASAPP, a customer experience company. 

“A key reason why NLP is seeing growing use and adoption is due to how perfectly suited it is for automating many tasks in customer service,” he said. “Customer service is in this Goldilocks zone for NLP: it has enough repeated/consistent elements that it is tractable for 2021's NLP techniques, but there is also enough variation and noise and art within a customer service interaction that you require 2021's NLP.”

Simpler, more rigid rules-based systems are not flexible or generalizable enough. The huge data pools and novel machine learning training techniques that are now available to organizations keep deployment costs down while the overwhelming demand to automate tasks create a perfect set of conditions for NLP to take off in the enterprise.

To illustrate this further, intent classification, a key aspect of NLP, can help determine what question to ask or action to take next for a customer based on what they have said. Effective NLP models know when to query the customer for further information, drawing from a customer’s complete history with a business, and when to complete a task for a customer. Sophisticated NLP models should additionally know what policy constraints are in place, such as honoring a refund request when it is within a company return policy.

Related Article: Where Big Data and Knowledge Management Intersect in 2021

Business Use Cases for Natural Language Processing

There are a wide range of applications. Some of the more interesting business applications of NLP include artificial machine learning generated ad copy, said Jenn Halweil of New York City-based Go Beyond. This is especially true in a world where growth marketers and social media ad experts focus on churning content with keywords and A/B testing this content to lower CPM costs. These are all functions that can be quantified and measured against specific outcomes. This means that they are functions that can be automated through NLP.

Chatbots are another framework utilizing NLP. For example, a traveler can call or text Delta to ask questions about their upcoming flight. Among millions of customers, it is likely that a number of these questions would be repetitive and would have similar answers. Rather than train humans at call centers to be robots from call scripts, why not let actual robots, or in this case, chatbots do it and free up human capital for problems that can't be automated?

Many of the speech-to-text applications could hold the key to breakthroughs in more effective real-time communication among people from different backgrounds, languages, or differently abled people such as the hearing impaired, Halweil said.

“A great example is Google Hangouts, which now does voice-to-text closed captioning in real time," she said. "Now imagine a world where that text could be translated by an AI in real time so a Spanish speaker and say a Chinese speaker could communicate on Google Hangouts across a language barrier in real time?”

The key to all these use cases is a guiding principle whereby organizations can use NLP to streamline repetitive tasks or overcome language barriers and free up human capital to solve more complex problems that machines cannot currently solve.

That said, NLP is not without its challenges, Halweil said. Think of the Twitter chatbot that, after processing too many disinformation campaigns, becomes pro-Hitler, or the GPT-3 AI model that is so creepily accurate the person on the other end is not sure they are talking to a person or not. There are ethical questions to be considered as well business considerations in using NLP. It is a form of manipulation and, like with any manipulation, it is more ethical to consider impacts upfront and make sure outcomes are inclusive, equitable, transparent and verifiable.

"The last thing anyone wants is chatbot armies running disinformation campaigns on social networks that undermine democracy, voting and other important institutions," she said. “So, it's important that we start to build frameworks for how to build ethical AI.”

Widespread Adoption of NLP

NLP techniques as a subset of AI technologies are getting wide adoption and acceptance among companies and individuals. Not without reason.

The “processing” piece means that text-based information can be understood in its context and intent can be discerned from unstructured data. This is a powerful cognitive ability that supports many business processes and increases human capability. For example, the Port of Montreal used NLP and AI models to detect and distribute important cargo during the most difficult months of the pandemic in 2020. NLP-enabled solutions took care of tedious or repetitive manual “reading” operations to extract insights and support human decision-making.

“The interesting part of NLP is its maturity as technology or set of technologies," Halweil said. "There are different levels of performance depending on the  implemented model, but also a realistic understanding of the trade-off between results and requirements. This is key for massive adoption."

While the biggest companies focus on state-of-the-art models such as GPT-3 or equivalents, other organizations are either implementing older techniques or leveraging cloud-native capabilities via Azure, AWS and GCP. Initial requirements and implementation times may vary, but the end goal is the same: Help humans make better and faster decisions using data.

Tags

Featured Research

Related Stories

records management the old fashioned way, in binders on a shelf

Information Management

Your Top 3 Records Management Questions Answered

traffic jam

Information Management

Mancini's Law Says: Information Chaos Has Consequences

A chaotic scene in which an elephant is storming through a restaurant, while a seated well-dressed woman listens to the mayhem

Information Management

How Master Data Management Can Help Tame the Data Governance Mayhem

Register Now: Digital Workplace Experience Summer Session

DWX21 - Q1