Natural Language Processing

If you've ever used a translation app, had predictive text spell that tricky word for you, or said the words, "Alexa, what's the weather like tomorrow?" then you've enjoyed the products of natural language processing.

It's no coincidence that we can now communicate with computers using human language - they were trained that way - and in this article, we're going to find out how. We'll begin by looking at a definition and the history behind natural language processing before moving on to the different types and techniques. Finally, we will look at the social impact natural language processing has had.

Definition of Natural Language Processing

Natural language processing (NLP) is a branch of artificial intelligence (AI) that assists in the process of programming computers/computer software to 'learn' human languages. The goal of NLP is to create software that understands language as well as we do.

Natural language processing has roots in linguistics, computer science, and machine learning and has been around for more than 50 years (almost as long as the modern-day computer!).

Today, we can see the results of NLP in things such as Apple's Siri, Google's suggested search results, and language learning apps like Duolingo.

Fig 1. We can talk to 'Alexa' because of natural language processing

History of Natural Language Processing

The beginnings of NLP as we know it today arose in the 1940s after the Second World War. The global nature of the war highlighted the importance of understanding multiple different languages, and technicians hoped to create a 'computer' that could translate languages for them.

The creation of such a computer proved to be pretty difficult, and linguists such as Noam Chomsky identified issues regarding syntax. For example, Chomsky found that some sentences appeared to be grammatically correct, but their content was nonsense. He argued that for computers to understand human language, they would need to understand syntactic structures.

By the 1990s, NLP had come a long way and now focused more on statistics than linguistics, 'learning' rather than translating, and used more Machine Learning algorithms. Using Machine Learning meant that NLP developed the ability to recognize similar chunks of speech and no longer needed to rely on exact matches of predefined expressions. For example, software using NLP would understand both "What's the weather like?" and "How's the weather?".

By 2011, Apple released the first successful and publicly available NLP virtual assistant, Siri.

How Does Natural Language Processing Work?

You're probably wondering by now how NLP works - this is where linguistics knowledge will come in handy.

NLP uses AI to take in real-world human language and perform processing tasks in order to turn the language into code the computer will understand. There are two parts to this process:

• Pre-processing (sometimes referred to as data processing) This involves breaking the language down and converting it into data that an algorithm can work with.

• Algorithm development - Once the language has been turned into data, an algorithm must be developed to process and use it.

Let's look at some of the most common pre-processing techniques now. These techniques are rooted in linguistics and linguistic analysis. We won't be looking at algorithm development today, as this is less related to linguistics.

Natural Language Processing Techniques

There are two main pre-processing types: syntactic and semantic analysis. Before we dive into these techniques, let's look at some definitions for these two terms.

Syntactic Analysis

Syntactic analysis involves looking at a sentence as a whole to understand its meaning rather than analyzing individual words. There are several syntactic analysis techniques NLP utilizes.

Parsing

Parsing involves breaking a sentence down into each of its constituents. A constituent is a unit of language that serves a function in a sentence; they can be individual words, phrases, or clauses. For example, the sentence "The cat plays the grand piano." comprises two main constituents, the noun phrase (the cat) and the verb phrase (plays the grand piano). The verb phrase can then be further divided into two more constituents, the verb (plays) and the noun phrase (the grand piano).

Conducting a parsing analysis involves representing each sentence's constituents in a parse tree, like so:

Fig 2. Example of a parse tree

Parse trees can show us the relationship between words in a sentence and how they work together to form constituents. For example, we can see that "the grand piano" is a constituent, but "plays the" isn't. This information can be turned into data for an NLP algorithm.

Stemming

Stemming is a morphological process that involves reducing conjugated words back to their root word.

Taking each word back to its original form can help NLP algorithms recognize that although the words may be spelled differently, they have the same essential meaning. It also means that only the root words need to be stored in a database, rather than every possible conjugation of every word.

Text Segmentation

Text segmentation is the process of separating language into meaningful units, such as morphemes (e.g., un-, luck, -y), words, sentences, paragraphs, and intent (i.e., what is the purpose of the language? does it ask a question, provide a statement, or give an order?).

Semantic Analysis

Sometimes sentences can follow all the syntactical rules but don't make semantical sense. This is why it's important to also conduct semantic analyses. These help the algorithms understand the tone, purpose, and intended meaning of language.

Sentiment Analysis

Sentiment analysis is an NLP technique that aims to understand whether the language is positive, negative, or neutral. It can also determine the tone of language, such as angry or urgent, as well as the intent of the language (i.e., to get a response, to make a complaint, etc.). Sentiment analysis works by finding vocabulary that exists within preexisting lists.

Disambiguation

Word disambiguation is the process of trying to remove lexical ambiguities. A lexical ambiguity occurs when it is unclear which meaning of a word is intended.

Removing lexical ambiguities helps to ensure the correct semantic meaning is being understood.

Natural Language Processing Examples

Now we have a good idea of what NLP is and how its works, let's look at some real-world examples of how NLP affects our day-to-day lives.

Fig 3. Language translation as we know it today wouldn't be possible without NLP

The Social Impact of Natural Language Processing

In 2016, the researchers Hovy & Spruit released a paper discussing the social and ethical implications of NLP. In it, they highlight how up until recently, it hasn't been deemed necessary to discuss the ethical considerations of NLP; this was mainly because conducting NLP doesn't involve human participants. However, researchers are becoming increasingly aware of the social impact the products of NLP can have on people and society as a whole.

Here are some of the main issues they identified:

• Exclusion - NLP may learn from dominant cultures, making it easier to use and more appropriate for those from those dominant cultures.

• Overgeneralization - NLP may lead to software making widespread assumptions about things like our gender, age, religion, and sexual orientation.

• Bias - Most NLP tools focus on English and can therefore produce more rich data for English speakers than for others.¹