Another dot in the blogosphere?

CrashCourse AI episode 7

Posted on: September 30, 2019

Video source

This episode focused on natural language processing (NLP) by AI.

Examples of relatively simple NLP might include spam email filtering, discerning if a search for “apple” refers to a grocery item or a phone, and understanding voice inputs for way-finding. According to the narrator, this type of AI performs natural language understanding.

A higher level of NLP is natural language generation. Examples include language translation (perhaps more accurately, transliteration), document summarisation, and chat bots.

The problem with words is that their meaning is largely contextual. Describing an experience as “Great!” can be positive (said in an upbeat tone) or negative (said in a sarcastic manner). So how does AI learn to distinguish word use?

AI needs distributional semantics, i.e., seeing which words appear and often they appear with other words in sentences. To do this, it needs to convert language to the more universal language of mathematics, e.g., count vectors — the number of times a word appears with other common words in the same article. The problem with count vectors is the amount of effort and data it would require.

This is where unsupervised learning, the topic of the previous episode, comes in. Using an encoder-decoder model, various partial sentences might be fed to encoders for decoders to complete. The video provided two examples of a simple and a complex sentence for encoding and decoding.

By using a recurrent neural network, we might understand the inner workings of Gmail’s predictive text, e.g., how it helps us compose replies by suggesting words and sentences. Unlike images, such words cannot be assigned value. Instead, it uses mechanisms in unsupervised learning to decide the proximity or value of the next word.

The series has built up in complexity while probably only scratching the surface over the last seven episodes. But it has provided me with invaluable insights into AI.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.


Usage policy

%d bloggers like this: