What is natural language processing (NLP)?

"Natural language" means the languages people speak and write, such as English or Chinese, as opposed to programming languages built for machines. Human language is messy: one word can mean different things (the bank of a river or a bank account), sentences can be ambiguous, and meaning depends on context and tone. NLP is the work of getting computers to cope with that mess.

Typical NLP tasks include:

Early NLP relied on hand-written rules and word lists, like a phrasebook that only works for sentences someone thought of in advance. From the 1990s, statistical methods learned by counting word patterns in large collections of text, which made translation and spam filtering much better.

Neural networks then learned to represent words as lists of numbers, called embeddings, so that words with similar meanings sit close together; word2vec (2013) made this popular. The transformer architecture, introduced in 2017, lets a model weigh every word against every other word in a passage, and pre-trained models such as BERT (2018) showed that one model could be adapted to many language tasks. Scaling this approach up produced today's large language models, which handle most NLP tasks from a plain-language instruction.

NLP is built into much of the software you use every day:

Processing language is not the same as understanding it the way a person does. NLP systems can miss sarcasm, slang, cultural references and ambiguous instructions, and a fluent answer from a language model can still be factually wrong.

Quality also varies by language. Systems usually perform best in languages with lots of digital text to learn from, such as English, and less well in languages with little. Some languages are also split into more tokens (word pieces) per sentence, which can make the same request cost more on a token-billed API.