Download as PDF, PPTX
Introduction to Natural Language Processing (NLP)
- 1. Natural Language Processing (NLP) Introduction !1 Presentedby, Venkatesh Murugadas Venkatesh Murugadas
- 2. Image source :www.google.com !2Venkatesh Murugadas
- 3. Problem of NaturalLanguage “Human language is highly ambiguous … It is also ever changing and evolving. People are great at producing language and understanding language, and are capable of expressing, perceiving, and interpreting very elaborate and nuanced meanings. At the same time, while we humans are great users of language, we are also very poor at formally understanding and describing the rules that govern language.” - Page 1, Neural Network Methods in Natural Language Processing, 2017. Source : https://machinelearningmastery.com/natural-language-processing/ 3Venkatesh Murugadas
- 4. “It is hardfrom the standpoint of the child, who must spend many years acquiring a language … it is hard for the adult language learner, it is hard for the scientist who attempts to model the relevant phenomena, and it is hard for the engineer who attempts to build systems that deal with natural language input or output. These tasks are so hard that Turing could rightly make fluent conversation in natural language the centrepiece of his test for intelligence.” - Page 248, Mathematical Linguistics, 2010. Source : https://machinelearningmastery.com/natural-language-processing/ 4Venkatesh Murugadas
- 5. Computer Linguistics Linguistics isthe scientific study of language, including its grammar, semantics, and phonetics. Computational linguistics is the modern study of linguistics using the tools of computer science. Yesterday’s linguistics may be today’s computational linguist as the use of computational tools and thinking has overtaken most fields of study. Source : https://machinelearningmastery.com/natural-language- processing/ 5Venkatesh Murugadas
- 6. Statistical NLP Statistical NLPaims to do statistical inference for the field of natural language. Statistical inference in general consists of taking some data (generated in accordance with some unknown probability distribution) and then making some inference about this distribution. — Page 191, Foundations of Statistical Natural Language Processing, 1999. Source : https://machinelearningmastery.com/natural-language- processing/ 6Venkatesh Murugadas
- 7. Natural language processing(NLP) is a collective term referring to automatic computational processing of human languages. This includes both algorithms that take human-produced text as input, and algorithms that produce natural looking text as outputs. — Page xvii, Neural Network Methods in Natural Language Processing, 2017. Source : https://machinelearningmastery.com/natural-language-processing/ Natural language processing is a subfield of computer science, information engineering, and artificial intelligence concerned with the interactions between computers and human languages, in particular how to program computers to process and analyze large amounts of natural language !7Venkatesh Murugadas
- 8. We will takeNatural Language Processing — or NLP for short –in a wide sense to cover any kind of computer manipulation of natural language. At one extreme, it could be as simple as counting word frequencies to compare different writing styles. At the other extreme, NLP involves “understanding” complete human utterances, at least to the extent of being able to give useful responses to them. — Page ix, Natural Language Processing with Python, 2009. Source : https://machinelearningmastery.com/natural-language- processing/ 8Venkatesh Murugadas
- 9. Areas of NLP •Natural Language Understanding • Natural Language Search • Natural Language Generation • Natural Language Interface Venkatesh Murugadas
- 10. Applications of NLP 1.Text classification and Categorisation 2. Named Entity Recognition 3. Conversational AI 4. Paraphrase detection 5. Language generation and Multi-document Summarisation 6. Machine Translation 7. Speech recognition 8. Spell Checking 10Venkatesh Murugadas
- 11. Corpus • “A corpusis a large body of natural language text used for accumulating statistics on natural language text. The plural is corpora. Corpora often include extra information such as a tag for each word indicating its part-of-speech, and perhaps the parse tree for each sentence.” Source : https://www.quora.com/In-NLP-what-is-the-difference-between-a-Lexicon-and-a-Corpus 11Venkatesh Murugadas
- 12. NLP Pipeline • WordTokenisation • Sentence Segmentation • Parts of Speech Tagging • Dependency Parsing • Named Entity Recognition • Relation Extraction 12Venkatesh Murugadas
- 13. Word Tokenisation Token : “Atoken is an instance of a sequence of characters in some particular document that are grouped together as a useful semantic unit for processing.” Eg. To sleep perhaps to dream. Type: “A type is the class of all tokens containing the same character sequence.” Term : “A term is a (perhaps normalized) type that is included in the dictionary.” Text Normalisation : “Token normalization is the process of creating tokens, so that matches occur despite superficial differences in the character sequences of the tokens” Source: nlp.anirbansaha.com Tokenization is an identification of basic units to be processed. Tokenizer must often be customised to the data in question. 13Venkatesh Murugadas
- 14. How is tokenisationdone? • NLTK ( Natural Language Tool Kit) and SpaCy language models use Regular Expressions (Regex) to create tokens from the running sequence of texts. • NLTK - Penn Treebank Tokenizer , Word Punct Tokenizer , Tweet Tokenizer , MWETokenizer (Multi word Expression Tokenizer) • This is language dependent. • Languages in which white spaces are not present, such as Chinese, Japanese and Korean they use the technique called Word Segmentation. !14Venkatesh Murugadas
- 15. • Problems • Hyphenatedwords - co-operative, self-esteem • URL’s - “https://www.google.com/" • Phone numbers - (541) 754-3010 • Compound nouns (Names , Places) - New York 15Venkatesh Murugadas
- 16. Sentence Segmentation • Itis splitting the running text by detecting the sentence boundary. • Sentence Boundary Detection. • NLTK uses the class Punkt Sentence Tokenizer. This is the most widely used sentence tokenizer. 16Venkatesh Murugadas
- 17. Punkt Architecture Source: UnsupervisedMultilingual Sentence Boundary Detection ( Tibor Kiss, Jan Strunk) Type based Classification (Initials, Ordinal numbers, Texts) 1. Strong Collocation 2. Internal Periods 3. Penalty Token based Classification 1. Orthographic Heuristics - Word shape 2. The Collocation Heuristics 3. Frequent sentence Starter Heuristic 17Venkatesh Murugadas
- 18.
- 19. Parts of SpeechTagging • Part-of-Speech tagging in itself may not be the solution to any particular NLP problem. It is however something that is done as a pre-requisite to simplify a lot of different problems • 8 Parts of Speech in English • There are open classes and closed classes. • Open class - Noun, Verb, Adverb, Adjective • There are languages in which there is no classification of Parts of Speech, such as Riau Indonesian. Korean language do not have Adjectives. NOUN. PRONOUN. VERB. ADJECTIVE. ADVERB. PREPOSITION. CONJUNCTION INTERJECTION. 19Venkatesh Murugadas
- 20. • There are8 to 45 POS tags present. • The mainly used tagged corpora : • Brown corpus - with a million word • Wall Street Journal corpus - with a million word • Switchboard : telephone speech corpus - with 2 million words Source: Speech and Language Processing, Daniel Jurafsky and James H. Martin 20Venkatesh Murugadas
- 21. Source: Speech andLanguage Processing, Daniel Jurafsky and James H. Martin Parts of Speech Tagging algorithm Generative Hidden Markov Model 21Venkatesh Murugadas
- 22. Parts of SpeechTagging algorithm Discriminative Maximum Entropy Markov Model Source: Speech and Language Processing, Daniel Jurafsky and James H. Martin Discriminative model to incorporate a lot of features based on which the classification will be better. There is a feature template. 22Venkatesh Murugadas
- 23. • The modernPOS tagging algorithms use Bidirectional methods. • The Stanford core NLP uses a log- linear Parts of Speech Tagger. • Based on the paper : https:// nlp.stanford.edu/~manning/papers/ tagging.pdf 23Venkatesh Murugadas
- 24. Dependency Parser • Dependencysyntax postulates that syntactic structure consists of lexical items linked by binary asymmetric relations (“arrows”) called dependencies. • The arrows are commonly typed with name of grammatical relations. • So dependencies form a tree (connected, acyclic, single-head) 24Venkatesh Murugadas
- 25. • Shallow parsing(also chunking, "light parsing") is an analysis of a sentence which first identifies constituent parts of sentences (nouns, verbs, adjectives, etc.) and then links them to higher order units that have discrete grammatical meanings (noun groups or phrases, verb groups, etc.). 25Venkatesh Murugadas
- 26. • The StanfordNLP core is based on the paper : https:// nlp.stanford.edu/~sebschu/ pubs/schuster-manning- lrec2016.pdf 26Venkatesh Murugadas
- 27. Information Extraction Architecture Source:https://www.nltk.org/book/ch07.html 27Venkatesh Murugadas
- 28. Named Entity Recognition Named-entityrecognition (NER) (also known as entity identification, entity chunking and entity extraction) is a subtask of information extraction that seeks to locate and classify named entity mentions in unstructured text into pre-defined categories such as the person names, organisations, locations, medical codes, time expressions, quantities, monetary values, percentages, etc. Source: https://en.wikipedia.org/wiki/Named-entity_recognition 28Venkatesh Murugadas
- 29. Noun Phrase Chunking •This is a basic technique used for entity detection. • Each of these larger boxes is called a chunk. • This is done with the help of Regular Expression. (RegEx) Source: https://www.nltk.org/book/ch07.html 29Venkatesh Murugadas
- 30. Shallow parsing (alsochunking, "light parsing") is an analysis of a sentence which first identifies constituent parts of sentences (nouns, verbs, adjectives, etc.) and then links them to higher order units that have discrete grammatical meanings (noun groups or phrases, verb groups, etc.). Image Source: https://www.nltk.org/book/ch07.html Source: https://en.wikipedia.org/wiki/Shallow_parsing 30Venkatesh Murugadas
- 31. • Noun chunkingusing Regular Expression 31Venkatesh Murugadas
- 32. • SpaCy basedNamed Entity Recognition • It is trained on the dataset OntoNotes5. • There are 7 pre-defined categories of Entities. 32 Applications of NER 1. NLU 2. NLS Venkatesh Murugadas
- 33. Discussion ! Venkatesh Murugadas!33 For further queries, contact me at edu.venkateshdas@gmail.com