Python | Named Entity Recognition (NER) using spaCy

Named Entity Recognition (NER) is used in Natural Language Processing (NLP) to identify and classify important information within unstructured text. These "named entities" include proper nouns like people, organizations, locations and other meaningful categories such as dates, monetary values and products. By tagging these entities, we can transform raw text into structured data that can be analyzed, indexed or used in applications.

Use of spaCy in NER

spaCy is efficient in NLP tasks and is available in Python. It offers:

• Optimized performance: spaCy is built for high-speed text processing making it ideal for large-scale NLP tasks.
• Pre-trained models: It includes various pre-trained NER models that recognize multiple entity types out of the box.
• Ease of use: With a user-friendly API allowing developers to implement NER with minimal effort.
• Deep learning integration: The library works seamlessly with deep learning frameworks like TensorFlow and PyTorch.
• Efficient pipeline processing: It can efficiently handle text processing tasks, including tokenization, part-of-speech tagging, dependency parsing and named entity recognition.
• Customizability: We can train custom models or manually defining new entities.

Implementation of NER using spaCy

Here is the step by step procedure to do NER using spaCy:

1. Install spaCy

We will download spaCy. We will use en_core_web_sm model which is used for english and is a lightweight model that includes pre-trained word vectors and an NER component. spaCy supports various entity types including:

• PERSON – Names of people
• ORG – Organizations
• GPE – Countries, cities, states
• DATE – Dates and time expressions
• MONEY – Monetary values
• PRODUCT – Products and brand names
• EVENT – Events (e.g., "Olympics")
• LAW – Legal documents

A full list of entity types can be found in the spaCy documentation.

!pip install spacy
!python - m spacy download en_core_web_sm

The following code demonstrates how to perform NER using spaCy:

• spacy.load("en_core_web_sm") loads the pre-trained English model.
• nlp(text) processes the input text and tokenizes it.
• doc.ents contains all recognized named entities.

import spacy
nlp = spacy.load('en_core_web_sm')
sentence = "Why Apple is looking at buying U.K. startup for $1 billion ?"
doc = nlp(sentence)

for ent in doc.ents:
    print(ent.text, ent.start_char, ent.end_char, ent.label_)

Output:

Apple 4 9 ORG
U.K. 31 35 GPE
$1 billion 48 58 MONEY

Here Apple is classified as an Organization (ORG), U.K. as a Geopolitical Entity (GPE) and $1 billion as Money (MONEY).

3. Effect of Case Sensitivity

Here we examine how capitalization affects entity recognition. Lowercasing an entity name may prevent it from being recognized correctly.

sentence = "Why apple is now looking at buying U.K. startup for $1 billion ?"
doc = nlp(sentence)

for ent in doc.ents:
    print(ent.text, ent.start_char, ent.end_char, ent.label_)

Output:

U.K. 35 39 GPE
$1 billion 52 62 MONEY

Since "apple" is in lowercase it is no longer recognised as an organization.

4. Customizing Named Entity Recognition

Here we manually add a new named entity to spaCy's output. This technique is useful when you want to recognize specific terms that are not in the pre-trained model.

• We use Span to define the new entity.
• The entity is added to doc.ents to update the output.

from spacy.tokens import Span
doc = nlp("Tesla is planning to launch a new product.")

custom_label = "ORG"
doc.ents = (Span(doc, 0, 1, label=custom_label),)

for ent in doc.ents:
    print(ent.text, ent.label_)

Output:

Tesla ORG

Here "Tesla" was manually added as an organization. In a full NER training setup you can retrain the model using annotated datasets.

Named Entity Recognition (NER) is an essential tool for extracting valuable insights from unstructured text for better automation and analysis across industries. spaCy’s flexible capabilities allow developers to quickly implement and customize entity recognition for specific applications. It also offers an efficient and scalable solution for handling named entity recognition in real-world text processing.

You can download source code from here.