How to mark identifiers like productid (AS12314_11) as entity ?

[vkgpt11]

If i have a query like

Update price for product XA12345 to $10.

How will i be able to get XA12345 as an entity?

Regards
Vikas Gupta

[ines]

spaCy's entity recognizer can be updated with more examples – either to detect more examples of an existing category (like PRODUCT) or to add a new custom category. Here are some resources to get you started:

• spaCy 101: Named entities
• Documentation: Training models
• Documentation: Training the named entity recognizer
• Documentation: Training tips and advice

If your product names follow a consistent scheme (for example, two letters plus 5 numbers), a rule-based approach might actually be a better place to start. Instead of using a statistical model, you'd use the Matcher to find entities in your text, and then add them to the doc.ents property. More info:

• Documentation: Rule-based matching
• Interactive demo: Rule-based Matcher explorer
• Documentation: Custom pipeline components

[vkgpt11]

I have tried that but it is not learning with ID's

Training:

import random
import spacy as spacy
TRAIN_DATA = [
("ASP123124 are too tall and they pretend to care about your feelings", {
'entities': [(0, 9, 'ID')]
}),

("Do they bite?", {
    'entities': []
}),

("12/234525-B are too tall and they pretend to care about your feelings", {
    'entities': [(0, 11, 'ID')]
}),

("23qwe5-B/1232 pretend to care about your feelings", {
    'entities': [(0, 13, 'ID')]
}),

("they pretend to care about your feelings, those Aewr2131242_1", {
    'entities': [(48, 61, 'ID')]
}),

("Aewr2131242_1?", {
    'entities': [(0, 13, 'ID')]
})]

LABEL = 'ID'
n_iter = 20
nlp = spacy.blank('en')
ner = nlp.create_pipe('ner')
nlp.add_pipe(ner)
ner.add_label(LABEL)
optimizer = nlp.begin_training()
other_pipes = [pipe for pipe in nlp.pipe_names if pipe != 'ner']
with nlp.disable_pipes(*other_pipes): # only train NER
for itn in range(n_iter):
random.shuffle(TRAIN_DATA)
losses = {}
for text, annotations in TRAIN_DATA:
nlp.update([text], [annotations], sgd=optimizer, drop=0.35,
losses=losses)
print(losses)

Testing

test the trained model

test_text = 'Do you like ewr214124243_1?'
doc = nlp(test_text)
print("Entities in '%s'" % test_text)
for ent in doc.ents:
print(ent.label_, ent.text)

[ines]

@vkgpt11 Well, the entities are recognised in context and you've given it only very few examples of completely different IDs, in the context of the original "horses" example.

Also note that the code example is really just a simple example that shows how the training loop works. If you really want to train a new category from scratch, you'll need a lot more examples, in real contexts, as similar as possible to the original data you want to use the model on later.

Here are some more details on training data:
https://spacy.io/usage/training#training-data

In your case, it might be better to start off with a rule-based approach first – especially if the IDs follow certain patterns that can be expressed via rules:
https://spacy.io/usage/linguistic-features#section-rule-based-matching