-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fuzzy assign bio tags #22
base: master
Are you sure you want to change the base?
Conversation
…names_filename automatically
…less memory efficient)
Dropping it gives a nice speedup because computations are now in Cython.
…simplify code and make it faster. If needed, it can be implemented as a global feature.
After reading the WebAnnotator docs people might get an idea "export" format is preferred which is wrong.
webstruct/utils.py
Outdated
if x != 'O': | ||
return x | ||
return 'O' | ||
return [reduce(select_tag, i) for i in zip(*tag_list)] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can it be implemented without reduce? :) Also, I think it can produce invalid sequences of tags, i.e. what will happen with [B-ORG, I-ORG, I-ORG] + [O, B-ADDR, O]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why not reduce? also the tags should not clashes, i added an assertion.
7cb7cc3
to
79460bf
Compare
this can be useful to generate training data when partialy labeled data is avaiable
79460bf
to
025c782
Compare
allow assign the BIO tags with fuzzy match. this can be useful when having some of partially labelled dataset.