Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BART summarizer to improve BERT results #5

Open
CypherousSkies opened this issue Sep 27, 2021 · 1 comment
Open

BART summarizer to improve BERT results #5

CypherousSkies opened this issue Sep 27, 2021 · 1 comment
Assignees
@CypherousSkies
Labels
enhancement New feature or request
Milestone
Long Term

Comments

@CypherousSkies
Copy link
Owner

DistilBERT can only take 512 tokens which is not enough for most passages.
So... maybe using a summarizer to give context outside of the sentence with a mask token to improve results.
Huggingface has a DistilBART model that I could use, but that would only increase the (already considerable) runtime.
Maybe make it optional?
@CypherousSkies CypherousSkies added the enhancement New feature or request label Sep 27, 2021
@CypherousSkies CypherousSkies added this to the Long Term milestone Sep 27, 2021
@CypherousSkies CypherousSkies self-assigned this Sep 27, 2021
@CypherousSkies
Copy link
Owner Author

This could be replaced by TrOCR (#13), although it might be neat to test which of these methods is faster.

@CypherousSkies CypherousSkies modified the milestones: Long Term, 0.1.0 Oct 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@CypherousSkies CypherousSkies

Labels
enhancement New feature or request
Projects
None yet
Milestone
Long Term
Development

No branches or pull requests
1 participant
@CypherousSkies