Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for 1024+ dim vectors in CAGRA search #1994

Merged

Conversation

enp1s0
Copy link
Member

@enp1s0 enp1s0 commented Nov 15, 2023

This PR updates the CAGRA search implementation to support 1024+ dim vectors.
For 1024+ dim vectors, the distance between a vector and the query vector is calculated by splitting the vector into multiple 1024 dim vectors and accumulating the distances of each sub-vector.

Rel: #1948

@enp1s0 enp1s0 requested a review from a team as a code owner November 15, 2023 14:02
Copy link

copy-pr-bot bot commented Nov 15, 2023

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@enp1s0 enp1s0 self-assigned this Nov 15, 2023
@github-actions github-actions bot added the cpp label Nov 15, 2023
@enp1s0 enp1s0 added feature request New feature or request non-breaking Non-breaking change labels Nov 15, 2023
@enp1s0
Copy link
Member Author

enp1s0 commented Nov 15, 2023

  • single-CAT
  • multi-CAT
  • multi-kernel
  • test (fix a bug in multi-kernel)

@cjnolet
Copy link
Member

cjnolet commented Nov 16, 2023

/ok to test

@enp1s0 enp1s0 added the DO NOT MERGE Hold off on merging; see PR for details label Nov 16, 2023
@enp1s0
Copy link
Member Author

enp1s0 commented Nov 17, 2023

No significant performance drop is observed
s1024+

@cjnolet
Copy link
Member

cjnolet commented Nov 20, 2023

/ok to test

@enp1s0 enp1s0 removed the DO NOT MERGE Hold off on merging; see PR for details label Nov 21, 2023
@enp1s0 enp1s0 changed the base branch from branch-23.12 to branch-24.02 November 22, 2023 00:52
@enp1s0 enp1s0 added the DO NOT MERGE Hold off on merging; see PR for details label Nov 22, 2023
@cjnolet cjnolet added the Lantern label Dec 2, 2023
@enp1s0 enp1s0 requested a review from tfeher December 4, 2023 09:09
@enp1s0 enp1s0 removed the DO NOT MERGE Hold off on merging; see PR for details label Dec 4, 2023
@cjnolet
Copy link
Member

cjnolet commented Dec 6, 2023

/ok to test

@cjnolet
Copy link
Member

cjnolet commented Dec 8, 2023

/ok to test

Copy link
Member

@cjnolet cjnolet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! This is an important feature and we have folks from Lucene and Lantern waiting on this. These changes look good, but we can follow up if anyone else sees any issues.

@cjnolet
Copy link
Member

cjnolet commented Dec 8, 2023

/merge

@rapids-bot rapids-bot bot merged commit e999100 into rapidsai:branch-24.02 Dec 8, 2023
58 checks passed
Copy link
Contributor

@tfeher tfeher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @enp1s0 for the PR, it looks good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cpp feature request New feature or request Lantern non-breaking Non-breaking change
Projects
Development

Successfully merging this pull request may close these issues.

3 participants