ADR for IBM Granite Embeddings

[jwm4]

This ADR proposes that we will use the IBM Granite Embeddings model for the vector retrieval component of our RAG solution.

[jwm4]

I received some comments about this proposal in a direct message from @akashgit . The pull request is updated with additional text to address the concerns that he raised.

[jwm4]

@instructlab/oversight-committee , this has approval from relevant stakeholders and no request for more changes so I think it is ready for final oversight now. Can someone review this and if it meets the criteria merge it too?

[danmcp]

Please squash commits to remove the intermediate changes

[jwm4]

Squashing is complete.

[danmcp]

@jwm4 Re:

I received some comments about this proposal in a direct message from @akashgit . The pull request is updated with additional text to address the concerns that he raised.

Are you confident these concerns were addressed or do we need @akashgit to re-review?

[jwm4]

I received some comments about this proposal in a direct message from @akashgit . The pull request is updated with additional text to address the concerns that he raised.

Are you confident these concerns were addressed or do we need @akashgit to re-review?

@akashgit was clear that he doesn't want to be the blocker, so I don't think we need to wait for a re-review. I let him know on January 9 that I had updated the text to address the issues he raised, so if he wanted to submit a formal review in github, I think he would have by now.

He is concerned that we don't have enough data to be really data driven on this issue. I agree with that assessment and I updated the Consequences section to be more explicit about the risk that we will need to pivot to another default as more data emerges. I think that's an acceptable risk because changing the default embedding model is a fairly low cost operation.

I think it is time to make a decision now for what we will do now and then pivot later if it becomes clear that there are better options available. I also think assessing the trade-offs between issues like hardware requirements, accuracy, IP/legal risk, etc. will always be very subjective. I am hoping the "key considerations" listed in this document will provide a framework for making these subjective judgements long after the specific decision about which model is the best one for January 2025 has become irrelevant.

[danmcp]

Thanks for the explanation!