Build and explore efficient retrieval-augmented generative models and applications
-:round_pushpin: Installation • :rocket: Components • :books: Examples • :red_car: Getting Started • :pill: Demos • :pencil2: Scripts • :bar_chart: Benchmarks + + +:round_pushpin: Installation • :rocket: Components • :books: Examples • :red_car: Getting Started • :pill: Demos • :pencil2: Scripts • :bar_chart: Benchmarks @@ -105,13 +107,21 @@ Preliminary requirements: - **Python** 3.8 or higher. - **PyTorch** 2.0 or higher. -To set up the software, clone the project and run the following, preferably in a newly created virtual environment: +To set up the software, install from `pip` or clone the project for the bleeding-edge updates. Run the following, preferably in a newly created virtual environment: + +via `pip` pypi: + +```bash +pip install fastrag +``` + +or from a local clone: ```bash pip install . ``` -There are several dependencies to consider, depending on your specific usage: +There are several dependencies to consider, depending on your specific usage (also works with `pip install fastrag[*]` package): ```bash # Additional engines/components @@ -122,9 +132,6 @@ pip install .[colbert] # Support for ColBERT+PLAID; requires FAISS pip install .[faiss-cpu] # CPU-based Faiss library pip install .[faiss-gpu] # GPU-based Faiss library -# Benchmarking -pip install .[benchmark] - # Development tools pip install .[dev] ``` diff --git a/benchmarks/README.md b/benchmarks/README.md index 8f7215d..ac6c429 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -4,6 +4,15 @@ We provide scripts for running some well known benchmarks with fastRAG. The goal We report results as compared with the original benchmark papers. +## Installation + +Running the benchmarks require some additional packages: + +``` sh +pip install beir +pip install git+https://github.com/facebookresearch/KILT.git +``` + ## :beers: BeIR Benchmark [BeIR](https://github.com/beir-cellar/beir) contains diverse IR tasks ([Thakur et al. 2021](#org29ef2fc)); here we focus on MSMARCO and NaturalQuestions (NQ). For each task, we test two pipelines: diff --git a/fastrag/__init__.py b/fastrag/__init__.py index 20eb0a1..ea20565 100644 --- a/fastrag/__init__.py +++ b/fastrag/__init__.py @@ -1,4 +1,4 @@ ### Essential imports for loading components and modifications to original Haystack components from fastrag import generators, rankers, retrievers, stores -__version__ = "3.0.0" +__version__ = "3.0.1" diff --git a/setup.cfg b/setup.cfg index 3121209..2d95f4d 100644 --- a/setup.cfg +++ b/setup.cfg @@ -1,6 +1,4 @@ [options] -packages = find: - install_requires = haystack-ai==2.1.2 transformers>=4.35.2 @@ -29,10 +27,6 @@ dev = pytest pre-commit -benchmark = - beir - kilt @ git+https://github.com/facebookresearch/KILT.git - elastic = elasticsearch-haystack diff --git a/setup.py b/setup.py index f4b0337..ffee518 100644 --- a/setup.py +++ b/setup.py @@ -1,14 +1,13 @@ -import codecs import os import pathlib -from setuptools import setup +from setuptools import find_namespace_packages, find_packages, setup here = pathlib.Path(__file__).parent.resolve() def read(rel_path): - with codecs.open(os.path.join(here, rel_path), "r") as fp: + with open(os.path.join(here, rel_path), "r", encoding="utf-8") as fp: return fp.read() @@ -21,15 +20,22 @@ def get_version(rel_path): raise RuntimeError("Unable to find version string.") -# Get the long description from the README file -long_description = (here / "README.md").read_text(encoding="utf-8") - setup( name="fastrag", author="Intel Labs", version=get_version("fastrag/__init__.py"), - description="A research framework for building and evaluating neural information retrieval and generative models.", - long_description=long_description, + packages=find_namespace_packages(include=["fastrag*"]), + description="An Efficient Retrieval Augmentation and Generation Framework for Intel Hardware.", + long_description=read("README.md"), long_description_content_type="text/markdown", + url="https://github.com/IntelLabs/fastRAG", + license="Apache-2.0", python_requires=">=3.8, <4", + classifiers=[ + "Programming Language :: Python :: 3", + "License :: OSI Approved :: Apache Software License", + "Operating System :: OS Independent", + "Topic :: Scientific/Engineering :: Artificial Intelligence", + "Topic :: Software Development :: Libraries", + ], )