bug: evaluation with qdrant randomly times out

[konrad-czarnota-ds]

Using evaluation pipeline with qdrant I noticed random errors with timeout. They vary across different runs, sometimes happens just after the start of evaluation, sometimes evaluation almost finishes. Attaching the stacktrace:

[2024-12-16 13:54:44,816][asyncio][ERROR] - Task exception was never retrieved
future: <Task finished name='Task-12' coro=<tqdm_asyncio.gather.<locals>.wrap_awaitable() done, defined at /home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/tqdm/asyncio.py:75> exception=ResponseHandlingException(ConnectTimeout(''))>
Traceback (most recent call last):
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 72, in map_httpcore_exceptions
    yield
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 377, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 256, in handle_async_request
    raise exc from None
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 236, in handle_async_request
    response = await connection.handle_async_request(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpcore/_async/connection.py", line 101, in handle_async_request
    raise exc
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpcore/_async/connection.py", line 78, in handle_async_request
    stream = await self._connect(request)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpcore/_async/connection.py", line 124, in _connect
    stream = await self._network_backend.connect_tcp(**kwargs)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpcore/_backends/auto.py", line 31, in connect_tcp
    return await self._backend.connect_tcp(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 113, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/qdrant_client/http/api_client.py", line 178, in send_inner
    response = await self._async_client.send(request)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1674, in send
    response = await self._send_handling_auth(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1702, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1739, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpx/_client.py", line 1776, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 376, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/httpx/_transports/default.py", line 89, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectTimeout

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/tqdm/asyncio.py", line 76, in wrap_awaitable
    return i, await f
  File "/home/konrad/Desktop/project/evaluation/main.py", line 36, in __call__
    elements = await self.document_search.search(data["question"])
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/ragbits/core/audit/__init__.py", line 106, in wrapper_async
    returned = await func(*args, **kwargs)  # type: ignore
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/ragbits/document_search/_main.py", line 115, in search
    entries = await self.vector_store.retrieve(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/ragbits/core/audit/__init__.py", line 106, in wrapper_async
    returned = await func(*args, **kwargs)  # type: ignore
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/ragbits/core/vector_stores/qdrant.py", line 121, in retrieve
    results = await self._client.query_points(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/qdrant_client/async_qdrant_client.py", line 509, in query_points
    return await self._client.query_points(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/qdrant_client/async_qdrant_remote.py", line 527, in query_points
    query_result = await self.http.points_api.query_points(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/qdrant_client/http/api/points_api.py", line 1193, in query_points
    return await self._build_for_query_points(
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/qdrant_client/http/api_client.py", line 151, in request
    return await self.send(request, type_)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/qdrant_client/http/api_client.py", line 168, in send
    response = await self.middleware(request, self.send_inner)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/qdrant_client/http/api_client.py", line 200, in __call__
    return await call_next(request)
  File "/home/konrad/Desktop/project/.venv/lib/python3.10/site-packages/qdrant_client/http/api_client.py", line 180, in send_inner
    raise ResponseHandlingException(e)
qdrant_client.http.exceptions.ResponseHandlingException

I was able to finish the evaluation after changing this line: pipe_outputs = await tqdm.gather(*[pipeline(data) for data in dataset], desc="Evaluation") to a synchronous loop:

for i, data in enumerate(dataset):
    pipe_outputs.append(await pipeline(data))

so probably there are too many concurrent requests that qdrant couldn't handle (I have 76 datapoints).

[mhordynski]

@micpst needs to check again - it may be fixed in last changes

[micpst]

Can't reproduce it, seems to be fixed. Evaluator now has a concurrency flag that limits the maximum number of concurrent requests. Closing the issue.