You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I noticed that there are two formats for the evaluation data - one flat and the other with an options section. When I try to run the evaluation with the neptune callback, I get an exception. We should choose one layout and fix the code for neptune.
To reproduce run example from examples/evaluation/document-search/evaluate.py and flip neptune.run flag in examples/evaluation/document-search/config/retrieval.yaml
How can we reproduce it?
No response
Relevant log output
Traceback (most recent call last):
File "/Users/michal/repos/ragbits/examples/evaluation/document-search/evaluate.py", line 63, in main
asyncio.run(bench(config))
File "/Users/michal/.local/share/uv/python/cpython-3.10.15-macos-aarch64-none/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/Users/michal/.local/share/uv/python/cpython-3.10.15-macos-aarch64-none/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
returnfuture.result()
File "/Users/michal/repos/ragbits/examples/evaluation/document-search/evaluate.py", line 33, in bench
run = setup_neptune(config)
File "/Users/michal/repos/ragbits/packages/ragbits-evaluate/src/ragbits/evaluate/utils.py", line 110, in setup_neptune
config.data.name,
omegaconf.errors.ConfigAttributeError: Key 'name' is not in struct
full_key: data.name
object_type=dict
The text was updated successfully, but these errors were encountered:
What happened?
I noticed that there are two formats for the evaluation data - one flat and the other with an
options
section. When I try to run the evaluation with the neptune callback, I get an exception. We should choose one layout and fix the code for neptune.To reproduce run example from
examples/evaluation/document-search/evaluate.py
and flipneptune.run
flag inexamples/evaluation/document-search/config/retrieval.yaml
How can we reproduce it?
No response
Relevant log output
The text was updated successfully, but these errors were encountered: