Final Deliverables

[error9098x]

• Added Chat History for Streamlit
• Updated Human_Eval wrt new backend
• Added auto eval - chainforge, script based
• Added analysis -
  - discussion directly puts latest discussion dataset from hugging face
  - previously conducted evaluation visualisation
• Added NextJs frontend

[error9098x]

apologies for the last-minute PR. It passed pycheck strict locally, but there are issues with the checks now. I'm currently in-flight to New Zealand for a week-long conference and will try to fix these issues as soon as possible. Let me know if any other updates are needed.

[luarss]

Also you need to signoff your commits. See the DCO section for help.

edit: You need to add the frontend/requirements-test.txt file for .github/workflows/mypy.yaml CI to work.,

  frontend:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: 3.12
      - name: Install dependencies
        run: |
          pip install -r frontend/requirements-test.txt
      - name: Run MyPy
        run: |
          python -m mypy --strict frontend
      - name: Run pre-commit
        run: |
          pre-commit run --files frontend/*

[luarss]

can you rename the dataset files to be separated with _?

instead of Human Eval Dataset.csv -> Human_Eval_Dataset.csv

[luarss]

file rename to README.md

[luarss]

why are there ._***.py files? what are their purposes

[luarss]

Python files should not contain spaces

[luarss]

filename change and put main logic under if __name__ == "__main__":

[luarss]

README.md

[luarss]

What is the default value for this i am wondering. also for production we need to think whether to enable some blocking of harmful outputs.

[error9098x]

What is the default value for this i am wondering. also for production we need to think whether to enable some blocking of harmful outputs.

its default, cause there was issue with the safety settings, sometimes it would block certain questions without having any harmful content which hinders the evaluation script.

[luarss]

That's fine, but I was curious what is the default safety level. I get that we have to disable it for evaluation, but for production we might need it back since we didn't set it for our backend code.

I guess my question is can we make the evaluation mode as close to production mode as close as possible.

[luarss]

any reason these src folders are nested in another folder orassistant-frontend? if not, prefer it to be under nextjs_frontend directly

[luarss]

README.md

[luarss]

There seems to be only one option...

[luarss]

@error9098x Please add the nextJS code in a separate PR. Thanks!

[error9098x]

@error9098x Please add the nextJS code in a separate PR. Thanks!

Sure.