Skip to content

Commit

Permalink
fix: update harness with link to eleuther docs (#197)
Browse files Browse the repository at this point in the history
* fix: update harness with link to eleuther docs

It does not seem to make a ton of sense if we duplicate the code here. Should probably have one central place to maintain it.

* lint
  • Loading branch information
Sparkier authored Dec 21, 2023
1 parent 1c4e623 commit d0fef2f
Show file tree
Hide file tree
Showing 3 changed files with 3 additions and 378 deletions.
25 changes: 3 additions & 22 deletions examples/eleuther_harness/README.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,8 @@
# EleutherAI Harness

Use Zeno to visualize the data from the [Eleuther LM Evaluation Harness][1]!
There is a notebook for running harness tasks on diferent models and uploading
the results to Zeno.

You can configure tasks, models, and more directly in the notebook, get a task
list at:
[Task List](https://github.com/EleutherAI/lm-evaluation-harness/blob/master/docs/task_table.md)

## Setup

To run this example, first install the requirements:

```bash
pip install -r requirements.txt
```

You'll then need to get an API key from Zeno Hub.
Create an account at [https://hub.zenoml.com](https://hub.zenoml.com) and navigate
to [your account page](https://hub.zenoml.com/account) to get an API key.
Add this key as an environment variable, `ZENO_API_KEY`.

You can now run the notebook to create a Zeno Project for any of the
harness tasks.
You can find all the details about how to upload harness results to Zeno in the
[lm-evaluation-harness repo][2].

[1]: https://github.com/EleutherAI/lm-evaluation-harness
[2]: https://github.com/EleutherAI/lm-evaluation-harness#visualizing-results
353 changes: 0 additions & 353 deletions examples/eleuther_harness/harness.ipynb

This file was deleted.

3 changes: 0 additions & 3 deletions examples/eleuther_harness/requirements.txt

This file was deleted.

0 comments on commit d0fef2f

Please sign in to comment.