-
Notifications
You must be signed in to change notification settings - Fork 33
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
fix: update harness with link to eleuther docs (#197)
* fix: update harness with link to eleuther docs It does not seem to make a ton of sense if we duplicate the code here. Should probably have one central place to maintain it. * lint
- Loading branch information
Showing
3 changed files
with
3 additions
and
378 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,27 +1,8 @@ | ||
# EleutherAI Harness | ||
|
||
Use Zeno to visualize the data from the [Eleuther LM Evaluation Harness][1]! | ||
There is a notebook for running harness tasks on diferent models and uploading | ||
the results to Zeno. | ||
|
||
You can configure tasks, models, and more directly in the notebook, get a task | ||
list at: | ||
[Task List](https://github.com/EleutherAI/lm-evaluation-harness/blob/master/docs/task_table.md) | ||
|
||
## Setup | ||
|
||
To run this example, first install the requirements: | ||
|
||
```bash | ||
pip install -r requirements.txt | ||
``` | ||
|
||
You'll then need to get an API key from Zeno Hub. | ||
Create an account at [https://hub.zenoml.com](https://hub.zenoml.com) and navigate | ||
to [your account page](https://hub.zenoml.com/account) to get an API key. | ||
Add this key as an environment variable, `ZENO_API_KEY`. | ||
|
||
You can now run the notebook to create a Zeno Project for any of the | ||
harness tasks. | ||
You can find all the details about how to upload harness results to Zeno in the | ||
[lm-evaluation-harness repo][2]. | ||
|
||
[1]: https://github.com/EleutherAI/lm-evaluation-harness | ||
[2]: https://github.com/EleutherAI/lm-evaluation-harness#visualizing-results |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.