InterProt

This repo contains tools for interpreting protein language models using sparse autoencoders (SAEs). Our SAE visualizer is available at interprot.com and our SAE models weights are on HuggingFace. For more information, check out our preprint.

viz contains the frontend app for visualizing SAE features. interprot is a Python package for SAE training, evaluation, and interpretation.

Getting started with our SAEs

Check out this demo notebook for SAE inference with a custom input sequence.

The visualizer

The visualizer is a React app with some RunPod serverless functions that serve our SAEs.

Running the visualizer locally

cd viz
pnpm install
pnpm run dev

RunPod endpoints

The RunPod serverless functions live in their own repos:

• SAE inference: https://github.com/liambai/sae-inference
• SAE steering: https://github.com/liambai/sae-steering

Running and developing the Python package

Setting up pre-commit

pip install pre-commit
pre-commit install

Building and running the Docker container

docker compose build
docker compose run --rm interprot bash