spsa-prune

git clone https://github.com/fabianwinter93/spsa-prune
cd spsa-prune


python -m venv .venv
source .venv/bin/activate

pip install -r requirements.txt

Relevant Papers

Simultaneous perturbation stochastic approximation

MeZO: Fine-Tuning Language Models with Just Forward Passes

ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and Uncertainty in Zeroth-Order Optimization

MINI-LLM: Memory-Efficient Structured Pruning for Large Language Models