search

ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware [arXiv] [Poster]

@inproceedings{
  cai2018proxylessnas,
  title={Proxyless{NAS}: Direct Neural Architecture Search on Target Task and Hardware},
  author={Han Cai and Ligeng Zhu and Song Han},
  booktitle={International Conference on Learning Representations},
  year={2019},
  url={https://arxiv.org/pdf/1812.00332.pdf},
}

News

• Next generation of ProxylessNAS: Once-for-All (First place in the 3rd and 4th Low-Power Computer Vision Challenge).
• First place in the Visual Wake Words Challenge, TF-lite track, @CVPR 2019
• Third place in the Low Power Image Recognition Challenge (LPIRC), classification track, @CVPR 2019

Performance

Without any proxy, directly and efficiently search neural network architectures on your target task and hardware!

Mobile settings	GPU settings
	Model Top-1 Top-5 Latency MobilenetV2 72.0 91.0 6.1ms ShufflenetV2(1.5) 72.6 - 7.3ms ResNet-34 73.3 91.4 8.0ms MNasNet(our impl) 74.0 91.8 6.1ms ProxylessNAS (GPU) 75.1 92.5 5.1ms
ProxylessNAS(Mobile) consistently outperforms MobileNetV2 under various latency settings.	ProxylessNAS(GPU) is 3.1% better than MobilenetV2 with 20% faster.

Specialization

People used to deploy one model to all platforms, but this is not good. To fully exploit the efficiency, we should specialize architectures for each platform.

Requirements

• Pytorch 1.0
• Python 3.6+

# Train a model: <path>/run.config; <path>/net.config
python python imagenet_run_exp.py --path <path> --train

# Train Proxyless (GPU)
python imagenet_run_exp.py --path Exp/proxyless_gpu --train --net proxyless_gpu --dropout 0.3

# Train Proxyless (CPU)
python imagenet_run_exp.py --path Exp/proxyless_cpu --train --net proxyless_cpu --dropout 0.2

# Train Proxyless (Mobile)
python imagenet_run_exp.py --path Exp/proxyless_mobile --train --net proxyless_mobile --dropout 0.1

# Train Proxyless (Mobile-14)
python imagenet_run_exp.py --path Exp/proxyless_mobile_14 --train --net proxyless_mobile_14 --dropout 0.3

# Eval Proxyless (GPU)
python imagenet_run_exp.py --path Exp/proxyless_gpu --net proxyless_gpu

# Eval Proxyless (CPU)
python imagenet_run_exp.py --path Exp/proxyless_cpu --net proxyless_cpu

# Eval Proxyless (Mobile)
python imagenet_run_exp.py --path Exp/proxyless_mobile --net proxyless_mobile

# Eval Proxyless (Mobile-14)
python imagenet_run_exp.py --path Exp/proxyless_mobile_14 --net proxyless_mobile_14

# Architecture Search
please refer to imagenet_arch_search.py