How to run HeatMap with your best pretrained NABirds model?

[LanceBao0313]

Thank you so much for the beautiful code.
I'm trying to use your pretrained model best.pt on NABirds dataset.

First, I set the PATH to the pretrained model in NABirds_SwinT.yaml
then I run:

python heat.py --c ./configs/NABirds_SwinT.yaml --img ./vis/001.jpg --save_img ./vis/001/

But I get errors:

---

Building...
Traceback (most recent call last):
File "C:\Users\xxxxxx\Desktopxxxxxx\heat.py", line 100, in
model.load_state_dict(checkpoint['model'])
File "C:\Users\xxxxxx\anaconda3\envs\xxxxxxx\lib\site-packages\torch\nn\modules\module.py", line 1667, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for PluginMoodel:
Unexpected key(s) in state_dict: "combiner.conv_qk1.weight", "combiner.conv_qk1.bias".
size mismatch for combiner.adj1: copying a param with shape torch.Size([85, 85]) from checkpoint, the shape in current model is torch.Size([15, 15]).
size mismatch for combiner.param_pool0.weight: copying a param with shape torch.Size([85, 2720]) from checkpoint, the shape in current model is torch.Size([15, 480]).
size mismatch for combiner.param_pool0.bias: copying a param with shape torch.Size([85]) from checkpoint, the shape in current model is torch.Size([15]).
size mismatch for combiner.param_pool1.weight: copying a param with shape torch.Size([1, 85]) from checkpoint, the shape in current model is torch.Size([1, 15]).

---

It seems that your best,pt model is not using default SwinT model, the num_selects is not matching
And there are unexpected keys in your best.pt model: "combiner.conv_qk1.weight", "combiner.conv_qk1.bias"

I wish to know what modification I should make to load your pretrained best.pt.

[LanceBao0313]

I managed to make it work by :

1. Change num_selects in NABirds_SwinT.yaml to
   num_selects:
   layer1: 2048
   layer2: 512
   layer3: 128
   layer4: 32
2. Delete keys "combiner.conv_qk1.weight" and "combiner.conv_qk1.bias" in best.pt model

Although the heatmap is now output as normal, I am concerned that this will affect the performance of the model. I hope to know a better solution.