FIX: Enable non-strict loading of state dicts

[BenjaminBossan]

What does this PR do?

Resolves #278

PyTorch allows to load state dicts with they strict=False argument to ignore missing keys. This is now also supported in optimum-quanto. Before this fix, a KeyError would be raised.

One context where this is important is for parameter-efficient fine-tuning adapters such as LoRA. There, we want to load only a small subset of parameters and leave the other model weights untouched. This requires non-strict loading.

Before submitting

• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you run all tests locally and make sure they pass. Only subset
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[BenjaminBossan]

@dacorvo I created the PR to fix non-strict loading. As the code changed a bit compared to what I had on the issue, and since I wanted to support int4 and int8, the changes are a bit different from what we discussed there. LMK if something should be changed or is still missing.

Apparently the commit is not "conventional", not sure what I missed there. Should I try to fix this via rebase or can this be fixed later via squash+merge?

[dacorvo]

Apparently the commit is not "conventional", not sure what I missed there. Should I try to fix this via rebase or can this be fixed later via squash+merge?

You can amend your commit to fix: Enable non-strict loading of state dicts to make it conventional and force-push.

[BenjaminBossan]

You can amend your commit to fix: Enable non-strict loading of state dicts to make it conventional and force-push.

Thanks, done.

[dacorvo]

Thank you very much for this neat pull-request. Looking forward to see how you used quanto in peft !