Pre-train in forward mode and use it as a starting point for inverse parameter estimation

[annien094]

Hello,

We had this idea of pre-training a model in forward mode first, save it, and then load it and use it as a starting point for inverse parameter estimation. We wanted to do this as we observed bumps and dips, large deflections from the ground truth before the parameter estimates could stabalise (see image below for example). We thought this is because the NN was trying to do two things at a time, and if we pre-train it such that it already works relatively well in forward mode, that could save some time for any further inverse parameter estimation tasks.

However, it doesn't seem to be possible at the moment as the network configurations for forward and inverse mode are different: loading a forwardly trained NN and start an inverse mode training results in error such as "Key Variable not found in checkpoint", whilst loading an inversely trained NN and continue in inverse mode works fine.

I was wondering if there may be any workaround for this, or would it be some feature that might be of interest? Would appreciate any thoughts on this. Thank you so much.

Annie

[lululxvi]

Add the trainable variable also for forward code.