Why only tune parameters in certain layers

[pengshancai]

Dear authors,
I have the following questions about the code:

1. Why are certain parameters (e.g. params in the 6th and 7th layers) being fine-tuned, is it a concern on training speed?
2. The code only only provide training using news titles, is there any concern on using abstracts or full passage? (e.g. training time)?
   Thanks!

[wuch15]

Hi all, thanks for the comment. Sorry that one line of code is missing here. Please set config.num_hidden_layers = 8 to ensure that first 8 layers are used. This is because in practice we find fine-tuning the first 8 layers is slightly better than fine-tuning the entire model (though may not be always true for all kinds of models). Thus, the 7th and 8-th layers (numbers 6 and 7) are finetuned.