You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the "Attentional Interfaces" section, there is a reference to "Bahdanau, et al. 2014: Neural machine translation by jointly learning to align and translate" (figure). In that paper, the attention vector is calculated through a feed-forward network, using the hidden states of the encoder and decoder as input (this is called "additive attention"). However, the schematic diagram of this section shows that the attention vector is calculated by using the dot product between the hidden states of the encoder and decoder (which is known as multiplicative attention). I believe that a short mention / clarification would be of benefit here.
The text was updated successfully, but these errors were encountered:
In the "Attentional Interfaces" section, there is a reference to "Bahdanau, et al. 2014: Neural machine translation by jointly learning to align and translate" (figure). In that paper, the attention vector is calculated through a feed-forward network, using the hidden states of the encoder and decoder as input (this is called "additive attention"). However, the schematic diagram of this section shows that the attention vector is calculated by using the dot product between the hidden states of the encoder and decoder (which is known as multiplicative attention). I believe that a short mention / clarification would be of benefit here.
The text was updated successfully, but these errors were encountered: