Question about potentially incomplete usage of 'logits_mask'

[ruohuali]

Hi @HobbitLong,
Thanks for this great work!
My question is that in these two lines of loss

SupContrast/losses.py

Line 88 in 331aab5

exp_logits = torch.exp(logits) * logits_mask

SupContrast/losses.py

Line 89 in 331aab5

log_prob = logits - torch.log(exp_logits.sum(1, keepdim=True))

the logit_mask are only applied to exp_logits and not logits themselves.
I cannot figure out the reason for this mathematically, can you please shed some light?

[forgotton-wind]

I have also been studying the code for this loss today, and I have come to understand that it works like this:
logits_mask is used to get the denominator(positives and negatives).
mask is used to get the numerator(positives).
You can see that in line 92, mask is applied to log_prob.

SupContrast/losses.py

Line 92 in 331aab5

mean_log_prob_pos = (mask * log_prob).sum(1) / mask.sum(1)