perform bad in test dataset

[Comet2dh]

it's so strange that i got this result:


it looks like there is an overfitting when i train the model, but i didn't change anything(only some ajustments with tensorboard).

[mli0603]

Hi @Comet2dh

This is not overfitting but very likely a problem of normalization.

• Have you seen fix BN issue due to torch version #8 #49? I did a recent PR to fix this issue.
• Are you on the latest code? If so, which torch version are you using? I can look into this.

[Comet2dh]

Thanks for your reply! @mli0603
but it seems not to be the case you mentioned, in fact,

1. I downloaded the latest release of your code, so the 'BN issue' is not the problem.
2. my torch version is 1.6.0, and my cuda version is 10.1.

[mli0603]

This is odd. Do you mind sending me your checkpoint so I can take a look?

[Comet2dh]

Sure. Thanks for your help. This is my checkpoint link:
https://drive.google.com/file/d/1lRkw-NgxgCMO4zG4TNw13hWngHNoF6Ec/view?usp=sharing

[mli0603]

Hi @Comet2dh

When I inference with your checkpoint, the result is much better than what you have. I used both pytorch 1.5.1, 1.6.0 and 1.9.0. They gave me the same results.

I am still looking into what is going on. Meanwhile, do you mind directly running inference with your checkpoint and see if they agree with the tensorboard stats? I wonder if somethig is odd with the validation logic.