Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

training time #5

Open
youngAt19 opened this issue May 20, 2022 · 4 comments
Open

training time #5

youngAt19 opened this issue May 20, 2022 · 4 comments

Comments

@youngAt19
Copy link

hi, thank you for sharing your codes.
I try to train densecl on imagenet with 4*2080ti, but it takes about 13 days to finish training, which is much longer time than yours.
Did you use imagenet in lmdb format so that training time is shorter?

@CoinCheung
Copy link
Owner

I used 16 tesla T4 gpu amount 2 nodes, and it takes me about 4 days to train it. From my observation of gpu occupance, the data loading is not the bottleneck of training speed.

@youngAt19
Copy link
Author

Thanks for your reply. I thought it just takes you 53 hours to train the model from here

1 similar comment
@youngAt19
Copy link
Author

Thanks for your reply. I thought it just takes you 53 hours to train the model from here

@CoinCheung
Copy link
Owner

This is mocov2 time from the original repo of mocov2.

Training denseCl is time-consuming, I recommand you to use the pretrained model if you are using resnet50/101 backbone. It will save you time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants