Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make global soln indices deterministic #1599

Merged
merged 2 commits into from
Jan 30, 2025
Merged

Make global soln indices deterministic #1599

merged 2 commits into from
Jan 30, 2025

Conversation

b-shi
Copy link
Contributor

@b-shi b-shi commented Jan 27, 2025

Currently, global solution indices can change between different builds. This PR sorts the global solution indices to make them deterministic.

@KKyang
Copy link
Contributor

KKyang commented Jan 27, 2025

What happens when a solution is removed in the middle?

@b-shi
Copy link
Contributor Author

b-shi commented Jan 27, 2025

What happens when a solution is removed in the middle?

Do you mean when we delete duplicate solutions? In that case it should fine, since, afaik, that step is done in serial. The non-determinism comes from creating the master library in parallel.

@KKyang
Copy link
Contributor

KKyang commented Jan 30, 2025

What happens when a solution is removed in the middle?

Do you mean when we delete duplicate solutions? In that case it should fine, since, afaik, that step is done in serial. The non-determinism comes from creating the master library in parallel.

No, I mean if a solution is replaced by a faster solution, or it is removed due to new optimizations that makes another existing kernel faster. You noticed that indices change between builds and you want to make them deterministic.

for localIdx, _, s in libraryIter(masterLibrary):
matchTable[s.index] = [s.srcName, localIdx]
LibraryIO.write("MatchTable", matchTable)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the MatchTable be written out after solution sorting?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, good catch, thanks!

@b-shi
Copy link
Contributor Author

b-shi commented Jan 30, 2025

What happens when a solution is removed in the middle?

Do you mean when we delete duplicate solutions? In that case it should fine, since, afaik, that step is done in serial. The non-determinism comes from creating the master library in parallel.

No, I mean if a solution is replaced by a faster solution, or it is removed due to new optimizations that makes another existing kernel faster. You noticed that indices change between builds and you want to make them deterministic.

Oh, I meant for the same commit (no changes to library logic files) if we rebuild multiple times the global solution indices may be different. We can see this by checking the contents of MatchTable.yaml it will change whenever we rebuild.

@b-shi b-shi merged commit d9063a8 into ROCm:develop Jan 30, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants