Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance the Model analysis script #981

Closed
chandrasekaranpradeep opened this issue Dec 30, 2024 · 2 comments · Fixed by #1059
Closed

Enhance the Model analysis script #981

chandrasekaranpradeep opened this issue Dec 30, 2024 · 2 comments · Fixed by #1059
Assignees
Labels
enhancement New feature or request

Comments

@chandrasekaranpradeep
Copy link
Contributor

chandrasekaranpradeep commented Dec 30, 2024

  1. Add a feature to the model analysis script to identify and sort issues for each component (FFE, MLIR, TTNN) based on the number of models they block. This analysis will helps to prioritize which issues to focus first. Info will be updated in the model_analysis_docs/ModelsInfo.md
  2. Sort the model present in the model_analysis_docs/ModelsInfo.md in alphabetical order
  3. Add the statistics about the number of models pass in each compiler component, Total number of models across the framework, number of models per framework, average passing rate across models for each component at the beginning of the model_analysis_docs/ModelsInfo.md file
  4. Instead of updating the date and time in the rows of the table in model_analysis_docs/ModelsInfo.md, update the data and time info above the table
  5. Include the commit id on which the model analysis script is triggered.
  6. Include row ids for the model_analysis_docs/ModelsInfo.md file and other sub markdown files
  7. Add error message for the UNKNOWN Compiler Component or N/A in addition to the question mark symbol.
@chandrasekaranpradeep chandrasekaranpradeep added the enhancement New feature or request label Dec 30, 2024
@chandrasekaranpradeep chandrasekaranpradeep self-assigned this Dec 30, 2024
@chandrasekaranpradeep chandrasekaranpradeep changed the title Component-Wise Issue Analysis with Model Blocking Impact Enhance the Model analysis script Dec 30, 2024
@nvukobratTT
Copy link
Contributor

Thanks! Regarding this point:

Add a feature to the model analysis script to identify and sort issues for each component (FFE, MLIR, TTNN) based on the number of models they block. This analysis will helps to prioritize which issues to focus first. The results should be generated as an XML file and uploaded as a model analysis artifact.

See if we need to generate additional XML. If we can cleanly represent all information in our main model view, we don't need to rely on having custom XML file as well.

@chandrasekaranpradeep
Copy link
Contributor Author

See if we need to generate additional XML. If we can cleanly represent all information in our main model view, we don't need to rely on having custom XML file as well.

I think we can represent the info in the model_analysis_docs/ModelsInfo.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants