Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle performance reports #9

Merged
merged 1 commit into from
Dec 19, 2024
Merged

Handle performance reports #9

merged 1 commit into from
Dec 19, 2024

Conversation

vmilosevic
Copy link
Contributor

@vmilosevic vmilosevic commented Dec 4, 2024

Transform tt-forge-fe performance reports into the format for upload to SFTP
Input performance JSON

{
    "model": "MNIST Linear",
    "config": "",
    "date": "24-12-03",
    "hash": "c47f41a",
    "machine_name": "8cb186cee6d2",
    "samples_per_sec": 0.23979727678872859,
    "total_samples": 1,
    "total_time": 4.170189142227173,
    "training": false,
    "batch_size": 1,
    "output": "forge-benchmark-e2e-mnist_33854708624.json",
    "arch": "",
    "chips": "",
    "device": "",
    "galaxy": "",
    "perf_analysis": "",
    "load_tti": "",
    "save_tti": "",
    "task": "",
    "evaluation_score": ""
}

Output

{
    "run_start_ts": "2024-12-03T14:30:15",
    "run_end_ts": "2024-12-03T14:30:20Z",
    "run_type": "",
    "git_repo_name": null,
    "git_commit_hash": "470c66d36e9392947e0e336923e67de5c8ce5b83",
    "git_commit_ts": null,
    "git_branch_name": "vmilosevic/upload_perf_report",
    "github_pipeline_id": 12141788622,
    "github_pipeline_link": "https://github.com/tenstorrent/tt-forge-fe/actions/runs/12141788622",
    "github_job_id": 33854708624,
    "user_name": "Vladimir Milosevic",
    "docker_image": null,
    "device_hostname": "f10cs11-n150",
    "device_ip": null,
    "device_info": null,
    "ml_model_name": "MNIST Linear",
    "ml_model_type": null,
    "num_layers": null,
    "batch_size": 1,
    "config_params": {},
    "precision": null,
    "dataset_name": null,
    "profiler_name": null,
    "input_sequence_length": null,
    "output_sequence_length": null,
    "image_dimension": null,
    "perf_analysis": null,
    "training": false,
    "measurements": [
        {
            "step_start_ts": "2024-12-03T14:30:15.466000",
            "step_end_ts": "2024-12-03T14:39:31Z",
            "iteration": 0,
            "step_name": "",
            "step_warm_up_num_iterations": null,
            "name": "samples_per_sec",
            "value": 0.23979727678872859,
            "target": null,
            "device_power": null,
            "device_temperature": null
        },
        {
            "step_start_ts": "2024-12-03T14:30:15.466000",
            "step_end_ts": "2024-12-03T14:39:31Z",
            "iteration": 0,
            "step_name": "",
            "step_warm_up_num_iterations": null,
            "name": "total_time",
            "value": 4.170189142227173,
            "target": null,
            "device_power": null,
            "device_temperature": null
        }
    ]
}

@vmilosevic vmilosevic requested a review from vcanicTT December 4, 2024 14:29
Copy link

github-actions bot commented Dec 4, 2024

Coverage

Coverage Report
FileStmtsMissCoverMissing
src
   benchmark.py40392%48–50
   cicd.py64395%85–87
   generate_data.py32778%48–66
   pydantic_models.py970100% 
   test_parser.py16288%33–34
   utils.py94397%97, 150–153
src/parsers
   junit_xml_utils.py47687%85–89, 93–97
   parser.py10280%23, 33
   python_pytest_parser.py68691%18, 51, 54, 80, 130–131
   python_unittest_parser.py610100% 
TOTAL5293294% 

Tests Skipped Failures Errors Time
5 0 💤 0 ❌ 0 🔥 0.669s ⏱️

Copy link

@vcanicTT vcanicTT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vmilosevic vmilosevic merged commit 1d3c452 into main Dec 19, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants