Feat (llm): benchmark for llm entrypoint

[pablomlago]

Reason for this PR

Enable running the LLM entrypoint in a grid-seach manner for a subset of argument combinations specified through a YAML file. Moreover, workloads can be divided between different process, each of them running the LLM entrypoint in a mutually exclusive set of GPUs.

Changes Made in this PR

Refactored the argument parsing logic of the LLM entrypoint, moving it to a separate file, thus enabling to leverage the LLM parser before doing any import to torch, which would cause the CUDA devices to be initialized. By doing so, the cuda initialization takes into account the different values for CUDA_VISIBLE_DEVICES that each process sets. Also, results from the LLM entrypoint are now grouped into a single dictionary to facilitate retrieving them.

Testing Summary

Ran the benchmark script locally.

Risk Highlight

• This PR includes code from another work (please detail).
• This PR contains API-breaking changes.
• This PR depends on work in another PR (please provide links/details).
• This PR introduces new dependencies (please detail).
• There are coverage gaps not covered by tests.
• Documentation updates required in subsequent PR.

Checklist

• Code comments added to any hard-to-understand areas, if applicable.
• Changes generate no new warnings.
• Updated any relevant tests, if applicable.
• No conflicts with destination dev branch.
• I reviewed my own code changes.
• Initial CI/CD passing.
• 1+ reviews given, and any review issues addressed and approved.
• Post-review full CI/CD passing.