[runtime] initial support for running model on device

[pilkicTT]

• moves device specific code from tt_torch_device/ into runtime/
• adds TTSystem class (singleton) for holding all info on present devices
• runs mlir compiler as a separate compile stage, which at the end generates flatbuffer binary
• implements CompiledModel class for running inference on compiled model
• run_binary() is the function which invokes tt-mlir runtime

NOTE: with this commit, the following tests are passing:

• pybuda/test/test_api.py
• pybuda/test/mlir/test_ops.py::test_add
• pybuda/test/mlir/test_ops.py::test_multiply

[nsmithtt]

moves device specific code from tt_torch_device/ into runtime/

It seems like tt_torch_device directory still exists, is this intended to stay or will it eventually be removed? Seems a bit confusing to have both, but if this is only intermediate step then seems OK!

[nsmithtt]

It seems like tt_torch_device directory still exists, is this intended to stay or will it eventually be removed? Seems a bit confusing to have both, but if this is only intermediate step then seems OK!

Nvm, it makes sense after taking a look. I think the only thing that's a bit confusing is that there are 2 sets of tt_device.hpp and tt_device.cpp. Maybe we can rename the tt_torch_device/tt_device*? It mostly holds the python bindings so maybe we could call it bindings? Or torch_runtime or something? Just throwing out ideas.

[nsmithtt]

Looks great!!

[pilkicTT]

It seems like tt_torch_device directory still exists, is this intended to stay or will it eventually be removed? Seems a bit confusing to have both, but if this is only intermediate step then seems OK!

Nvm, it makes sense after taking a look. I think the only thing that's a bit confusing is that there are 2 sets of tt_device.hpp and tt_device.cpp. Maybe we can rename the tt_torch_device/tt_device*? It mostly holds the python bindings so maybe we could call it bindings? Or torch_runtime or something? Just throwing out ideas.

Yeah, I was thinking of moving it to under runtime, e.g. runtime/torch - will be done in changes to follow (there is still a lot of refactoring to do 😄).