Make wrapper classes for making ML models compatible with tool

[KristofferSkare]

Not every ML model / onnx file is compatible with the mlfmu tool and i can be a bit hard/confusing to make an incompatible model compatible.

To make this process easier for our users we can make example classes in tensorflow and pytorch that they can use and extend to make a compatibility wrapper for their trained model.

The class could have implemented some functions and give the user an idea of what they should do to make the wrapper.

class TensorflowWrapper(tensorflow.keras.Model):
   def __init__(self, trained_model):
        self.trained_model = trained_model
    def format_inputs(self, inputs, states, time):
        return 
    def format_outputs(self, inputs, states, time, model_outputs):
         return
    def call(self, x):
           inputs, states, time = x
           formatted_inputs = self.format_inputs(inputs, state, time)
           model_outputs = self.trained_model(formatted_inputs)
           return self.format_outputs(inputs, states, time, model_outputs)

class PytorchWrapper(torch.nn.Modulel):
   def __init__(self, trained_model):
        self.trained_model = trained_model
    def format_inputs(self, inputs, states, time):
        return 
    def format_outputs(self, inputs, states, time, model_outputs):
         return
    def forward(self, x):
           inputs, states, time = x
           formatted_inputs = self.format_inputs(inputs, state, time)
           model_outputs = self.trained_model(formatted_inputs)
           return self.format_outputs(inputs, states, time, model_outputs)

In easy cases just implementing the format_inputs and format_outputs classes could be sufficient to make the compatibility wrapper for their model. For more complex cases one would maybe also need to modify more, but this could be a good starting point for our users.

If we include this we should also include tools for loading onnx files into pytorch/tensorflow and converting pytorch/tensorflow to onnx files. That way a user could go the entire way from an incompatible onnx file to an compatible onnx to a compiled fmu. All within the this tool.

[ClaasRostock]

[like] Rostock, Claas reacted to your message:

…

________________________________ From: KristofferSkare ***@***.***> Sent: Friday, March 15, 2024 2:45:49 PM To: dnv-innersource/mlfmu ***@***.***> Cc: Subscribed ***@***.***> Subject: [dnv-innersource/mlfmu] Make wrapper classes for making ML models compatible with tool (Issue #30) Caution: This email originated from outside of the DNV Organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. Not every ML model / onnx file is compatible with the mlfmu tool and i can be a bit hard/confusing to make an incompatible model compatible. To make this process easier for our users we can make example classes in tensorflow and pytorch that they can use and extend to make a compatibility wrapper for their trained model. The class could have implemented some functions and give the user an idea of what they should do to make the wrapper. class TensorflowWrapper(tensorflow.keras.Model): def __init__(self, trained_model): self.trained_model = trained_model def format_inputs(self, inputs, states, time): return def format_outputs(self, inputs, states, time, model_outputs): return def call(self, x): inputs, states, time = x formatted_inputs = self.format_inputs(inputs, state, time) model_outputs = self.trained_model(formatted_inputs) return self.format_outputs(inputs, states, time, model_outputs) class PytorchWrapper(torch.nn.Modulel): def __init__(self, trained_model): self.trained_model = trained_model def format_inputs(self, inputs, states, time): return def format_outputs(self, inputs, states, time, model_outputs): return def forward(self, x): inputs, states, time = x formatted_inputs = self.format_inputs(inputs, state, time) model_outputs = self.trained_model(formatted_inputs) return self.format_outputs(inputs, states, time, model_outputs) In easy cases just implementing the format_inputs and format_outputs classes could be sufficient to make the compatibility wrapper for their model. For more complex cases one would maybe also need to modify more, but this could be a good starting point for our users. If we include this we should also include tools for loading onnx files into pytorch/tensorflow and converting pytorch/tensorflow to onnx files. That way a user could go the entire way from an incompatible onnx file to an compatible onnx to a compiled fmu. All within the this tool. — Reply to this email directly, view it on GitHub<#30>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALT6QOHRODXROKVSPWJW2KLYYMCR3AVCNFSM6AAAAABEYEJNW2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGE4DQNZQGY4TGMI>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***> DNV SE, Registered Office Hamburg No. HRB 115442 • Chairman of the Supervisory Board: Remi Eriksen • Executive Board: Knut Ørbeck-Nilssen • Torsten Schramm • Björn-Olaf Borth • Gesa Heinacher-Lindemann • The latest edition of the General Terms and Conditions of DNV SE is applicable. ************************************************************************************** This e-mail and any attachments thereto may contain confidential information and/or information protected by intellectual property rights for the exclusive attention of the intended addressees named above. If you have received this transmission in error, please immediately notify the sender by return e-mail and delete this message and its attachments. Unauthorized use, copying or further full or partial distribution of this e-mail or its contents is prohibited. **************************************************************************************