Inquiring about RTOSIM's Compatibility with Camera-Based Joint Position Data

[AliAhmadi24]

Hello,

I am currently working on a project that involves analyzing human arm motion using OpenSim. As a newcomer to OpenSim, I have recently come across the RTOSIM library, which offers real-time inverse kinematics and inverse dynamics capabilities. However, I am unsure about its compatibility with my specific project requirements.

In my project, I intend to use a camera and neural network for joint position estimation rather than a motion capture system. The neural network will be used to process the camera data and estimate the joint positions. I am interested in using RTOSIM to connect joints' positions from camera and neural network with OpenSim and perform real-time inverse kinematics and inverse dynamics calculations on the human arm model. My main question is whether RTOSIM can handle joint position data from a camera instead of using motion capture system data?

[cpizzolato]

Hi @aliamd321,

As long as your system generates 3D keypoint/marker locations as a time series, you can use it to solve kinematics. My suggestion would be to first create an OpenSim .trc file of the data generated by your system and test that using OpenSim. You will likely have to adjust the OpenSim model to have the correct markerset, compatible with your data. If you can make your system work offline using opensim, then you can start workling on the real-time implementation using RTOSIM. However, to work within RTOSIM, you will have to create your own class to communicate with your camera system and stream in the correct format.

[AliAhmadi24]

@cpizzolato,
Thank you for your prompt reply and detailed explanations.