update 3D points (sparse) after optimize camera pose [Question]

[wow2006]

Hello, I want to integrate multi-sensor fusion (MSF) with another module (which consists of a semi-dense front-end with non-linear optimization back-end) and IMU module. I have two question.
1- Is it a good idea to replace nonlinear optimization with MSF or use MSF after non-linear optimization?


2- After update camera position using MSF. How can I update 3D points (sparse) per camera and make sure that I don't duplicate points i.e. there is one floor, not two?