Localize with different camera

[rebellojason7]

Hi,

I built a map using a stereo camera where the image size was 1280 by 1024 and I am trying to localize a different camera in the map with image size 1344 by 720. However I can't seem to localize the new camera. I have calibrated the new camera and fed in the right parameters. I looked at this issue xdspacelab/openvslam#388. but cant seem to get my data to work. Any thoughts ? I can share my data if need be.
Thanks

[ymd-stella]

@rebellojason7
Please align the resolution per angle of view. Is only one of your monocular or stereo cameras wide FoV?

If you can identify which step is failing you, we may be able to provide additional advice.

[ymd-stella]

If it is failing in the image search step, I recommend that you try increasing the number of features to be extracted. (Feature.max_num_keypoints)

[rebellojason7]

Hi @ymd-stella and @surfii3z. First of all, thanks again for offering to help. Let me state the problem. I have 3 cameras: a stereo pair and a monocular camera. The image sizes are mentioned below. I managed to change the monocular size from the above mentioned size but only a little.
Monocular camera (camera 1) size 1280 x 720
Stereo camera (camera 2 and camera 3) are each of size 1280 x 1024.

I also created 2 files in ros. Feel free to use them. I can also attach the launch files if you need. Please note that the online slam version, you will have to modify the topics in openvslam_ros.cc:

1. a file that allows for online slam or localization
2. a file for offline slam and localization.

online_slam.txt

offline_slam.txt

Problem 1. Using the online_slam version of my code. I can build a map using each of the 3 cameras individually as well as the stereo version (using camera2 and camera3) in stereo config. This proves that each camera is good for slam (in my opinion). Please note that when I build the stereo map, I use the -r option which rectifies the images ( i think). The problem comes when I try to localize each of the cameras individually in the stereo map. I can localize camera2 and camera3 individually, however my monocular camera (camera 1) doesnt localize in the stereo map. Should I rectify camera1 images before trying to localize ? I dont rectify camera2 or camera3 in monocular mode since I thought that the distortion parameters are included in the projection.

Situation 2. Lets say I build 3 monocular maps using camera1, camera2 and camera3. Lets call them map_camera1, map_camera2, map_camera3. I find that the best features are obtained in my map_camera1 and when I try to localize camera2 and camera3 (in mono mode) in my map_camera1, it works. However when I try to localize camera1 in map_camera2 or map_camera3 it doesnt work too well.

Thoughts: The main problem I am facing is that I need a map with scale, therefore I use the stereo camera to create the map. I then need to localize each of the 3 cameras individually in the stereo map. camera2 and camera3 localize well (makes sense since they were used in stereo mode to build the map) However camera1 doesnt localize at all. Any thoughts ? I would be open to chatting on slack if that would be better. Thanks for the help !!!

[surfii3z]

Thanks for the details debugging experiment. Could you show us a little bit of your experiment? Like a short video or snapshots.

What do you mean by best features are obtained in our map_camera1, what happens to the map in cam2 and cam3??

How about making the map in Stereo mode using cam2, cam3 and do localization in monocular mode using cam2, cam3, does it work? (cam2 mapping (stereo) -> cam2, cam3 mapping (mono)) (cam3 mapping (stereo) -> cam2, cam3 mapping (mono))

[surfii3z]

Otherwise you might try to make a denser map with stereo configuration with more variety of the angles such that the mono mode of the first cam could determine 3d-to-2d correspondence easier.

[libertyfree]

I built a map using a perspective camera ,location use monocular can not location.this stella_vslam lib is not suuport or my config is not correct?