Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modified code for exporting eval pcd #20

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

dongjae0107
Copy link

Hi :)

How about slightly modifying the logic in 'export_eval_pcd.cpp' that marks 'gt_point' as dynamic? Currently, with a voxel size of 0.1, if the 'nearest_point' in 'et_cloud' is farther than 0.05 (half of the voxel size), 'gt_point' is classified as dynamic. However, when using the voxel downsampling method, which marks the space that the voxel spans, a more accurate approach might be to check if 'gt_point' falls within the space that 'nearest_point' spans before marking gt_point as dynamic. This would improve the accuracy of the classification.

@Kin-Zhang
Copy link
Member

I see. Thanks for your pull request.

Your way is more on voxel-way using the grid bounding to check where the original one is mainly based on the nearest neighbor distance. I may prefer the latter one still since some methods can save at point-level, and voxel-way will inflate the score in some way.

Maybe we need comparison results to show the benchmark methods' score change. Could you also attache a table for that if it's possible?

@dongjae0107
Copy link
Author

I evaluated the performance of the benchmark methods on the KITTI 00 and 05 sequences using the approach I suggested. (I used GT poses, not SuMa poses.) While the performance rankings between the methods remain consistent, I observed slight changes in the SA and DA values.
result_table

I agree with you that the voxel-based approach may inflate the scores. I also think that using the original scoring method is more accurate for point-level methods. The voxel-based approach I mentioned would be appropriate for methods that classify dynamic elements at the voxel level. It would be a good option to be able to choose the scoring method depending on the method.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants