How to fully build this project?(I guess it links to CPP version since the cpp folder is empty)

[cdb0y511]

Hi, I am a newcomer. I am very interesting on SobolevFusion. I notice you have study on it more than a year, and have done some good jobs. Among all the codes, I think your InfiniTAM branch gets the best results except the author's, better than dgrzech/sobfu. But I still have some questions:
1.Why do you abandon the InfiniTAM branch?
I guess there are 2 reasons. First the multithreaded and CUDA problems. Second the SDF-2-SDF tracker problems. These two may not or difficult to solve under the InfiniTAM framework. So you start a new one. Am I right?
2.How to fully build this project?
I guess it requires the CPP version. But how to link them. I am not familiar with combining C++ with python. Could you provide a readme?
3.Does this project work well?
I am look forward it.
Thx

[Algomorph]

@cdb0y511 thanks for reaching out.

1. To put it simply, InfiniTAM was designed with static-scene fusion algorithms in mind. InfiniTAM is a well-structured codebase on the whole, but I eventually got fed up with lack of appropriate testing framework support, archaic use of C++ and CMake, and a myriad of other issues, not the least of which was that I found myself bending over backwards to build up a dynamic-scene level-set-based method and still fit within the general InfiniTAM architecture / not deviate too much from the established structure. It started looking really, really ugly.
   CUDA problems and other bugs were the least of my concern -- after all, bugs can be fixed... More easily when you can do test-driven development, of course. SDF-2-SDF tracker could, in theory, be easily implemented as well.

So, I decided that starting a fresh codebase, supporting my own experiments (not just SobolevFusion / TSDF-evolution methods, but warp-optimization methods, hierarchical optimization, etc.) would actually be easier and cleaner than continuing on InfiniTAM.

2. Run a git submodule init and a `git pull --recur' in the root of your repo -- and you'll get the CPP subproject. Run CMake, build, and install, and you're all set.

I can and will provide a README in due time, as soon as I feel like the code has enough value for people besides me and my collaborator. More critically, after my PhD proposal defense at the end of June.

3. Define "work well". If you're looking for something that works out-of-the box like InfiniTAM, this code base isn't there yet and far from it. There are only some python-based experimental utilities (which include the non-rigid SobolevFusion component for a toy 2D environment, the same for SDF-2-SDF rigid alignment and other things). Realistically thinking about it, I'd give it another year or so before there are some real results to show.

[cdb0y511]

Thank you, really appreciate your answer.
I am a PhD student too. But not very good at coding. Recently, I focus on RGD-D Dynamic reconstruction. A mature project needs at least one year hard work. However, I want to verify my idea quickly. So I look for some code to work with it. I am interesting on the vector field and H1 norm in SobolevFusion. I think there is a better way to replace the sobolev gradient flow. At least, I want to give a try.
Do you have any suggestions?
Should I just start with your DynamicFusion branch or this one?
By the way, a minor problem on DynamicFusion is that I compile with"Build with FFmpeg support?" ON", I use the default apt install libavcodec-dev, libavformat-dev and libavutil-dev on ubuntu 18.04, but the recorded video by InfiniTAM_bpo can not open by vlc or default player.

I sincerely wish you everything goes well and smooth on this summer.

[Algomorph]

You can try out https://github.com/dgrzech/sobfu, that's the only other working SobolevFusion implementation that I know off. The results are... As they are with my DynamicFusion branch... Underwhelming. But you're welcome to try.

What you're essentially telling me is that you don't want to put in the engineering effort to overcome the barrier to entry for this. I can understand that on some level. Unfortunately, both VolumeDeform authors and SobolevFusion authors are sitting on their code, and the reasons for this seem to be not just gripes about "intellectual property", if you take my meaning. There's a myriad of DynamicFusion-based projects (talking about the DynamicFusion 2015 paper), I haven't tried these out, but they might work relatively well -- all caveats related to using control nodes instead of voxel granularity for warps pertaining.

But if you were me, I'd advise you to bite the bullet and join effort with me on my codebase here. C++ code is going to be the main effort here. The python repo is just an interface to it (we can use matplotlib, opencv, and other handy things to plot data & experiment, as well as prototype, then implement in C++). I can teach you how to code well in the process. It's a higher-risk higher-reward approach (and would also require you to work your butt off), but, in my not-so-humble view, the potential payoff is well worth it.

[Algomorph]

P.S. Take a look at this issue for sofbu to understand what to expect.

[cdb0y511]

Thanks,
Sorry for reply late, my advisor put me into some other project. I understand and agree with you. Usually I will start to look into your test sample, the progress maybe slow. If you have anything want me to know, feel free to mail me ([email protected]).
P.S. tsdf.generation lack a member GenerationMethod (Maybe the python api doesnt finish yet)

[Algomorph]

@cdb0y511 that's fine, I'm working on my PhD proposal anyway for now, so take your time with the other projects.

As for the GenerationMethod -- it sounds like a regression for one of the experiment runnable scripts, very easy to fix. Those aren't tested in CI. If you could give me the name of the script you were running when you saw this error, that would be helpful.

[cdb0y511]

run_slavcheva_optimizer2d.py, tsdf.generation does not have GenerationMethod
Traceback (most recent call last):
File "/home/cd/ASources/LevelSetFusion-Python/run_slavcheva_optimizer2d.py", line 28, in
from experiment.multiframe_experiment import perform_multiple_tests, OptimizerChoice
File "/home/cd/ASources/LevelSetFusion-Python/experiment/multiframe_experiment.py", line 34, in
from tsdf.generation import GenerationMethod
ImportError: cannot import name 'GenerationMethod'

And run_sdf_2_sdf2d.py
File "", line 1, in
runfile('/home/cd/ASources/LevelSetFusion-Python/run_sdf_2_sdf2d.py', wdir='/home/cd/ASources/LevelSetFusion-Python')
File "/home/cd/pycharm/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "/home/cd/pycharm/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/home/cd/ASources/LevelSetFusion-Python/run_sdf_2_sdf2d.py", line 62, in
main()
File "/home/cd/ASources/LevelSetFusion-Python/run_sdf_2_sdf2d.py", line 56, in main
optimizer.optimize(data_to_use, narrow_band_width_voxels=narrow_band_width_voxels, iteration=iteration)
File "/home/cd/ASources/LevelSetFusion-Python/rigid_opt/sdf_2_sdf_optimizer2d.py", line 78, in optimize
method=cpp_module.tsdf.FilteringMethod.NONE)
File "/home/cd/ASources/LevelSetFusion-Python/rigid_opt/sdf_generation.py", line 34, in generate_2d_canonical_field
depth_image0 = depth_image0.astype(np.uint16) # mm
AttributeError: 'NoneType' object has no attribute 'astype'

[Algomorph]

@cdb0y511 yes, these are all very simple fixes due to refactorings, thanks for letting me know.

If you want to try out something that sort-of works right now in 3D, to checkout and build InfiniTAM DynamicFusion branch.

You can use these instructions to run it (ignore talk about branch names): Algomorph/InfiniTAM#138 (comment)