[WIP] Include new Panda3D

[SamFlt]

In the context of providing a new Render-based tracker, this PR introduces a new, easy to use renderer based on Panda3D.

This renderer can output:

• A color image, with support for textures and lighting
• Depth image (For now, this is a buffer between 0 and 1)
• Normal maps
  • In world space
  • In camera space

It only supports camera models with no distortion.

It is also possible to compute camera clipping values, depending on the pose of an object in the camera frame. This ensures that the depth buffer is as accurate as possible when considering this object.

here is a set of renders for a textured object

and for a metallic object

Multi-output rendering is performed via the vpPanda3DRendererSet class, which duplicates the scene across multiple renders and synchronizes changes to objects and the camera. Each Sub renderer implements a specific type of render: geometric (vpPanda3DGeometryRenderer) or color-based (vpPanda3DRGBRenderer) etc. They all inherit from vpPanda3DBaseRenderer, which implements basic functions for a panda renderer.

Before this PR is complete, multiple things must be done:

• Improve CMake support. As of now, Cmake will not work for windows, and probably not for Mac. Same goes for conda support;
• Write the doc linked to the tutorial. Improve the tutorial to take path to the model to display as an argument;
• Implement postprocessing filters. For instance, Running canny edge computation for depth maps via shaders is planned;
  • Base class vpPanda3DPostProcessFilter: which can take any fragment shader and compute an output image
  • grayscale conversion
  • Gaussian blur
  • Canny on grayscale image
• Support for resizing the buffers;
• Support for removing and adding sub renderers on the fly;
• Add an interface to add lights to rendering in a clean way;
• Use ViSP frames in normal output and object/camera position.

Other, potentially interesting, future tracks include:

• Segmentation mapping
• Light mapping: In the context of tracking textureless objects with high specularity or "metallicness", this information can be used to filter the features to use to compute pose update.

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 45.63%. Comparing base (9e24a98) to head (626ed6a).
Report is 85 commits behind head on master.

Files	Patch %	Lines
tutorial/ar/tutorial-panda3d-renderer.cpp	0.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1388   +/-   ##
=======================================
  Coverage   45.63%   45.63%           
=======================================
  Files        1069     1070    +1     
  Lines      108080   108116   +36     
  Branches    18709    18713    +4     
=======================================
+ Hits        49323    49340   +17     
- Misses      58757    58776   +19     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fspindle]

• How to build Panda3D from source on Ubuntu 22.04

  $ mkdir -p $VISP_WS/3rdparty/panda3d
  $ cd $VISP_WS/3rdparty/panda3d
  $ git clone https://github.com/panda3d/panda3d
  $ cd panda3d
  $ python3 makepanda/makepanda.py --everything --installer --no-egl --no-gles --no-gles2 --no-opencv
  

  At this point you can either

  1. install the produced Debian package (recommended) with

     $ sudo dpkg -i panda3d1.11_1.11.0_amd64.deb
     

  2. use the Panda3D libraries located in the built folder without installing the Debian package panda3d1.11_1.11.0_amd64.deb, but in that case you need to set LD_LIBRARY_PATH environment var:

     $ export LD_LIBRARY_PATH=$VISP_WS/3rdparty/panda3d/panda3d/built/lib:$LD_LIBRARY_PATH
     

     Without setting LD_LIBRARY_PATH you may experience the following error when running a binary that uses Panda3D capabilities:

     $ ./tutorial-panda3d-renderer 
     ./tutorial-panda3d-renderer: error while loading shared libraries: libp3dtoolconfig.so.1.11: cannot open shared object file: No such file or directory
     

• How to build ViSP with Panda3D support when Debian package panda3d1.11_1.11.0_amd64.deb is installed

  There is nothing specific to do, just run cmake as usual

  $ cd $VISP_WS/visp-build
  $ cmake ../visp
  $ make -j$(nproc)
  

• How to build ViSP with Panda3D support without installing Debian package panda3d1.11_1.11.0_amd64.deb

  • By setting Panda3D_DIR cmake var to the Panda3D cloned folder

    $ cd $VISP_WS/visp-build
    $ cmake ../visp -DPanda3D_DIR=$VISP_WS/3rdparty/panda3d/panda3d
    $ make -j$(nproc)
    

  • By setting Panda3D_DIR environment variable

    $ export Panda3D_DIR=$VISP_WS/3rdparty/panda3d/panda3d
    $ cd $VISP_WS/visp-build
    $ cmake ../visp
    $ make -j$(nproc)
    

[fspindle]

• How to build Panda3D from source on macOS

  • On macOS, you will need to download a set of precompiled third-party packages in order to compile Panda3D. Navigate to PandaED download page, select the lastest SDK (in our case SDK 1.10.14), and under </> Source Code section, download Thirdparty tools for macOS (in our case panda3d-1.10.14-tools-mac.tar.gz).
  • Extract third-party tools for macOS from downloaded archive

    $ cd ~/Downloads
    $ tar xvzf panda3d-1.10.14-tools-mac.tar.gz 
    

  • Once done clone Panda3D:

    $ mkdir -p $VISP_WS/3rdparty/panda3d
    $ cd $VISP_WS/3rdparty/panda3d
    $ git clone https://github.com/panda3d/panda3d
    $ cd panda3d
    

  • Move the downloaded third-party tools in Panda3D source code folder

    $ mv ~/Downloads/panda3d-1.10.14/thirdparty .
    

  • Build Panda3D from source

    $ python3 makepanda/makepanda.py --everything --installer --no-egl --no-gles --no-gles2 --no-opencv --no-python --threads $(sysctl -n hw.logicalcpu)
    

• At this point you can either

  1. install the produced Panda3D-1.11.0-py3.9.dmg file (recommended) just by double clicking on it. In the installer window, don't forget to enable the C++ Header Files check box before pressing the installation button. After that you have to set DYLIB_LIBRARY_PATH environment var:

     % export DYLD_LIBRARY_PATH=/Library/Developer/Panda3D/lib:$DYLD_LIBRARY_PATH
     

  2. or use the Panda3D libraries located in the built folder without installing .dmg file, but in that case you need to set DYLIB_LIBRARY_PATH environment var:

     $ export DYLD_LIBRARY_PATH=$VISP_WS/3rdparty/panda3d/panda3d/built/lib:$DYLD_LIBRARY_PATH
     

     Without setting DYLD_LIBRARY_PATH you may experience the following error when running a binary that uses Panda3D capabilities:

     % ./tutorial-panda3d-renderer
     dyld[257]: Library not loaded: @loader_path/../lib/libpanda.1.11.dylib
     

• How to build ViSP with Panda3D support when .dmg file Panda3D-1.11.0-py3.9.dmg is installed

  There is nothing specific to do, just run cmake as usual

  $ cd $VISP_WS/visp-build
  $ cmake ../visp -DUSE_PCL=OFF
  $ make -j$(sysctl -n hw.logicalcpu)
  

• How to build ViSP with Panda3D support without installing the .dmg file

  • By setting Panda3D_DIR cmake var to the Panda3D cloned folder

    $ cd $VISP_WS/visp-build
    $ cmake ../visp -DUSE_PCL=OFF -DPanda3D_DIR=$VISP_WS/3rdparty/panda3d/panda3d
    $ make -j$(sysctl -n hw.logicalcpu)
    

  • Or by setting Panda3D_DIR environment variable

    $ export Panda3D_DIR=$VISP_WS/3rdparty/panda3d/panda3d
    $ cd $VISP_WS/visp-build
    $ cmake ../visp -DUSE_PCL=OFF
    $ make -j$(sysctl -n hw.logicalcpu)
    

• How to install Panda3D latest SDK from existing packages

  • Installer are available for Ubuntu, macOS and Windows browsing the download page.
  • Note: For the latest Panda3D 1.10.14 SDK there is an Installer for macOS X 10.9+ that is only compatible with architecture x86_64. If you are using a Mac M1 or M2, there is no Panda3D SDK available yet for arm64 architecture. The solution is to build Panda3D from source.

• Known issue on macOS

  • Segfault: :framework(error): Unable to create window

    % ./tutorial-panda3d-renderer
    Initializing Panda3D rendering framework
    Known pipe types:
      CocoaGLGraphicsPipe
    (all display modules loaded.)
    :framework(error): Unable to create window.
    zsh: segmentation fault  ./tutorial-panda3d-renderer
    

    This issue is probably due to EIGEN_MAX_ALIGN_BYTES and HAVE_PNG macro redefinition that occurs when building ViSP with Panda3D support:

    $ cd visp-build
    $ make
    ...
    [100%] Building CXX object tutorial/ar/CMakeFiles/tutorial-panda3d-renderer.dir/tutorial-panda3d-renderer.cpp.o
    In file included from $VISP_WS/visp/tutorial/ar/tutorial-panda3d-renderer.cpp:17:
    In file included from $VISP_WS/visp/modules/ar/include/visp3/ar/vpPanda3DRGBRenderer.h:39:
    In file included from $VISP_WS/visp/modules/ar/include/visp3/ar/vpPanda3DBaseRenderer.h:42:
    In file included from $VISP_WS/3rdparty/panda3d/panda3d/built/include/pandaFramework.h:17:
    In file included from $VISP_WS/3rdparty/panda3d/panda3d/built/include/pandabase.h:21:
    In file included from $VISP_WS/3rdparty/panda3d/panda3d/built/include/dtoolbase.h:22:
    $VISP_WS/3rdparty/panda3d/panda3d/built/include/dtool_config.h:40:9: warning: 'HAVE_PNG' macro redefined [-Wmacro-redefined]
    #define HAVE_PNG 1
            ^
    /opt/homebrew/include/pcl-1.14/pcl/pcl_config.h:53:9: note: previous definition is here
    #define HAVE_PNG
        ^
    In file included from $VISP_WS/visp/tutorial/ar/tutorial-panda3d-renderer.cpp:17:
    In file included from $VISP_WS/visp/modules/ar/include/visp3/ar/vpPanda3DRGBRenderer.h:39:
    In file included from $VISP_WS/visp/modules/ar/include/visp3/ar/vpPanda3DBaseRenderer.h:42:
    In file included from $VISP_WS/3rdparty/panda3d/panda3d/built/include/pandaFramework.h:17:
    In file included from $VISP_WS/3rdparty/panda3d/panda3d/built/include/pandabase.h:21:
    In file included from $VISP_WS/3rdparty/panda3d/panda3d/built/include/dtoolbase.h:22:
    $VISP_WS/3rdparty/panda3d/panda3d/built/include/dtool_config.h:64:9: warning: 'HAVE_ZLIB' macro redefined [-Wmacro-redefined]
    #define HAVE_ZLIB 1
            ^
    /opt/homebrew/include/pcl-1.14/pcl/pcl_config.h:55:9: note: previous definition is here
    #define HAVE_ZLIB
            ^
    In file included from $VISP_WS/visp/tutorial/ar/tutorial-panda3d-renderer.cpp:17:
    In file included from $VISP_WS/visp/modules/ar/include/visp3/ar/vpPanda3DRGBRenderer.h:39:
    In file included from $VISP_WS/visp/modules/ar/include/visp3/ar/vpPanda3DBaseRenderer.h:42:
    In file included from $VISP_WS/3rdparty/panda3d/panda3d/built/include/pandaFramework.h:17:
    In file included from $VISP_WS/3rdparty/panda3d/panda3d/built/include/pandabase.h:21:
    $VISP_WS/3rdparty/panda3d/panda3d/built/include/dtoolbase.h:432:9: warning: 'EIGEN_MAX_ALIGN_BYTES' macro redefined [-Wmacro-redefined]
    #define EIGEN_MAX_ALIGN_BYTES MEMORY_HOOK_ALIGNMENT
            ^
    /opt/homebrew/include/eigen3/Eigen/src/Core/util/ConfigureVectorization.h:175:11: note: previous definition is here
      #define EIGEN_MAX_ALIGN_BYTES EIGEN_IDEAL_MAX_ALIGN_BYTES
              ^
    3 warnings generated.
    [100%] Linking CXX executable tutorial-panda3d-renderer
    [100%] Built target tutorial-panda3d-renderer
    

    The work around consists in disabling PCL usage during ViSP configuration

    $ cd $VISP_WS/visp-build
    $ cmake ../visp -DUSE_PCL=OFF
    $ make -j$(sysctl -n hw.logicalcpu)