[render_gl] Needs nominal support for glTF

[SeanCurtis-TRI]

What happened?

Problem statement

Currently, if a Mesh or Convex referencing anything other than a .obj file is passed to RenderEngineGl, it will simply do a one time warning about it only uses .obj files and then ignore the geometry. RenderEngineGl is the last hold out for supporting glTF files (in any way). By giving it (at least nominal) support for glTF, we are free to promote glTF as the preferred visual representation in Drake.

Ideal solution

RenderEngineGl should consume the glTF and render it with a PBR-compliant render pipeline.

Challenges

• RenderEngineGl doesn't currently have a PBR render pipeline (this would require extensive shader writing and delay the time before we'd be rendering anything.
• A glTF file can have multiple disjoint pieces. Each would be associated with the GeometryId. Currently RenderEngineGl only associates a single geometry "piece" with a GeometryId.

Proposed solution

• We won't expect PBR support out of the gate. Instead, we'll transform PBR to the current quasi-Phong model.
• We need to upgrade RenderEngineGl to support a one-to-many mapping between GeometryId and geometry "pieces".

The plan

• Fix Drake to support OBJs with multiple materials (and multiple objects) in rendering.

  • The following issue is a subset of that (multiple objects and multiple materials have the same obstacle): Support for multiple meshes in a single obj file #17266
    • [render] RenderMaterial offers more nuanced and complete warnings #20236
    • Parsing an .obj file can produce multiple RenderMesh instances #20250
      • Amended with [render] Parsing an .obj file can produce multiple RenderMesh instances #20285
    • RenderEngineGl supports multiple meshes from OBJ #20272
    • [render] RenderEngineVtk supports multiple meshes from OBJ #20235
  • glTF support will require multiple materials. Fixing the OBJ support will introduce the necessary infrastructure, fix a long-standing bug, silence some warnings, and provide a basis to test the framework without introducing any complexity related to glTF.

• Have RenderEngineGl instantiate geometries from glTF files.

  • RenderEngineGl can consume glTF files #21520
    • Update typedefs of C-style function pointers to std::function syoyo/tinygltf#489 merges.
    • [workspace] Upgrade tinygltf to latest release v2.8.23 #21569
  • Bring tinygltf in as a dependency #21521
  • Generalize RenderEngineGl's geometries and instances #21522
  • Do some analysis of rendering performance.
    • Pick a "representative" scene from LBM.
    • Render it three times:
      • RenderEngineVtk with current settings (e.g., shadows, env maps, etc.)
      • RenderEngineVtk with "fastest" settings.
      • RenderEngineGl with 'fast" settings.
    • For TRI folks, this internal document shows the result of the basic analysis by which we can draw the conclusion that using RenderEngineGl is 2-3X faster than RenderEngineVtk.

• examples/multibody/deformable/bubble_gripper.cc uses "package://drake_models/wsg_50_description/meshes/textured_bubble.obj". Make sure it works with a glTF version instead. Then convert to glTF and remove the .obj file in RobotLocomotion/models.

  • 2024-07-18 Update: RenderEngineGl supports glTF files by directly parsing the glTF data into OpenGl data structures that get dumped straight to the GPU. The example above uses the .obj file as a "driven mesh". Driven mesh parses the file into geometry::internal::RenderMesh as an intermediate data structure. We currently have no path to go from glTF file to RenderMesh data type. (An earlier version of the RenderEngineGl support started going down that path, but that level of conversion was much more demanding than simply ferrying data to the GPU.) While it is reasonable to want to use a glTF file as a driven mesh, we'd have to do a fair amount of heavy lifting to map glTF to RenderMesh to support the current DrivenMesh infrastructure. The needs to resolve this have been added to Deformable Visualization for Camera Sensors #18922..

Version

No response

What operating system are you using?

No response

What installation option are you using?

No response

Relevant log output

No response

[SeanCurtis-TRI]

I've inserted a link to my simple experiment for comparing performance between RenderEngineGl and RenderEngineVtk up above. PTAL.

[SeanCurtis-TRI]

The only thing left in this issue cannot be resolved until #18922 gets resolved. I've moved that remaining item into that issue.