Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collada DAE support #19109

Open
RussTedrake opened this issue Mar 31, 2023 · 11 comments
Open

Collada DAE support #19109

RussTedrake opened this issue Mar 31, 2023 · 11 comments
Assignees
Labels
component: geometry general Geometry infrastructure or topics that defy categorization into other geometry components priority: medium type: feature request

Comments

@RussTedrake
Copy link
Contributor

RussTedrake commented Mar 31, 2023

Original title: Provide tools to help convert Collada DAE to Wavefront OBJ

Related to #2941, #14436, and #18844.

Request: We now have stl2obj. I think we should provide something similar for dae. Or at least publish our recommended workflow.

This is most relevant for visual geometries, because the primary difficulties were in preserving the colors/textures during conversion. Meshcat actually supports DAE directly. But Drake's simulated cameras do not support dae, and currently perform badly (#19055) when the visual geometry is specified as a dae asset.

I'll document here my experience in converting this file, which was more challenging than it should have been.

Approach 1: Meshlab.

Result: Meshlab failed to load the colors; unsuprisingly the exported obj was missing the colors, too.

Approach 2: Assimp.

Usage: assimp export daefile objfile
Result: Drake doesn't like that the exported obj file has multiple objects

RuntimeError: The OBJ file contains multiple objects; only OBJs with a single object are supported: File name: '/home/russt/QR/assets/movo_description_drake/meshes/manipulation/robotiq/visual/robotiq_85_base_link.obj'

Approach 3: Trimesh

Usage: I wrote this simple script:

import pathlib
from trimesh import load_mesh
from trimesh.exchange.obj import export_obj

for mesh_file in pathlib.Path('.').glob('*.dae'):
    mesh = load_mesh(mesh_file)
    obj, data = export_obj(
                mesh, return_texture=True, mtl_name=mesh_file.with_suffix('.mtl'))
    obj_file = mesh_file.with_suffix('.obj')
    with open(obj_file, "w") as f:
        f.write(obj)
    # save the MTL and images                                
    for k, v in data.items():
        with open(k, 'wb') as f:
            f.write(v)

Result: Drake doesn't like that the exported obj file has multiple objects

RuntimeError: The OBJ file contains multiple objects; only OBJs with a single object are supported: File name: '/home/russt/QR/assets/movo_description_drake/meshes/manipulation/robotiq/visual/robotiq_85_base_link.obj'

Approach 4: Blender

Usage: I wrote this simple script:

# Run with e.g.
# blender --background --python convert_dae_to_obj.py -- kinova_robotiq_coupler.dae ../collision/kinova_robotiq_coupler.obj

import bpy
import sys

argv = sys.argv
argv = argv[argv.index("--") + 1 :]  # get all args after "--"

dae_in = argv[0]
obj_out = argv[1]

bpy.ops.object.delete()
bpy.ops.wm.collada_import(filepath=dae_in)
bpy.ops.export_scene.obj(filepath=obj_out, axis_forward="Y", axis_up="Z")

Result: That worked! The only drawback is that the generated asset is now infected with GPL.

@RussTedrake RussTedrake added type: feature request priority: medium component: geometry general Geometry infrastructure or topics that defy categorization into other geometry components labels Mar 31, 2023
@jwnimmer-tri
Copy link
Collaborator

jwnimmer-tri commented Mar 31, 2023

It's probably worth investing in some documentation along this front.

In terms of conserving implementation effort, though, my vote would be instead to (1) support DAEs natively in the first place for visualizers and renderers, and (2) support multi-mesh OBJ files for collision meshes (probably directly, or in the worst case with a pre-processing tool). I think that would be the best investment.

Possibly we should support DAEs for collision meshes as well?

... the generated asset is now infected with GPL.

Nope. The code snippet you posted is copyleft, but the artwork itself retains its original license. Similar to a compiler, the input and output files being processed retain their distinct license, without any change from blender.

See https://www.blender.org/about/license/ at "Your Artwork".

@adamconkey
Copy link

I've also encountered this OBJ with multiple objects problem several times. I think that's a fairly common scenario when the meshes are generated from CAD files (e.g. a STEP -> OBJ conversion), and indeed when I load such OBJs in Blender I can see they're composed of several different objects, one for each component in the CAD model.

In Blender you may be able to get away with merging all the objects together before exporting to OBJ.

@RussTedrake
Copy link
Contributor Author

Oh, great news about the license!

Re: merging the geometries; the output from the blender conversion worked (it loaded as a single object) without any additional modifications required.

@marcoag
Copy link
Contributor

marcoag commented Apr 4, 2023

Just came across this issue, wanted to point out there's some successful work converting .dae files to .obj using assimp here:

RobotLocomotion/drake-ros#83

More specifically on the format_model_and_generate_manifest.py file. I kinda remember there's some issues to take into account when doing this like assimp/assimp#849.

Maybe the code there could be extracted in favor of a dae2obj.py script?

@jwnimmer-tri
Copy link
Collaborator

@SeanCurtis-TRI I'd like to propose that https://drake.mit.edu/doxygen_cxx/group__geometry__file__formats.html not only list the formats we do support, but also specifically nack the commonly-requested formats that we don't support. So it would gain a section header for DAE that would say "not supported for any role", with a hyperlink to this issue. WDYT?

(Similarly for STL, but linking to a different issue.)

@SeanCurtis-TRI
Copy link
Contributor

SeanCurtis-TRI commented Sep 29, 2023

As I refreshed myself on this topic, I collected some thoughts regarding the current state of affiars:

  • Multiple-part objs
    • has always worked with meshcat,
    • is being resolved for rendering ([render] RenderEngineVtk supports multiple meshes from OBJ #20235),
    • proximity is the only thing left. Current proposals can be found in Support for multiple meshes in a single obj file #17266.
      • Note: if a dae has multiple materials, and a conversion process merges that into a single "object" with multiple materials, there is still a danger that our parsing technology will consider that to be "multiple objects" in the one place where we don't care about the material: proximity role. I'd have to investigate to state for certain, one way or the other. UPDATE Looking at tinyobjloader, it seems that we should be safe in this regard.
  • Meshcat basically supports collada already, but Drake isn't ready to send it yet. But that would be the only Drake-affiliated consumer ready to go.

re: updating the geometry file document

  • Is "nack" a typo or slang?
  • The document currently says, if a format isn't listed, it is definitely not supported. As there are a multiple of file formats, I don't want to have to explicitly exclude everything. You can never stay in front with that.
  • That said, I like the idea of a section of file formats that are "Drake adjacent". That section would be about file formats we're interested in or are close enough to Drake that we have tools, desires, whatever regarding them. This would give us the chance to call out the stl to obj conversion tool, link issues on things like dae, call for volunteers, etc.

@jwnimmer-tri
Copy link
Collaborator

"nack" is "negative ack".

As there are a multiple of file formats, I don't want to have to explicitly exclude everything. You can never stay in front with that.

I didn't say everything. I said commonly-requested formats. (It sounds like you agree, in any case.)

@jwnimmer-tri
Copy link
Collaborator

jwnimmer-tri commented Jul 9, 2024

To revisit the big picture, our call to action here is either:

(1) When given a DAE mesh, throw an error with a hyperlink to a troubleshooting guide that convincingly explains how to convert the model file to use a different supported file format -- either OBJ or glTF. Maybe we provide a small Python program that helps, using either blender or pyassimp.

(2) Actually parse and support the DAE meshes, for all geometry endpoints. (This means Meshcat, Meldis, all RenderEngines, implicit inertia calculations, etc.) I think the technical approach here would be to add assimp as a bazel dependency (building it from source), convert the DAE file to an in-memory meshfile, and then use #15263 to add that transmogrified meshfile to scene graph under the proximity role. For the illustration role, we could probably just add the DAE filename directly, and finish meshcat-dev/meshcat#170.

Obviously (2) is better if we can swing it, but I'm not sure how much work it would end up being.

@RussTedrake
Copy link
Contributor Author

What about something in between:

(1.5) We provide a python script like prepare_model_for_drake.py which crawls the urdf, sdf, mjcf, and automatically converts dae and stl files, etc.

I think this would be dramatically better for users. Especially when people are using the AddRemote workflow, and trying to quickly pull in resources from other repositories; we'd like to make it easy for them to just run one thing and have all the changes on disk.

@jwnimmer-tri
Copy link
Collaborator

Yes, that was basically the "Maybe we provide a small Python program that helps, using either blender or pyassimp" trailer on option 1. I'm happy to assign it number 1.5 so we talk about it concisely.

If we have such a program that converts just one mesh, teaching it to glob / recurse for all meshes (and perl-pie the citation in other xml files) is trivial after that. The only difficult part is converting any one mesh in the first place, given the giant number of traps that textured mesh-conversion can fall into.

My gestalt of the issue traffic so far is that maybe blender (with good import / export options selected) is the only tool that can convert without making mistakes. I guess we might already have a bpy script above, that does the conversion correctly? So maybe the only work for option 1.5 is to rework that script into a supported tool, and mention it from the RuntimeException when the user tried to load a DAE into Drake.

.. which crawls ... mjcf ...

nit: MJCF doesn't allow DAE anyway, so we don't necessarily need to touch that part here. For STL, it's relevant but in #19408, probably just loading STL directly is super simple so we hopefully won't need conversion.

@jwnimmer-tri
Copy link
Collaborator

jwnimmer-tri commented Jul 10, 2024

Let me toss out another option, let's call it Option 3:

  • Support DAE for illustration geometry only (by fixing the broken plumbing in Meshcat's support of it).
  • Warn (that we're ignoring it) when DAE is used for perception geometry.
  • Error (throw) when DAE is used for proximity geometry.

I anticipate that most of the DAE in the wild is <visual> only, so that might be another good compromise.

I don't love having a heterogeneous support matrix for a mesh format, but it might be the kindest option for real users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: geometry general Geometry infrastructure or topics that defy categorization into other geometry components priority: medium type: feature request
Projects
Status: No status
Development

No branches or pull requests

6 participants