Portable install space

[ahoarau]

Is there a way for OROCOS in a ROS environnement to get a 'portable' install space ?
So far rtt and ocl creates files () that contain hardcoded path. There must be something do to with the variables contained in setup.sh ?

Context:

mkdir ~/rtt_ws/src
cd ~/rtt_ws/src
git clone https://github.com/orocos-toolchain/rtt -b toolchain-2.9 --depth 1
git clone https://github.com/orocos-toolchain/log4cpp -b toolchain-2.9 --depth 1
git clone https://github.com/orocos-toolchain/ocl -b toolchain-2.9 --depth 1

cd ~/rtt_ws
catkin config --init --install --extend /opt/ros/kinetic
catkin build

# rename or move the install space somewhere else
mv install/ release-2.9/
# Source the env file
source release-2.9/setup.bash
# Try to launch the deployer
deployer

# -----> Could not find /home/myname/ws/install/bin/deployer-gnulinux

[meyerj]

Unfortunately not easily as of today. The primary reason is that Orocos creates libraries at locations below lib, e.g. in lib/orocos/gnulinux/<package>, and relies on compiled-in RPATH to still enable other libraries to link against them. This is especially the case for typekits.

Without changing this fundamental design of the Orocos plugin system there is no easy way to relocate an install-space because setting [DY]LD_LIBRARY_PATH alone would not help to fix the linking problems. One possibility would be to install all libraries to the default library destination lib and create other files (e.g. XML manifests) in the lib/orocos subfolder to help the ComponentLoader and PluginLoader to find the plugin libraries that are meant to be plugins and loaded at run-time. Another possibility is to disallow linking to a component, plugin or typekit library that is not installed to a path outside [DY]LD_LIBRARY_PATH. This would require splitting typekits into a "normal" library that contains all the explicitly instantiated templates and which gets linked to other Orocos libraryes, and a trivial typekit plugin library, which only has the plugin entry code.

We could leave this issue open as a feature request, but it will probably not be implemented soon and require quite some changes. Unless someone has a better idea how to solve it.

[snrkiwi]

@ahoarau What do you mean by "portable"? Do you want an install that you can move to an arbitrary location(s), or do you want the simpler version where you build/install in one location and then move the install to another a priori known location? The second one is mostly supported by DESTDIR type semantics in Orocos, though some of the hard coded paths might need fixing. The first one is much more difficult due to RPATH and other considerations, let alone the hard coded paths.

[Hugal31]

Couldn't we make use of $ORIGIN in the RPATH? For example a plugin installed in
${CMAKE_INSTALL_PREFIX}/lib/orocos${OROCOS_SUFFIX}/${PROJECT_NAME}/plugins
would have this kind of RPATH:

$ORIGIN/..
$ORIGIN/../types
$ORIGIN
$ORIGIN/../../..

instead of

${CMAKE_INSTALL_PREFIX}/lib/orocos${OROCOS_SUFFIX}/${PROJECT_NAME}
${CMAKE_INSTALL_PREFIX}/lib/orocos${OROCOS_SUFFIX}/${PROJECT_NAME}/types
${CMAKE_INSTALL_PREFIX}/lib/orocos${OROCOS_SUFFIX}/${PROJECT_NAME}/plugins
${CMAKE_INSTALL_PREFIX}/lib

(taken from orocos_set_install_rpath).

[meyerj]

I was not aware of $ORIGIN yet. That could work, given that all Orocos binaries are installed to the same prefix. I cannot foresee yet how it works if some packages are already installed in a separate install-space. For example with ROS, RTT and other packages installed from binary Debian packages are located in /opt/ros/distro/lib/orocos. User packages are then typically built and installed in an overlay (separate build- and install-space). The library paths of the underlay must still be added to the RPATH explicitly because they cannot be found relative to $ORIGIN.

The orocos_set_install_rpath() macro already adds all the library dirs of used dependencies (passed from ${USE_OROCOS_LIBRARY_DIRS} via ${ARGN}) and RTT and OCL (${OROCOS-RTT_LIBRARY_DIRS}, ${OROCOS-OCL_LIBRARY_DIRS}). So that could work as long as only the top-level install-space is relocated and not one of the underlays.

The absolute paths are also written to the pkg-config files generated by orocos_generate_package() and parsed by the orocos_find_package() and orocos_use_package() macros. Finding packages and libraries at build-time from a relocated install-space from only CMAKE_PREFIX_PATH and PKG_CONFIG_PATH should also be considered as part of the problem.

A pull request that suggest the necessary changes for further testing would be very welcome.

Wiki page about RPATH handling with CMake:
https://gitlab.kitware.com/cmake/community/wikis/doc/cmake/RPATH-handling

[snrkiwi]

In all honesty I think you might need both options, to support the different use cases; 1) for system level installs or alternate install spaces, where absolute paths are required, and 2) when installing in your application install space, and then relative paths are ok.

Also, certainly on the Mac you need full path for system installed shared libraries due to lack of DYLD_LIBRARY_PATH for security reasons.

[ahoarau]

In 'deployer' script, if we replace:

abspath="/the/hardcoded/path"

by

abspath="$(dirname $(readlink -f $0))"

It seems to work for loading everything.

Question2: Is there a way to just install eveything in '/lib', rather than '/lib/orocos//types/' etc ?

[Hugal31]

For the "$ORIGIN in Rpath solution", I have a CMake function that fixes the RPATH of already compiled .so. It could also be used to fix the RPATH before the compilation.

[ahoarau]

It seems to work for loading everything.

• extending the RTT_COMPONENT_PATH as well.

[meyerj]

Question2: Is there a way to just install eveything in '/lib', rather than '/lib/orocos//types/' etc ?

I totally agree that this would be much simpler, but unfortunately not without refactoring the ComponentLoader and PluginLoader (which should be unified anyway). See #303 (comment).

What was question 1? ;-)

Also, certainly on the Mac you need full path for system installed shared libraries due to lack of DYLD_LIBRARY_PATH for security reasons.

I agree with @snrkiwi that there are cases where absolute paths are required. But it is not because of DYLD_LIBRARY_PATH on macOS. The RPATH mechanism is exactly there to not have to depend on unsecure [DY]LD_LIBRARY_PATH environment settings.

It seems to work for loading everything.

Using a relative RPATH only works for packages found in the same install-space, and under the assumption that they will only be relocated together. With overlays, like in the case of Debian packages released into ROS (or if RTT is installed to /usr), dependent packages must still have the absolute paths to all linked libraries linked from /opt/ros/xxx/lib/orocos/gnulinux/... in their RPATH because they cannot or they will not be found relative to $ORIGIN.

So some magic would be required in orocos_set_install_rpath() to configure the INSTALL_RPATH depending on whether a linked library is installed in the same install-space (in ${CMAKE_INSTALL_PREFIX}) or not. This can be tricky, especially with the various options to build a catkin package. In case of catkin_build`, with a single non-isolated build directory for the whole workspace, other package's libraries are not yet installed at the time a dependent package is built.

I'll give it a try this week, but a pull request with whatever approaches you already tested would be helpful as a starting point.

[meyerj]

Nice explanation of RPATH handling and respective CMake options:
https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling