Skip to content

Commit f004cb0

Browse files
jsmith-bdaiMayankm96
authored and
mohanksriram
committed
Adds custom dependency installation from extension.toml inside Docker (isaac-sim#552)
This PR adds back in the functionality that was added with https://github.com/isaac-orbit/IsaacLab/pull/542 - New feature (non-breaking change which adds functionality) - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [ ] I have added my name to the `CONTRIBUTORS.md` or my name already exists there --------- Signed-off-by: James Smith <142246516+jsmith-bdai@users.noreply.github.com> Co-authored-by: Mayank Mittal <12863862+Mayankm96@users.noreply.github.com>
1 parent 8b0394e commit f004cb0

File tree

4 files changed

+162
-0
lines changed

4 files changed

+162
-0
lines changed

‎docker/Dockerfile.base

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,12 @@ COPY ../ ${ISAACLAB_PATH}
5252
# Set up a symbolic link between the installed Isaac Sim root folder and _isaac_sim in the Isaac Lab directory
5353
RUN ln -sf ${ISAACSIM_ROOT_PATH} ${ISAACLAB_PATH}/_isaac_sim
5454

55+
# Install apt dependencies for extensions that declare them in their extension.toml
56+
RUN --mount=type=cache,target=/var/cache/apt \
57+
${ISAACLAB_PATH}/isaaclab.sh -p ${ISAACLAB_PATH}/tools/install_deps.py apt ${ISAACLAB_PATH}/source/extensions && \
58+
apt -y autoremove && apt clean autoclean && \
59+
rm -rf /var/lib/apt/lists/*
60+
5561
# for singularity usage, have to create the directories that will binded
5662
RUN mkdir -p ${ISAACSIM_ROOT_PATH}/kit/cache && \
5763
mkdir -p ${DOCKER_USER_HOME}/.cache/ov && \

‎docker/Dockerfile.ros2

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,9 @@ RUN --mount=type=cache,target=/var/cache/apt \
2222
ros-humble-rmw-fastrtps-cpp \
2323
# This includes various dev tools including colcon
2424
ros-dev-tools && \
25+
# Install rosdeps for extensions that declare a ros_ws in
26+
# their extension.toml
27+
${ISAACLAB_PATH}/isaaclab.sh -p ${ISAACLAB_PATH}/tools/install_deps.py rosdep ${ISAACLAB_PATH}/source/extensions && \
2528
apt -y autoremove && apt clean autoclean && \
2629
rm -rf /var/lib/apt/lists/* && \
2730
# Add sourcing of setup.bash to .bashrc

‎docs/source/setup/developer.rst

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,6 +190,34 @@ important to note that Omniverse also provides a similar
190190
However, it requires going through the build process and does not support testing of the python module in
191191
standalone applications.
192192

193+
Extension Dependency Management
194+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
195+
196+
Certain extensions may have dependencies which need to be installed before the extension can be run.
197+
While Python dependencies can be expressed via the ``INSTALL_REQUIRES`` array in ``setup.py``, we need
198+
a separate installation pipeline to handle non-Python dependencies. We have therefore created
199+
an additional setup procedure, ``python tools/install_deps.py {dep_type} {extensions_dir}``, which scans the ``extension.toml``
200+
file of the directories under the ``{extensions_dir}`` (such as ``${ISAACLAB_PATH}/source/extensions``) for ``apt`` and ``rosdep`` dependencies.
201+
202+
This example ``extension.toml`` has both ``apt_deps`` and ``ros_ws`` specified, so both
203+
``apt`` and ``rosdep`` packages will be installed if ``python tools/install_deps.py all ${ISAACLAB_PATH}/source/extensions``
204+
is passed:
205+
206+
.. code-block:: toml
207+
208+
[isaaclab_settings]
209+
apt_deps = ["example_package"]
210+
ros_ws = "path/from/extension_root/to/ros_ws"
211+
212+
From the ``apt_deps`` in the above example, the package ``example_package`` would be installed via ``apt``.
213+
From the ``ros_ws``, a ``rosdep install --from-paths {ros_ws}/src --ignore-src`` command will be called.
214+
This will install all the `ROS package.xml dependencies <https://docs.ros.org/en/humble/Tutorials/Intermediate/Rosdep.html>`__
215+
in the directory structure below. Currently the ROS distro is assumed to be ``humble``.
216+
217+
``apt`` deps are automatically installed this way during the build process of the ``Dockerfile.base``,
218+
and ``rosdep`` deps during the build process of ``Dockerfile.ros2``.
219+
220+
193221
Standalone applications
194222
~~~~~~~~~~~~~~~~~~~~~~~
195223

‎tools/install_deps.py

Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Copyright (c) 2022-2024, The Isaac Lab Project Developers.
2+
# All rights reserved.
3+
#
4+
# SPDX-License-Identifier: BSD-3-Clause
5+
6+
"""
7+
A script with various methods of installing dependencies
8+
defined in an extension.toml
9+
"""
10+
11+
import argparse
12+
import os
13+
import shutil
14+
import toml
15+
from subprocess import run
16+
17+
# add argparse arguments
18+
parser = argparse.ArgumentParser(description="Utility to install dependencies based on an extension.toml")
19+
parser.add_argument("type", type=str, choices=["all", "apt", "rosdep"], help="The type of packages to install")
20+
parser.add_argument("extensions_dir", type=str, help="The path to the directory beneath which we search for extensions")
21+
22+
23+
def install_apt_packages(paths: list[str]):
24+
"""Attempts to install apt packages for Isaac Lab extensions.
25+
For each path in arg paths, it looks in {extension_root}/config/extension.toml for [isaac_lab_settings][apt_deps]
26+
and then attempts to install them. Exits on failure to stop the build process
27+
from continuing despite missing dependencies.
28+
29+
Args:
30+
paths: A list of paths to the extension root
31+
32+
Raises:
33+
RuntimeError: If 'apt' is not a known command
34+
"""
35+
for path in paths:
36+
if shutil.which("apt"):
37+
if not os.path.exists(f"{path}/config/extension.toml"):
38+
raise RuntimeError(
39+
"During the installation of an IsaacSim extension's dependencies, an extension.toml was unable to"
40+
" be found. All IsaacSim extensions must have a configuring .toml at"
41+
" (extension_root)/config/extension.toml"
42+
)
43+
with open(f"{path}/config/extension.toml") as fd:
44+
ext_toml = toml.load(fd)
45+
if "isaac_lab_settings" in ext_toml and "apt_deps" in ext_toml["isaac_lab_settings"]:
46+
deps = ext_toml["isaac_lab_settings"]["apt_deps"]
47+
print(f"[INFO] Installing the following apt packages: {deps}")
48+
run_and_print(["apt-get", "update"])
49+
run_and_print(["apt-get", "install", "-y"] + deps)
50+
else:
51+
print("[INFO] No apt packages to install")
52+
else:
53+
raise RuntimeError("Exiting because 'apt' is not a known command")
54+
55+
56+
def install_rosdep_packages(paths: list[str]):
57+
"""Attempts to install rosdep packages for Isaac Lab extensions.
58+
For each path in arg paths, it looks in {extension_root}/config/extension.toml for [isaac_lab_settings][ros_ws]
59+
and then attempts to install all rosdeps under that workspace.
60+
Exits on failure to stop the build process from continuing despite missing dependencies.
61+
62+
Args:
63+
path: A list of paths to the extension roots
64+
65+
Raises:
66+
RuntimeError: If 'rosdep' is not a known command
67+
"""
68+
for path in paths:
69+
if shutil.which("rosdep"):
70+
if not os.path.exists(f"{path}/config/extension.toml"):
71+
raise RuntimeError(
72+
"During the installation of an IsaacSim extension's dependencies, an extension.toml was unable to"
73+
" be found. All IsaacSim extensions must have a configuring .toml at"
74+
" (extension_root)/config/extension.toml"
75+
)
76+
with open(f"{path}/config/extension.toml") as fd:
77+
ext_toml = toml.load(fd)
78+
if "isaac_lab_settings" in ext_toml and "ros_ws" in ext_toml["isaac_lab_settings"]:
79+
ws_path = ext_toml["isaac_lab_settings"]["ros_ws"]
80+
if not os.path.exists("/etc/ros/rosdep/sources.list.d/20-default.list"):
81+
run_and_print(["rosdep", "init"])
82+
run_and_print(["rosdep", "update", "--rosdistro=humble"])
83+
run_and_print([
84+
"rosdep",
85+
"install",
86+
"--from-paths",
87+
f"{path}/{ws_path}/src",
88+
"--ignore-src",
89+
"-y",
90+
"--rosdistro=humble",
91+
])
92+
else:
93+
print("[INFO] No rosdep packages to install")
94+
else:
95+
raise RuntimeError("Exiting because 'rosdep' is not a known command")
96+
97+
98+
def run_and_print(args: list[str]):
99+
"""Runs a subprocess.run(args=args, capture_output=True, check=True),
100+
and prints the output
101+
102+
Args:
103+
args: a list of arguments to be passed to subprocess.run()
104+
"""
105+
completed_process = run(args=args, capture_output=True, check=True)
106+
print(f"{str(completed_process.stdout, encoding='utf-8')}")
107+
108+
109+
def main():
110+
args = parser.parse_args()
111+
# Get immediate children of args.extensions_dir
112+
extension_paths = [os.path.join(args.extensions_dir, x) for x in next(os.walk(args.extensions_dir))[1]]
113+
if args.type == "all":
114+
install_apt_packages(extension_paths)
115+
install_rosdep_packages(extension_paths)
116+
elif args.type == "apt":
117+
install_apt_packages(extension_paths)
118+
elif args.type == "rosdep":
119+
install_rosdep_packages(extension_paths)
120+
else:
121+
raise ValueError(f"'Invalid type dependency: '{args.type}'. Available options: ['all', 'apt', 'rosdep'].")
122+
123+
124+
if __name__ == "__main__":
125+
main()

0 commit comments

Comments
 (0)