Skip to content

Commit a7f486e

Browse files
Mayankm96mohanksriram
authored andcommitted
Cleans up instructions for custom dependency installation (isaac-sim#621)
# Description This MR mainly fixes the documentation to make instructions clearer. The script was unclear about where the workspace path needs to be and how it can be resolved. The MR makes sure these cases are dealt and reported correctly. ## Type of change - This change requires a documentation update ## Checklist - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there
1 parent d5e7d25 commit a7f486e

File tree

2 files changed

+125
-56
lines changed

2 files changed

+125
-56
lines changed

docs/source/setup/developer.rst

Lines changed: 34 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -191,29 +191,46 @@ standalone applications.
191191
Extension Dependency Management
192192
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
193193

194-
Certain extensions may have dependencies which need to be installed before the extension can be run.
195-
While Python dependencies can be expressed via the ``INSTALL_REQUIRES`` array in ``setup.py``, we need
196-
a separate installation pipeline to handle non-Python dependencies. We have therefore created
197-
an additional setup procedure, ``python tools/install_deps.py {dep_type} {extensions_dir}``, which scans the ``extension.toml``
198-
file of the directories under the ``{extensions_dir}`` (such as ``${ISAACLAB_PATH}/source/extensions``) for ``apt`` and ``rosdep`` dependencies.
194+
Certain extensions may have dependencies which require installation of additional packages before the extension
195+
can be used. While Python dependencies are handled by the `setuptools <https://setuptools.readthedocs.io/en/latest/>`__
196+
package and specified in the ``setup.py`` file, non-Python dependencies such as `ROS <https://www.ros.org/>`__
197+
packages or `apt <https://en.wikipedia.org/wiki/APT_(software)>`__ packages are not handled by setuptools.
198+
To handle these dependencies, we have created an additional setup procedure described in the next section.
199199

200-
This example ``extension.toml`` has both ``apt_deps`` and ``ros_ws`` specified, so both
201-
``apt`` and ``rosdep`` packages will be installed if ``python tools/install_deps.py all ${ISAACLAB_PATH}/source/extensions``
202-
is passed:
200+
There are two types of dependencies that can be specified in the ``extension.toml`` file
201+
under the ``isaac_lab_settings`` section:
202+
203+
1. **apt_deps**: A list of apt packages that need to be installed. These are installed using the
204+
`apt <https://ubuntu.com/server/docs/package-management>`__ package manager.
205+
2. **ros_ws**: The path to the ROS workspace that contains the ROS packages. These are installed using
206+
the `rosdep <https://docs.ros.org/en/humble/Tutorials/Intermediate/Rosdep.html>`__ dependency manager.
207+
208+
As an example, the following ``extension.toml`` file specifies the dependencies for the extension:
203209

204210
.. code-block:: toml
205211
206-
[isaaclab_settings]
207-
apt_deps = ["example_package"]
208-
ros_ws = "path/from/extension_root/to/ros_ws"
212+
[isaac_lab_settings]
213+
# apt dependencies
214+
apt_deps = ["libboost-all-dev"]
209215
210-
From the ``apt_deps`` in the above example, the package ``example_package`` would be installed via ``apt``.
211-
From the ``ros_ws``, a ``rosdep install --from-paths {ros_ws}/src --ignore-src`` command will be called.
212-
This will install all the `ROS package.xml dependencies <https://docs.ros.org/en/humble/Tutorials/Intermediate/Rosdep.html>`__
213-
in the directory structure below. Currently the ROS distro is assumed to be ``humble``.
216+
# ROS workspace
217+
# note: if this path is relative, it is relative to the extension directory's root
218+
ros_ws = "/home/user/catkin_ws"
214219
215-
``apt`` deps are automatically installed this way during the build process of the ``Dockerfile.base``,
216-
and ``rosdep`` deps during the build process of ``Dockerfile.ros2``.
220+
These dependencies are installed using the ``install_deps.py`` script provided in the ``tools`` directory.
221+
To install all dependencies for all extensions, run the following command:
222+
223+
.. code-block:: bash
224+
225+
# execute from the root of the repository
226+
# the script expects the type of dependencies to install and the path to the extensions directory
227+
# available types are: 'apt', 'rosdep' and 'all'
228+
python tools/install_deps.py all ${ISAACLAB_PATH}/source/extensions
229+
230+
.. note::
231+
Currently, this script is automatically executed during the build process of the ``Dockerfile.base``
232+
and ``Dockerfile.ros2``. This ensures that all the 'apt' and 'rosdep' dependencies are installed
233+
before building the extensions respectively.
217234

218235

219236
Standalone applications

tools/install_deps.py

Lines changed: 91 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,27 @@
44
# SPDX-License-Identifier: BSD-3-Clause
55

66
"""
7-
A script with various methods of installing dependencies
8-
defined in an extension.toml
7+
This script is a utility to install dependencies mentioned in an extension.toml file of an extension.
8+
9+
The script takes in two arguments:
10+
11+
1. type: The type of dependencies to install. It can be one of the following: ['all', 'apt', 'rosdep'].
12+
2. extensions_dir: The path to the directory beneath which we search for extensions.
13+
14+
The script will search for all extensions in the extensions_dir and then look for an extension.toml file in each
15+
extension's config directory. If the extension.toml file exists, the script will look for the following keys in the
16+
[isaac_lab_settings] section:
17+
18+
* **apt_deps**: A list of apt packages to install.
19+
* **ros_ws**: The path to the ROS workspace in the extension. If the path is not absolute, the script assumes that
20+
the path is relative to the extension root and resolves it accordingly.
21+
22+
If the type is 'all', the script will install both apt and rosdep packages. If the type is 'apt', the script will only
23+
install apt packages. If the type is 'rosdep', the script will only install rosdep packages.
24+
25+
For more information, please check the `documentation`_.
26+
27+
.. _documentation: https://isaac-sim.github.io/IsaacLab/source/setup/developer.html#extension-dependency-management
928
"""
1029

1130
import argparse
@@ -15,31 +34,36 @@
1534
from subprocess import run
1635

1736
# add argparse arguments
18-
parser = argparse.ArgumentParser(description="Utility to install dependencies based on an extension.toml")
19-
parser.add_argument("type", type=str, choices=["all", "apt", "rosdep"], help="The type of packages to install")
20-
parser.add_argument("extensions_dir", type=str, help="The path to the directory beneath which we search for extensions")
37+
parser = argparse.ArgumentParser(description="A utility to install dependencies based on extension.toml files.")
38+
parser.add_argument("type", type=str, choices=["all", "apt", "rosdep"], help="The type of packages to install.")
39+
parser.add_argument("extensions_dir", type=str, help="The path to the directory containing extensions.")
40+
parser.add_argument("--ros_distro", type=str, default="humble", help="The ROS distribution to use for rosdep.")
2141

2242

2343
def install_apt_packages(paths: list[str]):
24-
"""Attempts to install apt packages for Isaac Lab extensions.
25-
For each path in arg paths, it looks in {extension_root}/config/extension.toml for [isaac_lab_settings][apt_deps]
26-
and then attempts to install them. Exits on failure to stop the build process
27-
from continuing despite missing dependencies.
44+
"""Installs apt packages listed in the extension.toml file for Isaac Lab extensions.
45+
46+
For each path in the input list of paths, the function looks in ``{path}/config/extension.toml`` for
47+
the ``[isaac_lab_settings][apt_deps]`` key. It then attempts to install the packages listed in the
48+
value of the key. The function exits on failure to stop the build process from continuing despite missing
49+
dependencies.
2850
2951
Args:
30-
paths: A list of paths to the extension root
52+
paths: A list of paths to the extension's root.
3153
3254
Raises:
33-
RuntimeError: If 'apt' is not a known command
55+
FileNotFoundError: If the extension.toml file is not found.
56+
SystemError: If 'apt' is not a known command. This is a system error.
3457
"""
3558
for path in paths:
3659
if shutil.which("apt"):
60+
# Check if the extension.toml file exists
3761
if not os.path.exists(f"{path}/config/extension.toml"):
38-
raise RuntimeError(
39-
"During the installation of an IsaacSim extension's dependencies, an extension.toml was unable to"
40-
" be found. All IsaacSim extensions must have a configuring .toml at"
41-
" (extension_root)/config/extension.toml"
62+
raise FileNotFoundError(
63+
"During the installation of 'apt' dependencies, unable to find a"
64+
f" valid file at: {path}/config/extension.toml."
4265
)
66+
# Load the extension.toml file and check for apt_deps
4367
with open(f"{path}/config/extension.toml") as fd:
4468
ext_toml = toml.load(fd)
4569
if "isaac_lab_settings" in ext_toml and "apt_deps" in ext_toml["isaac_lab_settings"]:
@@ -48,77 +72,105 @@ def install_apt_packages(paths: list[str]):
4872
run_and_print(["apt-get", "update"])
4973
run_and_print(["apt-get", "install", "-y"] + deps)
5074
else:
51-
print("[INFO] No apt packages to install")
75+
print(f"[INFO] No apt packages specified for the extension at: {path}")
5276
else:
53-
raise RuntimeError("Exiting because 'apt' is not a known command")
77+
raise SystemError("Unable to find 'apt' command. Please ensure that 'apt' is installed on your system.")
78+
5479

80+
def install_rosdep_packages(paths: list[str], ros_distro: str = "humble"):
81+
"""Installs ROS dependencies listed in the extension.toml file for Isaac Lab extensions.
5582
56-
def install_rosdep_packages(paths: list[str]):
57-
"""Attempts to install rosdep packages for Isaac Lab extensions.
58-
For each path in arg paths, it looks in {extension_root}/config/extension.toml for [isaac_lab_settings][ros_ws]
59-
and then attempts to install all rosdeps under that workspace.
60-
Exits on failure to stop the build process from continuing despite missing dependencies.
83+
For each path in the input list of paths, the function looks in ``{path}/config/extension.toml`` for
84+
the ``[isaac_lab_settings][ros_ws]`` key. It then attempts to install the ROS dependencies under the workspace
85+
listed in the value of the key. The function exits on failure to stop the build process from continuing despite
86+
missing dependencies.
87+
88+
If the path to the ROS workspace is not absolute, the function assumes that the path is relative to the extension
89+
root and resolves it accordingly. The function also checks if the ROS workspace exists before proceeding with
90+
the installation of ROS dependencies. If the ROS workspace does not exist, the function raises an error.
6191
6292
Args:
63-
path: A list of paths to the extension roots
93+
path: A list of paths to the extension roots.
94+
ros_distro: The ROS distribution to use for rosdep. Default is 'humble'.
6495
6596
Raises:
66-
RuntimeError: If 'rosdep' is not a known command
97+
FileNotFoundError: If the extension.toml file is not found under the path.
98+
FileNotFoundError: If a valid ROS workspace is not found while installing ROS dependencies.
99+
SystemError: If 'rosdep' is not a known command. This is raised if 'rosdep' is not installed on the system.
67100
"""
68101
for path in paths:
69102
if shutil.which("rosdep"):
103+
# Check if the extension.toml file exists
70104
if not os.path.exists(f"{path}/config/extension.toml"):
71-
raise RuntimeError(
72-
"During the installation of an IsaacSim extension's dependencies, an extension.toml was unable to"
73-
" be found. All IsaacSim extensions must have a configuring .toml at"
74-
" (extension_root)/config/extension.toml"
105+
raise FileNotFoundError(
106+
"During the installation of 'rosdep' dependencies, unable to find a"
107+
f" valid file at: {path}/config/extension.toml."
75108
)
109+
# Load the extension.toml file and check for ros_ws
76110
with open(f"{path}/config/extension.toml") as fd:
77111
ext_toml = toml.load(fd)
78112
if "isaac_lab_settings" in ext_toml and "ros_ws" in ext_toml["isaac_lab_settings"]:
113+
# resolve the path to the ROS workspace
79114
ws_path = ext_toml["isaac_lab_settings"]["ros_ws"]
115+
if not os.path.abspath(ws_path):
116+
ws_path = os.path.join(path, ws_path)
117+
# check if the workspace exists
118+
if not os.path.exists(f"{ws_path}/src"):
119+
raise FileNotFoundError(
120+
"During the installation of 'rosdep' dependencies, unable to find a"
121+
f" valid ROS workspace at: {path}/{ws_path}."
122+
)
123+
# install rosdep if not already installed
80124
if not os.path.exists("/etc/ros/rosdep/sources.list.d/20-default.list"):
81125
run_and_print(["rosdep", "init"])
82-
run_and_print(["rosdep", "update", "--rosdistro=humble"])
126+
run_and_print(["rosdep", "update", f"--rosdistro={ros_distro}"])
127+
# install rosdep packages
83128
run_and_print([
84129
"rosdep",
85130
"install",
86131
"--from-paths",
87-
f"{path}/{ws_path}/src",
132+
f"{ws_path}/src",
88133
"--ignore-src",
89134
"-y",
90-
"--rosdistro=humble",
135+
f"--rosdistro={ros_distro}",
91136
])
92137
else:
93-
print("[INFO] No rosdep packages to install")
138+
print(f"[INFO] No rosdep packages specified for the extension at: {path}")
94139
else:
95-
raise RuntimeError("Exiting because 'rosdep' is not a known command")
140+
raise SystemError(
141+
"Unable to find 'rosdep' command. Please ensure that 'rosdep' is installed on your system."
142+
"You can install it by running:\n\t sudo apt-get install python3-rosdep"
143+
)
96144

97145

98146
def run_and_print(args: list[str]):
99-
"""Runs a subprocess.run(args=args, capture_output=True, check=True),
100-
and prints the output
147+
"""Runs a subprocess and prints the output to stdout.
148+
149+
This function wraps subprocess.run() and prints the output to stdout.
101150
102151
Args:
103-
args: a list of arguments to be passed to subprocess.run()
152+
args: A list of arguments to pass to subprocess.run().
104153
"""
105154
completed_process = run(args=args, capture_output=True, check=True)
106155
print(f"{str(completed_process.stdout, encoding='utf-8')}")
107156

108157

109158
def main():
159+
# Parse the command line arguments
110160
args = parser.parse_args()
111161
# Get immediate children of args.extensions_dir
112162
extension_paths = [os.path.join(args.extensions_dir, x) for x in next(os.walk(args.extensions_dir))[1]]
163+
164+
# Install dependencies based on the type
113165
if args.type == "all":
114166
install_apt_packages(extension_paths)
115-
install_rosdep_packages(extension_paths)
167+
install_rosdep_packages(extension_paths, args.ros_distro)
116168
elif args.type == "apt":
117169
install_apt_packages(extension_paths)
118170
elif args.type == "rosdep":
119-
install_rosdep_packages(extension_paths)
171+
install_rosdep_packages(extension_paths, args.ros_distro)
120172
else:
121-
raise ValueError(f"'Invalid type dependency: '{args.type}'. Available options: ['all', 'apt', 'rosdep'].")
173+
raise ValueError(f"'Invalid dependency type: '{args.type}'. Available options: ['all', 'apt', 'rosdep'].")
122174

123175

124176
if __name__ == "__main__":

0 commit comments

Comments
 (0)