Skip to content

Feature that enable modify hydra configuration group through command line #2737

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions docs/source/features/hydra.rst
Original file line number Diff line number Diff line change
Expand Up @@ -127,3 +127,137 @@ the post init update is as follows:

Here, when modifying ``env.decimation`` or ``env.sim.dt``, the user needs to give the updated ``env.sim.render_interval``,
``env.scene.height_scanner.update_period``, and ``env.scene.contact_forces.update_period`` as input as well.


Group Override
--------------
Group override lets you swap out entire groups of environment- or agent-level settings in one go.
Instead of overriding individual fields, you select a named preset defined in your code.


Group Presets
^^^^^^^^^^^^^
First define the available group override options


.. code-block:: python

@configclass
class StateNoNoiseObservationsCfg:
"""Observation specifications for the MDP."""

@configclass
class PolicyCfg(ObsGroup):
"""Observations for policy group."""

# observation terms (order preserved)
joint_pos = ObsTerm(func=mdp.joint_pos_rel)
# other terms .......

def __post_init__(self):
self.enable_corruption = False
self.concatenate_terms = True

# observation groups
policy: PolicyCfg = PolicyCfg()


@configclass
class EnvConfigurables:
env: dict[str, any] = {
"observations": {
"state_obs_no_noise": StateNoNoiseObservationsCfg(),
"state_obs_noisy": # other option,
},
"actions.arm_action": {
"joint_pos_arm_action": mdp.JointPositionActionCfg(
asset_name="robot", joint_names=["panda_joint.*"], scale=0.5, use_default_offset=True
),
"osc_arm_action": mdp.OperationalSpaceControllerActionCfg(
asset_name="robot",
# rest of fields
),
},
"events": {
"rand_joint_pos_friction": JointRandPositionFrictionEventCfg(),
"rand_joint_pos_friction_amarture": JointRandPositionFrictionAmartureEventCfg(),
},
"events.reset_robot_joints": {
"aggressive": EventTerm(
func=mdp.reset_joints_by_scale,
mode="reset",
params={
"position_range": (0.0, 2.0),
"velocity_range": (0.0, 1.0),
},
),
"easy": # easy EventTerm with narrower ranges
},
}



@configclass
class AgentConfigurables(EnvConfigurables):
agent: dict[str, any] = {
"policy": {
"large_network": RslRlPpoActorCriticCfg(
init_noise_std=1.0,
actor_hidden_dims=[512, 256, 128, 64],
critic_hidden_dims=[512, 256, 128, 64],
activation="elu",
),
"medium_network": RslRlPpoActorCriticCfg(
init_noise_std=1.0,
actor_hidden_dims=[256, 128, 64],
critic_hidden_dims=[256, 128, 64],
activation="elu",
),
"small_network": RslRlPpoActorCriticCfg(
init_noise_std=1.0,
actor_hidden_dims=[128, 64],
critic_hidden_dims=[128, 64],
activation="elu",
),
},
# algorithm cfg.....
}


Group Registration
^^^^^^^^^^^^^^^^^^
When you register your Gym environment, provide the ``configurable_entry_point`` pointing to your ``@configclass``:

.. code-block:: python

gym.register(
id="Isaac-Reach-Franka-v0",
entry_point="isaaclab.envs:ManagerBasedRLEnv",
disable_env_checker=True,
kwargs={
# … other cfg entry points …
"configurable_entry_point": f"{agents.__name__}.configurables:AgentConfigurables"
},
)


Override Syntax
^^^^^^^^^^^^^^^
Select one preset per group via Hydra-style CLI flags. For example::

python scripts/reinforcement_learning/rsl_rl/train.py \
--task=Isaac-Reach-Franka-v0 \
--headless \
env.events=rand_joint_pos_friction_amarture \
env.observations=state_obs_no_noise \
env.actions.arm_action=osc_arm_action \
agent.policy=large_network

Under the hood, Hydra will replace:

- ``env.events`` with ``EnvConfigurables.env["rand_joint_pos_friction_amarture"]``
- ``env.observations`` with ``EnvConfigurables.env["state_obs_no_noise"]``
- ``env.actions.arm_action`` with ``EnvConfigurables.env["actions.arm_action"]["osc_arm_action"]``
- ``agent.policy`` with ``AgentConfigurables.agent["large_network"]``

allowing you to switch qualitative modes of your experiments with a single flag.
2 changes: 1 addition & 1 deletion source/isaaclab_tasks/config/extension.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[package]

# Note: Semantic Versioning is used: https://semver.org/
version = "0.10.45"
version = "0.11.0"

# Description
title = "Isaac Lab Environments"
Expand Down
9 changes: 9 additions & 0 deletions source/isaaclab_tasks/docs/CHANGELOG.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,15 @@
Changelog
---------

0.11.0 (2025-07-05)
~~~~~~~~~~~~~~~~~~~~

Changed
^^^^^^^

* Add new feature that support hydra group config override, and provide example at Isaac-Reach-Franka-v0 env


0.10.45 (2025-07-16)
~~~~~~~~~~~~~~~~~~~~

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
"rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_ppo_cfg.yaml",
"rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:FrankaReachPPORunnerCfg",
"skrl_cfg_entry_point": f"{agents.__name__}:skrl_ppo_cfg.yaml",
"configurable_entry_point": f"{agents.__name__}.configurables:AgentConfigurables",
},
)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Copyright (c) 2022-2025, The Isaac Lab Project Developers (https://github.com/isaac-sim/IsaacLab/blob/main/CONTRIBUTORS.md).
# All rights reserved.
#
# SPDX-License-Identifier: BSD-3-Clause

from isaaclab.utils import configclass

from isaaclab_rl.rsl_rl import RslRlPpoActorCriticCfg, RslRlPpoAlgorithmCfg

from ..configurables import EnvConfigurables


@configclass
class AgentConfigurables(EnvConfigurables):
agent: dict[str, any] = {
"policy": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how would this work for other RL libraries that don't use yaml?

"large_network": RslRlPpoActorCriticCfg(
init_noise_std=1.0,
actor_hidden_dims=[512, 256, 128, 64],
critic_hidden_dims=[512, 256, 128, 64],
activation="elu",
),
"medium_network": RslRlPpoActorCriticCfg(
init_noise_std=1.0,
actor_hidden_dims=[256, 128, 64],
critic_hidden_dims=[256, 128, 64],
activation="elu",
),
"small_network": RslRlPpoActorCriticCfg(
init_noise_std=1.0,
actor_hidden_dims=[128, 64],
critic_hidden_dims=[128, 64],
activation="elu",
),
},
"algorithm": {
"standard": RslRlPpoAlgorithmCfg(
value_loss_coef=1.0,
use_clipped_value_loss=True,
clip_param=0.2,
entropy_coef=0.001,
num_learning_epochs=8,
num_mini_batches=4,
learning_rate=1.0e-3,
schedule="adaptive",
gamma=0.99,
lam=0.95,
desired_kl=0.01,
max_grad_norm=1.0,
),
"small_batch": RslRlPpoAlgorithmCfg(
value_loss_coef=1.0,
use_clipped_value_loss=True,
clip_param=0.2,
entropy_coef=0.001,
num_learning_epochs=8,
num_mini_batches=16,
learning_rate=1.0e-4,
schedule="adaptive",
gamma=0.99,
lam=0.95,
desired_kl=0.01,
max_grad_norm=1.0,
),
},
}
Loading
Loading