Skip to content

Adds RSL-RL symmetry example for cartpole and ANYmal locomotion #3057

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Aug 16, 2025

Conversation

Mayankm96
Copy link
Contributor

@Mayankm96 Mayankm96 commented Jul 30, 2025

Description

This MR introduces the following:

  • An agent argument to all scripts to allow selecting different entry points (each then get resolved to their respective settings file).
  • Symmetry function for ANYmal robot for the locomotion task and cartpole balancing task
  • Documentation on how to configure RL training agent using gym resgistry

Fixes #2835

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Screenshots

Cartpole

# without symmetry
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-v0 --headless --agent rsl_rl_with_symmetry_cfg_entry_point --run_name ppo_with_no_symmetry agent.algorithm.symmetry_cfg.use_data_augmentation=false

# with symmetry
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-v0--headless --agent rsl_rl_with_symmetry_cfg_entry_point --run_name ppo_with_symmetry_data_augmentation  agent.algorithm.symmetry_cfg.use_data_augmentation=true
Isaac-Cartpole-v0 (pink w/o symmetry, blue w/ symmetry)
image

Locomotion

# without symmetry
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Rough-Anymal-D-v0 --headless --agent rsl_rl_with_symmetry_cfg_entry_point --run_name ppo_with_no_symmetry agent.algorithm.symmetry_cfg.use_data_augmentation=false

# with symmetry
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Rough-Anymal-D-v0 --headless --agent rsl_rl_with_symmetry_cfg_entry_point --run_name ppo_with_symmetry_data_augmentation  agent.algorithm.symmetry_cfg.use_data_augmentation=true
Isaac-Velocity-Rough-Anymal-D-v0 (green w/o symmetry, purple w/ symmetry)
image

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@Mayankm96
Copy link
Contributor Author

@Toni-SM for skrl it seems you have users specify the agent/algorithm name. I didn't want to change the behavior there but if you think above solution is generic (or also not - we can discuss), let me know how to proceed for skrl with it.

@Mayankm96 Mayankm96 changed the title Adds RSL-RL symmetry example for locomotion training Adds RSL-RL symmetry example for ANYmal locomotion Jul 30, 2025
@Mayankm96
Copy link
Contributor Author

@T-K-233 , I see you were working on something similar in #2836. Let me know your thoughts on the approach in this MR.

@Toni-SM
Copy link
Contributor

Toni-SM commented Jul 30, 2025

@Toni-SM for skrl it seems you have users specify the agent/algorithm name. I didn't want to change the behavior there but if you think above solution is generic (or also not - we can discuss), let me know how to proceed for skrl with it.

While this solution is more generic and doesn't require any code update if a new agent/algorithm is added... at the same time is more tedious to type:

--agent rl_games_cfg_entry_point vs --agent PPO

Up to you... I am fine with the proposed change

@Mayankm96 Mayankm96 marked this pull request as ready for review August 5, 2025 13:26
@Mayankm96 Mayankm96 added the enhancement New feature or request label Aug 5, 2025
@Mayankm96 Mayankm96 changed the title Adds RSL-RL symmetry example for ANYmal locomotion Adds RSL-RL symmetry example for cartpole and ANYmal locomotion Aug 5, 2025
Copy link
Collaborator

@ClemensSchwarke ClemensSchwarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you create a new entry point instead of a new task?

@Mayankm96
Copy link
Contributor Author

Why do you create a new entry point instead of a new task?

To avoid having a lot of duplications in the gym registry for every setting. If there are different algorithms (SAC, PPO etc) I think allowing users to pass entry point cfg makes it easier to scale.

Copy link

github-actions bot commented Aug 8, 2025

Test Results Summary

2 419 tests   2 011 ✅  2h 15m 57s ⏱️
   90 suites    408 💤
    1 files        0 ❌

Results for commit 6e261f0.

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@ooctipus ooctipus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very nice!

@ooctipus
Copy link
Collaborator

Do we need to update CHANGELOG or extension.toml for this one?

@Mayankm96 Mayankm96 merged commit 71608f9 into main Aug 16, 2025
4 checks passed
@Mayankm96 Mayankm96 deleted the feature/symmetry-examples branch August 16, 2025 04:27
StephenWelch pushed a commit to StephenWelch/IsaacLab that referenced this pull request Aug 17, 2025
…c-sim#3057)

# Description

This MR introduces the following:

* An `agent` argument to all scripts to allow selecting different entry
points (each then get resolved to their respective settings file).
* Symmetry function for ANYmal robot for the locomotion task and
cartpole balancing task
* Documentation on how to configure RL training agent using gym
resgistry

Fixes isaac-sim#2835

## Type of change

- New feature (non-breaking change which adds functionality)
- This change requires a documentation update

## Screenshots

### Cartpole

```bash
# without symmetry
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-v0 --headless --agent rsl_rl_with_symmetry_cfg_entry_point --run_name ppo_with_no_symmetry agent.algorithm.symmetry_cfg.use_data_augmentation=false

# with symmetry
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Cartpole-v0--headless --agent rsl_rl_with_symmetry_cfg_entry_point --run_name ppo_with_symmetry_data_augmentation  agent.algorithm.symmetry_cfg.use_data_augmentation=true
```

| Isaac-Cartpole-v0 (pink w/o symmetry, blue w/ symmetry) |
| ------ |
| <img width="823" height="421" alt="image"
src="https://github.com/user-attachments/assets/9c33db99-0d79-4c1d-b437-e01275d613b5"
/> |

### Locomotion

```bash

# without symmetry
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Rough-Anymal-D-v0 --headless --agent rsl_rl_with_symmetry_cfg_entry_point --run_name ppo_with_no_symmetry agent.algorithm.symmetry_cfg.use_data_augmentation=false

# with symmetry
./isaaclab.sh -p scripts/reinforcement_learning/rsl_rl/train.py --task Isaac-Velocity-Rough-Anymal-D-v0 --headless --agent rsl_rl_with_symmetry_cfg_entry_point --run_name ppo_with_symmetry_data_augmentation  agent.algorithm.symmetry_cfg.use_data_augmentation=true
```

| Isaac-Velocity-Rough-Anymal-D-v0 (green w/o symmetry, purple w/
symmetry) |
| ------ |
| <img width="1241" height="414" alt="image"
src="https://github.com/user-attachments/assets/625c125d-db9f-4006-9a62-0d55701a9407"
/> |

## Checklist

- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`./isaaclab.sh --format`
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] I have updated the changelog and the corresponding version in the
extension's `config/extension.toml` file
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------

Co-authored-by: Kelly Guo <kellyg@nvidia.com>
Co-authored-by: ooctipus <zhengyuz@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Proposal] Provide example usage for RslRlSymmetryCfg
5 participants