Skip to content

Changes to allow learning for multiple agents together #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

MehaKaushik
Copy link
Collaborator

@MehaKaushik MehaKaushik commented Aug 6, 2018

This PR:

  1. Fixes the issue of opening of multiple TORCS windows, in the beginning of episodes.
  2. Fixes code crash when multiple agents are trained together.
  3. Introduces a new script multi_ddpg, where multiple agents learn using DDPG.
  4. State vector is changed from size 29 to 65, information about the opponents is included. This is the reading of 36 sensors on the vehicle, which correspond to the closest distance of any obstacle in line of that sensor.
  5. Now weights are stored for each agent in the folder weights/port_number after every 300 episode.
  6. Weights of training for 3 agents when trained together are present in weights.
  7. autostart.sh, has been added with extra commented lines for speeding up the TORCS simulator.

How to run:

  1. Run scripts/startTorcs.sh in a separate terminal.
  2. Run multi_ddpg.py in another terminal. Set the number of workers in the file multi_ddpg.py, in TORCS simulator select that many number of scr_server.
  3. In script/autostart.sh change the command "torcs" to "cd path_of_installation & ./torcs" if simply "torcs" does not launch TORCS for you.

ToDo:

  1. Readme to be changed once this PR is merged.
  2. In snakeoil3_gym.py n_fail needs to be adjusted as per the number of agents trained. For 6 agents, 10 was a suitable number.
  3. Some cleanups done previously are lost, need to redo them.
  4. Everytime a new episode starts, there is a redundant start where only the first client connects and rest cannot. This does not create any issues in training, but is not a good user experience, need to work this out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant