RL algorithms

[bobvehs]

Hi :)
I am interested in your robo-gym toolkit and i was hoping you could help me with some of the starting dilemmas. My idea is to use your toolkit and test different RL algorithms and evaluate their succesfulness in tasks similar to your end effector positioning.
If i understand correctly this is a toolkit where you created several predefined environments (both for UR and Mir100 robots) and the required setup/architecture (all the necessary code) for easy implementation of the learned behaviour to the real robot? (one can also create his/her personal environment)
Looking at the available code here on github (ur_ee_positioning.py for example) i don't actually understand where the reinforcement learning part of the code is. My interpretation is that this (ur_ee_positioning.py) is merely an enviroment with all the necessary definitions whereas the reinforcement learning part is happening elsewhere. Nevertheless there are parts of code that seem like they are a part of RL; like reward and step methods...
I can see in the stable-baselines directory that there is a td3_script.py which is a RL algorithm. If i understand corrctly this is the script into which i "import" your (or my) enviroment and then start the training? In this case the TD3 algorithm which is provided by Stable Baselines. So the reinforcement learning part was provided by the stable baselines?
My question here would be: which part or what do i need to change/implement to apply different algorithms? Just the _td3_script.py or does that depend on the algorithm that i would like to implement?
Thanks for your help!

[matteolucchi]

Hi @bobvehs and thank you for your interest in robo-gym!

If i understand correctly this is a toolkit where you created several predefined environments (both for UR and Mir100 robots) and the required setup/architecture (all the necessary code) for easy implementation of the learned behaviour to the real robot? (one can also create his/her personal environment)

Yes you are absolutely right, it is like that.

Looking at the available code here on github (ur_ee_positioning.py for example) i don't actually understand where the reinforcement learning part of the code is. My interpretation is that this (ur_ee_positioning.py) is merely an enviroment with all the necessary definitions whereas the reinforcement learning part is happening elsewhere. Nevertheless there are parts of code that seem like they are a part of RL; like reward and step methods...

Exactly, robo-gym provides RL environments using the standard OpenAI gym interface, you can then train a RL algorithm such as TD3 or the ones included in stable baselines on these environments.

I would suggest you start with the basic OpenAI gym environments, have a look at the documentation and find some tutorials online where it is showed how to apply open source implementations of RL algorithms to gym environments. Once you are familiar with the framework and successfully trained algorithms on these environments I would suggest you come back here and try the same on our environments. Some good examples of environments to start with are Cartpole and Lunar Lander.

Hope this helps.

Cheers,

Matteo