Choice of state vector for the "EndEffectorPositioningURSim-v0" env

[brunoabpereira]

Hey, great job on this framework!

I've been playing with different DRL algorithms on the "EndEffectorPositioningURSim-v0" env and I've managed to make it work. I have two questions regarding your choice of the state vector:

1. why use the target coordinates with respect to the end effector reference frame rather than to the simulation reference frame at the base of the robot? (is it related with visual servoing?)
2. why only normalize the joint_positions and not the entire state? Could normalizing the joint_positions and target_polar yield better results in training?

Thanks,
Bruno

[matteolucchi]

Hi @brunoabpereira! Thank you for your kind words!

why use the target coordinates with respect to the end effector reference frame rather than to the simulation reference frame at the base of the robot? (is it related with visual servoing?)

We found out that this makes the problem way easier to solve for the learning agent, and given that the transformation between end effector and robot base is known and can be calculated at any time we see no point in not providing this information to the agent.

why only normalize the joint_positions and not the entire state? Could normalizing the joint_positions and target_polar yield better results in training?

Unfortunately we did not have time to test the performance of this combination, I cannot say whether this would help or not but it would be definitely interesting to try that! If you get the chance, and try this, I would be happy if you can report back the results that you obtain!

I hope this answers your questions. If you any further questions don't hesitate to ask.

Cheers,
Matteo

[matteolucchi]

I'm definitely thinking of trying different state vectors.

That sounds great!

Also didn't know there was a discussion tab, sorry for posting this as an issue.

No worries, it is a quite new feature.

One last question: for the gazebo simulation I changed the .world file to have a real_time_update_rate=0 (similarly to the empty_world_fast.world) and I get a real_time_factor of ~7 on my pc. This is nice because I get faster training times (and seems to work) but is it generally advisable?

The thing is that if your simulation runs 7 times faster, then your algorithm should also be able to generate actions 7 times faster. So, if your algorithm can keep up with the simulation speed then there is no reason to not go faster.

Cheers,
Matteo

[brunoabpereira]

Ok, I might be taking too long and skipping frames then. I'll look into how the timings work.
The robo-gym 2020 paper still describes this current version correctly for the ur robots?

[matteolucchi]

The robo-gym 2020 paper still describes this current version correctly for the ur robots?

For some things yes but also a lot was added after that, the obstacle avoidance environments were added after that. The List of environments contains a minimal description of the environments, otherwise there are comments directly in the code. If you have any doubts or if something is not clear just ask :)

[brunoabpereira]

Ok, I'll look into the code.
Thanks for all the help!

[matteolucchi]

No problem, glad to help! :)