Implementation of Vectorized Environments Class for RL

[Ali-Hossam]

Overview

Added Vectorized Environments Wrapper to Reinforcement Learning Codebase.

Vectorized Environments are a method for stacking multiple independent environments into a single environment. Instead of training an RL agent on 1 environment per step, it allows us to train it on n environments per step.

Purpose

• Enhance RL Algorithms performance (Currently Supports only SAC).
• A first step in implementing PPO (Proximal Policy Optimization)

Code Changes

• Added VecEnv class which is the environment wrapper.
• Added VecEnvReplay class which is a wrapper for replay buffers that allows storing a vector of experiences of n Envs.
• Adapted SAC implementation to the new code.

Testing

• [vectorized_environment_test.cpp] Added tests to validate VecEnv.

Performance

By comparing SAC performance on the pendulum environment with and without Vectorized environment, the first one converged with almost 30 episodes less.

[mlpack-bot]

This issue has been automatically marked as stale because it has not had any recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions! 👍