RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learning

RLeXplore is a unified, highly-modularized and plug-and-play toolkit that currently provides high-quality and reliable implementations of eight representative intrinsic reward algorithms. It used to be challenging to compare intrinsic reward algorithms due to various confounding factors, including distinct implementations, optimization strategies, and evaluation methodologies. Therefore, RLeXplore is designed to provide unified and standardized procedures for constructing, computing, and optimizing intrinsic reward modules.

The workflow of RLeXplore is illustrated as follows:

Installation

• with pip recommended

Open a terminal and install rllte with pip:

conda create -n rllte python=3.8
pip install rllte-core 

• with git

Open a terminal and clone the repository from GitHub with git:

git clone https://github.com/RLE-Foundation/rllte.git
pip install -e .

Now you can invoke the intrinsic reward module by:

from rllte.xplore.reward import ICM, RIDE, ...

Module List

Type	Modules
Count-based	PseudoCounts, RND, E3B
Curiosity-driven	ICM, Disagreement, RIDE
Memory-based	NGU
Information theory-based	RE3

Tutorials

Click the following links to get the code notebook:

0. Quick Start
1. RLeXplore with RLLTE
2. RLeXplore with Stable-Baselines3
3. RLeXplore with CleanRL
4. Exploring Mixed Intrinsic Rewards
5. Custom Intrinsic Rewards

Benchmark Results

• RLLTE's PPO+RLeXplore on SuperMarioBros:

• CleanRL's PPO+RLeXplore's RND on Montezuma's Revenge:

Cite Us

To cite this repository in publications:

@software{yuan_roger@rlexplore,
	author = {Mingqi Yuan and Roger Creus Castanyer and Bo Li and Xin Jin and Glen Berseth and Wenjun Zeng},
	title = {RLeXplore: Accelerating Research in Intrinsically-Motivated Reinforcement Learning},
	url = {https://github.com/RLE-Foundation/RLeXplore},
	year = {2024},
}