This repository hosts the source code for the "Sophisticated Learning" algorithm as described in the research paper:
Hodson, et al. "Planning to Learn: A Novel Algorithm for Active Learning during Model-Based Planning."
The codebase includes implementations for all four algorithms presented in the paper: Sophisticated Learning, Sophisticated Inference, Bayes-Adaptive RL, and Bayes-Adaptive RL + UCB. Each MATLAB script within the 'Algorithms' directory is named according to the algorithm it implements.
This code was developed by:
- Rowan Hodson @RowanLIBR
- St John Grimbly @sgrimbly
- Evert Boonstra @eboonst
To run one simulation of 120 iterations of any algorithm, navigate to the 'matlab' directory and execute the main.m
script. Set run_SL
, run_SI
, run_BA
, or run_BAUCB
to 1 to activate Sophisticated Learning, Sophisticated Inference, Bayes-Adaptive RL, and Bayes-Adaptive RL + UCB, respectively.
To use the MATLAB's Parallel Computing Toolbox, use the parallel_runs_PC.m
script in src/MATLAB/Algorithms
. This script will run 30 simulations (seeds) of 120 iterations of the selected algorithm in parallel. To use the Weights & Biases feature, this requires Python integration. Check MATLAB Python requirements here.
The repository contains MATLAB and Python implementations intended for reproducing the research results and further development.
- MATLAB (R2018b or newer recommended)
- Python 3.6+ (with virtual environment recommended)
Ensure MATLAB is installed. The scripts should run in any standard MATLAB environment.
Set up a Python virtual environment and install dependencies:
python -m venv venv
source venv/bin/activate # On Windows use `venv\Scripts\activate`
pip install -r python/requirements.txt
Navigate to the matlab
folder and execute the scripts:
cd matlab
main
Ensure you are in the virtual environment and run the Python scripts:
cd python
python main.py
Here are a few quick examples to get started:
% Load data
data = load('example_data.mat');
% Run algorithm
results = sophisticated_learning(data);
# Load data
data = load_data('example_data.json')
# Run algorithm
results = sophisticated_learning(data)
If you use this code for your research, please cite it as follows:
@misc{hodson2023planning,
title={Planning to Learn: A Novel Algorithm for Active Learning during Model-Based Planning},
author={Rowan Hodson and St John Grimbly and Evert Boonstra and Bruce Bassett and Charel van Hoof and Benjamin Rosman and Mark Solms and Jonathan P. Shock and Ryan Smith},
year={2023},
eprint={2308.08029}
}
Contributions are welcome! Please open an issue or submit a pull request.
This project is licensed under the MIT License - see the LICENSE file for details.
This code is provided "as is", without warranty of any kind, express or implied. Please note that the MATLAB version may have dependencies on specific toolboxes.
For any questions or concerns, please open an issue on GitHub or contact the corresponding author of the associated papers.