Name		Name	Last commit message	Last commit date
parent directory ..
bin		bin
runs_tables		runs_tables
README.md		README.md
get_finetune_scores.py		get_finetune_scores.py
get_finetune_tables_and_plots.py		get_finetune_tables_and_plots.py
get_finetune_urls.py		get_finetune_urls.py
get_offline_scores.py		get_offline_scores.py
get_offline_tables_and_plots.py		get_offline_tables_and_plots.py
get_offline_urls.py		get_offline_urls.py

README.md

Reproducing figures and tables

To reproduce all figures and tables from the paper, do the following steps:

Run get_{offline, finetune}_urls.py if needed. These scripts collect all wandb logs into .csv files and save them into the runs_tables folder. We provide the tables, but you can recollect them.
Run get_{offline, finetune}_scores.py if needed. These scripts collect data from runs kept in .csv files and save evaluation scores (and regret in case of offline-to-online) into pickled files, which are stored in the bin folder. We provide the pickled data, but if you need to extract more data, you can modify scripts for your purposes.
Run get_{offline, finetune}_tables_and_plots.py. These scripts use pickled data, print all the tables, and save all figures into the out directory.