Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update user guide #446

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open

update user guide #446

wants to merge 14 commits into from

Conversation

KohlerHECTOR
Copy link
Collaborator

@KohlerHECTOR KohlerHECTOR commented Apr 3, 2024 •
edited

Description

PR for issues #325 #353
I remove some place holders in the user guide that we would probably never do and I did a user guide page for plotting and statitstics.

Checklist

  • My code follows the style guideline
    To check :
    black --check examples rlberry *py
    flake8 --select F401,F405,D410,D411,D412 --exclude=rlberry/check_packages.py --per-file-ignores="init.py:F401",
  • I have commented my code, particularly in hard-to-understand areas,
  • I have made corresponding changes to the documentation,
  • I have added tests that prove my fix is effective or that my feature works,
  • New and existing unit tests pass locally with my changes,
  • If updated the changelog if necessary,
  • I have set the label "ready for review" and the checks are all green.

@KohlerHECTOR KohlerHECTOR added documentation Improvements or additions to documentation ready for review labels Apr 3, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 3, 2024

Hello,
The build of the doc succeeded. The documentation preview is available here:
https://rlberry-py.github.io/rlberry/preview_pr

@KohlerHECTOR KohlerHECTOR added the Marathon To do during Marathon label Apr 4, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 4, 2024

Hello,
The build of the doc succeeded. The documentation preview is available here:
https://rlberry-py.github.io/rlberry/preview_pr

TimotheeMathieu
TimotheeMathieu reviewed Apr 8, 2024
View reviewed changes
Copy link
Collaborator

@TimotheeMathieu TimotheeMathieu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this addition. However, for now, it is a bit difficult to review this. Can you clean up all the outputs?

docs/basics/userguide/plot_stats.md Outdated Show resolved Hide resolved
docs/basics/userguide/plot_stats.md Show resolved Hide resolved
docs/basics/userguide/plot_stats.md
[INFO] 14:46: [Sb3-PPO[worker: 6]] | max_global_step = 10240 | time/iterations = 4 | rollout/ep_rew_mean = 44.09 | rollout/ep_len_mean = 44.09 | time/fps = 114 | time/time_elapsed = 71 | time/total_timesteps = 8192 | train/learning_rate = 0.0003 | train/entropy_loss = -0.635085660405457 | train/policy_gradient_loss = -0.019524750619893894 | train/value_loss = 51.484919738769534 | train/approx_kl = 0.009435750544071198 | train/clip_fraction = 0.088720703125 | train/loss = 15.843541145324707 | train/explained_variance = 0.25749021768569946 | train/n_updates = 30 | train/clip_range = 0.2 | 
INFO:rlberry_logger:[Sb3-PPO[worker: 6]] | max_global_step = 10240 | time/iterations = 4 | rollout/ep_rew_mean = 44.09 | rollout/ep_len_mean = 44.09 | time/fps = 114 | time/time_elapsed = 71 | time/total_timesteps = 8192 | train/learning_rate = 0.0003 | train/entropy_loss = -0.635085660405457 | train/policy_gradient_loss = -0.019524750619893894 | train/value_loss = 51.484919738769534 | train/approx_kl = 0.009435750544071198 | train/clip_fraction = 0.088720703125 | train/loss = 15.843541145324707 | train/explained_variance = 0.25749021768569946 | train/n_updates = 30 | train/clip_range = 0.2 |
[INFO] 14:46: [Sb3-PPO[worker: 9]] | max_global_step = 10240 | time/iterations = 4 | rollout/ep_rew_mean = 45.69 | rollout/ep_len_mean = 45.69 | time/fps = 117 | time/time_elapsed = 69 | time/total_timesteps = 8192 | train/learning_rate = 0.0003 | train/entropy_loss = -0.6443713787943125 | train/policy_gradient_loss = -0.012846545978391077 | train/value_loss = 55.9754634976387 | train/approx_kl = 0.008374381810426712 | train/clip_fraction = 0.0560546875 | train/loss = 18.558977127075195 | train/explained_variance = 0.21047407388687134 | train/n_updates = 30 | train/clip_range = 0.2 | 
INFO:rlberry_logger:[Sb3-PPO[worker: 9]] | max_global_step = 10240 | time/iterations = 4 | rollout/ep_rew_mean = 45.69 | rollout/ep_len_mean = 45.69 | time/fps = 117 | time/time_elapsed = 69 | time/total_timesteps = 8192 |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not include colab stuffs. They are not useful for the doc and may cause bugs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't know how to remove, this was generated automatically with a jupyter to markdown converter.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just remove them manually.

docs/basics/userguide/plot_stats.md
[INFO] 14:46: [Sb3-PPO[worker: 6]] | max_global_step = 10240 | time/iterations = 4 | rollout/ep_rew_mean = 44.09 | rollout/ep_len_mean = 44.09 | time/fps = 114 | time/time_elapsed = 71 | time/total_timesteps = 8192 | train/learning_rate = 0.0003 | train/entropy_loss = -0.635085660405457 | train/policy_gradient_loss = -0.019524750619893894 | train/value_loss = 51.484919738769534 | train/approx_kl = 0.009435750544071198 | train/clip_fraction = 0.088720703125 | train/loss = 15.843541145324707 | train/explained_variance = 0.25749021768569946 | train/n_updates = 30 | train/clip_range = 0.2 | 
INFO:rlberry_logger:[Sb3-PPO[worker: 6]] | max_global_step = 10240 | time/iterations = 4 | rollout/ep_rew_mean = 44.09 | rollout/ep_len_mean = 44.09 | time/fps = 114 | time/time_elapsed = 71 | time/total_timesteps = 8192 | train/learning_rate = 0.0003 | train/entropy_loss = -0.635085660405457 | train/policy_gradient_loss = -0.019524750619893894 | train/value_loss = 51.484919738769534 | train/approx_kl = 0.009435750544071198 | train/clip_fraction = 0.088720703125 | train/loss = 15.843541145324707 | train/explained_variance = 0.25749021768569946 | train/n_updates = 30 | train/clip_range = 0.2 |
[INFO] 14:46: [Sb3-PPO[worker: 9]] | max_global_step = 10240 | time/iterations = 4 | rollout/ep_rew_mean = 45.69 | rollout/ep_len_mean = 45.69 | time/fps = 117 | time/time_elapsed = 69 | time/total_timesteps = 8192 | train/learning_rate = 0.0003 | train/entropy_loss = -0.6443713787943125 | train/policy_gradient_loss = -0.012846545978391077 | train/value_loss = 55.9754634976387 | train/approx_kl = 0.008374381810426712 | train/clip_fraction = 0.0560546875 | train/loss = 18.558977127075195 | train/explained_variance = 0.21047407388687134 | train/n_updates = 30 | train/clip_range = 0.2 | 
INFO:rlberry_logger:[Sb3-PPO[worker: 9]] | max_global_step = 10240 | time/iterations = 4 | rollout/ep_rew_mean = 45.69 | rollout/ep_len_mean = 45.69 | time/fps = 117 | time/time_elapsed = 69 | time/total_timesteps = 8192 |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, write an explanation on how to interpret the results.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with a todo

docs/user_guide.md Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 8, 2024

Hello,
The build of the doc succeeded. The documentation preview is available here:
https://rlberry-py.github.io/rlberry/preview_pr

@github-actions GitHub Actions
Copy link

github-actions bot commented Apr 8, 2024

Hello,
The build of the doc succeeded. The documentation preview is available here:
https://rlberry-py.github.io/rlberry/preview_pr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TimotheeMathieu TimotheeMathieu TimotheeMathieu left review comments

@JulienT01 JulienT01 Awaiting requested review from JulienT01

Assignees
No one assigned
Labels
documentation Improvements or additions to documentation Marathon To do during Marathon ready for review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@KohlerHECTOR @TimotheeMathieu