[BugFix,Feature] Allow non-tensor data in envs

[vmoens]

Depends on pytorch/tensordict#685

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1944

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 34 New Failures, 3 Unrelated Failures

As of commit f7bb04b with merge base 13bef42 ():

NEW FAILURES - The following jobs have failed:

• Continuous Benchmark (PR) / CPU Pytest benchmark (gh)
  Process completed with exit code 4.
• Continuous Benchmark (PR) / GPU Pytest benchmark (gh)
  Process completed with exit code 4.
• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 9df1bf2421cdef69b0062c97d3d4c7d05398b9a0f5d08a9f1059d1a1f543fa9b /exec failed with exit code 1
• Generate documentation / build-docs (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t c8efc8beb6eaddddec19d30335b343f1576a81800234935ec03867bc2581e40a /exec failed with exit code 1
• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 08fb2883488a143df070ecf03fb3109f0b69afc8e4560a9b1bc615a7549c3ee0 /exec failed with exit code 1
• Libs Tests on Linux / unittests-brax (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 92c092d075821888bbf8efd37cb3491bdaee6c56e9fcff80d50eb3137430dd16 /exec failed with exit code 1
• Libs Tests on Linux / unittests-gym (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 256cc89998f4d6b62a667edaccd3fce4d5b480cbbc802dee8fb0717a08dee5eb /exec failed with exit code 1
• Libs Tests on Linux / unittests-jumanji (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 17f5de8457bb94384243760079f4d67b03f36d20ddef7071b57fedda6d7ad30e /exec failed with exit code 1
• Libs Tests on Linux / unittests-pettingzoo / linux-job (gh)
  RuntimeError: Command docker exec -t 7db3afeeb9823e7dace76c4d6ccbe5ea253eea1ff0e5965199cda9811ae21b28 /exec failed with exit code 1
• Libs Tests on Linux / unittests-robohive (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t ab9a0afb192cbba8d137778c02083bda82303a00f87f78f0f567563f70eab952 /exec failed with exit code 1
• Libs Tests on Linux / unittests-sklearn (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 808cdc6fb1a0960100e036c93ab744870529a83739ef29fdf3600c6178f38f5a /exec failed with exit code 1
• Libs Tests on Linux / unittests-vmas (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 1ef49cd0e86b34a4d2e40330e2e3845c52c9fb5cf163bbb232b34c34aa1eef49 /exec failed with exit code 1
• RLHF Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t dfc50b11009cf8ce1c66c562502c30081eab9ad4b96e3642899a5de82a361a95 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.10) / linux-job (gh)
  RuntimeError: Command docker exec -t e63c39e12682048f75e70cd7119cf39890e8dbe9e0a3302d9eb5cc5854b4dec7 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.11) / linux-job (gh)
  RuntimeError: Command docker exec -t 9d13ed891775158d6716a2dca2caa858149bc918e3a6d87afde79a22d715dc37 /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 4e229922c44e3c7a4a2a0b52372f8304a7946201a7667483ab85fd53c610fe1a /exec failed with exit code 1
• Unit-tests on Linux / tests-cpu (3.9) / linux-job (gh)
  RuntimeError: Command docker exec -t 2de804c99f308369e8df7db5bf2a5af57c0cea5e93dd7bac93a817111c8eb11c /exec failed with exit code 1
• Unit-tests on Linux / tests-gpu (3.8, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t 8601b91dc065735c956c3e49f3ef8f2d2cbb886efde29d0e9bc8a6bcab96ce21 /exec failed with exit code 1
• Unit-tests on Linux / tests-olddeps (3.8, 11.6) / linux-job (gh)
  RuntimeError: Command docker exec -t 346753412e8ba0bf3cb38b04ff4db92349e719a195a6911d5d3308fc36efbe13 /exec failed with exit code 1
• Unit-tests on Linux / tests-optdeps (3.9, 12.1) / linux-job (gh)
  RuntimeError: Command docker exec -t b9b0b33803e089f658f33430af3bf999a70daa3a63b7aefe595570916cda5791 /exec failed with exit code 1
• Unit-tests on Linux / tests-stable-gpu (3.8, 11.8) / linux-job (gh)
  RuntimeError: Command docker exec -t 4de2e830d2f43852f3a60cb7a2c35fc8e6647222aadb3321fc98646de212b8f1 /exec failed with exit code 1
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
  RuntimeError: Command bash /Users/runner/work/_temp/exec_script failed with exit code 1
• Unit-tests on Windows / unittests-cpu / windows-job (gh)
  ImportError: cannot import name 'NonTensorStack' from 'tensordict.tensorclass' (c:\actions-runner\_work\rl\rl\pytorch\rl\tensordict\tensordict\tensorclass.py)
• Wheels / test-wheel (linux, ubuntu-20.04, 3.10) (gh)
  ##[error]The operation was canceled.
• Wheels / test-wheel (linux, ubuntu-20.04, 3.11) (gh)
  ##[error]The operation was canceled.
• Wheels / test-wheel (linux, ubuntu-20.04, 3.8) (gh)
  ImportError: cannot import name 'NonTensorStack' from 'tensordict.tensorclass' (/opt/hostedtoolcache/Python/3.8.18/x64/lib/python3.8/site-packages/tensordict/tensorclass.py)
• Wheels / test-wheel (linux, ubuntu-20.04, 3.9) (gh)
  ##[error]The operation was canceled.
• Wheels / test-wheel (mac, macos-11, 3.10) (gh)
  ##[error]The operation was canceled.
• Wheels / test-wheel (mac, macos-11, 3.11) (gh)
• Wheels / test-wheel (mac, macos-11, 3.8) (gh)
• Wheels / test-wheel (mac, macos-11, 3.9) (gh)
  ##[error]The operation was canceled.
• Wheels / test-wheel-windows (3.11) (gh)
  ImportError: cannot import name 'NonTensorStack' from 'tensordict.tensorclass' (C:\hostedtoolcache\windows\Python\3.11.8\x64\Lib\site-packages\tensordict\tensorclass.py)
• Wheels / test-wheel-windows (3.8) (gh)
  ImportError: cannot import name 'NonTensorStack' from 'tensordict.tensorclass' (C:\hostedtoolcache\windows\Python\3.8.10\x64\lib\site-packages\tensordict\tensorclass.py)

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Wheels / test-wheel-windows (3.10) (gh)
  ##[error]The operation was canceled.
• Wheels / test-wheel-windows (3.9) (gh)
  ##[error]The operation was canceled.

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Unit-tests on Windows / unittests-gpu / windows-job (gh)
  ##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}1$. Worsened: $\large\color{#d91a1a}20$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	60.7350ms	60.4393ms	16.5455 Ops/s	15.5067 Ops/s	$\textbf{\color{#35bf28}+6.70\%}$
test_sync	34.9953ms	33.0404ms	30.2660 Ops/s	29.7062 Ops/s	$\color{#35bf28}+1.88\%$
test_async	62.1604ms	31.1414ms	32.1116 Ops/s	31.8264 Ops/s	$\color{#35bf28}+0.90\%$
test_simple	0.4476s	0.4256s	2.3494 Ops/s	2.3164 Ops/s	$\color{#35bf28}+1.42\%$
test_transformed	0.6215s	0.5776s	1.7312 Ops/s	1.7025 Ops/s	$\color{#35bf28}+1.69\%$
test_serial	1.4601s	1.4160s	0.7062 Ops/s	0.6905 Ops/s	$\color{#35bf28}+2.28\%$
test_parallel	1.4022s	1.3491s	0.7412 Ops/s	0.7278 Ops/s	$\color{#35bf28}+1.85\%$
test_step_mdp_speed[True-True-True-True-True]	0.1115ms	22.4382μs	44.5668 KOps/s	47.4049 KOps/s	$\textbf{\color{#d91a1a}-5.99\%}$
test_step_mdp_speed[True-True-True-True-False]	36.2580μs	13.6330μs	73.3516 KOps/s	77.8222 KOps/s	$\textbf{\color{#d91a1a}-5.74\%}$
test_step_mdp_speed[True-True-True-False-True]	45.7250μs	12.9134μs	77.4387 KOps/s	81.0827 KOps/s	$\color{#d91a1a}-4.49\%$
test_step_mdp_speed[True-True-True-False-False]	32.3790μs	7.9339μs	126.0412 KOps/s	134.7276 KOps/s	$\textbf{\color{#d91a1a}-6.45\%}$
test_step_mdp_speed[True-True-False-True-True]	59.2600μs	23.8864μs	41.8649 KOps/s	44.3676 KOps/s	$\textbf{\color{#d91a1a}-5.64\%}$
test_step_mdp_speed[True-True-False-True-False]	58.3380μs	14.9035μs	67.0985 KOps/s	70.3882 KOps/s	$\color{#d91a1a}-4.67\%$
test_step_mdp_speed[True-True-False-False-True]	33.5420μs	14.2287μs	70.2803 KOps/s	73.4755 KOps/s	$\color{#d91a1a}-4.35\%$
test_step_mdp_speed[True-True-False-False-False]	41.8480μs	9.1520μs	109.2657 KOps/s	113.9686 KOps/s	$\color{#d91a1a}-4.13\%$
test_step_mdp_speed[True-False-True-True-True]	51.8170μs	25.3700μs	39.4167 KOps/s	42.1234 KOps/s	$\textbf{\color{#d91a1a}-6.43\%}$
test_step_mdp_speed[True-False-True-True-False]	54.2310μs	16.4073μs	60.9483 KOps/s	64.1913 KOps/s	$\textbf{\color{#d91a1a}-5.05\%}$
test_step_mdp_speed[True-False-True-False-True]	60.6730μs	14.3757μs	69.5618 KOps/s	73.1842 KOps/s	$\color{#d91a1a}-4.95\%$
test_step_mdp_speed[True-False-True-False-False]	25.2470μs	9.2295μs	108.3481 KOps/s	114.9200 KOps/s	$\textbf{\color{#d91a1a}-5.72\%}$
test_step_mdp_speed[True-False-False-True-True]	90.8690μs	26.8029μs	37.3094 KOps/s	39.7122 KOps/s	$\textbf{\color{#d91a1a}-6.05\%}$
test_step_mdp_speed[True-False-False-True-False]	38.8720μs	17.6830μs	56.5514 KOps/s	59.7236 KOps/s	$\textbf{\color{#d91a1a}-5.31\%}$
test_step_mdp_speed[True-False-False-False-True]	52.1160μs	15.4341μs	64.7915 KOps/s	68.3215 KOps/s	$\textbf{\color{#d91a1a}-5.17\%}$
test_step_mdp_speed[True-False-False-False-False]	33.7020μs	10.3986μs	96.1663 KOps/s	100.3976 KOps/s	$\color{#d91a1a}-4.21\%$
test_step_mdp_speed[False-True-True-True-True]	73.7870μs	25.3252μs	39.4864 KOps/s	42.2014 KOps/s	$\textbf{\color{#d91a1a}-6.43\%}$
test_step_mdp_speed[False-True-True-True-False]	59.9210μs	16.4860μs	60.6575 KOps/s	64.3992 KOps/s	$\textbf{\color{#d91a1a}-5.81\%}$
test_step_mdp_speed[False-True-True-False-True]	36.9080μs	16.7620μs	59.6588 KOps/s	63.5291 KOps/s	$\textbf{\color{#d91a1a}-6.09\%}$
test_step_mdp_speed[False-True-True-False-False]	47.0270μs	10.5248μs	95.0139 KOps/s	99.9707 KOps/s	$\color{#d91a1a}-4.96\%$
test_step_mdp_speed[False-True-False-True-True]	37.6100μs	27.0594μs	36.9558 KOps/s	39.3647 KOps/s	$\textbf{\color{#d91a1a}-6.12\%}$
test_step_mdp_speed[False-True-False-True-False]	47.8090μs	17.6179μs	56.7605 KOps/s	60.1916 KOps/s	$\textbf{\color{#d91a1a}-5.70\%}$
test_step_mdp_speed[False-True-False-False-True]	63.0870μs	18.0575μs	55.3787 KOps/s	58.7691 KOps/s	$\textbf{\color{#d91a1a}-5.77\%}$
test_step_mdp_speed[False-True-False-False-False]	36.5280μs	11.7770μs	84.9110 KOps/s	89.9181 KOps/s	$\textbf{\color{#d91a1a}-5.57\%}$
test_step_mdp_speed[False-False-True-True-True]	90.5490μs	27.9864μs	35.7317 KOps/s	38.1119 KOps/s	$\textbf{\color{#d91a1a}-6.25\%}$
test_step_mdp_speed[False-False-True-True-False]	62.3960μs	19.0110μs	52.6011 KOps/s	55.3291 KOps/s	$\color{#d91a1a}-4.93\%$
test_step_mdp_speed[False-False-True-False-True]	44.2120μs	17.7773μs	56.2515 KOps/s	58.6570 KOps/s	$\color{#d91a1a}-4.10\%$
test_step_mdp_speed[False-False-True-False-False]	54.4110μs	11.7594μs	85.0387 KOps/s	89.4428 KOps/s	$\color{#d91a1a}-4.92\%$
test_step_mdp_speed[False-False-False-True-True]	60.3320μs	29.0393μs	34.4360 KOps/s	36.4230 KOps/s	$\textbf{\color{#d91a1a}-5.46\%}$
test_step_mdp_speed[False-False-False-True-False]	63.0170μs	20.0667μs	49.8338 KOps/s	51.7028 KOps/s	$\color{#d91a1a}-3.61\%$
test_step_mdp_speed[False-False-False-False-True]	64.0390μs	18.9764μs	52.6971 KOps/s	55.2339 KOps/s	$\color{#d91a1a}-4.59\%$
test_step_mdp_speed[False-False-False-False-False]	44.1330μs	12.8789μs	77.6462 KOps/s	81.4389 KOps/s	$\color{#d91a1a}-4.66\%$
test_values[generalized_advantage_estimate-True-True]	9.3379ms	9.1084ms	109.7892 Ops/s	112.3929 Ops/s	$\color{#d91a1a}-2.32\%$
test_values[vec_generalized_advantage_estimate-True-True]	35.0795ms	32.8703ms	30.4226 Ops/s	30.3468 Ops/s	$\color{#35bf28}+0.25\%$
test_values[td0_return_estimate-False-False]	0.2236ms	0.1634ms	6.1200 KOps/s	6.0887 KOps/s	$\color{#35bf28}+0.52\%$
test_values[td1_return_estimate-False-False]	25.4497ms	22.3961ms	44.6507 Ops/s	44.6532 Ops/s	$-0.01\%$
test_values[vec_td1_return_estimate-False-False]	34.5498ms	33.1053ms	30.2066 Ops/s	30.3140 Ops/s	$\color{#d91a1a}-0.35\%$
test_values[td_lambda_return_estimate-True-False]	35.6803ms	32.4916ms	30.7772 Ops/s	30.4305 Ops/s	$\color{#35bf28}+1.14\%$
test_values[vec_td_lambda_return_estimate-True-False]	34.4784ms	33.1851ms	30.1340 Ops/s	30.3584 Ops/s	$\color{#d91a1a}-0.74\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	10.1786ms	8.0253ms	124.6065 Ops/s	127.4064 Ops/s	$\color{#d91a1a}-2.20\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	2.4730ms	1.9889ms	502.7904 Ops/s	511.2677 Ops/s	$\color{#d91a1a}-1.66\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5276ms	0.3388ms	2.9516 KOps/s	2.9565 KOps/s	$\color{#d91a1a}-0.16\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	49.1525ms	46.3154ms	21.5911 Ops/s	25.3295 Ops/s	$\textbf{\color{#d91a1a}-14.76\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	3.7596ms	2.9990ms	333.4462 Ops/s	333.6020 Ops/s	$\color{#d91a1a}-0.05\%$
test_dqn_speed	60.7375ms	1.4563ms	686.6801 Ops/s	720.3995 Ops/s	$\color{#d91a1a}-4.68\%$
test_ddpg_speed	4.6773ms	2.7751ms	360.3485 Ops/s	359.9869 Ops/s	$\color{#35bf28}+0.10\%$
test_sac_speed	8.8906ms	8.1958ms	122.0140 Ops/s	121.5992 Ops/s	$\color{#35bf28}+0.34\%$
test_redq_speed	14.2755ms	12.7272ms	78.5716 Ops/s	77.2451 Ops/s	$\color{#35bf28}+1.72\%$
test_redq_deprec_speed	13.9277ms	13.0287ms	76.7539 Ops/s	76.5729 Ops/s	$\color{#35bf28}+0.24\%$
test_td3_speed	8.3827ms	8.1597ms	122.5542 Ops/s	121.3489 Ops/s	$\color{#35bf28}+0.99\%$
test_cql_speed	36.5589ms	35.8036ms	27.9302 Ops/s	27.7862 Ops/s	$\color{#35bf28}+0.52\%$
test_a2c_speed	8.9367ms	7.2932ms	137.1132 Ops/s	137.0317 Ops/s	$\color{#35bf28}+0.06\%$
test_ppo_speed	7.8257ms	7.5587ms	132.2980 Ops/s	132.2282 Ops/s	$\color{#35bf28}+0.05\%$
test_reinforce_speed	7.8410ms	6.6622ms	150.0995 Ops/s	152.8759 Ops/s	$\color{#d91a1a}-1.82\%$
test_iql_speed	33.7130ms	32.1423ms	31.1116 Ops/s	30.9179 Ops/s	$\color{#35bf28}+0.63\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	4.0858ms	2.6919ms	371.4874 Ops/s	372.4931 Ops/s	$\color{#d91a1a}-0.27\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6173ms	0.5042ms	1.9835 KOps/s	1.9716 KOps/s	$\color{#35bf28}+0.60\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.5777ms	0.4767ms	2.0978 KOps/s	2.0865 KOps/s	$\color{#35bf28}+0.54\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.1607ms	2.7126ms	368.6452 Ops/s	371.1604 Ops/s	$\color{#d91a1a}-0.68\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.8521ms	0.5017ms	1.9932 KOps/s	1.9990 KOps/s	$\color{#d91a1a}-0.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.7040ms	0.4738ms	2.1105 KOps/s	2.1048 KOps/s	$\color{#35bf28}+0.27\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.9808ms	2.7823ms	359.4122 Ops/s	351.3513 Ops/s	$\color{#35bf28}+2.29\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8533ms	0.6187ms	1.6162 KOps/s	1.6115 KOps/s	$\color{#35bf28}+0.30\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8396ms	0.5894ms	1.6967 KOps/s	1.6878 KOps/s	$\color{#35bf28}+0.52\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.9013ms	2.6528ms	376.9551 Ops/s	370.1662 Ops/s	$\color{#35bf28}+1.83\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.6182ms	0.5037ms	1.9853 KOps/s	1.9676 KOps/s	$\color{#35bf28}+0.90\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.7156ms	0.4845ms	2.0639 KOps/s	2.0886 KOps/s	$\color{#d91a1a}-1.19\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.0206ms	2.6802ms	373.1011 Ops/s	374.9654 Ops/s	$\color{#d91a1a}-0.50\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7784ms	0.4986ms	2.0055 KOps/s	1.9882 KOps/s	$\color{#35bf28}+0.87\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.9055ms	0.4816ms	2.0766 KOps/s	2.0763 KOps/s	$\color{#35bf28}+0.01\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.1567ms	2.8033ms	356.7245 Ops/s	360.0460 Ops/s	$\color{#d91a1a}-0.92\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.9332ms	0.6175ms	1.6194 KOps/s	1.5881 KOps/s	$\color{#35bf28}+1.97\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8447ms	0.5932ms	1.6858 KOps/s	1.6581 KOps/s	$\color{#35bf28}+1.67\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	98.5099ms	7.4936ms	133.4473 Ops/s	131.3338 Ops/s	$\color{#35bf28}+1.61\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	92.7376ms	14.7112ms	67.9754 Ops/s	68.6005 Ops/s	$\color{#d91a1a}-0.91\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	4.4195ms	2.5406ms	393.6081 Ops/s	391.9720 Ops/s	$\color{#35bf28}+0.42\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	84.4221ms	7.3633ms	135.8084 Ops/s	136.2178 Ops/s	$\color{#d91a1a}-0.30\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	15.4188ms	13.0261ms	76.7687 Ops/s	77.0369 Ops/s	$\color{#d91a1a}-0.35\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	4.5627ms	2.5277ms	395.6109 Ops/s	395.8976 Ops/s	$\color{#d91a1a}-0.07\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	85.7372ms	9.2419ms	108.2025 Ops/s	107.5855 Ops/s	$\color{#35bf28}+0.57\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	15.0205ms	13.1709ms	75.9252 Ops/s	76.0530 Ops/s	$\color{#d91a1a}-0.17\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	5.0004ms	2.7829ms	359.3340 Ops/s	357.8457 Ops/s	$\color{#35bf28}+0.42\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}22$. Worsened: $\large\color{#d91a1a}9$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1134s	0.1116s	8.9607 Ops/s	8.4363 Ops/s	$\textbf{\color{#35bf28}+6.22\%}$
test_sync	95.7010ms	95.4840ms	10.4730 Ops/s	10.4460 Ops/s	$\color{#35bf28}+0.26\%$
test_async	0.1801s	91.2424ms	10.9598 Ops/s	10.9364 Ops/s	$\color{#35bf28}+0.21\%$
test_single_pixels	0.1304s	0.1278s	7.8246 Ops/s	7.9137 Ops/s	$\color{#d91a1a}-1.13\%$
test_sync_pixels	82.8612ms	79.7931ms	12.5324 Ops/s	12.4656 Ops/s	$\color{#35bf28}+0.54\%$
test_async_pixels	0.1496s	65.3572ms	15.3005 Ops/s	13.6706 Ops/s	$\textbf{\color{#35bf28}+11.92\%}$
test_simple	0.7973s	0.7944s	1.2587 Ops/s	1.2132 Ops/s	$\color{#35bf28}+3.75\%$
test_transformed	1.0816s	1.0252s	0.9754 Ops/s	0.9551 Ops/s	$\color{#35bf28}+2.13\%$
test_serial	2.3882s	2.3331s	0.4286 Ops/s	0.4123 Ops/s	$\color{#35bf28}+3.95\%$
test_parallel	2.0877s	2.0132s	0.4967 Ops/s	0.4779 Ops/s	$\color{#35bf28}+3.93\%$
test_step_mdp_speed[True-True-True-True-True]	97.4310μs	32.4797μs	30.7885 KOps/s	29.3217 KOps/s	$\textbf{\color{#35bf28}+5.00\%}$
test_step_mdp_speed[True-True-True-True-False]	54.9310μs	19.1515μs	52.2151 KOps/s	50.3943 KOps/s	$\color{#35bf28}+3.61\%$
test_step_mdp_speed[True-True-True-False-True]	43.6710μs	18.0970μs	55.2578 KOps/s	52.4320 KOps/s	$\textbf{\color{#35bf28}+5.39\%}$
test_step_mdp_speed[True-True-True-False-False]	84.2820μs	10.8376μs	92.2713 KOps/s	86.8548 KOps/s	$\textbf{\color{#35bf28}+6.24\%}$
test_step_mdp_speed[True-True-False-True-True]	55.0510μs	34.4309μs	29.0436 KOps/s	28.0001 KOps/s	$\color{#35bf28}+3.73\%$
test_step_mdp_speed[True-True-False-True-False]	45.1610μs	21.1460μs	47.2903 KOps/s	45.6016 KOps/s	$\color{#35bf28}+3.70\%$
test_step_mdp_speed[True-True-False-False-True]	59.0910μs	19.9782μs	50.0544 KOps/s	47.5762 KOps/s	$\textbf{\color{#35bf28}+5.21\%}$
test_step_mdp_speed[True-True-False-False-False]	41.5500μs	12.7395μs	78.4961 KOps/s	73.9039 KOps/s	$\textbf{\color{#35bf28}+6.21\%}$
test_step_mdp_speed[True-False-True-True-True]	63.1720μs	36.0400μs	27.7469 KOps/s	26.5882 KOps/s	$\color{#35bf28}+4.36\%$
test_step_mdp_speed[True-False-True-True-False]	95.7720μs	22.6306μs	44.1880 KOps/s	41.7306 KOps/s	$\textbf{\color{#35bf28}+5.89\%}$
test_step_mdp_speed[True-False-True-False-True]	46.4710μs	19.7242μs	50.6991 KOps/s	47.7539 KOps/s	$\textbf{\color{#35bf28}+6.17\%}$
test_step_mdp_speed[True-False-True-False-False]	29.3300μs	12.7590μs	78.3760 KOps/s	74.1641 KOps/s	$\textbf{\color{#35bf28}+5.68\%}$
test_step_mdp_speed[True-False-False-True-True]	64.3810μs	37.2625μs	26.8366 KOps/s	25.2602 KOps/s	$\textbf{\color{#35bf28}+6.24\%}$
test_step_mdp_speed[True-False-False-True-False]	58.5710μs	24.4137μs	40.9606 KOps/s	39.5215 KOps/s	$\color{#35bf28}+3.64\%$
test_step_mdp_speed[True-False-False-False-True]	83.9220μs	21.5655μs	46.3703 KOps/s	43.8290 KOps/s	$\textbf{\color{#35bf28}+5.80\%}$
test_step_mdp_speed[True-False-False-False-False]	36.3210μs	14.3979μs	69.4546 KOps/s	66.3391 KOps/s	$\color{#35bf28}+4.70\%$
test_step_mdp_speed[False-True-True-True-True]	63.7510μs	35.8806μs	27.8702 KOps/s	26.4715 KOps/s	$\textbf{\color{#35bf28}+5.28\%}$
test_step_mdp_speed[False-True-True-True-False]	47.4310μs	22.8802μs	43.7058 KOps/s	41.2279 KOps/s	$\textbf{\color{#35bf28}+6.01\%}$
test_step_mdp_speed[False-True-True-False-True]	49.4110μs	24.0453μs	41.5881 KOps/s	40.0856 KOps/s	$\color{#35bf28}+3.75\%$
test_step_mdp_speed[False-True-True-False-False]	32.0610μs	14.2832μs	70.0124 KOps/s	66.3650 KOps/s	$\textbf{\color{#35bf28}+5.50\%}$
test_step_mdp_speed[False-True-False-True-True]	70.0420μs	37.9791μs	26.3303 KOps/s	25.1529 KOps/s	$\color{#35bf28}+4.68\%$
test_step_mdp_speed[False-True-False-True-False]	47.9400μs	24.6973μs	40.4903 KOps/s	38.2921 KOps/s	$\textbf{\color{#35bf28}+5.74\%}$
test_step_mdp_speed[False-True-False-False-True]	51.1710μs	25.3671μs	39.4211 KOps/s	37.3846 KOps/s	$\textbf{\color{#35bf28}+5.45\%}$
test_step_mdp_speed[False-True-False-False-False]	38.1710μs	16.2151μs	61.6711 KOps/s	58.9152 KOps/s	$\color{#35bf28}+4.68\%$
test_step_mdp_speed[False-False-True-True-True]	69.5510μs	39.2630μs	25.4693 KOps/s	24.7829 KOps/s	$\color{#35bf28}+2.77\%$
test_step_mdp_speed[False-False-True-True-False]	55.5210μs	26.5270μs	37.6975 KOps/s	35.9725 KOps/s	$\color{#35bf28}+4.80\%$
test_step_mdp_speed[False-False-True-False-True]	62.8210μs	25.4387μs	39.3102 KOps/s	37.0174 KOps/s	$\textbf{\color{#35bf28}+6.19\%}$
test_step_mdp_speed[False-False-True-False-False]	40.0310μs	16.3356μs	61.2161 KOps/s	59.3980 KOps/s	$\color{#35bf28}+3.06\%$
test_step_mdp_speed[False-False-False-True-True]	71.9220μs	40.6515μs	24.5993 KOps/s	23.7563 KOps/s	$\color{#35bf28}+3.55\%$
test_step_mdp_speed[False-False-False-True-False]	63.0510μs	28.2249μs	35.4296 KOps/s	33.9065 KOps/s	$\color{#35bf28}+4.49\%$
test_step_mdp_speed[False-False-False-False-True]	59.8710μs	26.9592μs	37.0931 KOps/s	35.4639 KOps/s	$\color{#35bf28}+4.59\%$
test_step_mdp_speed[False-False-False-False-False]	39.6610μs	17.9626μs	55.6712 KOps/s	53.1874 KOps/s	$\color{#35bf28}+4.67\%$
test_values[generalized_advantage_estimate-True-True]	26.2852ms	25.8033ms	38.7547 Ops/s	41.2773 Ops/s	$\textbf{\color{#d91a1a}-6.11\%}$
test_values[vec_generalized_advantage_estimate-True-True]	81.7968ms	3.2191ms	310.6458 Ops/s	311.8005 Ops/s	$\color{#d91a1a}-0.37\%$
test_values[td0_return_estimate-False-False]	0.1004ms	65.8211μs	15.1927 KOps/s	17.0172 KOps/s	$\textbf{\color{#d91a1a}-10.72\%}$
test_values[td1_return_estimate-False-False]	57.5181ms	56.7741ms	17.6137 Ops/s	19.2712 Ops/s	$\textbf{\color{#d91a1a}-8.60\%}$
test_values[vec_td1_return_estimate-False-False]	2.1352ms	1.7439ms	573.4376 Ops/s	572.8088 Ops/s	$\color{#35bf28}+0.11\%$
test_values[td_lambda_return_estimate-True-False]	91.8767ms	89.6879ms	11.1498 Ops/s	12.1630 Ops/s	$\textbf{\color{#d91a1a}-8.33\%}$
test_values[vec_td_lambda_return_estimate-True-False]	3.8195ms	1.7772ms	562.6898 Ops/s	561.8121 Ops/s	$\color{#35bf28}+0.16\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	25.2114ms	23.1964ms	43.1101 Ops/s	43.7604 Ops/s	$\color{#d91a1a}-1.49\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.9444ms	0.7371ms	1.3567 KOps/s	1.4601 KOps/s	$\textbf{\color{#d91a1a}-7.08\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7731ms	0.6798ms	1.4710 KOps/s	1.5698 KOps/s	$\textbf{\color{#d91a1a}-6.29\%}$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.4705ms	1.4363ms	696.2300 Ops/s	694.6934 Ops/s	$\color{#35bf28}+0.22\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9347ms	0.6585ms	1.5187 KOps/s	1.5325 KOps/s	$\color{#d91a1a}-0.90\%$
test_dqn_speed	8.2195ms	1.4633ms	683.3746 Ops/s	666.9959 Ops/s	$\color{#35bf28}+2.46\%$
test_ddpg_speed	3.1854ms	2.7851ms	359.0514 Ops/s	345.4048 Ops/s	$\color{#35bf28}+3.95\%$
test_sac_speed	8.4876ms	8.0742ms	123.8508 Ops/s	120.1363 Ops/s	$\color{#35bf28}+3.09\%$
test_redq_speed	11.1183ms	10.3466ms	96.6498 Ops/s	95.1702 Ops/s	$\color{#35bf28}+1.55\%$
test_redq_deprec_speed	11.8542ms	11.4655ms	87.2181 Ops/s	90.1003 Ops/s	$\color{#d91a1a}-3.20\%$
test_td3_speed	16.1147ms	8.2409ms	121.3463 Ops/s	119.5127 Ops/s	$\color{#35bf28}+1.53\%$
test_cql_speed	25.9275ms	25.0058ms	39.9908 Ops/s	39.1375 Ops/s	$\color{#35bf28}+2.18\%$
test_a2c_speed	5.6546ms	5.4650ms	182.9830 Ops/s	182.5292 Ops/s	$\color{#35bf28}+0.25\%$
test_ppo_speed	6.3927ms	5.8213ms	171.7818 Ops/s	172.2259 Ops/s	$\color{#d91a1a}-0.26\%$
test_reinforce_speed	4.8321ms	4.5442ms	220.0627 Ops/s	223.8495 Ops/s	$\color{#d91a1a}-1.69\%$
test_iql_speed	19.5687ms	19.1692ms	52.1670 Ops/s	51.6712 Ops/s	$\color{#35bf28}+0.96\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.8196ms	3.6629ms	273.0069 Ops/s	267.9246 Ops/s	$\color{#35bf28}+1.90\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	0.7080ms	0.5628ms	1.7769 KOps/s	1.7734 KOps/s	$\color{#35bf28}+0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6724ms	0.5340ms	1.8727 KOps/s	1.8615 KOps/s	$\color{#35bf28}+0.60\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	4.0493ms	3.7004ms	270.2409 Ops/s	267.3406 Ops/s	$\color{#35bf28}+1.08\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	0.7070ms	0.5581ms	1.7919 KOps/s	1.7955 KOps/s	$\color{#d91a1a}-0.20\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6929ms	0.5312ms	1.8825 KOps/s	1.8921 KOps/s	$\color{#d91a1a}-0.51\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	4.0158ms	3.7958ms	263.4489 Ops/s	258.3220 Ops/s	$\color{#35bf28}+1.98\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8265ms	0.6921ms	1.4449 KOps/s	1.3019 KOps/s	$\textbf{\color{#35bf28}+10.98\%}$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.7831ms	0.6624ms	1.5097 KOps/s	1.5203 KOps/s	$\color{#d91a1a}-0.69\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.7928ms	3.6768ms	271.9746 Ops/s	267.0616 Ops/s	$\color{#35bf28}+1.84\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	93.7662ms	0.6397ms	1.5633 KOps/s	1.7817 KOps/s	$\textbf{\color{#d91a1a}-12.25\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	0.6797ms	0.5390ms	1.8552 KOps/s	1.8659 KOps/s	$\color{#d91a1a}-0.57\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	3.9197ms	3.7174ms	269.0050 Ops/s	266.0392 Ops/s	$\color{#35bf28}+1.11\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	98.5428ms	0.6654ms	1.5029 KOps/s	1.8031 KOps/s	$\textbf{\color{#d91a1a}-16.65\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	0.6554ms	0.5306ms	1.8846 KOps/s	1.6041 KOps/s	$\textbf{\color{#35bf28}+17.49\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	3.9678ms	3.8144ms	262.1674 Ops/s	258.6649 Ops/s	$\color{#35bf28}+1.35\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	0.8302ms	0.6901ms	1.4491 KOps/s	1.4504 KOps/s	$\color{#d91a1a}-0.09\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	0.8215ms	0.6625ms	1.5095 KOps/s	1.5049 KOps/s	$\color{#35bf28}+0.30\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1025s	9.3339ms	107.1366 Ops/s	89.7043 Ops/s	$\textbf{\color{#35bf28}+19.43\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	18.6536ms	16.1478ms	61.9280 Ops/s	61.5534 Ops/s	$\color{#35bf28}+0.61\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	6.2141ms	3.0534ms	327.4995 Ops/s	316.4995 Ops/s	$\color{#35bf28}+3.48\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1004s	11.1434ms	89.7391 Ops/s	89.7731 Ops/s	$\color{#d91a1a}-0.04\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	18.7508ms	16.1317ms	61.9899 Ops/s	61.5411 Ops/s	$\color{#35bf28}+0.73\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	6.5846ms	3.0580ms	327.0126 Ops/s	311.9197 Ops/s	$\color{#35bf28}+4.84\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1046s	11.5199ms	86.8060 Ops/s	104.7808 Ops/s	$\textbf{\color{#d91a1a}-17.15\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	18.2040ms	15.9810ms	62.5744 Ops/s	54.2421 Ops/s	$\textbf{\color{#35bf28}+15.36\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	7.1218ms	3.3006ms	302.9795 Ops/s	293.9909 Ops/s	$\color{#35bf28}+3.06\%$