[Feature] Add chance node support to OpenSpiel wrapper#3883
Open
itwasabhi wants to merge 1 commit into
Open
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3883
Note: Links to docs will display an error until the docs builds have been completed. ❌ 2 New Failures, 1 Unrelated FailureAs of commit 17e0d24 with merge base ae403e6 ( NEW FAILURES - The following jobs have failed:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
github-actions
Bot
added
Environments
Implement automatic chance node resolution in OpenSpielWrapper and OpenSpielEnv, allowing environments with stochastic outcomes (like backgammon, dice-based games) to be used without modification. Add guard in OpenSpielWrapper for batch size, as only single-batch environments are supported.
itwasabhi
force-pushed
the
openspiel_chance
branch
from
f3436e8 to
17e0d24
Compare
Contributor
Benchmark Results: PR
|
| Benchmark | main ops | PR ops | Change |
|---|---|---|---|
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] |
59.23 | 499.24 | +742.85% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] |
193.53 | 54.27 | -71.96% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
2,964 | 3,465 | +16.91% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-same] |
20.65 | 23.80 | +15.27% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-same] |
29.37 | 24.94 | -15.09% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
2,794 | 3,174 | +13.59% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[50-img_shape0-small] |
4,001 | 4,509 | +12.71% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
2,770 | 3,120 | +12.64% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] |
881.05 | 989.09 | +12.26% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,187 | 1,921 | -12.19% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[50-img_shape0-small] |
3,193 | 3,578 | +12.04% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] |
523.43 | 466.22 | -10.93% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] |
840.45 | 755.48 | -10.11% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[200-img_shape3-large_batch] |
306.32 | 331.14 | +8.10% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[200-img_shape3-large_batch] |
287.77 | 310.71 | +7.97% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-None] |
648.20 | 698.72 | +7.79% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
3,018 | 2,789 | -7.60% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] |
772.34 | 716.21 | -7.27% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[200-img_shape3-large_batch] |
130.65 | 140.14 | +7.27% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
2,691 | 2,501 | -7.06% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-backward] |
110.78 | 118.57 | +7.03% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-backward] |
56.45 | 60.25 | +6.74% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] |
599.09 | 638.06 | +6.50% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] |
35,499 | 37,781 | +6.43% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] |
118.10 | 125.56 | +6.32% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-True] |
19,542 | 20,770 | +6.28% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-True] |
16,849 | 17,868 | +6.05% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-True] |
22,144 | 23,449 | +5.90% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-True] |
20,695 | 21,905 | +5.84% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-True] |
28,644 | 30,307 | +5.81% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,114 | 1,992 | -5.80% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-False] |
29,597 | 31,274 | +5.66% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[reduce-overhead-None] |
79.81 | 84.31 | +5.63% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-None] |
1,808 | 1,707 | -5.58% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] |
646.49 | 681.74 | +5.45% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] |
36,040 | 37,997 | +5.43% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-backward] |
131.62 | 138.69 | +5.37% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-True] |
18,652 | 19,640 | +5.30% |
benchmarks/test_envs_benchmark.py::test_parallel |
0.9782 | 0.9267 | -5.27% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] |
1,025 | 1,079 | +5.24% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-False] |
32,405 | 34,090 | +5.20% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[pickle] |
11,680 | 12,269 | +5.04% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-True-32-512] |
27.49 | 28.86 | +4.99% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-True] |
28,499 | 29,882 | +4.85% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] |
25,592 | 26,804 | +4.74% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-False] |
41,747 | 43,721 | +4.73% |
benchmarks/test_objectives_benchmarks.py::test_values[generalized_advantage_estimate-True-True] |
103.77 | 98.90 | -4.69% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[True-None] |
226.83 | 216.20 | -4.68% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] |
1,855 | 1,768 | -4.68% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] |
159.33 | 166.68 | +4.61% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-False] |
29,928 | 31,298 | +4.58% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-False] |
53,892 | 56,343 | +4.55% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] |
1,050 | 1,002 | -4.53% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape2-large_img] |
549.72 | 524.96 | -4.50% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-False] |
27,555 | 28,793 | +4.49% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
2,681 | 2,561 | -4.46% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb[200-img_shape1-large_batch] |
14.90 | 15.54 | +4.30% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-None] |
253.60 | 264.48 | +4.29% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_with_rb[200-img_shape1-large_batch] |
13.05 | 13.61 | +4.24% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-False] |
40,231 | 41,894 | +4.13% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape1-atari] |
4,937 | 5,135 | +4.00% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] |
906.28 | 870.00 | -4.00% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] |
161.77 | 168.24 | +4.00% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-True] |
17,768 | 18,479 | +4.00% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape1-atari] |
263.87 | 274.41 | +4.00% |
benchmarks/test_envs_benchmark.py::test_transformed |
0.8856 | 0.9202 | +3.90% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-True] |
18,941 | 19,676 | +3.88% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-False] |
60,574 | 62,905 | +3.85% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] |
156.82 | 162.79 | +3.81% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-False] |
51,213 | 53,146 | +3.77% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-None] |
269.21 | 279.30 | +3.75% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_with_rb[100-img_shape0-atari] |
25.72 | 26.67 | +3.70% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-False-0-gru] |
1.2492 | 1.2950 | +3.66% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] |
31,106 | 32,241 | +3.65% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] |
186.06 | 192.73 | +3.58% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] |
157.24 | 162.86 | +3.57% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb[100-img_shape0-atari] |
29.47 | 30.51 | +3.52% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-1-512] |
2,372 | 2,290 | -3.46% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] |
154.31 | 159.61 | +3.43% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] |
160.49 | 165.93 | +3.39% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-backward] |
86.65 | 89.58 | +3.39% |
benchmarks/test_objectives_benchmarks.py::test_values[vec_td1_return_estimate-False-False] |
53.03 | 54.81 | +3.37% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-False] |
36,194 | 37,412 | +3.37% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-backward] |
402.82 | 416.36 | +3.36% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[reduce-overhead-None] |
211.71 | 218.81 | +3.35% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-constant] |
4,298 | 4,440 | +3.30% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[50-img_shape0-small] |
833.68 | 860.61 | +3.23% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] |
47,488 | 48,986 | +3.15% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-True] |
26,971 | 27,822 | +3.15% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-gru] |
2.8815 | 2.9719 | +3.14% |
benchmarks/test_objectives_benchmarks.py::test_values[vec_td_lambda_return_estimate-True-False] |
53.23 | 54.89 | +3.12% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-True] |
40,497 | 41,749 | +3.09% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-True] |
19,932 | 20,536 | +3.03% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-False] |
73,833 | 76,051 | +3.00% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-False] |
47,146 | 48,507 | +2.89% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sampler_sample_scale[1000000-cpu] |
96.30 | 99.04 | +2.85% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-lstm] |
2.0030 | 1.9459 | -2.85% |
benchmarks/test_objectives_benchmarks.py::test_values[vec_generalized_advantage_estimate-True-True] |
53.31 | 54.80 | +2.79% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[False-backward] |
54.34 | 55.84 | +2.76% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-True] |
33,679 | 34,598 | +2.73% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-backward] |
61.46 | 63.13 | +2.73% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-True-1-512] |
669.93 | 651.85 | -2.70% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] |
87.93 | 90.23 | +2.62% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-True-0-gru] |
1.3568 | 1.3915 | +2.56% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-False] |
32,830 | 33,651 | +2.50% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-None] |
349.99 | 341.88 | -2.32% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[200-img_shape3-large_batch] |
744.75 | 761.87 | +2.30% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
2,773 | 2,710 | -2.28% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[numpy] |
377,613 | 369,123 | -2.25% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] |
0.2127 | 0.2175 | +2.25% |
benchmarks/test_envs_benchmark.py::test_simple |
1.7741 | 1.8125 | +2.16% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] |
540.49 | 551.82 | +2.10% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-False-0-lstm] |
0.8113 | 0.8281 | +2.07% |
benchmarks/test_collectors_benchmark.py::test_sync |
16.59 | 16.93 | +2.07% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-backward] |
112.34 | 114.51 | +1.94% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] |
54.31 | 55.35 | +1.91% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[50-img_shape0-small] |
7,094 | 7,223 | +1.82% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-False] |
60,985 | 62,088 | +1.81% |
benchmarks/test_objectives_benchmarks.py::test_values[td_lambda_return_estimate-True-False] |
26.61 | 26.16 | -1.68% |
benchmarks/test_envs_benchmark.py::test_serial |
0.5718 | 0.5811 | +1.62% |
| ... | ... | ... | Showing 120 of 192 comparisons, sorted by absolute change. |
GPU
Compared 202 benchmarks. Regressions over 5%: 9. Improvements over 5%: 11.
| Benchmark | main ops | PR ops | Change |
|---|---|---|---|
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] |
44.55 | 191.15 | +329.10% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] |
299.71 | 689.84 | +130.17% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] |
191.80 | 49.22 | -74.34% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
2,793 | 3,628 | +29.89% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[reduce-overhead-None] |
98.34 | 120.44 | +22.48% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
3,205 | 2,491 | -22.26% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] |
1,000 | 807.30 | -19.30% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
3,134 | 2,534 | -19.14% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
1,940 | 2,297 | +18.41% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
2,858 | 3,333 | +16.61% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
3,084 | 2,681 | -13.06% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] |
719.14 | 803.75 | +11.77% |
benchmarks/test_collectors_benchmark.py::test_single_with_rb_pixels |
5.2729 | 4.6768 | -11.30% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] |
459.08 | 506.29 | +10.29% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] |
809.66 | 728.95 | -9.97% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
3,276 | 2,982 | -8.98% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
2,072 | 1,892 | -8.70% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] |
490.85 | 520.85 | +6.11% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape1-atari] |
3,970 | 4,185 | +5.42% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[200-img_shape3-large_batch] |
134.21 | 140.96 | +5.03% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
1,873 | 1,965 | +4.92% |
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-None] |
735.51 | 702.14 | -4.54% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape2-large_img] |
569.52 | 543.82 | -4.51% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,103 | 2,011 | -4.37% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-backward] |
343.98 | 329.06 | -4.34% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] |
362.87 | 348.02 | -4.09% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] |
974.56 | 1,013 | +3.93% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] |
709.66 | 683.71 | -3.66% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-None] |
1,853 | 1,921 | +3.65% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-True] |
20,997 | 20,240 | -3.61% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-None] |
355.31 | 368.05 | +3.58% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-single-True] |
1.3125 | 1.3594 | +3.57% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
2,670 | 2,760 | +3.38% |
benchmarks/test_objectives_benchmarks.py::test_values[td_lambda_return_estimate-True-False] |
12.04 | 11.65 | -3.31% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape1-atari] |
275.78 | 266.70 | -3.29% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-False] |
58,030 | 56,148 | -3.24% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-True] |
30,426 | 29,499 | -3.05% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] |
1,896 | 1,839 | -3.04% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] |
37,946 | 36,808 | -3.00% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] |
501.78 | 486.87 | -2.97% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape2-large_img] |
380.94 | 392.23 | +2.96% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] |
32,663 | 31,748 | -2.80% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-backward] |
76.03 | 78.08 | +2.71% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-False] |
77,317 | 75,242 | -2.68% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-False] |
55,198 | 53,719 | -2.68% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[200-img_shape3-large_batch] |
300.57 | 308.58 | +2.66% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-True] |
18,580 | 18,087 | -2.65% |
benchmarks/test_envs_benchmark.py::test_simple |
1.2280 | 1.1970 | -2.52% |
benchmarks/test_objectives_benchmarks.py::test_values[td1_return_estimate-False-False] |
19.68 | 19.19 | -2.49% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-False] |
31,557 | 30,773 | -2.49% |
benchmarks/test_objectives_benchmarks.py::test_values[vec_generalized_advantage_estimate-True-True] |
300.16 | 307.45 | +2.43% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-backward] |
273.61 | 266.97 | -2.43% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-None] |
757.22 | 775.23 | +2.38% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-backward] |
243.87 | 238.08 | -2.37% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[200-img_shape3-large_batch] |
321.44 | 328.78 | +2.28% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-cuda_storage_cpu_sampler] |
87.50 | 89.44 | +2.22% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-True] |
33,951 | 33,275 | -1.99% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-True] |
19,408 | 19,045 | -1.87% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[reduce-overhead-None] |
798.79 | 813.52 | +1.84% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] |
27,109 | 26,615 | -1.82% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-False] |
29,275 | 28,749 | -1.80% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] |
96.34 | 98.07 | +1.79% |
benchmarks/test_envs_benchmark.py::test_serial |
0.4175 | 0.4249 | +1.77% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-True] |
21,474 | 21,851 | +1.76% |
benchmarks/test_envs_benchmark.py::test_parallel |
0.5323 | 0.5232 | -1.71% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] |
165.78 | 168.60 | +1.70% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-same] |
6.4412 | 6.5501 | +1.69% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] |
651.41 | 640.45 | -1.68% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] |
49,479 | 48,648 | -1.68% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] |
23.23 | 23.60 | +1.60% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-True] |
30,455 | 29,971 | -1.59% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-False] |
44,088 | 43,399 | -1.56% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-False] |
64,348 | 63,355 | -1.54% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] |
167.45 | 169.92 | +1.47% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] |
162.61 | 164.98 | +1.46% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sampler_sample_scale[1000000-cuda] |
2,209 | 2,241 | +1.46% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-True] |
42,645 | 42,025 | -1.45% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] |
48.86 | 48.19 | -1.38% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-same] |
5.3650 | 5.4387 | +1.37% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape2-large_img] |
398.98 | 404.41 | +1.36% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-constant] |
4,640 | 4,703 | +1.35% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-gru] |
47.92 | 48.56 | +1.33% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[untyped_storage] |
7.8072 | 7.7059 | -1.30% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] |
37,600 | 37,113 | -1.30% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-backward] |
254.90 | 258.18 | +1.28% |
benchmarks/test_envs_benchmark.py::test_transformed |
0.7086 | 0.6998 | -1.25% |
benchmarks/test_objectives_benchmarks.py::test_values[generalized_advantage_estimate-True-True] |
47.24 | 47.82 | +1.23% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[safetensors] |
23,122 | 22,846 | -1.19% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-lstm] |
75.69 | 74.78 | -1.19% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] |
184.95 | 187.08 | +1.15% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[50-img_shape0-small] |
4,347 | 4,297 | -1.15% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] |
860.29 | 850.70 | -1.11% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-backward] |
228.83 | 231.34 | +1.10% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-serial-no-buffers-False] |
0.6887 | 0.6812 | -1.08% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb_cuda[200-img_shape1-large_batch] |
8.7966 | 8.7017 | -1.08% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-None] |
110.52 | 111.70 | +1.07% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-None] |
717.82 | 710.18 | -1.07% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-backward] |
125.94 | 127.28 | +1.06% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-None] |
338.05 | 341.56 | +1.04% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-serial-no-buffers-True] |
0.5964 | 0.5903 | -1.03% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] |
0.6025 | 0.6086 | +1.02% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[reduce-overhead-None] |
812.60 | 820.79 | +1.01% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-backward] |
344.82 | 348.29 | +1.01% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[False-backward] |
68.84 | 68.15 | -1.00% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[reduce-overhead-None] |
105.90 | 104.84 | -1.00% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-True] |
20,453 | 20,256 | -0.96% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[numpy] |
354,928 | 351,547 | -0.95% |
benchmarks/test_objectives_benchmarks.py::test_values[td0_return_estimate-False-False] |
11,464 | 11,571 | +0.94% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-serial-buffers-False] |
0.5981 | 0.5925 | -0.94% |
benchmarks/test_collectors_benchmark.py::test_sync_pixels |
9.9969 | 10.09 | +0.92% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-32-512] |
1,290 | 1,302 | +0.90% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[200-img_shape3-large_batch] |
735.07 | 741.53 | +0.88% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] |
0.2223 | 0.2204 | -0.86% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[False-None] |
98.39 | 97.57 | -0.84% |
benchmarks/test_objectives_benchmarks.py::test_td3_speed[False-backward] |
81.39 | 82.07 | +0.84% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-None] |
388.36 | 391.55 | +0.82% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-None] |
412.71 | 416.10 | +0.82% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape2-large_img] |
167.26 | 168.62 | +0.81% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[pickle] |
12,334 | 12,237 | -0.79% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[False-None] |
53.21 | 53.62 | +0.77% |
| ... | ... | ... | Showing 120 of 202 comparisons, sorted by absolute change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses #3882 and #3881
Implement automatic chance node resolution in OpenSpielWrapper and OpenSpielEnv, allowing environments with stochastic outcomes (like backgammon, dice-based games) to be used without modification. This matches the solution provided by openspiel: https://github.com/google-deepmind/open_spiel/blob/6c8edc829962967730e5ff353340df75847fa184/open_spiel/python/rl_environment.py#L137
Add guard in OpenSpielWrapper for batch size, as only single-batch environments are supported.
Description
Motivation and Context
Types of changes
What types of changes does your code introduce? Remove all that do not apply:
Checklist
Go over all the following points, and put an
xin all the boxes that apply.If you are unsure about any of these, don't hesitate to ask. We are here to help!