| [2025-02-21 23:33:42,998][09089] Saving configuration to /root/autodl-tmp/train_dir/default_experiment/config.json... | |
| [2025-02-21 23:33:43,000][09089] Rollout worker 0 uses device cpu | |
| [2025-02-21 23:33:43,001][09089] Rollout worker 1 uses device cpu | |
| [2025-02-21 23:33:43,001][09089] Rollout worker 2 uses device cpu | |
| [2025-02-21 23:33:43,001][09089] Rollout worker 3 uses device cpu | |
| [2025-02-21 23:33:43,001][09089] Rollout worker 4 uses device cpu | |
| [2025-02-21 23:33:43,002][09089] Rollout worker 5 uses device cpu | |
| [2025-02-21 23:33:43,002][09089] Rollout worker 6 uses device cpu | |
| [2025-02-21 23:33:43,002][09089] Rollout worker 7 uses device cpu | |
| [2025-02-21 23:33:43,525][09089] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-02-21 23:33:43,526][09089] InferenceWorker_p0-w0: min num requests: 2 | |
| [2025-02-21 23:33:43,635][09089] Starting all processes... | |
| [2025-02-21 23:33:43,636][09089] Starting process learner_proc0 | |
| [2025-02-21 23:33:54,402][09144] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-02-21 23:33:54,403][09144] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0 | |
| [2025-02-21 23:33:54,465][09144] Num visible devices: 1 | |
| [2025-02-21 23:33:54,490][09089] Starting all processes... | |
| [2025-02-21 23:33:54,499][09144] Starting seed is not provided | |
| [2025-02-21 23:33:54,500][09144] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-02-21 23:33:54,500][09144] Initializing actor-critic model on device cuda:0 | |
| [2025-02-21 23:33:54,500][09144] RunningMeanStd input shape: (3, 72, 128) | |
| [2025-02-21 23:33:54,502][09144] RunningMeanStd input shape: (1,) | |
| [2025-02-21 23:33:54,517][09144] ConvEncoder: input_channels=3 | |
| [2025-02-21 23:33:54,554][09089] Starting process inference_proc0-0 | |
| [2025-02-21 23:33:54,557][09089] Starting process rollout_proc0 | |
| [2025-02-21 23:33:54,557][09089] Starting process rollout_proc1 | |
| [2025-02-21 23:33:54,558][09089] Starting process rollout_proc2 | |
| [2025-02-21 23:33:54,559][09089] Starting process rollout_proc3 | |
| [2025-02-21 23:33:54,561][09089] Starting process rollout_proc4 | |
| [2025-02-21 23:33:54,562][09089] Starting process rollout_proc5 | |
| [2025-02-21 23:33:54,562][09089] Starting process rollout_proc6 | |
| [2025-02-21 23:33:54,563][09089] Starting process rollout_proc7 | |
| [2025-02-21 23:33:54,761][09144] Conv encoder output size: 512 | |
| [2025-02-21 23:33:54,761][09144] Policy head output size: 512 | |
| [2025-02-21 23:33:54,791][09144] Created Actor Critic model with architecture: | |
| [2025-02-21 23:33:54,792][09144] ActorCriticSharedWeights( | |
| (obs_normalizer): ObservationNormalizer( | |
| (running_mean_std): RunningMeanStdDictInPlace( | |
| (running_mean_std): ModuleDict( | |
| (obs): RunningMeanStdInPlace() | |
| ) | |
| ) | |
| ) | |
| (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace) | |
| (encoder): VizdoomEncoder( | |
| (basic_encoder): ConvEncoder( | |
| (enc): RecursiveScriptModule( | |
| original_name=ConvEncoderImpl | |
| (conv_head): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Conv2d) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| (2): RecursiveScriptModule(original_name=Conv2d) | |
| (3): RecursiveScriptModule(original_name=ELU) | |
| (4): RecursiveScriptModule(original_name=Conv2d) | |
| (5): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| (mlp_layers): RecursiveScriptModule( | |
| original_name=Sequential | |
| (0): RecursiveScriptModule(original_name=Linear) | |
| (1): RecursiveScriptModule(original_name=ELU) | |
| ) | |
| ) | |
| ) | |
| ) | |
| (core): ModelCoreRNN( | |
| (core): GRU(512, 512) | |
| ) | |
| (decoder): MlpDecoder( | |
| (mlp): Identity() | |
| ) | |
| (critic_linear): Linear(in_features=512, out_features=1, bias=True) | |
| (action_parameterization): ActionParameterizationDefault( | |
| (distribution_linear): Linear(in_features=512, out_features=5, bias=True) | |
| ) | |
| ) | |
| [2025-02-21 23:33:57,417][09144] Using optimizer <class 'torch.optim.adam.Adam'> | |
| [2025-02-21 23:33:59,500][09223] Worker 6 uses CPU cores [36, 37, 38, 39, 40, 41] | |
| [2025-02-21 23:33:59,526][09225] Worker 5 uses CPU cores [30, 31, 32, 33, 34, 35] | |
| [2025-02-21 23:34:01,219][09221] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-02-21 23:34:01,220][09221] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0 | |
| [2025-02-21 23:34:01,281][09221] Num visible devices: 1 | |
| [2025-02-21 23:34:02,748][09227] Worker 7 uses CPU cores [42, 43, 44, 45, 46, 47] | |
| [2025-02-21 23:34:02,906][09226] Worker 4 uses CPU cores [24, 25, 26, 27, 28, 29] | |
| [2025-02-21 23:34:04,255][09219] Worker 1 uses CPU cores [6, 7, 8, 9, 10, 11] | |
| [2025-02-21 23:34:04,368][09224] Worker 3 uses CPU cores [18, 19, 20, 21, 22, 23] | |
| [2025-02-21 23:34:05,950][09217] Worker 0 uses CPU cores [0, 1, 2, 3, 4, 5] | |
| [2025-02-21 23:34:07,037][09218] Worker 2 uses CPU cores [12, 13, 14, 15, 16, 17] | |
| [2025-02-21 23:34:07,061][09089] Heartbeat connected on Batcher_0 | |
| [2025-02-21 23:34:07,065][09089] Heartbeat connected on InferenceWorker_p0-w0 | |
| [2025-02-21 23:34:07,067][09089] Heartbeat connected on RolloutWorker_w4 | |
| [2025-02-21 23:34:07,070][09089] Heartbeat connected on RolloutWorker_w5 | |
| [2025-02-21 23:34:07,072][09089] Heartbeat connected on RolloutWorker_w6 | |
| [2025-02-21 23:34:07,073][09089] Heartbeat connected on RolloutWorker_w7 | |
| [2025-02-21 23:34:07,074][09089] Heartbeat connected on RolloutWorker_w1 | |
| [2025-02-21 23:34:07,076][09089] Heartbeat connected on RolloutWorker_w3 | |
| [2025-02-21 23:34:07,077][09089] Heartbeat connected on RolloutWorker_w0 | |
| [2025-02-21 23:34:07,230][09089] Heartbeat connected on RolloutWorker_w2 | |
| [2025-02-21 23:34:08,121][09144] No checkpoints found | |
| [2025-02-21 23:34:08,124][09144] Did not load from checkpoint, starting from scratch! | |
| [2025-02-21 23:34:08,126][09144] Initialized policy 0 weights for model version 0 | |
| [2025-02-21 23:34:08,172][09144] LearnerWorker_p0 finished initialization! | |
| [2025-02-21 23:34:08,177][09144] Using GPUs [0] for process 0 (actually maps to GPUs [0]) | |
| [2025-02-21 23:34:08,187][09089] Heartbeat connected on LearnerWorker_p0 | |
| [2025-02-21 23:34:08,314][09221] RunningMeanStd input shape: (3, 72, 128) | |
| [2025-02-21 23:34:08,318][09221] RunningMeanStd input shape: (1,) | |
| [2025-02-21 23:34:08,335][09221] ConvEncoder: input_channels=3 | |
| [2025-02-21 23:34:08,484][09221] Conv encoder output size: 512 | |
| [2025-02-21 23:34:08,486][09221] Policy head output size: 512 | |
| [2025-02-21 23:34:08,583][09089] Inference worker 0-0 is ready! | |
| [2025-02-21 23:34:08,589][09089] All inference workers are ready! Signal rollout workers to start! | |
| [2025-02-21 23:34:08,636][09225] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:34:08,637][09223] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:34:09,154][09223] Decorrelating experience for 0 frames... | |
| [2025-02-21 23:34:09,205][09226] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:34:09,205][09224] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:34:09,206][09217] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:34:09,207][09218] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:34:09,209][09219] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:34:09,252][09227] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:34:09,370][09089] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-02-21 23:34:09,572][09223] Decorrelating experience for 32 frames... | |
| [2025-02-21 23:34:09,988][09223] Decorrelating experience for 64 frames... | |
| [2025-02-21 23:34:10,002][09224] Decorrelating experience for 0 frames... | |
| [2025-02-21 23:34:10,334][09223] Decorrelating experience for 96 frames... | |
| [2025-02-21 23:34:10,428][09217] Decorrelating experience for 0 frames... | |
| [2025-02-21 23:34:10,431][09227] Decorrelating experience for 0 frames... | |
| [2025-02-21 23:34:10,677][09219] Decorrelating experience for 0 frames... | |
| [2025-02-21 23:34:11,179][09217] Decorrelating experience for 32 frames... | |
| [2025-02-21 23:34:11,290][09219] Decorrelating experience for 32 frames... | |
| [2025-02-21 23:34:13,082][09224] Decorrelating experience for 32 frames... | |
| [2025-02-21 23:34:13,188][09226] Decorrelating experience for 0 frames... | |
| [2025-02-21 23:34:13,418][09227] Decorrelating experience for 32 frames... | |
| [2025-02-21 23:34:13,576][09218] Decorrelating experience for 0 frames... | |
| [2025-02-21 23:34:13,886][09219] Decorrelating experience for 64 frames... | |
| [2025-02-21 23:34:13,885][09217] Decorrelating experience for 64 frames... | |
| [2025-02-21 23:34:14,315][09218] Decorrelating experience for 32 frames... | |
| [2025-02-21 23:34:14,360][09089] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 27.2. Samples: 136. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-02-21 23:34:14,365][09089] Avg episode reward: [(0, '3.503')] | |
| [2025-02-21 23:34:14,701][09217] Decorrelating experience for 96 frames... | |
| [2025-02-21 23:34:14,840][09227] Decorrelating experience for 64 frames... | |
| [2025-02-21 23:34:14,977][09224] Decorrelating experience for 64 frames... | |
| [2025-02-21 23:34:15,824][09226] Decorrelating experience for 32 frames... | |
| [2025-02-21 23:34:16,349][09219] Decorrelating experience for 96 frames... | |
| [2025-02-21 23:34:16,940][09144] Signal inference workers to stop experience collection... | |
| [2025-02-21 23:34:16,983][09221] InferenceWorker_p0-w0: stopping experience collection | |
| [2025-02-21 23:34:17,140][09227] Decorrelating experience for 96 frames... | |
| [2025-02-21 23:34:17,141][09218] Decorrelating experience for 64 frames... | |
| [2025-02-21 23:34:17,961][09224] Decorrelating experience for 96 frames... | |
| [2025-02-21 23:34:18,355][09218] Decorrelating experience for 96 frames... | |
| [2025-02-21 23:34:19,364][09089] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 232.9. Samples: 2328. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0) | |
| [2025-02-21 23:34:19,370][09089] Avg episode reward: [(0, '3.866')] | |
| [2025-02-21 23:34:20,406][09226] Decorrelating experience for 64 frames... | |
| [2025-02-21 23:34:21,282][09226] Decorrelating experience for 96 frames... | |
| [2025-02-21 23:34:23,659][09144] Signal inference workers to resume experience collection... | |
| [2025-02-21 23:34:23,666][09221] InferenceWorker_p0-w0: resuming experience collection | |
| [2025-02-21 23:34:24,361][09089] Fps is (10 sec: 409.6, 60 sec: 273.2, 300 sec: 273.2). Total num frames: 4096. Throughput: 0: 155.3. Samples: 2328. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0) | |
| [2025-02-21 23:34:24,367][09089] Avg episode reward: [(0, '3.866')] | |
| [2025-02-21 23:34:29,364][09089] Fps is (10 sec: 2457.9, 60 sec: 1229.1, 300 sec: 1229.1). Total num frames: 24576. Throughput: 0: 254.3. Samples: 5084. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2025-02-21 23:34:29,374][09089] Avg episode reward: [(0, '3.799')] | |
| [2025-02-21 23:34:32,513][09221] Updated weights for policy 0, policy_version 10 (0.0234) | |
| [2025-02-21 23:34:34,366][09089] Fps is (10 sec: 4094.4, 60 sec: 1802.4, 300 sec: 1802.4). Total num frames: 45056. Throughput: 0: 438.8. Samples: 10968. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:34:34,386][09089] Avg episode reward: [(0, '4.150')] | |
| [2025-02-21 23:34:39,360][09089] Fps is (10 sec: 4097.0, 60 sec: 2185.1, 300 sec: 2185.1). Total num frames: 65536. Throughput: 0: 565.7. Samples: 16968. Policy #0 lag: (min: 0.0, avg: 0.4, max: 2.0) | |
| [2025-02-21 23:34:39,362][09089] Avg episode reward: [(0, '4.395')] | |
| [2025-02-21 23:34:42,814][09221] Updated weights for policy 0, policy_version 20 (0.0013) | |
| [2025-02-21 23:34:44,368][09089] Fps is (10 sec: 4096.0, 60 sec: 2457.8, 300 sec: 2457.8). Total num frames: 86016. Throughput: 0: 574.3. Samples: 20100. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:34:44,383][09089] Avg episode reward: [(0, '4.381')] | |
| [2025-02-21 23:34:49,358][09089] Fps is (10 sec: 4096.3, 60 sec: 2663.0, 300 sec: 2663.0). Total num frames: 106496. Throughput: 0: 653.8. Samples: 26146. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:34:49,361][09089] Avg episode reward: [(0, '4.339')] | |
| [2025-02-21 23:34:49,363][09144] Saving new best policy, reward=4.339! | |
| [2025-02-21 23:34:52,897][09221] Updated weights for policy 0, policy_version 30 (0.0010) | |
| [2025-02-21 23:34:54,364][09089] Fps is (10 sec: 4097.0, 60 sec: 2822.0, 300 sec: 2822.0). Total num frames: 126976. Throughput: 0: 718.2. Samples: 32316. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:34:54,374][09089] Avg episode reward: [(0, '4.309')] | |
| [2025-02-21 23:34:59,361][09089] Fps is (10 sec: 4095.3, 60 sec: 2949.5, 300 sec: 2949.5). Total num frames: 147456. Throughput: 0: 782.6. Samples: 35354. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:34:59,366][09089] Avg episode reward: [(0, '4.471')] | |
| [2025-02-21 23:34:59,370][09144] Saving new best policy, reward=4.471! | |
| [2025-02-21 23:35:03,243][09221] Updated weights for policy 0, policy_version 40 (0.0014) | |
| [2025-02-21 23:35:04,361][09089] Fps is (10 sec: 4096.9, 60 sec: 3053.8, 300 sec: 3053.8). Total num frames: 167936. Throughput: 0: 864.7. Samples: 41238. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:35:04,369][09089] Avg episode reward: [(0, '4.595')] | |
| [2025-02-21 23:35:04,391][09144] Saving new best policy, reward=4.595! | |
| [2025-02-21 23:35:09,360][09089] Fps is (10 sec: 3686.6, 60 sec: 3072.4, 300 sec: 3072.4). Total num frames: 184320. Throughput: 0: 994.1. Samples: 47062. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) | |
| [2025-02-21 23:35:09,365][09089] Avg episode reward: [(0, '4.632')] | |
| [2025-02-21 23:35:09,536][09144] Saving new best policy, reward=4.632! | |
| [2025-02-21 23:35:13,538][09221] Updated weights for policy 0, policy_version 50 (0.0013) | |
| [2025-02-21 23:35:14,373][09089] Fps is (10 sec: 3683.5, 60 sec: 3412.9, 300 sec: 3150.7). Total num frames: 204800. Throughput: 0: 998.5. Samples: 50024. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:35:14,399][09089] Avg episode reward: [(0, '4.621')] | |
| [2025-02-21 23:35:19,363][09089] Fps is (10 sec: 4095.3, 60 sec: 3754.8, 300 sec: 3218.6). Total num frames: 225280. Throughput: 0: 997.5. Samples: 55854. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:35:19,370][09089] Avg episode reward: [(0, '4.503')] | |
| [2025-02-21 23:35:23,952][09221] Updated weights for policy 0, policy_version 60 (0.0014) | |
| [2025-02-21 23:35:24,362][09089] Fps is (10 sec: 4098.6, 60 sec: 4027.7, 300 sec: 3277.0). Total num frames: 245760. Throughput: 0: 994.8. Samples: 61738. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:35:24,369][09089] Avg episode reward: [(0, '4.394')] | |
| [2025-02-21 23:35:29,362][09089] Fps is (10 sec: 4096.1, 60 sec: 4027.8, 300 sec: 3328.3). Total num frames: 266240. Throughput: 0: 993.4. Samples: 64800. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:35:29,368][09089] Avg episode reward: [(0, '4.441')] | |
| [2025-02-21 23:35:34,148][09221] Updated weights for policy 0, policy_version 70 (0.0012) | |
| [2025-02-21 23:35:34,363][09089] Fps is (10 sec: 4096.1, 60 sec: 4027.9, 300 sec: 3373.4). Total num frames: 286720. Throughput: 0: 995.9. Samples: 70964. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:35:34,371][09089] Avg episode reward: [(0, '4.501')] | |
| [2025-02-21 23:35:34,417][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000070_286720.pth... | |
| [2025-02-21 23:35:39,362][09089] Fps is (10 sec: 4095.9, 60 sec: 4027.6, 300 sec: 3413.6). Total num frames: 307200. Throughput: 0: 994.1. Samples: 77048. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) | |
| [2025-02-21 23:35:39,368][09089] Avg episode reward: [(0, '4.781')] | |
| [2025-02-21 23:35:39,373][09144] Saving new best policy, reward=4.781! | |
| [2025-02-21 23:35:44,362][09089] Fps is (10 sec: 3686.5, 60 sec: 3959.7, 300 sec: 3406.4). Total num frames: 323584. Throughput: 0: 989.9. Samples: 79900. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:35:44,368][09089] Avg episode reward: [(0, '4.816')] | |
| [2025-02-21 23:35:44,516][09221] Updated weights for policy 0, policy_version 80 (0.0014) | |
| [2025-02-21 23:35:44,523][09144] Saving new best policy, reward=4.816! | |
| [2025-02-21 23:35:49,360][09089] Fps is (10 sec: 3687.3, 60 sec: 3959.5, 300 sec: 3440.9). Total num frames: 344064. Throughput: 0: 992.1. Samples: 85882. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) | |
| [2025-02-21 23:35:49,367][09089] Avg episode reward: [(0, '4.789')] | |
| [2025-02-21 23:35:54,363][09089] Fps is (10 sec: 4095.4, 60 sec: 3959.4, 300 sec: 3472.0). Total num frames: 364544. Throughput: 0: 1004.1. Samples: 92248. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:35:54,370][09089] Avg episode reward: [(0, '4.681')] | |
| [2025-02-21 23:35:54,410][09221] Updated weights for policy 0, policy_version 90 (0.0015) | |
| [2025-02-21 23:35:59,359][09089] Fps is (10 sec: 4096.0, 60 sec: 3959.6, 300 sec: 3500.5). Total num frames: 385024. Throughput: 0: 1005.4. Samples: 95260. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:35:59,362][09089] Avg episode reward: [(0, '4.723')] | |
| [2025-02-21 23:36:04,359][09089] Fps is (10 sec: 4097.5, 60 sec: 3959.5, 300 sec: 3526.4). Total num frames: 405504. Throughput: 0: 1011.1. Samples: 101350. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:36:04,362][09089] Avg episode reward: [(0, '4.750')] | |
| [2025-02-21 23:36:04,426][09221] Updated weights for policy 0, policy_version 100 (0.0019) | |
| [2025-02-21 23:36:09,362][09089] Fps is (10 sec: 4504.6, 60 sec: 4095.9, 300 sec: 3584.2). Total num frames: 430080. Throughput: 0: 1017.2. Samples: 107510. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:36:09,371][09089] Avg episode reward: [(0, '4.708')] | |
| [2025-02-21 23:36:14,230][09221] Updated weights for policy 0, policy_version 110 (0.0011) | |
| [2025-02-21 23:36:14,363][09089] Fps is (10 sec: 4504.0, 60 sec: 4096.4, 300 sec: 3604.6). Total num frames: 450560. Throughput: 0: 1016.3. Samples: 110534. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:36:14,369][09089] Avg episode reward: [(0, '4.888')] | |
| [2025-02-21 23:36:14,412][09144] Saving new best policy, reward=4.888! | |
| [2025-02-21 23:36:19,362][09089] Fps is (10 sec: 3686.5, 60 sec: 4027.8, 300 sec: 3592.1). Total num frames: 466944. Throughput: 0: 1014.6. Samples: 116622. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:36:19,370][09089] Avg episode reward: [(0, '4.965')] | |
| [2025-02-21 23:36:19,468][09144] Saving new best policy, reward=4.965! | |
| [2025-02-21 23:36:24,364][09089] Fps is (10 sec: 3686.0, 60 sec: 4027.6, 300 sec: 3610.6). Total num frames: 487424. Throughput: 0: 1011.3. Samples: 122558. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:36:24,372][09089] Avg episode reward: [(0, '4.871')] | |
| [2025-02-21 23:36:24,595][09221] Updated weights for policy 0, policy_version 120 (0.0013) | |
| [2025-02-21 23:36:29,358][09089] Fps is (10 sec: 4096.9, 60 sec: 4027.9, 300 sec: 3628.1). Total num frames: 507904. Throughput: 0: 1014.1. Samples: 125534. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:36:29,360][09089] Avg episode reward: [(0, '4.921')] | |
| [2025-02-21 23:36:34,359][09089] Fps is (10 sec: 4097.9, 60 sec: 4027.9, 300 sec: 3644.2). Total num frames: 528384. Throughput: 0: 1015.6. Samples: 131586. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:36:34,361][09089] Avg episode reward: [(0, '4.864')] | |
| [2025-02-21 23:36:34,978][09221] Updated weights for policy 0, policy_version 130 (0.0013) | |
| [2025-02-21 23:36:39,359][09089] Fps is (10 sec: 4096.0, 60 sec: 4027.9, 300 sec: 3659.3). Total num frames: 548864. Throughput: 0: 1013.4. Samples: 137848. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:36:39,361][09089] Avg episode reward: [(0, '4.587')] | |
| [2025-02-21 23:36:44,360][09089] Fps is (10 sec: 3686.0, 60 sec: 4027.8, 300 sec: 3646.9). Total num frames: 565248. Throughput: 0: 997.4. Samples: 140144. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:36:44,366][09089] Avg episode reward: [(0, '4.773')] | |
| [2025-02-21 23:36:45,971][09221] Updated weights for policy 0, policy_version 140 (0.0016) | |
| [2025-02-21 23:36:49,363][09089] Fps is (10 sec: 3685.1, 60 sec: 4027.5, 300 sec: 3660.9). Total num frames: 585728. Throughput: 0: 992.3. Samples: 146008. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:36:49,371][09089] Avg episode reward: [(0, '4.850')] | |
| [2025-02-21 23:36:54,363][09089] Fps is (10 sec: 4095.0, 60 sec: 4027.8, 300 sec: 3674.1). Total num frames: 606208. Throughput: 0: 999.8. Samples: 152502. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:36:54,369][09089] Avg episode reward: [(0, '5.085')] | |
| [2025-02-21 23:36:54,490][09144] Saving new best policy, reward=5.085! | |
| [2025-02-21 23:36:55,784][09221] Updated weights for policy 0, policy_version 150 (0.0011) | |
| [2025-02-21 23:36:59,359][09089] Fps is (10 sec: 4507.1, 60 sec: 4096.0, 300 sec: 3710.7). Total num frames: 630784. Throughput: 0: 999.4. Samples: 155502. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:36:59,361][09089] Avg episode reward: [(0, '5.363')] | |
| [2025-02-21 23:36:59,364][09144] Saving new best policy, reward=5.363! | |
| [2025-02-21 23:37:04,362][09089] Fps is (10 sec: 4096.5, 60 sec: 4027.6, 300 sec: 3698.2). Total num frames: 647168. Throughput: 0: 996.4. Samples: 161462. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) | |
| [2025-02-21 23:37:04,371][09089] Avg episode reward: [(0, '5.298')] | |
| [2025-02-21 23:37:05,729][09221] Updated weights for policy 0, policy_version 160 (0.0013) | |
| [2025-02-21 23:37:09,367][09089] Fps is (10 sec: 3684.1, 60 sec: 3959.2, 300 sec: 3709.2). Total num frames: 667648. Throughput: 0: 1002.5. Samples: 167670. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:37:09,378][09089] Avg episode reward: [(0, '5.278')] | |
| [2025-02-21 23:37:14,363][09089] Fps is (10 sec: 4505.1, 60 sec: 4027.8, 300 sec: 3741.9). Total num frames: 692224. Throughput: 0: 1008.1. Samples: 170902. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:37:14,370][09089] Avg episode reward: [(0, '5.175')] | |
| [2025-02-21 23:37:15,550][09221] Updated weights for policy 0, policy_version 170 (0.0012) | |
| [2025-02-21 23:37:19,369][09089] Fps is (10 sec: 4096.8, 60 sec: 4027.6, 300 sec: 3729.6). Total num frames: 708608. Throughput: 0: 1008.0. Samples: 176952. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:37:19,388][09089] Avg episode reward: [(0, '5.454')] | |
| [2025-02-21 23:37:19,405][09144] Saving new best policy, reward=5.454! | |
| [2025-02-21 23:37:24,363][09089] Fps is (10 sec: 3686.5, 60 sec: 4027.9, 300 sec: 3739.0). Total num frames: 729088. Throughput: 0: 998.7. Samples: 182792. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:37:24,369][09089] Avg episode reward: [(0, '5.523')] | |
| [2025-02-21 23:37:24,397][09144] Saving new best policy, reward=5.523! | |
| [2025-02-21 23:37:25,907][09221] Updated weights for policy 0, policy_version 180 (0.0017) | |
| [2025-02-21 23:37:29,359][09089] Fps is (10 sec: 4097.5, 60 sec: 4027.7, 300 sec: 3748.0). Total num frames: 749568. Throughput: 0: 1015.8. Samples: 185856. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) | |
| [2025-02-21 23:37:29,363][09089] Avg episode reward: [(0, '5.846')] | |
| [2025-02-21 23:37:29,366][09144] Saving new best policy, reward=5.846! | |
| [2025-02-21 23:37:34,364][09089] Fps is (10 sec: 4095.7, 60 sec: 4027.5, 300 sec: 3756.4). Total num frames: 770048. Throughput: 0: 1029.5. Samples: 192336. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:37:34,371][09089] Avg episode reward: [(0, '5.789')] | |
| [2025-02-21 23:37:34,435][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000189_774144.pth... | |
| [2025-02-21 23:37:35,542][09221] Updated weights for policy 0, policy_version 190 (0.0010) | |
| [2025-02-21 23:37:39,363][09089] Fps is (10 sec: 4504.3, 60 sec: 4095.8, 300 sec: 3784.0). Total num frames: 794624. Throughput: 0: 1025.9. Samples: 198666. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:37:39,371][09089] Avg episode reward: [(0, '5.650')] | |
| [2025-02-21 23:37:44,361][09089] Fps is (10 sec: 4096.6, 60 sec: 4095.9, 300 sec: 3772.2). Total num frames: 811008. Throughput: 0: 1026.6. Samples: 201702. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:37:44,365][09089] Avg episode reward: [(0, '5.627')] | |
| [2025-02-21 23:37:45,486][09221] Updated weights for policy 0, policy_version 200 (0.0012) | |
| [2025-02-21 23:37:49,362][09089] Fps is (10 sec: 4096.3, 60 sec: 4164.3, 300 sec: 3798.2). Total num frames: 835584. Throughput: 0: 1034.4. Samples: 208010. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:37:49,374][09089] Avg episode reward: [(0, '5.561')] | |
| [2025-02-21 23:37:54,361][09089] Fps is (10 sec: 4505.8, 60 sec: 4164.4, 300 sec: 3804.8). Total num frames: 856064. Throughput: 0: 1035.5. Samples: 214262. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:37:54,367][09089] Avg episode reward: [(0, '5.481')] | |
| [2025-02-21 23:37:55,317][09221] Updated weights for policy 0, policy_version 210 (0.0015) | |
| [2025-02-21 23:37:59,362][09089] Fps is (10 sec: 3686.4, 60 sec: 4027.6, 300 sec: 3793.3). Total num frames: 872448. Throughput: 0: 1032.9. Samples: 217382. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:37:59,369][09089] Avg episode reward: [(0, '5.762')] | |
| [2025-02-21 23:38:04,360][09089] Fps is (10 sec: 4096.3, 60 sec: 4164.4, 300 sec: 3817.2). Total num frames: 897024. Throughput: 0: 1038.1. Samples: 223662. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:38:04,363][09089] Avg episode reward: [(0, '6.131')] | |
| [2025-02-21 23:38:04,413][09144] Saving new best policy, reward=6.131! | |
| [2025-02-21 23:38:06,025][09221] Updated weights for policy 0, policy_version 220 (0.0017) | |
| [2025-02-21 23:38:09,362][09089] Fps is (10 sec: 4505.8, 60 sec: 4164.5, 300 sec: 3823.0). Total num frames: 917504. Throughput: 0: 1040.0. Samples: 229590. Policy #0 lag: (min: 0.0, avg: 1.2, max: 2.0) | |
| [2025-02-21 23:38:09,368][09089] Avg episode reward: [(0, '6.270')] | |
| [2025-02-21 23:38:09,375][09144] Saving new best policy, reward=6.270! | |
| [2025-02-21 23:38:14,359][09089] Fps is (10 sec: 4096.4, 60 sec: 4096.2, 300 sec: 3828.6). Total num frames: 937984. Throughput: 0: 1041.9. Samples: 232740. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:38:14,360][09089] Avg episode reward: [(0, '6.051')] | |
| [2025-02-21 23:38:15,183][09221] Updated weights for policy 0, policy_version 230 (0.0011) | |
| [2025-02-21 23:38:19,361][09089] Fps is (10 sec: 4096.3, 60 sec: 4164.5, 300 sec: 3834.0). Total num frames: 958464. Throughput: 0: 1033.6. Samples: 238846. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:38:19,367][09089] Avg episode reward: [(0, '5.944')] | |
| [2025-02-21 23:38:24,363][09089] Fps is (10 sec: 4094.6, 60 sec: 4164.2, 300 sec: 3839.1). Total num frames: 978944. Throughput: 0: 1031.3. Samples: 245076. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:38:24,369][09089] Avg episode reward: [(0, '6.240')] | |
| [2025-02-21 23:38:25,068][09221] Updated weights for policy 0, policy_version 240 (0.0017) | |
| [2025-02-21 23:38:29,360][09089] Fps is (10 sec: 4096.2, 60 sec: 4164.2, 300 sec: 3844.0). Total num frames: 999424. Throughput: 0: 1029.0. Samples: 248006. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:38:29,364][09089] Avg episode reward: [(0, '6.350')] | |
| [2025-02-21 23:38:29,368][09144] Saving new best policy, reward=6.350! | |
| [2025-02-21 23:38:34,360][09089] Fps is (10 sec: 3277.5, 60 sec: 4027.9, 300 sec: 3817.9). Total num frames: 1011712. Throughput: 0: 990.4. Samples: 252576. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:38:34,363][09089] Avg episode reward: [(0, '6.873')] | |
| [2025-02-21 23:38:34,377][09144] Saving new best policy, reward=6.873! | |
| [2025-02-21 23:38:37,010][09221] Updated weights for policy 0, policy_version 250 (0.0014) | |
| [2025-02-21 23:38:39,360][09089] Fps is (10 sec: 2867.3, 60 sec: 3891.4, 300 sec: 3807.9). Total num frames: 1028096. Throughput: 0: 975.1. Samples: 258142. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:38:39,364][09089] Avg episode reward: [(0, '6.574')] | |
| [2025-02-21 23:38:44,365][09089] Fps is (10 sec: 3684.9, 60 sec: 3959.3, 300 sec: 3813.0). Total num frames: 1048576. Throughput: 0: 962.0. Samples: 260674. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:38:44,374][09089] Avg episode reward: [(0, '7.186')] | |
| [2025-02-21 23:38:44,401][09144] Saving new best policy, reward=7.186! | |
| [2025-02-21 23:38:48,103][09221] Updated weights for policy 0, policy_version 260 (0.0019) | |
| [2025-02-21 23:38:49,369][09089] Fps is (10 sec: 4092.7, 60 sec: 3890.8, 300 sec: 3818.1). Total num frames: 1069056. Throughput: 0: 951.3. Samples: 266480. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:38:49,375][09089] Avg episode reward: [(0, '7.242')] | |
| [2025-02-21 23:38:49,383][09144] Saving new best policy, reward=7.242! | |
| [2025-02-21 23:38:54,363][09089] Fps is (10 sec: 4096.7, 60 sec: 3891.1, 300 sec: 3823.0). Total num frames: 1089536. Throughput: 0: 953.0. Samples: 272478. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:38:54,369][09089] Avg episode reward: [(0, '8.013')] | |
| [2025-02-21 23:38:54,401][09144] Saving new best policy, reward=8.013! | |
| [2025-02-21 23:38:58,330][09221] Updated weights for policy 0, policy_version 270 (0.0012) | |
| [2025-02-21 23:38:59,362][09089] Fps is (10 sec: 4098.7, 60 sec: 3959.5, 300 sec: 3827.7). Total num frames: 1110016. Throughput: 0: 953.9. Samples: 275666. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:38:59,367][09089] Avg episode reward: [(0, '7.802')] | |
| [2025-02-21 23:39:04,366][09089] Fps is (10 sec: 4094.8, 60 sec: 3890.8, 300 sec: 3832.2). Total num frames: 1130496. Throughput: 0: 955.8. Samples: 281860. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:39:04,374][09089] Avg episode reward: [(0, '8.360')] | |
| [2025-02-21 23:39:04,402][09144] Saving new best policy, reward=8.360! | |
| [2025-02-21 23:39:08,534][09221] Updated weights for policy 0, policy_version 280 (0.0018) | |
| [2025-02-21 23:39:09,364][09089] Fps is (10 sec: 3685.3, 60 sec: 3822.7, 300 sec: 3887.7). Total num frames: 1146880. Throughput: 0: 942.3. Samples: 287480. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:39:09,374][09089] Avg episode reward: [(0, '8.151')] | |
| [2025-02-21 23:39:14,365][09089] Fps is (10 sec: 3687.0, 60 sec: 3822.6, 300 sec: 3957.1). Total num frames: 1167360. Throughput: 0: 943.0. Samples: 290444. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:39:14,374][09089] Avg episode reward: [(0, '8.512')] | |
| [2025-02-21 23:39:14,409][09144] Saving new best policy, reward=8.512! | |
| [2025-02-21 23:39:18,381][09221] Updated weights for policy 0, policy_version 290 (0.0011) | |
| [2025-02-21 23:39:19,362][09089] Fps is (10 sec: 4097.1, 60 sec: 3822.9, 300 sec: 4012.7). Total num frames: 1187840. Throughput: 0: 982.3. Samples: 296780. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:39:19,370][09089] Avg episode reward: [(0, '8.206')] | |
| [2025-02-21 23:39:24,363][09089] Fps is (10 sec: 4096.8, 60 sec: 3822.9, 300 sec: 4012.7). Total num frames: 1208320. Throughput: 0: 994.3. Samples: 302890. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:39:24,370][09089] Avg episode reward: [(0, '9.249')] | |
| [2025-02-21 23:39:24,559][09144] Saving new best policy, reward=9.249! | |
| [2025-02-21 23:39:28,863][09221] Updated weights for policy 0, policy_version 300 (0.0031) | |
| [2025-02-21 23:39:29,368][09089] Fps is (10 sec: 4094.4, 60 sec: 3822.6, 300 sec: 4012.7). Total num frames: 1228800. Throughput: 0: 1003.1. Samples: 305816. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:39:29,377][09089] Avg episode reward: [(0, '9.990')] | |
| [2025-02-21 23:39:29,385][09144] Saving new best policy, reward=9.990! | |
| [2025-02-21 23:39:34,362][09089] Fps is (10 sec: 4096.1, 60 sec: 3959.4, 300 sec: 4012.7). Total num frames: 1249280. Throughput: 0: 1007.0. Samples: 311790. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:39:34,368][09089] Avg episode reward: [(0, '10.637')] | |
| [2025-02-21 23:39:34,408][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000305_1249280.pth... | |
| [2025-02-21 23:39:35,072][09144] Removing /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000070_286720.pth | |
| [2025-02-21 23:39:35,107][09144] Saving new best policy, reward=10.637! | |
| [2025-02-21 23:39:38,848][09221] Updated weights for policy 0, policy_version 310 (0.0014) | |
| [2025-02-21 23:39:39,362][09089] Fps is (10 sec: 4097.4, 60 sec: 4027.6, 300 sec: 4012.7). Total num frames: 1269760. Throughput: 0: 1013.2. Samples: 318070. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) | |
| [2025-02-21 23:39:39,368][09089] Avg episode reward: [(0, '9.894')] | |
| [2025-02-21 23:39:44,364][09089] Fps is (10 sec: 4095.4, 60 sec: 4027.8, 300 sec: 4012.6). Total num frames: 1290240. Throughput: 0: 1015.1. Samples: 321346. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:39:44,379][09089] Avg episode reward: [(0, '10.753')] | |
| [2025-02-21 23:39:44,453][09144] Saving new best policy, reward=10.753! | |
| [2025-02-21 23:39:48,737][09221] Updated weights for policy 0, policy_version 320 (0.0014) | |
| [2025-02-21 23:39:49,360][09089] Fps is (10 sec: 4096.7, 60 sec: 4028.2, 300 sec: 4012.7). Total num frames: 1310720. Throughput: 0: 1006.6. Samples: 327150. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:39:49,366][09089] Avg episode reward: [(0, '12.119')] | |
| [2025-02-21 23:39:49,371][09144] Saving new best policy, reward=12.119! | |
| [2025-02-21 23:39:54,394][09089] Fps is (10 sec: 4083.5, 60 sec: 4025.6, 300 sec: 4012.2). Total num frames: 1331200. Throughput: 0: 1014.1. Samples: 333144. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:39:54,400][09089] Avg episode reward: [(0, '12.567')] | |
| [2025-02-21 23:39:54,430][09144] Saving new best policy, reward=12.567! | |
| [2025-02-21 23:39:59,363][09089] Fps is (10 sec: 3685.7, 60 sec: 3959.4, 300 sec: 3998.8). Total num frames: 1347584. Throughput: 0: 1011.5. Samples: 335958. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) | |
| [2025-02-21 23:39:59,369][09089] Avg episode reward: [(0, '14.597')] | |
| [2025-02-21 23:39:59,422][09221] Updated weights for policy 0, policy_version 330 (0.0013) | |
| [2025-02-21 23:39:59,426][09144] Saving new best policy, reward=14.597! | |
| [2025-02-21 23:40:04,364][09089] Fps is (10 sec: 4108.5, 60 sec: 4027.9, 300 sec: 4026.5). Total num frames: 1372160. Throughput: 0: 1010.8. Samples: 342268. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:40:04,374][09089] Avg episode reward: [(0, '14.089')] | |
| [2025-02-21 23:40:09,236][09221] Updated weights for policy 0, policy_version 340 (0.0012) | |
| [2025-02-21 23:40:09,361][09089] Fps is (10 sec: 4506.4, 60 sec: 4096.3, 300 sec: 4026.7). Total num frames: 1392640. Throughput: 0: 1011.9. Samples: 348424. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:40:09,369][09089] Avg episode reward: [(0, '12.779')] | |
| [2025-02-21 23:40:14,363][09089] Fps is (10 sec: 4096.6, 60 sec: 4096.1, 300 sec: 4026.6). Total num frames: 1413120. Throughput: 0: 1018.8. Samples: 351660. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:40:14,368][09089] Avg episode reward: [(0, '12.875')] | |
| [2025-02-21 23:40:18,690][09221] Updated weights for policy 0, policy_version 350 (0.0014) | |
| [2025-02-21 23:40:19,363][09089] Fps is (10 sec: 4095.3, 60 sec: 4096.0, 300 sec: 4026.6). Total num frames: 1433600. Throughput: 0: 1025.1. Samples: 357920. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:40:19,369][09089] Avg episode reward: [(0, '12.599')] | |
| [2025-02-21 23:40:24,361][09089] Fps is (10 sec: 4096.6, 60 sec: 4096.1, 300 sec: 4026.6). Total num frames: 1454080. Throughput: 0: 1021.0. Samples: 364014. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:40:24,364][09089] Avg episode reward: [(0, '13.304')] | |
| [2025-02-21 23:40:28,651][09221] Updated weights for policy 0, policy_version 360 (0.0014) | |
| [2025-02-21 23:40:29,369][09089] Fps is (10 sec: 4094.4, 60 sec: 4096.0, 300 sec: 4026.5). Total num frames: 1474560. Throughput: 0: 1018.0. Samples: 367160. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:40:29,390][09089] Avg episode reward: [(0, '12.757')] | |
| [2025-02-21 23:40:34,364][09089] Fps is (10 sec: 4095.3, 60 sec: 4096.0, 300 sec: 4026.6). Total num frames: 1495040. Throughput: 0: 1030.5. Samples: 373526. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:40:34,369][09089] Avg episode reward: [(0, '12.563')] | |
| [2025-02-21 23:40:38,520][09221] Updated weights for policy 0, policy_version 370 (0.0014) | |
| [2025-02-21 23:40:39,363][09089] Fps is (10 sec: 4097.5, 60 sec: 4096.0, 300 sec: 4040.5). Total num frames: 1515520. Throughput: 0: 1039.7. Samples: 379898. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:40:39,371][09089] Avg episode reward: [(0, '11.384')] | |
| [2025-02-21 23:40:44,364][09089] Fps is (10 sec: 3686.1, 60 sec: 4027.7, 300 sec: 4026.5). Total num frames: 1531904. Throughput: 0: 1051.3. Samples: 383268. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:40:44,370][09089] Avg episode reward: [(0, '11.243')] | |
| [2025-02-21 23:40:49,362][09089] Fps is (10 sec: 3686.6, 60 sec: 4027.6, 300 sec: 4026.6). Total num frames: 1552384. Throughput: 0: 1023.3. Samples: 388314. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:40:49,369][09089] Avg episode reward: [(0, '12.853')] | |
| [2025-02-21 23:40:49,455][09221] Updated weights for policy 0, policy_version 380 (0.0022) | |
| [2025-02-21 23:40:54,362][09089] Fps is (10 sec: 4096.5, 60 sec: 4029.9, 300 sec: 4026.5). Total num frames: 1572864. Throughput: 0: 1016.7. Samples: 394178. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:40:54,369][09089] Avg episode reward: [(0, '14.396')] | |
| [2025-02-21 23:40:59,364][09089] Fps is (10 sec: 4095.4, 60 sec: 4095.9, 300 sec: 4026.5). Total num frames: 1593344. Throughput: 0: 1012.7. Samples: 397234. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:40:59,373][09089] Avg episode reward: [(0, '15.257')] | |
| [2025-02-21 23:40:59,472][09221] Updated weights for policy 0, policy_version 390 (0.0014) | |
| [2025-02-21 23:40:59,482][09144] Saving new best policy, reward=15.257! | |
| [2025-02-21 23:41:04,370][09089] Fps is (10 sec: 4093.2, 60 sec: 4027.4, 300 sec: 4012.6). Total num frames: 1613824. Throughput: 0: 1004.8. Samples: 403144. Policy #0 lag: (min: 1.0, avg: 1.1, max: 2.0) | |
| [2025-02-21 23:41:04,386][09089] Avg episode reward: [(0, '15.909')] | |
| [2025-02-21 23:41:04,455][09144] Saving new best policy, reward=15.909! | |
| [2025-02-21 23:41:09,364][09089] Fps is (10 sec: 3686.5, 60 sec: 3959.3, 300 sec: 3998.8). Total num frames: 1630208. Throughput: 0: 989.4. Samples: 408538. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:41:09,372][09089] Avg episode reward: [(0, '14.538')] | |
| [2025-02-21 23:41:10,600][09221] Updated weights for policy 0, policy_version 400 (0.0018) | |
| [2025-02-21 23:41:14,361][09089] Fps is (10 sec: 3689.6, 60 sec: 3959.6, 300 sec: 4012.7). Total num frames: 1650688. Throughput: 0: 986.0. Samples: 411526. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:41:14,367][09089] Avg episode reward: [(0, '14.325')] | |
| [2025-02-21 23:41:19,362][09089] Fps is (10 sec: 3686.7, 60 sec: 3891.2, 300 sec: 3998.8). Total num frames: 1667072. Throughput: 0: 957.3. Samples: 416602. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:41:19,369][09089] Avg episode reward: [(0, '14.923')] | |
| [2025-02-21 23:41:22,420][09221] Updated weights for policy 0, policy_version 410 (0.0019) | |
| [2025-02-21 23:41:24,364][09089] Fps is (10 sec: 3685.7, 60 sec: 3891.1, 300 sec: 3998.8). Total num frames: 1687552. Throughput: 0: 934.9. Samples: 421970. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:41:24,377][09089] Avg episode reward: [(0, '15.341')] | |
| [2025-02-21 23:41:29,361][09089] Fps is (10 sec: 3687.0, 60 sec: 3823.3, 300 sec: 3984.9). Total num frames: 1703936. Throughput: 0: 927.9. Samples: 425020. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:41:29,366][09089] Avg episode reward: [(0, '15.000')] | |
| [2025-02-21 23:41:32,569][09221] Updated weights for policy 0, policy_version 420 (0.0015) | |
| [2025-02-21 23:41:34,377][09089] Fps is (10 sec: 3682.2, 60 sec: 3822.2, 300 sec: 3984.7). Total num frames: 1724416. Throughput: 0: 946.4. Samples: 430912. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:41:34,394][09089] Avg episode reward: [(0, '15.993')] | |
| [2025-02-21 23:41:34,490][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000421_1724416.pth... | |
| [2025-02-21 23:41:35,513][09144] Removing /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000189_774144.pth | |
| [2025-02-21 23:41:35,519][09144] Saving new best policy, reward=15.993! | |
| [2025-02-21 23:41:39,363][09089] Fps is (10 sec: 4095.3, 60 sec: 3822.9, 300 sec: 3998.8). Total num frames: 1744896. Throughput: 0: 940.2. Samples: 436488. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:41:39,370][09089] Avg episode reward: [(0, '15.972')] | |
| [2025-02-21 23:41:43,474][09221] Updated weights for policy 0, policy_version 430 (0.0017) | |
| [2025-02-21 23:41:44,362][09089] Fps is (10 sec: 4101.0, 60 sec: 3891.3, 300 sec: 3998.8). Total num frames: 1765376. Throughput: 0: 933.1. Samples: 439224. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:41:44,367][09089] Avg episode reward: [(0, '16.470')] | |
| [2025-02-21 23:41:44,391][09144] Saving new best policy, reward=16.470! | |
| [2025-02-21 23:41:49,364][09089] Fps is (10 sec: 3686.4, 60 sec: 3822.9, 300 sec: 3984.9). Total num frames: 1781760. Throughput: 0: 933.3. Samples: 445138. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:41:49,379][09089] Avg episode reward: [(0, '16.558')] | |
| [2025-02-21 23:41:49,392][09144] Saving new best policy, reward=16.558! | |
| [2025-02-21 23:41:53,967][09221] Updated weights for policy 0, policy_version 440 (0.0012) | |
| [2025-02-21 23:41:54,359][09089] Fps is (10 sec: 3687.2, 60 sec: 3823.1, 300 sec: 3971.0). Total num frames: 1802240. Throughput: 0: 944.4. Samples: 451034. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0) | |
| [2025-02-21 23:41:54,362][09089] Avg episode reward: [(0, '17.539')] | |
| [2025-02-21 23:41:54,372][09144] Saving new best policy, reward=17.539! | |
| [2025-02-21 23:41:59,360][09089] Fps is (10 sec: 4097.1, 60 sec: 3823.2, 300 sec: 3984.9). Total num frames: 1822720. Throughput: 0: 943.1. Samples: 453966. Policy #0 lag: (min: 0.0, avg: 1.3, max: 2.0) | |
| [2025-02-21 23:41:59,364][09089] Avg episode reward: [(0, '17.649')] | |
| [2025-02-21 23:41:59,368][09144] Saving new best policy, reward=17.649! | |
| [2025-02-21 23:42:04,345][09221] Updated weights for policy 0, policy_version 450 (0.0014) | |
| [2025-02-21 23:42:04,364][09089] Fps is (10 sec: 3684.8, 60 sec: 3755.0, 300 sec: 3971.1). Total num frames: 1839104. Throughput: 0: 961.9. Samples: 459890. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:42:04,370][09089] Avg episode reward: [(0, '17.725')] | |
| [2025-02-21 23:42:04,442][09144] Saving new best policy, reward=17.725! | |
| [2025-02-21 23:42:09,361][09089] Fps is (10 sec: 4095.5, 60 sec: 3891.3, 300 sec: 3971.1). Total num frames: 1863680. Throughput: 0: 981.6. Samples: 466142. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:42:09,368][09089] Avg episode reward: [(0, '17.132')] | |
| [2025-02-21 23:42:14,319][09221] Updated weights for policy 0, policy_version 460 (0.0011) | |
| [2025-02-21 23:42:14,362][09089] Fps is (10 sec: 4506.5, 60 sec: 3891.1, 300 sec: 3984.9). Total num frames: 1884160. Throughput: 0: 978.9. Samples: 469072. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:42:14,368][09089] Avg episode reward: [(0, '17.697')] | |
| [2025-02-21 23:42:19,363][09089] Fps is (10 sec: 3685.9, 60 sec: 3891.2, 300 sec: 3971.0). Total num frames: 1900544. Throughput: 0: 984.2. Samples: 475188. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:42:19,371][09089] Avg episode reward: [(0, '17.568')] | |
| [2025-02-21 23:42:24,189][09221] Updated weights for policy 0, policy_version 470 (0.0015) | |
| [2025-02-21 23:42:24,363][09089] Fps is (10 sec: 4095.7, 60 sec: 3959.5, 300 sec: 3984.9). Total num frames: 1925120. Throughput: 0: 1000.0. Samples: 481486. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:42:24,369][09089] Avg episode reward: [(0, '18.011')] | |
| [2025-02-21 23:42:24,414][09144] Saving new best policy, reward=18.011! | |
| [2025-02-21 23:42:29,359][09089] Fps is (10 sec: 4506.9, 60 sec: 4027.8, 300 sec: 3985.0). Total num frames: 1945600. Throughput: 0: 1007.0. Samples: 484538. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:42:29,364][09089] Avg episode reward: [(0, '19.107')] | |
| [2025-02-21 23:42:29,368][09144] Saving new best policy, reward=19.107! | |
| [2025-02-21 23:42:34,099][09221] Updated weights for policy 0, policy_version 480 (0.0013) | |
| [2025-02-21 23:42:34,359][09089] Fps is (10 sec: 4097.2, 60 sec: 4028.7, 300 sec: 3971.1). Total num frames: 1966080. Throughput: 0: 1013.0. Samples: 490720. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:42:34,361][09089] Avg episode reward: [(0, '21.363')] | |
| [2025-02-21 23:42:34,371][09144] Saving new best policy, reward=21.363! | |
| [2025-02-21 23:42:39,359][09089] Fps is (10 sec: 4096.0, 60 sec: 4027.9, 300 sec: 3984.9). Total num frames: 1986560. Throughput: 0: 1020.5. Samples: 496958. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) | |
| [2025-02-21 23:42:39,362][09089] Avg episode reward: [(0, '20.985')] | |
| [2025-02-21 23:42:44,364][09089] Fps is (10 sec: 3685.1, 60 sec: 3959.4, 300 sec: 3957.1). Total num frames: 2002944. Throughput: 0: 1021.8. Samples: 499950. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:42:44,372][09089] Avg episode reward: [(0, '22.192')] | |
| [2025-02-21 23:42:44,392][09221] Updated weights for policy 0, policy_version 490 (0.0014) | |
| [2025-02-21 23:42:44,401][09144] Saving new best policy, reward=22.192! | |
| [2025-02-21 23:42:49,373][09089] Fps is (10 sec: 3681.9, 60 sec: 4027.1, 300 sec: 3957.0). Total num frames: 2023424. Throughput: 0: 1015.3. Samples: 505588. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:42:49,383][09089] Avg episode reward: [(0, '22.602')] | |
| [2025-02-21 23:42:49,405][09144] Saving new best policy, reward=22.602! | |
| [2025-02-21 23:42:54,359][09089] Fps is (10 sec: 4097.5, 60 sec: 4027.7, 300 sec: 3971.1). Total num frames: 2043904. Throughput: 0: 1000.0. Samples: 511142. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:42:54,361][09089] Avg episode reward: [(0, '23.683')] | |
| [2025-02-21 23:42:54,374][09144] Saving new best policy, reward=23.683! | |
| [2025-02-21 23:42:55,852][09221] Updated weights for policy 0, policy_version 500 (0.0014) | |
| [2025-02-21 23:42:59,368][09089] Fps is (10 sec: 3688.2, 60 sec: 3959.0, 300 sec: 3943.2). Total num frames: 2060288. Throughput: 0: 1001.4. Samples: 514138. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:42:59,382][09089] Avg episode reward: [(0, '23.949')] | |
| [2025-02-21 23:42:59,397][09144] Saving new best policy, reward=23.949! | |
| [2025-02-21 23:43:04,359][09089] Fps is (10 sec: 3686.4, 60 sec: 4028.0, 300 sec: 3943.3). Total num frames: 2080768. Throughput: 0: 985.3. Samples: 519524. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:43:04,362][09089] Avg episode reward: [(0, '24.824')] | |
| [2025-02-21 23:43:04,373][09144] Saving new best policy, reward=24.824! | |
| [2025-02-21 23:43:06,182][09221] Updated weights for policy 0, policy_version 510 (0.0020) | |
| [2025-02-21 23:43:09,359][09089] Fps is (10 sec: 4099.1, 60 sec: 3959.6, 300 sec: 3943.3). Total num frames: 2101248. Throughput: 0: 975.7. Samples: 525390. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:43:09,361][09089] Avg episode reward: [(0, '25.098')] | |
| [2025-02-21 23:43:09,362][09144] Saving new best policy, reward=25.098! | |
| [2025-02-21 23:43:14,361][09089] Fps is (10 sec: 3685.9, 60 sec: 3891.3, 300 sec: 3929.4). Total num frames: 2117632. Throughput: 0: 973.3. Samples: 528336. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:43:14,365][09089] Avg episode reward: [(0, '25.132')] | |
| [2025-02-21 23:43:14,386][09144] Saving new best policy, reward=25.132! | |
| [2025-02-21 23:43:17,186][09221] Updated weights for policy 0, policy_version 520 (0.0012) | |
| [2025-02-21 23:43:19,371][09089] Fps is (10 sec: 3273.9, 60 sec: 3890.8, 300 sec: 3915.4). Total num frames: 2134016. Throughput: 0: 952.0. Samples: 533570. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:43:19,377][09089] Avg episode reward: [(0, '24.565')] | |
| [2025-02-21 23:43:24,359][09089] Fps is (10 sec: 4096.6, 60 sec: 3891.4, 300 sec: 3929.4). Total num frames: 2158592. Throughput: 0: 947.6. Samples: 539602. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:43:24,361][09089] Avg episode reward: [(0, '22.460')] | |
| [2025-02-21 23:43:27,348][09221] Updated weights for policy 0, policy_version 530 (0.0013) | |
| [2025-02-21 23:43:29,363][09089] Fps is (10 sec: 4098.0, 60 sec: 3822.7, 300 sec: 3943.2). Total num frames: 2174976. Throughput: 0: 947.5. Samples: 542586. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:43:29,370][09089] Avg episode reward: [(0, '21.017')] | |
| [2025-02-21 23:43:34,375][09089] Fps is (10 sec: 3680.8, 60 sec: 3822.0, 300 sec: 3957.0). Total num frames: 2195456. Throughput: 0: 955.9. Samples: 548604. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:43:34,383][09089] Avg episode reward: [(0, '18.629')] | |
| [2025-02-21 23:43:34,436][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000537_2199552.pth... | |
| [2025-02-21 23:43:35,324][09144] Removing /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000305_1249280.pth | |
| [2025-02-21 23:43:37,489][09221] Updated weights for policy 0, policy_version 540 (0.0012) | |
| [2025-02-21 23:43:39,363][09089] Fps is (10 sec: 4096.1, 60 sec: 3822.7, 300 sec: 3957.2). Total num frames: 2215936. Throughput: 0: 968.4. Samples: 554724. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:43:39,369][09089] Avg episode reward: [(0, '18.616')] | |
| [2025-02-21 23:43:44,368][09089] Fps is (10 sec: 4099.3, 60 sec: 3891.0, 300 sec: 3957.2). Total num frames: 2236416. Throughput: 0: 968.7. Samples: 557730. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:43:44,382][09089] Avg episode reward: [(0, '18.730')] | |
| [2025-02-21 23:43:47,986][09221] Updated weights for policy 0, policy_version 550 (0.0010) | |
| [2025-02-21 23:43:49,361][09089] Fps is (10 sec: 4096.9, 60 sec: 3891.9, 300 sec: 3957.2). Total num frames: 2256896. Throughput: 0: 980.8. Samples: 563662. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:43:49,366][09089] Avg episode reward: [(0, '20.026')] | |
| [2025-02-21 23:43:54,364][09089] Fps is (10 sec: 4097.3, 60 sec: 3890.9, 300 sec: 3957.1). Total num frames: 2277376. Throughput: 0: 983.4. Samples: 569646. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:43:54,370][09089] Avg episode reward: [(0, '21.039')] | |
| [2025-02-21 23:43:58,059][09221] Updated weights for policy 0, policy_version 560 (0.0012) | |
| [2025-02-21 23:43:59,359][09089] Fps is (10 sec: 4096.6, 60 sec: 3960.0, 300 sec: 3957.2). Total num frames: 2297856. Throughput: 0: 985.5. Samples: 572680. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:43:59,361][09089] Avg episode reward: [(0, '22.013')] | |
| [2025-02-21 23:44:04,360][09089] Fps is (10 sec: 4097.4, 60 sec: 3959.4, 300 sec: 3971.1). Total num frames: 2318336. Throughput: 0: 1006.1. Samples: 578836. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:44:04,363][09089] Avg episode reward: [(0, '22.824')] | |
| [2025-02-21 23:44:08,260][09221] Updated weights for policy 0, policy_version 570 (0.0015) | |
| [2025-02-21 23:44:09,358][09089] Fps is (10 sec: 4096.1, 60 sec: 3959.5, 300 sec: 3971.1). Total num frames: 2338816. Throughput: 0: 1010.9. Samples: 585094. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:44:09,361][09089] Avg episode reward: [(0, '23.010')] | |
| [2025-02-21 23:44:14,359][09089] Fps is (10 sec: 4096.2, 60 sec: 4027.8, 300 sec: 3971.1). Total num frames: 2359296. Throughput: 0: 1011.8. Samples: 588112. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:44:14,360][09089] Avg episode reward: [(0, '23.702')] | |
| [2025-02-21 23:44:18,163][09221] Updated weights for policy 0, policy_version 580 (0.0027) | |
| [2025-02-21 23:44:19,365][09089] Fps is (10 sec: 3684.3, 60 sec: 4028.0, 300 sec: 3957.1). Total num frames: 2375680. Throughput: 0: 1012.5. Samples: 594156. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:44:19,375][09089] Avg episode reward: [(0, '23.722')] | |
| [2025-02-21 23:44:24,361][09089] Fps is (10 sec: 3685.8, 60 sec: 3959.4, 300 sec: 3957.2). Total num frames: 2396160. Throughput: 0: 1000.5. Samples: 599746. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:44:24,364][09089] Avg episode reward: [(0, '23.178')] | |
| [2025-02-21 23:44:28,667][09221] Updated weights for policy 0, policy_version 590 (0.0011) | |
| [2025-02-21 23:44:29,365][09089] Fps is (10 sec: 4096.2, 60 sec: 4027.7, 300 sec: 3957.1). Total num frames: 2416640. Throughput: 0: 1000.9. Samples: 602770. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:44:29,376][09089] Avg episode reward: [(0, '24.540')] | |
| [2025-02-21 23:44:34,366][09089] Fps is (10 sec: 4094.3, 60 sec: 4028.4, 300 sec: 3957.1). Total num frames: 2437120. Throughput: 0: 1007.9. Samples: 609024. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:44:34,383][09089] Avg episode reward: [(0, '24.220')] | |
| [2025-02-21 23:44:38,866][09221] Updated weights for policy 0, policy_version 600 (0.0012) | |
| [2025-02-21 23:44:39,359][09089] Fps is (10 sec: 4098.0, 60 sec: 4028.0, 300 sec: 3957.2). Total num frames: 2457600. Throughput: 0: 1007.6. Samples: 614984. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:44:39,365][09089] Avg episode reward: [(0, '24.410')] | |
| [2025-02-21 23:44:44,363][09089] Fps is (10 sec: 4096.9, 60 sec: 4028.0, 300 sec: 3957.1). Total num frames: 2478080. Throughput: 0: 1007.2. Samples: 618006. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:44:44,370][09089] Avg episode reward: [(0, '23.036')] | |
| [2025-02-21 23:44:49,132][09221] Updated weights for policy 0, policy_version 610 (0.0011) | |
| [2025-02-21 23:44:49,359][09089] Fps is (10 sec: 4096.0, 60 sec: 4027.8, 300 sec: 3957.6). Total num frames: 2498560. Throughput: 0: 999.5. Samples: 623814. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:44:49,363][09089] Avg episode reward: [(0, '22.229')] | |
| [2025-02-21 23:44:54,362][09089] Fps is (10 sec: 4096.4, 60 sec: 4027.8, 300 sec: 3971.0). Total num frames: 2519040. Throughput: 0: 1004.2. Samples: 630286. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:44:54,370][09089] Avg episode reward: [(0, '20.930')] | |
| [2025-02-21 23:44:58,708][09221] Updated weights for policy 0, policy_version 620 (0.0013) | |
| [2025-02-21 23:44:59,368][09089] Fps is (10 sec: 4092.7, 60 sec: 4027.2, 300 sec: 3957.1). Total num frames: 2539520. Throughput: 0: 1007.0. Samples: 633436. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:44:59,382][09089] Avg episode reward: [(0, '21.703')] | |
| [2025-02-21 23:45:04,364][09089] Fps is (10 sec: 4095.5, 60 sec: 4027.5, 300 sec: 3957.1). Total num frames: 2560000. Throughput: 0: 1006.4. Samples: 639442. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) | |
| [2025-02-21 23:45:04,372][09089] Avg episode reward: [(0, '22.670')] | |
| [2025-02-21 23:45:08,941][09221] Updated weights for policy 0, policy_version 630 (0.0014) | |
| [2025-02-21 23:45:09,358][09089] Fps is (10 sec: 4099.4, 60 sec: 4027.7, 300 sec: 3957.2). Total num frames: 2580480. Throughput: 0: 1022.6. Samples: 645760. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:45:09,360][09089] Avg episode reward: [(0, '23.525')] | |
| [2025-02-21 23:45:14,363][09089] Fps is (10 sec: 4096.2, 60 sec: 4027.5, 300 sec: 3957.1). Total num frames: 2600960. Throughput: 0: 1024.7. Samples: 648880. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:45:14,370][09089] Avg episode reward: [(0, '23.143')] | |
| [2025-02-21 23:45:19,155][09221] Updated weights for policy 0, policy_version 640 (0.0011) | |
| [2025-02-21 23:45:19,362][09089] Fps is (10 sec: 4094.8, 60 sec: 4096.2, 300 sec: 3957.1). Total num frames: 2621440. Throughput: 0: 1016.6. Samples: 654770. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:45:19,365][09089] Avg episode reward: [(0, '22.828')] | |
| [2025-02-21 23:45:24,363][09089] Fps is (10 sec: 4096.1, 60 sec: 4095.9, 300 sec: 3957.2). Total num frames: 2641920. Throughput: 0: 1018.9. Samples: 660838. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0) | |
| [2025-02-21 23:45:24,370][09089] Avg episode reward: [(0, '23.021')] | |
| [2025-02-21 23:45:28,991][09221] Updated weights for policy 0, policy_version 650 (0.0014) | |
| [2025-02-21 23:45:29,362][09089] Fps is (10 sec: 4096.2, 60 sec: 4096.2, 300 sec: 3957.2). Total num frames: 2662400. Throughput: 0: 1020.1. Samples: 663910. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:45:29,373][09089] Avg episode reward: [(0, '21.104')] | |
| [2025-02-21 23:45:34,363][09089] Fps is (10 sec: 4095.7, 60 sec: 4096.1, 300 sec: 3957.1). Total num frames: 2682880. Throughput: 0: 1029.7. Samples: 670154. Policy #0 lag: (min: 0.0, avg: 0.6, max: 3.0) | |
| [2025-02-21 23:45:34,369][09089] Avg episode reward: [(0, '21.414')] | |
| [2025-02-21 23:45:34,396][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000655_2682880.pth... | |
| [2025-02-21 23:45:35,114][09144] Removing /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000421_1724416.pth | |
| [2025-02-21 23:45:39,025][09221] Updated weights for policy 0, policy_version 660 (0.0011) | |
| [2025-02-21 23:45:39,362][09089] Fps is (10 sec: 4095.7, 60 sec: 4095.8, 300 sec: 3971.1). Total num frames: 2703360. Throughput: 0: 1022.0. Samples: 676276. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:45:39,369][09089] Avg episode reward: [(0, '23.129')] | |
| [2025-02-21 23:45:44,363][09089] Fps is (10 sec: 4096.1, 60 sec: 4096.0, 300 sec: 3971.0). Total num frames: 2723840. Throughput: 0: 1022.1. Samples: 679424. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:45:44,369][09089] Avg episode reward: [(0, '23.125')] | |
| [2025-02-21 23:45:48,992][09221] Updated weights for policy 0, policy_version 670 (0.0012) | |
| [2025-02-21 23:45:49,366][09089] Fps is (10 sec: 4094.5, 60 sec: 4095.6, 300 sec: 3971.0). Total num frames: 2744320. Throughput: 0: 1025.7. Samples: 685600. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:45:49,381][09089] Avg episode reward: [(0, '22.648')] | |
| [2025-02-21 23:45:54,359][09089] Fps is (10 sec: 4097.5, 60 sec: 4096.2, 300 sec: 3971.1). Total num frames: 2764800. Throughput: 0: 1020.2. Samples: 691668. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:45:54,361][09089] Avg episode reward: [(0, '23.695')] | |
| [2025-02-21 23:45:58,706][09221] Updated weights for policy 0, policy_version 680 (0.0012) | |
| [2025-02-21 23:45:59,362][09089] Fps is (10 sec: 4097.2, 60 sec: 4096.3, 300 sec: 3971.1). Total num frames: 2785280. Throughput: 0: 1020.7. Samples: 694810. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:45:59,369][09089] Avg episode reward: [(0, '22.734')] | |
| [2025-02-21 23:46:04,360][09089] Fps is (10 sec: 4095.7, 60 sec: 4096.2, 300 sec: 3985.0). Total num frames: 2805760. Throughput: 0: 1022.5. Samples: 700782. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:46:04,363][09089] Avg episode reward: [(0, '22.546')] | |
| [2025-02-21 23:46:09,360][09089] Fps is (10 sec: 3687.3, 60 sec: 4027.6, 300 sec: 3971.0). Total num frames: 2822144. Throughput: 0: 1016.3. Samples: 706570. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:46:09,364][09089] Avg episode reward: [(0, '23.811')] | |
| [2025-02-21 23:46:09,384][09221] Updated weights for policy 0, policy_version 690 (0.0013) | |
| [2025-02-21 23:46:14,363][09089] Fps is (10 sec: 3685.7, 60 sec: 4027.8, 300 sec: 3984.9). Total num frames: 2842624. Throughput: 0: 1011.9. Samples: 709444. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:46:14,374][09089] Avg episode reward: [(0, '25.638')] | |
| [2025-02-21 23:46:14,404][09144] Saving new best policy, reward=25.638! | |
| [2025-02-21 23:46:19,359][09089] Fps is (10 sec: 4096.4, 60 sec: 4027.9, 300 sec: 3985.0). Total num frames: 2863104. Throughput: 0: 1004.7. Samples: 715360. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0) | |
| [2025-02-21 23:46:19,361][09089] Avg episode reward: [(0, '25.260')] | |
| [2025-02-21 23:46:19,944][09221] Updated weights for policy 0, policy_version 700 (0.0012) | |
| [2025-02-21 23:46:24,361][09089] Fps is (10 sec: 4096.3, 60 sec: 4027.8, 300 sec: 3998.8). Total num frames: 2883584. Throughput: 0: 1001.8. Samples: 721358. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:46:24,364][09089] Avg episode reward: [(0, '25.017')] | |
| [2025-02-21 23:46:29,360][09089] Fps is (10 sec: 4095.6, 60 sec: 4027.8, 300 sec: 3999.0). Total num frames: 2904064. Throughput: 0: 997.9. Samples: 724328. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:46:29,362][09089] Avg episode reward: [(0, '24.846')] | |
| [2025-02-21 23:46:30,054][09221] Updated weights for policy 0, policy_version 710 (0.0011) | |
| [2025-02-21 23:46:34,359][09089] Fps is (10 sec: 4096.7, 60 sec: 4028.0, 300 sec: 3998.8). Total num frames: 2924544. Throughput: 0: 998.6. Samples: 730530. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:46:34,361][09089] Avg episode reward: [(0, '22.410')] | |
| [2025-02-21 23:46:39,363][09089] Fps is (10 sec: 4095.2, 60 sec: 4027.7, 300 sec: 3998.8). Total num frames: 2945024. Throughput: 0: 1003.8. Samples: 736844. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:46:39,369][09089] Avg episode reward: [(0, '21.388')] | |
| [2025-02-21 23:46:39,948][09221] Updated weights for policy 0, policy_version 720 (0.0010) | |
| [2025-02-21 23:46:44,363][09089] Fps is (10 sec: 4094.9, 60 sec: 4027.8, 300 sec: 4012.7). Total num frames: 2965504. Throughput: 0: 1004.0. Samples: 739988. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:46:44,372][09089] Avg episode reward: [(0, '21.363')] | |
| [2025-02-21 23:46:49,362][09089] Fps is (10 sec: 4096.4, 60 sec: 4028.0, 300 sec: 4012.7). Total num frames: 2985984. Throughput: 0: 1004.6. Samples: 745990. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:46:49,368][09089] Avg episode reward: [(0, '22.426')] | |
| [2025-02-21 23:46:49,912][09221] Updated weights for policy 0, policy_version 730 (0.0016) | |
| [2025-02-21 23:46:54,359][09089] Fps is (10 sec: 4097.1, 60 sec: 4027.7, 300 sec: 4012.7). Total num frames: 3006464. Throughput: 0: 1007.5. Samples: 751906. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:46:54,361][09089] Avg episode reward: [(0, '22.966')] | |
| [2025-02-21 23:46:59,360][09089] Fps is (10 sec: 3686.9, 60 sec: 3959.7, 300 sec: 4012.7). Total num frames: 3022848. Throughput: 0: 1010.1. Samples: 754898. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:46:59,365][09089] Avg episode reward: [(0, '24.591')] | |
| [2025-02-21 23:47:00,257][09221] Updated weights for policy 0, policy_version 740 (0.0012) | |
| [2025-02-21 23:47:04,365][09089] Fps is (10 sec: 3684.6, 60 sec: 3959.2, 300 sec: 3998.8). Total num frames: 3043328. Throughput: 0: 1010.9. Samples: 760856. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:47:04,370][09089] Avg episode reward: [(0, '24.284')] | |
| [2025-02-21 23:47:09,365][09089] Fps is (10 sec: 4094.3, 60 sec: 4027.5, 300 sec: 3998.8). Total num frames: 3063808. Throughput: 0: 1007.8. Samples: 766712. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:47:09,372][09089] Avg episode reward: [(0, '24.271')] | |
| [2025-02-21 23:47:10,751][09221] Updated weights for policy 0, policy_version 750 (0.0030) | |
| [2025-02-21 23:47:14,366][09089] Fps is (10 sec: 4095.6, 60 sec: 4027.5, 300 sec: 4012.7). Total num frames: 3084288. Throughput: 0: 1009.4. Samples: 769754. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:47:14,375][09089] Avg episode reward: [(0, '24.504')] | |
| [2025-02-21 23:47:19,365][09089] Fps is (10 sec: 4096.3, 60 sec: 4027.5, 300 sec: 3998.8). Total num frames: 3104768. Throughput: 0: 1008.3. Samples: 775906. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:47:19,373][09089] Avg episode reward: [(0, '24.691')] | |
| [2025-02-21 23:47:20,797][09221] Updated weights for policy 0, policy_version 760 (0.0011) | |
| [2025-02-21 23:47:24,370][09089] Fps is (10 sec: 4094.5, 60 sec: 4027.2, 300 sec: 3998.7). Total num frames: 3125248. Throughput: 0: 1002.6. Samples: 781968. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:47:24,382][09089] Avg episode reward: [(0, '25.479')] | |
| [2025-02-21 23:47:29,360][09089] Fps is (10 sec: 4097.4, 60 sec: 4027.8, 300 sec: 3998.8). Total num frames: 3145728. Throughput: 0: 998.4. Samples: 784914. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:47:29,363][09089] Avg episode reward: [(0, '26.157')] | |
| [2025-02-21 23:47:29,368][09144] Saving new best policy, reward=26.157! | |
| [2025-02-21 23:47:30,769][09221] Updated weights for policy 0, policy_version 770 (0.0012) | |
| [2025-02-21 23:47:34,359][09089] Fps is (10 sec: 4100.0, 60 sec: 4027.7, 300 sec: 3998.8). Total num frames: 3166208. Throughput: 0: 1001.7. Samples: 791066. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0) | |
| [2025-02-21 23:47:34,361][09089] Avg episode reward: [(0, '24.682')] | |
| [2025-02-21 23:47:34,370][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000773_3166208.pth... | |
| [2025-02-21 23:47:34,950][09144] Removing /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000537_2199552.pth | |
| [2025-02-21 23:47:39,358][09089] Fps is (10 sec: 4096.3, 60 sec: 4027.9, 300 sec: 4012.7). Total num frames: 3186688. Throughput: 0: 1009.7. Samples: 797344. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:47:39,361][09089] Avg episode reward: [(0, '24.843')] | |
| [2025-02-21 23:47:40,545][09221] Updated weights for policy 0, policy_version 780 (0.0011) | |
| [2025-02-21 23:47:44,364][09089] Fps is (10 sec: 4094.6, 60 sec: 4027.7, 300 sec: 4012.8). Total num frames: 3207168. Throughput: 0: 1012.8. Samples: 800476. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:47:44,372][09089] Avg episode reward: [(0, '25.508')] | |
| [2025-02-21 23:47:49,362][09089] Fps is (10 sec: 4504.6, 60 sec: 4096.0, 300 sec: 4026.5). Total num frames: 3231744. Throughput: 0: 1025.0. Samples: 806980. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:47:49,365][09089] Avg episode reward: [(0, '24.163')] | |
| [2025-02-21 23:47:50,194][09221] Updated weights for policy 0, policy_version 790 (0.0012) | |
| [2025-02-21 23:47:54,359][09089] Fps is (10 sec: 4507.1, 60 sec: 4096.0, 300 sec: 4040.6). Total num frames: 3252224. Throughput: 0: 1040.5. Samples: 813528. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:47:54,361][09089] Avg episode reward: [(0, '24.068')] | |
| [2025-02-21 23:47:59,364][09089] Fps is (10 sec: 4095.3, 60 sec: 4164.0, 300 sec: 4040.4). Total num frames: 3272704. Throughput: 0: 1046.4. Samples: 816840. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:47:59,372][09089] Avg episode reward: [(0, '25.311')] | |
| [2025-02-21 23:47:59,595][09221] Updated weights for policy 0, policy_version 800 (0.0012) | |
| [2025-02-21 23:48:04,368][09089] Fps is (10 sec: 4093.0, 60 sec: 4164.1, 300 sec: 4040.4). Total num frames: 3293184. Throughput: 0: 1046.7. Samples: 823012. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:48:04,387][09089] Avg episode reward: [(0, '25.604')] | |
| [2025-02-21 23:48:09,370][09089] Fps is (10 sec: 4094.4, 60 sec: 4164.0, 300 sec: 4054.3). Total num frames: 3313664. Throughput: 0: 1046.0. Samples: 829038. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:48:09,393][09089] Avg episode reward: [(0, '26.591')] | |
| [2025-02-21 23:48:09,408][09144] Saving new best policy, reward=26.591! | |
| [2025-02-21 23:48:10,485][09221] Updated weights for policy 0, policy_version 810 (0.0012) | |
| [2025-02-21 23:48:14,359][09089] Fps is (10 sec: 4099.0, 60 sec: 4164.7, 300 sec: 4068.4). Total num frames: 3334144. Throughput: 0: 1046.3. Samples: 831998. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:48:14,361][09089] Avg episode reward: [(0, '27.619')] | |
| [2025-02-21 23:48:14,381][09144] Saving new best policy, reward=27.619! | |
| [2025-02-21 23:48:19,362][09089] Fps is (10 sec: 4098.1, 60 sec: 4164.3, 300 sec: 4054.3). Total num frames: 3354624. Throughput: 0: 1050.0. Samples: 838320. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:48:19,370][09089] Avg episode reward: [(0, '28.302')] | |
| [2025-02-21 23:48:19,379][09144] Saving new best policy, reward=28.302! | |
| [2025-02-21 23:48:20,904][09221] Updated weights for policy 0, policy_version 820 (0.0012) | |
| [2025-02-21 23:48:24,359][09089] Fps is (10 sec: 4095.8, 60 sec: 4164.9, 300 sec: 4068.3). Total num frames: 3375104. Throughput: 0: 1045.6. Samples: 844398. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:48:24,362][09089] Avg episode reward: [(0, '27.462')] | |
| [2025-02-21 23:48:29,364][09089] Fps is (10 sec: 4095.9, 60 sec: 4164.1, 300 sec: 4068.4). Total num frames: 3395584. Throughput: 0: 1045.7. Samples: 847532. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:48:29,370][09089] Avg episode reward: [(0, '26.865')] | |
| [2025-02-21 23:48:29,806][09221] Updated weights for policy 0, policy_version 830 (0.0012) | |
| [2025-02-21 23:48:34,364][09089] Fps is (10 sec: 4094.2, 60 sec: 4163.9, 300 sec: 4068.2). Total num frames: 3416064. Throughput: 0: 1037.9. Samples: 853688. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:48:34,370][09089] Avg episode reward: [(0, '26.236')] | |
| [2025-02-21 23:48:39,359][09089] Fps is (10 sec: 4097.2, 60 sec: 4164.2, 300 sec: 4068.3). Total num frames: 3436544. Throughput: 0: 1033.0. Samples: 860012. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:48:39,362][09089] Avg episode reward: [(0, '25.000')] | |
| [2025-02-21 23:48:39,489][09221] Updated weights for policy 0, policy_version 840 (0.0011) | |
| [2025-02-21 23:48:44,365][09089] Fps is (10 sec: 4096.0, 60 sec: 4164.2, 300 sec: 4068.2). Total num frames: 3457024. Throughput: 0: 1024.6. Samples: 862946. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:48:44,374][09089] Avg episode reward: [(0, '24.275')] | |
| [2025-02-21 23:48:49,368][09089] Fps is (10 sec: 4093.6, 60 sec: 4095.7, 300 sec: 4068.2). Total num frames: 3477504. Throughput: 0: 1022.4. Samples: 869018. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:48:49,384][09089] Avg episode reward: [(0, '24.077')] | |
| [2025-02-21 23:48:49,613][09221] Updated weights for policy 0, policy_version 850 (0.0011) | |
| [2025-02-21 23:48:54,363][09089] Fps is (10 sec: 4096.5, 60 sec: 4095.8, 300 sec: 4068.2). Total num frames: 3497984. Throughput: 0: 1026.8. Samples: 875240. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0) | |
| [2025-02-21 23:48:54,370][09089] Avg episode reward: [(0, '24.014')] | |
| [2025-02-21 23:48:59,370][09089] Fps is (10 sec: 4094.6, 60 sec: 4095.6, 300 sec: 4068.1). Total num frames: 3518464. Throughput: 0: 1032.3. Samples: 878462. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:48:59,378][09089] Avg episode reward: [(0, '25.307')] | |
| [2025-02-21 23:48:59,600][09221] Updated weights for policy 0, policy_version 860 (0.0011) | |
| [2025-02-21 23:49:04,363][09089] Fps is (10 sec: 4095.9, 60 sec: 4096.2, 300 sec: 4068.2). Total num frames: 3538944. Throughput: 0: 1028.5. Samples: 884602. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:49:04,370][09089] Avg episode reward: [(0, '23.918')] | |
| [2025-02-21 23:49:09,359][09089] Fps is (10 sec: 4099.9, 60 sec: 4096.5, 300 sec: 4068.2). Total num frames: 3559424. Throughput: 0: 1029.5. Samples: 890726. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:49:09,360][09089] Avg episode reward: [(0, '24.007')] | |
| [2025-02-21 23:49:09,507][09221] Updated weights for policy 0, policy_version 870 (0.0012) | |
| [2025-02-21 23:49:14,360][09089] Fps is (10 sec: 4097.2, 60 sec: 4095.9, 300 sec: 4082.2). Total num frames: 3579904. Throughput: 0: 1027.8. Samples: 893780. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0) | |
| [2025-02-21 23:49:14,363][09089] Avg episode reward: [(0, '24.433')] | |
| [2025-02-21 23:49:19,361][09089] Fps is (10 sec: 4095.1, 60 sec: 4096.0, 300 sec: 4082.1). Total num frames: 3600384. Throughput: 0: 1028.6. Samples: 899972. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:49:19,366][09089] Avg episode reward: [(0, '25.447')] | |
| [2025-02-21 23:49:19,482][09221] Updated weights for policy 0, policy_version 880 (0.0010) | |
| [2025-02-21 23:49:24,363][09089] Fps is (10 sec: 4095.1, 60 sec: 4095.8, 300 sec: 4082.1). Total num frames: 3620864. Throughput: 0: 1022.1. Samples: 906008. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:49:24,370][09089] Avg episode reward: [(0, '25.227')] | |
| [2025-02-21 23:49:29,361][09089] Fps is (10 sec: 4096.2, 60 sec: 4096.1, 300 sec: 4082.2). Total num frames: 3641344. Throughput: 0: 1026.1. Samples: 909118. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:49:29,370][09089] Avg episode reward: [(0, '25.091')] | |
| [2025-02-21 23:49:29,585][09221] Updated weights for policy 0, policy_version 890 (0.0011) | |
| [2025-02-21 23:49:34,378][09089] Fps is (10 sec: 4090.6, 60 sec: 4095.2, 300 sec: 4081.9). Total num frames: 3661824. Throughput: 0: 1028.5. Samples: 915310. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:49:34,400][09089] Avg episode reward: [(0, '25.438')] | |
| [2025-02-21 23:49:34,438][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000894_3661824.pth... | |
| [2025-02-21 23:49:34,862][09144] Removing /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000655_2682880.pth | |
| [2025-02-21 23:49:39,367][09089] Fps is (10 sec: 4094.0, 60 sec: 4095.6, 300 sec: 4082.1). Total num frames: 3682304. Throughput: 0: 1025.0. Samples: 921366. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:49:39,381][09089] Avg episode reward: [(0, '24.803')] | |
| [2025-02-21 23:49:39,625][09221] Updated weights for policy 0, policy_version 900 (0.0011) | |
| [2025-02-21 23:49:44,365][09089] Fps is (10 sec: 4100.8, 60 sec: 4096.0, 300 sec: 4082.1). Total num frames: 3702784. Throughput: 0: 1018.7. Samples: 924298. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:49:44,371][09089] Avg episode reward: [(0, '26.769')] | |
| [2025-02-21 23:49:49,358][09089] Fps is (10 sec: 4098.8, 60 sec: 4096.4, 300 sec: 4082.2). Total num frames: 3723264. Throughput: 0: 1018.2. Samples: 930418. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:49:49,360][09089] Avg episode reward: [(0, '26.870')] | |
| [2025-02-21 23:49:49,819][09221] Updated weights for policy 0, policy_version 910 (0.0011) | |
| [2025-02-21 23:49:54,359][09089] Fps is (10 sec: 4098.0, 60 sec: 4096.3, 300 sec: 4082.2). Total num frames: 3743744. Throughput: 0: 1014.0. Samples: 936356. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:49:54,360][09089] Avg episode reward: [(0, '25.790')] | |
| [2025-02-21 23:49:59,359][09089] Fps is (10 sec: 4095.9, 60 sec: 4096.6, 300 sec: 4082.2). Total num frames: 3764224. Throughput: 0: 1013.2. Samples: 939374. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:49:59,361][09089] Avg episode reward: [(0, '25.416')] | |
| [2025-02-21 23:50:00,518][09221] Updated weights for policy 0, policy_version 920 (0.0019) | |
| [2025-02-21 23:50:04,361][09089] Fps is (10 sec: 4095.2, 60 sec: 4096.1, 300 sec: 4082.1). Total num frames: 3784704. Throughput: 0: 1006.5. Samples: 945266. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:50:04,365][09089] Avg episode reward: [(0, '25.588')] | |
| [2025-02-21 23:50:09,359][09089] Fps is (10 sec: 3686.4, 60 sec: 4027.7, 300 sec: 4068.3). Total num frames: 3801088. Throughput: 0: 1012.1. Samples: 951550. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:50:09,361][09089] Avg episode reward: [(0, '23.933')] | |
| [2025-02-21 23:50:10,166][09221] Updated weights for policy 0, policy_version 930 (0.0013) | |
| [2025-02-21 23:50:14,362][09089] Fps is (10 sec: 4095.5, 60 sec: 4095.9, 300 sec: 4082.1). Total num frames: 3825664. Throughput: 0: 1009.9. Samples: 954566. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:50:14,368][09089] Avg episode reward: [(0, '25.025')] | |
| [2025-02-21 23:50:19,362][09089] Fps is (10 sec: 4504.7, 60 sec: 4096.0, 300 sec: 4082.1). Total num frames: 3846144. Throughput: 0: 1014.6. Samples: 960952. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:50:19,368][09089] Avg episode reward: [(0, '26.400')] | |
| [2025-02-21 23:50:19,880][09221] Updated weights for policy 0, policy_version 940 (0.0012) | |
| [2025-02-21 23:50:24,365][09089] Fps is (10 sec: 4095.3, 60 sec: 4095.9, 300 sec: 4082.1). Total num frames: 3866624. Throughput: 0: 1024.6. Samples: 967472. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:50:24,382][09089] Avg episode reward: [(0, '27.890')] | |
| [2025-02-21 23:50:29,363][09089] Fps is (10 sec: 4095.1, 60 sec: 4095.8, 300 sec: 4082.1). Total num frames: 3887104. Throughput: 0: 1028.3. Samples: 970570. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0) | |
| [2025-02-21 23:50:29,368][09089] Avg episode reward: [(0, '28.699')] | |
| [2025-02-21 23:50:29,422][09221] Updated weights for policy 0, policy_version 950 (0.0012) | |
| [2025-02-21 23:50:29,421][09144] Saving new best policy, reward=28.699! | |
| [2025-02-21 23:50:34,366][09089] Fps is (10 sec: 4095.5, 60 sec: 4096.7, 300 sec: 4082.1). Total num frames: 3907584. Throughput: 0: 1024.5. Samples: 976528. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0) | |
| [2025-02-21 23:50:34,371][09089] Avg episode reward: [(0, '29.349')] | |
| [2025-02-21 23:50:34,444][09144] Saving new best policy, reward=29.349! | |
| [2025-02-21 23:50:39,362][09089] Fps is (10 sec: 4096.7, 60 sec: 4096.3, 300 sec: 4082.1). Total num frames: 3928064. Throughput: 0: 1025.6. Samples: 982510. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:50:39,368][09089] Avg episode reward: [(0, '29.527')] | |
| [2025-02-21 23:50:39,373][09144] Saving new best policy, reward=29.527! | |
| [2025-02-21 23:50:40,708][09221] Updated weights for policy 0, policy_version 960 (0.0019) | |
| [2025-02-21 23:50:44,363][09089] Fps is (10 sec: 4096.9, 60 sec: 4096.1, 300 sec: 4082.2). Total num frames: 3948544. Throughput: 0: 1022.1. Samples: 985374. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0) | |
| [2025-02-21 23:50:44,370][09089] Avg episode reward: [(0, '28.423')] | |
| [2025-02-21 23:50:49,363][09089] Fps is (10 sec: 4095.4, 60 sec: 4095.7, 300 sec: 4082.1). Total num frames: 3969024. Throughput: 0: 1034.4. Samples: 991818. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0) | |
| [2025-02-21 23:50:49,373][09089] Avg episode reward: [(0, '28.184')] | |
| [2025-02-21 23:50:49,545][09221] Updated weights for policy 0, policy_version 970 (0.0014) | |
| [2025-02-21 23:50:54,362][09089] Fps is (10 sec: 4096.4, 60 sec: 4095.8, 300 sec: 4082.1). Total num frames: 3989504. Throughput: 0: 1037.9. Samples: 998258. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0) | |
| [2025-02-21 23:50:54,369][09089] Avg episode reward: [(0, '27.794')] | |
| [2025-02-21 23:50:57,764][09144] Stopping Batcher_0... | |
| [2025-02-21 23:50:57,769][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-02-21 23:50:57,767][09089] Component Batcher_0 stopped! | |
| [2025-02-21 23:50:57,776][09089] Component RolloutWorker_w5 process died already! Don't wait for it. | |
| [2025-02-21 23:50:57,772][09144] Loop batcher_evt_loop terminating... | |
| [2025-02-21 23:50:57,949][09223] Stopping RolloutWorker_w6... | |
| [2025-02-21 23:50:57,954][09223] Loop rollout_proc6_evt_loop terminating... | |
| [2025-02-21 23:50:57,953][09089] Component RolloutWorker_w6 stopped! | |
| [2025-02-21 23:50:58,022][09221] Weights refcount: 2 0 | |
| [2025-02-21 23:50:58,034][09221] Stopping InferenceWorker_p0-w0... | |
| [2025-02-21 23:50:58,037][09221] Loop inference_proc0-0_evt_loop terminating... | |
| [2025-02-21 23:50:58,041][09089] Component InferenceWorker_p0-w0 stopped! | |
| [2025-02-21 23:50:58,334][09144] Removing /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000773_3166208.pth | |
| [2025-02-21 23:50:58,356][09144] Saving /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-02-21 23:50:58,717][09217] Stopping RolloutWorker_w0... | |
| [2025-02-21 23:50:58,724][09089] Component RolloutWorker_w0 stopped! | |
| [2025-02-21 23:50:58,736][09217] Loop rollout_proc0_evt_loop terminating... | |
| [2025-02-21 23:50:58,866][09089] Component RolloutWorker_w1 stopped! | |
| [2025-02-21 23:50:58,865][09219] Stopping RolloutWorker_w1... | |
| [2025-02-21 23:50:58,876][09219] Loop rollout_proc1_evt_loop terminating... | |
| [2025-02-21 23:50:58,880][09089] Component RolloutWorker_w4 stopped! | |
| [2025-02-21 23:50:58,883][09089] Component RolloutWorker_w2 stopped! | |
| [2025-02-21 23:50:58,884][09089] Component RolloutWorker_w3 stopped! | |
| [2025-02-21 23:50:58,880][09226] Stopping RolloutWorker_w4... | |
| [2025-02-21 23:50:58,880][09218] Stopping RolloutWorker_w2... | |
| [2025-02-21 23:50:58,881][09224] Stopping RolloutWorker_w3... | |
| [2025-02-21 23:50:58,890][09226] Loop rollout_proc4_evt_loop terminating... | |
| [2025-02-21 23:50:58,892][09218] Loop rollout_proc2_evt_loop terminating... | |
| [2025-02-21 23:50:58,893][09224] Loop rollout_proc3_evt_loop terminating... | |
| [2025-02-21 23:50:58,893][09089] Component RolloutWorker_w7 stopped! | |
| [2025-02-21 23:50:58,893][09227] Stopping RolloutWorker_w7... | |
| [2025-02-21 23:50:58,905][09227] Loop rollout_proc7_evt_loop terminating... | |
| [2025-02-21 23:50:58,929][09144] Stopping LearnerWorker_p0... | |
| [2025-02-21 23:50:58,932][09089] Component LearnerWorker_p0 stopped! | |
| [2025-02-21 23:50:58,934][09144] Loop learner_proc0_evt_loop terminating... | |
| [2025-02-21 23:50:58,935][09089] Waiting for process learner_proc0 to stop... | |
| [2025-02-21 23:51:09,974][09089] Waiting for process inference_proc0-0 to join... | |
| [2025-02-21 23:51:09,980][09089] Waiting for process rollout_proc0 to join... | |
| [2025-02-21 23:51:09,982][09089] Waiting for process rollout_proc1 to join... | |
| [2025-02-21 23:51:12,473][09089] Waiting for process rollout_proc2 to join... | |
| [2025-02-21 23:51:12,517][09089] Waiting for process rollout_proc3 to join... | |
| [2025-02-21 23:51:12,688][09089] Waiting for process rollout_proc4 to join... | |
| [2025-02-21 23:51:12,693][09089] Waiting for process rollout_proc5 to join... | |
| [2025-02-21 23:51:12,695][09089] Waiting for process rollout_proc6 to join... | |
| [2025-02-21 23:51:12,697][09089] Waiting for process rollout_proc7 to join... | |
| [2025-02-21 23:51:12,811][09089] Batcher 0 profile tree view: | |
| batching: 289.4114, releasing_batches: 0.2914 | |
| [2025-02-21 23:51:12,816][09089] InferenceWorker_p0-w0 profile tree view: | |
| wait_policy: 0.0000 | |
| wait_policy_total: 311.1377 | |
| update_model: 7.7092 | |
| weight_update: 0.0012 | |
| one_step: 0.0020 | |
| handle_policy_step: 645.3836 | |
| deserialize: 90.5514, stack: 3.3978, obs_to_device_normalize: 126.9975, forward: 299.7272, send_messages: 25.9621 | |
| prepare_outputs: 76.1270 | |
| to_cpu: 39.2102 | |
| [2025-02-21 23:51:12,821][09089] Learner 0 profile tree view: | |
| misc: 0.0784, prepare_batch: 48.9298 | |
| train: 112.8418 | |
| epoch_init: 0.0456, minibatch_init: 0.0591, losses_postprocess: 0.9255, kl_divergence: 1.2063, after_optimizer: 7.5373 | |
| calculate_losses: 33.4497 | |
| losses_init: 0.0332, forward_head: 4.0769, bptt_initial: 11.0658, tail: 3.9279, advantages_returns: 0.9545, losses: 5.6210 | |
| bptt: 6.8021 | |
| bptt_forward_core: 6.5459 | |
| update: 66.4915 | |
| clip: 6.7269 | |
| [2025-02-21 23:51:12,824][09089] RolloutWorker_w0 profile tree view: | |
| wait_for_trajectories: 1.2503, enqueue_policy_requests: 77.0597, env_step: 752.7879, overhead: 28.9796, complete_rollouts: 1.6788 | |
| save_policy_outputs: 52.6730 | |
| split_output_tensors: 17.2437 | |
| [2025-02-21 23:51:12,827][09089] RolloutWorker_w7 profile tree view: | |
| wait_for_trajectories: 1.2557, enqueue_policy_requests: 76.1079, env_step: 755.9259, overhead: 28.0805, complete_rollouts: 1.6689 | |
| save_policy_outputs: 51.2851 | |
| split_output_tensors: 17.6030 | |
| [2025-02-21 23:51:12,834][09089] Loop Runner_EvtLoop terminating... | |
| [2025-02-21 23:51:12,840][09089] Runner profile tree view: | |
| main_loop: 1049.2047 | |
| [2025-02-21 23:51:12,843][09089] Collected {0: 4005888}, FPS: 3818.0 | |
| [2025-02-21 23:51:15,882][09089] Loading existing experiment configuration from /root/autodl-tmp/train_dir/default_experiment/config.json | |
| [2025-02-21 23:51:15,883][09089] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2025-02-21 23:51:15,883][09089] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'push_to_hub'=False that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'hf_repository'=None that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2025-02-21 23:51:15,884][09089] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2025-02-21 23:51:15,885][09089] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2025-02-21 23:51:15,885][09089] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2025-02-21 23:51:15,916][09089] Doom resolution: 160x120, resize resolution: (128, 72) | |
| [2025-02-21 23:51:15,920][09089] RunningMeanStd input shape: (3, 72, 128) | |
| [2025-02-21 23:51:15,921][09089] RunningMeanStd input shape: (1,) | |
| [2025-02-21 23:51:15,946][09089] ConvEncoder: input_channels=3 | |
| [2025-02-21 23:51:16,071][09089] Conv encoder output size: 512 | |
| [2025-02-21 23:51:16,073][09089] Policy head output size: 512 | |
| [2025-02-21 23:51:16,314][09089] Loading state from checkpoint /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-02-21 23:51:17,440][09089] Num frames 100... | |
| [2025-02-21 23:51:19,033][09089] Num frames 200... | |
| [2025-02-21 23:51:20,855][09089] Num frames 300... | |
| [2025-02-21 23:51:22,696][09089] Num frames 400... | |
| [2025-02-21 23:51:25,018][09089] Num frames 500... | |
| [2025-02-21 23:51:25,906][09089] Num frames 600... | |
| [2025-02-21 23:51:26,078][09089] Avg episode rewards: #0: 11.720, true rewards: #0: 6.720 | |
| [2025-02-21 23:51:26,079][09089] Avg episode reward: 11.720, avg true_objective: 6.720 | |
| [2025-02-21 23:51:26,130][09089] Num frames 700... | |
| [2025-02-21 23:51:26,296][09089] Num frames 800... | |
| [2025-02-21 23:51:26,461][09089] Num frames 900... | |
| [2025-02-21 23:51:26,616][09089] Num frames 1000... | |
| [2025-02-21 23:51:26,771][09089] Num frames 1100... | |
| [2025-02-21 23:51:26,927][09089] Num frames 1200... | |
| [2025-02-21 23:51:27,079][09089] Num frames 1300... | |
| [2025-02-21 23:51:27,238][09089] Num frames 1400... | |
| [2025-02-21 23:51:27,395][09089] Num frames 1500... | |
| [2025-02-21 23:51:27,555][09089] Num frames 1600... | |
| [2025-02-21 23:51:27,713][09089] Num frames 1700... | |
| [2025-02-21 23:51:27,800][09089] Avg episode rewards: #0: 18.095, true rewards: #0: 8.595 | |
| [2025-02-21 23:51:27,801][09089] Avg episode reward: 18.095, avg true_objective: 8.595 | |
| [2025-02-21 23:51:27,945][09089] Num frames 1800... | |
| [2025-02-21 23:51:28,101][09089] Num frames 1900... | |
| [2025-02-21 23:51:28,255][09089] Num frames 2000... | |
| [2025-02-21 23:51:28,408][09089] Num frames 2100... | |
| [2025-02-21 23:51:28,563][09089] Num frames 2200... | |
| [2025-02-21 23:51:28,718][09089] Num frames 2300... | |
| [2025-02-21 23:51:28,916][09089] Avg episode rewards: #0: 16.970, true rewards: #0: 7.970 | |
| [2025-02-21 23:51:28,917][09089] Avg episode reward: 16.970, avg true_objective: 7.970 | |
| [2025-02-21 23:51:28,936][09089] Num frames 2400... | |
| [2025-02-21 23:51:29,089][09089] Num frames 2500... | |
| [2025-02-21 23:51:29,243][09089] Num frames 2600... | |
| [2025-02-21 23:51:29,401][09089] Num frames 2700... | |
| [2025-02-21 23:51:29,566][09089] Num frames 2800... | |
| [2025-02-21 23:51:29,727][09089] Num frames 2900... | |
| [2025-02-21 23:51:29,882][09089] Num frames 3000... | |
| [2025-02-21 23:51:30,038][09089] Num frames 3100... | |
| [2025-02-21 23:51:30,199][09089] Num frames 3200... | |
| [2025-02-21 23:51:30,356][09089] Num frames 3300... | |
| [2025-02-21 23:51:30,511][09089] Num frames 3400... | |
| [2025-02-21 23:51:30,670][09089] Num frames 3500... | |
| [2025-02-21 23:51:30,827][09089] Num frames 3600... | |
| [2025-02-21 23:51:30,997][09089] Num frames 3700... | |
| [2025-02-21 23:51:31,160][09089] Num frames 3800... | |
| [2025-02-21 23:51:31,317][09089] Num frames 3900... | |
| [2025-02-21 23:51:31,473][09089] Num frames 4000... | |
| [2025-02-21 23:51:31,627][09089] Num frames 4100... | |
| [2025-02-21 23:51:31,785][09089] Num frames 4200... | |
| [2025-02-21 23:51:31,961][09089] Num frames 4300... | |
| [2025-02-21 23:51:32,098][09089] Num frames 4400... | |
| [2025-02-21 23:51:32,296][09089] Avg episode rewards: #0: 27.227, true rewards: #0: 11.228 | |
| [2025-02-21 23:51:32,297][09089] Avg episode reward: 27.227, avg true_objective: 11.228 | |
| [2025-02-21 23:51:32,320][09089] Num frames 4500... | |
| [2025-02-21 23:51:32,472][09089] Num frames 4600... | |
| [2025-02-21 23:51:32,634][09089] Num frames 4700... | |
| [2025-02-21 23:51:32,793][09089] Num frames 4800... | |
| [2025-02-21 23:51:32,950][09089] Num frames 4900... | |
| [2025-02-21 23:51:33,107][09089] Num frames 5000... | |
| [2025-02-21 23:51:33,263][09089] Num frames 5100... | |
| [2025-02-21 23:51:33,418][09089] Num frames 5200... | |
| [2025-02-21 23:51:34,184][09089] Num frames 5300... | |
| [2025-02-21 23:51:35,560][09089] Num frames 5400... | |
| [2025-02-21 23:51:36,100][09089] Num frames 5500... | |
| [2025-02-21 23:51:36,268][09089] Num frames 5600... | |
| [2025-02-21 23:51:36,437][09089] Num frames 5700... | |
| [2025-02-21 23:51:36,608][09089] Num frames 5800... | |
| [2025-02-21 23:51:36,831][09089] Avg episode rewards: #0: 28.598, true rewards: #0: 11.798 | |
| [2025-02-21 23:51:36,833][09089] Avg episode reward: 28.598, avg true_objective: 11.798 | |
| [2025-02-21 23:51:36,837][09089] Num frames 5900... | |
| [2025-02-21 23:51:37,007][09089] Num frames 6000... | |
| [2025-02-21 23:51:37,184][09089] Num frames 6100... | |
| [2025-02-21 23:51:37,334][09089] Num frames 6200... | |
| [2025-02-21 23:51:37,489][09089] Num frames 6300... | |
| [2025-02-21 23:51:37,644][09089] Num frames 6400... | |
| [2025-02-21 23:51:37,798][09089] Num frames 6500... | |
| [2025-02-21 23:51:37,970][09089] Num frames 6600... | |
| [2025-02-21 23:51:38,125][09089] Num frames 6700... | |
| [2025-02-21 23:51:38,292][09089] Num frames 6800... | |
| [2025-02-21 23:51:38,456][09089] Num frames 6900... | |
| [2025-02-21 23:51:38,609][09089] Num frames 7000... | |
| [2025-02-21 23:51:38,767][09089] Num frames 7100... | |
| [2025-02-21 23:51:38,923][09089] Num frames 7200... | |
| [2025-02-21 23:51:39,078][09089] Num frames 7300... | |
| [2025-02-21 23:51:39,239][09089] Num frames 7400... | |
| [2025-02-21 23:51:39,410][09089] Num frames 7500... | |
| [2025-02-21 23:51:39,574][09089] Num frames 7600... | |
| [2025-02-21 23:51:39,731][09089] Num frames 7700... | |
| [2025-02-21 23:51:39,887][09089] Num frames 7800... | |
| [2025-02-21 23:51:40,045][09089] Num frames 7900... | |
| [2025-02-21 23:51:40,269][09089] Avg episode rewards: #0: 33.665, true rewards: #0: 13.332 | |
| [2025-02-21 23:51:40,270][09089] Avg episode reward: 33.665, avg true_objective: 13.332 | |
| [2025-02-21 23:51:40,273][09089] Num frames 8000... | |
| [2025-02-21 23:51:40,443][09089] Num frames 8100... | |
| [2025-02-21 23:51:40,602][09089] Num frames 8200... | |
| [2025-02-21 23:51:40,761][09089] Num frames 8300... | |
| [2025-02-21 23:51:40,934][09089] Num frames 8400... | |
| [2025-02-21 23:51:41,087][09089] Num frames 8500... | |
| [2025-02-21 23:51:41,246][09089] Num frames 8600... | |
| [2025-02-21 23:51:41,408][09089] Num frames 8700... | |
| [2025-02-21 23:51:41,572][09089] Num frames 8800... | |
| [2025-02-21 23:51:41,730][09089] Num frames 8900... | |
| [2025-02-21 23:51:41,896][09089] Num frames 9000... | |
| [2025-02-21 23:51:42,056][09089] Num frames 9100... | |
| [2025-02-21 23:51:42,236][09089] Num frames 9200... | |
| [2025-02-21 23:51:42,394][09089] Num frames 9300... | |
| [2025-02-21 23:51:42,549][09089] Num frames 9400... | |
| [2025-02-21 23:51:42,707][09089] Num frames 9500... | |
| [2025-02-21 23:51:42,862][09089] Num frames 9600... | |
| [2025-02-21 23:51:42,999][09089] Num frames 9700... | |
| [2025-02-21 23:51:43,144][09089] Num frames 9800... | |
| [2025-02-21 23:51:43,303][09089] Num frames 9900... | |
| [2025-02-21 23:51:43,460][09089] Num frames 10000... | |
| [2025-02-21 23:51:43,734][09089] Avg episode rewards: #0: 37.712, true rewards: #0: 14.427 | |
| [2025-02-21 23:51:43,735][09089] Avg episode reward: 37.712, avg true_objective: 14.427 | |
| [2025-02-21 23:51:43,738][09089] Num frames 10100... | |
| [2025-02-21 23:51:43,897][09089] Num frames 10200... | |
| [2025-02-21 23:51:44,053][09089] Num frames 10300... | |
| [2025-02-21 23:51:44,207][09089] Num frames 10400... | |
| [2025-02-21 23:51:44,361][09089] Num frames 10500... | |
| [2025-02-21 23:51:44,444][09089] Avg episode rewards: #0: 33.896, true rewards: #0: 13.146 | |
| [2025-02-21 23:51:44,445][09089] Avg episode reward: 33.896, avg true_objective: 13.146 | |
| [2025-02-21 23:51:44,575][09089] Num frames 10600... | |
| [2025-02-21 23:51:44,732][09089] Num frames 10700... | |
| [2025-02-21 23:51:44,888][09089] Num frames 10800... | |
| [2025-02-21 23:51:45,043][09089] Num frames 10900... | |
| [2025-02-21 23:51:45,197][09089] Num frames 11000... | |
| [2025-02-21 23:51:45,354][09089] Num frames 11100... | |
| [2025-02-21 23:51:45,507][09089] Num frames 11200... | |
| [2025-02-21 23:51:45,661][09089] Num frames 11300... | |
| [2025-02-21 23:51:45,815][09089] Num frames 11400... | |
| [2025-02-21 23:51:45,972][09089] Num frames 11500... | |
| [2025-02-21 23:51:46,102][09089] Avg episode rewards: #0: 32.934, true rewards: #0: 12.823 | |
| [2025-02-21 23:51:46,103][09089] Avg episode reward: 32.934, avg true_objective: 12.823 | |
| [2025-02-21 23:51:46,202][09089] Num frames 11600... | |
| [2025-02-21 23:51:46,363][09089] Num frames 11700... | |
| [2025-02-21 23:51:46,513][09089] Num frames 11800... | |
| [2025-02-21 23:51:46,667][09089] Num frames 11900... | |
| [2025-02-21 23:51:46,825][09089] Num frames 12000... | |
| [2025-02-21 23:51:46,965][09089] Avg episode rewards: #0: 30.453, true rewards: #0: 12.053 | |
| [2025-02-21 23:51:46,966][09089] Avg episode reward: 30.453, avg true_objective: 12.053 | |
| [2025-02-21 23:52:19,952][09089] Replay video saved to /root/autodl-tmp/train_dir/default_experiment/replay.mp4! | |
| [2025-02-21 23:52:19,990][09089] Loading existing experiment configuration from /root/autodl-tmp/train_dir/default_experiment/config.json | |
| [2025-02-21 23:52:19,991][09089] Overriding arg 'num_workers' with value 1 passed from command line | |
| [2025-02-21 23:52:19,991][09089] Adding new argument 'no_render'=True that is not in the saved config file! | |
| [2025-02-21 23:52:19,991][09089] Adding new argument 'save_video'=True that is not in the saved config file! | |
| [2025-02-21 23:52:19,991][09089] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file! | |
| [2025-02-21 23:52:19,991][09089] Adding new argument 'video_name'=None that is not in the saved config file! | |
| [2025-02-21 23:52:19,991][09089] Adding new argument 'max_num_frames'=100000 that is not in the saved config file! | |
| [2025-02-21 23:52:19,991][09089] Adding new argument 'max_num_episodes'=10 that is not in the saved config file! | |
| [2025-02-21 23:52:19,991][09089] Adding new argument 'push_to_hub'=True that is not in the saved config file! | |
| [2025-02-21 23:52:19,992][09089] Adding new argument 'hf_repository'='ashkid/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file! | |
| [2025-02-21 23:52:19,992][09089] Adding new argument 'policy_index'=0 that is not in the saved config file! | |
| [2025-02-21 23:52:19,992][09089] Adding new argument 'eval_deterministic'=False that is not in the saved config file! | |
| [2025-02-21 23:52:19,992][09089] Adding new argument 'train_script'=None that is not in the saved config file! | |
| [2025-02-21 23:52:19,992][09089] Adding new argument 'enjoy_script'=None that is not in the saved config file! | |
| [2025-02-21 23:52:19,992][09089] Using frameskip 1 and render_action_repeat=4 for evaluation | |
| [2025-02-21 23:52:20,020][09089] RunningMeanStd input shape: (3, 72, 128) | |
| [2025-02-21 23:52:20,022][09089] RunningMeanStd input shape: (1,) | |
| [2025-02-21 23:52:20,046][09089] ConvEncoder: input_channels=3 | |
| [2025-02-21 23:52:20,097][09089] Conv encoder output size: 512 | |
| [2025-02-21 23:52:20,097][09089] Policy head output size: 512 | |
| [2025-02-21 23:52:20,149][09089] Loading state from checkpoint /root/autodl-tmp/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth... | |
| [2025-02-21 23:52:20,727][09089] Num frames 100... | |
| [2025-02-21 23:52:20,876][09089] Num frames 200... | |
| [2025-02-21 23:52:21,030][09089] Num frames 300... | |
| [2025-02-21 23:52:21,183][09089] Num frames 400... | |
| [2025-02-21 23:52:21,336][09089] Num frames 500... | |
| [2025-02-21 23:52:21,493][09089] Num frames 600... | |
| [2025-02-21 23:52:21,649][09089] Num frames 700... | |
| [2025-02-21 23:52:21,784][09089] Avg episode rewards: #0: 19.510, true rewards: #0: 7.510 | |
| [2025-02-21 23:52:21,785][09089] Avg episode reward: 19.510, avg true_objective: 7.510 | |
| [2025-02-21 23:52:21,867][09089] Num frames 800... | |
| [2025-02-21 23:52:22,020][09089] Num frames 900... | |
| [2025-02-21 23:52:22,173][09089] Num frames 1000... | |
| [2025-02-21 23:52:22,327][09089] Num frames 1100... | |
| [2025-02-21 23:52:22,408][09089] Avg episode rewards: #0: 13.080, true rewards: #0: 5.580 | |
| [2025-02-21 23:52:22,408][09089] Avg episode reward: 13.080, avg true_objective: 5.580 | |
| [2025-02-21 23:52:22,545][09089] Num frames 1200... | |
| [2025-02-21 23:52:22,700][09089] Num frames 1300... | |
| [2025-02-21 23:52:22,854][09089] Num frames 1400... | |
| [2025-02-21 23:52:23,009][09089] Num frames 1500... | |
| [2025-02-21 23:52:23,161][09089] Num frames 1600... | |
| [2025-02-21 23:52:23,314][09089] Num frames 1700... | |
| [2025-02-21 23:52:23,485][09089] Num frames 1800... | |
| [2025-02-21 23:52:23,639][09089] Num frames 1900... | |
| [2025-02-21 23:52:23,792][09089] Num frames 2000... | |
| [2025-02-21 23:52:23,946][09089] Num frames 2100... | |
| [2025-02-21 23:52:24,068][09089] Avg episode rewards: #0: 14.800, true rewards: #0: 7.133 | |
| [2025-02-21 23:52:24,069][09089] Avg episode reward: 14.800, avg true_objective: 7.133 | |
| [2025-02-21 23:52:24,163][09089] Num frames 2200... | |
| [2025-02-21 23:52:24,321][09089] Num frames 2300... | |
| [2025-02-21 23:52:24,471][09089] Num frames 2400... | |
| [2025-02-21 23:52:24,623][09089] Num frames 2500... | |
| [2025-02-21 23:52:24,775][09089] Num frames 2600... | |
| [2025-02-21 23:52:24,930][09089] Num frames 2700... | |
| [2025-02-21 23:52:25,095][09089] Num frames 2800... | |
| [2025-02-21 23:52:25,252][09089] Num frames 2900... | |
| [2025-02-21 23:52:25,408][09089] Num frames 3000... | |
| [2025-02-21 23:52:25,565][09089] Num frames 3100... | |
| [2025-02-21 23:52:25,672][09089] Avg episode rewards: #0: 17.330, true rewards: #0: 7.830 | |
| [2025-02-21 23:52:25,672][09089] Avg episode reward: 17.330, avg true_objective: 7.830 | |
| [2025-02-21 23:52:25,782][09089] Num frames 3200... | |
| [2025-02-21 23:52:25,934][09089] Num frames 3300... | |
| [2025-02-21 23:52:26,088][09089] Num frames 3400... | |
| [2025-02-21 23:52:26,246][09089] Num frames 3500... | |
| [2025-02-21 23:52:26,399][09089] Num frames 3600... | |
| [2025-02-21 23:52:26,567][09089] Num frames 3700... | |
| [2025-02-21 23:52:26,712][09089] Num frames 3800... | |
| [2025-02-21 23:52:26,864][09089] Num frames 3900... | |
| [2025-02-21 23:52:27,022][09089] Num frames 4000... | |
| [2025-02-21 23:52:27,175][09089] Num frames 4100... | |
| [2025-02-21 23:52:27,366][09089] Avg episode rewards: #0: 18.576, true rewards: #0: 8.376 | |
| [2025-02-21 23:52:27,367][09089] Avg episode reward: 18.576, avg true_objective: 8.376 | |
| [2025-02-21 23:52:27,389][09089] Num frames 4200... | |
| [2025-02-21 23:52:27,543][09089] Num frames 4300... | |
| [2025-02-21 23:52:27,696][09089] Num frames 4400... | |
| [2025-02-21 23:52:27,846][09089] Num frames 4500... | |
| [2025-02-21 23:52:27,999][09089] Num frames 4600... | |
| [2025-02-21 23:52:28,149][09089] Num frames 4700... | |
| [2025-02-21 23:52:28,304][09089] Num frames 4800... | |
| [2025-02-21 23:52:28,473][09089] Num frames 4900... | |
| [2025-02-21 23:52:28,640][09089] Num frames 5000... | |
| [2025-02-21 23:52:28,805][09089] Num frames 5100... | |
| [2025-02-21 23:52:28,968][09089] Num frames 5200... | |
| [2025-02-21 23:52:29,137][09089] Num frames 5300... | |
| [2025-02-21 23:52:29,296][09089] Num frames 5400... | |
| [2025-02-21 23:52:29,457][09089] Num frames 5500... | |
| [2025-02-21 23:52:29,611][09089] Num frames 5600... | |
| [2025-02-21 23:52:29,767][09089] Num frames 5700... | |
| [2025-02-21 23:52:29,921][09089] Num frames 5800... | |
| [2025-02-21 23:52:30,082][09089] Num frames 5900... | |
| [2025-02-21 23:52:30,239][09089] Num frames 6000... | |
| [2025-02-21 23:52:30,392][09089] Num frames 6100... | |
| [2025-02-21 23:52:30,552][09089] Num frames 6200... | |
| [2025-02-21 23:52:30,749][09089] Avg episode rewards: #0: 25.647, true rewards: #0: 10.480 | |
| [2025-02-21 23:52:30,750][09089] Avg episode reward: 25.647, avg true_objective: 10.480 | |
| [2025-02-21 23:52:30,774][09089] Num frames 6300... | |
| [2025-02-21 23:52:30,930][09089] Num frames 6400... | |
| [2025-02-21 23:52:31,087][09089] Num frames 6500... | |
| [2025-02-21 23:52:31,254][09089] Num frames 6600... | |
| [2025-02-21 23:52:31,407][09089] Num frames 6700... | |
| [2025-02-21 23:52:31,563][09089] Num frames 6800... | |
| [2025-02-21 23:52:31,717][09089] Num frames 6900... | |
| [2025-02-21 23:52:31,884][09089] Num frames 7000... | |
| [2025-02-21 23:52:32,044][09089] Num frames 7100... | |
| [2025-02-21 23:52:32,179][09089] Avg episode rewards: #0: 24.360, true rewards: #0: 10.217 | |
| [2025-02-21 23:52:32,179][09089] Avg episode reward: 24.360, avg true_objective: 10.217 | |
| [2025-02-21 23:52:32,259][09089] Num frames 7200... | |
| [2025-02-21 23:52:32,422][09089] Num frames 7300... | |
| [2025-02-21 23:52:32,580][09089] Num frames 7400... | |
| [2025-02-21 23:52:32,744][09089] Num frames 7500... | |
| [2025-02-21 23:52:32,908][09089] Num frames 7600... | |
| [2025-02-21 23:52:33,074][09089] Num frames 7700... | |
| [2025-02-21 23:52:33,237][09089] Num frames 7800... | |
| [2025-02-21 23:52:33,390][09089] Num frames 7900... | |
| [2025-02-21 23:52:33,555][09089] Num frames 8000... | |
| [2025-02-21 23:52:33,702][09089] Num frames 8100... | |
| [2025-02-21 23:52:33,877][09089] Avg episode rewards: #0: 23.970, true rewards: #0: 10.220 | |
| [2025-02-21 23:52:33,878][09089] Avg episode reward: 23.970, avg true_objective: 10.220 | |
| [2025-02-21 23:52:33,909][09089] Num frames 8200... | |
| [2025-02-21 23:52:34,062][09089] Num frames 8300... | |
| [2025-02-21 23:52:34,228][09089] Num frames 8400... | |
| [2025-02-21 23:52:34,383][09089] Num frames 8500... | |
| [2025-02-21 23:52:34,534][09089] Num frames 8600... | |
| [2025-02-21 23:52:34,691][09089] Num frames 8700... | |
| [2025-02-21 23:52:34,853][09089] Num frames 8800... | |
| [2025-02-21 23:52:35,015][09089] Num frames 8900... | |
| [2025-02-21 23:52:35,161][09089] Num frames 9000... | |
| [2025-02-21 23:52:35,311][09089] Num frames 9100... | |
| [2025-02-21 23:52:35,468][09089] Num frames 9200... | |
| [2025-02-21 23:52:35,623][09089] Num frames 9300... | |
| [2025-02-21 23:52:35,779][09089] Num frames 9400... | |
| [2025-02-21 23:52:35,934][09089] Num frames 9500... | |
| [2025-02-21 23:52:36,100][09089] Num frames 9600... | |
| [2025-02-21 23:52:36,257][09089] Num frames 9700... | |
| [2025-02-21 23:52:36,410][09089] Num frames 9800... | |
| [2025-02-21 23:52:36,529][09089] Avg episode rewards: #0: 26.155, true rewards: #0: 10.933 | |
| [2025-02-21 23:52:36,530][09089] Avg episode reward: 26.155, avg true_objective: 10.933 | |
| [2025-02-21 23:52:36,643][09089] Num frames 9900... | |
| [2025-02-21 23:52:36,816][09089] Num frames 10000... | |
| [2025-02-21 23:52:36,981][09089] Num frames 10100... | |
| [2025-02-21 23:52:37,145][09089] Num frames 10200... | |
| [2025-02-21 23:52:37,300][09089] Num frames 10300... | |
| [2025-02-21 23:52:37,459][09089] Num frames 10400... | |
| [2025-02-21 23:52:37,601][09089] Num frames 10500... | |
| [2025-02-21 23:52:37,743][09089] Num frames 10600... | |
| [2025-02-21 23:52:37,898][09089] Num frames 10700... | |
| [2025-02-21 23:52:38,054][09089] Num frames 10800... | |
| [2025-02-21 23:52:38,210][09089] Num frames 10900... | |
| [2025-02-21 23:52:38,363][09089] Num frames 11000... | |
| [2025-02-21 23:52:38,509][09089] Num frames 11100... | |
| [2025-02-21 23:52:38,633][09089] Avg episode rewards: #0: 27.043, true rewards: #0: 11.143 | |
| [2025-02-21 23:52:38,633][09089] Avg episode reward: 27.043, avg true_objective: 11.143 | |
| [2025-02-21 23:53:09,124][09089] Replay video saved to /root/autodl-tmp/train_dir/default_experiment/replay.mp4! | |