kertob commited on
Commit
9b41750
·
verified ·
1 Parent(s): 66ea9f0

Training in progress, step 170

Browse files
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:364ceb7ae52affb9ad78d06f425d9e44a05ff6e3fca9692068e69486a4a5b2a1
3
  size 261131840
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2ff90d45fcdd96c6dd647095ba4ed56702716ef0e6ff6e49e993ee5135e7459a
3
  size 261131840
runs/Mar13_11-36-50_7acbe5a74b63/events.out.tfevents.1710329812.7acbe5a74b63.337.1 CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e98363a4f26fe3b45c3e867e197a44bda281b9b57c9673ee857f00d8ad6bcb98
3
- size 8878
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36eb5f3e8f96692af95264ce761cb81a744277e7e764d026e48bbb5d53a01c73
3
+ size 9089
wandb/debug-internal.log CHANGED
@@ -1252,3 +1252,49 @@
1252
  2024-03-13 12:00:48,489 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1253
  2024-03-13 12:00:52,351 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1254
  2024-03-13 12:00:54,352 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1252
  2024-03-13 12:00:48,489 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1253
  2024-03-13 12:00:52,351 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1254
  2024-03-13 12:00:54,352 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1255
+ 2024-03-13 12:00:56,119 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1256
+ 2024-03-13 12:00:56,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1257
+ 2024-03-13 12:00:56,160 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1258
+ 2024-03-13 12:01:00,340 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1259
+ 2024-03-13 12:01:05,341 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1260
+ 2024-03-13 12:01:10,341 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1261
+ 2024-03-13 12:01:11,119 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1262
+ 2024-03-13 12:01:11,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1263
+ 2024-03-13 12:01:11,159 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1264
+ 2024-03-13 12:01:16,311 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1265
+ 2024-03-13 12:01:21,312 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1266
+ 2024-03-13 12:01:22,352 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1267
+ 2024-03-13 12:01:26,119 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1268
+ 2024-03-13 12:01:26,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1269
+ 2024-03-13 12:01:26,160 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1270
+ 2024-03-13 12:01:27,230 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1271
+ 2024-03-13 12:01:32,230 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1272
+ 2024-03-13 12:01:37,232 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1273
+ 2024-03-13 12:01:41,120 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1274
+ 2024-03-13 12:01:41,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1275
+ 2024-03-13 12:01:41,161 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1276
+ 2024-03-13 12:01:42,297 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1277
+ 2024-03-13 12:01:47,298 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1278
+ 2024-03-13 12:01:52,298 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1279
+ 2024-03-13 12:01:52,357 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1280
+ 2024-03-13 12:01:56,120 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1281
+ 2024-03-13 12:01:56,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1282
+ 2024-03-13 12:01:56,161 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1283
+ 2024-03-13 12:01:58,210 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1284
+ 2024-03-13 12:02:03,210 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1285
+ 2024-03-13 12:02:08,211 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1286
+ 2024-03-13 12:02:11,120 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1287
+ 2024-03-13 12:02:11,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1288
+ 2024-03-13 12:02:11,160 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1289
+ 2024-03-13 12:02:13,242 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1290
+ 2024-03-13 12:02:16,992 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: partial_history
1291
+ 2024-03-13 12:02:16,993 DEBUG SenderThread:5135 [sender.py:send():378] send: history
1292
+ 2024-03-13 12:02:16,993 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: summary_record
1293
+ 2024-03-13 12:02:16,994 INFO SenderThread:5135 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end
1294
+ 2024-03-13 12:02:17,343 INFO Thread-12 :5135 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240313_112023-t8o614bo/files/wandb-summary.json
1295
+ 2024-03-13 12:02:18,995 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1296
+ 2024-03-13 12:02:22,354 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1297
+ 2024-03-13 12:02:24,354 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1298
+ 2024-03-13 12:02:26,120 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1299
+ 2024-03-13 12:02:26,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1300
+ 2024-03-13 12:02:26,161 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
wandb/run-20240313_112023-t8o614bo/files/wandb-summary.json CHANGED
@@ -1 +1 @@
1
- {"train/loss": 0.1411, "train/grad_norm": 0.7663830518722534, "train/learning_rate": 1.012785947186397e-05, "train/epoch": 12.8, "train/global_step": 160, "_timestamp": 1710331246.485993, "_runtime": 2423.1039838790894, "_step": 19, "train/train_runtime": 361.3918, "train/train_samples_per_second": 1.328, "train/train_steps_per_second": 0.083, "train/total_flos": 1585771096988928.0, "train/train_loss": 1.3836020787556966}
 
1
+ {"train/loss": 0.1313, "train/grad_norm": 0.5366930365562439, "train/learning_rate": 5.785225463498828e-06, "train/epoch": 13.6, "train/global_step": 170, "_timestamp": 1710331336.9920022, "_runtime": 2513.6099932193756, "_step": 20, "train/train_runtime": 361.3918, "train/train_samples_per_second": 1.328, "train/train_steps_per_second": 0.083, "train/total_flos": 1585771096988928.0, "train/train_loss": 1.3836020787556966}
wandb/run-20240313_112023-t8o614bo/logs/debug-internal.log CHANGED
@@ -1252,3 +1252,49 @@
1252
  2024-03-13 12:00:48,489 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1253
  2024-03-13 12:00:52,351 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1254
  2024-03-13 12:00:54,352 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1252
  2024-03-13 12:00:48,489 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1253
  2024-03-13 12:00:52,351 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1254
  2024-03-13 12:00:54,352 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1255
+ 2024-03-13 12:00:56,119 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1256
+ 2024-03-13 12:00:56,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1257
+ 2024-03-13 12:00:56,160 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1258
+ 2024-03-13 12:01:00,340 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1259
+ 2024-03-13 12:01:05,341 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1260
+ 2024-03-13 12:01:10,341 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1261
+ 2024-03-13 12:01:11,119 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1262
+ 2024-03-13 12:01:11,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1263
+ 2024-03-13 12:01:11,159 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1264
+ 2024-03-13 12:01:16,311 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1265
+ 2024-03-13 12:01:21,312 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1266
+ 2024-03-13 12:01:22,352 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1267
+ 2024-03-13 12:01:26,119 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1268
+ 2024-03-13 12:01:26,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1269
+ 2024-03-13 12:01:26,160 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1270
+ 2024-03-13 12:01:27,230 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1271
+ 2024-03-13 12:01:32,230 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1272
+ 2024-03-13 12:01:37,232 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1273
+ 2024-03-13 12:01:41,120 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1274
+ 2024-03-13 12:01:41,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1275
+ 2024-03-13 12:01:41,161 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1276
+ 2024-03-13 12:01:42,297 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1277
+ 2024-03-13 12:01:47,298 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1278
+ 2024-03-13 12:01:52,298 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1279
+ 2024-03-13 12:01:52,357 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1280
+ 2024-03-13 12:01:56,120 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1281
+ 2024-03-13 12:01:56,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1282
+ 2024-03-13 12:01:56,161 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1283
+ 2024-03-13 12:01:58,210 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1284
+ 2024-03-13 12:02:03,210 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1285
+ 2024-03-13 12:02:08,211 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1286
+ 2024-03-13 12:02:11,120 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1287
+ 2024-03-13 12:02:11,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1288
+ 2024-03-13 12:02:11,160 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
1289
+ 2024-03-13 12:02:13,242 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1290
+ 2024-03-13 12:02:16,992 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: partial_history
1291
+ 2024-03-13 12:02:16,993 DEBUG SenderThread:5135 [sender.py:send():378] send: history
1292
+ 2024-03-13 12:02:16,993 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: summary_record
1293
+ 2024-03-13 12:02:16,994 INFO SenderThread:5135 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end
1294
+ 2024-03-13 12:02:17,343 INFO Thread-12 :5135 [dir_watcher.py:_on_file_modified():288] file/dir modified: /content/wandb/run-20240313_112023-t8o614bo/files/wandb-summary.json
1295
+ 2024-03-13 12:02:18,995 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1296
+ 2024-03-13 12:02:22,354 DEBUG SenderThread:5135 [sender.py:send():378] send: stats
1297
+ 2024-03-13 12:02:24,354 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: status_report
1298
+ 2024-03-13 12:02:26,120 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: stop_status
1299
+ 2024-03-13 12:02:26,120 DEBUG SenderThread:5135 [sender.py:send_request():405] send_request: stop_status
1300
+ 2024-03-13 12:02:26,161 DEBUG HandlerThread:5135 [handler.py:handle_request():146] handle_request: internal_messages
wandb/run-20240313_112023-t8o614bo/run-t8o614bo.wandb CHANGED
Binary files a/wandb/run-20240313_112023-t8o614bo/run-t8o614bo.wandb and b/wandb/run-20240313_112023-t8o614bo/run-t8o614bo.wandb differ