End of training

README.md

@@ -0,0 +1,79 @@
+---
+tags:
+- generated_from_trainer
+datasets:
+- funsd
+model-index:
+- name: layoutlm-funsd
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# layoutlm-funsd
+
+This model is a fine-tuned version of [microsoft/layoutlm-base-uncased](https://huggingface.co/microsoft/layoutlm-base-uncased) on the funsd dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.7049
+- Answer: {'precision': 0.7178051511758119, 'recall': 0.792336217552534, 'f1': 0.7532314923619271, 'number': 809}
+- Header: {'precision': 0.2803030303030303, 'recall': 0.31092436974789917, 'f1': 0.29482071713147406, 'number': 119}
+- Question: {'precision': 0.7570815450643776, 'recall': 0.828169014084507, 'f1': 0.7910313901345292, 'number': 1065}
+- Overall Precision: 0.7123
+- Overall Recall: 0.7827
+- Overall F1: 0.7459
+- Overall Accuracy: 0.8046
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 3e-05
+- train_batch_size: 16
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 15
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Answer                                                                                                      | Header                                                                                                         | Question                                                                                                     | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------:|:-----------------:|:--------------:|:----------:|:----------------:|
+| 1.747         | 1.0   | 10   | 1.5873          | {'precision': 0.018205461638491547, 'recall': 0.0173053152039555, 'f1': 0.01774397972116603, 'number': 809} | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                    | {'precision': 0.19186046511627908, 'recall': 0.15492957746478872, 'f1': 0.17142857142857143, 'number': 1065} | 0.1099            | 0.0898         | 0.0988     | 0.3629           |
+| 1.4357        | 2.0   | 20   | 1.2348          | {'precision': 0.2678787878787879, 'recall': 0.273176761433869, 'f1': 0.27050183598531213, 'number': 809}    | {'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}                                                    | {'precision': 0.4725705329153605, 'recall': 0.5661971830985916, 'f1': 0.5151644596326356, 'number': 1065}    | 0.3922            | 0.4134         | 0.4025     | 0.5730           |
+| 1.0578        | 3.0   | 30   | 0.9336          | {'precision': 0.4918864097363083, 'recall': 0.5995055624227441, 'f1': 0.5403899721448469, 'number': 809}    | {'precision': 0.027777777777777776, 'recall': 0.008403361344537815, 'f1': 0.012903225806451613, 'number': 119} | {'precision': 0.6016260162601627, 'recall': 0.6948356807511737, 'f1': 0.644880174291939, 'number': 1065}     | 0.5444            | 0.6152         | 0.5776     | 0.7071           |
+| 0.814         | 4.0   | 40   | 0.7665          | {'precision': 0.5914935707220573, 'recall': 0.7391841779975278, 'f1': 0.6571428571428571, 'number': 809}    | {'precision': 0.14084507042253522, 'recall': 0.08403361344537816, 'f1': 0.10526315789473685, 'number': 119}    | {'precision': 0.6415562913907285, 'recall': 0.7276995305164319, 'f1': 0.6819181698196215, 'number': 1065}    | 0.6039            | 0.6939         | 0.6458     | 0.7567           |
+| 0.6755        | 5.0   | 50   | 0.7183          | {'precision': 0.6652221018418202, 'recall': 0.7589616810877626, 'f1': 0.7090069284064666, 'number': 809}    | {'precision': 0.2692307692307692, 'recall': 0.23529411764705882, 'f1': 0.25112107623318386, 'number': 119}     | {'precision': 0.7006039689387403, 'recall': 0.7624413145539906, 'f1': 0.7302158273381295, 'number': 1065}    | 0.6651            | 0.7296         | 0.6959     | 0.7844           |
+| 0.5514        | 6.0   | 60   | 0.6832          | {'precision': 0.6699134199134199, 'recall': 0.765142150803461, 'f1': 0.7143681477207153, 'number': 809}     | {'precision': 0.2625, 'recall': 0.17647058823529413, 'f1': 0.21105527638190955, 'number': 119}                 | {'precision': 0.7032, 'recall': 0.8253521126760563, 'f1': 0.7593952483801296, 'number': 1065}                | 0.6739            | 0.7622         | 0.7153     | 0.7894           |
+| 0.4846        | 7.0   | 70   | 0.6668          | {'precision': 0.6698513800424628, 'recall': 0.7799752781211372, 'f1': 0.7207310108509423, 'number': 809}    | {'precision': 0.2773109243697479, 'recall': 0.2773109243697479, 'f1': 0.2773109243697479, 'number': 119}       | {'precision': 0.719932716568545, 'recall': 0.8037558685446009, 'f1': 0.7595385980479148, 'number': 1065}     | 0.6756            | 0.7627         | 0.7165     | 0.7922           |
+| 0.4323        | 8.0   | 80   | 0.6610          | {'precision': 0.6934065934065934, 'recall': 0.7799752781211372, 'f1': 0.7341477603257708, 'number': 809}    | {'precision': 0.3113207547169811, 'recall': 0.2773109243697479, 'f1': 0.2933333333333334, 'number': 119}       | {'precision': 0.7377892030848329, 'recall': 0.8084507042253521, 'f1': 0.771505376344086, 'number': 1065}     | 0.6986            | 0.7652         | 0.7304     | 0.7999           |
+| 0.3889        | 9.0   | 90   | 0.6681          | {'precision': 0.7054871220604704, 'recall': 0.7787391841779975, 'f1': 0.7403055229142185, 'number': 809}    | {'precision': 0.2975206611570248, 'recall': 0.3025210084033613, 'f1': 0.3, 'number': 119}                      | {'precision': 0.7395309882747069, 'recall': 0.8291079812206573, 'f1': 0.7817618415227978, 'number': 1065}    | 0.7015            | 0.7772         | 0.7374     | 0.8046           |
+| 0.3514        | 10.0  | 100  | 0.6881          | {'precision': 0.7018909899888766, 'recall': 0.7799752781211372, 'f1': 0.7388758782201406, 'number': 809}    | {'precision': 0.2857142857142857, 'recall': 0.3025210084033613, 'f1': 0.2938775510204082, 'number': 119}       | {'precision': 0.7391304347826086, 'recall': 0.8300469483568075, 'f1': 0.7819548872180452, 'number': 1065}    | 0.6983            | 0.7782         | 0.7361     | 0.7977           |
+| 0.3179        | 11.0  | 110  | 0.6895          | {'precision': 0.6925566343042071, 'recall': 0.7935723114956736, 'f1': 0.73963133640553, 'number': 809}      | {'precision': 0.3185840707964602, 'recall': 0.3025210084033613, 'f1': 0.3103448275862069, 'number': 119}       | {'precision': 0.7578125, 'recall': 0.819718309859155, 'f1': 0.7875507442489851, 'number': 1065}              | 0.7076            | 0.7782         | 0.7412     | 0.8011           |
+| 0.3008        | 12.0  | 120  | 0.6971          | {'precision': 0.7183257918552036, 'recall': 0.7849196538936959, 'f1': 0.7501476668635558, 'number': 809}    | {'precision': 0.2923076923076923, 'recall': 0.31932773109243695, 'f1': 0.3052208835341365, 'number': 119}      | {'precision': 0.7574978577549272, 'recall': 0.8300469483568075, 'f1': 0.7921146953405018, 'number': 1065}    | 0.7139            | 0.7812         | 0.7460     | 0.8036           |
+| 0.2844        | 13.0  | 130  | 0.7024          | {'precision': 0.710352422907489, 'recall': 0.7972805933250927, 'f1': 0.751310425160163, 'number': 809}      | {'precision': 0.2868217054263566, 'recall': 0.31092436974789917, 'f1': 0.2983870967741935, 'number': 119}      | {'precision': 0.7601031814273431, 'recall': 0.8300469483568075, 'f1': 0.7935368043087971, 'number': 1065}    | 0.7118            | 0.7858         | 0.7470     | 0.8027           |
+| 0.2735        | 14.0  | 140  | 0.7061          | {'precision': 0.7136514983351832, 'recall': 0.7948084054388134, 'f1': 0.752046783625731, 'number': 809}     | {'precision': 0.2824427480916031, 'recall': 0.31092436974789917, 'f1': 0.29600000000000004, 'number': 119}     | {'precision': 0.7575236457437661, 'recall': 0.8272300469483568, 'f1': 0.7908438061041293, 'number': 1065}    | 0.7112            | 0.7832         | 0.7455     | 0.8040           |
+| 0.2676        | 15.0  | 150  | 0.7049          | {'precision': 0.7178051511758119, 'recall': 0.792336217552534, 'f1': 0.7532314923619271, 'number': 809}     | {'precision': 0.2803030303030303, 'recall': 0.31092436974789917, 'f1': 0.29482071713147406, 'number': 119}     | {'precision': 0.7570815450643776, 'recall': 0.828169014084507, 'f1': 0.7910313901345292, 'number': 1065}     | 0.7123            | 0.7827         | 0.7459     | 0.8046           |
+
+
+### Framework versions
+
+- Transformers 4.28.0
+- Pytorch 2.0.1+cu118
+- Datasets 2.12.0
+- Tokenizers 0.13.3

logs/events.out.tfevents.1684833889.97d68b1e9c63.4236.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f96570ddde1361a8f35cb3ad53e8b11e12e96b52f5fb8d65284355936c0d666c
-size 14018
+oid sha256:1435d4c03678ea67a0633df2f9d80ca549a35d7898233b7a0c2eef12d707423b
+size 14372

preprocessor_config.json

@@ -0,0 +1,14 @@
+{
+  "apply_ocr": true,
+  "do_resize": true,
+  "feature_extractor_type": "LayoutLMv2FeatureExtractor",
+  "image_processor_type": "LayoutLMv2ImageProcessor",
+  "ocr_lang": null,
+  "processor_class": "LayoutLMv2Processor",
+  "resample": 2,
+  "size": {
+    "height": 224,
+    "width": 224
+  },
+  "tesseract_config": ""
+}

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:99130ce20201129da28ed4cce60de80798cd32ea0abf44c40a52887f5a11a9ce
+oid sha256:a27969e2598183471deb32692f24b8f93ebeab844dbd4a5aea12f5e3ac0f4a68
 size 450608389

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,38 @@
+{
+  "additional_special_tokens": null,
+  "apply_ocr": false,
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "cls_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "only_label_first_subword": true,
+  "pad_token": "[PAD]",
+  "pad_token_box": [
+    0,
+    0,
+    0,
+    0
+  ],
+  "pad_token_label": -100,
+  "processor_class": "LayoutLMv2Processor",
+  "sep_token": "[SEP]",
+  "sep_token_box": [
+    1000,
+    1000,
+    1000,
+    1000
+  ],
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "LayoutLMv2Tokenizer",
+  "unk_token": "[UNK]"
+}