| results = benchmark.run() | |
| print(results) | |
| results = benchmark.run() | |
| print(results) | |
| ==================== INFERENCE - SPEED - RESULT ==================== | |
| Model Name Batch Size Seq Length Time in s | |
| google-bert/bert-base-uncased 8 8 0.005 | |
| google-bert/bert-base-uncased 8 32 0.008 | |
| google-bert/bert-base-uncased 8 128 0.022 | |
| google-bert/bert-base-uncased 8 512 0.105 | |
| ==================== INFERENCE - MEMORY - RESULT ==================== | |
| Model Name Batch Size Seq Length Memory in MB | |
| google-bert/bert-base-uncased 8 8 1330 | |
| google-bert/bert-base-uncased 8 32 1330 | |
| google-bert/bert-base-uncased 8 128 1330 | |
| google-bert/bert-base-uncased 8 512 1770 | |
| ==================== ENVIRONMENT INFORMATION ==================== | |
| transformers_version: 2.11.0 | |
| framework: Tensorflow | |
| use_xla: False | |
| framework_version: 2.2.0 | |
| python_version: 3.6.10 | |
| system: Linux | |
| cpu: x86_64 | |
| architecture: 64bit | |
| date: 2020-06-29 | |
| time: 09:26:35.617317 | |
| fp16: False | |
| use_multiprocessing: True | |
| only_pretrain_model: False | |
| cpu_ram_mb: 32088 | |
| use_gpu: True | |
| num_gpus: 1 | |
| gpu: TITAN RTX | |
| gpu_ram_mb: 24217 | |
| gpu_power_watts: 280.0 | |
| gpu_performance_state: 2 | |
| use_tpu: False | |
| By default, the time and the required memory for inference are benchmarked. |