Cloned from Intel/dpt-large

.gitattributes

@@ -1,35 +1,28 @@
 *.7z filter=lfs diff=lfs merge=lfs -text
 *.arrow filter=lfs diff=lfs merge=lfs -text
 *.bin filter=lfs diff=lfs merge=lfs -text
+*.bin.* filter=lfs diff=lfs merge=lfs -text
 *.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
 *.ftz filter=lfs diff=lfs merge=lfs -text
 *.gz filter=lfs diff=lfs merge=lfs -text
 *.h5 filter=lfs diff=lfs merge=lfs -text
 *.joblib filter=lfs diff=lfs merge=lfs -text
 *.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
 *.model filter=lfs diff=lfs merge=lfs -text
 *.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
 *.onnx filter=lfs diff=lfs merge=lfs -text
 *.ot filter=lfs diff=lfs merge=lfs -text
 *.parquet filter=lfs diff=lfs merge=lfs -text
 *.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
 *.pt filter=lfs diff=lfs merge=lfs -text
 *.pth filter=lfs diff=lfs merge=lfs -text
 *.rar filter=lfs diff=lfs merge=lfs -text
-*.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text 
 *.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
 *.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
 *.xz filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
+*.zstandard filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,176 @@
+---
+license: apache-2.0
+tags:
+- vision
+- depth-estimation
+widget:
+- src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/tiger.jpg
+  example_title: Tiger
+- src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/teapot.jpg
+  example_title: Teapot
+- src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/palace.jpg
+  example_title: Palace
+
+model-index:
+- name: dpt-large
+  results:
+  - task:
+      type: monocular-depth-estimation
+      name: Monocular Depth Estimation
+    dataset:
+      type: MIX-6
+      name: MIX-6
+    metrics:
+    - type: Zero-shot transfer
+      value: 10.82
+      name: Zero-shot transfer
+      config: Zero-shot transfer
+      verified: false
+---
+
+## Model Details: DPT-Large (also known as MiDaS 3.0)
+
+Dense Prediction Transformer (DPT) model trained on 1.4 million images for monocular depth estimation. 
+It was introduced in the paper [Vision Transformers for Dense Prediction](https://arxiv.org/abs/2103.13413) by Ranftl et al. (2021) and first released in [this repository](https://github.com/isl-org/DPT). 
+DPT uses the Vision Transformer (ViT) as backbone and adds a neck + head on top for monocular depth estimation.
+![model image](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/dpt_architecture.jpg)
+
+The model card has been written in combination by the Hugging Face team and Intel.
+
+| Model Detail | Description |
+| ----------- | ----------- | 
+| Model Authors - Company | Intel | 
+| Date | March 22, 2022 | 
+| Version | 1 | 
+| Type | Computer Vision - Monocular Depth Estimation | 
+| Paper or Other Resources | [Vision Transformers for Dense Prediction](https://arxiv.org/abs/2103.13413) and [GitHub Repo](https://github.com/isl-org/DPT) | 
+| License | Apache 2.0 |
+| Questions or Comments | [Community Tab](https://huggingface.co/Intel/dpt-large/discussions) and [Intel Developers Discord](https://discord.gg/rv2Gp55UJQ)|
+
+| Intended Use | Description |
+| ----------- | ----------- | 
+| Primary intended uses | You can use the raw model for zero-shot monocular depth estimation. See the [model hub](https://huggingface.co/models?search=dpt) to look for fine-tuned versions on a task that interests you. | 
+| Primary intended users | Anyone doing monocular depth estimation | 
+| Out-of-scope uses | This model in most cases will need to be fine-tuned for your particular task.  The model should not be used to intentionally create hostile or alienating environments for people.|
+
+
+### How to use
+
+The easiest is leveraging the pipeline API:
+
+```
+from transformers import pipeline
+
+pipe = pipeline(task="depth-estimation", model="Intel/dpt-large")
+result = pipe(image)
+result["depth"]
+```
+
+In case you want to implement the entire logic yourself, here's how to do that for zero-shot depth estimation on an image:
+
+```python
+from transformers import DPTImageProcessor, DPTForDepthEstimation
+import torch
+import numpy as np
+from PIL import Image
+import requests
+
+url = "http://images.cocodataset.org/val2017/000000039769.jpg"
+image = Image.open(requests.get(url, stream=True).raw)
+
+processor = DPTImageProcessor.from_pretrained("Intel/dpt-large")
+model = DPTForDepthEstimation.from_pretrained("Intel/dpt-large")
+
+# prepare image for the model
+inputs = processor(images=image, return_tensors="pt")
+
+with torch.no_grad():
+    outputs = model(**inputs)
+    predicted_depth = outputs.predicted_depth
+
+# interpolate to original size
+prediction = torch.nn.functional.interpolate(
+    predicted_depth.unsqueeze(1),
+    size=image.size[::-1],
+    mode="bicubic",
+    align_corners=False,
+)
+
+# visualize the prediction
+output = prediction.squeeze().cpu().numpy()
+formatted = (output * 255 / np.max(output)).astype("uint8")
+depth = Image.fromarray(formatted)
+```
+
+For more code examples, we refer to the [documentation](https://huggingface.co/docs/transformers/master/en/model_doc/dpt).
+
+
+| Factors | Description | 
+| ----------- | ----------- | 
+| Groups | Multiple datasets compiled together | 
+| Instrumentation | - |
+| Environment | Inference completed on Intel Xeon Platinum 8280 CPU @ 2.70GHz with 8 physical cores and an NVIDIA RTX 2080 GPU. |
+| Card Prompts | Model deployment on alternate hardware and software will change model performance |
+
+| Metrics | Description | 
+| ----------- | ----------- | 
+| Model performance measures | Zero-shot Transfer |
+| Decision thresholds | - | 
+| Approaches to uncertainty and variability | - | 
+
+| Training and Evaluation Data | Description | 
+| ----------- | ----------- | 
+| Datasets | The dataset is called MIX 6, and contains around 1.4M images. The model was initialized with ImageNet-pretrained weights.|
+| Motivation | To build a robust monocular depth prediction network |
+| Preprocessing | "We resize the image such that the longer side is 384 pixels and train on random square crops of size 384. ... We perform random horizontal flips for data augmentation." See [Ranftl et al. (2021)](https://arxiv.org/abs/2103.13413) for more details. | 
+
+## Quantitative Analyses
+| Model | Training set | DIW WHDR | ETH3D AbsRel | Sintel AbsRel | KITTI δ>1.25 | NYU δ>1.25 | TUM δ>1.25 |
+| --- | --- | --- | --- | --- | --- | --- | --- | 
+| DPT - Large | MIX 6 | 10.82 (-13.2%) | 0.089 (-31.2%) | 0.270 (-17.5%) | 8.46 (-64.6%) | 8.32 (-12.9%) | 9.97 (-30.3%) |
+| DPT - Hybrid | MIX 6 | 11.06 (-11.2%) | 0.093 (-27.6%) | 0.274 (-16.2%) | 11.56 (-51.6%) | 8.69 (-9.0%) | 10.89 (-23.2%) | 
+| MiDaS  | MIX 6  | 12.95 (+3.9%)  | 0.116 (-10.5%)  | 0.329 (+0.5%)  | 16.08 (-32.7%)  | 8.71 (-8.8%)  | 12.51 (-12.5%)
+| MiDaS [30]  | MIX 5  | 12.46  | 0.129  | 0.327  | 23.90  | 9.55  | 14.29 | 
+ | Li [22]  | MD [22]  | 23.15  | 0.181  | 0.385  | 36.29  | 27.52  | 29.54 | 
+ | Li [21]  | MC [21]  | 26.52  | 0.183  | 0.405  | 47.94  | 18.57  | 17.71 | 
+ | Wang [40]  | WS [40]  | 19.09  | 0.205  | 0.390  | 31.92  | 29.57  | 20.18 | 
+ | Xian [45]  | RW [45]  | 14.59  | 0.186 |  0.422  | 34.08 |  27.00 |  25.02 | 
+ | Casser [5]  | CS [8]  | 32.80  | 0.235  | 0.422  | 21.15  | 39.58  | 37.18 | 
+
+Table 1. Comparison to the state of the art on monocular depth estimation. We evaluate zero-shot cross-dataset transfer according to the
+protocol defined in [30]. Relative performance is computed with respect to the original MiDaS model [30]. Lower is better for all metrics. ([Ranftl et al., 2021](https://arxiv.org/abs/2103.13413))
+
+
+| Ethical Considerations | Description | 
+| ----------- | ----------- | 
+| Data | The training data come from multiple image datasets compiled together. |
+| Human life | The model is not intended to inform decisions central to human life or flourishing. It is an aggregated set of monocular depth image datasets. | 
+| Mitigations | No additional risk mitigation strategies were considered during model development. |
+| Risks and harms | The extent of the risks involved by using the model remain unknown. |
+| Use cases | - | 
+
+| Caveats and Recommendations |
+| ----------- | 
+| Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. There are no additional caveats or recommendations for this model. |
+
+
+
+### BibTeX entry and citation info
+
+```bibtex
+@article{DBLP:journals/corr/abs-2103-13413,
+  author    = {Ren{\'{e}} Ranftl and
+               Alexey Bochkovskiy and
+               Vladlen Koltun},
+  title     = {Vision Transformers for Dense Prediction},
+  journal   = {CoRR},
+  volume    = {abs/2103.13413},
+  year      = {2021},
+  url       = {https://arxiv.org/abs/2103.13413},
+  eprinttype = {arXiv},
+  eprint    = {2103.13413},
+  timestamp = {Wed, 07 Apr 2021 15:31:46 +0200},
+  biburl    = {https://dblp.org/rec/journals/corr/abs-2103-13413.bib},
+  bibsource = {dblp computer science bibliography, https://dblp.org}
+}
+```

config.json

@@ -0,0 +1,47 @@
+{
+  "architectures": [
+    "DPTForDepthEstimation"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "auxiliary_loss_weight": 0.4,
+  "backbone_out_indices": [
+    5,
+    11,
+    17,
+    23
+  ],
+  "fusion_hidden_size": 256,
+  "head_in_index": -1,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.0,
+  "hidden_size": 1024,
+  "image_size": 384,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-12,
+  "model_type": "dpt",
+  "neck_hidden_sizes": [
+    256,
+    512,
+    1024,
+    1024
+  ],
+  "num_attention_heads": 16,
+  "num_channels": 3,
+  "num_hidden_layers": 24,
+  "patch_size": 16,
+  "qkv_bias": true,
+  "readout_type": "project",
+  "reassemble_factors": [
+    4,
+    2,
+    1,
+    0.5
+  ],
+  "semantic_classifier_dropout": 0.1,
+  "semantic_loss_ignore_index": 255,
+  "torch_dtype": "float32",
+  "transformers_version": "4.18.0.dev0",
+  "use_auxiliary_head": true,
+  "use_batch_norm_in_fusion_residual": false
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c173e7fd575c4d6c2164621063fed3a0e7e2eb6adbfb7c1845b364bc0fed9ce8
+size 1367456044

preprocessor_config.json

@@ -0,0 +1,19 @@
+{
+  "do_normalize": true,
+  "do_resize": true,
+  "ensure_multiple_of": 1,
+  "feature_extractor_type": "DPTFeatureExtractor",
+  "image_mean": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "image_std": [
+    0.5,
+    0.5,
+    0.5
+  ],
+  "keep_aspect_ratio": false,
+  "resample": 2,
+  "size": 384
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:71150941604c39c1c770a72a7b2f56487669f0e3a4d99c8759e233fb1be24080
+size 1367581165