README.md

metadata

title: README
emoji: 👀
colorFrom: gray
colorTo: gray
sdk: static
pinned: false

Want to contribute?

• 🥙 For naming models, spaces, datasets, and metrics, employ kebab-case. Use lowercase letters, except for acronyms, which may be capitalized.
• 🔒 Please mind about making the models, spaces and datasets public or private. Metrics can (and have to) be public.
• 🤐 Do not expose passwords or tokens, use secrets.

Structure. (Click to expand)

Your metric should have the following structure:

• _compute(references, predictions)
  • Calls a metric engine, defined for example in the seametrics package, or other
• compute_from_payload(paylaod)
  • Call the module.compute method internally after converting payload -> references, predictions
• All the metric's parameters, such as iou_threshold, area_ranges, etc.. should be moved to the __init__ method.
• Output should look like this:

  {
  "ahoy_IR_b2_engine_3_6_0_49_gd81d3b63_oversea": {
      "overall": {
          "all": {
              "f1": 0.15967351103175614,
              "fn": 2923.0,
              "fp": 3666.0,
              "num_gt_ids": 10,
              "precision": 0.14585274930102515,
              "recall": 0.1763877148492533,
              "recognition_0.3": 0.1,
              "recognition_0.5": 0.1,
              "recognition_0.8": 0.1,
              "recognized_0.3": 1,
              "recognized_0.5": 1,
              "recognized_0.8": 1,
              "tp": 626.0
          }
      },
      "per_sequence": {
          "Sentry_2023_02_08_PROACT_CELADON_@6m_MOB_2023_02_08_12_51_49": {
              "all": {
                  "f1": 0.15967351103175614,
                  "fn": 2923.0,
                  "fp": 3666.0,
                  "num_gt_ids": 10,
                  "precision": 0.14585274930102515,
                  "recall": 0.1763877148492533,
                  "recognition_0.3": 0.1,
                  "recognition_0.5": 0.1,
                  "recognition_0.8": 0.1,
                  "recognized_0.3": 1,
                  "recognized_0.5": 1,
                  "recognized_0.8": 1,
                  "tp": 626.0
              }
          }
      }
  }
  }```
  

Looking for metrics?

• https://huggingface.co/spaces/SEA-AI/det-metrics
  • Object detection metrics based on pycocotools and torchmetrics' Mean Avergae Precision.
• https://huggingface.co/spaces/SEA-AI/box-metrics
  • Bounding box statistics, including IOU, BEP (bottom edge proximity), and others.
• https://huggingface.co/spaces/SEA-AI/horizon-metrics
  • Comparing horizons in an image w.r.t their midpoint and slope errors
• https://huggingface.co/spaces/SEA-AI/mot-metrics
  • Multi-object-tracking metrics using py-motmetrics
• https://huggingface.co/spaces/SEA-AI/panoptic-quality
  • Evaluating panoptic models