SEA-AI (SEA.AI)

Organization Card

Community About org cards

Want to contribute?

🥙 For naming models, spaces, datasets, and metrics, employ kebab-case. Use lowercase letters, except for acronyms, which may be capitalized.
🔒 Please mind about making the models, spaces and datasets public or private. Metrics can (and have to) be public.
🤐 Do not expose passwords or tokens, use secrets.

Structure. (Click to expand)

Your metric should have the following structure:

_compute(references, predictions)
- Calls a metric engine, defined for example in the seametrics package, or other
compute_from_payload(paylaod)
- Call the module.compute method internally after converting payload -> references, predictions
All the metric's parameters, such as iou_threshold, area_ranges, etc.. should be moved to the __init__ method.

Output should look like this:

{
"ahoy_IR_b2_engine_3_6_0_49_gd81d3b63_oversea": {
    "overall": {
        "all": {
            "f1": 0.15967351103175614,
            "fn": 2923.0,
            "fp": 3666.0,
            "num_gt_ids": 10,
            "precision": 0.14585274930102515,
            "recall": 0.1763877148492533,
            "recognition_0.3": 0.1,
            "recognition_0.5": 0.1,
            "recognition_0.8": 0.1,
            "recognized_0.3": 1,
            "recognized_0.5": 1,
            "recognized_0.8": 1,
            "tp": 626.0
        }
    },
    "per_sequence": {
        "Sentry_2023_02_08_PROACT_CELADON_@6m_MOB_2023_02_08_12_51_49": {
            "all": {
                "f1": 0.15967351103175614,
                "fn": 2923.0,
                "fp": 3666.0,
                "num_gt_ids": 10,
                "precision": 0.14585274930102515,
                "recall": 0.1763877148492533,
                "recognition_0.3": 0.1,
                "recognition_0.5": 0.1,
                "recognition_0.8": 0.1,
                "recognized_0.3": 1,
                "recognized_0.5": 1,
                "recognized_0.8": 1,
                "tp": 626.0
            }
        }
    }
}
}```