## Protocol Section
### Identification Module
**NCT ID:** NCT06428799

**Acronym:** EMBLEDDA-MMG

**Brief Title:** Evaluation of Carebot AI MMG Medical Device for Breast Lesion Detection and Density Assessment

**Official Title:** Evaluation of Carebot AI MMG Medical Device for Detection and Radiographic Assessment of Breast Lesions and Quantitative Analysis of Breast Density: A Multicentric, Multi-Reader Study

#### Organization Study ID Info
**ID:** 00003
#### Organization
**Class:** INDUSTRY
**Full Name:** Carebot s.r.o.
### Status Module
#### Completion Date
**Date:** 2024-04-24

**Type:** ACTUAL

#### Expanded Access Info
#### Last Update Post Date
**Date:** 2024-05-24

**Type:** ACTUAL

**Last Update Submit Date:** 2024-05-20

**Overall Status:** COMPLETED

#### Primary Completion Date
**Date:** 2024-04-24

**Type:** ACTUAL

#### Start Date
**Date:** 2022-01-14

**Type:** ACTUAL

**Status Verified Date:** 2024-05

#### Study First Post Date
**Date:** 2024-05-24
**Type:** ACTUAL
**Study First Submit Date:** 2024-05-07
**Study First Submit QC Date:** 2024-05-20
### Sponsor Collaborators Module

#### Lead Sponsor

**Class:** INDUSTRY

**Name:** Carebot s.r.o.

#### Responsible Party

**Type:** SPONSOR

### Oversight Module

**Is FDA Regulated Device:** False

**Is FDA Regulated Drug:** False

**Oversight Has DMC:** False

### Description Module

**Brief Summary:** Comparison of accuracy of clinician and DLAD image evaluation (Carebot AI MMG v2.2)

1. Comparison of the Accuracy of Density Assessment by Clinician and DLAD (DENS)
2. Comparison of Accuracy of Lesion Assessment by Clinician and DLAD (MASS, CLASS)

**Detailed Description:** The mammography studies were acquired from three independent sites: Site 1 (EUC Mamocentrum Brno) and Site 2 (Hospital Šumperk, a.s.) specialise in routine screening mammography, and Site 3 (Masaryk Memorial Cancer Institute) is a comprehensive oncology facility primarily dedicated to diagnostic mammography, i.e. performing additional examinations in case of a suspicious finding (recall).

The ground truth was obtained by consensus of two board-certified radiologists with expertise in radiology and diagnostic methods, and 13 and 27 years of experience with mammography image interpretation, respectively.

For comparative analysis, a team of five independent radiologists with clinical experience in interpreting mammography images was established. Three of the clinicians were junior (2, 2, and 4 years of experience, respectively) without board-certification; two physicians were senior (7 and 8 years of experience, respectively), board-certified.

### Conditions Module

**Conditions:**

- Breast Cancer

- Breast Tumor Benign

### Design Module

#### Design Info

**Observational Model:** COHORT

**Time Perspective:** RETROSPECTIVE

#### Enrollment Info

**Count:** 122

**Type:** ACTUAL

**Study Type:** OBSERVATIONAL

### Arms Interventions Module

#### Arm Group 1

**Description:** A total of 60 mammographic studies were retrospectively collected from Site 1 (EUC Mamocentrum Brno).

The acquisition of mammography studies from Site 1 was enabled by the contract for the transfer of mammography images for medical research purposes, signed on 14 January 2022.

**Intervention Names:**

- Device: Carebot AI MMG

**Label:** Retrospective collection of DICOM patient files for Site 1

#### Arm Group 2

**Description:** A total of 28 mammographic studies were retrospectively collected from Site 2 (Hospital Šumperk, a.s.).

The acquisition of mammography studies from Institution 2 was enabled by the contract for the transfer of mammography images for medical research purposes, signed on 31 January 2023.

**Intervention Names:**

- Device: Carebot AI MMG

**Label:** Retrospective collection of DICOM patient files for Site 2

#### Arm Group 3

**Description:** A total of 34 mammographic studies were retrospectively collected from Site 3 (Masaryk Memorial Cancer Institute).

The acquisition of mammography studies from Institution 3 was enabled by the amendment to the contract for the transfer of X-ray images for medical research purposes, signed on 21 February 2023, which follows the contract for the transfer of X-ray images for medical research purposes, signed on 3 January 2022.

**Intervention Names:**

- Device: Carebot AI MMG

**Label:** Retrospective collection of DICOM patient files for Site 3

### Interventions

#### Intervention 1

**Arm Group Labels:**

- Retrospective collection of DICOM patient files for Site 1

- Retrospective collection of DICOM patient files for Site 2

- Retrospective collection of DICOM patient files for Site 3

**Description:** Carebot AI MMG is a software solution that utilizes artificial intelligence methods, specifically deep learning and computer vision algorithms, to evaluate and localize suspicious regions of potential lesions during the interpretation of digital breast x-rays as part of standard mammography screening procedures. The Carebot AI MMG medical device is not intended for use in diagnostic mammography. The Carebot AI MMG is intended for use in women over the age of 18.

The predictive outputs of the Carebot AI MMG medical device are intended to aid decision-making in screening clinical practice, always in conjunction with other relevant patient information and based on the professional judgment of the examining clinician. The Carebot AI MMG is specifically designed to provide a supporting layer of analysis that helps in evaluating or prioritizing mammography images with additional patient information and the professional judgment of the examining physician.

**Name:** Carebot AI MMG

**Type:** DEVICE

### Outcomes Module

#### Primary Outcomes

**Description:** A multicenter, multi-reader, retrospective study was designed to validate the clinical efficacy of the proposed Carebot AI MMG (also referred to as "DLAD"). Using a non-certified medical device, a test set of retrospectively collected mammography studies in standard projections (CC and MLO). The performance of the DLAD was evaluated against the ground truth for individual indications (breast density evaluation, breast lesion detection) using Accuracy.

**Measure:** Performance Test

**Time Frame:** 2024

#### Secondary Outcomes

**Description:** Carebot AI MMG v2.2 classified mammography studies according to the ACR BI-RADS 5th edition, i.e. breast tissue density assessment into A/B/C/D classes. The performance of the Carebot AI MMG device was assessed relative to the ground truth and then compared with the performance of five independent radiologists with varying levels of experience (RAD 1-RAD 5). A rigorous statistical analysis was used in the BI-RADS breast density classification to evaluate the performance of each method - the proposed Carebot AI MMG v2.2 medical device and the compared radiologists in the multi-reader study. The analysis focused on key metrics including Accuracy, F1 Score (Macro-Averaged), Precision (Macro-Averaged), Recall (Macro-Averaged) and Cohen's Kappa (κ) to assess the strength of agreement. Given that all scans were evaluated by all radiologists in the comparison, a bootstrapping method that involves resampling the test data 1000 times.

**Measure:** Comparison of Accuracy of Clinician and DLAD Image Evaluation (Carebot AI MMG v2.2) in Breast Density Assessment

**Time Frame:** 2024

**Description:** The medical device (DLAD, Carebot AI MMG v2.2) analyzed mammography studies in standard projections (CC and MLO) and classified the presence of lesions ("present" x "absent") at the mammography study level. DLAD performance was assessed relative to the ground truth and then compared to the performance of five independent radiologists with varying levels of experience (RAD 1-RAD 5). The investigators quantified the performance of diagnostic tests based on Sensitivity, Specificity and Balanced Accuracy. The investigators further assessed the statistical significance of differences between DLAD and individual radiologists using appropriate statistical tests. To determine the reliability of the metrics examined, the investigators calculated 95% confidence intervals using Wilson scores. To evaluate the statistical significance of differences in Sensitivity and Specificity between DLAD and individual radiologists, the investigators applied McNemar's test with a continuity correction.

**Measure:** Comparison of Accuracy of Clinician and DLAD Image Evaluation (Carebot AI MMG v2.2) in Breast Lesion Detection

**Time Frame:** 2024

### Eligibility Module

**Eligibility Criteria:** Inclusion Criteria:

* The medical device is intended for use in women over 18 years of age who are indicated for screening mammography using digital mammography.

Exclusion Criteria:

* The medical device cannot be used in patients with breast implants.
* The medical device cannot be used in male breast examination.
* The medical device cannot be used in patients under 18 years of age.

**Gender Based:** True

**Gender Description:** The medical device cannot be used in male breast examination.

**Minimum Age:** 18 Years

**Sampling Method:** NON_PROBABILITY_SAMPLE

**Sex:** FEMALE

**Standard Ages:**

- ADULT

- OLDER_ADULT

**Study Population:** A total of 122 mammography studies (488 images, the so-called "test data") were obtained from the reference sites for the evaluation by radiologists and the Carebot AI MMG v2.2. The target sites used different mammography X-ray machines: Site 1 and Site 2 used the Senographe Essential mammography machine from GE HealthCare to image patients, and Site 3 used the Selenia Dimensions mammography machine from Hologic and the MAMMOMAT Revelation mammography machine from Siemens Healthineers.

### Contacts Locations Module

#### Locations

**Location 1:**

**City:** Brno

**Country:** Czechia

**Facility:** EUC Mamocentrum Brno

**Zip:** 60200

**Location 2:**

**City:** Brno

**Country:** Czechia

**Facility:** Masaryk Memorial Cancer Institute

**Zip:** 60200

**Location 3:**

**City:** Šumperk

**Country:** Czechia

**Facility:** Hospital Šumperk

**Zip:** 78701

### IPD Sharing Statement Module

**IPD Sharing:** UNDECIDED

## Derived Section

### Condition Browse Module - Ancestors

- ID: D000009371

- Term: Neoplasms by Site

- ID: D000009369

- Term: Neoplasms

- ID: D000001941

- Term: Breast Diseases

- ID: D000012871

- Term: Skin Diseases

### Condition Browse Module - Browse Branches

- Abbrev: BC04

- Name: Neoplasms

- Abbrev: BC17

- Name: Skin and Connective Tissue Diseases

- Abbrev: All

- Name: All Conditions

### Condition Browse Module - Browse Leaves

- ID: M5220

- Name: Breast Neoplasms

- Relevance: HIGH

- As Found: Breast Tumor

- ID: M5218

- Name: Breast Diseases

- Relevance: LOW

- As Found: Unknown

- ID: M15674

- Name: Skin Diseases

- Relevance: LOW

- As Found: Unknown

### Condition Browse Module - Meshes

- ID: D000001943

- Term: Breast Neoplasms

### Intervention Browse Module - Browse Branches

- Abbrev: PhSol

- Name: Pharmaceutical Solutions

- Abbrev: All

- Name: All Drugs and Chemicals

### Intervention Browse Module - Browse Leaves

- ID: M21860

- Name: Pharmaceutical Solutions

- Relevance: LOW

- As Found: Unknown

### Misc Info Module

- Version Holder: 2024-05-31
