How to calculate the mAP on this network?

[Sebi2106]

System Info

I want to evaluate my network with the mean Average Precision. I don't know how to get the class-id of my gt data. Are there any examples to calculate the mAP with this library?

I use the DetrForObjectDetection with my own dataset.

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

this is my code to save the loss in a csv file. I also want to save the mAP in this file.

def on_train_epoch_end(self, trainer, pl_module):
            train_loss = trainer.callback_metrics.get("training_loss").item()
            val_loss = trainer.callback_metrics.get("validation/loss").item()
            with open(self.file_path, 'a', newline='') as csvfile:
                writer = csv.writer(csvfile)
                if not self.header_written:
                    writer.writerow(["Epoch", "Train Loss", "Validation Loss"])
                    self.header_written = True
                writer.writerow([pl_module.current_epoch, train_loss, val_loss])

Expected behavior

I tried to get the data with this code:

    gt_boxes = []
    detected_boxes = []
    for batch in self.val_dataloader:
        pixel_values = batch['pixel_values'].to(pl_module.device)
        pixel_mask = batch['pixel_mask'].to(pl_module.device)
        labels = batch['labels']
        # train_idx = batch['train_idx']
        outputs = pl_module(pixel_values=pixel_values, pixel_mask=pixel_mask)
        
        target_sizes = torch.tensor([image.shape[-2:] for image in pixel_values]).to(pixel_values.device)
        detections = image_processor.post_process_object_detection(outputs, target_sizes=target_sizes, threshold=0.5)[0]

        for i in range(len(detections['scores'])):
            prob_score = detections['scores'][i].item()
            class_pred = detections['labels'][i].item()
            box = detections['boxes'][i].detach().cpu().numpy()

            detected_boxes.append([class_pred, prob_score, *box])
        
        for label in labels:
            gt_box = label['boxes']
            for box in gt_box:
                gt_boxes.append(box)

    image_height = 2048
    image_width = 2048

    gt_boxes_abs = []
    for box in gt_boxes:
        x_min, y_min, width, height = box
        x_max = x_min + width
        y_max = y_min + height
        x_min_abs = int(x_min * image_width)
        y_min_abs = int(y_min * image_height)
        x_max_abs = int(x_max * image_width)
        y_max_abs = int(y_max * image_height)
        
        class_id = ???
        difficult = ???
        crowd = ???
        
        gt_boxes_abs.append([x_min_abs, y_min_abs, x_max_abs, y_max_abs, class_id, difficult, crowd])
    
    adjusted_detected_boxes = []
    converted_boxes = []
    for box in detected_boxes:
        class_id = box[0]
        confidence = box[1]
        x_min = box[2]
        y_min = box[3]
        x_max = box[4]
        y_max = box[5]
        converted_boxes.append([x_min, y_min, x_max, y_max, class_id, confidence])

[ArthurZucker]

Hey 🤗 thanks for opening an issue! We try to keep the github issues for bugs/feature requests.
Could you ask your question on the forum instead? I'm sure the community will be of help!

Thanks!

[NielsRogge]

It's a valid question though, part of #29964

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[NielsRogge]

cc @qubvel

[qubvel]

Hi @Sebi2106, you can find an updated example how to compute mAP in this PR
#30422

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.