The MIDOG 2022 comes with extended metrics over its predecessor. Following popular request, we added to the threshold-based metrics (F1/precision/recall) also the average precision (AP), as is commonly used for object detection.
Additionally, participants will be able to see the performance on each of the tumor types (four in the preliminary test set and ten in the final test set).
Changes in the Docker container
These new metrics, however, also mean a small change in the container. The reference docker container has already been updated to reflect these changes.
In particular, the output format now shows for each detection:
In the probability field, we expect a number between 0 and 1, which reflects the detection probability (e.g., after softmax).
The “name” field is of particular importance: It needs to be set to “mitotic figure” whenever a detection is above threshold. Else, it may just have any other value (for instance: “non-mitotic figure”). We use this field for the threshold-based metrics (like F1, precision, and recall). For the AP metric, this field is not evaluated, but the “probability” is.