Welcome

This app provides distance-based and volume-overlap-based agreement measures for 3D structures stored in mesh files. The general approach is to calculate agreement measures for all pairwise comparisons, and aggregate the individual results. The application motivating development of MeshAgreement is to compare delineated structures for radiotherapy treatment planning.

In order to export 3D mesh files in PLY format from Varian Eclipse, you can use an ESAPI script .

This is a joint project by the Institute of Medical Biostatistics, Epidemiology and Informatics (IMBEI) together with the Department of Radiation Oncology and Radiotherapy at the University Medical Center of the Johannes Gutenberg-University Mainz . The section gives more information on the calculated agreement measures, and lists contributors as well as software tools.

Usage

  • I - Green area: navigation between sections
    • Load data: Upload mesh files defining volumes which should be compared
    • Agreement: View results for pairwise agreement as well as for average agreement
    • View: Interactive 3D plots for a pair of meshes with their union and intersection
    • About: Background information on methods and contributors
  • II - Orange area: Main output region
  • III - Purple area: Toggle fullscreen mode

Comparisons

Information on selected meshes

Note: If information on mesh volume and mesh centroid are missing ('NA'), the mesh is not proper. This means that the mesh may not be closed, may not bound a volume, may have self-intersections, or may contain duplicated faces. Some agreement measures will then be unavailable. Ticking the 'fix mesh issues' box or using surface reconstruction on import may help. Otherwise, inspection of the mesh with a tool such as MeshLab is advised.

Options for volume-overlap calculations

Options for VCG metro distance calculations

For the definition of the following options, see the documentation of the Rvcg::vcgMetro() function as well the VCG metro page

Pairwise agreement

Average agreement

Pairwise agreement

Average agreement

Calculated agreement measures

See section for a definition of calculated measures

Settings

Methods

Geometric agreement measures

  • Distance-based agreement measures
    • DCOM: Distance between centers of mass
    • ASD: Average symmetric surface distance
    • RMSD: Root mean squared symmetric surface distance
    • HD_max: Hausdorff distance - worst case, maximum of both directed Hausdorff distances
    • HD_avg: Hausdorff distance - average, mean of both directed Hausdorff distances
  • Volume-overlap based measures
    • JSC: Jaccard similarity coefficient
    • DSC: Dice similarity coefficient

The calculated agreement measures are defined and discussed in these publications:

  • Fotina et al. Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy. Strahlenther Onkol 2012; 188: 160-167.
  • Hanna et al. Geometrical Analysis of Radiotherapy Target Volume Delineation: a Systematic Review of Reported Comparison Methods. Clin Oncol 2010; 22, 515-525.
  • Heimann et al. Comparison and Evaluation of Methods for Liver Segmentation From CT Datasets. IEEE Trans Med Imaging 2009; 28: 1251-1265.
  • Sherer et al. Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review. Radiother Oncol 2021; 160: 185-191.

The geometric agreement is first calculated for all possible volume pairs, and then averaged over these pairs.

  • Coefficient of variation - for aggregating pairwise similarity measures
    • CV: SD(x) / Mean(x)
    • CV_ln: sqrt(exp(var(log(x))) - 1) - assuming log-normal distribution

Project contributors

People

University Medical Center of the Johannes Gutenberg-University Mainz

Software

This web application is part of the R package MeshAgreement which itself is based on the following software: