To comprehensively evaluate the predictive performance and investigate the effect of the special neural network architecture, we performed 10-fold cross-validation experiments on hands and feet individually. The primal evaluation metric is Pearson's correlation between predictions and ground truth labels created by human professionals. We also considered a secondary metric, the root mean square error (RMSE) between prediction and ground truth. We benchmarked the neural network models with or without the segmentation of the joint space region. In both hands and feet, the neural network with segmentation achieved significantly higher correlations than the network without segmentation, as well as lower RMSEs (Fig. 5a-b).
Since joint space narrowing and erosion often occur simultaneously, a joint with narrowing damage is likely to have bone erosion. We therefore hypothesize that the neural network architecture with segmentation will also improve the prediction accuracy of erosion damages. Similar to joint space narrowing, this model significantly increased the correlations and decreased RMSEs for erosion prediction in both hands and feet (Fig. 5c-d).