Discussion

In this study, we develop a novel machine learning approach for integrating multiple images and automatically quantifying joint space narrowing and bone erosion in rheumatoid arthritis. We designed a special neural network architecture that simultaneously scores joint damage levels and segments the joint space regions. This design not only significantly improves the prediction accuracy but also highlights the regions of interest to assist further analysis in clinical settings. The idea of introducing segmentation into an image-based regression deep learning model should not be limited to the joint damage scoring in RA. In fact, many biomedical imaging problems have a similar situation - the quantification or diagnosis closely depends on parts of an image. The segmentation of the disease-related regions of interest will be crucial to guide a neural network to focus on those regions, improve the performance, and facilitate subsequent error analysis or clinical diagnosis. This is especially true when the sample size is relatively small and the segmentation will serve as a “teacher” that helps the model training well with a limited number of samples.
Inspired by the widely observed symmetrical symptoms in patients with RA, the method we developed learns multilevel symmetry and dependence across images. This approach is novel in that it seamlessly integrates multiple layers of information from different images to guide prediction, which can be extended to other medical image fields. Additionally, we investigated the relationships among joints and damage types in our machine learning model and revealed the disease-specific map; this data-driven RA-specific map is instructive to clinical decisions. This study design can be applied to many biomedical imaging problems and biological studies, with or without symmetrical patterns. Through analyzing the contributions of different, multiple images used in a machine learning model, the hidden relationships between different disease manifestations will be revealed from a new computational perspective, complementing direct experimental observations and current knowledge.

Conclusion

[Summarize here key points in your findings along with the implications of your study.]

Acknowledgements

Acknowledgements should include contributions from anyone who does not meet the criteria for authorship (for example, to recognize contributions from people who provided technical help, collation of data, writing assistance, acquisition of funding, or a department chairperson who provided general support), as well as any funding or other supporting information.

Conflict of interest

You will be asked to provide a conflict of interest statement during the submission process. Please check the journal’s author guidelines for more information.

Supporting Information

[Please don't insert supporting information here! We encourage you to include all your results and data in the main article. However, if you need to submit a supporting information document, you can upload it to AISY's Supporting Information collection on Authorea via https://authorea.com/inst/21456 and insert the DOI here.]