Key Points
- Atelectasis and attic retraction pocket are two common tympanic
membrane changes.
- Otoscope is widely accepted for diagnosing and managing OME,
atelectasis and attic retraction pocket.
- General practitioners, pediatricians and otolaryngologists showed low
diagnostic accuracy for ear diseases.
- We developed a deep learning model to diagnose atelectasis and attic
retraction pocket using otoscopic images and assess the performance of
deep learning model.
- It may be used to improve the procedure of OME diagnosis and
management, such as saving time and improving diagnostic accuracy.
INTRODUCTION
Atelectasis and attic retraction pocket occur because of tympanic
membranes architectural deformity and bad ventilation, then tympanic
membranes collapse toward to the tympanic cavity. Tympanic membrane
retraction is the most common change of tympanic membrane in pediatric
otitis media with effusion (OME) patients 1.
Atelectasis and attic retraction pocket also could be sequela of OME,
and it’s more frequently in surgery cases 2,3.
Patients with mild atelectasis or attic retraction pocket may have no
symptoms, however, tiny attic retraction pocket may conceal attic
cholesteatoma 4. Severe atelectasis and attic
retraction pocket may cause erosion of ossicular chain, outer wall of
tympanic cavity and hearing loss. Moreover, whether there is atelectasis
or attic retraction pocket is important part in the OME diagnosis
procedure 5. Cholesteatoma and adhesive otitis media
are common severe sequela of
atelectasis and attic retraction
pocket 6. Although surgery serves as an effective
method to treat severe atelectasis and attic retraction pocket, but
surgery is associated with economic burden, and surgery risk, such as
sensorineural hearing loss, facial palsy. And some patients may cease to
retract and returned to normal condition, so prophylactic surgery would
not be recommended 7. However, early diagnosis with
appropriate follow up is a reasonable policy to manage
atelectasis and attic retraction
pocket, irrespective of surgery or not 8,9.
Otoscope is widely accepted for diagnosing and managing OME, atelectasis
and attic retraction pocket 5,10. And many types of
smartphone adaptable otoscopes can be
used to acquire tympanic membranes images by no-specialty or
no-clinicians 11-13. However, Diagnosis of ear disease
only with manual examination shows low accuracy, which may lead to
improper referral, delayed or improper treatment and pointless
follow-up.
The progressive use of telemedicine and artificial medicine in the
otologic setting may gradually change the procedure of disease
management. Wu et. al 14 developed a deep learning
model to diagnosis pediatric otitis media using otoscopic images and
tested in a smartphone-enabled otoscope set. Shie et. al15 extracted color, geometric and texture features to
develop a classification system for differentiating most type of otitis
media, achieving an accuracy of 88.06% in 865 otoscopic images. Cha et.
al 16 developed a deep learning model to detect 6
common ear diseases acquiring an accuracy of 93.67%. All the previous
artificial intelligence studies didn’t classify retraction of tympanic
membranes as atelectasis and attic retraction pocket.
The purpose of this study was to develop a
deep
learning model to diagnose atelectasis and attic retraction pocket using
otoscopic images and assess the performance of deep learning model. It
may be used to improve the procedure of OME diagnosis and management,
such as saving time and improving diagnostic accuracy.
METHOD
Participant selection and otoscopic images acquisition
Otoscopic images from inpatients and outpatients were collected
retrospectively from 3 hospitals between year 2015 to 2019. Otoscopic
images were taken with 4 mm (KARL STORZ, Germany) or 2.7 mm (TIAN SONG,
China) 0-degree otoscope by otolaryngologists. OME cases were confirmed
with criteria of clinical guideline5, including
disease history, medical examination and auditory test. 1 to 3 best
quality otoscopic images from different angles with complete pars tensa
and pars flaccida were adopted from each ear with at least 500
× 500 pixels. White light, eardrum
size more than 50% in the otoscopic images and light reflection without
overexposure and underexposure were optimal. Otoscopic images with
tympanostomy tube, secretion and earwax more than 25% of tympanic
membranes were excluded in this study.
Clinical labelling of otoscopic images
Only a few parts of otoscopic images have been recorded the presence of
attic retraction pocket and atelectasis in the electronic medical record
systems. To achieve a consistent ground truth label, we didn’t adopt
these records as ground truth label. Firstly, JBZ with more than 3 years
clinical experience in otology were assigned to address the presence of
attic retraction pocket and atelectasis according to the first
widespread standard independently 17,18. Because attic
retraction pocket and atelectasis may present in the same otoscopic
images, these two lesions were labeled separately. All otoscopic images
were labeled the presence of attic retraction pocket and atelectasis
without region annotation. Then, two otologists with more than 10 years
clinical experience in otology were assigned to review labels
independently, and any discrepancy will be discussed with another
otologist with more than 20 years clinical experience in otology until
consensus was reached. As in actual clinical practice, the prevalence of
different stage of attic retraction pocket and atelectasis was heavily
skewed in our dataset, stage III and IV attic retraction pocket and
atelectasis with less than 5%. To ensure that there was sufficient data
to develop and assess the performance of this model, we only address the
presence of attic retraction pocket and atelectasis without stage
classification. Other clinical demographic data wasn’t used to develop
deep learning model, such as acoustic test results, age and gender.
Deep learning model development
3-fold random cross validation was adopted to divided dataset into
training set and validation set. The output of this model was a standard
two-class task for determining whether the input otoscopic image
contained attic retraction pocket or atelectasis. We used a CNN model
pretrained on the ImageNet dataset (http://www.image-net.org), then
otoscopic images of this dataset were used to fine-tune the
hyperparameters of the pretrained CNN model. During the process of
training, online data was used for data expansion, including random
vertical and horizontal flip, and constant aspect ratio scaling.
Considering our previous experience, Google Inception-V3 were suitable
for developing deep learning model based on otoscopic images. So, Google
Inception-V3 CNN model was adopted as the backbone network and trained,
tuned and evaluated19. All the otoscopic images were
turned into 299 × 299 pixels as input data. CNN model consisted of a
convolutional neural network to implicitly recognize characteristics of
attic retraction pocket and atelectasis from otoscopic images.
To evaluate the CNN model performance in clinical practice, we compared
the predicted diagnosis with the labeled standard diagnosis using the
3-fold
average classification accuracy, sensitivity, specificity of the model
(normal pars flaccida vs attic retraction pocket, normal pars tensor vs
atelectasis). We also used receiver operating characteristic (ROC) curve
and corresponding area under ROC curve to show the diagnostic ability of
the deep learning model in identifying the presence of attic retraction
pocket and atelectasis.
Class Activation Mapping
Class Activation Mapping (CAM) was employed to visualize the
discriminative region in the otoscopic images. CAM used different colors
to show different values of deep learning model ranging from blue (no
specific region) to red (most discriminative region). Right
identification of lesion region with red color in the otoscopic images
are essential for clinician to trust the deep learning model. All
experiments were operated with Python 3.6 in Keras using Python
programming language. The diagnostic model was developed based on
TensorFlow and carried out with 4 Titan XP 256 GB GPU.
RESULTS
We collected an image dataset consist of 6393 OME otoscopic images,
of which 3564 (55.74%) otoscopic
images were assessed for attic retraction pocket, and
atelectasis
was diagnosed in 2460 (38.48%) otoscopic images. Each otoscopic images
were reviewed by at least 3 expert otologists. We used 3-fold
cross-validation for developing and testing the deep convolutional
neural network (DCNN) model to detect OME referable attic retraction
pocket and atelectasis.