ANALYSIS AND RESULTS
Model Performance
Table 2 compares the performance of the OLS and DNN classifiers in
predicting premature births in 2018. The overall accuracy of all the
outputs was high, however, the OLS outputs showed very poor sensitivity.
The highest specificity of the OLS classifier is achieved with the
cutoff of 0.1 and was 0.22 for LBW and 0.23 for PTB. It steeply
decreased as the cutoff increased.
Compared to the OLS approach, the DNN classifiers showed more promising
results, particularly in terms of increased sensitivity. The highest
sensitivity for predicting LBW using the DNN classifier was 0.64 with
one hidden layer and 12:1 of class weighting on positive cases.
Similarly, using two hidden layers and a class weight ratio of 11:1, the
highest sensitivity for predicting PTB was 0.64. However, the results
also show that a higher class weight on positive cases sacrifices
accuracy and specificity while increasing sensitivity.
[Table 2 about here]
Figure 2 shows experiments with different levels of the threshold in
classifying the output in the output layer. In the plots, the Y-axis
indicates the score of performance metrics. Various thresholds from
0.501 to 0.509 are on the X-axis. For example, in Figure 2A, 0.501 means
that records are classified as LBW if the predicted outputs are 0.501 or
higher. In these experiments, the number of hidden layers and the class
weight were respectively set to 1 and 11 that showed the best
performance among the settings in Table 2.
By tuning the thresholds, the highest sensitivity for predicting LBW
reached 0.69 with a threshold of 0.5012. With this setting, the accuracy
and specificity were 0.75 and 0.76, respectively. However, the threshold
tuning for predicting PTB showed relatively lower performance than the
results based on the default threshold of 0.5 in Table 1. Therefore, the
result for predicting PTB with the highest sensitivity in this study was
0.64, with an accuracy of 0.74 and a specificity of 0.75.
[Figure 2 about here]