ANALYSIS AND RESULTS

Model Performance

Table 2 compares the performance of the OLS and DNN classifiers in predicting premature births in 2018. The overall accuracy of all the outputs was high, however, the OLS outputs showed very poor sensitivity. The highest specificity of the OLS classifier is achieved with the cutoff of 0.1 and was 0.22 for LBW and 0.23 for PTB. It steeply decreased as the cutoff increased.
Compared to the OLS approach, the DNN classifiers showed more promising results, particularly in terms of increased sensitivity. The highest sensitivity for predicting LBW using the DNN classifier was 0.64 with one hidden layer and 12:1 of class weighting on positive cases. Similarly, using two hidden layers and a class weight ratio of 11:1, the highest sensitivity for predicting PTB was 0.64. However, the results also show that a higher class weight on positive cases sacrifices accuracy and specificity while increasing sensitivity.
[Table 2 about here]
Figure 2 shows experiments with different levels of the threshold in classifying the output in the output layer. In the plots, the Y-axis indicates the score of performance metrics. Various thresholds from 0.501 to 0.509 are on the X-axis. For example, in Figure 2A, 0.501 means that records are classified as LBW if the predicted outputs are 0.501 or higher. In these experiments, the number of hidden layers and the class weight were respectively set to 1 and 11 that showed the best performance among the settings in Table 2.
By tuning the thresholds, the highest sensitivity for predicting LBW reached 0.69 with a threshold of 0.5012. With this setting, the accuracy and specificity were 0.75 and 0.76, respectively. However, the threshold tuning for predicting PTB showed relatively lower performance than the results based on the default threshold of 0.5 in Table 1. Therefore, the result for predicting PTB with the highest sensitivity in this study was 0.64, with an accuracy of 0.74 and a specificity of 0.75.
[Figure 2 about here]