Table 5. Comparison of overall accuracy and Cohen’s kappa coefficient of operators’ ICT kits based on rRT-PCR Ct value categories.
Table 5: Comparative analysis of overall accuracy and Cohen’s kappa coefficient among the operators’ ICT kits based on their Ct value categories. N represents the total number of samples analyzed, while n represents only samples that tested positive. Thirty-seven (37) samples tested negative.
Discussion
This study evaluated the diagnostic performance characteristics of these brands of COVID-19 ICT test kits on the Ghanaian market, namely, DG Rapid (DGT), SD Rapid (SD), and SS Rapid (SS). Rapid antigen tests offer several advantages, including affordability, faster turnaround time, and the ability to diagnose patients at their point-of-care. These advantages are essential and critical, especially in resource-limited settings, where rRT-PCR testing may not be readily available. This study demonstrated that DG Rapid and SD Rapid antigen test kits performed relatively better in detecting SARS-CoV-2 than SS Rapid.
The findings for DG Rapid and SD Rapid are consistent with previous studies that have reported the effectiveness of rapid antigen tests in detecting COVID-19 (13). These studies showed a relatively lower sensitivity with SARS-CoV-2 antigen rapid diagnostic test kits (Ag-RDTs) compared to the clinical reference standard, which is the real-time reverse transcriptase- polymerase chain reaction (rRT-PCR). A recent systematic review and meta-analysis that evaluated the accuracy of commercially available SARS-CoV-2 Ag-RDTs revealed a pooled sensitivity of 71.2% (13). A sensitivity of 70% and 59% (95% CI) respectively was observed in 262 study participants in Uganda and Cameroon, respectively, using the SD Rapid RDT (14,15). In Cameroon, the RDTs’ sensitivity of 59% (95% CI) increased to 69% (95% CI) when only symptomatic participants were considered. Another study conducted at a teaching hospital in northern Ghana evaluated the sensitivity of the SD Rapid RDT with 193 participants as 64% (95% CI) (16). Our study, however, found a sensitivity of 74% (95% CI) for SD Rapid RDT, which is comparable to what was observed in Uganda and Cameroon. Moreover, among the three brands, DG Rapid demonstrated a higher sensitivity of 79% (95% CI), followed by SD Rapid with 74% (95% CI). However, the SS Rapid RDT demonstrated a lower sensitivity of 50% (95% CI) compared to the other two brands. This lower sensitivity for SS Rapid may limit its usefulness as a standalone diagnostic tool, as it may lead to false negative results. False negative results may result in a delay in diagnosis, thereby increasing the risk of virus transmission. It is noteworthy that these values of sensitivity observed in our study are much below the performance reported by the manufacturers (DG Rapid, 2022; SD Rapid, 2020; SS Rapid, 2021). Possible explanations for the lower sensitivity observed could be due to factors such as variations in the batch of ICT kits used, variations in the concentration of extracted antigens, differences in processing techniques, and variations in the storage conditions of the kits, especially in the market (18).
In this study, the specificity was 100% (95% CI), which is comparable to the 100% (95% CI) stated by the manufacturers but higher than the 92% (95% CI) documented in the studies conducted in both Uganda and Cameroon. The possible explanations given for the lower specificity in these other studies were cross-reacting antibodies from previous infections or variations in environmental testing temperatures (24°C - 37°C) in the general wards and the COVID-19 isolation center where the tests were carried out (16). The high specificity exhibited by all three test kits in our study is an important attribute as it ensures that individuals without the virus are correctly identified, reducing the risk of false-positive results. False positives can lead to unwarranted quarantine, isolation, and treatment, with significant social and economic consequences.
Regarding overall accuracy, this study reported values of 89% (for DG Rapid), 87% (SD Rapid), and 75% (SS Rapid) at 95% CI for the three SARS-CoV-2 ICT kits, which are lower than the manufacturers’ claims of 99%, 98%, and 99%, respectively. However, it is essential to recognize that overall accuracy can vary with disease prevalence, making it less reliable as a single summary measure of a test’s validity. The prevalence-dependent nature of overall accuracy introduces challenges, leading to warnings against its use. Estimates of overall accuracy can be misleading when obtained from populations with significantly different disease prevalence from the target population where the test is intended for application (19).
To assess the agreement between the ICT kits and rRT-PCR, Cohen’s kappa coefficient was employed. Specifically, it was used to determine the level of agreement between the performance of the ICT kits and rRT-PCR. Our findings indicated that only DG Rapid demonstrated excellent agreement, while SD Rapid and SS Rapid exhibited good agreement compared to rRT-PCR. This observation could be because DG Rapid demonstrated a higher sensitivity of 79% and a higher NPV of 82% compared to the other brands. Other performance indicators, specifically, specificity and PPV, were the same (100%) for all three brands and thus did not have an impact on the kappa value calculation. In a study conducted in India, Cohen’s kappa calculated for SD Rapid and rRT-PCR showed a good agreement, with a Cohen’s kappa of 64.4%(20). Another study conducted in Ethiopia found that a SARS-CoV-2 antigen rapid test kit and rRT-PCR had a kappa value of agreement of 73.5% which indicates good agreement between the two tests (21). While we could not find specific research articles on DG and SS Rapid kits, the evidence from the study conducted in both India and Ethiopia suggests a consistent trend of good agreement between SARS-CoV-2 antigen rapid test kits such as SD Rapid and rRT-PCR.
By convention, a lower Ct value signifies a higher viral load, while a higher Ct value suggests a lower viral load (13). All three brands demonstrated better detection limits for higher viral load (Ct values ≤29.99), which is often the case in the pre-symptomatic phase (1–3 days before the onset of symptoms) and the early symptomatic phase (during the first 5 – 7 days of illness) of SARS-CoV-2 infection (22). Conversely, they displayed less favorable detection limits for Ct values >35.00.. Consequently, the DG Rapid ICT kits exhibit a lower detection threshold when Ct values exceed 35.00, despite their excellent agreement with rRT-PCR. It is essential to acknowledge that our study utilized frozen archived nasopharyngeal samples, which may not represent the current situation accurately. The performance of these antigen test kits can be influenced by factors such as viral load and the type of specimen used for testing(13). Therefore, it is crucial to evaluate the performance of these rapid antigen test kits on fresh samples and in real-life settings.