Our approach
In this paper, we discuss a method of “repeated bootstrapping” (rBS) where a number of iterations is chosen, and the resampling is repeated a specified number of times. In the traditional bootstrapping method, a diagnosis is made after a specified number of iterations (e.g. 100 iterations). However, if the test were to be repeated, the same diagnosis may not be recommended given the random sampling involved in the bootstrapping procedure11Note that this limitation is also relevant for any number of diagnostic tests that are performed on the same individual at different points in time or by different services or technologies.. When considering the efficacy of a diagnostic test it is vital to report its precision, or how similar the results of the test are upon re-testing. To our knowledge, no previous study has evaluated the repeatability of the 100 iterations bootstrap test in the P300-based concealed information detection literature, so that is what we aim to do here. Additionally, the rBS technique could be a useful tool to calculate confidence intervals for diagnostic metrics like the BSITER score, improving upon standard methods in our field similar to ERP data quality metrics such as the one recently proposed by Luck et al. (2021).