Summary and Aim of the Paper
Bootstrap iteration (BSITER) scores are often used in the P300 CIT literature to make classification decisions about individual participants. The BSITER score is typically derived using 100 or 1,000 iterations. Although random sampling is inherent to the calculation of these scores, it is generally assumed that this variability does not meaningfully impact classification performance. However, this assumption has never been tested, to our knowledge, particularly regarding the 100 iterations version of the bootstrap test. Although Rosenfeld et al., (2017b) observed robust correlations between the 100, 1,000 and 10,000 iterations tests, which suggests high precision across the majority of subjects, a correlation approach is not a thorough evaluation of the internal validity of the 100 iterations test. Therefore, we opted for a more rigorous alternative to evaluate the precision of the 100 iterations bootstrap test in the current report.
To evaluate the precision of the 100 iterations test, we repeated it 100 times per participant. Because each test yields a BSITER score, our procedure (henceforth the “repetition bootstrap” or rBS) produced 100 scores per participant, allowing us to observe and report their variability within each individual subject.