Summary and Aim of the Paper
Bootstrap iteration (BSITER) scores are often used in the P300 CIT
literature to make classification decisions about individual
participants. The BSITER score is typically derived using 100 or 1,000
iterations. Although random sampling is inherent to the calculation of
these scores, it is generally assumed that this variability does not
meaningfully impact classification performance. However, this assumption
has never been tested, to our knowledge, particularly regarding the 100
iterations version of the bootstrap test. Although Rosenfeld et al.,
(2017b) observed robust correlations between the 100, 1,000 and 10,000
iterations tests, which suggests high precision across the majority of
subjects, a correlation approach is not a thorough evaluation of the
internal validity of the 100 iterations test. Therefore, we opted for a
more rigorous alternative to evaluate the precision of the 100
iterations bootstrap test in the current report.
To evaluate the precision of the 100 iterations test, we repeated it 100
times per participant. Because each test yields a BSITER score, our
procedure (henceforth the “repetition bootstrap” or rBS) produced 100
scores per participant, allowing us to observe and report their
variability within each individual subject.