Results
Our assessment data indicate that student understanding of foundational
ecological forecasting and uncertainty communication concepts increased
after module completion (Figures 3, 4). Specifically, students
identified significantly more ways to communicate uncertainty in a
forecast, and were significantly more likely to identify
‘decision-making’ and ‘prediction’ as important benefits of ecological
forecasts (Figure 4). Across the two LOs, students scored higher in
foundational ecological forecasting concepts after completing the
module, but showed strong growth in understanding both ecological
forecasting and uncertainty communication concepts from pre- to
post-module surveys (Figure 5).Student
understanding of uncertainty communication
Students were more likely to correctly identify and describe multiple
ways to communicate uncertainty in forecast visualizations after
completion of the module (Figures 3, 4b, 4d; Table 4). The percent of
students able to correctly distinguish among different ways to visualize
forecast uncertainty increased from 33% pre-module to 60% after module
completion (p < 0.001; Figure 3e, Table 4). In addition,
students were significantly more likely to identify and interpret
differences between two visualizations that had varying representations
of uncertainty after module completion (33% of students pre-module vs.
49% post-module; p <0.001; Figure 3d, Table 4). We also
observed post-module increases in the percent of students who correctly
interpreted a forecast visualization (49% pre-module; 52% post-module)
and matched a forecast visualization with a forecast user decision need
(42% pre-module; 49% post-module), but these increases were not
statistically significant (p = 0.34 and 0.28, respectively; Figure 3c,
3f; Table 4).
Students also showed increased comprehension of uncertainty
communication after module completion in our qualitative assessment.
When asked to describe two different ways to communicate uncertainty,
the number of correct answers students provided increased from an
average of 0.2 ± 0.5 on the pre-survey (S.D.) to 1.1 ± 0.8 on the
post-survey (p < 0.001; Figure 4d, Table 4). Specifically, the
number of students who identified numeric, visual, or probabilistic
methods to visualize uncertainty increased significantly after module
completion (all p < 0.001; Figure 4b), while student responses
which included text or multiple predictions increased, but not
significantly (both p = 0.18; Figure 4b, Table 4). The number of
students who identified ‘numeric’ methods to visualize uncertainty
(e.g., ‘standard deviation’, ‘statistical confidence intervals’,
‘intervals’, or ‘ranges’) increased from 8% on the pre-survey to 29%
on the post-survey (p < 0.001; Figure 4b, Table 4).
Additionally, student responses which included ‘visual’ descriptions of
uncertainty (e.g., ‘boxplots’, ‘shaded area around a line’, ‘error
bars’) increased from 8% to 41% (p < 0.001; Figure 4b, Table
4), while responses including ‘probabilistic’ methods to visualize
uncertainty (e.g., ‘percentage likelihood’, ‘probability of exceeding a
certain threshold’) increased from 2% to 15% from the pre-survey to
the post-survey (p < 0.001; Figure 4b, Table 4).
Several keywords were significantly more prevalent in post-module than
pre-module responses to the question about identifying uncertainty
communication methods (Q4, Table 3). None of our keywords were
identified in the pre-survey responses, but 4% (n = 9) of students
included the word ‘icon’ and 13% (n = 31) of students included ‘color’
in their answers when asked to identify ways of communicating
uncertainty in the post-survey. In addition, 11% (n = 27) of students
named ‘forecast output’ or ‘forecast index’ in their post-survey
responses. However, we note that of the 27 students who listed ‘forecast
index/output’ as a way to visualize uncertainty in their post-module
response, only three students correctly described these terms in the
context of uncertainty communication. For example, one student
explicitly described what they meant by a ‘forecast index/output’
(“Forecasts can be visualized through figures that show forecast
output, which have direct information on it, and forecast index, which
contains a meaningful threshold that is based off what decision is being
made”), demonstrating a deeper understanding than students who
mentioned forecast index/output without a definition (e.g., “You can
visualize uncertainty with a forecast index and a forecast output”).