Results
Our assessment data indicate that student understanding of foundational ecological forecasting and uncertainty communication concepts increased after module completion (Figures 3, 4). Specifically, students identified significantly more ways to communicate uncertainty in a forecast, and were significantly more likely to identify ‘decision-making’ and ‘prediction’ as important benefits of ecological forecasts (Figure 4). Across the two LOs, students scored higher in foundational ecological forecasting concepts after completing the module, but showed strong growth in understanding both ecological forecasting and uncertainty communication concepts from pre- to post-module surveys (Figure 5).Student understanding of uncertainty communication
Students were more likely to correctly identify and describe multiple ways to communicate uncertainty in forecast visualizations after completion of the module (Figures 3, 4b, 4d; Table 4). The percent of students able to correctly distinguish among different ways to visualize forecast uncertainty increased from 33% pre-module to 60% after module completion (p < 0.001; Figure 3e, Table 4). In addition, students were significantly more likely to identify and interpret differences between two visualizations that had varying representations of uncertainty after module completion (33% of students pre-module vs. 49% post-module; p <0.001; Figure 3d, Table 4). We also observed post-module increases in the percent of students who correctly interpreted a forecast visualization (49% pre-module; 52% post-module) and matched a forecast visualization with a forecast user decision need (42% pre-module; 49% post-module), but these increases were not statistically significant (p = 0.34 and 0.28, respectively; Figure 3c, 3f; Table 4).
Students also showed increased comprehension of uncertainty communication after module completion in our qualitative assessment. When asked to describe two different ways to communicate uncertainty, the number of correct answers students provided increased from an average of 0.2 ± 0.5 on the pre-survey (S.D.) to 1.1 ± 0.8 on the post-survey (p < 0.001; Figure 4d, Table 4). Specifically, the number of students who identified numeric, visual, or probabilistic methods to visualize uncertainty increased significantly after module completion (all p < 0.001; Figure 4b), while student responses which included text or multiple predictions increased, but not significantly (both p = 0.18; Figure 4b, Table 4). The number of students who identified ‘numeric’ methods to visualize uncertainty (e.g., ‘standard deviation’, ‘statistical confidence intervals’, ‘intervals’, or ‘ranges’) increased from 8% on the pre-survey to 29% on the post-survey (p < 0.001; Figure 4b, Table 4). Additionally, student responses which included ‘visual’ descriptions of uncertainty (e.g., ‘boxplots’, ‘shaded area around a line’, ‘error bars’) increased from 8% to 41% (p < 0.001; Figure 4b, Table 4), while responses including ‘probabilistic’ methods to visualize uncertainty (e.g., ‘percentage likelihood’, ‘probability of exceeding a certain threshold’) increased from 2% to 15% from the pre-survey to the post-survey (p < 0.001; Figure 4b, Table 4).
Several keywords were significantly more prevalent in post-module than pre-module responses to the question about identifying uncertainty communication methods (Q4, Table 3). None of our keywords were identified in the pre-survey responses, but 4% (n = 9) of students included the word ‘icon’ and 13% (n = 31) of students included ‘color’ in their answers when asked to identify ways of communicating uncertainty in the post-survey. In addition, 11% (n = 27) of students named ‘forecast output’ or ‘forecast index’ in their post-survey responses. However, we note that of the 27 students who listed ‘forecast index/output’ as a way to visualize uncertainty in their post-module response, only three students correctly described these terms in the context of uncertainty communication. For example, one student explicitly described what they meant by a ‘forecast index/output’ (“Forecasts can be visualized through figures that show forecast output, which have direct information on it, and forecast index, which contains a meaningful threshold that is based off what decision is being made”), demonstrating a deeper understanding than students who mentioned forecast index/output without a definition (e.g., “You can visualize uncertainty with a forecast index and a forecast output”).