Note: Test statistics, p-values, and effect sizes are for paired,
two-sided Wilcoxon signed-rank tests. Significant p-values (p
< 0.05) are shown in bold.
FIGURE CAPTIONS
Figure 1. Conceptual diagram of the two learning objectives (LOs)
associated with uncertainty communication and foundational ecological
forecasting concepts taught in Macrosystems EDDIE Module 8: “Using
Ecological Forecasts to Guide Decision-Making.” Within the larger
circles, each bubble shows the corresponding assessment questions which
tested the effectiveness of the module in meeting these LOs.
Descriptions of the assessment questions are in Table 3 and the full
assessment questions can be found in Appendix S1: Tables S1-2.
Ecological forecast is abbreviated as
EF.Figure 2. Screenshots of activities from
the module R Shiny application showing A) Activity B, in which students
(1) make decisions about how to manage a drinking water reservoir, using
two different forecast visualizations (2a and 2b, which were shown
separately to students within the application), while weighing the
consequences on multiple objectives, such as maintaining good drinking
water quality, preserving ecological health, maximizing economic
benefit, and ensuring swimmer safety (3). In Activity C, students chose
a forecast user and customized a visualization for that particular
users’ decision needs; for example, (B) shows a visualization and
decision chosen for a swimmer, while (C) shows a visualization and
decision chosen for a local policymaker.Figure
3. Percentage of students who answered multiple-choice questions
correctly in the pre- and post-module surveys. Asterisks (***) indicate
a statistically significant difference between the pre- and post-module
survey according to a Wilcoxon signed-rank test (p < 0.001).
Colors of the bars correspond to LO1: Foundational ecological
forecasting (EF) concepts (green) and LO2: Uncertainty Communication
(orange; Figure 1, Table 3). A description of questions can be found in
Table 3 and Appendix S1: Table S3.Figure 4.Percentage of students who identified a) different benefits of
ecological forecasting (Q3), and b) different ways to visualize
uncertainty (Q4) in the pre- and post-module responses. These responses
correspond to the c) total number of benefits identified by individual
students in Q3, and d) the total number of ways to visualize uncertainty
identified by individual students in Q4. Students were also given the
option to state “I don’t know,” represented as “IDK” in panels a and
b. The categories listed on the x-axis in panels a and b were determined
through the methods outlined in Appendix S1: Text S1 and are listed in
Table 4, respectively. Asterisks (***) indicate a significant difference
(p < 0.001) between the pre- and post-survey responses
according to paired Wilcoxon signed-rank tests. A full description of
questions can be found in Appendix S1: Table
S4.Figure 5. Pre- and post-survey results
across the two LOs showing a) the total percent correct for all students
across each category, and b) the change in the number of correct answers
for each student after module completion relative to each student’s
percent correct before taking the module. Color in (b) corresponds to
the number correct on the pre-survey only and points are jittered to
improve legibility of individual points. Note that the number of
questions corresponding to each LO varied, with three questions
assessing foundational ecological forecasting concepts and five
questions assessing uncertainty communication concepts. The percentage
of correct answers are standardized to the total number of questions per
LO in (a), while the number of questions answered correctly is shown in
(b). “Foundational” corresponds to LO1: Ecological Forecasting
Concepts, and “Communication” corresponds to LO2: Uncertainty
Communication.