Note: Test statistics, p-values, and effect sizes are for paired, two-sided Wilcoxon signed-rank tests. Significant p-values (p < 0.05) are shown in bold.
FIGURE CAPTIONS Figure 1. Conceptual diagram of the two learning objectives (LOs) associated with uncertainty communication and foundational ecological forecasting concepts taught in Macrosystems EDDIE Module 8: “Using Ecological Forecasts to Guide Decision-Making.” Within the larger circles, each bubble shows the corresponding assessment questions which tested the effectiveness of the module in meeting these LOs. Descriptions of the assessment questions are in Table 3 and the full assessment questions can be found in Appendix S1: Tables S1-2. Ecological forecast is abbreviated as EF.Figure 2. Screenshots of activities from the module R Shiny application showing A) Activity B, in which students (1) make decisions about how to manage a drinking water reservoir, using two different forecast visualizations (2a and 2b, which were shown separately to students within the application), while weighing the consequences on multiple objectives, such as maintaining good drinking water quality, preserving ecological health, maximizing economic benefit, and ensuring swimmer safety (3). In Activity C, students chose a forecast user and customized a visualization for that particular users’ decision needs; for example, (B) shows a visualization and decision chosen for a swimmer, while (C) shows a visualization and decision chosen for a local policymaker.Figure 3. Percentage of students who answered multiple-choice questions correctly in the pre- and post-module surveys. Asterisks (***) indicate a statistically significant difference between the pre- and post-module survey according to a Wilcoxon signed-rank test (p < 0.001). Colors of the bars correspond to LO1: Foundational ecological forecasting (EF) concepts (green) and LO2: Uncertainty Communication (orange; Figure 1, Table 3). A description of questions can be found in Table 3 and Appendix S1: Table S3.Figure 4.Percentage of students who identified a) different benefits of ecological forecasting (Q3), and b) different ways to visualize uncertainty (Q4) in the pre- and post-module responses. These responses correspond to the c) total number of benefits identified by individual students in Q3, and d) the total number of ways to visualize uncertainty identified by individual students in Q4. Students were also given the option to state “I don’t know,” represented as “IDK” in panels a and b. The categories listed on the x-axis in panels a and b were determined through the methods outlined in Appendix S1: Text S1 and are listed in Table 4, respectively. Asterisks (***) indicate a significant difference (p < 0.001) between the pre- and post-survey responses according to paired Wilcoxon signed-rank tests. A full description of questions can be found in Appendix S1: Table S4.Figure 5. Pre- and post-survey results across the two LOs showing a) the total percent correct for all students across each category, and b) the change in the number of correct answers for each student after module completion relative to each student’s percent correct before taking the module. Color in (b) corresponds to the number correct on the pre-survey only and points are jittered to improve legibility of individual points. Note that the number of questions corresponding to each LO varied, with three questions assessing foundational ecological forecasting concepts and five questions assessing uncertainty communication concepts. The percentage of correct answers are standardized to the total number of questions per LO in (a), while the number of questions answered correctly is shown in (b). “Foundational” corresponds to LO1: Ecological Forecasting Concepts, and “Communication” corresponds to LO2: Uncertainty Communication.