Performance across ecological forecasting and uncertainty communication learning objectives
Student performance improved from pre- to post-module in correctly answering questions on both foundational ecological forecasting concepts (LO1) and uncertainty communication (LO2; Figure 5). The increase in performance was stronger for students who scored lower on the pre-survey (Figure 5b). Students were more likely to correctly answer questions on ecological forecasting concepts than questions on uncertainty communication concepts on the pre-module assessment (Figure 5a). Both LOs showed strong growth after module completion, with many students who answered zero questions correctly on the pre-survey answering all questions correctly on the post-survey in both categories (Figure 5b). More students scored 100% (all LO-specific questions answered correctly) on foundational ecological forecasting concepts than on uncertainty communication both before and after module completion, though we note that the number of questions in the two categories differed (Figure 1).

Discussion

Our results indicate that completion of a 3-hour module can significantly improve undergraduate ecology students’ understanding of uncertainty communication and ecological forecasting. While the percentage of correct answers increased for all assessment questions after module completion, students were more likely to perform higher on foundational ecological forecasting concepts than uncertainty communication concepts prior to module completion. Higher initial performance on ecological forecasting questions may be because students were more familiar with ecological forecasting concepts relative to uncertainty communication concepts before completing the module. However, students showed more growth in describing multiple ways to communicate uncertainty than in identifying benefits of ecological forecasting (Figure 4c, 4d). Below, we explore the implications of our results for undergraduate education in uncertainty communication, ecological forecasting, and ecology broadly.Improved uncertainty communication skills and implications for visualization literacy
Students identified significantly more ways to communicate uncertainty following module completion, indicating that the module introduced students to a toolbox of approaches for developing and understanding uncertainty in ecological visualizations. Before completing our module, the majority (85%) of students were unable to describe any ways to communicate uncertainty, while after module completion 72% were able to describe one or more ways to communicate uncertainty (Figure 4d). Being able to identify and describe multiple methods for uncertainty communication is an important skill, as the method used to visualize uncertainty can have a substantial effect on user comprehension and decision-making (Nadav-Greenberg et al. 2008, Ramos et al. 2013, Cheong et al. 2016, McKenzie et al. 2016, Kinkeldey et al. 2017). For example, using summary visualizations (e.g., boxplots) can decrease users’ cognitive load and increase the speed of decision-making, but are more likely to lead to misinterpretation (Ruginski et al. 2016, Correll et al. 2018). In contrast, ensemble-based visualizations (i.e., forecast visualizations that show all possible model outputs) may provide users with more information about the whole spread of uncertainty, but viewers may overweight certain ensemble members, leading to inconsistent decision-making (Padilla et al. 2017a). Given that there is no single “best” visualization method for uncertainty communication due to differences in decision-making needs (Spiegelhalter et al. 2011), the ability to create a variety of visualization options and adapt visualizations based on forecast user feedback is critical for developing effective uncertainty visualizations.
All of the methods for uncertainty communication included in the students’ post-module responses are aligned with current state-of-the-art practices for uncertainty communication in visualization science. “Visual” and “numeric,” the two uncertainty communication methods most commonly reported by students in post-module responses (Figure 4b), mirror the two key uncertainty representation techniques (“visualization” and “quantification”) identified in a recent review (Kamal et al. 2021). Probabilistic methods, which were also significantly more common in student responses following module completion, can decrease cognitive load and increase use and understanding of visualizations for decision support (Kox et al. 2018). Some students (n = 8) also identified “text” as a useful method for uncertainty communication (Figure 4b), but “text” was almost always (7/8 students) reported in addition to another form of communication (e.g., “visual,” “numeric,” or “probability”), following literature which shows that text is most useful for explaining and providing context for visualizations (Carr et al. 2018). Additionally, common keywords throughout student responses included “color” and “icon.” Thoughtful use of color palettes (e.g., by using discrete rather than continuous color palettes) has been shown to be a powerful tool in representing ranges of uncertainty (Padilla et al. 2017b, Correll et al. 2018). Similarly, the use of icons or symbols has been shown to improve user understanding and usability of decision support tools in diverse settings (Galesic et al. 2009, Garcia-Retamero et al. 2010, Zikmund-Fisher et al. 2014, Kamal et al. 2021), potentially by decreasing the cognitive load required to interpret the communication. While neither text, icons, or numbers alone are typically most effective in scientific communication of complex ideas (Larkin and Simon 1987, Tait et al. 2010), the student post-module responses are reflective of a common theme in the visualization literature that using multiple communication forms increases user comprehension and confidence in decision-making (Fagerlin et al. 2005, Spiegelhalter et al. 2011). Ultimately, the module increased students’ ability to communicate uncertainty using multiple approaches, a key skill for developing decision support tools for forecast users.
Overall, the module shows promise for increasing visualization literacy and introducing much needed skills in uncertainty communication to undergraduate ecology students. Most students who completed this module had little to no prior experience with uncertainty communication, but showed substantial improvement in performance after module completion, indicating that a 3-hour module can help build these critical skills (Figure 5). Students’ lack of previous exposure to uncertainty communication is likely because undergraduate ecology classes do not currently include uncertainty communication and visualization literacy topics as often as, e.g., ecological modeling and prediction (Willson et al. 2022). Because the communication of uncertainty is just as important as the quantification of uncertainty in forecasts for ensuring that forecast visualizations guide end users’ decision-making, it is critical that science communication and visualization science, including incorporation of end user decision needs in visualization development, are included in ecological forecasting training (Robinson et al. 2012, Schwartz et al. 2017, Eisenhauer et al. 2021).
Overall, given the importance of uncertainty communication not only in ecology, but across scientific disciplines broadly (e.g., medicine, meteorology, economics; Tait et al. 2010, Ferstl et al. 2017, Wesslen et al. 2022), improving students’ ability to interpret and produce uncertainty visualizations may help enable student participation in a variety of scientific disciplines. Moreover, providing students with improved visualization literacy and uncertainty communication skills will yield a more data-literate population, regardless of students’ future careers. This module provides an important first step for incorporating visualization literacy coursework across undergraduate curricula broadly and initiating training in critical visualization interpretation and communication skills.Increase in student understanding of foundational ecological forecasting concepts
In addition to expanding students’ uncertainty communication skills, completion of the 3-hour module improved students’ understanding of foundational ecological forecasting concepts. Following module completion, students were significantly more likely to correctly define an ecological forecast as a future prediction of environmental conditions with uncertainty (Figure 3a; Appendix S1: Table S1). Overall, developing a common definition of “forecast” is important for furthering the field of ecological forecasting, as having common definitions enables meaningful discourse on topics within and across disciplines, providing a scaffold for interdisciplinary work to address complex socioecological problems (Lélé and Norgaard 2005, Robinson et al. 2012). Given that ecological forecasting is an emerging field (Woelmer et al. 2021), codifying definitions in training materials enables undergraduate ecology students to more effectively discuss and learn forecasting topics.
Student understanding of the benefits of ecological forecasting also significantly increased after module completion. Specifically, we found a significant increase in the number of students who identified ‘decision-making’ and ‘prediction’ as benefits, but saw only a minimal increase for ‘policy,’ and decreases for ‘management’ and ‘understanding.’ Since ‘decision-making’ and ‘prediction’ were emphasized throughout the module, it is unsurprising that these two benefits of ecological forecasting were most commonly provided in student responses. However, the small decrease in responses related to management and understanding is surprising, and may indicate that the module did not sufficiently focus on the benefits of forecasts for management or ecological understanding. For example, while Activity B used a management-centered role-playing example, the activity was primarily focused on the effect of visualization type on decision-making, rather than how forecasts could be integrated into management workflows. Similarly, the module did not emphasize how ecological forecasts can advance understanding of ecosystems and testing of ecological theory or be integrated into policy-making decisions, leaving an opportunity to bolster these forecast applications in future iterations of the module. Alternatively, it is possible that students who identified “decision-making” as a benefit of forecasts may have had policy or management decisions in mind, but not specifically stated this.
We note that students’ ability to correctly identify that uncertainty should increase the further into the future a forecast is made (Q2) showed only marginal growth (Figure 3b), leaving room for improvement in teaching this foundational concept of ecological forecasting (sensu Dietze et al. 2018). To complement this module and provide additional training in foundational ecological forecasting concepts, we suggest pairing this module with other Macrosystems EDDIE modules (Module 5: Introduction to Ecological Forecasting, Moore et al. 2022b; Module 6: Understanding Uncertainty in Ecological Forecasts, Moore et al. 2021; or Module 7: Using Data to Improve Ecological Forecasts, Lofton et al. 2022).
Integration of decision support concepts into ecology curricula Integrating applied decision-making concepts into ecological forecasting and uncertainty communication lessons heeds a widespread call to make ecological research more societally relevant (e.g., Belovsky et al. 2004, Ruhl et al. 2022). Training that incorporates components of translational ecology (e.g., science communication, end user engagement, structured decision-making, multidisciplinary training) has long been recommended for ecologists at all career stages (Robinson et al. 2012, Schwartz et al. 2017, Eisenhauer et al. 2021), but resources targeted at the undergraduate level have been lacking (Bakermans and Pfeifer 2018). Our module aims to close this gap by incorporating complex water management decision-making scenarios with multiple end users to engage students in solving real-world problems, while also developing visualization literacy skills. While we did not quantitatively assess the engagement of students in the module, we received open-ended positive feedback from many students. Students reported that the module was enjoyable and important (“I really enjoyed looking at decision analysis in an ecology class;” “I think it’s very important to talk about this in science classes”). Additionally, student responses suggest that they found that the module was interesting (“This was informative and a really interesting way for me to realize the actual impacts of ecological forecasting”) and novel (“It was helpful because I came in with no information”).
Ultimately, introducing students to applications of ecological forecasting for real-world decision-making may help recruit students to work in this subdiscipline, as well as highlight the importance of using ecology to produce actionable tools to address societal problems (e.g., Enquist et al. 2017). Many ecological forecasters have already begun to integrate decision support and uncertainty communication components into their forecasts by making forecasts which are actionable, useful, and targeted towards forecast users (e.g., Gerst et al. 2019, Turner et al. 2020, Jackson-Blake et al. 2022). Our experience with this module indicates that even a short (3-hour) exposure to decision support and uncertainty communication concepts can increase students’ understanding of potential applications and benefits of ecological forecasting.