Module caveats and opportunities for future use
The impact of this module on student learning of decision science and uncertainty communication skills could be improved in several aspects. First, during module development, we intentionally introduced students to a single method of structured decision-making (PROACT) and a limited number of uncertainty communication methods (i.e., visual, numeric, probabilistic) to provide a simplified introduction to the decision and visualization sciences. While students showed successful understanding of the PrOACT tool (Appendix S1: Figure S4), the addition of other decision support components, including solutions-oriented decision-making theory (Deitrick and Wentz 2015) or additional methods of structured decision-making (Gregory et al. 2012), would increase students’ breadth of understanding of decision science. Inclusion of a broader variety of decision support concepts could also lead to improved performance on the decision-related questions (e.g., Figure 3c, 3f). Second, allowing students more control over visualizations within the Shiny app (e.g., additional visualization or personalization options, inclusion of R-based coding activities) would likely increase students’ visualization literacy (Huron et al. 2014, Alper et al. 2017, Börner et al. 2016, Börner et al. 2019). Third, due to time constraints, our module asked students to imagine what type of forecast visualization would best meet different end users’ needs, rather than asking them to actively engage in co-development of visualizations with different forecast users, which would likely increase the utility of the visualization (Raftery 2016, Padilla et al. 2017b, Gerst et al. 2019). Fourth, shifting the focus of the case study in Activity B to be customizable for specific, nearby ecosystems which are directly relevant to students’ everyday lives could potentially increase engagement and student learning (Cid and Pouyat 2013, Henri et al. 2022, Vance-Chalcraft and Osborne Jelks, 2022). For example, students living in areas where wildfires are common may be more engaged analyzing a case study presenting a decision-making scenario on wildfire forecasts. While we recognize the value in including additional content on uncertainty communication, decision science, and ecological forecasting, we note that expanding the module may make it less feasible for instructors to add into their ecology curricula.
Several caveats should be considered when interpreting the assessment results from our module. First, we used a pre- and post-module methodology because instructors were unable to divide their classes into treatment and control groups for instruction. Second, there were many factors which were not held constant across the classrooms that tested our module, including student experience level, instructor experience level, classroom size, institutional familiarity with forecasting and others that could influence the effect of the module on individual student learning. Third, due to the length of the module and limitations of our assessment survey, our analysis provides only a limited understanding of students’ knowledge gain. Future longer-term assessments are needed to assess student growth over a longer duration of time.