Module caveats and opportunities for future use
The impact of this module on student learning of decision science and
uncertainty communication skills could be improved in several aspects.
First, during module development, we intentionally introduced students
to a single method of structured decision-making (PROACT) and a limited
number of uncertainty communication methods (i.e., visual, numeric,
probabilistic) to provide a simplified introduction to the decision and
visualization sciences. While students showed successful understanding
of the PrOACT tool (Appendix S1: Figure S4), the addition of other
decision support components, including solutions-oriented
decision-making theory (Deitrick and Wentz 2015) or additional methods
of structured decision-making (Gregory et al. 2012), would increase
students’ breadth of understanding of decision science. Inclusion of a
broader variety of decision support concepts could also lead to improved
performance on the decision-related questions (e.g., Figure 3c, 3f).
Second, allowing students more control over visualizations within the
Shiny app (e.g., additional visualization or personalization options,
inclusion of R-based coding activities) would likely increase students’
visualization literacy (Huron et al. 2014, Alper et al. 2017, Börner et
al. 2016, Börner et al. 2019). Third, due to time constraints, our
module asked students to imagine what type of forecast visualization
would best meet different end users’ needs, rather than asking them to
actively engage in co-development of visualizations with different
forecast users, which would likely increase the utility of the
visualization (Raftery 2016, Padilla et al. 2017b, Gerst et al. 2019).
Fourth, shifting the focus of the case study in Activity B to be
customizable for specific, nearby ecosystems which are directly relevant
to students’ everyday lives could potentially increase engagement and
student learning (Cid and Pouyat 2013, Henri et al. 2022,
Vance-Chalcraft and Osborne Jelks, 2022). For example, students living
in areas where wildfires are common may be more engaged analyzing a case
study presenting a decision-making scenario on wildfire forecasts. While
we recognize the value in including additional content on uncertainty
communication, decision science, and ecological forecasting, we note
that expanding the module may make it less feasible for instructors to
add into their ecology curricula.
Several caveats should be considered when interpreting the assessment
results from our module. First, we used a pre- and post-module
methodology because instructors were unable to divide their classes into
treatment and control groups for instruction. Second, there were many
factors which were not held constant across the classrooms that tested
our module, including student experience level, instructor experience
level, classroom size, institutional familiarity with forecasting and
others that could influence the effect of the module on individual
student learning. Third, due to the length of the module and limitations
of our assessment survey, our analysis provides only a limited
understanding of students’ knowledge gain. Future longer-term
assessments are needed to assess student growth over a longer duration
of time.