A user assessment of decision support tools
-
Upload
soil-and-water-conservation-society -
Category
Environment
-
view
236 -
download
0
description
Transcript of A user assessment of decision support tools
![Page 1: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/1.jpg)
A User Assessment of Decision Support Tools for Addressing Nonpoint Source Pollution in the Saginaw Basin Watershed:
Environmental Learning Using Computer Interactive Decisions (ELUCID) and the Great Lakes Watershed Management System (GLWMS)
Laura Young, Research AssistantSaichon Seedang, Research Specialist
Michigan State University Institute of Water Research
69th Soil Water Conservation Society International Annual ConferenceMaking Waves in ConservationJuly 29, Lombard, Illinois
![Page 2: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/2.jpg)
CONTEXT & SYSTEMS OVERVIEW
![Page 3: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/3.jpg)
Context
Awarded Great Lakes Restoration Initiative (GLRI) grant
Enhance technical assistance
Accelerate BMP implementation
Reduce non‐point source pollution
![Page 4: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/4.jpg)
Trainings
• May 2014• Overview and walkthrough of systems• Tutorials• Evaluation
![Page 5: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/5.jpg)
ELUCID
Environmental Learning Using Computer Interactive Decisions
Developed with input from various stakeholders
Flint River Watershed
![Page 6: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/6.jpg)
ELUCID
Engage and inform various users groupsField staff, watershed groups, teachers, the public
![Page 7: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/7.jpg)
ELUCIDSubwatershed Prioritization Field‐level Prioritization
![Page 8: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/8.jpg)
ELUCID
Seamlessly linked with other systemsEPA databases, GLWMS
![Page 9: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/9.jpg)
Great Lakes Watershed Management System
Evolved through various multi‐partner efforts
Four GLRI priority watersheds (Saginaw Bay)
![Page 10: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/10.jpg)
Great Lakes Watershed Management System
Subwatershed prioritization for NPS pollutants
![Page 11: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/11.jpg)
Great Lakes Watershed Management System
Provides baseline NPS pollution model estimates at field‐scales
![Page 12: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/12.jpg)
Great Lakes Watershed Management System
Conducts field‐scale scenario evaluations of land cover changes or best management practices (BMPs)
![Page 13: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/13.jpg)
Great Lakes Watershed Management System
![Page 14: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/14.jpg)
FRAMEWORK FOR EVALUATION
![Page 15: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/15.jpg)
Assessment PurposeTo evaluate:
– Perceived usefulness of system2
• Prioritizing high risk sites• Engaging producers
– Impacts of usage on efficiency at work– Integration into conservation work processes1
– Ease of use1– Transparency1– Anticipated Usage– Identify additional features needed to improve DSS’s– Participants’ understanding of material presented at the training
1. Inman, D., M. et. al. 2011.
2. Ives., B., M. Olson, J. Baroudi. 1983.
![Page 16: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/16.jpg)
Survey Instrument
• Types of Questions– ELUCID, GLWMS, Training– Five‐point Likert Scale (51)– Multiple Choice (7)– Short Answer (12)
• Administered online through Survey Gizmo
![Page 17: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/17.jpg)
RESULTS
![Page 18: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/18.jpg)
Survey Population
Sample size: 17 trainees
Response Rate: 82%– 14 completions– 1 partial completion– 2 did not take survey
![Page 19: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/19.jpg)
Profession
![Page 20: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/20.jpg)
Will help me work with producers to place BMPs on high risk areas
Will help me work with producers to place BMPs on high risk areas
Will increase my ability engage with a producer
Will increase my ability engage with a producer
A producer will more motivated to implement a BMP on a high risk area
after reviewing data in ELUCID
A producer will more motivated to implement a BMP on a high risk area
after reviewing data in GLWMS
ELUCID
GLWMS
Perceived Usefulness
![Page 21: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/21.jpg)
ELUCID
GLWMS
Will help me work with producers to place BMPs on high risk areas
Will help me work with producers to place BMPs on high risk areas
Perceived Usefulness
![Page 22: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/22.jpg)
A producer will be more motivated to implement a BMP on a high risk area
after reviewing data in ELUCID
A producer will be more motivated to implement a BMP on a high risk area
after reviewing data in GLWMS
ELUCID
GLWMS
Perceived Usefulness
![Page 23: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/23.jpg)
Will increase my ability to engage with a producer
Will increase my ability to engage with a producer
ELUCID
GLWMS
Perceived Usefulness
![Page 24: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/24.jpg)
Will make my work easier
Will make my work easier
Will reduce the amount of effort I spend on certain aspects of my work.
Will reduce the amount of effort I spend on certain aspects of my work.
Will reduce the amount of time I spend on certain aspects of my work.
Will reduce the amount of time I spend on certain aspects of my work.
Perceived Impacts on Efficiency
ELUCID
GLWMS
![Page 25: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/25.jpg)
Will make my work easier
Will make my work easier
Perceived Impacts on Efficiency
ELUCID
GLWMS
![Page 26: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/26.jpg)
Will reduce the amount of effort I spend on certain aspects of my work.
Will reduce the amount of effort I spend on certain aspects of my work.
Perceived Impacts on Efficiency
ELUCID
GLWMS
![Page 27: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/27.jpg)
Will reduce the amount of time I spend on certain aspects of my work.
Will reduce the amount of time I spend on certain aspects of my work.
Perceived Impacts on Efficiency
ELUCID
GLWMS
![Page 28: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/28.jpg)
ELUCID will easily integrate into my conservation work process.
The GLWMS will easily integrate into my conservation work process.
Perceived Potential for Integration
ELUCID fills a gap in the current set of tools I frequently use to do my job.
The GLWMS fills a gap in the current set of tools I frequently use to do my job.
ELUCID
GLWMS
![Page 29: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/29.jpg)
ELUCID fills a gap in the current set of tools I frequently use to do my job.
The GLWMS fills a gap in the current set of tools I frequently use to do my job.
Perceived Potential for Integration
ELUCID
GLWMS
![Page 30: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/30.jpg)
ELUCID will easily integrate into my conservation work process.
The GLWMS will easily integrate into my conservation work process.
Perceived Potential for Integration
ELUCID
GLWMS
![Page 31: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/31.jpg)
![Page 32: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/32.jpg)
![Page 33: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/33.jpg)
Future Work
• Follow‐up survey– Barriers to targeted conservation– How respondents have used the system
• Focus groups• Examining Google Analytics data with evaluation results
![Page 34: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/34.jpg)
ReferencesInman, D., M. Blind, I. Ribarova, A. Krause, O. Roosenschoon, A. Kassahun, H. Scholten, G. Arampatzis, G.
Abrami, B. McIntosh, P. Jeffery. 2011. Perceived effectiveness of environmental decision support systems in participatory planning: Evidence from small groups of end‐users. Environmental Modelling & Software. 26, 302‐309.
Ives, B. M. Olson, and J. J. Baroudi. 1983. The Measurement of User Information Systems. Communications of the ACM. 26 (10): 785‐793.
Matthews, K.B., M. Rivington, K.L. Blackstock, G. McCrum, K. Buchan, and D.G. Miller. 2010. Raising the bar? The challenges of evaluating the outcomes of environmental modelling and software. Environmental Modelling & Software. 26: 247‐257.
Rhee, C. and H. Raghav Rao. 2008. Evaluation of Decision Support Systems. International Handbook on Information Systems.
![Page 35: A user assessment of decision support tools](https://reader033.fdocuments.us/reader033/viewer/2022052621/5584e93bd8b42a2f5c8b48db/html5/thumbnails/35.jpg)
QUESTIONS?
www.iwr.msu.edu/flintriverelucid.iwr.msu.eduwww.iwr.msu.edu/glwms
Laura [email protected]
517‐353‐8587