National Public Health Institute, Finlandw
ww
.ktl.
fi
1
http://en.opasnet.org
Open assessment and metal risksO
Jouni TuomistoKTL, Finland
National Public Health Institute, Finlandw
ww
.ktl.
fi
2
http://en.opasnet.org
Yleismalli-wiki
National Public Health Institute, Finlandw
ww
.ktl.
fi
3
http://en.opasnet.org
Assessment page: diagram
National Public Health Institute, Finlandw
ww
.ktl.
fi
4
http://en.opasnet.org
Assessment page: variables
National Public Health Institute, Finlandw
ww
.ktl.
fi
5
http://en.opasnet.org
Module
National Public Health Institute, Finlandw
ww
.ktl.
fi
6
http://en.opasnet.org
General model of metal risks
National Public Health Institute, Finlandw
ww
.ktl.
fi
7
http://en.opasnet.org
Analytica variable and Wiki link
National Public Health Institute, Finlandw
ww
.ktl.
fi
8
http://en.opasnet.org
Variable: scope
National Public Health Institute, Finlandw
ww
.ktl.
fi
9
http://en.opasnet.org
Variable: definition
National Public Health Institute, Finlandw
ww
.ktl.
fi
10
http://en.opasnet.org
Variable: result
National Public Health Institute, Finlandw
ww
.ktl.
fi
11
http://en.opasnet.org
The attributes of a variable• Name (an identifier)• Scope (research question and boundaries)• Definition
– Data (relevant information)– Causality (dependencies)– Unit– Formula (how to compute based on the above)
• Result (what is the current estimate/answer to the research question?)
National Public Health Institute, Finlandw
ww
.ktl.
fi
12
http://en.opasnet.org
The attributes of a variable
National Public Health Institute, Finlandw
ww
.ktl.
fi
13
http://en.opasnet.org
Connections between variable attributes
National Public Health Institute, Finlandw
ww
.ktl.
fi
14
http://en.opasnet.org
Parts of an attribute• Actual content: what is known• Narrative description: any explanations or
background information that is useful to understand the actual content
• Discussion: (formal) discussions about the actual content
National Public Health Institute, Finlandw
ww
.ktl.
fi
15
http://en.opasnet.org
What is the acceptability of the idea of open assessment?
• Poll (informal, based on observations of several audiences): – 30 % think it is a stupid idea– 50 % think it cannot work– 15 % find it interesting, but…– 5 % are fond of the idea
• In which category do you fall in?
National Public Health Institute, Finlandw
ww
.ktl.
fi
16
http://en.opasnet.org
Interfaces between tools• Opasnet: Open website for any assessment-related
descriptions (en.opasnet.org, fi.opasnet.org)• Erac website: A project website for Finmerac material (to
be extended to other projects)• Analytica: modelling software (Monte Carlo, causal
diagrams and models. www.lumina.com)– Models uploaded to Opasnet– Computing in Analytica, descriptions in Opasnet.
• Result base: Interpreted results in a model-friendly format (www.pyrkilo.fi/resultdb, soon base.opasnet.org)
• M-files: For original and processed data (FMS)
National Public Health Institute, Finlandw
ww
.ktl.
fi
17
http://en.opasnet.org
Lessons learnt• Finmerac was not originally planned as an open
assessment. This caused confusion when the way of working changed during project.
• Researchers not experienced in OA or wiki – reluctance to use.
• Difficulties in finding the role and place of own work.– The role was only found when the assessment
structure became clear enough
National Public Health Institute, Finlandw
ww
.ktl.
fi
18
http://en.opasnet.org
End
National Public Health Institute, Finlandw
ww
.ktl.
fi
19
http://en.opasnet.org
What are the challenges of the current risk assessment?
• Limited area of application• Lack of flexibility and breadth• Inefficiency and slowliness of the process• Deliberate biases towards "safety"• Communication problems• Lack of acceptability among stakeholders
National Public Health Institute, Finlandw
ww
.ktl.
fi
20
http://en.opasnet.org
What is needed from the new assessment?
Stakeholders must have a say on everything in advanceValue judgements included in the assessment
Lack of acceptability among stakeholders
Everything available for clarification questionsCommunication problems
Best estimates (incl uncertainty) usedDeliberate biases towards "safety"
Info structured & directly reusableDelegation, non-experts includedRoutines automated
Inefficiency and slowliness of the process
Fully scalable to very simple and very complex questions
Lack of flexibility and breadth
Adoptable by any area of administration or policy-making
Limited area of application
National Public Health Institute, Finlandw
ww
.ktl.
fi
21
http://en.opasnet.org
Paradigm shift• Thomas Kuhn (1922-1996)• Science progresses in a regular
way until too many faults are identified in the current paradigm. Then, there is a period of extraordinary science, which leads into a shift of paradigm
National Public Health Institute, Finlandw
ww
.ktl.
fi
22
http://en.opasnet.org
Open assessment• The objectives:
– Find solutions to ALL the challenges at the same time– Systematize and "industrialize" the risk assessment– Maintain high scientific quality
• The current situation with open assessment: there are suggested methods to all challenges listed previously– Many of the suggestions have not been tested in practice– Not everything will probably work
• However, there is already a critical mass of solutions available so that full-scale testing can be started
• Further problems should be solved as they appear
National Public Health Institute, Finlandw
ww
.ktl.
fi
23
http://en.opasnet.org
Assessments in practice• They look like Wikipedia articles• They are written in much the same way• Substance is on the main page
– Research question– Definition: how to find an answer– Result
• Discussion about the substance is on a separate discussion page– Discussion can be free or structured
National Public Health Institute, Finlandw
ww
.ktl.
fi
24
http://en.opasnet.org
Open assessment• The research question for the (pyrkilo) method:
– "How can scientific information and value judgements be organised for societal decision-making in such a way that open participation is possible?"
• Full range of development– a new ontological foundation– strictly object-oriented approach– a new structure for information objects– traditional RA methods for processing information, but
organised in a more systematic way– tools that enable open collaboration– data sources that are directly available and applicable
National Public Health Institute, Finlandw
ww
.ktl.
fi
25
http://en.opasnet.org
Highlights of the theory behind• The basic principles are
– Open participation at all phases of the assessment– Requirement of scientific method (openness to scientific
critique) at all phases– Reusability of information from one assessment to another
• A uniform information structure is used:– Assessment: specific information need for a policy decision– Variable: a truthful description of a particular part of reality; it is
independent of assessments given its scope.– Both objects have 4 attributes: name, scope (research
question), definition (how to answer), and result
National Public Health Institute, Finlandw
ww
.ktl.
fi
26
http://en.opasnet.org
Assessments are performed openly in the Internet
• Parts needed to run open assessments– Web pages with method descriptions for the
assessors (Guidebook)– Web pages about useful data (Resource Centre)– Web pages about actual assessments
• Descriptions of assessments and models used (Warehouse)• Actual models (Toolbox)• Results of assessments in a uniform structure (Result
database)– Web pages for collaborative work on assessments
(Collaborative workspace)Red parts are officially parts of the Intarese project
National Public Health Institute, Finlandw
ww
.ktl.
fi
27
http://en.opasnet.org
Quality control• Is quality control possible if pages are available for
all to contribute?• Our first thoughts (no experience yet, because
work is practically within research projects)– Pages can be freely added and edited– Advanced pages are protected from direct edits
• Then, the edits are made through an open discussion on a separate page; results are transferred to the actual page
– Very important pages go through a peer review and get a quality label
National Public Health Institute, Finlandw
ww
.ktl.
fi
28
http://en.opasnet.org
The ORA report
National Public Health Institute, Finlandw
ww
.ktl.
fi
29
http://en.opasnet.org
Falsification• Karl Popper (1902-1994)• Science consists of statements
(theories) that can be falsified• Science is an evolutionary
process where poor theories are falsified
• The current knowledge consists of those theories that have not (yet) been falsified
National Public Health Institute, Finlandw
ww
.ktl.
fi
30
http://en.opasnet.org
Argumentation• Frans van Eemeren• Disputes can be solved by using
formal argumentation that consists of attacks and defends of specified statements
National Public Health Institute, Finlandw
ww
.ktl.
fi
31
http://en.opasnet.org
Mass collaboration• Don Tapscott, Anthony Williams• A large group of unorganised
people are able to produce complex artefacts, if the product is information or culture, the work can be chopped into bite-size pieces, and the pieces can be effectively synthesised.
National Public Health Institute, Finlandw
ww
.ktl.
fi
32
http://en.opasnet.org
Summary• The purpose of open assessment is to improve
and make more efficient the use of scientific information
• The basic principles are– Open participation at all phases of the assessment– Requirement of scientific method (openness to
scientific critique) at all phases– Reusability of information from one assessment to
another• Researchers are needed to test the ideas of open
assessment• If successful, scientific practices may change
Top Related