Post on 06-Apr-2018
04-09-2013
1
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
MSR- Methodologies for Scientific Research
1. Methods and Techniques for scientific Research
A possible vision about Theories on the development and
characterization of Scientific Research
http://paginas.fe.up.pt/~eol/PRODEI/mic1314_i.htm
Eugénio Oliveira- 2013/2014
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
MSR- Methodologies for Scientific Research
1. Abstract of my Lectures
MSR- Methodologies for Scientific Research
Understanding about Science
Scientific Methods and Techniques
Writing scientific papers
Evaluating scientific papers
Consulting scientific Data Bases and searching for information ( Dr. Ana Azevedo)
To criticize a scientific paper
To organize a workshop
To apply scientific methods
To write your own scientific paper
To present a scientific paper
2. Abstract of your practical Work
04-09-2013
2
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
1.1 Basic Motivations: Issues on Scientific
Methods
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Source: GETA and Graduate School of Electrical and Communications Engineering course:
Introduction to Research Methodology, Aarne Mammela, VTT-
Helsinki University of Technology,
04-09-2013
3
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Source: http://www.experiment-resources.com/
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Questioning Scientific Research- T.Kuhn
• Basic Characteristics:
• Cumulative versus reformulating always from the principles
• However:
•Thomas Kuhn
(Harvard, Berkley, Princeton, MIT until 91. +96)
“The Structure of Scientific Revolutions” 1962.
• He points out the role in scientific
research of the emergence of NEW
PARADIGMS.
MSR- Methodologies for Scientific Research
04-09-2013
4
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Questioning Scientific Research- T.Kuhn
• He points out the role in scientific research of the emergence of
NEW PARADIGMS.
•A PARADIGM is defined as:
Scientific discoveries, which universally recognized during a
certain period of time, create a new model for both problems
and their respective solutions to be used by a community of
practicians ans experimentalists
MSR- Methodologies for Scientific Research
What is your interpretation
Of the figure?
It depends on your mental,
A priori perspective….
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Paradigms support theories based on concepts, phenomena and
techniques for helping on explaining new facts or information .
• Science progresses through successive ruptures depending on
the current accepted paradigms.
Paradigms correspond to “different views” over the reality which
are influenced by aspects other than scientific ones.
Questioning Scientific Research- L. Wittgenstein
L.Wittgenstein “Tractatus Logico-Philosophicus Wttgeinstein’s Poker”, David Edmonds, J. Eidinow
MSR- Methodologies for Scientific Research Eugénio Oliveira- 2010
04-09-2013
5
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
•
Wittgenstein and duck-??????…
Questioning Scientific Research- L. Wittgenstein
MSR- Methodologies for Scientific Research
That’s the rabbit
Mouth!!
Eugénio Oliveira- 2010
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Paradigms correspond to “different views” over the reality which
are influenced by aspects other than scientific ones.
Questioning Scientific Research- L.Wittgenstein
MSR- Methodologies for Scientific Research
“ Well, God has arrived. I met him on the 5:15 train. He has a
plan to stay in Cambridge permanently”.
John Maynard Keynes, after meeting with Wittgenstein at his
arrival in Cambridge, in a letter to his wife
04-09-2013
6
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Scientific interpretation (and perception) is mostly affected by
Social as well as Psychological factors.
• Does it lead to Irrationality dominating Science?
No !
• A good discussion on prevalence of science can be found in:
• Kuhn, T.S.
“Black-Body Theory and the Quantum Discontinuity”, 1987.
However: Paradigm change non-cumulative Science (sometimes)
Questioning Scientific Research- A. Einstein & C.
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Examples of important Paradigm changes:
• Solar System Ptolemy's vision changed by Copernicus
vision).
Questioning Scientific Research- A. Einstein & C.
MSR- Methodologies for Scientific Research
• Going from Newtonian Physics to Quantic Physics (Uncertainty
Versus Determinism..) and Relativity (Space-Time relationship).
04-09-2013
7
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Karl Popper (London S. E., + 94)
• “The Logic of Scientific Discover”, Basic Books, 1959 (1ªEd Ing)
• Conjecture and Refutations, Harper & Row, 1968
Questioning Scientific Research- K.Popper
MSR- Methodologies for Scientific Research
• Idealistic. He developed a Scientific Method as an Epistomology
(a way of “knowing”) independently of the intrinsic value.
• T. Khun and Feyerabend (1993) criticize this point of view showing how
science is guided by value and, therefore being non-idealistic.
• This is a debate on “Consciousness over Existence”
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Popper argues that scientific theory will always be conjectural and
provisional.
It is not possible to confirm veracity (the truth) of a theory by means
of simply recognizing that the results of a preview already done based
on that very same theory, hold.
That Theory should only be seen as a theory not yet contradicted by
facts.
• “Critical Rationalism” proposal defends “falsifiability”
as a criterion for demarcating (separating) science and non-science
Induction Baaaad!!! - Falsifiability gooood!!!
Questioning Scientific Research- K. Popper
MSR- Methodologies for Scientific Research
04-09-2013
8
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Verificationism:
a sentence should be either verifiable or falsifiable
• Critics addressed to Verificationism: • Generalization: specific instances lead to universal
assumptions
Observation X, being an instance of Theory T
Increases the probability of Theory T being correct
The “Black Crow paradox”:
• The “Black Crow paradox”: by Hempel:
“All Crows are Black" is logically equivalent to
“Everything which is Non-Black is not a Crow“
x(Cx → Bx) ≡ x(¬Bx → ¬Cx)
Questioning Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• The “Black Crow paradox”: by Hempel:
“All Crows are Black" is logically equivalent to
“Everything which is Non-Black is not a Crow".
x(Cx → Bx) ≡ x(¬Bx → ¬Cx)
Questioning Scientific Research
• Thus, IF x(Cx Bx) reassures that x(Cx→Bx),
Then x(¬Bx ¬Cx) reassures that x(¬Bx→¬Cx)
which, being equivalent to x(Cx→Bx),
this last implication would also see its probability
of being True increased !!?
04-09-2013
9
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Crítics addressed to Verificationism: • Generalization: specific instances lead to
universal assumptions
• “There are Apples (non-Crows) not Black”
Increases the probability that the sentence
“All Crows are Black” is True !!?
• If verificationism (generalization) was
acceptable, any trivial evidence would support
a Theory
Questioning Scientific Research
There are attempts to overcome this paradox.
“falsifiability” proposed by KP states that keeping
just the Existential assertion permit us to know
that if another existential proposition that contradicts it, arrises,
this falsifies the Theory T !!
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research
MSR- Methodologies for Scientific Research
04-09-2013
10
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research Exploration
MSR- Methodologies for Scientific Research
Motivation: ambiguous…
Problem: a new way to “India”
Hypothesis: over the Atlantic and not around
south Africa
Experiment: Yes he have done it !!
Conclusions: ?????
An exploration Enterprise:
Christopher Columbus – the “Explorer”
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research Exploration
MSR- Methodologies for Scientific Research
research: discover new knowledge
• basic research (no specific application in mind)
• applied research (ideas into operational form)
• development: systematic use of the existing
knowledge
04-09-2013
11
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research Object
•
MSR- Methodologies for Scientific Research
Science, Technology and Engineering [Jain97]
• science: organized or systematic body of knowledge
• technology: application of scientific knowledge for
practical ends in engineering, medicine, agriculture, etc.
•natural sciences and engineering sciences differ in
the object of study:
• natural sciences: objects in the nature
• engineering sciences: objects (products,
services, methods) not found in the nature, using
results of mathematics and natural sciences
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific ResearchERS
MSR- Methodologies for Scientific Research
Requirements for success
• You have to be:
• analytical and curious,
• autonomous and collaborative
• criticism and tolerant
Science, Technology and Engineering [Jain97]
Verification, confirmation, and justification are
synonymous terms in philosophy of science. The
opposite is falsification or refutation. R. K. Jain and H. C. Triandis, Management of Research and Development
Organizations: Managing the Unmanageable. John Wiley & Sons, 1997.
04-09-2013
12
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific ResearchERS
MSR- Methodologies for Scientific Research
How does a scientific researcher work?
• make always notes in a “notebook”
• make summaries about what has been learned.
• make plans for the future all the time (outlines,
roadmaps)
• discuss, ask questions and argue (criticism)
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research and Engineering
MSR- Methodologies for Scientific Research
Learning carefully by induction (bottom up, generalization
from examples to models) [Felder88]
• Defending theories by deduction (top down, from
models to results)
Break the problem down and then generalize the results
(divide and conquer)
R. M. Felder and L. K. Silverman, “Learning and teaching styles in
engineering education, ”Engineering Education, pp. 674681, April 1988.
04-09-2013
13
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research
MSR- Methodologies for Scientific Research
1.2 Supporting Theories
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research
•B- Techniques refer to the means and specific Tools
enabling relevant information acquisition, the respective data
analysis, as well as the inferences that can be made to get
the results.
• Basic Characteristics
• Research involves Methodologie(s) and Techniques
MSR- Methodologies for Scientific Research
• A- Methodology includes the study of several methods that are
applicable to a class of problems
• i.e., the set of processes to conduct each specific Research
Project
04-09-2013
14
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Research Methods Dimensions
• ‘Instrumentalist Research’:
• It contributes for making human intervention in Real world
environments, more effective.
• A- Methodologies
• depending on the MOTIVATION:
• ‘Pure Research’:
• It contributes to a deep theoretical understanding and for a
more abstract formulation of the phenomena.
• Two major sub-categories are identified:
• ‘Applied Research’
• ‘Problem - Oriented Research’
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Research Methods Dimensions
• ‘Apllied Research’.
• Starting from a technology (devices, specific techniques, both)
use them for dealing with processes (physic, organizational, social,
individual,...)
• ‘Problem- Oriented Research’.
• Starting with the problem description and then looking for
the adequated techniques to solve it.
MSR- Methodologies for Scientific Research
Who wants to classify the research (s)he is willing to pursue?
04-09-2013
15
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Research Methods Dimensions
• Depending on the Theory underlying the Research:
• ‘Descriptive’, of all significative aspects of the domain
• ‘Explicative’, of the behaviour of all the phenomena
• ‘Predictive’ of the future phenomena behaviour
• ‘Prescriptive’, beyond the prediction it should also prescribe and apply
norms and processes in identified specific circumstances
Research in Informatics,IE, IS, Electronics and Telecommunications
should be of this kind
MSR- Methodologies for Scientific Research
To solve predicted problems in a flexible way !!
Or even to deal with some unexpected situations
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Research Methods Dimensions
• Depending on the tradition in that scientific area or Knowledge Domain:
i) ‘Traditional Scientific Research’ which is more quantitative
ii) ‘Interpretative Research’ which is more qualitative
iii) ‘Engineering-oriented Research’
• iii) most appropriated for Informatics, Electronics and Telecommunications
• But do not ignore the other approaches!
MSR- Methodologies for Scientific Research
04-09-2013
16
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research Traditional Methods
1.3 Positivist (classical) Research method
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research Traditional Methods
• From observations we may build up Theories trying to
explain what has been observed
• Theories are expressed in Deductive form or using
Axioms and Postulates which are then operated through
Logics.
• Traditional Scientific Research is based on positivist rational
thinking:
MSR- Methodologies for Scientific Research
Deduction Vs Induction Vs Abduction
04-09-2013
17
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research Traditional Methods
MSR- Methodologies for Scientific Research
• Scientific Theories, following K. Popper, are capable of
generating Inferences which are, in principle, falsifiable
through references to the real world.
• Results coming from the process of testing hypothesis
give feedback to the Theory, enabling verification of its
trueness or detecting possible “ab-normalities” (T.Kuhn).
Do Not Forget :
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Whenever the Observation comes from the Real World it is called empirical
and it becomes relevant in two situations:
Scientific Research Traditional Methods
MSR- Methodologies for Scientific Research
• during the formation of the theoretical knowledge through the so-called
passive observations
• during Hypothesis verification, gathering information in a more
active way, although guided by that same theory.
04-09-2013
18
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• When we do not have a suitable Theory available, we have to pursue
‘exploratory research’ where experiments and interpretation are open,
not guided by a Theory but making available Empirical Knowledge
that may postulate other Theories.
Scientific Research Traditional Methods
MSR- Methodologies for Scientific Research
• Informatics, IS and Telecommunications are disciplines (scientific areas)
without a given stable theory.
• They encompass applied subjects using a partial theory or co-opting a
theory from “reference disciplines” such as:
• Mathematics
• Information Theory (C.Shannon)
• Physics
• “Theory of Organizations”,
• “Management Sciences”,
• “Computer Science”.
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
RECAP
Methods and Techniques for guiding Research activities
• Cumulative versus reformulating always from the principles
• emergence of NEW PARADIGMS
• “Critical Rationalism” proposal defends “falsifiability”
as a criterion for demarcating science and non-science
• Critics addressed to Verificationism
• ‘Instrumentalist Research’:
• It contributes for making human intervention in Real world
environments, more effective.
• ‘Applied Research’
• ‘Problem - Oriented Research’
A- Methodology
B- Techniques refer to the means and specific Tools
‘Pure Research’ X
04-09-2013
19
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
•A- Methodology
•‘Descriptive’, of all significative aspects of the domain
• ‘Explicative’, of the behaviour of all the phenomena
• ‘Predictive’ of the future phenomena behaviour
• ‘Prescriptive’, apply norms and processes in pre-
identified specific circumstances
• Depending on the tradition in that scientific area or Knowledge Domain:
i) ‘Classical Scientific Research’ which is more quantitative
ii) ‘Interpretative Research’ which is more qualitative
iii) ‘Engineering-oriented Research’
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Scientific Research Traditional Methods
Theory of
Disciplines
of Reference
Theory
Axioms
Postulates
Deductive Logic
Hypothesis
2
3
4
5
7
Real World 1
1- There exists a Real World that we can Perceive.
2- Knowledge is Organized in Theories making inferences possible
3- Applied Disciplines like IS or T rely on Theories of Reference of other Disciplines
4- A Research Project permits testing formal Hypothesis formulated through inferences expressed in the
Theory.
5- In Empirical Research, Hypothesis derived from the Theory are directly cheked in the Real World
6- Testing Hypothesis through Simulation
7- Results comming out of the Research give feedback to the Theory
Research
Project
Sim
ula
tio
n
6
04-09-2013
20
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Summarizing: In traditional (or conventional) Science, first new Hypothesis
are extracted from the existent Theory, they are tested and new results are
added to that Theory.
Scientific Research Traditional Methods
• Thus, this implies the pre-existance of:
• A theoretical Body of Knowledge
• An explicit theoretical Framework to guide the research
• Definition of what issues have to be investigated
• What are the explicit Hypotheses (and how could they be refuted (falsifiability))
• Technique for Applied Research and well defined techniques for
testing the Hypothesis.
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
1.4 Interpretative Research method
Scientific Research Methods
MSR- Methodologies for Scientific Research
Is concerned with trying to understand lived
experience and with how participants themselves
make sense of their experiences.
IRM is phenomenological in that it wishes to explore
an individual’s personal perception or account of an
event or state as opposed to attempting to produce an
objective record of the event or state itself.
http://www.psyc.bbk.ac.uk/ipa/ (Birkbeck- U. London)
Therefore it is concerned with the meanings which
those experiences hold for the participants.
04-09-2013
21
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Interpretative Method (or philosophy):
• post-positivist
• criticizes the (conventional) scientific “chimera”
• declares the impossibility of objective observation
• Observations and Interpretations depend on the Observer.
• Addresses critics to both “hard Sciences” and Social Sciences,
Interpretative Approach to Scientific Research
MSR- Methodologies for Scientific Research
• Difficulties for Objectivity:
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Concepts, Factors, relationships that cannot be accessed
Interpretative Approach to Scientific Research
• Researchers involvement in the research domain
• Results depending on the researcher own perspective:
- On selecting and defining the research (sub-) domain
- On selecting a pre-existant theory
- On setting the issues to be investigated
- On setting the precise scope of the research
- On selecting and defining the Variables to be instantiated
- On measuring those variables possible values
• Different attitudes on all those issues may lead to many different
interpretations of the same observed phenomena.
MSR- Methodologies for Scientific Research
04-09-2013
22
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
1.5 Research in Engineering disciplines
MSR- Methodologies for Scientific Research
Engineering-oriented Research Approach
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• For this approach to research, technology is very important!
(devices, artefacts, tools, practical techniques to be used)
Engineering-oriented Research Approach
• At least up to a certain extent is the most suitable for research in
Informatics, Information Systems, Electronics, Telecommunications.
• Research in Informatics, IS and Telecomm is applied and
problem-oriented.
• Implying:
- application of suitable technology,
- conceptualization: requirements, models
- prototyping,
- implementation
- demonstration of (new) technology (real world/sim)
- Evaluation phase
MSR- Methodologies for Scientific Research
04-09-2013
23
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Observation and Theory [Wohlin99]
• In engineering an hypothesis (also defined in system
specifications) is usually an idea of the relationship between the
cause and effect (defined in system requirements)
• Theoretical model is always only an approximation of observation
in real world
C. Wohlin et al., Experimentation in Software Engineering: An Introduction, Springer, 1999.
Engineering-oriented Research Approach
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
In engineering we are fundamentally interested in
how efficiently the basic resources are used for a
needed system performance.
MSR- Methodologies for Scientific Research
Engineering-oriented Research Approach
Moreover
when finding solutions for a problem:
04-09-2013
24
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Research Techniques
MSR- Methodologies for Scientific Research
1.6 Techniques associated to Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
I. Non empírical Techniques
II. Scientific Research positivist Techniques
III. Scientific Research Interpretative Techniques
IV. Scientific Research Techniques at the positivist/interpretative boundery
V. Scientific Research Engineering-based Techniques
Research Techniques
B- Techniques
MSR- Methodologies for Scientific Research
04-09-2013
25
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
I. Non empirical Techniques depend on:
• Artificially created data, or
• Conceptual thinking about abstractions
• Non empirical Techniques include:
• conceptual research based on opinion and speculation.
( argumentative and dialectic analysis)
• theorem proving: This applies to mathematical abstractions:
• formal methods application,
• Induction (in the sense that results are “abstractions”),
• mathematical abstractions
• Models verification
• Simulation:
• Mapping a complex environment in a simplified Model.
MSR- Methodologies for Scientific Research
Research Techniques
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Other non empiric Techniques:
• Using hypothetic future scenarios
• Mapping a real scenario into Games
• ...
• Reviewing the State-of-the-art and doing Meta-analysis
MSR- Methodologies for Scientific Research
Research Techniques
04-09-2013
26
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
FEUP-LIACC / Eugénio Oliveira
Using Games for Research
RoboSoccer Competition-middle size
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Março 2007 FEUP-LIACC / Eugénio Oliveira
IA- Futebol Robótico- Sony legged
04-09-2013
27
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
II. Scientific Research positivist Techniques
(applicable to Informatics, IS and other Engineering disciplines):
• Techniques for forecasting applying
• regression algorithms (Technique for fitting a simple equation to real data points)
• time-series based algorithms to extrapolate on historic
data ; to extract meaningful statistics from data; • Field Experiments :
• Possibly isolating and controlling a set of phenomena
to be looked at
• Laboratory Experiments:
• Creating artificial setups in which phenomena, factors
and variables are isolated and controlled.
MSR- Methodologies for Scientific Research
Research Techniques
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Research approach is Descriptive/Interpretative:
Going from empirical observations to a limited rigour analysis.
Need for controlling the researcher’s perspective:
- Critical self- Examination
- Revising all the postulates and biases
- Varying Observations
- Submission to peer reviewing
III. Scientific Research Interpretativist Techniques:
• Group Research:
• frequent discussion with a group of people including
who may be affected by the application (or the technology)
• Use Collaborative Work Tools.
MSR- Methodologies for Scientific Research
Research Techniques
04-09-2013
28
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Action-based Research:
- Researcher is like an “Agent” changing existent
conditions to measure the respective reaction
• Other Methods for Social Sciences(Ethnographic..)
In http://www.qual.auckland.ac.nz/ Michael Myers Ed. Ass.Inf.Sys.
It is advocated qualitative research, for example, for IS.
We also can do (but it is not recommended) R&D just through
Case-study analysis
MSR- Methodologies for Scientific Research
Research Techniques
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
IV. Scientific Research Techniques at the positivist/interpretativist
boundery
Including:
- Field Work : Data related to the object of study is
directly observed by the Researcher in the original
context
- Questionary-based : Structured Data collection comming
from interviewees
- Case Study: - Isolate and describe a relevant real-world
situation,
- Gather a collection of data from multiple
sources
MSR- Methodologies for Scientific Research
Research Techniques
Eugénio Oliveira- 2010
04-09-2013
29
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Secondary Research:
• Analysis starting from pre-existent documents (texts,
papers, descriptions, memoirs).
• Analysis is done under new perspective and using
different techniques or procedures.
MSR- Methodologies for Scientific Research
Research Techniques
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
v. Scientific Research Engineering-based Techniques
(Informatics Engineering, IS, Electronic Engineering, Telecomm...)
• Divided into two categories:
1- Construction Technique:
• Includes Design, Project, Implementation
(or prototyping) of a system
(ex: Computer program System, or physical device)
- objective:
-- explicitly testing an hypothesis or
-- to solve a class of problems.
- it implies:
-- Existence of either a theory or a model
for explaining the results
(testing the hypothesis)
MSR- Methodologies for Scientific Research
Research Techniques
04-09-2013
30
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• theoretical model is defined (preferably a mathematical model),
including boundary conditions (define the environment)
and initial conditions (define the initial values of the
parameters of the model) [Losee01]
J. Losee, A Historical Introduction to the Philosophy of Science,
4th ed. Oxford Univ. Press, 2001.
MSR- Methodologies for Scientific Research
Research Techniques
Materials and Methods
• in exact sciences materials include definitions,
model, and related assumptions
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• rules of analysis follow deductive reasoning,
statistical analysis and approximations (analysis
must often be replaced by simulations due to
mathematical tractability problems)
• the theoretical model is verified by comparing the results
with reality (measurements with a prototype, known as
hypothetical-deductive method)
MSR- Methodologies for Scientific Research
Research Techniques
• methods include:
rules of analysis and
rules of verification and validation
04-09-2013
31
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• rules of analysis follow deductive reasoning,
statistical analysis and approximations (analysis
must often be replaced by simulations due to
mathematical tractability problems)
• the theoretical model is verified by comparing the results
with reality (measurements with a prototype, known as
hypothetical-deductive method)
MSR- Methodologies for Scientific Research
Research Techniques
• methods include:
rules of analysis and
rules of verification and validation
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• use analytical results in simple limited cases to obtain reference
values
• the value of the results depend on how well they can be
generalized (bottom up approach will help you)
MSR- Methodologies for Scientific Research
Research Techniques
• Results
• results can be analytical (deductive), from simulation, or
measurement results
• it is important to present numerical results that
verify (or not) your own thesis,
04-09-2013
32
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Conclusions (“constructive “-based technique report example)
(1) “We have shown that [some] algorithms proved right…”
(2) “Our approach is simpler to implement than previous algorithms and is
practically feasible …. On the theoretical side, the greatest deficiency in our
results is …”
(3) “In a practical situation the advantages and limitations of our
approach are…”
(4) “These results can be extended in a number of directions…”
MSR- Methodologies for Scientific Research
Research Techniques
1. What is shown by that work and its significance.
2. Limitations and advantages.
3. Applications of the results.
4. Recommendations for further work.
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Destructive Techniques:
• To analyse a system, or a set of methods to gather
new information about those technology procedures
under analysis.
- Recognized information and characteristics are
then reused may be in a different way
- Example: Reverse Engineering
MSR- Methodologies for Scientific Research
Research Techniques
04-09-2013
33
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• There are philosophies permitting the Classification of Research
Methods through Paradigms or basic Epistemologies:
I. Positivist
II. Interpretative (post-positivist)
III. Critical
IV. (Constructivism)
At Harvard Business School they use just the three categories
Phylosophy Underlying Research
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Positivist Method:
• Assumes that reality is objectively accessible, it can be
measured and described by the observer and measured by
the same observer through their instruments and devices.
Tests are made making predictions possible over future
phenomena
• IS research (also Telecomm research?) can be of the
positivist kind (for IS, Orlikowski and Baroudi,1995)
- Use math/logic formalisms
- use quantifiable and measurable variables
- Test an Hypothesis and make inferences from what is
known.
MSR- Methodologies for Scientific Research
Philosophy Underlying Research
04-09-2013
34
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Interpretativist Method
• Assumes that accessing to reality is only possible through
social constructions like language, consciousness and
ontologies (sharing of concepts’ meaning).
• Hermeneutics-based philosophy (based on the human
understanding and text interpretation) as well as in
phenomenology
(describing, understanding and interpretation perceived
phenomena)
Proposes to abolish separation between "subject“ and
"object",
It opposes to XIX century positivist thinking.
MSR- Methodologies for Scientific Research
Philosophy Underlying Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Can Informatics, IS and Telecomm studies influence or be
influenced by the context?
• Here, the focus is not as much on the variables,
measurements and processes as it is on interpretation of
the results of the test, the measurement or implementation
MSR- Methodologies for Scientific Research
Philosophy Underlying Research
04-09-2013
35
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Research Criticism:
• It is a radical enterprise once it tries to identify what constraints
research:
• socially, culturally,
• Historically, politically,
• economically, physically
trying to emancipate science by eliminating causes for ignorance,
alienation etc...
Eg: Why is it not possible to do research on X?
Eg; Why cannot we implement systems of type Y?
Habermas from Frankfurt School following the way of Adorno,
Lukacs, Marcuse
MSR- Methodologies for Scientific Research
Philosophy Underlying Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Junk Science (?) Stories of the Past Decade
Dial “F” for Fear.
Since the 1993 Larry King Live broadcast featuring a man suing a
cell phone maker claiming his wife died from a cell phone-
induced brain cancer, many cell phone users have worried
about phone safety.
But studies failed to identify any risk for many, many years …
MSR- Methodologies for Scientific Research
Philosophy Underlying Research
04-09-2013
36
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Junk Science (?) Stories of the Past Decade
MSR- Methodologies for Scientific Research
Philosophy Underlying Research
Powerline scare unplugged.
Fears that electric and magnetic fields (EMFs) created by power
lines and appliances caused cancer started in 1978. Parents
worried about power lines over schools.
Power companies worried about burying power lines. The National
Academy of Sciences finally unplugged the scare in October 1996,
concluding that no evidence showed EMFs presented a health
hazard
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Quality of Research
1.7 Quality of Research
MSR- Methodologies for Scientific Research
04-09-2013
37
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Quality of Research
• Quality depends on the Objectives to reach
1. To increase available (and also general) Knowledge
• Objectives and Quality criteria are internal to the discipline
• RIGOUR is very critical
2. Developing Computer Systems– (Instrumentalisme)
• Objectives and Criteria are external (regarding R&D)
• Relevance can be measured by the impact and is of the
most importance
• Relevance Versus Rigour
• Not antagonic but taken with different emphasis
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Case-studies based research is mainly descriptive concerning
simple instances and then less scientific
• Empirical research is now dominated by the descriptive
approaches
• Specific Models have limited capabilities for explaining a
phenomen as well as little predictive power.
• Computer Science (and other Engineering-like areas) tendency:
• Until the 80ies conceptual (not empirical) papers
dominated
• After the 80 ies Less theoretical papers, more
instantiated, less general and Knowledge less cumulative
MSR- Methodologies for Scientific Research
Quality and directions of Research
04-09-2013
38
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Include support to:
• Logistics, procurement, contracting, trading
• Reorganization and reconfiguration
• Informatics and Computer Science Research:
• Services, Platforms and Frameworks for Electronic
inter-operability between Organizations (enterprises,
schools, institutions…) inside themselves and interfacing
to society (markets)
MSR- Methodologies for Scientific Research
Quality and directions of Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Supporting Sciences:
• Biotechnology
• Graphic Computing
• Medicine
• Law
• Astronomy, Phisics, History, Management….
• Informatics and Computer Science Research:
• Algorithms and Systems for Information management:
• Visual (image, graphic…)
• Textual (Text Corpora, data in DB)
• Signals (traffic, biologic, environmental….)
MSR- Methodologies for Scientific Research
Quality and directions of Research
04-09-2013
39
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Quality of Research
• Requirements for the Research in CS and TC
• Choose an appropriate Research Method:
• Make explicit a Body of Theory sometimes coming from
other adjacent disciplines (ex: Computer Science, Organization
Theory, Information Theory, Physics, Statistics…)
• Traditional/positivist, “interpretativiste” or
Engineering-oriented
• Extend the Theory and Infer predictions to be checked
(at least in the case of positivist/traditional research)
MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
(following Björn Lisper, Datorteknik, Mälardalens högskola
Västerås, Sweeden)
http://www.idt.mdh.se/kurser/ct3340/ht03/
Theoretical methods:
Create formal models (mathematics, logic)
Define concepts within thesis
Prove properties of the concepts
Abstraction, hide details to make the whole more understandable
(and to make it possible to prove properties of it)
Proofs of properties by deductive methods
Scientific Methods, General
04-09-2013
40
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Empirical methods:
Perform experiments
See how it turned out
Draw conclusions
Simulation:
Start with a formal model at some "easy-to-understand" level
Make "artificial experiments" in your computer
Collect statistics and draw conclusions
Scientific Methods, General
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Common pattern in Computing Science:
The system is constructed to behave according to some
theoretical model
Deviations are seen as construction errors rather than
deficiencies in the theory (hardware error, bug in OS, : : :)
In both cases: the theory helps us understand and predict, but in
different ways!
Scientific Methods
In Physics:
Make hypotheses about the surrounding world (theory),
observe it (experiment)
Relate the result of experiment to the theory
Adjust the theory if it doesn’t predict the reality well enough
Theory is used to predict the future (e.g., if a bridge will hold for a
certain load, or an asteroid fall down on our heads)
04-09-2013
41
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Scientific Methods
Theoretical vs. Empirical Methods in Computing Science
Computing Science really has a “spectrum”, from “extreme
constructivism” to the use of theory close to the one in physics:
• “Extreme constructivism”: (ideal) programming language design:
Formal semantics for the language, pure construction of model
defining the mathematical meaning of each program
Abstraction of details to make the meaning of the language
simpler (for instance, assume that data structures can grow
arbitrarily big)
Implement the language according to the semantics
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Scientific Methods
Theoretical vs. Empirical Methods in Computing Science
One can prove formally within the model that a
program is correct –valuable!
But the model does not cover all kinds of failures.
E.g., hardware errors, or stack overflow (or an
asteroid falling down on the computer)
04-09-2013
42
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Scientific Methods
Theoretical vs. Empirical Methods in Computing Science
Extreme “physics” approach:
performance modeling of complex computer-
and communication systems
Extremely hard to make analytical calculations
Simplified performance models, tested against
experiments (e.g., long suites of benchmarks)
Discrepancy leads to a modified theory, as in physics
Often simulation (desire to evaluate systems before
building them)
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Scientific Methods
Theoretical vs. Empirical Methods in Computing Science
In-between: algorithm analysis
Build on some form of formal model for how the
algorithm executes (meta-language with formal
semantics), and some performance model (how long
does a step in the algorithm take, how much memory
is needed to store an entity)
04-09-2013
43
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research MSR- Methodologies for Scientific Research
Scientific Methods
Theoretical vs. Empirical Methods in Computing Science
In-between: algorithm analysis
Given that the performance model is correct, one
proves
mathematically that the algorithm needs certain
resources (time, memory) to be carried out
But the performance model is often very approximate.
Sometimes is possible to refine the performance model,
but this can make it impossible to calculate the resource
needs of the algorithm
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Quality of Research
• Characteristics of the Research in CS
• Always Report the most recent S-o-A. (always moving...)
• Prove you made an advancement in Knowledge
• Combination of Techniques trying to compensate one’s
weaknesses with other one strengths
• Report about the object of study and the nature of the domain
• Validate Data
• Analysis using statistics
• Extract relationships between variables
• Praticable according to the resources
• Relevant results (at different levels: community,society...)
• Publish
• Ambitious
MSR- Methodologies for Scientific Research
04-09-2013
44
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Characterístics of the research in CS:
• Research may have as an objective:
• Building a Theory
• Testing a Theory
• Extending a Theory
MSR- Methodologies for Scientific Research
Quality of Research
• The practical research objectives may include:
• Exploring a specific domain
• Describing a domain
• Explaining a domain
• Forecasting over a domain
• (Testing formalized hipothesis)
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Other characteristics:
• Concerning Data:
• Extensive
• Representative
• What is Data validity (ex. temporal, precision…)
• Granularity:
• Simple or agregate
• Specific or large generic experiments
• Temporal Horizon:
• “Snapshot” of the domain at a specific point in time
• “Longitudinal cut section” (múltiple points in time)
• Methods for colecting and analising data; quantification...
MSR- Methodologies for Scientific Research
Quality of Research
04-09-2013
45
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
1.8 Challenges
MSR- Methodologies for Scientific Research
Quality of Research
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
Research in CS and TC Challenges
• Challenges:
• There exists shortage of theory. Mainly theories in related disciplines
• Phenomena are unstable:
• Data may get old quickly, usually a snapshot that may vary
- Pay attention to the validity period
• Organizations and Markets are always moving
• Significant cultural variations:
• Different national/regional cultures imply different speed/rhythm
• When you are doing research (requirements, interaction,
questionnaires, demos) you influence the domain
• Pressure for publishing during the research action
MSR- Methodologies for Scientific Research
04-09-2013
46
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Pressure for technical instantiation:
• Prototyping without a theoretical ground or well defined
objective
• Economics and business models either not known or consolidate.
• Impact of the automation and rationalization:
• Depends on time dynamics concerns
• Depends on who is directly affected
• Depending on if researchers are also objects of the study
• Adopting a new technology may take long time
• Impact on other partners, markets, standards, tools…
MSR- Methodologies for Scientific Research
Research in CS and TC Challenges
Eugénio Oliveira- 2013 MSR- Methodologies for Scientific Research
• Conclusions on Research actions in CS and TC
• Consider tradoff Rigour Versus Relevance
• Identify challenges BEFORE starting doing research
• Be clear about conventions related with research
to be done
• Select a significative sub-domain
• Define questions to be answered through that research
• Select the apropriate Research Method and Techniques
(possibly a good combination of some) and justify the choice.
Whenever the selected aproach is instrumentalist, you need
to be sure about the external quality, data validation and rigour
as well as:
Relevance
MSR- Methodologies for Scientific Research
Research in CS and TC Challenges