ael.chungbuk.ac.krael.chungbuk.ac.kr/ael/연구기초특론/참고문헌... · 2014. 1. 15. ·...
Transcript of ael.chungbuk.ac.krael.chungbuk.ac.kr/ael/연구기초특론/참고문헌... · 2014. 1. 15. ·...
1/3/2008
1
���� ���������� ������������������ �����
1. Methods and Techniques for developing Research projects A possible vision about Theories on the development and
characterization of Scientific Research
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2007/2008
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
���� ���������� ������������������ �����
1. Abstract of my Lectures
Understanding about ScienceScientific Methods and TechniquesWriting scientific papersEvaluating scientific papersConsulting scientific Data Bases and searching for information ( Dr. Ana Azevedo)
2. Abstract of your Work
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Criticizing a scientific paperOrganizing a workshopWriting a scientific paperPresenting a scientific paper
2. Abstract of your Work
1/3/2008
2
1.1 Basic Motivations: Issues on Scientific Methods
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Source: GETA and Graduate School of Electrical and Communications Engineering course: Introduction to Research Methodology, Aarne Mammela, VTT-Helsinki University of Technology,
1/3/2008
3
� �� ��������������������� �����
• Basic Characteristics:• Cumulative versus reformulating always from the principles
• However:• However:•Thomas Kuhn(Harvard, Berkley, Princeton, MIT until 91. +96) “The Structure of Scientific Revolutions” 1962.
• He points out the role in scientific research of the emergence of NEW PARADIGMS.
•A PARADIGM is defined as:
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
•A PARADIGM is defined as:Scientific discoveries, which universally recognized during a certain period of time, create a new model for both problems and their respective solutions to be used by a community of practicians ans experimentalists
MSR- Methodologies for Scientific Research
• Paradigms support theories based on concepts, phenomena and techniques for helping on explaining new facts or information
.
� �� ��������������������� �����
• Science progresses through successive ruptures depending on the current accepted paradigms.
Paradigms correspond to “different views” over the reality which are influenced by aspects other than scientific ones.
Wittgenstein and duck-??????…
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Wittgenstein and duck-??????…
Eugénio Oliveira- 2007
L.Wittgenstein “Tractatus Logico-PhilosophicusWttgeinstein’s Poker”, David Edmonds, J. Eidinow
MSR- Methodologies for Scientific Research
1/3/2008
4
• Scientific interpretation (and perception) is mostly affected bySocial as well as Psychological factors.
• Does it lead to Irrationality dominating Science?
� �� ��������������������� �����
• Example of important Paradigm changes:
• Does it lead to Irrationality dominating Science?No !
• A good discussion on prevalence of science can be found in:• Kuhn, T.S.“Black-Body Theory and the Quantum Discontinuity”, 1987.
However: Paradigm change � non-cumulative Science (sometimes)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• Solar System Ptolemy's vision changed by Copernicusvision • Going from Newtonian Phisics to Quantic Physics (Uncertainty Versus Determinism..) and Relativity (Space-Time relationship).
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• Karl Popper (London S. E., + 94)
� �� ��������������������� �����
• Karl Popper (London S. E., + 94)
• “The Logic of Scientific Discover”, Basic Books, 1959 (1ªEd Ing)
• Conjecture and Refutations, Harper & Row, 1968
• Idealistic. He developed a Scientific Method as an Epistomology(a way of “knowing”) independently of the intrinsic value.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• T. Khun e Feyerabend (1993) criticize this point of view showing how science is guided by value and, therefore being non-idealistic.
• This is a debate on “Consciousness over Existence”
1/3/2008
5
• “Critical Rationalism” proposal� defends “falsifiability”as a criterion for demarcating (separating) science and non-science
Induction Baaaad!!! - Falsifiability gooood!!!
� �� ��������������������� �����
• Popper argues that scientific theory will always be conjectural andprovisional.
It is not possible to confirm veracity (the truth) of a theory by meansof simply recognizing that the results of a preview already done basedon that very same theory, hold.
Induction Baaaad!!! - Falsifiability gooood!!!
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
on that very same theory, hold.
That Theory should only be seen as a theory not (or not yet)contradicted by facts.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Verificationism: a sentence should be either verifiable or falsifiable
Critics addressed to Verificationism:Generalization: specific instances lead to universal
assumptions
� �� ��������������������� �����
assumptions
Observation X, being an instance of Theory TIncreases the probability of Theory T being correctThe “Black Crow paradox”:
• The “Black Crow paradox”: by Hempel:“All Crows are Black" is logically equivalent to“Everything which is Non-Black is not a Crow".
∀x(Cx � Bx) � ∀ x(¬Bx � ¬Cx)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MIC- Metodologias de Investigação Científica
∀x(Cx � Bx) � ∀ x(¬Bx � ¬Cx)
• Thus, If ∃ x(Cx ∧ Bx) reassures that ∀ x(Cx�Bx), Then ∃ x(¬Bx ∧ ¬Cx) reassures that ∀ x(¬Bx�¬Cx) which, being equivalent to ∀ x(Cx�Bx), this last implication would also see its probabilityof being True increased.
1/3/2008
6
• Crítics addressed to Verificationism:• Generalization: specific instances lead to
universal assumptions
• “There are Apples (non-Crows) not Black”
� �� ��������������������� �����
• “There are Apples (non-Crows) not Black” Increases the probability that the sentence“All Crows are Black” is True
• If verificationism (generalization) was acceptable, any trivial evidence would support a Theory
There are attempts to overcome this paradox..
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
There are attempts to overcome this paradox..“falsifiability” proposed by KP states that keepingjust the Existential assertion permit us to know that if another existential proposition that contradicts it, arrises, this falsifies the Theory T !!
MSR- Methodologies for Scientific Research
������������� �����
•
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
7
A������������� ������ ����������
An exploration Enterprise:
Christopher Columbus – the “Explorer”Motivation: ambiguous…Problem: a new way to “India”Hypothesis: over the Atlantic and not around south Africa
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
south AfricaExperiment: Yes he have done it !!Conclusions: ?????
A������������� ������� ����������
research: discover new knowledge
• basic research (no specific application in mind)
• applied research (ideas into operational form)
• development: systematic use of the existing
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• development: systematic use of the existing knowledge
1/3/2008
8
A������������� ������ � �����
Science, Technology and Engineering [Jain97]
• science: organized or systematic body of knowledge
•• technology: application of scientific knowledge for practical ends in engineering, medicine, agriculture, etc.
•natural sciences and engineering sciences differ inthe object of study:
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• natural sciences: objects in the nature• engineering sciences: objects (products, services, methods) not found in the nature, usingresults of mathematics and natural sciences
A������������� ��������
Science, Technology and Engineering [Jain97]
Verification, confirmation, and justification are
Requirements for success• You have to be:
Verification, confirmation, and justification aresynonymous terms in philosophy of science. Theopposite is falsification or refutation.R. K. Jain and H. C. Triandis, Management of Research and Development Organizations: Managing the Unmanageable. John Wiley & Sons, 1997.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• You have to be:• analytical and curious, • autonomous and collaborative• criticism and tolerant
1/3/2008
9
A������������� ��������
How does a researcher work?
•• make always notes in a “notebook”• make summaries on what has been learned.• make plans for the future all the time (outlines, roadmaps)• discuss, ask questions and argue (criticism)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
A������������� �����
Learning carefully by induction (bottom up, generalizationfrom examples to models) [Felder88]
• Defending theories by deduction (top down, from
•
• Defending theories by deduction (top down, frommodels to results)
Break the problem down and then generalize the results(divide and conquer)
R. M. Felder and L. K. Silverman, “Learning and teaching styles in
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
R. M. Felder and L. K. Silverman, “Learning and teaching styles inengineering education, ”Engineering Education, pp. 674681, April 1988.
1/3/2008
10
A������������� �����
1.2 Supporting Theories
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
������������� �����
• Basic Characteristics
• Research involves Methodologie(s) and Techniques
• A- Methodology includes the study of several methods that areapplicable to a class of problems
• i.e., the set of processes to conduct each specific Research Project
• B- Techniques refer to the means and specific Toolsenabling relevant information acquisition, the respective data
• Research involves Methodologie(s) and Techniques
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
enabling relevant information acquisition, the respective dataanalysis, as well as the inferences that can be made.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
11
�� ����������� ���� �� ���
• A- Methodologies• depending on the MOTIVATION:
• ‘Instrumentalist Research’:• It contributes for making human intervention in Real world
environments, more effective.
• ‘Pure Research’:• It contributes to a deep theoretical understanding and for a more abstract formulation of the phenomena.
• Two major sub-categories are identified:
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Two major sub-categories are identified:• ‘Applied Research’• ‘Problem - Oriented Research’
MSR- Methodologies for Scientific Research
�� ����������� ���� �� ���
• ‘Apllied Research’. • Starting from a technology (devices, specific techniques, both) use them for dealing with processes (physic, organizational, social, use them for dealing with processes (physic, organizational, social, individual,...)
• ‘Problem- Oriented Research’. • Starting with the problem description and then looking for
the adequated techniques to solve it.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
12
�� ������������� ��� �� ���
• Depending on the Theory underlying the Research:
• ‘Descriptive’, of all significative aspects of the domain• ‘Descriptive’, of all significative aspects of the domain
• ‘Explicative’, of the behaviour of all the phenomena
• ‘Predictive’ of the future phenomena behaviour
• ‘Prescriptive’, beyond the prediction it should also prescribe and apply Norms and processes in identified specific circumstances
(Research in IS, Electronics and Telecommunications should be of
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
(Research in IS, Electronics and Telecommunications should be ofthis kind)
MSR- Methodologies for Scientific Research
�� ������������� ��� �� ���
• Depending on the tradition in that scientific area or Knowledge Domain:
i) ‘Traditional Scientific Research’ which is more quantitative
ii) ‘Interpretative Research’ which is more qualitative
iii) ‘Engineering-oriented Research’
• iii) most appropriated for Electronics, Telecommunications and CS
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• iii) most appropriated for Electronics, Telecommunications and CS
• But do not ignore the other approaches!
MSR- Methodologies for Scientific Research
1/3/2008
13
������������� ������������� �����
1.3 Positivist (classical) Research method
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
������������� ������������� �����
• Traditional Scientific Research is based on positivist rational thinking:
• From observations we may build up Theories trying to explain what has been observed
• Theories are expressed in Deductive form or using Axioms and Postulates which are then operated through
thinking:
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Axioms and Postulates which are then operated through Logics.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
14
������������� ������������� �����
Do Not Forget :
• Scientific Theories, following K. Popper, are capable of generating Inferences which are, in principle, falsifiablethrough references to the real world.
• Results coming from the process of testing hypothesis give feedback to the Theory, enabling verification of its trueness or detecting possible “ab-normalities” (T.Kuhn).
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
trueness or detecting possible “ab-normalities” (T.Kuhn).
• Whenever the Observation comes from the Real World it is called empirical
������������� ������������� �����
• Whenever the Observation comes from the Real World it is called empiricaland it becomes relevant in two situations:
• during the formation of the theoretical knowledge through the so-calledpassive observations
• during Hypothesis verification, gathering information in a moreactive way, although guided by that same theory.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
15
• When we do not have a suitable Theory available, we have to pursue‘exploratory research’ where experiments and interpretation are open,not guided by a Theory and making available Empirical Knowledge that may postulate other Theories.
������������� ������������� �����
that may postulate other Theories.
• IS and Telecommunications are disciplines (scientific areas) without a given stable theory.
• They encompass applied subjects using a partial theory or co-opting a theory from “reference disciplines” such as:
• Mathematics• Information Theory (C.Shannon)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• Information Theory (C.Shannon)• Physics• “Theory of Organizations”, • “Management Sciences”, • “Computer Science”.
������������� ������������� �����
Theory ofDisciplines of Reference
Theory Axioms
2
3
AxiomsPostulatesDeductive Logic
Hypothesis
4
5
7
Real World1
1- There exists a Real World that we can Perceive.
ResearchProject
Sim
ulat
ion
6
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
2- Knowledge is Organized in Theories making inferences possible3- Applied Disciplines like IS or T rely on Theories of Reference of other Disciplines
4- A Research Project permits testing formal Hypothesis formulated through inferences expressed in theTheory.5- In Empirical Research, Hypothesis derived from the Theory are directly cheked in the Real Worldl
6- Testing Hypothesis through Simulation7- Results comming out of the Research give feedback to the Theory
1/3/2008
16
• Summarizing: In traditional (or conventional) Science, new Hypothesis areextracted from the existent Theory, they are tested and new results are added to that Theory.
������������� ������������� �����
added to that Theory.
• Thus, it implies the pre-existance of:• A theoretical Body of Knowledge • An explicit theoretical Framework to guide the research• Definition of what issues have to be investigated
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Definition of what issues have to be investigated• What are the explicit Hypotheses that could be refuted (falsifiability)• Method for Applied Research and well defined techniquesfor testing the Hypothesis.
MSR- Methodologies for Scientific Research
1.4 Interpretative Research method
������������� �����������
is concerned with trying to understand lived experience and with how participants themselves make sense of their experiences. Therefore it is concerned with the meanings which those experiences hold for the participants. IRM is phenomenological in that it wishes to explore an individual’s personal perception or account of
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
individual’s personal perception or account of an event or state as opposed to attempting to produce an objective record of the event or state itself. http://www.psyc.bbk.ac.uk/ipa/ (Birkback- U. London)
1/3/2008
17
• Interpretative Method (or philosophy):
����������� ������������������������� �����
• post-positivist• criticizes the (conventional) scientific “chimera”• declares the impossibility of objective observation • Observations and Interpretations depend on the Observer.• Addresses critics to both “hard Sciences” and Social Sciences,
• Difficulties for Objectivity:
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• Concepts, Factors, relationships that can’t be accessed
������������ ��������������������������� �����
• Researchers involvement in the research domain• Researchers involvement in the research domain
• Results depending on the researcher own perspective:- On selecting and defining the research (sub-) domain- On selecting a pre-existant theory- On setting the issues to be investigated- On setting the precise scope of the research- On selecting and defining the Variables to be instantiated- On measuring those variables possible values
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
- On measuring those variables possible values
• Different attitudes on all those issues may lead to many different interpretations of the same observed phenomena.
MSR- Methodologies for Scientific Research
1/3/2008
18
Engineering-oriented Research Approach
1.5 Research in Engineering disciplines
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• For this approach to research, technology is very important!• (devices, artefacts, practical techniques to be used)
Engineering-oriented Research Approach
• At least up to a certain extent is the most suitable for research in Information Systems, Electronics, Telecommunications.
• Research in IS and Telecomm is applied and problem-oriented.• Implying:
- application of suitable technology, - conceptualization: requirements, models- prototyping,
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
- prototyping, - implementation - demonstration of (new) technology (real world/sim)- Evaluation phase
MSR- Methodologies for Scientific Research
1/3/2008
19
Engineering-oriented Research Approach
Observation and Theory [Wohlin99]
• In engineering an hypothesis (defined in system specifications) is usually an idea of the relationship between the cause and effect (defined in system requirements)
• Theoretical model is always only an approximation of observation
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• Theoretical model is always only an approximation of observation in real world
C. Wohlin et al., Experimentation in Software Engineering: An Introduction, Springer, 1999.
Engineering-oriented Research Approach
In engineering we are fundamentally interested inhow efficiently the basic resources are used for aneeded performance.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
20
• From Dr. Leung Yee Hong Australia TC Research Institut
Engineering-oriented Research Approach
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Research Techniques
1.6 Techniques associated to Research
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
21
I. Non empírical Techniques
Research Techniques
B- Techniques
II. Scientific Research positivist Techniques
III. Scientific Research Interpretative Techniques
IV. Scientific Research Techniques at the positivist/interpretative boundery
V. Scientific Research Engineering-based Techniques
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
I. Non empirical Techniques depend on:
• Artificially created data, or• Conceptual thinking about abstractions
Research Techniques
• Conceptual thinking about abstractions
• Includes:• conceptual research based on opinion and speculation.
( argumentative and dialectic analysis)• theorem proof: This applies to mathematical abstractions:
• formal methods application, • induction, • mathematical abstractions
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• mathematical abstractions• Models verification
• Simulation: • Mapping a complex environment in a simplified Model.
MSR- Methodologies for Scientific Research
1/3/2008
22
• Other non empiric Techniques:
Research Techniques
• Using hypothetic future scenarios• Mapping a real scenario into Games• ...
• Reviewing the State-of-the-art and doing Meta-analysis
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
������������ ���� ��
����� ��� ��������������� ���
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchMarço 2007 FEUP-LIACC / Eugénio Oliveira
1/3/2008
23
II. Scientific Research positivist Techniques(applicable to IS and other Engineering disciplines):
• Techniques for forecasting applying
Research Techniques
• Techniques for forecasting applying • regression algorithms and • time-series based algorithms to extrapolate on historic
data
• Field Experiments :• Possibly isolating and controlling a set of phenomena
to be looked at
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Laboratory Experiments:• Creating artificial setups in which phenomena, factors
and variables are isolated and controlled.
MSR- Methodologies for Scientific Research
• Research approach is Descriptive/Interpretative:
III. Scientific Research Interpretativist Techniques:
Research Techniques
• Research approach is Descriptive/Interpretative: Going from empirical observations to a limited rigour analysis.Need for controlling the researcher’s perspective:
- Critical self- Examination - Revising all the postulates and biases- Varying Observations- Submission to peer reviewing
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Group Research: • frequent discussion with a group of people including
who may be affected by the application (or the technology)• Use Collaborative Work Tools.
MSR- Methodologies for Scientific Research
1/3/2008
24
• Action-based Research:
- Researcher is like an “Agent” changing existent
Research Techniques
- Researcher is like an “Agent” changing existent conditions to measure the respective reaction
• Other Methods for Social Sciences(Ethnographic..)
In http://www.qual.auckland.ac.nz/ Michael Myers Ed. Ass.Inf.Sys.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
In http://www.qual.auckland.ac.nz/ Michael Myers Ed. Ass.Inf.Sys.It is advocated qualitative research, for example, for IS.
We also can do (but it is not recommended) R&D just through Case-study analysis
MSR- Methodologies for Scientific Research
IV. Scientific Research Techniques at the positivist/interpretativistboundery
Including:
Research Techniques
Including:- Field Work : Data related to the object of study is directly observed by the Researcher in the original context
- Questionary-based : Structured Data collection comming from interviewees
- Case Study: - Isolate and describe a relevant real-world
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
- Case Study: - Isolate and describe a relevant real-worldsituation,
- Gather a collection of data from multiple sources
MSR- Methodologies for Scientific Research
1/3/2008
25
• Secondary Research:
Research Techniques
• Analysis starting from pre-existent documents (texts, papers, descriptions, memoirs).
• Analysis is done under new perspective and using different techniques or procedures.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
v. Scientific Research Engineering-based Techniques(Informatics Engineering, IS, Electronic Engineering, Telecomm...)
• Divided into two categories:
Research Techniques
• Divided into two categories:
1- Construction Technique: • Includes Design, Project, Implementation
(or prototyping) of a system(ex: Computer program System, or physical device)
- objective:-- explicitly testing an hypothesis or
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
-- explicitly testing an hypothesis or -- to solve a class of problems.
- it implies:-- Existence of either a theory or a modelfor explaining the results (testing the hypothesis)
MSR- Methodologies for Scientific Research
1/3/2008
26
Research Techniques
Materials and Methods
• in exact sciences materials include definitions,
• theoretical model is defined (preferably a mathematical model),including boundary conditions (define the environment)and initial conditions (define the initial values of the
• in exact sciences materials include definitions,model, and related assumptions
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
and initial conditions (define the initial values of theparameters of the model) [Losee01]
J. Losee, A Historical Introduction to the Philosophy of Science,4th ed. Oxford Univ. Press, 2001.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Research Techniques
• methods include:rules of analysis and rules of verification and validation
• rules of analysis follow deductive reasoning,statistical analysis and approximations (analysismust often be replaced by simulations due tomathematical tractability problems)
• the theoretical model is verified by comparing the results
rules of verification and validation
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• the theoretical model is verified by comparing the results with reality (measurements with a prototype, known as hypothetical-deductive method)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
27
Research Techniques
• Results
• results can be analytical (deductive), from simulation, or
• use analytical results in simple limited cases to obtain referencevalues
• results can be analytical (deductive), from simulation, ormeasurement results
• it is important to present numerical results thatverify (or not) your own thesis,
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
values
• the value of the results depend on how well they can be generalized (bottom up approach will help you)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Conclusions (constructive technique report)
(1) “We have shown that [some] algorithms proved right…”
(2) “Our approach is simpler to implement than previous algorithms and is
Research Techniques
(2) “Our approach is simpler to implement than previous algorithms and is practically feasible …. On the theoretical side, the greatest deficiency in our results is …”
(3) “In a practical situation the advantages and limitations of our approach are…”
(4) “These results can be extended in a number of directions…”
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1. What is shown by this work and its significance.2. Limitations and advantages.3. Applications of the results.4. Recommendations for further work.
1/3/2008
28
• Destructive Techniques:
Research Techniques
• To analyse a system, or a set of methods to gather new information about those technology procedures under analysis.
- Recognized information and characteristics are then reused may be in a different way
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
- Example: Reverse Engineering
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• There are philosophies permitting the Classification of Research Methods through Paradigms or basic Epistemologies:
Phylosophy Underlying Research
I. PositivistII. Interpretative (post-positivist)III. CriticalIV. (Constructivism)
At Harvard Business School they use just the three categories
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
At Harvard Business School they use just the three categories
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
29
• Positivist Method:
• Assumes that reality is objectively accessible, it can be
Philosophy Underlying Research
• Assumes that reality is objectively accessible, it can be measured and described by the observer and measured by the same observer through their instruments and devices.Tests are made making predictions possible over future phenomena
• IS research (also Telecomm research?) can be of the positivist kind (for IS, Orlikowski and Baroudi,1995)- Use math/logic formalisms
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
- Use math/logic formalisms- use quantifiable and measurable variables- Test an Hypothesis and make inferences from what is known.
MSR- Methodologies for Scientific Research
• Interpretativist Method
• Assumes that accessing to reality is only possible through social constructions like language, consciousness and
Philosophy Underlying Research
social constructions like language, consciousness and ontologies (sharing of concepts’ meaning).
• Hermeneutics-based philosophy (based on the human understanding and text interpretation) as well as in phenomenology(describing, understanding and interpretation perceived phenomena)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
phenomena)
Proposes to abolish separation between "subject“ and "object", It opposes to XIX century positivist thinking.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
30
• Can IS and Telecomm studies influence or be influenced by
Philosophy Underlying Research
the context?
• Here, the focus is not as much on the variables, measurements and processes as it is on interpretation of what is being the results of the test, the measurement or implementation
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• Research Criticism:
• It is a radical enterprise once it tries to identify what constraintsresearch:
Philosophy Underlying Research
research:• socially, culturally,• Historically, politically, • economically, physicallytrying to emancipate science by eliminating causes for ignorance,alienation etc...
Ex: Why is it not possible to do research on X?
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
Ex; Why cannot we implement systems of type Y?
Habermas from Frankfurt School following the way of Adorno, Lukacs, Marcuse
MSR- Methodologies for Scientific Research
1/3/2008
31
• Junk Science (?) Stories of the Past Decade
Dial “F” for Fear.
Philosophy Underlying Research
Dial “F” for Fear.
Since the 1993 Larry King Live broadcast featuring a man suing a cell phone maker claiming his wife died from a cell phone-induced brain cancer, many cell phone users have worried about phone safety.
But studies failed to identify any risk.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• Junk Science (?) Stories of the Past Decade
Philosophy Underlying Research
Powerline scare unplugged.
Fears that electric and magnetic fields (EMFs) created by power lines and appliances caused cancer started in 1978. Parents worried about power lines over schools.
Power companies worried about burying power lines. The National
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Power companies worried about burying power lines. The National Academy of Sciences finally unplugged the scare in October 1996, concluding that no evidence showed EMFs presented a health hazard
1/3/2008
32
• Junk Science Stories of the Past Decade
The Lone-Tree Theory.
Philosophy Underlying Research
The Lone-Tree Theory.
It nearly took an act of Congress to get the researcher behind the notorious “hockey stick” graph, which purports to show a steep rise in global temperature in the 20th century following a millennium of stable temperatures, to release his publicly funded data and computer code.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
Among other dubious presumptions, the graph is derived from data that bases climate estimates for the entire 15th century on the tree ring measurements of a single tree.
MSR- Methodologies for Scientific Research
Quality of Research
1.7 Quality of Research
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
33
Quality of Research
• Quality depends on the Objectives to reach1. To increase available (and also general) Knowledge
• Objectives and Quality criteria are internal to the discipline• Objectives and Quality criteria are internal to the discipline• RIGOUR is very critical
2. Developing Computer Systems– (Instrumentalisme)• Objectives and Criteria are external (regarding R&D)• Relevance can be measured by the impact and is of the
most importance
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Relevance Versus Rigour• Not antagonic but taken with different emphasis
MSR- Methodologies for Scientific Research
• Computer Science (and other Engineering-like areas) tendency:
• Until the 80ies � conceptual (not empirical) papers
Quality of Research
• Case-studies based research is mainly descriptive concerningsimple instances and then less scientific
• Until the 80ies � conceptual (not empirical) papers dominated
• After the 80 ies� Less theoretical papers, more instantiated, less general and Knowledge less cumulative
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Empirical research is now dominated by the descriptive approaches
• Specific Models have limited capabilities for explaining aphenomen as well as little predictive power.
MSR- Methodologies for Scientific Research
1/3/2008
34
Quality of Research
• Requirements for the Research in CS and TC
• Choose an appropriate Research Method:• Choose an appropriate Research Method:
• Make explicit a Body of Theory sometimes coming fromother adjacent disciplines (ex: Computer Science, Organization
• Traditional/positivist, “interpretativiste” or Engineering-oriented
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
other adjacent disciplines (ex: Computer Science, OrganizationTheory, Information Theory, Physics, Statistics…)
• Extend the Theory and Infer predictions to be checked(at least in the case of positivist/traditional research)
MSR- Methodologies for Scientific Research
(following Björn Lisper, Datorteknik, Mälardalens högskolaVästerås, Sweeden)http://www.idt.mdh.se/kurser/ct3340/ht03/
Scientific Methods, General
Theoretical methods:
Create formal models (mathematics, logic)
Define concepts within these
Prove properties of the concepts
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Prove properties of the concepts
Abstraction, hide details to make the whole more understandable(and to make it possible to prove properties of it)
Proofs of properties by deductive methods
1/3/2008
35
Empirical methods:
Perform experiments
See how it turned out
Scientific Methods, General
See how it turned out
Draw conclusions
Simulation:Start with a formal model at some "easy-to-understand" level
Make "artificial experiments" in your computer
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Collect statistics and draw conclusions
In Physics:Make hypotheses about the surrounding world (theory), observe it (experiment)Relate the result of experiment to the theory
Scientific Methods
Adjust the theory if it doesn’t predict the reality well enoughTheory is used to predict the future (e.g., if a bridge will hold for acertain load, or an asteroid fall down on our heads)
Common pattern in Computing Science:The system is constructed to behave according to some theoretical model
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
theoretical modelDeviations are seen as construction errors rather than deficiencies in the theory (hardware error, bug in OS, : : :)
In both cases: the theory helps us understand and predict, but indifferent ways!
1/3/2008
36
Scientific MethodsTheoretical vs. Empirical Methods in Computing Science
Computing Science really has a “spectrum”, from “extremeconstructivism” to a use of theory close to the one in physics:constructivism” to a use of theory close to the one in physics:
• “Extreme constructivism”: (ideal) programming language design:
Formal semantics for the language, pure construction of modeldefining the mathematical meaning of each program
Abstraction of details to make the meaning of the language
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Abstraction of details to make the meaning of the languagesimpler (for instance, assume that data structures can growarbitrarily big)Implement the language according to the semantics
Scientific MethodsTheoretical vs. Empirical Methods in Computing Science
One can prove formally within the model that a program is correct –valuable!
But the model does not cover all kinds of failures. E.g., hardware errors, or stack overflow (or an asteroid falling down on the computer)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
asteroid falling down on the computer)
1/3/2008
37
Scientific MethodsTheoretical vs. Empirical Methods in Computing Science
Extreme “physics” approach: performance modeling of complex computer-and communication systemsand communication systems
Extremely hard to make analytical calculationsSimplified performance models, tested against experiments (e.g., long suites of benchmarks)
Discrepancy leads to a modified theory, as in physics
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Discrepancy leads to a modified theory, as in physicsOften simulation (desire to evaluate systems before building them)
Scientific MethodsTheoretical vs. Empirical Methods in Computing Science
In-between: algorithm analysis
Build on some form of formal model for how the algorithm executs (meta-language with formal semantics), and some performance model (how long does a step in the algorithm take, how much memory is needed to store an entity)
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
1/3/2008
38
Scientific MethodsTheoretical vs. Empirical Methods in Computing Science
In-between: algorithm analysis
Given that the performance model is correct, one provesmathematically that the algorithm needs certain resources (time, memory) to be carried outBut the performance model is often very approximate.
Sometimes is possible to refine the performance model,
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Sometimes is possible to refine the performance model, but this can make it impossible to calculate the resource needs of the algorithm
Quality of Research
• Characteristics of the Research in CS and TC(?)
• Report the S-o-A. • Report the S-o-A. • Prove you made an advancement in Knowledge• Combination of Techniques trying to compensate one’s
weaknesses with other one strengths
• Report about the object of study and the nature of the domain• Validate Data• Analysis using statistics• Extract relationships between variables
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Praticable according to the resources• Relevant results (at different levels: community,society...)• Publish• Ambitious
MSR- Methodologies for Scientific Research
1/3/2008
39
• Characterístics of the research in CS:
• Research may have as an objective:
Quality of Research
• Research may have as an objective:• Building a Theory• Testing a Theory• Extending a Theory
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
• Other characteristics:
• Concerning Data:• Extensive
Quality of Research
• Extensive• Representative• What is Data validity (ex. temporal, precision…)• Granularity:
• Simple or agregate• Specific or large generic experiments
• Temporal Horizon:
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Temporal Horizon:
• “Snapshot” of the domain at a specific point in time• “Longitudinal cut section” (múltiple points in time)
• Methods for colecting and analising data; quantification...
MSR- Methodologies for Scientific Research
1/3/2008
40
Quality of Research
1.8 Challenges
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research
Research in CS and TC Challenges
• Challenges:• There exists shortage of theory. Mainly theories in related disciplines• Phenomena are unstable:
• Data may get old quickly, usually a snapshot that may varyPay attention to the validity period
• Organizations and Markets are always moving
• Significant cultural variations:• Different national/regional cultures imply different speed/rhythm
• When you are doing research (requirements, interaction,questionnaires, demos) you influence the domain
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
questionnaires, demos) you influence the domain
• Pressure for publishing during the research action
MSR- Methodologies for Scientific Research
1/3/2008
41
• Pressure for technical instantiation:
• Prototyping without a theoretical ground or well defined objective
Research in CS and TC Challenges
objective• Economics and business models either not known or consolidate.
• Impact of the automation and rationalization:• Depends on time dynamics concerns• Depends on who are directly affected
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
• Adopting a new technology may take long time• Impact on other partners, markets, standards, tools…
MSR- Methodologies for Scientific Research
• Conclusions on Research actions in CS and TC
• Consider tradoff Rigour Versus Relevance• Identify challenges BEFORE starting doing research
Research in CS and TC Challenges
• Be clear about conventions related with researchto be done
• Select a significative sub-domain • Define questions to be answered through that research• Select the apropriate Research Method and Techniques
(possibly a good combination of some) and justify the choice.
Whenever the selected aproach is instrumentalist, you need
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007
Whenever the selected aproach is instrumentalist, you need to be sure about the external quality, data validation and rigouras well as:Relevance
MSR- Methodologies for Scientific Research
1/3/2008
42
Other Bibliography
• D. Sternberg, How to Complete and Survive a Doctoral Thesis. • D. Sternberg, How to Complete and Survive a Doctoral Thesis. St. Martin’s Press, 1981.
Eugénio Oliveira- 2007 MSR- Methodologies for Scientific ResearchEugénio Oliveira- 2007 MSR- Methodologies for Scientific Research