Is PhD About• Doing more course work? Taking more

interesting, higher-level courses?• Gaining broader knowledge in your

general area (e.g., CS, IS, SE, etc.)?• Developing more sophisticated

systems, better tools/programs?• Positioning yourself for a better career?

So, What’s It All About?• PhD is about discovery through research

and

• Research is about advancing human knowledge (not just yours) in a particular area of inquiry

• The Illustrated Guide to a Ph.D. – By Matt Might

But, then ….• Why take more courses?• Why take comprehensive or breadth

exams?• Where would I use all this research

stuff in the future, anyway?• What if I just want to teach?

– E.g., at a “teaching university”

Research is about “knowing”

• First things first: Why do you want to know?

• More important: When do you really know something?

??? Researchknowledge

“Knowing”• When did we know:

– Earth not being flat?– How to fly?– Structure of the DNA?– Laws of gravitation?– Six degrees of separation in social networks?

• What does it take to really know something?

Richard Feynman on “Knowing”

Research

• Word has a broad spectrum of meanings–“After some research on this

topic ….”–“Years of research has produced

a new ….”

What Research is Not• Not mere information gathering

–Computer Security–HTML 5 for Web Pages

• Not the mere transportation of facts from one location to the next–Computer Crime and Digital Signatures–HTML 5 versus HTML 4

• Not merely rummaging for information

What Research Is• Originates with a question the answer

to which:–makes a difference; is worthy, relevant– leads to new knowledge–builds on relevant existing knowledge

• Requires a clear articulation of a goal–What problem to you intend to solve?–And how do you propose to solve it?

What Research Is• Follows a specific plan or procedure

– how you will reach your goal

• Usually divided into manageable sub-problems• Guided by a hypothesis• Accepts certain critical assumptions• Grounded in objective reality and prior

knowledge• Requires collection and interpretation of data

Scientific Research• Science is an objective, logical, and

systematic method of analyzing and explaining phenomena, devised to permit the accumulation of reliable knowledge.

• The product of science is knowledge

Scientific Research• Premises of science are

– empiricism – objectivity– falsifiability– control

• What this means: – rationality and skepticism are how we learn about the universe and

shape new principles – If the arguments and experiments are sound, if the theory can

withstand skeptical scrutiny, if the work was undertaken within the framework of past research and provides a basis for further discovery, then it is science

Scientific Research• The search for knowledge, with an open mind, to establish

novel facts, usually using a scientific method. – Basic Research (as opposed to applied research) is

discovering, interpreting, and the developing methods and systems for the advancement of knowledge in a given area.

– Scientific Method: systematically observe, formulate hypotheses, make predictions by determining logical consequences of the hypotheses, gather/analyze data, test to see if the world behaves as predicted by hypotheses, revise hypotheses, formulate theories, examines their logical consistency and subjects them to further empirical test.

“Knowing Revisited”• And that’s how we can move toward really

knowing something:

Richard Feynman on the Scientific Method

What is Good Research?• Problem clearly specified and hypotheses

formulated based on observations• Objectives clearly defined - scope, limitations are

fully specified• Process detailed - can be repeated and except

when secrecy is imposed reveal the sources of data and means by which they were obtained

• Design thoroughly planned - make as objective as possible

What is Good Research?• High ethical standards applied• Limitations frankly revealed - there are very few

perfect designs• adequate analysis - data classified to clearly reveal

findings, probability of error should be estimated• Threats to validity (external, internal, construct)

clearly identified and addressed• findings presented unambiguously• conclusions justified

Research Life Cycle• Definition

– Exploratory research defines a new problem, new constraints, new opportunity, or a new approach.

– This initial research must be guided by a hypotheses that will later be further instantiated and specified.

• Initial Solutions– Initial algorithms, designs, theorems, programs are developed.

• Evaluation of Initial Solutions– Initial solutions are evaluated and refined in isolation.

• Comparison of Solutions– Solutions are compared to one another and also to ideal solutions.

• Technology Transfer– Best approaches are transferred to users.

Possible Research Results• a definition of a problem or task • a unit for solving a problem, performing a task • identification of factors influencing the cost, effectiveness, or

applicability of a unit (and relative importance of the factors) • development of an ideal model• a finished unit that can be distributed to users • measurement of some properties of a unit: run time, chip area,

accuracy, efficiency, etc.• more reliability, usability, etc. • identification of problems and shortcomings in a unit. • a demonstration that one unit is better than another. • a definition, demonstration or analysis of a tradeoff • a generative (explanatory) theory for some set of units

Possible Research Methods• writing programs • developing systems • developing architectures • developing content architectures (ontologies, knowledge bases,

class libraries, graphics toolboxes, etc.) • measuring properties of units • finding and proving theorems • analyzing and consolidating previous research results• interviewing experts, customers • performing experiments, surveys, observations, simulation• Importing/adapting techniques and results from other fields • measuring and predicting constraints

Research Project Phases• An individual research project (such as a Ph.D.

dissertation) follows a lifecycle related to the research life cycle: – Choose research question/problem; formulate hypotheses– Determine current state of knowledge– Apply appropriate methods to produce research results

• To verify the hypotheses• To evaluate the proposed solutions

– Write up research results

• Research is not complete until it is written up (and published) – Peer Review is critical!

A Research Project Checklist (I)• Are ideas clear and consistent?• Is the problem clearly specified?• Is it worthy of investigation?• Are there proposed solutions to be explored?• Does the project have appropriate scope?• What are the specific hypotheses / research

questions• What would disprove the hypothesis?• What are the assumptions and are they “sensible”?

A Research Project Checklist (II)• Has the research plan/design been critically

assessed? Are you convinced that it is based on sound science?

• What forms of evidence are to be used?• How are outcomes to be evaluated? What are the

metrics? Why are the selected methods appropriate for verifying the hypothesis or evaluating the proposed solutions?

• What are the likely weaknesses of your solutions?

A Research Project Checklist (III)

• Is there a detailed written research plan?• Have milestones, timelines, and deadlines been

identified?• Do the deadlines leave enough time to receive

feedback on the drafts and to allow colleagues to contribute?

• Has the literature been adequately explored? Once the work is done – and your perspective has changed – does in need to be explored again?

Unstructured Environment• especially in PhD• need to develop a routine• make research a priority• regular meetings with your advisor• join or set up a research group• role of course work• keep a journal - get in habit of writing down ideas

Research Is About Reading• Foundation of good research: find and critically read

related scientific articles• Locate important research tools in your area

– journals and conferences– workshops– important or seminal papers in your area– good research sites and groups– other faculty web sites– develop your own Web repository (why?)

• How to read papers– first look at abstract, keywords, references– next: for some papers look at intro. and conclusions– next: for some, read in detail and write notes

What to consider when critically reading papers (write a summary for every paper you read)

• what is the main problem? why is it important?• what is the hypothesis? how precise are the claims?• what are the main results (in your own words)?• how does it relate to previous work? what are the most

important references?• what is the evidence and how was it gathered?• what are the metrics and how are measurements taken?• how carefully are the algorithms and experiments described?

what would the reproduction of the results involve?• why and how are the results significant?• ideas for future work?

Exercise

• Select one recent paper relevant to you area of interest

• Critically read the paper (using criteria in previous slide)

• Submit the paper and your notes in two weeks

• You may be asked to summarize in class

Research Is About Writing• writing is the most fundamental part of research• writing not just an end, but also a tool• writing <=> logical reasoning• take a writing course• when reading papers, mark those that are especially well-

written• keeping a journal, revisited• consider writing a survey paper• write one or more research proposals• Note: programming or math are not writing, by

themselves; programming is also not research

More Practical Advice

How to Be a Good Graduate Student– Marie desJardins

http://www.cs.indiana.edu/how.2b/how.2b.html