Post on 31-Dec-2015
P.B. Medawar. Advice to a young scientist. Basic Books. 1979.
Medawar’s Experimentation Models & Computer Science
Peter Medawar
Nobel Prize for Medicine 1960• 1915- 1987, born in Rio de Janeiro, son of a
Lebanese business man who was a naturalized British subject.
• Bachelor’s degree from Oxford in 1932. • Worked on tissue grafts and transplants
Solutions
Solving a problem simply means representing it so as to make the solution transparent.
Herbert Simon
Research is the art of the soluble. Peter Medawar
How Do we look at things?
El Greco test
Medawar’s Experiments and Discovery
Four kinds– Baconian (observe)– Aristotelian (effect)– Galilean (hypothesis)– Kantian (thought)
What does it mean to do experiments in CS?
1. Baconian Experimentation
Find truths by careful examination of things as they are Compilation of facts Contrived performance rather than natural occurrance No control group, no theory Examples
– Magnetising nails– Static electricity in silk
Trying things out or mucking about
Baconian experimentation in CS
Early IR KWIC/ KWOC indices Zipf distribution Counting word occurrences and distributions
2. Aristotelian Experimentation(John Glanville, Royal Soc. 1636-84)
Demonstrate some preconceived idea– Ring a bell before giving the dog his dinner
Effect without theory Examples of X CS??
Aristolelian Experiments in CS
Eliza
Bob
Pop up ads
IR data visualizations
Post hoc, ergo prompter hoc
Psych: Why do you flail your arms around like that?
Patient: Keeps the wild elephants at bay.
Psych: But there aren’t any wild elephants here.
Patient: That’s right. Effective, isn’t it!
4. Kantian Experiment
Thought experiments Examples
– non-Euclidian spaces– Parallel lines that meet
Let’s look at that differently
Kant meets CS
N-dimensional vector spaces
Shneiderman data walls
Hypercube Web graph Data visualization
3. Galilean Experimentation
Expose hypothesis to a test Dropping of canon balls off Pisa tower to test his
hypothesis of gravitational acceleration Leads to the null hypothesis
– Experiments can not really prove anything!– Best you can do is refute the null hypothesis– I.e., that you have done better than wild good luck– Looking at results of differences of observations
Be prepared to take “no difference” as an answer
Hypotheses
I cannot give any scientist of any age better advice than this: the intensity of the conviction that a hypothesis is true has no bearing on whether it is true or not.
Medawar
Galilean Experimentation in CS
Algorithm efficiency Algorithm effectiveness User preference Etc
Not withstanding
Simpson’s Paradox– 2 data sets -> separately support a conclusion– BUT the union supports the opposite conclusion
Will Roger’s Phenomena– In a patient study, it is possible to transfer a patient from one
group to another and improve the statistics of both groups
Mark Twain’s Observation– Lies, damned lies and statistics!
How to be prepared to do research I
Mastering the literature– Too much
Confine the imagination Psychological substitute for research
– Too little Make an idiot of yourself
– Mix some eclectic breadth with selected depth Eg. ACM Communications and IJHCI
How to prepare II
Get on with it– Get results
Repeat others work Try variations Try other data
– Join the discussion When I tried that … I got exactly the same results when I… I agree, for this purpose x is better then y
How to prepare III
Follow the art of the soluble Start with a “soft underbelly problem”
– Quantification of vague phenomenon– Isolating factors– Selecting feature sets
To quantify is not to be a scientist, but it does help. (Medawar)
Also part of the Scientific Process
Devising hypotheses that can be tested in a practical manner
Imaginative guesswork Exercise of common sense All experimentation is a form of criticism Having the right slot in your mind to put a new
observation or idea Good luck counts Accept flux. Science as a Maoist microcosm of
continuing revolution.