Writing & Reading Workshop
description
Transcript of Writing & Reading Workshop
Writing & Reading Workshop
Today´s programmeWorkshop on:
writing experiment reports how to read research papers
– expect to be challenged ...
Will not go through pensum in detail – focus on utilizing your acquired knowledge Ask questions as we get to each section of
the reports
Today´s programme Exercise: Writing a review of Law et al.: ”Understanding,
Scoping and Defining User Experience: A Survey Approach” Following the guidelines in Philip Fong: ”How to Read a CS
Research Paper?”
We will do the first parts in steps throughout the day, the latter parts you will complete at home and hand in next week
Some points may be to answer because they require knowledge of the field of user experience – take your best shot or skip them.
Practical information
We have been approved (horray!) with breaking from 11.45 to 12.45 – same exercise room (4A58)
Re. the 28/9 – heard from 2 students – weren´t there 3???
Today´s test if you read the curriculum: Handout by Graham Hole is exact copy of chapter 9 in the book – how many discoverd this?
Handouts for exercises on the course website
The game lecture series
Center for Computer Games Research Game Lecture Series
Runs on Thursdays at 15.00 around every 2 weeks in the semester
Talks of interest to you: 24th of September: Janus Sørensen from
IO Interactive on user testing 19th November: Lennart Nacke on bio-
physiological measures of user experience
Paper review - assignment1-2 page review following the recipe
of Philip Fong
1) Summary of the paper in your own words
2) Evaluation of the work Is the research problem significant? Are the contributions significant? Are the claims valid?
3) Synthesis: summarise your thoughts on the paper
Writing scientific reports
Writing scientific reports
Reports are based on the idealised scientific journal article
Reports are divided into sections which provide a specific piece of information each
This standard format is used in virtually all the sciences (not humanities) and with minor modification, the industry
Being able to write reports is a key skill
Writing scientific reports
Purpose:
To communicate research findings to others in a clear, systematic and standardised way.
Clear: So we get it ...Systematic and standardized: So
we can quickly read the paper, know where things are
Sections Sections of a report:
Title Abstract (summary) Introduction Method: Participants Design Procedure Results Discussion/Conclusion References Appendices
10-20%
50-70%
15-25%
Aims of a report 1. "Why?". Why did I do this particular experiment? What
did I expect to find out by doing it? This question is dealt with in the Introduction.
2. "How?". How did I actually carry it out - what procedures and apparatus did I use?
This question is covered in the Method section.
3."What?". What did I find? What were my results? This information is provided in the results section.
4."So What?" What does it all mean? How do my results relate to previous research on the same topic, and what are their theoretical implications? What are my conclusions?These issues are all dealt with in the Discussion and Conclusion sections.
Other components
Title Provides quick idea what the report is
about
Abstract Summary of the contents and main
conclusions Should keep the attention of the reader
References Documents arguments in report and
provides means for locating further knowledge
Title Ask yourself: Do you want anyone to
actually read the report?
If yes, use a title that catches people´s attention If no, use a really long and boring title
Informative but succinct (10-12 words maximum) - e.g.:
“Gender differences in attitudes to "Bambi" amongst Navaho Indians”
“The effects of nationality and age on sun-bed claiming behaviour”
Report titles ...
Abstract
An abstract is a brief summary of the report
Typically 120-150 words
Give reader: (a) what you did: Quick overview of
contents (b) what you found: The main results (c) what it means: Theoretical/practical
implications
Typically easiest to write the last
Keywords
Typically 3-5 keywords of the report Nominally provides the reader another quick
idea about the contents + is used as search words in databases/browsers
Consider One thing is what the paper is about, another
is what the current buzzwords in your field is Ensure you know which keywords to use to
make people find/not find your report in library/company databases
Executive Review
Used business reports and white papers, or experiment reports in the industry
The same thing as an abstract, but tuned to the specific audience/stakeholder Design department Management Shareholders Etc.
Introduction Introduces the reader to the overall problem
area first, then focuses on the specific problem under investigation
General problem introduction and statement (why?), e.g.: - gaps in previous knowledge; - methodological flaws in previous work.
Review of current work in the field Discussion of any relevant literature and theories Overview of the report contents
Brief outline of our experiment, and its possible outcomes and their implications.
Why are we bothering to do our study?
The METHOD section (dun-dun!)
Method Describes how the experiments were
run Describes what data that were
collected and how they were analyzed
Argues for why the approach is correct
Introduction + hypotheses Design Participants Apparatus Procedure
Method
Provide enough detail that experiment can be
replicated, but not any irrelevant information
Hypotheses
In HCI research papers, there will normally be specific hypotheses and research questions that are sought answered
These are normally – not always - described in the method section
In commercial HCI reports, often there are no hypotheses, but specific purposes with the test/experiment (sort of the same thing...)
Participants
Who took part in the experiment?What were their characteristics?
(demographics – age, gender, nationality, experience)
Volunteers or paid?Randomly selected or not?How were they allocated to the
different conditions of the experiment?
Experiment conditions
Typically we have a control group and an experiment group
The experiment group we do something with – ”Will this pill turn people purple?”
The control group we do NOT submit to an experiment – they permit us to measure the effect of the experiment reliably
Materials (apparatus, equipment...)Also called ”setup” in many HCI
situations
Basically describes the equipment of the experiment + measurement instruments
Equipment: computers, watches, cameras, software etc.
Instruments: surveys, pulse watches etc.
Diagrams and photographs are often used
Procedure
Explain how the experiment was carried out
Usually pretty detailed with exact description of what was done to the participants, and what they had to do
Order of the tests being administered
Temporal frame: How long? When? Etc.
Results
An objective walkthrough of what results that have come out of the experiment
Do not interpret numbers here – just report them
Only critical raw data and summary statistics are included in the report ”76% of people turned purple...” The rest is in appendices
Results
Two overall parts in the results
1) Descriptive statistics These describe datasets ”35 samples, mean weight 2 kgs, SD=0.2
kgs”
2) Inferential statistics These allow us to answer research
questions ”An ANOVA shoved that the iron meteors
had statistically different weights than ice meteors”
Results
Descriptive statistics
Start by providing details for each group or experiment condition, e.g. Means
Follow up with inferential statistics (the results of statistical tests), to decide whether there was any difference between the groups or conditions
Results
inferential statistics ...
State which test you used The value of the ”test statistic”Number of degrees of freedom
(where appropriate)Significance level
We will get back to these – just remember that you
need this stuff in the results!
Results
Example:
”An independent-means t-test was performed. This showed that participants who had received 40kg of the drug ‘Coffee' recalled significantly fewer words than those who had consumed 20 mg (t (29) = 3.65, p < . 001)"
Statistical test
Significance levelTest statistic value
Results
If you have many results, summarise them in tables and graphs
Do not dublicate information (in both tables and graphs)
All graphs and tables should be clearly labeled with self-explanatory titles and legends (i.e. no need to read the main text to understand what the figure shows)
Results
Figure X: An initial analysis of the causes of death in Fragile Alliance as function of time. The figure charts the percentage distribution of kills for AI-controlled autonomous agents and players.
Axis labels + measurementunits Graph label
Title
Figure text
Clear data presentation
Discussion
Interpretation of the results What do the results mean? Explain what the statistics mean
Relation to other works What have other people found?
Impact What is the importance?
Critical reflection Criticize your own work, indicate possible
weaknesses, design flaws, limits of application of results?
Research agenda (what now?)
Discussion: Research agendaWorthwhile suggestions for future
research? Extension to other situations, to see
if the results are generalisable [applicable to entire population we are interested in]
Replication with a better-matched set of participants.
Explain briefly why these studies would be worth doing.
Conclusion
Can be sub-sumed under discussion
Quick summary of the results, their meaning and importance, and setting of future research agenda.
If these are in conclusion, remove from discussion
Acknowledgements
This section should give thanks to the major people (supervisors, associates) and organizations (sponsoring agencies, funders) that helped you.
Mostly in research articles
For example: “The authors thank Spiderman, who provided constructive feedback on earlier drafts of this report. Also a heartfelt thanks to Captain Kirk, who financed the project”.
?
Task 1: Research problem You get 1 hour to give the paper a quick
read-through (”skimming”)
Use this time to orient yourself in the paper (hint: start with the abstract), and discuss in the group if there are components you do not understand
After 45 minutes, discuss in the group: What is the research problem/-s the paper attempts to address? (use Fong, section 2 point 1 as a guide)
Sections Sections of a report:
Title Abstract (summary) Introduction Method: Participants Design Procedure Results Discussion/Conclusion References Appendices
10-20%
50-70%
15-25%
References
ALL papers, reports, artwork or similar was cited in the report, are listed here in full detail
In alphabetical order, very standardized format
E.g.: Buonaparte, N. and Nelson, H. (1805,
March 21). An analysis of sun-bed claiming behavior in Western Europe. Seafaring Weekly, 75, 46-49.
References
References must provide enough informtion for the reader to be able to track it down
(unless you don´t want to of course)
This allows the reader to: Seek further knowledge on the subject Evaluate if you have used the reference
correctly (science is all about mistrust, as Graham and
Andy puts it)
References
There are many referencing formats
Journals, conferences and companies have different standards
Some common standards, like APA, CHI, Science, Chicago, ACM, Harvard etc.
Here we use the APA format – described in detail in our textbook
You must use this in all reports for this course
References
In-text references usually something along the lines of (single-author): (Spock, 2005) Studies by Spock (2005) indicate that ... Studies by Spock (2005, 2007) indicate
that ... It has been claimed that aliens are
illogical (Spock, 2005) The results of Spock (2005) are at odds
with those of Kirk (2004) on the matter of aliens
References
If multiple authors
”Spock and Kirk (2006) agreed that aliens are indeed weird”
If more than 2 authors
”Spock et al. (2007) argue that aliens are green”
Figure 1: Are aliens weird?
References
Multiple references ...
”Aliens have been claimed to be alien (Kirk, 2004; Spock, 2005; Spock et al., 2007)”
References Primary and secondary references
Primary: Those you have read (ideally, ALL your references
Secondary: Those you quote because it makes you look
smart ... Because someone else referenced them and you
should probably do it too ... (looks important) Because the primary is in Swahili and you don´t
read Swahili To avoid getting bogged down with detail – if
someone else has done the work, quote that -
References
People say you should be honest about it if you use a secondary
reference:
"It has been claimed that aliens can be cured by killing them (Kirk, 1995, as cited in Spock, 2000)".
“Kirk (1995, as cited in Spock, 2000) claimed that aliens can be cured by killing them".
I say: Adopt and adapt ...
References
LEARN TO DO PROPER REFERENCING!
Appendices (research+industry)Experimental materials
Location of programs, instructions, questionnaires, test materials, transcripts of sessions, photos of screen presentations if required, etc.
Pilot study results Preliminary/pilot experiment record, what you did and saw, how and why you modified the experiment.
Appendices (research+industry)Raw Data
Raw data elicited from subjects, verbal comments from subjects, videotapes, etc. Large amounts of data are best kept on discs. Public data should be available on ftp for replication purposes.
Statistical runs/output from softwarePrintouts of your statistical analyses.
Experimental consent forms/participant instructionProof that your subjects consented to do the experiment, and any other relevant documentation
Stimulus material
SummaryA report is build over a standardized
model
A report always contains specific sections, each with their specific purpose
Always use a consistent referencing style
Task 2: Contributions – 20 mins.Re-browse the abstract, introduction
and conclusion sections
What contributions do the authors claim to make with the paper? Answering a question? Providing new
knowledge? Evidence for something?
(use Fong, section 2 point 2 as a guide)
More on report writing
Yet more on report writing ...
Quotations
The standard format
Target audience
Writing style
Quotations
You use quotations when you want to cite something someone has said word by word
Quotations are usually <40 words, and marked by: ” ”
Either inserted in-text, or in a separate paragraph, indented
”I do believe that Kirk is wrong when he says that
we come in peace (Spock 2005, p. 123)”.
Quotations
Supply the full reference for the text you quoted in the reference list
Quotations must always be accurate – you may not change anything in formatting or wording
If you do, add: ”[whatever change done]” – to show you have changed the quote
Quotations
If you omit text from the original quote, use ”...”
”Spock is a stuck-up vulcan clown who does not understand aliens, planets, quasars, starships or space navigation”
”Spock is a stuck-up vulcan clown who does not understand aliens ... or space navigation”
Be careful: Omitting text can change the meaning of a quote!
Quotations
Quotations should not substitute trying to use your own words to explain things
That being said ... adopt and adapt
(don´t copy!)
Thesaurus: Learn to love it.
Task 3: Claims – 20 mins.
This is where it gets difficult: How do the authors substantiate their claims? Answer the research question?
What is the methodology used and does it fit with the purpose?
What makes their claims scientific?
(use Fong, section 2 point 3 as a guide)
The standardized format
The advantage of the standardized format is that it is standardized
Once you know how to do it, it is easy to do it again – like filling in the blanks
As everything in HCI is written in this format, it is easy finding the information you need in reports/papers You know exactly where stuff is ...
Target audience Keep your audience in mind
Who are you writing to? What do they know?
What style of language should you use for them to be able to read the report?
Normally the reader is an interested, intelligent person who is familiar with the general area of your work (a non-specialist)
Only expect the reader to know what you have already written in the report
Substantiting claims
Substantiate your claims
When you claim something, make sure you argue for it – either by providing evidence or referencing someone who does
”Aliens are not from Earth” – claim”Aliens are not from Earth (Spock,
2005)” – substantiated claim
Writing style
Write in a clear but formal style, use past tense
Avoid biased language in your writing
Use only abbreviations if they are commonly used by your target audience
10 or above, use numbers. If part of a comparison, use numbers
under 10: ”3 of 20 students are awake”
SummaryPurpose of reports is to communicate
research findings in a clear and systematic way
Your reader knows only what you have told them so far
Substantiate all claimsUse the passive past tense, avoid
jargon
Be prepared to rewrite your report a few times
Don’t worry - writing good reports takes practice!
Task 4: Conclusions – 20 mins.This is the easy one (all you need to
do is read the conclusion section)
What have we learned from the paper?
In what areas can we use the results?
(use Fong, section 2 point 4 as a guide)
Questions?