Science, Research Ques3ons, Hypotheses, &...
20
2/12/16 1 Science, Research Ques3ons, Hypotheses, & Visualiza3on Today • Reading discussion: “LineUp: Visual Analysis of Mul3- AHribute Rankings” • How to read a research paper? (What makes a good research paper?) • Science, Research Ques3ons, and Hypotheses • Friday Crayon Exercise • No class on Tuesday! Reading for Friday
Transcript of Science, Research Ques3ons, Hypotheses, &...
2/12/16
1
Science,ResearchQues3ons,
Hypotheses,&Visualiza3on
Today
• Readingdiscussion:“LineUp:VisualAnalysisofMul3-AHributeRankings”
• Howtoreadaresearchpaper?(Whatmakesagoodresearchpaper?)
• Science,ResearchQues3ons,andHypotheses• FridayCrayonExercise• NoclassonTuesday!ReadingforFriday
2/12/16
2
ScaHerplotMatrix
hHp://www.mathworks.com/help/stats/examples/visualizing-mul3variate-data.html
Colorcodedby#ofcylinders
“Thetablelens:merginggraphicalandsymbolicrepresenta3onsinaninterac3vefocus+contextvisualiza3onfortabularinforma3on”RamanaRao,SIGCHIConferenceonHumanFactorsinCompu3ngSystems1994 Focus+context
2/12/16
3
“Slopegraph”
from:EdwardTuaeTheVisualDisplayofQuan3ta3veInforma3on1983
“Bumpchart”
TimeGranger,UniversityofCambridge
2/12/16
4
ParallelCoordinates
hHp://eagereyes.org/techniques/parallel-coordinates
2/12/16
5
Tableau
hHp://www.tableausoaware.com/
ReadingforToday
• “LineUp:VisualAnalysisofMul3-AHributeRankings”,Gratzl,Lex,Gehlenborg,PfisterandStreit,IEEEINFOVIS2013
2/12/16
6
• Visualizingoverallrankingissimple,interpreta3onofrankingisnotsimple
• Alterna3verankingsexists• Customizeweightsofmul3pleheterogeneousaHributes
• Customizefunc3onforapar3cularaHribute:linear,logisthiseasyfortheusertodo?Givealargersetoftypicalop3onsratherthantypejavascript?
• Comparedifferentpublica3onsrankings,comparerankingsover3me
• Cost-benefitanalysisforimprovingrankings• Rankingsshouldbepersonalized,weeachhaveourownpriori3es
• Take“rankings”toanewlevel• Previouslythoughtbarchartswereboring,notcrea3ve
Requirementsanalysis• Listof10detailedrequirements,including:– Nestedmodelforvisualiza3ondesignandvalida3on– Domainproblemcharacteriza3on– Datamustbenormalized,allowfiltersofdata,handlemissingdata(unfortunatelydiscussion/solu3onrathervagueinpaper)
– Butkeepingtrackof10criteriawasaliHlehardtofollow– Someofthe10requirementsseemedredundant
• Didgoodjobdeveloping&describingtheproblem&requirements
• Formalizesgeneral(intui3ve,agreedupon)criteria• Interac3oniskeytoeffec3veuserinterpreta3onofdata
2/12/16
7
Features• Columntypes
– Rankcolumns– TextualaHributecolumn–labels,nominalaHributes– CategoricalaHributecolumn– NumericalaHributecolumn–encodedasbars!– CombinedaHributecolumn
• HistogramabovetheaHributes–confusing,notsufficientlyexplained• Interac3vity
– Interac3vebargraphploHerwithcustomizablefilters¶meters– Immediatefeedbackwhenweightsarechanged– Animatedtransi3ons,colorhighlight–strength/lengthof3meapplied,basedon
amountofchange– “Snapshot”forcomparingrankingsbefore/aaerweightedits– Collapsing/compressingcolumns
• Usefewerpixelstodisplaysamedata(canbedonewithbargraphs,notwords/rawnumbers)• “Fisheye”viewallowingagradualfalloffinlevel-of-detail/screenspace• CleanupcluHer• Replaceconnec3onlinestoinvisibletargetswitharrows
FeaturesII• Memopad:placetoputaHributesnotcurrentlyinuse.• Consistent,customize-ablemappingbetweenaHributeandcolor– toomanycolorsmightbecometoobusy/confusing
• Parallelmappingeditor• Orthogonalmappingeditor• AllowsusertoinfertheimportanceofaHributestoranking
• Isthistoomanyfeaturesinoneindustrialstrengthapplica3on?WoulditbebeHertohaveacollec3onofindividualtools/plugins?
• Toomanycontrols,notenoughguidancefornovicesonhowtousethem– Tool(and/orpresenta3onoftool)isneedlesslycomplex?
2/12/16
8
Evalua3on• HeavyfocusonUserStudyinpaper• Comparetoexis3ngtools:ConfirmedthatexpertsinExcelcould
completetasksthatnoviceswouldbeaskedtoperformwithLineUp–However,didnotdoformaluserstudycomparisonbetweenExcel&Lineupbecauseexper3seissuchahugecomponent
• Checkforcolorblindness–why?• 12separatetasks• NASATLXworkloadassessmentques3onnaire,taskcomple3on3mes• Jus3fica3onfortheirexperimentswasearlyinpaper&detailed,synced
wellwithstudy&results,studymeasuredtheappropriatethings• Welldesignedstudy,butonlya3ny#ofpar3cipants(8)• Usersinstudywereunrealis3c?Targetaudienceis“expertamateurs”?
2/12/16
9
Results• Authorssurprisedthatfiltertasktooklongerthanmostothertasks
• Usersthoughtitwasvisuallypleasing,poten3allyhelpful,easytounderstand,and(comparedtoExcel)wouldsave3me,helpgathermoreinsights
• Drag&dropiseasiertocreatecomplexrankingsthantypingformulas
• Somenega3vecommentsaboutrankingschangefromuserwhodidnot[knowto]usethesnapshotfeature
• Doesthisscaletobiggerdatasets?Isitvisuallyclear?Isitreasonabletonavigate?
Futurework• Featurestohelpop3mizerankings,whatneedstobechangedtoachieveacertainranking?– Whyisthisanimportantfeature?(askanyuniversityadministrator…)
• Sta3s3caltechniquestodealwithlargenumberofaHributes
• Largedifferencesinrankingsbecomesteepslopeswhicharehardtointerpret
• Needtotestactualusersinreal-worldanalysesandobserveadop3onrates– Web-basedimplementa3on– Applytoolfororiginalpurpose:rankingofgenes,clusters,pathwaysforanalysisofgenomicdata
2/12/16
10
Wri3ngQuality• PaperwaswellwriHen• Jus3fica3onfortheirexperimentswasearlyinpaper&detailed,syncedwell
withstudy&results,studymeasuredtheappropriatethings• Whoistheaudience?
– Clientsdesigningrankings– Orpeoplewhoneedtolookatrankings– Domainexperts?– Novices?
• Lotsoffigures!Great.Butsomedetailsinsomefigureswereconfusing/hardtofollow
• Wouldhavebeennicetoincludealgorithminpaper• Mul3plereal-worldexamplesgood--Nutri3onexamplehelpedexplainthe
datamappingeditor• Scrollinguptothetopofthepaperevery3metheymen3onFigure1ora
specific#requirementfromthelistof10wasannoying• Gooduseofboldtext• Somepartswererepe33ve
Defini3ons/Misc.
• Likertscale:onasurvey,responsefromasetofchoicesonascale,commonly:stronglydisagree,disagree,neitheragreenordisagree,agree,strongagree
• Algorithmdesign,encoding/interac3ondesign– “Anestedmodelforvisualiza3ondesignandvalida3on”,TamaraMunzer,TVCG2009
– hHp://www.cs.ubc.ca/labs/imager/tr/2009/NestedModel/NestedModel.pdf
2/12/16
11
Today
• Readingdiscussion:“LineUp:VisualAnalysisofMul3-AHributeRankings”
• Howtoreadaresearchpaper?(Whatmakesagoodresearchpaper?)
• Science,ResearchQues3ons,andHypotheses• FridayCrayonExercise• NoclassonTuesday!ReadingforFriday
Howtoreadaresearchpaper?
(especiallyanadvancedpaperinanewarea)
2/12/16
12
Howtoreadaresearchpaper?
(especiallyanadvancedpaperinanewarea)
• Mul3plereadingsareoaennecessary• Don'tnecessarilyreadfromfronttoback• Lookupimportantterms
• Targetapplica3on&claimedcontribu3ons• Experimentalprocedure• Howwellresults&examplessupporttheclaims
• Scalabilityofthetechnique(ordernota3on)• Limita3onsoftechnique,placesforfutureresearch• Possibili3esforhybridsystemswithotherwork
Componentsofawell-wriHenresearchpaper?
2/12/16
13
Componentsofawell-wriHenresearchpaper?
• Mo3va3on/context/relatedwork
• Contribu3onsofthiswork• Cleardescrip3onofalgorithm– Sufficiently-detailedtoallowworktobereproduced
– Workistheore3callysound(hacks/arbitraryconstantsdiscouraged)
• Results– wellchosenexamples
– cleartables/illustra3ons/visualiza3ons• Conclusions– limita3onsofthemethodareclearlystated
Today
• Readingdiscussion:“LineUp:VisualAnalysisofMul3-AHributeRankings”
• Howtoreadaresearchpaper?(Whatmakesagoodresearchpaper?)
• Science,ResearchQues3ons,andHypotheses• FridayCrayonExercise• NoclassonTuesday!ReadingforFriday
2/12/16
14
HomeworkAssignment4:DataCollec3on&DataPrepara3on• Iden3fyaninteres3ng(toyou)andinteres3nglylargedata
sourcethatiscurrently“untapped”– Shouldn’tjustbea“downloadafile”dataset– Canberesearch-related[Required?forgradstudents]
• Useyourprogrammingskillsto:– Obtainthedata(whatshouldyoucollect?)– Parse/Organize/Simplify/Post-Processthedata
• Visualizethedatausingatoolthat’snewtoyou…Excel,LineUp,Tableau,GoogleAnaly3cs,orPlotly– One“obvious”plot– Oneplotthatreorganizesthedatatoansweramorecomplexor
subtleques3on• [Op%onal]Workinateamof2(someoneyoudidn’tknow
beforethisclass(?)
2/12/16
15
Theobviousplot…
Butwhatarethemorecomplexorsubtleques%onswecanask(andanswer!)nowthatwehavethisdata?
• Doesanindividualstudent’sgraderiseover%me(repeatedsubmissions)?• DostudentswhostartsubmiIngearlierintheweekhaveahigherfinalgradeforthathomework?• Dostudentswhosubmitmore%mesgethighergrades?
hHps://oasis.cs.rpi.edu/
2/12/16
16
WhatisScience?fromwikipedia&dic3onary.com
• Science:theintellectualandprac3calac3vityencompassingthesystema3cstudyofthestructureandbehaviorofthephysicalandnaturalworldthroughobserva3onandexperiment
• Hypothesis:asupposi3onorproposedexplana3onmadeonthebasisoflimitedevidenceasastar3ngpointforfurtherinves3ga3on
WhatisResearch?fromwikipedia&dic3onary.com• Researchques3on:Specifyingtheresearchques3onisoneofthefirst
methodologicalstepstheinves3gatorhastotakewhenundertakingresearch.Theresearchques3onmustbeaccuratelyandclearlydefined.Choosingaresearchques3onisthecentralelementofbothquan3ta3veandqualita3veresearchandinsomecasesitmayprecedeconstruc3onoftheconceptualframeworkofstudy.Inallcases,itmakesthetheore3calassump3onsintheframeworkmoreexplicit,mostofallitindicateswhattheresearcherwantstoknowmostandfirst.– ToasktheNSF(Na3onalScienceFounda3on)forresearchmoney,wehave
tohaveaclearresearchques3on.Manycomputerscien3stresearchersfallinthetrapofsayingtheywilldevelopcode,orimplementsomethingtosolveaproblem.Describingourworkwiththistermscandoomaproposal,becausetheseac3vi3esarenotscience(it’snottheNa3onalR&DFounda3on!)
• ThesisStatement:Athesisstatementisashortstatementthatsummarizesthemainpointorclaimofanessay,researchpaper,etc.,andisdeveloped,supported,andexplainedinthetextbymeansofexamplesandevidence.
2/12/16
17
Today
• Readingdiscussion:“LineUp:VisualAnalysisofMul3-AHributeRankings”
• Howtoreadaresearchpaper?(Whatmakesagoodresearchpaper?)
• Science,ResearchQues3ons,andHypotheses• FridayCrayonExercise• NoclassonTuesday!ReadingforFriday
Today’sCrayonExercise• Teamupwithsomeoneyoudidn’tknowbeforethisclass
(notthesamepersonaslastweek)• Brainstormatleast5new,big,“untapped”datasources
– Atleastonethatisprac3callyobtainableforHomework4– Atleastonethatyoucouldobtain…butitwouldtakemore
%meand/ormoneythanisprac%calforHomework4– Atleastonethatisunobtainable..becauseit’sillegal,it’sunethical,
thesensorhasn’tbeeninventedyet,etc.• Pickoneofthesedatasetsandcreateadetailedformatforthedata
(thecolumnsoftherawspreadsheet)andinsertafewrowsofsamplenumbers/text.
• Writedown2specificResearchQues3onsthatcanbesolvedbystudyingthisdata.One“obvious”andonemorecomplexorsubtle.
• WhatisyourHypothesisfortheseques3ons?• Sketchtwosimpleplots(line/barchart,node/edgegraph,etc.)thatwill
answertheseques3ons.Labeltheaxesclearlyanddrawrealis3clookingdetailsforthedatathatwouldconfirmyourhypotheses.
2/12/16
18
hHp://www.theresearchassistant.com/tutorial/2-1.asp
hHp://www.theresearchassistant.com/tutorial/2-1.asp
2/12/16
19
hHp://www.theresearchassistant.com/tutorial/2-1.asp
hHp://www.theresearchassistant.com/tutorial/2-1.asp
2/12/16
20
Today
• Readingdiscussion:“LineUp:VisualAnalysisofMul3-AHributeRankings”
• Howtoreadaresearchpaper?(Whatmakesagoodresearchpaper?)
• Science,ResearchQues3ons,andHypotheses• FridayCrayonExercise• NoclassonTuesday!ReadingforFriday
"AngularBrushingofExtendedParallelCoordinates",
Hauser,Ledermann,andDoleisch,InfoVis2002
ReadingfornextFridayPostacomment/ques%ononLMSbyFriday@10am
