ShadowDraw Real-Time User Guidance for Freehand Drawing
25
ShadowDraw Real-Time User Guidance for Freehand Drawing Larry Zitnick, Michael Cohen Microsoft Research Yong Jae Lee U. of Texas at Austin
description
ShadowDraw Real-Time User Guidance for Freehand Drawing. Larry Zitnick, Michael Cohen Microsoft Research. Yong Jae Lee U. o f Texas at Austin. Drawing. Subject 1. Subject 2. We need help. Tracing. Difficult to find photos Limits creativity. Drawing a face. Learn a set of rules - PowerPoint PPT Presentation
Transcript of ShadowDraw Real-Time User Guidance for Freehand Drawing
ShadowDraw Real-Time User Guidance for Freehand Drawing
ShadowDraw Real-Time User Guidance for Freehand DrawingLarry Zitnick, Michael CohenMicrosoft ResearchYong Jae LeeU. of Texas at AustinDrawing
Subject 1Subject 2We need help..Tracing
Difficult to find photosLimits creativity
Drawing a faceLearn a set of rulesRequires significant training
4
Our Idea: ShadowDrawShadowDrawCreating shadows
=ShadowShadow gives gist of many images simultaneously
+Collection of imagesCreating shadowsWhat if the only input is the users drawing?
ShadowDrawingRequires partial matching and dynamic updatesSpeedReal-time updating of shadows is critical
0.1 seconds feels right0.5 seconds useful2 seconds useless
OverviewUser DrawingFor each sub-windowHistogram(ID x dx x dy)Top 100Fine AlignmentSpatial ScoringShadow
VerifyQuery time (online)Database Image
EdgesFor each sub-windowSketch, Img_ID, x, y.Sketch, Img_ID, x, ySketch, Img_ID, x, ySketch, Img_ID, x, yMin Hash
Database (offline)Database
DatabaseEfficient (sub-linear) retrievalStores image index and offsetInverted file structure:
[Zitnick, ECCV 2010]
Edge descriptor:PositionOrientationLinear lengthDatabase Image
EdgesFor each sub-windowSketch, Img_ID, x, y.Sketch, Img_ID, x, ySketch, Img_ID, x, ySketch, Img_ID, x, yMin Hash
Database (offline)MatchingSecond stage: Refine alignment and compute blending weightsFirst stage: Use inverted file structure to find candidate set User DrawingFor each sub-windowHistogram(ID x dx x dy)Top 100
Fine AlignmentSpatial ScoringShadow
VerifyQuery time (online)Fine alignmentHough transform (3D)Break into three 1D problems:YXXYS (scale)SUser DrawingFor each sub-windowHistogram(ID x dx x dy)Top 100
Fine AlignmentSpatial ScoringShadow
Verify
Blending weights
=Shadows are the composite of many images**+Pen StrokesTop matchesWeights
Blending weights=**+
Shadows are the composite of many images
Top matchesWeightsPen Strokes
Rendering
+Pen StrokesPen PositionShadow
*+=OutputHigher contrast near pen positionMain focus is users drawingRelated workInteractive drawing interfacesTeddy [Igarashi et al., 1999]Fluid Sketches [Arvo and Novins, 2000]3D drawing system [Igarashi and Hughes, 2000]iCanDraw [Dixon et al., 2010]
Drawing studiesWhere do people draw lines? [Cole et al., 2008]
User studies30,000 images, 20 categories16 drawers (8 men, 8 women), 8 evaluators 5 objects (shoe, face, bicycle, butterfly, rabbit)
User studiesGood drawersWith ShadowDrawWithout ShadowDraw
User studiesBad drawersWith ShadowDrawWithout ShadowDraw
User studiesAverage drawersWith ShadowDraw
Without ShadowDrawUser studiesAverage drawers
With ShadowDrawWithout ShadowDrawUser studies
Significant improvement for Average groupUser studies
Improvement for most object categoriesRabbit is control variableUser studies
After training all users improved:Poor skillGood skillSubjects personal style is maintained!Future workMatching against drawings and not photographsTemporal strokesColor, shading, etc.Drawing priors
Not future work:Changing peoples strokesLive demo
