The accuracy and reliability of perceived depth from linear perspective ... · The accuracy and...

The accuracy and reliability of perceived depth fromlinear perspective as a function of image size

Department of Psychology University of PennsylvaniaPhiladelphia PA USAJeffrey A Saunders

Department of Psychology University of PennsylvaniaPhiladelphia PA USABenjamin T Backus

We investigated the ability to use linear perspective to perceive depth from monocular images Specifically we focused onthe information provided by convergence of parallel lines in an image due to perspective projection Our stimuli weretrapezoid-shaped projected contours which appear as rectangles slanted in depth If converging edges of a contour areassumed to be parallel edges of a 3D object then it is possible in principle to recover its 3D orientation and relativedimensions This 3D interpretation depends on projected size hence if an image contour were scaled accurate use ofperspective predicts changes in perceived slant and shape We tested this prediction and measured the accuracy andprecision with which observers can judge depth from perspective alone Observers viewed monocular images of slantedrectangles and judged whether the rectangles appeared longer versus wider than a square The projected contours hadvarying widths (7 14 or 21 deg) and side angles (7 or 25 deg) and heights were varied by a staircase procedure tocompute a point of subjective equality and 75 threshold for each condition Observers were able to reliably judge aspectratios from the monocular images Weber fractions were 6ndash9 for the largest rectangles increasing to as high as 17 forsmall rectangles with high simulated slant Overall the contours judged to be squares were taller than the projections ofactual squares consistent with perceptual underestimation of depth Judgments were modulated by image size in thedirection expected from perspective geometry but the effect of size was only about 20ndash30 of what was predicted Wesimulated the performance of a Bayesian ideal observer that integrated perspective information with an a priori bias towardcompression of depth and which was able to qualitatively model the pattern of results

Keywords linear perspective depth perception slant perception picture perception Bayesian model vision

Introduction

We can perceive 3D structure from photographs andlinear perspective pictures in an effortless and stablemanner despite the absence of depth cues like binoculardisparities or motion parallax The effectiveness ofpictures is only possible because they reproduce regular-ities present in real-world views of a structured environ-ment A static monocular image typically has manyregularities that can be used to infer 3D structure calledpictorial depth cues (for taxonomies see Cutting andVishton 1995 or Kubovy 1986) For example in Figure 1depth can be inferred from the gradient of size for thesquare tiles or from the convergence (in the image) oflines that recede in the world These cues are typicallycorrelated but each is conditional on different assump-tions that squares lying on a surface have constant sizein the world or that lines on the surface are parallel inthe world In this article we focus on the informationprovided by the latter cue which we will refer to as per-spective convergencePsychophysical studies of 3D perception from perspec-

tive convergence date back more than 50 years Earlystudies demonstrated that observers make systematic slant

judgments from minimal stimuli that contain perspec-tive convergence (Clark Smith amp Rabe 1955 1956Freeman 1966a 1966b Rosinski Mulholland Degelmanamp Farber 1980 Smith 1967 Stavrianos 1945) and thatconvergence can dominate other slant information in cueconflict situations (Attneave amp Olson 1966 Banks ampBackus 1998 Braunstein amp Payne 1969 Gillam 1968Smith 1967) Recent slant-from-texture experiments byAndersen Braunstein and Saidpour (1998) and ToddThaler and Dijkstra (2005) included conditions in whichconvergence was the only available cue and found thatthese stimuli were effective for supporting slant judg-ments Modulations in perspective convergence have alsobeen found to contribute to the perceived 3D shape ofcurved surfaces (Li amp Zaidi 2000)Although it is clear that linear perspective can contrib-

ute to 3D perception significant clarification is stillneeded as to how the visual system uses this informationNo attempts have been made to measure the reliability ofperceived depth from solely perspective convergenceinformation except in the special case of surfaces thatare close to frontal (Freeman 1966a Gillam 1968) Anumber of studies have measured the ability to discrim-inate slant-from-texture but for these studies perspectiveconvergence was either absent (Knill 1998a 1998b

Journal of Vision (2006) 6 933ndash954 httpjournalofvisionorg697 933

doi 10 1167 6 9 7 Received June 20 2005 published August 17 2006 ISSN 1534-7362 ARVO

Knill amp Saunders 2003) or confounded with other textureinformation (Hillis Watt Landy amp Banks 2004 RosasWichmann amp Wagemans 2004 Saunders amp Backus2006) There have been studies that measure the accuracyof perceived slant when only convergence information isavailable (Andersen et al 1998 Rosinski et al 1980Smith 1967 Todd et al 2005) However these data allowonly limited interferences about whether perspective con-vergence is used in a geometrically consistent mannerThe goal of the experiments reported here was to attain a

more quantitative psychophysical measure of the percep-tion of depth from perspective convergence Specificallywe measured the accuracy and reliability of 3D judgmentswhen the only information indicating depth was perspec-tive convergence and tested whether perceived depthchanged in a geometrically consistent way in responseto size scaling of images We used a match-to-known-standard paradigm that indirectly probed perceived depthSubjects were presented with images of slanted rectanglesand judged whether the perceived 3D shape was taller orwider than a square As we illustrate in the next sectionthis task can be performed using monocular informationand the correct response depends on image sizeA similar aspect ratio judgment task has been used for

2D rectangular shapes viewed frontally and observersexhibit discrimination thresholds as low as 1 (ReganHajdur amp Hong 1996 Regan amp Hamstra 1992 Zankeramp Quenzer 1999) Like the 2D task the 3D task dependson the ability to measure aspect ratio and to make adecision However the 3D task also forces the observer tointerpret perspective convergence as slant Thresholds arein general lower when observers are instructed to make2D judgments than when they are instructed to make 3Djudgments indicating that performance in the 3D task islimited by the observerrsquos ability to recover slant fromperspective convergence (see Experiment 1)

Size dependence of perspective information

Consider the geometry of lines projected onto a planarimage If a pair of lines is parallel in the world they

project to lines that converge to a vanishing point in theimage If multiple sets of parallel lines are present withdifferent orientations but lying on the same planar surfacethen their multiple vanishing points uniquely define ahorizon and thereby specify 3D orientation of the surfacethat is its slant and tilt (Stevens 1983) relative to a line ofsight that intersects the plane Once the slant and tilt areknown it would then be possible to reconstruct the shapeof the object up to a scale factor for distance (eg by backprojecting onto the slanted plane)It is unnecessary to compute the horizon line explicitly

to use the convergence cue For example it would sufficeto learn a direct mapping from differences in projectedline orientations to possible parallel interpretations in 3DFigure 2 illustrates the relationship between perspectiveconvergence and 3D orientation for an image of aslanted rectangle In this special case where one pair ofedges remains parallel in the image (Bconverging[ to apoint at infinite distance in the horizontal direction) thereis a simple relationship between slant and perspectiveconvergence

tanethaTHORN frac14 tan w=2eth THORN tanethsTHORN eth1THORN

where a is the angle of the converging side edges relativeto the tilt direction (ie tan(a) is the slope relative tovertical) s is the egocentric slant of the surface (as mea-sured at the origin) and w is the angular width measuredthrough the center This formulation follows thatof Freeman (1966b equivalent formulations includeBraunstein amp Payne 1969 and Flock 1962) A similarrelation holds for more generic poses but would involveboth the slant and tilt of the surface We describeperspective convergence in terms of projection onto animage plane because it is convenient the same geometric

Figure 1 Example of an image in which depth can be perceivedfrom pictorial cues The distorted checkerboard pattern is seen asa slanted rectangular surface covered with uniform texturePerceived slant in depth of the surface could be based ondifferent assumptions about the texture such as that the squaresare uniform sized in the world or that the converging lines in depthare parallel edges in the world

Figure 2 Illustration of the relationship between perspectiveconvergence and surface slant for the projected image of aslanted rectangular surface aligned with its direction of tilt Wedefine the angular width w and slant s to be relative to a referencepoint at the center of the 3D object (left panels) and assume aprojection plane with a distance of 1 The sides of the projectedcontour converge to a point on the horizon (right panel) which is adistance of tan(90 deg j s) or 1tan(s) away from the centralreference point Note that the reference point can be identified inthe image as the intersection between the diagonals of theprojected contour The slope of the sides relative to verticaltan(a) is equal to tan(w2) tan(s)

Journal of Vision (2006) 6 933ndash954 Saunders amp Backus 934

relationship could also be described in spherical coordi-natesAn interesting aspect of the geometry is that the visual

angle subtended by an object factors into its 3Dinterpretation (ie the parameter w in Equation 1)Accurate computation of slant in depth from perspectivewould therefore require the visual system to know theabsolute angular size of a projected image One way tointuitively understand the size dependence is to think ofperspective convergence as identifying the location of thehorizon within a projected image Surface slant is thecomplement of the visual angle between a reference pointand the horizon To obtain this visual angle from aprojected contour one must also know the visual anglesubtended by the contour itself which we will refer to asits projected size As illustrated in Figure 3 similarshapes when scaled to different sizes correspond todifferent 3D interpretations Thus if the visual systemdoes use perspective convergence in a geometricallycorrect way one would expect that rescaling a perspectiveimage would change perceived 3D structure in predictableways As projected size increases the perceived aspectratio of the rectangle (lengthndashwidth) should decrease and

it should appear less slanted The experiments presentedhere test this prediction as a means to identify thecontribution of perspective convergence

Previous work on scaled images

In the context of picture perception other researchershave pointed out that the 3D interpretation of perspectiveinformation is changed when an image is scaled (egFarber amp Rosinski 1978 Nichols amp Kennedy 1993Sedgwick 1980 1991) This is sometimes described asviewing an image from the wrong distance rather thanviewing a scaled image although these are geometricallyequivalent A perspective picture or photograph is onlygeometrically accurate when viewed from a particularvantage point corresponding to the optical center ofprojection of the image If an observer moves further froma picture its projected image subtends a smaller visualangle but remains otherwise similar hence its 3Dinterpretation has greater depth relative to width andheight (as in Figure 3) Thus the problem of interpreting aperspective picture viewed from the wrong distance isequivalent to the problem of interpreting a perspectivepicture when the size of an image has been rescaled as inour experimentsPrevious studies have tested whether perceived depth in

photographs is affected by rescaling an image or bychanging viewing distance (Bengston Stergios Ward ampJester 1980 Lumsden 1983 Smith 1958a 1958b Smithamp Gruber 1958) Most of these studies compared scaledand unscaled photographs with matching perspectiveinformation Bengston et al (1980) found close agreementbetween judgments for scaled and unscaled photos thatwould be expected to appear equivalent based onperspective geometry Other experiments found morelimited correspondence (Lumsden 1983 Smith 1958a1958b) Regardless of the degree of correspondence onecannot infer from these results whether perceived depth inscaled images changes accurately because any biases inthe interpretation of perspective information would haveaffected the perception of both scaled and unscaledimages Smith and Gruber (1958) compared depth judg-ments based on either photographs or actual views of acorridor To the extent that perspective informationcontributes to perceived depth of the real scene how-ever this paradigm has the same limitation as the otherstudiesTo our knowledge Smith (1967) is the only previous

study that directly tested whether perceived depth fromperspective is accurately modulated by projected sizeSmith compared slant judgments for stimuli with similarshapes but varied projected sizes He found that slantestimates changed depending on size in the expecteddirection but by a much smaller magnitude than predictedby perspective geometry in agreement with our resultshere

Figure 3 Illustration of how the 3D interpretation of perspectiveinformation depends on projected size The upper left panelshows two trapezoid-shaped projected contours that have differ-ent angular sizes but identical shapes If these contours areprojections of rectangles with parallel sides their slants can bedetermined (see Figure 2) which in turn specifies the dimensionsof the rectangular object For the large size the 3D interpretationis a wide rectangle at intermediate slant (upper right) whereas forthe small size the 3D interpretation is a long rectangle with highslant (middle right) The lower graph plots the slant specified byperspective for this example contour shape across a range ofprojected sizes The small black rectangles illustrate objectdimensions for some sample points on the graph


Size-invariant monocular cues

Perspective convergence differs from other pictorialcues in its dependence on size One size-invariant cue isforeshortening or compression of a projected contour Ifthe overall shape of a planar object is assumed to beisotropic (Garding 1993 Witkin 1981) then the aspectratio of its projected contour provides a cue to its slantFigure 4a illustrates this cue for the case of trapezoidalprojected contoursAnother size-invariant cue is skew symmetry (Kanade

1981 Perkins 1976 Saunders amp Knill 2001) If a pair ofaxes (or edges) that intersect at a skewed angle in animage is assumed to intersect at a right angle in the worldthen the skewed axes provide a cue to 3D orientation Asillustrated in Figure 4b the skew of a set of axes canprovide a slant cue that is independent of projected sizeIn scaled images these size-invariant image cues willconflict with perspective convergence and are expected toreduce the effect of image size on the perceived slants ofsurfaces in the depicted scene

There is evidence that observers are sensitive to differ-ences between the 3D structure specified by perspectiveand by size-invariant cues Nichols and Kennedy (1993)presented subjects with line drawings of cubes viewedfrom their corners some of which were generated byrescaling a perspective image A given drawing wasjudged to be a good cube over a range of sizes but thesize that was geometrically accurate for a cube was ratedbest Yang and Kubovy (1999) performed a similarexperiment presenting cube-like line drawings withvarying relationships between projected size and amountof perspective and asking subjects to identify the bestcube They also observed a weak preference for geometri-cally accurate images consistent with the work of Nicholsand Kennedy In both studies a wide range of stimuliwere rated to be good cubes which is consistent withreliance on size-invariant cuesWe wanted to isolate perspective convergence from

other pictorial cues We therefore used trapezoidalprojected images like those of Figure 3 where the sideedges are symmetric and where the top and bottom edgesare parallel in the image This image is a special caseAssuming right-angle intersections does not provide thenormal size-invariant slant cue One way to understandthis is that when viewing a real rectangle the vanishingpoints for opposite edge pairs have visual directions thatare separated by 90 deg This angle normally becomessmaller or larger when the image of the rectangle isscaled For a trapezoidal image however this angleremains 90 deg (because one Bvanishing point[ is atinfinity) and therefore provides no information aboutwhether the image has been scaledAlthough the use of trapezoid-shaped stimuli eliminates

the skew symmetry cue isotropy is still a potentiallyuseful constraint Observers could assume that the imagesare the projections of square objects in 3D which wouldgenerally lead to conflict with perspective convergence InExperiment 1 the task was to judge whether the objectwas taller versus wider than a square thus heavy relianceon an assumption of isotropy would cause judgments to beunreliable but would not by itself lead to bias InExperiment 2 subjects matched perceived length in depthto the height of a frontally oriented bar and projectedaspect ratio was an independent variable which allowedus to assess its effect

Experiment 1

In this experiment stimuli were monocular images likethose shown in Figure 5 and subjects judged whether theobjects appeared to be taller versus wider than a squarewhen considered as objects in 3D There is a correctanswer in this task based on the perspective convergencecue We tested two different side angles 25 and 7125

Figure 4 Two monocular depth cues that provide information thatdoes not depend on absolute projected size foreshortening (a)and skew symmetry (b) The top center figure shows the projectedoutline contours of two slanted squares with the same slant butdifferent sizes These trapezoids have different amounts ofconvergence but their mean aspect ratios are the same (upperright) equal to the cosine of surface slant If objects are assumedto be isotropic slant could be recovered from projected aspectratio even if size were not known A size-invariant measure offoreshortening can alternatively be formulated in terms of theslope of the line connecting the geometric center of the trapezoidto one of its corners (Braunstein amp Payne 1969) The bottommiddle figure shows the projected contours of two squares withdifferent sizes that have been rotated by 45 deg within their planeSize affects the amount of convergence but not the angle formedbetween symmetry axes at the geometric center (bottom right) Ifobjects are assumed to be mirror symmetric rotated in deptharound a horizontal axis slant could be recovered from the skewangle without knowledge of size


deg which we will refer to as the high-slant and low-slantconditions respectively (Figure 5 top and bottom rows)Both shapes were presented with a range of projectedsizes If subjects use perspective convergence correctly ata given projected size one would expect a large differenceacross size conditions in the projected shape that is judgedto be square with the largest projected size having to bethe tallest to appear square Figure 6 shows the contourshapes that accurately correspond to the projections of asquare for each of the six size and slant conditions Inaddition to overall shape we also varied the presence orabsence of an internal grid texture (Figure 5 right vsleft) The textured stimuli contained an additional cue todepth the gradient of compression of vertical spacingThis cue is effective in its own right (Andersen et al1998) As with perspective convergence the 3D interpre-tation of the compression cue depends on projected sizehence varying size might have a larger effect for texturedthan untextured rectangles

MethodsApparatus and display

Stimuli were rear projected from an InFocus LP350projector with 1024768 resolution onto a 166125 cmregion of a large screen positioned 2 m from the observerThe rectangular projected region subtended 45 deg hori-zontally and 35 deg vertically and its boundaries weredimly visible Subjects wore a patch over their left eyethroughout the experiment Subjects were seated on a stool

and were instructed to remain stationary during judgmentsbut were not otherwise restricted (no chin rest or bite barwas used) Images were grayscale and antialiased renderedusing OpenGL on a workstation with Nvidia Quadro FX1000 graphics boardStimuli simulated perspective views of slanted rectan-

gular surfaces filled either with uniform gray (untextured)or with a 9 9 checkerboard pattern (textured) on ablack background Slant was always around a horizontalaxis (ie the tilt direction was vertical) The left and rightsides of the projected contours had 2D orientations thatwere either T25 or T7125 deg relative to vertical in thehigh- and low-slant conditions respectively Both shapeswere presented at three different sizes with widths of25 cm (71 deg) 50 cm (14 deg) or 75 cm (206 deg) asmeasured horizontally through the geometric center of theprojected contours By Bcenter[ we mean the intersectionpoint of the trapezoid diagonals which is also theprojected location of the center of the original 3Drectangle The screen location of the center of thetrapezoid was the same for all stimuli The slants specifiedby perspective convergence were as follows 82 and63 deg (small) 75 and 45 deg (medium) or 68 and 34 deg(large) respectively

Procedure

Subjects made forced-choice judgments whether thesimulated 3D object was longer versus wider than a

Figure 5 The four classes of perspective images used inExperiment 1 Images were consistent with a perspective viewof planar rectangular objects slanted in depth Objects wereeither of uniform color (untextured conditions left) or covered witha checkerboard pattern (textured conditions right) The conver-gence angle of the sides of the trapezoids was either 25 deg(high-slant conditions top) or 7125 deg (low-slant conditionsbottom) Each base image was presented at different sizes withthe width of the centerline subtending 7 14 or 21 deg (not shownin this figure) Here the trapezoids are shown as dark on a whitebackground but in the actual displays the contrast was reversed

Figure 6 Predicted results if performance were veridical Eachtrapezoid is an accurate projection of a square for various widthsand slants The convergence angles are matched across thethree high-slant conditions (top row) and the three low-slantconditions (bottom row) but the slants that these correspond todiffer depending on size In the high-slant conditions the slantsspecified by perspective convergence under an assumption ofparallelism are 82 75 and 68 deg (from left to right) In the low-slant conditions the slants specified by perspective convergenceare 63 45 and 34 deg The mean aspect ratios of the projectedtrapezoids are equal to the cosine of the slant


square They were instructed to base their judgments onthe perceived shape of the slanted 3D object not on thescreen projection In the case of textured stimuli subjectswere told to base their responses on the rectangle as awhole not component rectangles Trials were self-pacedand subjects received no feedbackThe aspect ratios of projected contours were varied

across trials using a new adaptive method (see Appendix A)The set of judgments from each condition was fit to a cu-mulative Gaussian psychometric function using maximum-likelihood criteria The mean of the best fitting function wastaken as the point of subjective equality (PSE) which forthis task was the aspect ratio of the image Discriminationability was measured as the difference between the PSE andthe 75 points divided by the PSE This is a Weber frac-tion that corresponds to the change in aspect ratio requiredfor 75 discriminationWe define the projected aspect ratio of a trapezoid to be

(wbottom + wtop)(2h) where wbottom and wtop are the widthsof its bottom and top edges and h is its projected heightThis ratio is related to the aspect ratio of the correspond-ing 3D rectangle by a factor of the cosine of slant(Braunstein amp Payne 1969) Thus the same Weberfraction describes discriminablility for the 3D rectanglesand for their projections Note that the Bmean width[ usedto compute aspect ratio for the Weber fraction is not sameas width measured through the center of the trapezoidTextured and untextured stimuli were presented in

separate blocks whereas size and slant were randomizedwithin blocks The experiment consisted of two 1-hrsessions each with one block of textured stimuli and oneblock of untextured stimuli with order randomized acrosssubjects and sessions Each block contained 300 trialsyielding a total of 100 trials for each of the 12 conditionsIn an additional control condition separate subjects

judged the aspect ratios of 2D projected contours Thestimuli were either trapezoidal or rectangular contoursand subjects reported whether the contourrsquos height wasgreater or less than half the central width of the contourNo feedback was given The trapezoidal contours used inthe control experiment were the same as in the untexturedhigh-slant condition of the main experiment The rectan-gular contours were presented at the same three sizes asthe trapezoidal contours Contour height was varied acrosstrials using the same adaptive procedure and the PSEsand 75 thresholds for the 2D task were computed asbefore Subjects performed two blocks of 300 trials eachand both contour shape and size were randomized withinblocks

Subjects

Twelve paid subjects participated in the main experi-ment All had normal or corrected-to-normal vision andwere naive to the purposes of the experiment Fiveadditional naive subjects along with the first authorparticipated in the control experiment using the 2D task

Subjects gave informed consent in accordance with aprotocol approved by the IRB panel at the University ofPennsylvania

Results

Figure 7 shows mean contour aspect ratios derived fromsubjectsrsquo judgments averaged across subjects as a functionof projected size The two graphs plot results for the high-and low-slant conditions respectively and the two solidlines on each graph correspond to the textured and untex-tured conditions In all cases aspect ratios were larger forlarger projected sizes which is in the predicted directionANOVA for high slant F(255) = 1933 p G 001 lowslant F(255) = 3953 p G 001 There was no evidencethat the presence or absence of texture made a difference inthe effect of size high slant F(255) = 0628 p = 47 (ns)low slant F(255) = 057 p = 57 (ns) Eleven of 12 sub-jects showed the size effect There were also small but sig-nificant main effects of texture for both high- and low-slantconditions high slant F(155) = 6119 p = 016 low slantF(155) = 1096 p = 002Although the effect of projected size was reliable it was

much smaller in magnitude than would be expected basedon perspective geometry The dashed lines on each plotshow the ratios for veridical use of this cue For the high-slant conditions (Figure 7 left) the observed height-to-width ratios for the largest and smallest trapezoids differedby 19 for textured stimuli and 26 for untexturedstimuli whereas the predicted difference is 180 For thelow-slant conditions (Figure 7 right) PSEs were closer toveridical but the effect of size was still compressed (12and 8 for the textured and untextured conditions vs apredicted difference of 86) In all cases the projectionsof apparently square rectangles were taller than theprojections of actual squaresFigure 8 shows the mean 75 discrimination thresholds

for the same conditions as in Figure 7 Discriminationthresholds were relatively low indicating that althoughjudgments were biased they were reliable For the high-slant conditions textured stimuli produced significantlylower thresholds F(155) = 87 p = 005 whereas for thelow-slant conditions there was no reliable difference fortexture F(155) = 15 p = 23 (ns) Threshold decreasedas projected size increased in the high-slant conditionsF(255) = 126 p G 001 this trend was not significant inthe low-slant conditions F(255) = 14 p = 25 (ns) Therewas no interaction between size and the presence of tex-ture for either slant condition high slant F(255) = 036p = 70 (ns) low slant F(255) = 014 p = 87 (ns)

DiscussionConsistency of depth judgments

Our results demonstrate that subjects can make con-sistent judgments about 3D lengths from monocular


images containing perspective convergence At the largestprojected sizes the average discrimination thresholds foruntextured stimuli were only 6 for the low-slantcondition and 9 for the high-slant conditionThe presence of internal texture resulted in lower

discrimination thresholds This is not surprising as thereare a number of ways that the texture could have served toimprove performance First texture provides additionalindependent sources of information about depth includingthe following the compression of individual textureelements the gradient of texture compression and thegradients of texture size and spacing The contributions ofany of these texture cues which have all been observed tocontribute in at least some conditions (eg Knill 1998bRosenholtz amp Malik 1997) could have improved theprecision of depth estimates Second the textured stimulicontained multiple converging lines (the columns of thetexture) which could have improved image measurementsof perspective convergence and thereby improved theprecision of depth estimatesIn the case of the textured stimuli improvement in

discrimination performance at large projected sizes couldpotentially be explained by better estimates of the size andshapes of texture elements rather than by better use ofinformation from perspective convergence Howeverthresholds for the untextured stimuli improved withprojected size by as much as or more than those for the

textured stimuli hence any effect of size on the reliabilityof texture information must have been comparativelysmall

Perceptual biases

Although judgments were relatively reliable theyshowed large biases relative to veridical especially forthe high-slant conditions In the worst caseVhigh slantand small widthVthe contours that appeared to be viewsof square objects were on average over three times theheight of an actual projection of a squareSome component of the overall bias could be related to

a known bias in perception of 2D dimensions thehorizontalvertical illusion For example subjects mighthave been comparing perceived dimensions of the slanted3D object to some biased internal standard of a squareHowever the magnitude of the horizontalvertical bias hasbeen measured to be around 4 in the case of 2D shapes(Henriques Flanders amp Soechting 2005) thus it couldnot fully account for the much larger biases we observedThe presence of the checkerboard texture had only a

small effect on biases despite the fact that the texturedstimuli in principle provide more information and aresubjectively more compelling This suggests that texturecues added little depth information beyond perspectiveconvergence for our displays This is in partial agreement

Figure 7 Mean PSEs from the data of Experiment 1 The graphs plot the mean projected aspect ratios of projected contours that would beperceived as images of square objects as a function of projected size The left and right graphs show results for the high- and low-slantconditions respectively closed and open symbols correspond to textured and untextured conditions respectively The dashed lines plotveridical responses based on accurate use of perspective convergence


with previous studies Todd et al (2005) measuredjudgments of the dihedral angle formed by two texturedsurfaces and found a reliable but small difference betweenruled textures which isolated perspective convergenceinformation and plaid textures Andersen et al (1998)also report modest differences in judgments of dihedralangles for ruled surfaces and grids although they found acomparatively larger advantage for grid textures whensubjects judged the slant of a single surface

Effect of projected size

The contours that appeared to be square objects inaddition to being biased overall also did not change withprojected size as much as predicted by perspective geom-etry This is in agreement with the results of Smith (1967)Smith measured direct estimates of slant for trapezoidalcontour stimuli similar to our untextured stimuli and in-cluded conditions in which contour shape was matchedacross different sizes Smith also found that judgments weredependent on size but that this effect was much smaller thanpredictedAlthough its effect was modest projected size did

reliably modulate subjectsrsquo judgments in our experimentand the magnitude of the effect exceeded discrimination

thresholds Thus the smaller-than-expected size modula-tion cannot be attributed to simply poor sensory measure-ment of projected sizeOne depth cue that we were not able to isolate even in

the untextured condition was the overall foreshortening ofthe projected contour However any bias caused by theforeshortening cue would be in the direction of anisotropic interpretation that is the 3D interpretation ofthe object would always be closer to being square For thetask we used this might make discrimination moredifficult but it could not directly account for size-dependent biases in what projected shapes appear to besquares

Control for 2D strategy

One potential concern is that subjects might not havebased their responses on a 3D percept Rather they mighthave made 2D shape judgments comparing the projectedaspect ratio of a contour with some imagined 2D standardIf subjects were using 2D projected shapes directly ratherthan the perceived shapes of slanted 3D objects then ourexperiments would have little relevance to the question ofhow the visual system uses perspective convergence toperceive 3D slant and shape

Figure 8 Mean discrimination thresholds from the data of Experiment 1 Thresholds are expressed as Weber fractions computed bytaking the difference between aspect ratios at the PSE and 75 points and dividing by the PSE aspect ratios The left and right graphsshow results for the high- and low-slant conditions respectively closed and open symbols correspond to textured and untexturedconditions respectively The small figures beside the y-axis labels are graphical representations of the range of aspect ratios cor-responding to a given threshold value with the shaded regions depicting T1 threshold units around the mean


There are several reasons to believe that subjects used3D shape First for a cognitive 2D strategy it is notobvious why judgments would be modulated by size at allOne would have to assume that the internal 2D standardsused for comparison had varying aspect ratios dependingon projected size Second if subjects were judging 2Dshape thresholds would be expected to show the samegeneral pattern as previously observed for 2D aspect ratiojudgments However aspect ratio discrimination for 2Dshapes is unaffected by large variations in overall scale(Regan amp Hamstra 1992) In our experiments thresholdsdecreased as image size increasedOur additional control experiment also addresses this

issue Instead of judging the perceived shape of a slanted3D object subjects were instructed to judge the aspectratio of the 2D trapezoidal contour whether thecontourrsquos height was greater or less than half of itscentral width For comparison we also had subjectsperform the same task for rectangular 2D contours withthe same sizes The results are shown in Figure 9Thresholds were higher overall for the trapezoids relativeto the rectangles but projected size had no effectComparing these thresholds to those obtained using a 3Dtask (Figure 8 high-slant conditions) it is clear that the2D shape task produces qualitatively different results ThePSEs from our control experiment also did not depend onprojected sizeGiven that thresholds and PSEs both varied with size in

our 3D task but not in our 2D task and that performancewas worse when subjects were instructed to use the 3D

strategy we conclude that subjectsrsquo judgments in the 3Dtask were in fact based on 3D percepts In that case ourresults indicate that perceived depth from perspectiveconvergence is both more reliable and more accurate forlarge images As we will describe later an ideal observerexhibits similar performance

Perceptual compression of depth

The direction of the observed biases is consistent withunderestimation of the perceived slant of the rectangleswhich has been observed in other studies that isolatedperspective information (Andersen et al 1998 Smith1967 Todd et al 2005) Consider for example thecontour shown in the upper left panel of Figure 6 Theslant implied by perspective convergence is very high inthis case (82 deg) Suppose that observers tended to seethe figure as being less slanted The resulting perceived3D object would be compressed in length relative toveridical To be perceived as a 3D square it would thenhave to be stretched vertically as in our resultsThis interpretation agrees with our phenomenal impres-

sions of the stimuli They appear much less slanted thanthey should based on perspective convergence Anotheraspect of the phenomenal appearance that is also con-sistent with underestimation of slant is that for thetextured surfaces individual texture elements did notappear to have uniform aspect ratios along the 3D surfaceRather the upper elements appeared to be more com-pressed in length If perceived slant were underestimatedthen the compression gradient in a projected image wouldbe greater than would be expected for a homogeneoussurface with the (biased) perceived slant which couldaccount for the inhomogeneous appearance of the texturedstimuliThere are a number of factors that could have reduced

perceived slant in our stimuli Real slanted surfacesproduce an accommodative gradient which is known tocontribute to slant perception (Watt Akeley Ernst ampBanks 2005) The absence of an accommodative gradientin our displays indicated a frontal surface Another factoris the frame provided by the projection screen Althoughthe images were subjectively compelling as 3D objectssubjects were aware that they were looking at projectedimages and the boundaries of the screen were visibleThis could also have acted as conflicting information orBcross talk[ specifying a frontal surface as suggested bySedgwick (1991) Eliminating cues that specify a flatpictorial surface has been observed to enhance perceptionof depth (Koenderink van Doorn amp Kappers 1994) Thevisual system may also have an a priori bias toward seeinguniform depth when 3D cues are weak or absent (Gogel1965) All of these factors would have the effect ofcompressing perceived depth toward the frontal planeIf perceived slant were some blend between the slant

specified by perspective and the slant indicated by con-flicting cues and any prior assumptions then one could

Figure 9 Mean 75 discrimination thresholds for 2D aspect ratiojudgment task


also explain the smaller-than-expected effect of projectedsize That is because to the extent that the perspective cueis only partially Bweighted[ changes in the perspectiveinformation would have less influence on the final perceptWe will discuss this possibility in more detail in a latersectionIt is also possible that the visual system has simply

learned an incorrect mapping between rectangular 3Dobjects and their perspective projections Rectangular orsquare planar objects are common in normal environ-ments thus the visual system would have ample exposureto this class of objects However there may normally beno cost to underestimating the slant or length in depth ofrectangles Informal observation suggests that realsquares (eg on sidewalks) may also be misperceivedas being relatively wider than a square consistent withcompression of perceived depth even under full-cueconditions

Experiment 2

Figure 10a illustrates what PSEs from Experiment 1represent the shapes of projected contours that areperceived to be square 3D objects for various projected

sizes One could also ask How does perceived 3D objectshape vary across different sized contours with the sameshape (Figure 10b) From the data from Experiment 1 wecan infer that a contour that appears to be a square objectat an intermediate size would appear as an elongatedrectangle when presented at smaller sizes and as ashortened object at larger sizes However the extent towhich the perceived 3D objects appear elongated orshortened cannot necessarily be determinedIn particular the aspect ratio of a projected contour

might interact with perspective information in determin-ing perceived 3D slant and shape For example ifforeshortening contributed as a slant cue varying pro-jected aspect ratio would change both perceived slant andperceived 3D shape This would tend toward making the3D objects appear closer to being square and therebyreduce the effect of projected size on perceived depth It isalso possible that projected aspect ratio interacts in theopposite direction for example tall projected contoursmight be perceived as more slanted than shorter contourswith the same perspective convergenceThese possibilities are tested in Experiment 2 Figure 11

depicts the stimuli and task Subjects viewed images withtrapezoid-shaped figures which appeared as 3D rectan-gles and adjusted the length of vertical (frontal) compar-ison rectangles until its length matched the apparentlength of the 3D rectangle This task allows one tocompare perceived shape for different-sized but identi-cally shaped contours

Figure 10 (a) Interpretation of the PSEs from Experiment 1 Foreach projected size the contour that is perceived as a square 3Dobject has a different shape The contours shown are consistentwith the mean PSEs in the high-slant condition (b) A relatedquestion for a given contour shape what is the perceived 3Dobject shape across different projected sizes The degree towhich the perceived 3D object is stretched in depth at small sizes(left) or compressed at large sizes (right) cannot be determinedfrom Experiment 1 alone

Figure 11 The stimuli and length-matching task used inExperiment 2 Subjects adjusted a pair of vertical bars to appearthe same length as the slanted 3D rectangle Surfaces weretextured with random planks (see text) Convergence angles andprojected sizes were the same as in Experiment 1 In the high-slant conditions the trapezoids had aspect ratios of 04 05 or06 (top left to right) In the low-slant conditions the aspect ratioswere 07 08 or 09 (bottom left to right) From the results ofExperiment 1 the short trapezoids (left) would be expected toappear as wide rectangles and the taller trapezoids (right) as longrectangles with the intermediate trapezoids appearing close tosquare


The slant of a rectangular object is determined by thewidth and side angles of its projected contour Thecontourrsquos aspect ratio also factors into the 3D shape ofthe rectangle but not into its slant In the extreme ifperceived slant were determined solely by perspectiveconvergence then trapezoidal projected contours with thesame width and side angles would be perceived to havethe same slant regardless of the aspect ratio of theprojected contour This would in turn imply a relationshipbetween the aspect ratio of a projected contour and its 3Dinterpretation Figure 12 illustrates this mapping for arange of possible slants


The display apparatus was the same as in Experiment 1Stimuli consisted of a textured trapezoid and an adjoiningpair of (identical) variable-height comparison rectanglesFigure 11 shows the six trapezoidal shapes that were usedas projected contours Two convergence angles weretested (same as Experiment 1) 25 and 7125 deg Foreach convergence angle three projected aspect ratios weretested 03 04 and 05 for the high-slant condition (toprow left to right) and 07 08 and 09 for the low-slantcondition (bottom row left to right) These aspect ratioswere chosen because they spanned the range of shapesthat were judged to be the projections of squares based onthe mean data from Experiment 1 That is we would

expect on average the leftmost stimuli to be seen aswider than longer and the rightmost stimuli as longer thanwider Each projected shape was presented at threedifferent sizes with horizontal widths of 71 14 or206 deg (same as Experiment 1)In Experiment 1 judgments were more reliable (lower

thresholds) for the checkerboard textured surface than forthe blank surface although PSEs were similar One pos-sibility is that the texture helped stabilize the interpre-tation of the 3D object To achieve this potential benefitwithout introducing a strong cue based on the gradient ofvertical spacing we used a random plank surface texture(illustrated in Figure 11) similar to that of Andersen et al(1998) This texture tiled the surface with 10 columns ofrectangles with uniform width but variable length Eachcolumn was subdivided into separate tiles at 10 locationschosen randomly from a uniform distribution A differentrandom pattern of planks and brightness values was gen-erated for each trial The comparison rectangles were sim-ulated to have the same width as the rectangle with heightbeing controlled by the subject

Procedure

Subjectsrsquo task was to adjust the height of the compar-ison rectangles until they appeared to match the length ofthe slanted rectangular object The initial comparisonheight on a trial was chosen randomly to be between 70and 140 of the contourrsquos projected width and subjects

Figure 12 Predicted results for Experiment 2 if judgments were consistent with some constant planar surface across changes in contouraspect ratio When a projected contour is back projected onto a slanted surface (left) the length of the resulting 3D object (middle)depends on both surface slant and contour height The rightmost graph plots the length-to-width ratio of a 3D object (y-axis) as a functionof the aspect ratio of its projection contour (x-axis) for various possible surface slants (curve labels) If perceived slant depends onconvergence but not contour height (or aspect ratio) judgments of 3D object dimensions should lie along one of these lines The twoexample cases shown on the left have been highlighted on the graph


increased or decreased the height in 14 steps using akeyboard Trials were self-paced with no feedback The ex-periment consisted of two blocks of 180 trials in a single1-hr session yielding 20 trials for each of 18 conditions(3 sizes 2 slants 3 aspect ratios) per subject Condi-tions were randomized within blocks

Subjects

Six subjects participated in Experiment 2 One of thesubjects was the first author The others were naive to thepurposes of the experiment and were paid for participat-ing All subjects had normal or corrected-to-normal

vision Subjects gave informed consent in accordance toa protocol approved by the IRB panel of University ofPennsylvania

Results

Height settings were averaged across trials in a conditionfor a given subject then divided by projected width tonormalize for scale The resulting measure represents thelength-to-width ratio of the perceived 3D rectangle Figure 13plots mean length-to-width ratios for individual subjects asa function of projected aspect ratio for each of the three

Figure 13 Results of Experiment 2 The black lines plot mean matching length settings for individual subjects as a function of projectedaspect ratio The six graphs correspond to high- and low-slant conditions (top and bottom) and to the different size conditions (left to right)The gray lines depict the patterns of responses that would be consistent with a constant surface orientation for various possible slants(see Figure 10 for description) Overall the data plots were close to being aligned with predicted curves indicating that subjectsrsquojudgments changed as a function of aspect ratio at a rate that was consistent with their mean bias as would be expected if they perceivedthe objects as having a constant (but biased) surface orientation Note that the one low outlier on each graph is the same subject


sizes (left to right) and two side angles (top and bottom)The labeled gray lines show how length-to-width ratio var-ies for a given slant as the contourrsquos aspect ratio changesAcross conditions with the same size and side angle themean length-to-width settings for a given observer lie alongone these curves which means that their settings were con-sistent with perceiving the same 3D slant regardless of thecontour aspect ratioAs in Experiment 1 the effect of projected size was

significant but smaller than expected based on perspectivegeometry If responses were veridical length-to-widthratios would lie along the curves corresponding to slantsof 82 75 and 68 deg for the high-slant conditions (smallto large widths) and slants of 63 45 and 34 deg for thelow-slant conditions As can be seen in Figure 13 sub-jectsrsquo judgments corresponded to surface slants that hadmuch less size modulation In all subjects the slants im-plied by judgments decreased with projected size but thisdecrease was on average only 38 deg for the high-slantconditions and 64 deg for the low-slant conditions cor-responding to gains of 27 and 34 respectively Oneoutlier subject made responses that were consistent withlower overall perceived slants but similarly exhibited thesmaller-than-expected size modulationFigure 14 plots inferred slant as a function of contour

side angle (high slant vs low slant) and projected sizecombining data across projected aspect ratio conditionsThe filled circles plot the mean inferred slants averagedacross the six subjects The dashed lines depict predictedresults based on accurate use of perspective convergenceRelative to veridical subjectsrsquo perceived slants are biased

toward zero (frontal) and show less change as a function ofprojected size consistent with the results of Experiment 1For comparison we computed inferred perceived slantsbased on the PSEs from Experiment 1 which are alsoplotted in Figure 14 (open circles) The biases are similaracross the two experiments

Discussion

The results of Experiment 2 demonstrate that for ourstimuli small changes in the aspect ratio of a projectedcontour have little effect on perceived slant in depthAcross conditions with different aspect ratios but with thesame perspective convergence and projected size judg-ments were consistent with a constant slantThis finding is consistent with earlier results of

Braunstein and Payne (1969) In their study slantinformation from perspective convergence and fromforeshortening were independently varied by manipulatingthe horizontal and vertical spacing of a texture composedof a grid of dots Braunstein and Payne similarly observedthat convergence was the dominant factor in determiningperceived slant The trapezoidal contour stimuli used bySmith (1967) also varied aspect ratio independently ofside angle Unfortunately the conditions cannot be easilycompared to ours Smith varied the width rather than theheight of projected contours which changes both fore-shortening and perspective informationAlthough our results suggest that perceived slant in depth

was determined primarily by perspective convergence thedata do not allow a strong test of this hypothesis It remainspossible that projected aspect ratio does influence per-ceived slant for our class of stimuli but that its effect is toosmall to be observed across the modest range of aspectratios we testedWhat we can conclude is that any effect of contour aspect

ratio is not sufficient to account for the smaller-than-expected effect of projected size on judgments of length indepth The projected aspect ratio for which the objectappears square in 3D (as was measured in Experiment 1)can be inferred from the data in Experiment 2 by lookingat the point where perceived length-to-width ratio equals 1(ie where y = 1 on the plots in Figure 13) If there was asignificant interaction between perspective convergenceand aspect ratio these points might be similar acrossprojected size conditions even if the perceived length-to-width ratio for a given aspect ratio changed by a largeamount This is clearly not the case

Ideal observer model

In both experiments subjectsrsquo judgments were biasedrelative to veridical We hypothesized that a model that

Figure 14 Replotting of data as inferred perceived slants Opencircles and filled circles show slants inferred from Experiments 1and 2 respectively Dashed lines plot the veridical slantsassuming parallel sides


correctly internalized perspective geometry might still showthe observed pattern of biases if the perspective cue wasnot strong enough by itself to overcome conflicting infor-mation and prior assumptionsAs described earlier a bias toward perceiving depth as

flattened toward the frontal plane would explain whyimage trapezoids would have to be taller relative toveridical to be perceived as 3D squares Such an overallbias might be due to absent or conflicting cues awarenessof the pictorial surface or some prior assumption of con-stant depth Additionally if perspective information wereonly partially weighted relative to conflicting informationthis could also explain why projected size had less effecton subjectsrsquo judgments than predicted by perspectivegeometryThe simplest variant of this cue conflict explanation

would be if perceived slant were some constant weightedaverage of the slant from perspective and the slantspecified by other information Because the latter slant iszero perceived slant would be linearly related toperspective slant by a constant factor This clearly doesnot fit our data the discrepancy between subjectsrsquo judg-ments and veridical performance varies greatly dependingon both contour size and shapeA constant-weight linear model is also overly simplistic

in principle because it ignores how the informationspecified by perspective varies across conditions Firstsensory measures of contour shape and size would havevarying amounts of noise For example it is known thatdiscrimination of line orientations depends on both linelength and overall orientation (Heeley Buchanan-SmithCromwell amp Wright 1997 Regan amp Price 1986 Snippeamp Koenderink 1994) and that discrimination of anglesdepends on the reference angle (Chen amp Levi 1996Heeley amp Buchanan-Smith 1996 Regan Gray ampHamstra 1996) Second noise in image measurementspropagates to different amounts of uncertainty in slantFinally because the slant from perspective depends onprojected size the amount of conflict between theperspective cue and other information effectively varieswith size as wellTo incorporate such factors in our hypothesized explan-

ation we simulated the performance of a nonlinearBayesian ideal observer for the task and stimuli we usedA bias toward perceptual compression of depth wasmodeled as a prior distribution over the set of possiblesurface slants which was integrated with perspectiveinformationThe task of the ideal observer model was to estimate the

3D shape and slant of planar object based on a trapezoid-shaped projected contour The projected contour wasspecified by its projected width (w) aspect ratio (rproj)and side angles (aproj) We assumed that slant was arounda horizontal axis (vertical tilt direction) thus the 3Dobject would be a symmetric trapezoid as well The 3Dobject was also specified by its size length-to-width ratio(robj) and the angle of its sides relative to its midline

(aobj) There is an unavoidable ambiguity with respect tothe overall size and distance of the 3D object hencewithout loss of generality we ignore its size parameterThus in terms of the defined parameters the modelrsquos taskwas to estimate the slant (s) and shape (robj aobj) ofthe 3D object from a projected contour with a givenprojected width (w) and shape (rproj aproj) The results ofExperiment 2 suggest that perceived slant does not dependon projected aspect ratio thus we initially estimated s andaobj based solely on w and aproj The estimate of slant wascombined with rproj to determine robj which was what themodel (and subjects) judgedThe modelrsquos estimate of s and aobj was the combination

that maximized the posterior probability function P(s aobjordfaproj w) By applying Bayesrsquo rule this can be expressed interms of the likelihood function P(aprojordfs aobj w) and thepriors on s and aobj

P s aobjkaprojw

Egrave P aprojks aobjw

P seth THORNP aobj

eth2THORN

To compute the likelihood function P(aprojordfs aobj w) weassumed that the image measures of w and aproj were un-biased but corrupted by noise and then marginalized overthe possible true values Details of the noise model aregiven in Appendix BThe bottom left panel of Figure 15 shows the likelihood

function P(aprojordfs aobj w) computed for an exampleimage trapezoid (top left) There is a range of 3Dinterpretations with high likelihood lying along a curveTwo particular points along the curve are marked forillustration One is the zero slant interpretation in thiscase the 3D object has the same trapezoid shape as theprojected contour The other special case marked in thefigure is where the high likelihood curve intersects the axisaobj = 0 which corresponds to the 3D interpretationassuming parallel sides The other points with highlikelihood are intermediate cases where the 3D shape isa trapezoid with less steeped sides than the 2D projectedcontour and is less slanted than the parallel-sidesinterpretationThe middle and right panels in the bottom part of

Figure 15 show the result of combining the likelihoodfunction P(aprojordfs aobj w) with the different priors for sand aobj The top middle panel shows P(s aobj) assuming auniform prior for s and a Gaussian prior for the shapeparameter aobj centered around zero with standard deviationApersp = 6 deg This prior assigns higher likelihood tointerpretations for which the objectrsquos sides are near parallelThe bottom middle panel shows the result of multiplyingthese priors with P(aprojordfs aobj w) to obtain the posteriorP(s aobjordfaproj w) As might be expected the maximumof this function is very close to the parallel-sides interpre-tation The top right panel shows a different set of priorsThe prior on aobj is the same but the prior on s is weightedtoward zero P(s) = cos(s) This particular prior has beensuggested by Hillis et al (2004) who point out that it


describes the distribution of viewer-relative slants in anenvironment where all 3D surface orientations are equallylikely When this biased slant prior is integrated withperspective information the peak of P(s aobjordfaproj w) isshifted away from the correct parallel interpretation to apoint with lower slant (bottom right)The final step in our model simulations was to convert

slant estimates into simulated performance in the 3Ddimension judgment task performed by subjects Given aprojected contour and its estimated slant the shape of thecorresponding 3D object can be determined by simplyback projecting the contour We assumed no decisionnoise hence judgments were directly determined by thelength-to-width ratio of the back-projected shape On anygiven trial however image measures of the width andshape of a projected contour would be perturbed by noiseThe same noise models used in deriving the likelihoodfunctions served as generative models for trial-to-trialvariability By integrating model judgments over thepossible perturbations we computed expected psychomet-ric functions for a given stimulusFigure 16 plots the simulated performance of the model

with a biased cosine prior on slant for the conditionstested in Experiment 1 The modelrsquos PSEs (left) exhibitboth an overall bias and reduced modulation by projectedsize as in the human data The modelrsquos discriminationthresholds (right) are lower overall than the human resultsbut otherwise show a similar pattern This agreement pro-vides confirmation that our choice of noise parameters wasreasonable The fact that these parameters also result in agood fit to PSE data supports our hypothesis demonstrat-

ing that a general bias toward underestimating depth couldaccount for much of the deviations from veridicality ob-served in our resultsOn the other hand the quantitative agreement is clearly

imperfect and is not easily improved by choice of parameterswithin our simple formulation With a fixed slant prior andnoise parameters determined from psychophysical datathe only remaining freedom is in the choice of theparallelism prior P(aobj) which in our model is specifiedby the single parameter Apersp Setting this parametermuch higher than in our simulations can lead toqualitative discrepancies In addition there is one dis-crepancy between model performance and human datathat cannot be explained within our formulation regard-less of parameters In the 21 deg low-slant conditionsubjectsrsquo judgments on average were close to veridicaland for some subjects they were biased slightly in theopposite direction as in the other conditions In our modelthe prior toward low slants is the only factor that producesdeviations from veridical performance hence one wouldnever expect biases in the direction opposite to perceptualcompression of depthThus there must be some factors other than conflicting

depth cues or priors contributing to biases in subjectsrsquoperformance We have argued that foreshortening infor-mation could not account for the smaller-than-predictedsize modulation However an influence of this cue mightbe sufficient to shift overall biases The possibility thatthere are simply errors in the learned mapping fromperspective convergence to slant or depth also remains Inour formulation this would correspond to f(aobj s w para)

Figure 15 Illustration of ideal observer computation (see text)


being inaccurate Perceptual distortions have beenobserved even for a very simple shape matching task(Henriques et al 2005) hence the possibility of system-atic distortions in 3D interpretations cannot be discountedAlthough the model fit is not perfect the simulations

demonstrate that much of the observed pattern of biasescould potentially be explained by a general perceptualbias toward an absence of depth which has been observedin many settings Thus despite the fact that subjectsrsquojudgments were not veridical it remains possible that thevisual system was interpreting perspective information ina geometrically appropriate way but that the isolatedperspective cue is relatively weak compared to conflictinginformation

General discussion

Our results demonstrate that subjects are able to makereliable judgments of length in depth based on relativelyimpoverished images that isolated perspective conver-gence as 3D information Judgments showed large biasesrelative to veridical but had relatively low variability Theperceived length of slanted objects as indicated byobserversrsquo judgments was larger (relative to width) forsmall images than large images as predicted by perspec-tive geometry The only monocular depth cue thatspecified nonzero slant besides perspective convergencewas foreshortening of the projected contour (ie itsprojected aspect ratio) The slant implied by this cue does

not vary with overall image size and therefore could notexplain the size effect observed in the data Moreover theresults of Experiment 2 suggest that foreshortening hadlittle effect on judgments in our task and conditions Thusour results clearly implicate perspective convergence as thebasis for perceived 3D structure in our stimuliAn earlier attempt by Freeman (1966a) to measure the

ability to discriminate slant from perspective wascriticized on the grounds that the task could have beenperformed as 2D shape comparisons between sequentialpresentations (see Flock 1965 or Smith 1967) Ourmethod avoids this problem because judgments wererelative to a fixed familiar standard To account for ourresults in terms of a 2D shape-based strategy one wouldhave to assume that the shape standard used forcomparison varied systematically depending on projectedsize In addition we demonstrated with a control experi-ment that 2D shape discrimination judgments for ourstimuli are invariant to projected size whereas perfor-mance in the 3D judgment task showed significant im-provement with increased size We conclude that ourexperiment was effective in probing perception of 3D shapefrom perspectiveOne distinguishing characteristic of our paradigm is that

absolute accuracy can be assessed because there wasalways a correct response based solely on the monocularinformation which depended on a well-learned standard(a square) We found that across conditions judgmentsshowed large overall biases in the direction consistentwith perceptual compression of depth This is in agree-ment with results from other studies using direct report orprobe-matching tasks which have consistently found

Figure 16 PSEs and 75 thresholds from the model simulations (see text)


underestimation of slant and depth based on perspectiveconvergence (Andersen et al 1998 Rosinski et al 1980Smith 1967 Todd et al 2005)Judgments were also inaccurate in that the effect of

image size was not as large as would be expected based onperspective geometry A similar finding was reported bySmith (1967) for the case of slant judgments for simplecontour stimuli The fact that some size modulation wasconsistently observed indicates that this was not simplydue to poor sensitivity to image size and the results ofExperiment 2 rule out the foreshortening cue as a primarycausal factorWe hypothesized that both the overall bias in perceived

depth and the smaller-than-expected effect of size weredue to a general perceptual compression of depthPerceptual compression of depth has been observed undervarious conditions for a variety of measures (for a reviewsee Todd amp Norman 2003) In our experiment perceptualcompression could be due to some a priori bias andor theinfluence of conflicting information specifying flatnesssuch as lack of accommodation influence of the screenframe and so forthTo test the viability of this explanation we simulated the

performance of a Bayesian ideal observer that incorporateda probabilistic version of a parallelism assumption With auniform prior for slant average performance of the modelwas veridical but with a prior that is weighted toward lowslants (ie biased toward an absence of depth) estimatesexhibited biases that were qualitatively similar to thehuman data Thus the deviations from veridicality weobserved could arise even for a model that accuratelyinternalized perspective geometry if one assumes thatperspective information by itself is not Bstrong[ enough(ie a weak parallelism assumption) to fully counteracteither a priori assumptions or conflicting informationindicating an absence of depth We cannot rule out thepossibility that the visual system has simply instantiated abiased model of perspective projection However thesimulations demonstrate that such errors need not beassumed to account for most of the pattern of our resultsparticularly given that there is prior reason to expect someoverall bias toward compression of depthSome previous studies have compared slant judgments

for slanted rectangles with different sizes but with thesame slant (Freeman 1966b Stavrianos 1945) This isdifferent from what we tested because when the slant isfixed scaling a rectangular object increases the amount ofconvergence along with projected size In contrast we variedprojected size while keeping convergence constant How-ever from our data we can infer how perceived slantwould change if the object size was varied while holdingslant constant As can be seen in Figure 7 the slantspecified by perspective (dashed line) was similar for thesmall contours with low convergence and the largecontours with high convergence The contours judged tobe squares however were much taller for the smallstimuli implying that they were perceived as less slanted

Extrapolating from our data this difference would beexpected to remain even if slant from perspective wasexactly matched Thus our results indicate that a largerectangle would be perceived as more slanted than asmaller rectangle with the same slant which is consistentwith earlier reports (Freeman 1966b Stavrianos 1945)Our results are in partial conflict with those of Nichols

and Kennedy (1993) and Yang and Kubovy (1999) Inboth studies subjects rated images as most cube-like whentheir projected size was consistent with perspectiveinformation across a range of sizes One interpretation isthat observers interpret perspective information in a waythat accurately depends on projected size For examplesubjects might have used perspective convergence toperceive extent in depth and then based their judgmentson whether the corners appeared stretched or compressedrelative to that of a cube By this interpretation apreference for geometrically consistent sizes conflictswith our results because we observed that perceived depthwas compressed overall and did not scale with projectedsize by the predicted amount In Yang and Kubovyrsquosexperiment biases could have been obscured by therelatively coarse sampling of sizes However given thelarge overall biases we observed relative to veridicalsome detectable effect would be expectedA significant difference between our stimuli and those

used by both Nichols and Kennedy and Yang and Kubovyis that their stimuli contained size-independent depth cuesIf the faces of the cube-like object were implicitlyassumed to have right-angle corners or if they wereassumed to be isotropic then a unique 3D interpretation ispossible even if projected size were unknown (seeFigure 4) Thus the stimuli where the perspective cuewas inconsistent with projected size were cue conflictsituations with the task being to judge the degree ofconflict In contrast we chose restricted conditions toisolate the information available from perspective con-vergence In particular our stimuli lacked a skew symme-try cue which is scale invariant and which has been shownto be an effective cue to slant (Saunders amp Knill 2001) Fortrapezoid-shaped contours such as the ones we used anassumption of parallel sides implies right-angle cornersand vice versa hence there is no conflict betweenperspective and skew symmetry information In a follow-up study we are exploring whether perceived depth fromperspective is affected by the presence of scale-invariantinformation from skew symmetry

Implications for picture perception

Finally we consider the implications of our resultsregarding the perception of 3D structure in perspectivepictures and photographs As many others have notedobservers are not typically positioned at the correct centerof projection when viewing pictures Consequently thepictorial cues presented to the observer would generally


specify a 3D structure that is distorted relative to thedepicted sceneMuch of the previous work on perception of 3D struc-

ture in pictures has focused on the effect of viewing picturesfrom an angle which can induce a variety of perceptual dis-tortions (Goldstein 1987 1988 Halloran 1993 Koenderinkvan Doorn Kappers amp Todd 2004 Perkins 1974) In thissituation the visual system could potentially use informa-tion about the 3D orientation of the pictorial surface tocompensate for a slanted viewpoint There is considerabledebate both as to the amount of robustness in perceived3D structure and the extent that pictorial surface cuescontribute to compensation (Farber amp Rosinski 1978Goldstein 1979 1987 1988 Halloran 1993 Koenderinket al 2004 Kubovy 1986 Perkins 1974 Rosinski ampFarber 1980 Rosinski et al 1980 Vishwanath Girshickamp Banks 2005 Wallach amp Marshall 1986) Although thisis an interesting issue it is not directly relevant to thepresent experiments because only projected size was variedWhen viewing pictures from the wrong distance rather thanfrom an angle knowledge about the pictorial surface nolonger provides useful information for perceptual compen-sation Projected size would still be required to interpret depthfrom perspective convergence regardless of the distance ofthe pictureIn normal viewing of pictures inconsistencies in pic-

torial information due to incorrect scaling may be largerand more common than inconsistencies due to viewingangle When allowed observers will likely choose view-ing positions that are roughly normal to the picture surface(eg sitting in the center of a theater holding a snapshotfrontally) In contrast large inconsistencies in projectedsize would likely remain The relationship between angu-lar field of view depicted in a picture and the angle thatit subtends when viewed depends on many factors suchas the power of the camera lens (ie telephoto or wideangle) the physical size of the picture and the distance ofthe observer from the pictureBased on our results the perceived 3D structure of a

scene based on a photograph would indeed be expected tochange as a function of its magnification The effect ofprojected size we observed was smaller than that predictedby accurate use of perspective information correspondingto a gain of 02ndash03 This size dependence while limitedwould still produce significant distortions in perceived depthwhen viewing photographs with varying magnificationOur finding that size modulation is partial might appear

to conflict with the results of some previous studies whichhave reported that 3D judgments from magnified orminified images are consistent with geometric predictions(eg Bengston et al 1980 Smith amp Gruber 1958) How-ever the notion of geometric consistency tested by theseexperiments is very different and there is no actual con-flict with our results Previous studies measured whetherscaled and unscaled stimuli with identical perspective in-formation were judged to have equivalent depth structure

This is essentially a cue conflict paradigm In the case ofthe experiment by Smith and Gruber (1958) which com-pared judgments for photos and actual scenes the conflictwould consist of any cues that differ between views of anactual scene and a photograph In other studies scaled andunscaled photos with matching perspective convergencewere compared (Bengston et al 1980 Lumsden 1983Smith 1958a 1958b) In this case conflicts would arisefrom size-invariant monocular depth cues such as texturecompression or familiar size One can imagine an analo-gous variant of our experiment in which judgments werebased on either rendered stimuli or monocular views ofactual checkerboard surfaces constructed to have match-ing projected images If results were similar it would notimply that perspective convergence was interpreted in anaccurate scale-dependent way Rather it would imply thatperspective convergence dominated other 3D cues Sim-ilarly to the extent that results differed it would implythat other cues influenced judgments Thus this type ofdesign addresses the question of how much perspectivecontributes relative to other depth information This is verydifferent from asking as in our experiment whether per-ceived depth from convergence changes in a geometricallyaccurate way when projected size is variedIf perception of 3D structure from perspective does

depend on projected size as suggested by our resultsthere remains a question as to why perception of picturesappears Brobust[Vthat is why we do not more frequentlyexperience noticeable distortions One possibility is thatsize-dependent distortions are tolerated because the (in-correctly) perceived scene is also plausible This sort ofexplanation is discussed in Koenderink et al (2004) Forexample in the special case of rectangular objects that arepartially aligned with the picture plane (as tested here)distortions due to image magnification preserve propertieslike mirror symmetry and right-angle corners (ie rect-angles remain rectangular) and therefore might not in-terfere with the ability to perceive the general structure ofthe scene More generally perception of 3D structure mightbe based on intrinsic geometric relations that are invariantto the sorts of distortions caused by changes in viewpoint(Busey Brady amp Cutting 1990 Gibson 1950 Perkins1972 Sedgwick 1983 1991)It is also possible that the size dependence we observed

was due to the degenerate nature of our stimuli Real-world scenes provide a variety of monocular depth cuesbesides perspective convergence some of which providesize-invariant information (eg skew symmetry) Present-ing an image at the wrong projected size (as when viewinga picture from the wrong distance) generally introduces aconflict between the depth specified by perspective andsize-invariant cues However in our stimuli this conflictwas intentionally minimized A natural question forfurther work is whether perceived depth from perspectiveis modulated by image size in a similar way when othermonocular cues are available


Appendix AMinimized expectedentropy staircase method

In choosing the stimulus aspect ratios to test on eachtrial we used a new adaptive procedure which we termminimized expected entropy staircase methodFor a given trial the probe ratios and responses

from previous trials in the same condition xkrkwere used to estimate a posterior probability distributionP(2Aordfx1r1x2r2Ixnrn) where 2 is the PSE and A isthe difference between the PSE and the 75 point Thenext probe xn + 1 was chosen to minimize the expectedentropy jp log(p) of the posttrial posterior functionP(2Aordfx1r1x2r2Ixn + 1rn + 1) The entropy cost func-tion rewards probes that would be expected to result in amore peaked and concentrated posterior distribution overthe space of possible combinations of 2 and A consistentwith the goal of estimating 2 and A with minimal boundsof uncertaintyThere are only two possibilities for the next response 0

or 1 and for each of these possibilities one can computewhat the new postresponse likelihood distribution wouldbe as well as its entropy The expected value of entropyis simply a weighted average of the two possible re-sults where weights are proportional to their probabilitiesP(rn + 1 = 0ordfxn + 1) and P(rn + 1 = 1ordfxn + 1) If 2 and s wereknown these probabilities would be directly determinedby the model psychometric function Thus to estimateP(rn+ 1ordfxn+ 1) we marginalized over 2 and A using theposterior distribution computed from previous responsehistory as an estimate of P(2A)

P rnthorn1kxnthorn1eth THORN ~2A

P rnthorn1kxnthorn12Aeth THORN

P 2Akx1 r1Ixn rneth THORN ethA1THORN

In our implementation we used a logistic function tomodel the psychometric function P(rn + 1ordfxn + 1 2 A)rather than a more standard cumulative Gaussian tosimplify computation during probe selection Also thefunction was scaled to range from 0025 to 0975 ratherthan from 0 to 1 to reduce the effect of lapses of attentionand guessing on the probe selection The space of possiblebias and threshold values was discretely sampled to carryout marginalization with A sampled linearly from theset 005 01 I 08 and 2 sampled exponentially fromthe set 026 0274 I 387 for low-slant conditionsand from the set 0094 0100 I 142 for high-slantconditionsOur staircase method is a greedy algorithm in that it

minimizes the expected entropy only after the succeedingtrial not for the whole future sequence We do not yetknow how much this greedy method diverges from a full

optimization However informal testing of the procedurerevealed it to be highly efficient and robustOne aspect of the methodrsquos behavior that could be

problematic in practice is that once the estimates of 2 andA have converged the probe choices tend to oscillatebetween two values symmetric around the PSE and beanticorrelated with the previous response This occursbecause the expected entropy function at this point hastwo local minima that are very similar such that a singleresponse switches their relative depths Consequentlyprobe values would tend to alternate which couldinfluence a subjectrsquos behavior In the experiment reportedhere there were many interleaved conditions in eachblock and there were a modest number of trials perstaircase hence temporal correlations were not a concernHowever in a design with few conditions and many trialsper staircase this would be a more serious problem Asimple solution is to use a random subset of the responsehistory to estimate the posterior function rather than thewhole history once a sufficient number of trials arerecorded Because the method converges to a roughestimate quickly (within 15ndash20 trials) excluding a subsetof trials has little effect on the final distribution of probesamples Note that with this modification there is no needto run multiple interleaved staircases using our method asis commonly done when using standard staircases

Appendix B Measurement noisefor the ideal observer model

In this appendix we describe how we modeled noise inimage measurements for our ideal observer simulationsWe modeled the noise in the shape parameter aproj as

being Gaussian with a width parameter Aa that was setbased on previous psychophysical measures of 2Dorientation discrimination Discrimination of 2D orienta-tions exhibits an oblique effect Thresholds are higheraway from the horizontal and vertical axes (Heeley et al1997 Regan amp Price 1986 Snippe amp Koenderink 1994)In the case of our stimuli this would imply thatorientations are encoded less reliably for our high-slantconditions than for our low-slant conditions Uncertaintyin shape measurement could alternatively be modeled as afunction of corner angles of the projected figure asopposed to the orientations of its side edges Thresholdsfor 2D angle discrimination follow an m-shaped functionof base angle with a local minimum at 90 deg (Chen andLevi 1996 Heeley and Buchanan-Smith 1996 ReganGray et al 1996) thus one would similarly expectgreater noise for the high-slant conditions On the basis ofthese various results we estimated that the effectiveorientationangle noise for the high-slant condition wouldbe about twice as high as for the low-slant condition withall other factors equal Orientation discrimination for 2D


lines has also been found to strongly depend on line lengthFor extended lines thresholds decrease roughly with thesquare root of length (Heeley amp Buchanan-Smith 1998Orban Vandenbussche and Vogels 1984) Incorporatingthis length dependence our model for noise in projectedshape was 25 degsqrt(L) for the low-slant conditions and5 degsqrt(L) for the high-slant conditions where L is thelength of the side edges (in degrees of visual angle)For uncertainty in measurement of the width of projected

figures we assumed proportional noise around the correctvalue such that log(w) is a Gaussian with deviation Aw =005 log units This would be consistent with results fromstudies of interval length discrimination which have foundthat thresholds increase proportionally with length withWeber fraction of approximately 005 across a range ofconditions (Burbeck 1987 Toet van Eekhout Simons ampKoenderink 1987 Whitaker amp Latham 1997) We foundthat this noise parameter could be varied somewhat with-out affecting the qualitative performance of the model pro-vided that it remains small compared with the uncertaintyintroduced by orientation noiseFor any combination of slant and 3D object shape the

true projected shape parameter aparaproj is determined by per-spective geometry hence only the projected width pa-rameter wpara needs to be explicitly marginalized Using thenoise models the desired likelihood function becomes

P aprojks aobjw

Egrave

ZZ aproj j f aobj swpara

=Aa

Z wjwparafrac12 =Aweth THORNdwpara ethB1THORN

where Z is a standard Gaussian distribution and f is theprojection function mapping aobj to aproj for a given slantand widthThe final step of the modeling was to simulate the effect

of measurement noise on performance of our experimentaltask (see main text) For this it was necessary to assume anoise model for the measurement of contour height inaddition to noise in measurement of width and side anglesWe assumed proportional noise for contour height withthe same Weber fraction 5 as for image measurementof contour width

Acknowledgment

This research was supported by NIH Grant EY-013988

Commercial relationships noneCorresponding author Jeffrey A SaundersEmail jeffrey_a_saundersyahoocomAddress 3401 Walnut Street Philadelphia PA 19104USA

References

Andersen G J Braunstein M L amp Saidpour A (1998)The perception of depth and slant from texture inthree-dimensional scenes Perception 27 1087ndash1106[PubMed]

Attneave F amp Olson R K (1966) Inferences aboutvisual mechanisms from monocular depth effectsPsychonomic Science 4 133ndash134

Banks M S amp Backus B T (1998) Extra-retinal andperspective cues cause the small range of the inducedeffect Vision Research 38 187ndash194 [PubMed]

Bengston J K Stergios J C Ward J L amp Jester R E(1980) Optic array determinants of apparent distanceand size in pictures Journal of Experimental Psy-chology Human Perception and Performance 6751ndash759 [PubMed]

Braunstein M L amp Payne J W (1969) Perspective andform ratio as determinants of relative slant judgmentsJournal of Experimental Psychology 81 584ndash590

Burbeck C A (1987) Position and spatial frequency inlarge-scale localization judgments Vision Research27 417ndash427 [PubMed]

Busey T A Brady N P amp Cutting J E (1990)Compensation is unnecessary for perception of facesin slanted pictures Perception amp Psychophysics 481ndash11 [PubMed]

Chen S amp Levi D M (1996) Angle judgement Is thewhole the sum of its parts Vision Research 361721ndash1735 [PubMed]

Clark W C Smith A H amp Rabe A (1955) Retinalgradients of outline as a stimulus for slant CanadianJournal of Psychology 9 247ndash253 [PubMed]

Clark W C Smith A H amp Rabe A (1956) Theinteraction of surface texture outline gradient andground in the perception of slant Canadian Journalof Psychology 10 1ndash8 [PubMed]

Cutting J E amp Vishton P M (1995) Perceiving layoutand knowing distances The interaction relativepotency and contextual use of different informationabout depth In W Epstein amp S Rogers (Eds)Perception of space and motion (pp 69ndash117) SanDiego CA Academic Press

Farber J amp Rosinski R R (1978) Geometric trans-formations of pictured space Perception 7 269ndash282[PubMed]

Flock H R (1965) Optical texture and linear perspectiveas stimuli for slant perception Psychological Review72 505ndash514 [PubMed]

Freeman R B Jr (1966a) Absolute threshold for visualslant The effect of stimulus size and retinal perspective


Journal of Experimental Psychology 71 170ndash176[PubMed]

Freeman R B Jr (1966b) Function of cues in theperceptual learning of visual slant An experimentaland theoretical analysis Psychological Monographs80 1ndash29 [PubMed]

Garding J (1993) Shape from texture and contour byweak isotropy Artificial Intelligence 64 243ndash297

Gibson J J (1950) The perception of the visual worldBoston Houghton Mifflin

Gillam B J (1968) Perception of slant when perspectiveand stereopsis conflict Experiments with aniseikoniclenses Journal of Experimental Psychology 78299ndash305 [PubMed]

Gogel W C (1965) Equidistance tendency and its conse-quencesPsychological Bulletin 64 153ndash163 [PubMed]

Goldstein E B (1979) Rotation of objects in picturesviewed at an angle Evidence for different propertiesof two types of pictorial space Journal of Exper-imental Psychology Human Perception and Perfor-mance 5 78ndash87 [PubMed]

Goldstein E B (1987) Spatial layout orientation relativeto the observer and perceived projection in picturesviewed at an angle Journal of Experimental Psychol-ogy Human Perception and Performance 13 256ndash266[PubMed]

Goldstein E B (1988) Geometry or not geometryPerceived orientation and spatial layout in picturesviewed at an angle Journal of Experimental Psychol-ogy Human Perception and Performance 14 312ndash314[PubMed]

Halloran T O (1993) The frame turns also Factors indifferential rotation in pictures Perception andPsychophysics 54 496ndash508 [PubMed]

Heeley D W amp Buchanan-Smith H M (1996)Mechanisms specialized for the perception of imagegeometry Vision Research 36 3607ndash3627[PubMed]

Heeley D W amp Buchanan-Smith H M (1998) Theinfluence of stimulus shape on orientation acuityExperimental Brain Research 120 217ndash222[PubMed]

Heeley D W Buchanan-Smith H M Cromwell J A ampWright J S (1997) The oblique effect in orientationacuity Vision Research 37 235ndash242 [PubMed]

Henriques D Y Flanders M amp Soechting J F (2005)Distortions in the visual perception of shape Exper-imental Brain Research 160 384ndash393 [PubMed]

Hillis J M Watt S J Landy M S amp Banks M S(2004) Slant from texture and disparity cues optimalcue combination Journal of Vision 4(12) 967ndash992

httpjournalofvisionorg4121 doi1011674121[PubMed] [Article]

Kanade T (1981) Recovery of the three-dimensionalshape of an object from a single view ArtificialIntelligence 17 409ndash460

Knill D C (1998a) Discrimination of planar surface slantfrom texture Human and ideal observers comparedVision Research 38 1683ndash1711 [PubMed]

Knill D C (1998b) Ideal observer perturbation analysisreveals human strategies for inferring surface orien-tation from texture Vision Research 38 2635ndash2656[PubMed]

Knill D C amp Saunders J A (2003) Do humansoptimally integrate stereo and texture information forjudgments of surface slant Vision Research 432539ndash2558 [PubMed]

Koenderink J J van Doorn A J amp Kappers A M(1994) On so-called paradoxical monocular stereo-scopy Perception 23 583ndash594 [PubMed]

Koenderink J J van Doorn A J Kappers A M ampTodd J T (2004) Pointing out of the picturePerception 33 513ndash530 [PubMed]

Kubovy M (1986) The psychology of perspective andrenaissance art Cambridge Cambridge UniversityPress

Li A amp Zaidi Q (2000) Perception of three-dimen-sional shape from texture is based on patterns oforiented energy Vision Research 40 217ndash242[PubMed]

Lumsden E A (1983) Perception of radial distance as afunction of magnification and truncation of depictedspatial layout Perception amp Psychophysics 33177ndash182 [PubMed]

Nichols A L amp Kennedy J M (1993) Angularsubtense effects on perception of polar and parallelprojections of cubes Perception amp Psychophysics54 763ndash772 [PubMed]

Orban G A Vandenbussche E amp Vogels R (1984)Human orientation discrimination tested with longstimuli Vision Research 24 121ndash128 [PubMed]

Perkins D N (1972) Visual discrimination betweenrectangular and nonrectangular parallelepipeds Per-ception amp Psychophysics 12 396ndash400

Perkins D N (1974) Compensation for distortion in view-ing pictures obliquely Perception amp Psychophysics14 13ndash18

Perkins D N (1976) How good a bet is good formPerception 5 393ndash406 [PubMed]

Regan D Gray R amp Hamstra S J (1996) Evidencefor a neural mechanism that encodes angles VisionResearch 36 323ndash330 [PubMed]


Regan D Hajdur L V amp Hong H (1996) Two-dimensional aspect ratio discrimination for shapedefined by orientation texture Vision Research 363695ndash3702 [PubMed]

Regan D ampHamstra S J (1992) Shape discrimination andthe judgment of perfect symmetry Dissociation of shapefrom size Vision Research 32 1845ndash1864 [PubMed]

Regan D amp Price P (1986) Periodicity in orientationdiscrimination and the unconfounding of visual in-formation Vision Research 26 1299ndash1302 [PubMed]

Rosas P Wichmann F A amp Wagemans J (2004)Some observations on the effects of slant and texturetype on slant-from-texture Vision Research 441511ndash1535 [PubMed]

Rosenholtz R amp Malik J (1997) Surface orientationfrom texture Isotropy or homogeneity (or both)Vision Research 37 2283ndash2293 [PubMed]

Rosinski R R amp Farber J (1980) Compensation forviewing point in the perception of pictured space InM A Hagen (Ed) The perception of pictures (Vol 1pp 137ndash176) New York Academic Press

Rosinski R R Mulholland T Degelman D amp Farber J(1980) Picture perception An analysis of visual com-pensation Perception amp Psychophysics 28 521ndash526[PubMed]

Saunders J A amp Backus B T (2006) Perception of slantfrom oriented textures Journal of Vision 6(9) 882ndash897httpjournalofvisionorg693 doi101167693[PubMed] [Article]

Saunders J A amp Knill D C (2001) Perception of 3Dsurface orientation from skew symmetry VisionResearch 41 3163ndash3183 [PubMed]

Sedgwick H A (1980) The geometry of spatial layout inpictorial representation In M A Hagen (Ed) Theperception of pictures (Vol 1 pp 33ndash90) New YorkAcademic Press

Sedgwick H A (1983) Environment-centered represen-tation of spatial layout Available visual informationfrom texture and perspective In J Beck B Hope ampA Rosenfield (Eds) Human and machine vision(pp 425ndash458) New York Academic Press

Sedgwick H A (1991) The effect of viewpoint on thevirtual space of pictures In S R Ellis M K Kaiseramp A C Grunwald (Eds) Pictorial communicationin virtual and real environments (pp 460ndash469)London Taylor amp Francis

Smith A H (1967) Perceived slant as a function ofstimulus contour and vertical dimension Perceptualand Motor Skills 24 167ndash173

Smith O W (1958a) Comparison of apparent depth in aphotograph viewed form two distances Perceptualand Motor Skills 8 79ndash81

Smith O W (1958b) Judgments of size and distance inphotographs American Journal of Psychology 71529ndash538 [PubMed]

Smith O W amp Gruber H (1958) Perception of depth inphotographs Perceptual and Motor Skills 8 307ndash313

Snippe H P amp Koenderink J J (1994) Discriminationof geometric angle in the fronto-parallel planeSpatial Vision 8 309ndash328 [PubMed]

Stavrianos B K (1945) The relation of shape perceptionto explicit judgments of inclination Archives ofPsychology No 296

Stevens K A (1983) Slant-tilt The visual encoding ofsurface orientation Biological Cybernetics 46183ndash195 [PubMed]

Todd J T amp Norman J F (2003) The visual perceptionof 3-D shape from multiple cues Are observerscapable of perceiving metric structure Perceptionamp Psychophysics 65 31ndash47 [PubMed]

Todd J T Thaler L amp Dijkstra T M (2005) Theeffects of field of view on the perception of 3D slantfrom texture Vision Research 45 1501ndash1517[PubMed]

Toet A van Eekhout M P Simons H L ampKoenderink J J (1987) Scale invariant features ofdifferential spatial displacement discriminationVision Research 27 441ndash451 [PubMed]

Vishwanath D Girshick A R amp Banks M S (2005)Why pictures look right when viewed from thewrong place Nature Neuroscience 8 1401ndash1410[PubMed]

Wallach H amp Marshall F J (1986) Shape constancy inpictorial representation Perception amp Psychophysics39 233ndash235 [PubMed]

Watt S J Akeley K Ernst M O amp Banks M S(2005) Focus cues affect perceived depth Journal of Vi-sion 5(10) 834ndash862 httpjournalofvisionorg5107doi1011675107 [PubMed] [Article]

Whitaker D amp Latham K (1997) Disentangling the roleof spatial scale separation and eccentricity in Weberrsquoslaw for position Vision Research 37 515ndash524[PubMed]

Witkin A P (1981) Recovering surface shape andorientation from texture Artificial Intelligence 1717ndash45

Yang T amp Kubovy M (1999) Weakening the robust-ness of perspective Evidence for a modified theory ofcompensation in picture perception Perception ampPsychophysics 61 456ndash467 [PubMed]

Zanker J M amp Quenzer T (1999) How to tell cir-cles from ellipses Perceiving the regularity of simpleshapes Naturwissenschaften 86 492ndash495 [PubMed]


Knill amp Saunders 2003) or confounded with other textureinformation (Hillis Watt Landy amp Banks 2004 RosasWichmann amp Wagemans 2004 Saunders amp Backus2006) There have been studies that measure the accuracyof perceived slant when only convergence information isavailable (Andersen et al 1998 Rosinski et al 1980Smith 1967 Todd et al 2005) However these data allowonly limited interferences about whether perspective con-vergence is used in a geometrically consistent mannerThe goal of the experiments reported here was to attain a

more quantitative psychophysical measure of the percep-tion of depth from perspective convergence Specificallywe measured the accuracy and reliability of 3D judgmentswhen the only information indicating depth was perspec-tive convergence and tested whether perceived depthchanged in a geometrically consistent way in responseto size scaling of images We used a match-to-known-standard paradigm that indirectly probed perceived depthSubjects were presented with images of slanted rectanglesand judged whether the perceived 3D shape was taller orwider than a square As we illustrate in the next sectionthis task can be performed using monocular informationand the correct response depends on image sizeA similar aspect ratio judgment task has been used for

2D rectangular shapes viewed frontally and observersexhibit discrimination thresholds as low as 1 (ReganHajdur amp Hong 1996 Regan amp Hamstra 1992 Zankeramp Quenzer 1999) Like the 2D task the 3D task dependson the ability to measure aspect ratio and to make adecision However the 3D task also forces the observer tointerpret perspective convergence as slant Thresholds arein general lower when observers are instructed to make2D judgments than when they are instructed to make 3Djudgments indicating that performance in the 3D task islimited by the observerrsquos ability to recover slant fromperspective convergence (see Experiment 1)

Size dependence of perspective information

Consider the geometry of lines projected onto a planarimage If a pair of lines is parallel in the world they

project to lines that converge to a vanishing point in theimage If multiple sets of parallel lines are present withdifferent orientations but lying on the same planar surfacethen their multiple vanishing points uniquely define ahorizon and thereby specify 3D orientation of the surfacethat is its slant and tilt (Stevens 1983) relative to a line ofsight that intersects the plane Once the slant and tilt areknown it would then be possible to reconstruct the shapeof the object up to a scale factor for distance (eg by backprojecting onto the slanted plane)It is unnecessary to compute the horizon line explicitly

to use the convergence cue For example it would sufficeto learn a direct mapping from differences in projectedline orientations to possible parallel interpretations in 3DFigure 2 illustrates the relationship between perspectiveconvergence and 3D orientation for an image of aslanted rectangle In this special case where one pair ofedges remains parallel in the image (Bconverging[ to apoint at infinite distance in the horizontal direction) thereis a simple relationship between slant and perspectiveconvergence

tanethaTHORN frac14 tan w=2eth THORN tanethsTHORN eth1THORN

where a is the angle of the converging side edges relativeto the tilt direction (ie tan(a) is the slope relative tovertical) s is the egocentric slant of the surface (as mea-sured at the origin) and w is the angular width measuredthrough the center This formulation follows thatof Freeman (1966b equivalent formulations includeBraunstein amp Payne 1969 and Flock 1962) A similarrelation holds for more generic poses but would involveboth the slant and tilt of the surface We describeperspective convergence in terms of projection onto animage plane because it is convenient the same geometric

Figure 1 Example of an image in which depth can be perceivedfrom pictorial cues The distorted checkerboard pattern is seen asa slanted rectangular surface covered with uniform texturePerceived slant in depth of the surface could be based ondifferent assumptions about the texture such as that the squaresare uniform sized in the world or that the converging lines in depthare parallel edges in the world

Figure 2 Illustration of the relationship between perspectiveconvergence and surface slant for the projected image of aslanted rectangular surface aligned with its direction of tilt Wedefine the angular width w and slant s to be relative to a referencepoint at the center of the 3D object (left panels) and assume aprojection plane with a distance of 1 The sides of the projectedcontour converge to a point on the horizon (right panel) which is adistance of tan(90 deg j s) or 1tan(s) away from the centralreference point Note that the reference point can be identified inthe image as the intersection between the diagonals of theprojected contour The slope of the sides relative to verticaltan(a) is equal to tan(w2) tan(s)

















Experiment 1









Procedure










Subjects



Results











Perceptual biases





























Experiment 2













Procedure





Subjects



Results








Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References





























































































Experiment 1









Procedure










Subjects



Results











Perceptual biases





























Experiment 2













Procedure





Subjects



Results








Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References



















































































Procedure










Subjects



Results











Perceptual biases





























Experiment 2













Procedure





Subjects



Results








Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References



















































































Subjects



Results











Perceptual biases





























Experiment 2













Procedure





Subjects



Results








Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References


















































































Perceptual biases





























Experiment 2













Procedure





Subjects



Results








Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References





































































































Experiment 2













Procedure





Subjects



Results








Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References



















































































Procedure





Subjects



Results








Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References















































































Subjects



Results








Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References


















































































Discussion

















P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References






















































































P s aobjkaprojw


P seth THORNP aobj

eth2THORN















General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References
























































































General discussion













































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References




















































































































P aprojks aobjw

Egrave


=Aa




Acknowledgment



References














































































The accuracy and reliability of perceived depth from linear perspective ... · The accuracy and...

Documents

Transcript of The accuracy and reliability of perceived depth from linear perspective ... · The accuracy and...