From Mnemosyne to Terpsichore - the Bilderatlas after the Image [email protected]ÉcolePolytechniqueFédéraledeLausanne,[email protected]ÉcolePolytechniqueFédéraledeLausanne,Switzerland
Introduction ThisstudyconcernsAbyWarburg’s lastandmost
ambitiousproject:theAtlasMnemosyne(orBilderat-las), conceived in August 1926 and truncated threeyearslater,unfinished,byWarburg’ssuddendeathinOctober1929.Mnemosyneconsistsofaseriesoflargeblackpanels,onwhichareattachedavariablenumberof black-and-white photographs of paintings, sculp-tures,tarotcards,stamps,coins,andothertypesofim-ages. The version we use is the one Warburg wasworkingonatthetimeofhisdeath,alsoknownasthe“1-79version”:itincludesaroundathousandimagespinnedon63panels(ThisversionoftheAtlasispub-lishedinvariousprinteditions,andavailableonline).TheBilderatlasisaconceptualmaze-theculminationofalife’sscholarshipinimagesandmemory-throughwhichperhapstheclearestthreadistheconceptofPa-thosformel,orformulafor(theexpressionof)passion.Muchexcellentworkhasbeenwrittenontheconceptbut,tothebestofourknowledge,nobodyhasyettriedto “operationalise” it - to turn it into a sequence ofquantitativeoperations,orinotherwords,intoanal-gorithm(Moretti,2013).
The Pathosformel and its Operationalisation Onthemostbasiclevel,thePathosformeldescribes
theportrayalpassionateemotionsthroughaformula,arepeatablevisualparadigm.ThePathosformelowesmuchofitsforce,asSalvatoreSettishaspointedout,toitscombinationofsemanticopposites:an“oxymo-ronic word, in that it merges in the same term themovementofpathosand the rigidityof the formula-schema”(Settis1997).
Ratherthanattemptingtooperationalisetheentireconceptatonce,wefirstbreakitdownintoitsconstit-uent components: first the morphology of formula,thenthedynamismofpathos.
Weturn to twowell-studiedcasesofWarburgianformula: theDeathofOrpheus(whereWarburg firstnamedtheconceptofPathosformel),andtheNymph,headhuntressandFortuna(Bilderatlaspanels46-48).Looking at Warburg’s examples of the Orpheus-for-mula,reproducedinFigure1,wecanhardlystopour-selves fromspottinga formulawhich repeats acrossthe ages; but how could this intuitive similarity bemeasured?
Ouralgorithmcomesasfollows: 1. Weisolateeachindividualbodyfromits
context. It is clear the Pathosformel re-latestoindividualcharacters-theobjectofstudythusbecomesnotpanelsorpho-tographsintheBilderatlas,butindividualbodies.
2. Wetakeonlytheskeletonsofsuchbodies.Herewe are eliminating colour, clothes,hands, faces, gender, age. This is not tosay that such factors aren’t important:but they are not elementary to the for-mula(seeforexample,AndréJolles’lettertoWarburg23rdDecember1900,wheretheformulaofGhirlandaio’sNymphhopsbetween Judith, Salome, Tobias, Gabrieletc.–seeGhelardi,2004;ortheformulainBilderatlas Panel 47 shared betweenGiambolo-gna’s Samson and Donatello’sJudith.)
3. Wecomparetheseskeletonsbymeasur-ing the angles of the main limbs of thebody,asdescribedinSection5.
Each of these steps is not merely a convenientquantification, but a conceptual wager. This is thestrength of operationalisation as a critical tool: itforcesonetobeexplicitabouttheconceptualchoicesonemakes.
Fig 1: the Death of Orpheus: details from Warburg’s
example figures from classical antiquity to Dürer. Reproduced from Warburg 1905/1999
Anatomy, Emotion and Pose
WarburgwascertainlyinfluencedbyDarwin’sTheExpressionof theEmotions inManandAnimals(Dar-win1872) -which,when talkingabouthumanemo-tions,largelyconcentratesontheface.Indeed,thefirstfigureofthebookisananatomicaldiagramoftheface-“Ishalloftenhavetorefer[...]tothemusclesofthehuman face” (Darwin 1872 p.23).Warburgwas cer-tainly struck by the book - writing in his diary “einBuch,dasmirhilft!”(Gombrich,p.72).Hewasalsoin-terestedandcapableofstudyingthefaceinart(seee.g.hisdiscussionoffacesinGhirlandaio’sConfirmationinSantaTrinita,inTheArtofPortraitureintheFloren-tineBourgeoisie(inWarburg1999p.185),yetneverin relation to Pathos - his descriptions of the Pa-thosformelrelateexclusivelytothebody.
WecanrelateWarburg’sdecisiontothelargepsy-chological literature on emotional recognition frombodies. Psychological studies are based on the LightSpotsModelbyJohansson(1973),oftencalled‘biolog-ical motion’, in which reducing body pose to 10-12points - quite comparable to our own reduction - isjudged to give a ‘compelling impression of humanwalking,running,dancingetc.’.UsingonlyLightSpots,observers can reliably tell gender andemotion fromdynamicpose(Kozlowski1977,Montepare1987).In-deed,ithasbeensuggestedthatouremotionalunder-standingoffacesismoreinfluencedbyourperceptionofthebodythanviceversa(VandenStock2007). Encoding Pathos through Pose
The Atlas is, even by today’s dataset standards,quitesizeable:1000imagesacross63panels,contain-inganorderof103-104depictedhumanfigures.Scala-ble manual annotation is only therefore possiblethrough crowdsourcing, which we did through theCrowdFlowerplatform.
Accuratelyannotatingeveryvisiblefigureinanim-ageisadifficultandambiguoustasks.Additionally,ifdifferent workers annotate different figures in thesameimage,theannotationscannotbecollatedorav-eraged.We thereforedevelopeda two-stageannota-tionprocess:
Humanfiguresareextractedfromthepaintingbydrawingbounding-boxes.Thisisdonethreetimesperimage(bythreeseparateworkers).
Having aggregated the information from the firststage,separate imagesareproducedforeachhumanfigure.Detailedposeinformation(thepositionofma-jorbody-points)isthenaddedbythreeseparatework-ers,withtheinformationaggregated.
Itshouldbeclarifiedthatthedecisiontoannotatebodies in isolation (for annotationaccuracyand just
workercompensation)isquiteseparatetotheearlierconceptual decision to analyse bodies individually,whichrelatestotheobjectofstudy.Itwouldbequitepossibletodoeitheronewithouttheother.
Using this two-stageannotationprocess,wehavepresentlyannotated⅓oftheBilderatlas(bypanels),resulting in1,665aggregatedhumanposes.The col-lection and aggregation of the data are described ingreatertechnicaldetailelsewhere(Impett,2016).
Data Analysis: dimensionality reduction and dimensioned reductionism
Havingencodedourstaticposes,howdoweana-lyseandcompareacollectionofhumanfiguresofdif-ferent sizes, proportions and orientations? We mir-roredtheposeshorizontallyandcontrolledforglobalrotation, ending upwith a 11-dimensional vector P,describingtheanglesofthemainlimbs.
Fromthisangularposevector,wecanusecircular
statisticstofindamorphologicaldistanceDa,bbetweentwoposesPaandPb:
WherePa,iistheithangleofposevectorPa,and||α
istheangularradiandistance:
Thesemorphologicalpose-differencesarepercep-
tually meaningful over short distances. On a largerscale, they become less perceptually significant: is asittingperson‘closer’toalyingorstandingperson?
Inordertomakeourdistanceanalysisperceptuallyrelevant,therefore,wefirstclusteredour1,665posesinto 16 clusters by rotational K-means clustering(Dordet-Berdanet and Wicker 2008). Our two-stageclusteringsystemisthereforeasfollows:
I. K-meansclustering(toproducemeaning-fulclusters)
II. Hierarchical clustering (for within-clus-termorphologicalinformation)
Thenumberof clustersK is chosenby lookingattheinter-clustervarianceoverK.Theresultofthefirststage of clustering is shown in Figure 2; Figure 3showsanexamplesectionfromahierarchicalmapofCluster1.
Fig 2: our 16 pose-clusters
Fig 3: a detail from a dendrogram of different poses within the Bilderatlas, produced by second-stage clustering within
Cluster 2
Unity of the Pathosformeln: from distant to close reading
Someoftheclustersclearlyrepresentphysicalac-tivities -sitting,praying,embracing,dancing-whilstothersseemmoresubtlycommunicativeorexpressiveinnature.Havingreorganisedthe1,665figuresinto16mean-centredclusters,weproceededtotracetheclas-sicalPathosformeln-identifiedintheAtlasbyprimaryandsecondaryliterature-throughourclusters.
ThecanonicalPathosformelnaremainlymytholog-icalfigures(Perseus,Pentheus,Orpheus)orrecurringallegories(Graces,Nymphs,Fortuna).Theywerepre-viouslydescribedasdistinct,andweexpectedtofinda taxonomyof such formulae throughour clusteringanalysis.
On the contrary, the statistical result was muchstronger: a complete morphological unity. AlmosteveryidentifiedPathosformelfallsintoCluster1,withfewfalsepositives-over80%ofthefiguresinCluster1areanidentifiedPathosformel.Thehandfulofexcep-tionsareallborderlinecases,placedinperipheraltoCluster1(Clusters7and13).
Lookingmorecloselyattheimagesthemselves,asinFigure4,thisbecomesvisuallyclear:notonlydothePathosformelnsharecertainposefeatures(most im-portantly,araisedarm)presentnowhereelse in thedataset.Todate,however,theauthorsknowofnoart-historical literaturethathas identifiedsuchmorpho-logicalunity.
Figure 4, clockwise from top-left: Laocoön, Orpheus, Fortuna, Nymph, Judith, Perseus - all except Orpheus are in the Atlas. The identified Pathosformeln share distinguishing features from the other characters in the Atlas: a raised arm,
most often accompanied by a lowered second arm, and a slight twist of the body.
Concluding remarks
Ourmorphologicalmodel forPathosformel issta-tisticallystrong:butwhataretheart-historicalimpli-cations?TheoppositionalsymmetryandraisedarmofCluster1(Fig.2)remindsusofaContrapposto,butthebodiesthemselvesarefarremovedfromsuchclassicalbalance(e.g.Fig.4,top).Ratherthanmovement,ten-sion(betweenupperandlowerbody)seemstobethefundamentalelementofPathosformel-thenatureofwhichwillbethesubjectofasubsequentpublication.
Ourmorphological analysishas shown that staticpose can identifyPathosformeln, and thata studyofstaticposethroughalargecollectionofartisticworkscanidentifylinksacrossstyles,periodsandcultures.
Automaticdetectionofposeisafocusofthecurrentresearch,andwillallowustoexpandourart-historicalmodelsbeyondtheBilderatlas.Humans(thusbodies,andposes)areunsurprisinglythemostcommonfea-tureof humanart, and thereforemake excellent ob-jectsof study foranarthistoryof theLongueDurée(Robb2015).
Bibliography
Darwin,C.(1872).Theexpressionoftheemotionsinmanandanimals.London,JohnMurray
Dortet-Bernadet,J.L.andWicker,N.,(2008).Model-based
clusteringontheunitspherewithanillustrationusinggeneexpressionprofiles. InBiostatistics,9(1),pp.66-80.
Gombrich,E.H.(1970).AbyWarburg,anintellectualbiog-
raphy.ChicagoUniversityPress.Impett, L. and Süsstrunk, S., (2016). Pose and Pa-
thosformelinAbyWarburg’sBilderatlas.InEuropeanConference on Computer Vision (pp. 888-902).SpringerInternationalPublishing.
Johansson,G., (1973).Visualperceptionofbiologicalmo-
tionandamodelforitsanalysis.InPerception&psy-chophysics,14(2),pp.201-211.
Kozlowski,L.T.andCutting,J.E.,(1977).Recognizingthe
sexofawalkerfromadynamicpoint-lightdisplay.InPerception&Psychophysics,21(6),pp.575-580.
Montepare,J.M.,Goldstein,S.B.andClausen,A.,(1987).
The identificationof emotions fromgait information.InJournalofNonverbalBehavior,11(1),pp.33-42.
Moretti,F.,(2013).“Operationalizing”;or,TheFunctionof
Measurement in Modern Literary. Stanford LiteraryLab Pamphlet Series. litlab.stanford. edu/Liter-aryLabPamphlet6
Robb,J.andHarris,O.J.T.(2013).“Bodyworldsandtheir
history:someworkingconcepts.”TheBodyinHistory:EuropefromthePalaeolithictothefuture:7(31).
Settis,“PathosundEthos,MorphologieundFunktion”,inW.
Kemp, G. Mattenklott, M. Wagner, M. Warnke, eds,VorträgeausdemWarburg-Haus,Band1,DeGruyter,Berlin1997,pp.39-44.
VandenStock, J.,Righart,R.andDeGelder,B., (2007).
Bodyexpressionsinfluencerecognitionofemotionsinthefaceandvoice.InEmotion,7(3),p.487.
Warburg,A., trans.D.Britt.(1999)TheRenewalofPaganAntiquity.LosAngeles,GettyResearchInstitute.
Top Related