Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan...
-
Upload
constance-goodman -
Category
Documents
-
view
215 -
download
1
Transcript of Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan...
![Page 1: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/1.jpg)
Ranking and Classifying Ranking and Classifying Attractiveness of Attractiveness of Photos in FolksonomiesPhotos in Folksonomies
Jose San Pedro and Stefan SiersdorferJose San Pedro and Stefan Siersdorfer
University of Sheffield, L3S Research CenterUniversity of Sheffield, L3S Research Center
WWW 2009WWW 2009
![Page 2: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/2.jpg)
22
OutlineOutline
IntroductionIntroduction
Features for Image AttractivenessFeatures for Image Attractiveness
Experiment DataExperiment Data
Experiment ResultsExperiment Results
ConclusionConclusion
![Page 3: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/3.jpg)
33
OutlineOutline
IntroductionIntroduction
Features for Image AttractivenessFeatures for Image Attractiveness
Experiment SetupExperiment Setup
Experiment ResultsExperiment Results
ConclusionConclusion
![Page 4: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/4.jpg)
44
IntroductionIntroduction
Thousands of new photos are uploaded to Thousands of new photos are uploaded to Flickr every minute.Flickr every minute.– Effective automatic content filtering is necessary.Effective automatic content filtering is necessary.
Meta data for Flickr photosMeta data for Flickr photos– TagsTags
– Number of viewsNumber of views
– User commentsUser comments
– Upload dateUpload date
– Save as favoriteSave as favorite
![Page 5: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/5.jpg)
55
Attractive or Not?Attractive or Not?
Attractiveness of imagesAttractiveness of images– A highly subjective conceptA highly subjective concept
– Semantic aspects are associated but not crucialSemantic aspects are associated but not crucial People expression, picture composition… etcPeople expression, picture composition… etc
– The artistic component is an important factorThe artistic component is an important factor
– Low-level features are shown to provide high Low-level features are shown to provide high correlation with the attractivenesscorrelation with the attractiveness
![Page 6: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/6.jpg)
66
ExampleExample
Figure: Attractive vs. Unattractive images. Each column Figure: Attractive vs. Unattractive images. Each column represents the same semantic concept (animal, landscape, represents the same semantic concept (animal, landscape, portrait, flower)portrait, flower)
![Page 7: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/7.jpg)
77
OutlineOutline
IntroductionIntroduction
Features for Image AttractivenessFeatures for Image Attractiveness
Experiment SetupExperiment Setup
Experiment ResultsExperiment Results
ConclusionConclusion
![Page 8: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/8.jpg)
88
Feature TypesFeature Types
Visual featuresVisual features– Color-associatedColor-associated
– Coarseness (sharpness)Coarseness (sharpness)
– Various color spaces are adoptedVarious color spaces are adopted RGB, YUV, HSL …etcRGB, YUV, HSL …etc
Text featuresText features– TagsTags
![Page 9: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/9.jpg)
99
Color Features - 1Color Features - 1
Brightness Brightness – To measure the intensity of light waveTo measure the intensity of light wave
– YUV color spaceYUV color space
SaturationSaturation– To measure the vividnessTo measure the vividness
( Illustration of YUV color space, Y=0.5)
![Page 10: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/10.jpg)
1010
Color Features - 2Color Features - 2
ColorfulnessColorfulness– To measure the difference against greyTo measure the difference against grey
![Page 11: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/11.jpg)
1111
Color Features - 3Color Features - 3
Naturalness (Naturalness ( 自然度自然度 // 真實度真實度 ))– To measure the degree of correspondence between To measure the degree of correspondence between
images and human perception of realityimages and human perception of reality– HSL (Hue-Saturation-Lightness) color spaceHSL (Hue-Saturation-Lightness) color space– Pixels with 20 L 80 and S≦ ≦Pixels with 20 L 80 and S≦ ≦ >> 0.1 are grouped into 3 sets: 0.1 are grouped into 3 sets:
‘A – Skin’, ‘B – Grass’, ‘C – Sky’‘A – Skin’, ‘B – Grass’, ‘C – Sky’
![Page 12: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/12.jpg)
1212
Color Features - 4Color Features - 4
ContrastContrast– To measure the relative variation of luminanceTo measure the relative variation of luminance
– RMS-contrastRMS-contrast
![Page 13: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/13.jpg)
1313
Coarseness FeatureCoarseness Feature
Coarseness represents the degree of detail Coarseness represents the degree of detail contained in an image.contained in an image.
The most commonly used metric: The most commonly used metric: SharpnessSharpness– Be determined as a function of its Laplacian, normalized Be determined as a function of its Laplacian, normalized
by the local average luminanceby the local average luminance
where μwhere μxyxy denotes the average luminance around pixel (x,y) denotes the average luminance around pixel (x,y)
![Page 14: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/14.jpg)
1414
Indication of TagsIndication of Tags
![Page 15: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/15.jpg)
1515
OutlineOutline
IntroductionIntroduction
Features for Image AttractivenessFeatures for Image Attractiveness
Experiment SetupExperiment Setup
Experiment ResultsExperiment Results
ConclusionConclusion
![Page 16: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/16.jpg)
1616
Experiment SetupExperiment Setup
Data from FlickrData from Flickr– 2.2M photos uploaded between June 1 and 7, 20072.2M photos uploaded between June 1 and 7, 2007– Among which 35,000 photos are with at least 2 favorite Among which 35,000 photos are with at least 2 favorite
assignmentsassignments– A random sample of 40,000 photos without any favorite A random sample of 40,000 photos without any favorite
assignment as the negative examplesassignment as the negative examples– The number of favorites are used as relevance valuesThe number of favorites are used as relevance values
Attractiveness classificationAttractiveness classification– Classifier: Support Vector Machine (SVMlight)Classifier: Support Vector Machine (SVMlight)
Ranking by attractivenessRanking by attractiveness– Regression model: Support Vector RegressionRegression model: Support Vector Regression
![Page 17: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/17.jpg)
1717
OutlineOutline
IntroductionIntroduction
Features for Image AttractivenessFeatures for Image Attractiveness
Experiment SetupExperiment Setup
Experiment ResultsExperiment Results
ConclusionConclusion
![Page 18: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/18.jpg)
1818
Classification Results - Classification Results - 11 Table: Classification results (BEP) of 500 Table: Classification results (BEP) of 500
“attractive/unattractive” training photos“attractive/unattractive” training photos
![Page 19: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/19.jpg)
1919
Classification Results - Classification Results - 22 Table: Classification results (BEP) of 8000 Table: Classification results (BEP) of 8000
“attractive/unattractive” training photos“attractive/unattractive” training photos
![Page 20: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/20.jpg)
2020
Classification Results - Classification Results - 33
Figure: Precision-recall curves for visual and textual Figure: Precision-recall curves for visual and textual dimensions and their combination (8000 training photos per dimensions and their combination (8000 training photos per class, numFav 5)≧class, numFav 5)≧
![Page 21: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/21.jpg)
2121
Ranking ResultRanking Result
Evaluated by Kendall’s Tau-bEvaluated by Kendall’s Tau-b
Table: Ranking using regressionTable: Ranking using regression
![Page 22: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/22.jpg)
2222
OutlineOutline
IntroductionIntroduction
Features for Image AttractivenessFeatures for Image Attractiveness
Experiment SetupExperiment Setup
Experiment ResultsExperiment Results
ConclusionsConclusions
![Page 23: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/23.jpg)
2323
ConclusionsConclusions
We have used favorite assignments in Flickr to We have used favorite assignments in Flickr to obtain training data for attractiveness obtain training data for attractiveness classification and ranking.classification and ranking.
The best performance is achieved by The best performance is achieved by combining tags and visual information.combining tags and visual information.
We plan to extend and generalize this work to We plan to extend and generalize this work to consider various kinds of resources such as consider various kinds of resources such as videos or text.videos or text.
![Page 24: Ranking and Classifying Attractiveness of Photos in Folksonomies Jose San Pedro and Stefan Siersdorfer University of Sheffield, L3S Research Center WWW.](https://reader036.fdocuments.us/reader036/viewer/2022062719/56649ef25503460f94c0366f/html5/thumbnails/24.jpg)
2424