Socially‐DrivenWebSitesfortheMasses

FrankUyedaDiwakerGupta,AminVahdat,GeorgeVarghese

UniversityofCalifornia,SanDiego

•  Grass‐rootscommuniDeswishtohavewebsitesthatallowthemtosubmitandflexiblysearchfordata.

•  ThesecommuniDesrequiretoolsthataresimplerthanthosecurrentlyavailable(e.g.Apache,PHP,MySQL).

•  Tobuildsuchtoolsweneedmodelsfor:–  dataobjects,pagelayout,and,mostimportantly,search.

•  WehaveinstanDatedthesemodelinaneasytouselanguagecalledGrassRoots,andcompilercalledGR.

ApplicaDonLogic(PHP,Ruby,Python)

PresentaDon(Dreamweaver)

Wikis,Blogs,CMS(WordPress,Joomla)

WebFrameworks(Zend,Rails,Django)

PresentaDon

ApplicaDonModel

ApplicaDonLogic(AutoGenerated)

ConfiguraDon

DatabaseConfiguraDon(Pre‐specified)

DatabaseConfiguraDonDatabaseConfiguraDon

(AutoGenerated)

WebApplicaDon

Database(MySQL)

Pre‐packaged Custom GrassRoots

•  MoDvaDon•  Modeling

•  Results

•  Bigsocialnetworks‐‐internaDonalphenomenon.–  NorthAmerica:Facebook(250M)–  SouthAmerica/India:Orkut(67Musers)

•  Notjust“socialnetworks”–  YouTube(videosharing),Digg(socialbookmarking)

•  Growinginterestinsmallersitesspecializedbyindustry,enterpriseandcommuniDes.– Wellpoint(insurance)–  Cisco(companyspecific)

•  Lotsofpeople!•  Example:PhysicsResearchers

– Wanttosharedatasets,taginteresDngfeatures

•  Example:DigitalArDsts– WanttosharedatavisualizaDonprograms&collaborate

•  Example:LocalParents&Baby‐sigers– WantjobposDngs,referralnetwork

•  Requiredatabase&applicaDonlogic.ThesecommuniDeslackresourcesandexperDse.

Needcheap,easy‐to‐usetools!

•  Difficulttoprototypenewideas–  Notclearwhichwebdevelopmentframeworkwillworkbest–  Requireknowledgeofdatabaseschemas,webprogramming

languages,designtechniques

•  SignificantDmeandexperDseneededtodevelopanoperaDonalsite–  Large,complexcodebase–  IntegraDonwithusermanagement,accesscontrolandweb

API’s–  Engineerforsecurityandprivacy

•  Generaltechniquestoscaleareunknown–  Performancetuningis“blackmagic”–  Hireaconsultant

Addressedinthistalk

Site Objects Search

Flickr Images Keyword,Tags,Comparison(geo‐tags)

YouTube Video Keyword,Tags

Last.fm Audio Tags,Structural

Del.icio.us URLs Tags

Digg URLs Taxonomy,Keyword

Craigslist LisDngs(Image+Text) Taxonomy,Keyword,Comparison

Wikipedia ArDcles(Text) Keyword,Structural

Facebook UserProfile(Image+Text) Structural,Tags

•  Userscreateanduploadcontent•  Contentorganized/rankedbaseduserinput•  Searchbasedon:– Associatedkeywords(e.g.,Del.icio.ustags)– StructuralrelaDonships(e.g.,friendsinFacebook)– Taxonomy/Hierarchy(e.g.,categoriesinDigg)

– Comparison/Proximity(e.g.,geo‐tagginginFlickr)

•  Model:Createanabstractmodelforcommunitydrivenwebsites.

•  Specify:AllowdeveloperstoexpressanabstractsitemodelintheGrassRootslanguage.

•  Compile:TheGrassRootscompilergenerateswebcodeandconfiguresstorage.

•  MoDvaDon•  Modeling

•  Results

•  Insight1(Layout):Pagesarecomposedofpanesthatarepopulatedbysearchresults.

•  Insight2(NavigaDon):AllnavigaDonissearch.

•  Insight3(Search):Graph‐basedsearchwithagributefilteringcoversexisDngsocialsearchmechanisms.

•  GrassRootsobjects:– High‐leveltypes(e.g.,video,image,text),

– CompositetypeslikeC‐structs– Builtinagributes:taggable,commentable

•  RelaDonshipsasGraphs– GeneralGraph(e.g.friendsinFacebook)– DirectedGraph(e.g.YouTubeSubscribers)– Hierarchy/Tree(e.g.Craigslistcategories)

Whatis?•  Communityphoto‐sharingwebsite

•  Usersassociatewitheachother•  Imagesorganizedusing:– Keywordtags– User“photosets”– Group“pools”

PictureSummaryPane

SearchPane

SetSummaryPane

StaDcPane

•  Apageiscomposedofoneormorepanes– Panesarepopulatedbyembeddedsearches

•  Pane– AregionwithinthePage– HandlestheinputandoutputofaparDculardatacollecDon,ordisplaysstaDccontent

– Definedonce,andreusedacrossmanypages– PaneaestheDcscustomizedwithCSS

User‐specifiedSearch

Pre‐specifiedSearch

•  NavigaDonassociatesclickingonadataobjecttoapageandsearchparameters.–  Pagesembedsearches.–  the“linkto”keywordprovidesparameterstosearches.

•  Syntax:[objectinpane]‐>linkto[page]([params]);

•  Example:Clickingonauser’snamedisplaysallpicturesownedbythatuser.user‐>linktoall_users_pictures(user);

Hierarchy

AgributeFilter

Ordering

SELECT<collecDon>[FROM<structuralrelaDon>][WHERE<filtercondiDon>…][ORDERBY<rankingfuncDon>…]

•  StructuralrelaDons:–  Graphs:neighbor,–  Tree:subtree,parent,children

•  FiltercondiDons:–  matches,contains,greaterthan,between,withindistance,taggedby

•  Ranking:–  CombinaDonofagributesorgraphproperDes(e.g.nodedegree)

•  MoDvaDon•  Modeling

•  Results

•  Claim:SmallchangestothespecificaDonprovideimportantfeaturesatlowcost.

•  Example:– Flickrtagsphotoswithkeywords– FacebooktagsphotoswithUsers

•  HowdowechangeourFlickrspecificaDontoincorporateUsertagging?

COMPOSITEPicture{IMAGEpic;

TEXTpic_Dtle; TEXTpic_descripDon;}(taggable,taggablebyUSER,commentable);

PAGEpic_detail(Picturep){ Detail(Picture)main:LOOKUPPicturep;}

DetailPane:Picture{ _owner‐>linktouser_profile(_owner); _tag‐>linktotag_result(_tag); _tag.USER‐>linktouser_profile(_tag.USER); "addtag"‐>linktoadd_tag(this); "addtoset"‐>linktoadd_to_set(this); "addtogroup"‐>linktoadd_to_group(this);}

UserTags

•  ImplementedinJava(~15Klinesofcode)

•  PagegeneraDoninvariouslanguages.–  CurrentlysupportsPHP

Compiler

SpecificaDon ParserAbstractModel

DatabasePlanner

DatabaseSchema

PageGenerator

ServerScripts

DatabaseWebServerHTMLPages

ImplementedFlickr‐likesiteusingRailsplug‐ins&GR•  CodeComplexity–  50linesofcodeacross19files(vs.180linesin1file)

•  Pictureretrievalthroughput:– Grassrootsgives2xmaxthroughput.

•  Grassrootsonlygeneratesnecessarycode•  Rubyhaslargecalltree.

•  Tagsearchthroughput–20mostrecent–  500:1performancedifference,favoringGrassRoots.–  SuspectpoorSQLqueries&failuretoparallelize.

•  NeedbegertoolsforconstrucDngsocialsites.•  LeveragethecommonaliDesamongsites.

•  WeprovideabstracDonstoeasedevelopment.– Pagesarecomposedofpanes– AllnavigaDonissearch– Graph‐basedsearchwithagributefiltering

•  AbstracDonsprovideflexibility&opportunityforopDmizaDon.