SEO AJAX Crawlability in a Responsive Publisher World
50
AJAX in a responsive publishing world CRAWLABILITY
-
Upload
eric-wu -
Category
Technology
-
view
3.278 -
download
4
description
With the emergence of heavy javascript / AJAX heavy frameworks and the growing popularity of things like AngularJS, Ember, Backbone.js, CanJS, and even JQuery; making sites and single page apps crawlable to search engines are becoming increasingly difficult. It doesn't have to be. This presentation takes a look at some of the largest and trending publishers and some of the AJAX features they employ.
Transcript of SEO AJAX Crawlability in a Responsive Publisher World
AJAX in a responsive publishing world CRAWLABILITY
google.me/+EricWu
@eywu
linkedin.com/in/ericywu
github.com/eywu
AJAX is like violence if it doesn't solve
your problems, you’re not using
enough of it.
16ms
10kb
100ms SPEED,
PERFORMANCE,
HUMAN PERCEPTION
PAGES CRAWLED PER DAY
80% INCREASE
20,000
80,000
May 2013 – Aug 2013
80% INCREASE
ORGANIC SEARCH SESSIONS
600,000
1,200,000
May 2013 – Aug 2013
Mar 2004: “Googlebot/Test” External JS
Mar 2006: Googlebot Uses Onsite Live Chat
June 2010: Caffeine (Full Rollout)
Nov 2010: Instant Preview
May 2014: GWT Fetch & Render
May 2012: Matt PSA. Don’t Block JS & CSS
Oct 2009: AJAX Crawlability _escaped_fragment_
Nov 2007: Spider’s View on Web 2.0
May 2013: Matt Video. Googlebot & AJAX
SCROLL
http://googlewebmastercentral.blogspot.com/2014/02/infinite-scroll-search-friendly.html
THINK BASIC
PAGINATION
NAVIGATE / PAGINATE
NAVIGATE / PAGINATE
<a href="/page/2/">Load More</a>
LOAD MORE LINK
NAVIGATE / PAGINATE
<a href="/page/2/">Load More</a>
LOAD MORE LINK
NAVIGATE / PAGINATE
REL=NEXT / PREV
<link href="/page/10/" rel="prev" /> <link href="/page/11/" rel="canonical" /> <link href="/page/12/" rel="next" />
POINTER URLS*
example.com/?last-post=42
example.com/?time=1401424133
CONTINUOUS CONTENT
PUSHSTATE OR SHEBANG #! http://caniuse.com/#search=history
history.pushState( {}, titleOfPage, newURL
); history.replaceState();
AJAX GALLERIES
CONTINUOUS CONTENT
1. PUSHSTATE
2. VIEW-SOURCE
NEXT SLIDE LINKS
3. REL=NEXT/ PREV
DEFERED IMAGE LOADING
I WISH CRAWLERS SUPPORTED
REL=CANONICAL HTTP HEADERS FOR IMAGES
80% DECREASE
IMAGE SEARCH SESSIONS
400,000
800,000
Nov 2012 – Mar 2013
LAZY LOADING • 1x1 “BLANKS”
• SKELETON SCREENS
• LOW RESOLUTION Z Z
Z
RESPONSIVE IMAGES • <PICTURE>
• SRCSET
• POLYFILL
• UA DETECTION
NOSCRIPT
<img data-src='<480:small.jpg, <960:medium.jpg, >960:large.jpg' />
<noscript> <img src='full.jpg' /> </noscript>
CRAWLABLE PARTIALS
CRAWLABLE PARTIALS LYRIC SEARCH LINKS TO http://rock.rapgenius.com/1884632/Pharrell-williams-happy/ It-might-seem-crazy-what-im-bout-to-say-sunshine-shes-here-you-can-take-a-break
CRAWLABLE PARTIALS JS ENABLED REDIRECTS http://rock.rapgenius.com/Pharrell-williams-happy-lyrics#note-1884632
CRAWLABLE PARTIALS JS DISABLED RENDERS http://rock.rapgenius.com/1884632/Pharrell-williams-happy/It-might-seem-crazy-what-im-bout-to-say-sunshine-shes-here-you-can-take-a-break
4X PAGES CRAWLED PER DAY
QUADRUPALED
75,000
300,000
May 2013 – Aug 2013
+1 MILLION
ORGANIC SEARCH SESSIONS
2,000,000
4,000,000
May 2013 – Aug 2013
Technical SEO Community
HelpDeskHangouts.com
SearchSignals.com
bit.ly/ajax-crawlability
APPENDIX
ARCHITECTURE PRERENDER SERVERSIDE VS
PRERENDER
PRERENDER
PRERENDER
SEO.js
CONS PROS • SINGLE VIEW TEMPLATE
• SINGLE ROUTING
• REVERSE PROXY CACHING
• _ESCAPED_FRAGMENT_
• INTERPOLATED SOURCE
SERVER SIDE
SERVER SIDE
SERVER SIDE
SERVER SIDE
SERVER SIDE
SERVER SIDE
SERVER SIDE RENDER
CONS PROS • CLEAN URLS*
• ALL REQUESTS ARE EQUAL
• TRUE DYNAMIC RESPONSE
• 2 ROUTERS
• 2 VIEW TEMPLATES*
Quote conflated from my favorite ruby XML parser » http://nokogiri.org/ Speed, Performance, and Human Perception » https://www.youtube.com/watch?v=7ubJzEi3HuA SERoundtable Timeline Links » http://www.seroundtable.com/google-javascript-webmaster-tools-18602.html Googlebot/Test External JS » http://www.seroundtable.com/archives/000236.html Googlebot Uses Onsite Live Chat » http://www.seroundtable.com/archives/003492.html Spider’s View on Web 2.0 » http://googlewebmastercentral.blogspot.com/2007/11/spiders-view-of-web-20.html AJAX Crawlability Proposal » http://googlewebmastercentral.blogspot.com/2009/10/proposal-for-making-ajax-crawlable.html Caffine Rollout » http://googlewebmastercentral.blogspot.com/2010/06/our-new-search-index-caffeine.html Instant Previews »
http://googleblog.blogspot.com/2010/11/beyond-instant-results-instant-previews.html http://googlewebmastercentral.blogspot.com/2010/11/instant-previews.html
http://googlewebmastercentral.blogspot.com/2011/11/get-post-and-safely-surfacing-more-of.html https://sites.google.com/site/webmasterhelpforum/en/faq-instant-previews Matt Cutts PSA: Don’t Block JS & CSS » http://www.seroundtable.com/googlebot-javascript-css-14930.html Matt Video: How Does Googlebot handle content loaded via AJAX? » https://www.youtube.com/watch?v=_6mtiwQ3nvw
REFERENCES
GWT Fetch & Render » http://googlewebmastercentral.blogspot.com/2014/05/rendering-pages-with-fetch-as-google.html Google Blog: Infinite Scroll Recommendations & Example » http://googlewebmastercentral.blogspot.com/2014/02/infinite-scroll-search-friendly.html LA Times Reimagined by Code and Theory » http://www.codeandtheory.com/things-we-make/the-los-angeles-times-reimagined Google Blog: Specify your canonical » http://googlewebmastercentral.blogspot.com/2009/02/specify-your-canonical.html Google Blog: Pagination with rel=“next” and rel=“prev” » http://googlewebmastercentral.blogspot.com/2011/09/pagination-with-relnext-and-relprev.html Google Blog: Video about Pagination » http://googlewebmastercentral.blogspot.com/2012/03/video-about-pagination-with-relnext-and.html One Page Wonder: Coverage on QZ » http://www.foliomag.com/2013/one-page-wonder-infinite-scroll The Next Web Redesign Coverage » http://www.niemanlab.org/2012/10/the-next-web-redesigns-to-be-more-app-like/ The Next Web Press Release » http://thenextweb.pr.co/010a893a11df2bb61d981b2b0607c1b6784a5ab07b5ab100790b2bb3168a35f8
REFERENCES
USA Today Redesign » http://blog.f-i.com/usatoday-com-redesigning-one-of-americas-most-popular-news-site/ http://designenvy.aiga.org/usa-today-website-redesign-fantasy-interactive/ http://www.businessinsider.com/usa-todays-homepage-redesigns-2012-9
Gawker 1 Year Later Success » http://thenextweb.com/insider/2012/02/02/remember-that-gawker-redesign-a-years-worth-of-data-says-it-worked/ http://www.businessinsider.com/nick-denton-loses-bet-that-the-gawker-redesign-wouldnt-hurt-traffic-2011-10 http://www.businessinsider.com/gawker-media-traffic-numbers-2011-4 Gawker Failed Coverage » http://www.catchmyfame.com/2013/05/02/how-gawker-sabotaged-their-own-network-with-a-horrible-new-layout/ http://www.theatlantic.com/technology/archive/2011/04/gawkers-traffic-numbers-are-worse-than-anyone-anticipated/237594/ http://www.webmonkey.com/2011/02/gawker-learns-the-hard-way-why-hash-bang-urls-are-evil/ Paul Irish to Matt Cutts Video » https://www.youtube.com/watch?v=yiAF9VdvRPw Google Developer Documentation on AJAX Crawlability » https://developers.google.com/webmasters/ajax-crawling/ Browser Compatibility Chart » http://caniuse.com/#search=history Breaking The Web With Hash Bangs » http://isolani.co.uk/blog/javascript/BreakingTheWebWithHashBangs
REFERENCES
Vox Cards: Legalization of Marijuana » http://www.vox.com/cards/marijuana-legalization/learn-more-about-marijuana-legalization Bing’s Duane Forrester says still no rel=canonical in http headers » https://twitter.com/DuaneForrester/status/459387860358295552 Google Blog: A Faster Image Search » http://googlewebmastercentral.blogspot.com/2013/01/faster-image-search.html Google Says It’s Better for Webmasters » http://www.seroundtable.com/google-image-search-design-16259.html Ilya Grigork discussion around <plaintext> injection » https://plus.google.com/+IlyaGrigorik/posts/S6j45VxNESB Vox Workflow for Creating SVG Images » http://product.voxmedia.com/2013/11/25/5426880/polygon-feature-design-svg-animations-for-fun-and-profit One Solution to Responsive Images » http://www.smashingmagazine.com/2014/02/03/one-solution-to-responsive-images/ Truly Responsive Images » http://davidwalsh.name/responsive-design AngularJS NYC Meetup: Server-side Template Rendering by HBO » http://youtu.be/iB7hfvqyZpg?t=58m20s
REFERENCES
Vox Cards: Legalization of Marijuana » http://www.vox.com/cards/marijuana-legalization/learn-more-about-marijuana-legalization Bing’s Duane Forrester says still no rel=canonical in http headers » https://twitter.com/DuaneForrester/status/459387860358295552 Google Blog: A Faster Image Search » http://googlewebmastercentral.blogspot.com/2013/01/faster-image-search.html Google Says It’s Better for Webmasters » http://www.seroundtable.com/google-image-search-design-16259.html Ilya Grigork discussion around <plaintext> injection » https://plus.google.com/+IlyaGrigorik/posts/S6j45VxNESB Vox Workflow for Creating SVG Images » http://product.voxmedia.com/2013/11/25/5426880/polygon-feature-design-svg-animations-for-fun-and-profit One Solution to Responsive Images » http://www.smashingmagazine.com/2014/02/03/one-solution-to-responsive-images/ Truly Responsive Images » http://davidwalsh.name/responsive-design Serious Angular SEO » http://www.ng-newsletter.com/posts/serious-angular-seo.html AngularJS NYC Meetup: Server-side Template Rendering by HBO » http://youtu.be/iB7hfvqyZpg?t=58m20s
REFERENCES
Josh Kadis Quartz on VIP Wordpress Video » http://vip.wordpress.com/2013/09/26/josh-kadis-qz-wordpress/ https://docs.google.com/file/d/0B2Z4K6ynFLg5TVdvWVV1aTRmYUU/edit?pli=1
AirBNB: Our First Node.js App » http://nerds.airbnb.com/weve-launched-our-first-nodejs-app-to-product/ AirBNB: Rendr (Backbone in the Browser and Node) » http://nerds.airbnb.com/weve-open-sourced-rendr-run-your-backbonejs-a/ StackOverflow: PushState, Backbone, and Node » http://stackoverflow.com/questions/7098130/reusing-backbone-views-routes-on-the-server-when-using-backbone-js-pushstate-for Google: How do I create an HTML Snapshot (HIJAX) » https://developers.google.com/webmasters/ajax-crawling/docs/html-snapshot
REFERENCES
