Site Crawling: What To Do & What To Look For
-
Upload
outspoken-media -
Category
Internet
-
view
1.832 -
download
0
description
Transcript of Site Crawling: What To Do & What To Look For
![Page 1: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/1.jpg)
Site CrawlingWhat To Do & What To Look For
@katemorris
![Page 2: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/2.jpg)
@katemorris
The “Oh Shit” Moment
![Page 3: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/3.jpg)
@katemorris
The “WTF” Moment
![Page 4: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/4.jpg)
@katemorris
WHEN IS THE
LAST TIME YOU
CRAWLED
YOUR SITE?
![Page 5: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/5.jpg)
@katemorris
CRAWLING TIMELINESCRAWLING FUN
CRAWLING TOOLS CRAWLING DATA
![Page 6: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/6.jpg)
@katemorris
CRAWLING
TOOLS
![Page 7: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/7.jpg)
@katemorris
Moz
Best For: • In-House Marketers
• Beginning SEOs
Cost: $99/month+
User Score: 1-2
![Page 8: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/8.jpg)
@katemorris
Raven
Best For: • In-House Marketers
• Small Agencies
Cost: $99/month+
User Score: 3
![Page 9: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/9.jpg)
@katemorris
Xenu Link Sleuth
Best For: • Technical Experts
• Developers
Cost: Free
User Score: 4
![Page 10: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/10.jpg)
@katemorris
Screaming Frog
Best For: • Technical Marketers
• SEOs
• Developers
Cost: $160/year
User Score: 5
![Page 11: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/11.jpg)
@katemorris
Deep Crawl
Best For: • Expert SEOs
• Sites with Large
Footprints
Cost: $80/ month+
User Score: 6
![Page 12: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/12.jpg)
@katemorris
Crawler User Scale
Unrelated to SEO/Marketing
Novice Level SEO
Knowledge,
Good SEO Knowledge,
Non-Technical
Good Technical,
Minimal SEO
1 2 3 4
5 6 7
Good SEO and Technical
Knowledge
Expert Level SEO Knowledge,
Minimal Technical
Expert Technical and SEO
Knowledge
![Page 13: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/13.jpg)
@katemorris
Crawler User Scale
Unrelated to SEO/Marketing
Novice Level SEO
Knowledge,
Good SEO Knowledge,
Non-Technical
Good Technical,
Minimal SEO
Good SEO and Technical
Knowledge
Expert Level SEO Knowledge,
Minimal Technical
Expert Technical and SEO
Knowledge
BUILD YOUR OWN
![Page 14: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/14.jpg)
@katemorris
THE
CRAW
LAll Examples Use Screaming
Frog or DeepCrawl
![Page 15: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/15.jpg)
@katemorris
DeepCrawl Dashboard
![Page 16: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/16.jpg)
@katemorris
4XX/5XX Errors
![Page 17: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/17.jpg)
@katemorris
4XX/5XX Errors
![Page 18: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/18.jpg)
@katemorris
Internal Redirects
![Page 19: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/19.jpg)
@katemorris
External Redirects
![Page 20: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/20.jpg)
@katemorris
Duplicate Titles
![Page 21: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/21.jpg)
@katemorris
Duplicate Content
![Page 22: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/22.jpg)
@katemorris
URL Issues
![Page 23: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/23.jpg)
@katemorris
Meta Descriptions
![Page 24: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/24.jpg)
@katemorris
Meta Descriptions
DeepCrawl breaks it down for you.
![Page 25: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/25.jpg)
@katemorris
Canonicals
![Page 26: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/26.jpg)
@katemorris
Image Alt Text
![Page 27: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/27.jpg)
@katemorris
PDF Files
![Page 28: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/28.jpg)
@katemorris
HOW OFTEN
DO I HAVE TO
DO THIS?
![Page 29: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/29.jpg)
@katemorris
Every Quarter
![Page 30: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/30.jpg)
@katemorris
Before and After All Major Site Updates
WEBSITE UPDATE WEBSITE UPDATE WEBSITE UPDATE WEBSITE UPDATE WEBSITE UP
![Page 31: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/31.jpg)
@katemorris
Any Major Changes in Organic
Traffic
![Page 32: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/32.jpg)
@katemorris
FUN CRAWLER
USES
![Page 33: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/33.jpg)
@katemorris
Testing Sitemaps
![Page 34: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/34.jpg)
@katemorris
Testing Sitemaps
![Page 35: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/35.jpg)
@katemorris
Testing Backlinks
Issue: Backlink Analysis is time consuming,
especially checking backlink lists for live
links.
Solution: Make a text list of all the backlinks
and crawl. For Screaming Frog, set a
custom search for the client site.
![Page 36: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/36.jpg)
@katemorris
Testing Backlinks
![Page 37: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/37.jpg)
@katemorris
Testing Backlinks
![Page 38: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/38.jpg)
@katemorris
Content Inventory
![Page 39: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/39.jpg)
@katemorris
Confirm GA Implementation
![Page 40: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/40.jpg)
@katemorris
Want more?http://www.seerinteractive.com/blog/screaming-frog-guide
![Page 41: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/41.jpg)
@katemorris
Do This
Download and find your favorite crawler.
Make a calendar entry that reoccurs every three months to remind you to crawl your site.
At minimum, review:– Errors
– Redirects
– Duplicated Titles
– XML Sitemaps
![Page 42: Site Crawling: What To Do & What To Look For](https://reader033.fdocuments.us/reader033/viewer/2022060121/559441ad1a28abfe248b45e9/html5/thumbnails/42.jpg)
Thanks
Icons provided by: Medialoot