WebResearchCredibility Truth Trust on the Web

8/2/2019 WebResearchCredibility Truth Trust on the Web

http://slidepdf.com/reader/full/webresearchcredibility-truth-trust-on-the-web 1/78

CredibilityTruth & Trust on the Web (v6. 07/19/11)

Daniel Russell Search Researcher Natasha Bergson-Michelson, Trent Maverick Curriculum Fellows



Learning objectives

1.Discuss what makes something credible2.Learn differences between truth, credible,

authoritative, useful...

3.Learn common mistakes people make incredibility assessments4.Learn methods for assessing credibility



Big Disclaimer This is a difficult area to teach. This class is about what makes something credible (or not) WRT your belief system. If we happen mention something that you believe in —please know

that we ’re not trying discredit anything in particular. But to

teach this class effectively, we need to use examples —mostly to illustrate the points of what makes something credible. We are not attacking your beliefs. Take it as an example of the points we ’re trying to make.



To start… What’s credible to you?

Pair up

Open today’s assignment in the Web Research

folder within ANGEL. Week 5 / Lessons / WebResearch



Your side-by-side should look like this…



NOW….

1.Work your way through the pairs of web pages,showing them side-by-side (the person on the leftbrings up the A page, the person on the right brings upthe B page).

1.Look at both pages side-by-side and discuss which ofthe pair you find credible… and why.

1.As you work through the pages, fill out the form foreach pair.



First, thispage… then…

This one..Fill out the formhere, then pop backto the “CredibilityAnalysis Tasks” form and continue



Start to work.

• Start working through the examples.

• We’ll give you 15 minutes to compare 5 pairs ofpages

• TAs will help with discussion (and filling out theform)

• In 15 minutes, we’ll discuss your thoughts aboutthe pairs of pages.



Common beliefs…

• .COM is more credible than .ORG• .NET is more credible than .COM • (Or vice versa)

• Demo: Go to Godaddy.com – check for what top-level domains areavailable…



What’s credible… authoritative… informative….

• Is there a distinction between these? (consider singleassertions at the moment)

• Credibility: how much you find X believable / consistent with

your beliefs

• Authoritative: how much does this site have a degree of“official” or “formal recognition”

• Informative: does the site have a good quantity of informationabout the topic • Useful: is the information useful for the task you have



What’s Spam?

• Attempt to manipulate or alter search engines’ results rankingand relevancy by using various techniques prohibited by ourwebmaster quality guidelines.

Examples:

• Avoid hidden text or hidden links.

• Don’t do cloaking.• Don't load pages with irrelevant keywords.• Don't create pages containing a substantial

amount of unrelated links.



Google warnings

• we show this warning message for search results that we believe may havebeen hacked or otherwise compromised. If a site has been hacked, it

typically means that a third party has taken control of the site without theowner’s permission.



What’s believable? Images?

• From the Dove“Evolution

”video:http://www.youtube.com/watch?v=iYhCn0jf46U

http://www.youtube.com/watch?v=iYhCn0jf46U

http://www.youtube.com/watch?v=iYhCn0jf46U



http://area.autodesk.com/fakeorfoto/

• Which ones are computer-generated?



CG CG CG

CGCG CG



Suppose you do the following query:

Context: Your intent is to find out “all about explorers.”

[ all about explorers ]

Click on result #1



The real problem

http://xkcd.com/250/



Example: Did NASA fake the moon landing?

[ lunar landing hoax ] • Click on first result:

www.ufosaliens.co.uk/cosmicapollo.html



Credibility / Authenticity / Validation



Heuristics to check credibility

• Look at the contact info for websiteo usually on the home page, not a deep linked page

• Look at ads that run on the site o what do they tell you about the content / operator?

• Use the link operator:

[ link:www.ufos-aliens.co.uk ] to see what other web sites link to this site



Top 10 PointsAbout Credibility



1. Credibility is a Conversation

Credibility is a subjectivemeasure of many differentinterlocking factors. There is nohard and fast way to determinecredibility. Credibility changes by:- topic - time - source - user - goal



Recognizing Point-of-View

QUESTION Are GMOs unnatural and unsafe? VISIT http://bio.org/news/pressreleases/newsitem.asp?id=2010_0414_05 http://www.iowasource.com/food/2010_05_gmo.html PROBLEM Both of these articles are very scientific, but they are contradictory. CUE Read each carefully, and think about the motivation behind thewriting of each article, and what that says about the facts they

chose to offer.

http://bio.org/news/pressreleases/newsitem.asp?id=2010_0414_05


http://www.iowasource.com/food/2010_05_gmo.html

http://www.iowasource.com/food/2010_05_gmo.html





• IowaSource

• BioSource.org



2. Credibility is Not Truth

Just because a source iscredible does not mean thecontent is true. Just because asource is not credible does notmean the content is untrue. Has a doctor ever given youincorrect medical advice? Has aprofessor ever been mistaken

about recent work in the field? Even credible sources makemistakes.



Recognizing Academic Tradition

QUESTION Did Christopher Columbus really live in a time when everyone believed theworld was flat? VISIT http://books.google.com/books?id=aEr07wJ21NYC&lpg=PR3&dq=the%20discoverers&pg=PA107#v=onepage&q&f=false and search for [a flat earth returns] PROBLEM That's what I learned in school.... CUE Use Google to fact-check by searching [christopher columbus earth flat]

http://books.google.com/books?id=aEr07wJ21NYC&lpg=PR3&dq=the%20discoverers&pg=PA107






Source: Boorstin, Daniel J. The Discoverers. New York: Vintage Books, 1985.

Th T h Ch



3. The Truth Changes

What we know to be truechanges, both with time and astopics expand. What we know about the human

genome today is vastly differentfrom what we knew ten yearsago. Think of how scientific truth haschanged over the pastmillenium.

T th Ch



Truth Changes

QUESTION What kind of dinosaur was T-Rex, anyway? (carnivore? herbivore? scavenger?) VISIT http://select.nytimes.com/gst/abstract.html?res=F1061FFA3C5512738DDDA90B94DA415B868CF1D3

PROBLEM I've seen books that talk about T-Rex being the biggest predator, but also sourcessaying that it might have been a scavenger.

CUE

Check the dates on different sources, to see if some are older than others.

Look for articles spanning the history of the topic, which might call out a change inexperts' beliefs over time

http://select.nytimes.com/gst/abstract.html?res=F1061FFA3C5512738DDDA90B94DA415B868CF1D3






Source: New York Times. December 30, 1906, p.21.

Scavenger! (?)



Scavenger! (?)

• http://www.ucmp.berkeley.edu/diapsids/saurischia/tyrannosauridae.html

5 Aesthetics Matter

http://www.ucmp.berkeley.edu/diapsids/saurischia/tyrannosauridae.html

http://www.ucmp.berkeley.edu/diapsids/saurischia/tyrannosauridae.html



5. Aesthetics Matter

Like it or not, there aresubconscious influences at play.The font, layout or color schemeof a page could influencewhether you believe, trust, oruse the information presented. The Page Ranking (or positionon the Search Results Page)can also affect your impression

of a site's credibility. Strictlyspeaking, there is no connectionbetween a site's Page Rank andits credibility.

If It Is Beautiful It Must Be True



If It Is Beautiful, It Must Be True

QUESTION

Is this a reputable hospital/medical resource??

VISIT http://www.rythospital.com/2008/ PROBLEM It must be a really good company to have such a nice-looking site.

CUE Fact-check. Just carefully looking over the search results page for[ ryt hospital ] should be enough. (Note the “[Contains fictitious information”] )

Finding the Credible in the Maybe Credible



Finding the Credible in the Maybe-Credible

QUESTION Is Wikipedia right when it says that that Leo Tolstoy deeply influenced

Mahatma Gandhi? VISIT http://en.wikipedia.org/wiki/Leo_Tolstoy#Religious_and_political_beliefs

PROBLEM I've been told that Wikipedia is not always reliable, so how can I talk aboutthis cool discovery in my paper?

CUE Pull the search terms [tolstoy gandhi correspondence] and use Google BookSearch to find the full text of their letters to each other.

What Wikipedia says about Tolstoy and

http://en.wikipedia.org/wiki/Leo_Tolstoy

http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=tolstoy+gandhi+correspondence

http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=tolstoy+gandhi+correspondence

http://en.wikipedia.org/wiki/Leo_Tolstoy



Gandhi.…

…What you find when you leverage thatinformation with a Google Bookssearch.

Source: Tolstoy, Leo Nikolayevich. Correspondences of Leo Tolstoy: Easy ReadLarge Bold Edition. Readhowyouwant.com, 2008.

Seeing the Context



Seeing the Context

QUESTION Was there really a mummy smuggled on board the Titanic, upsetting ancient gods

causing the ship to sink?

VISIT http://www.catchpenny.org/titanic.html

PROBLEM See, the story is repeated right there on the page--several times in fact. It's evenindented, so that must be a quote from some kind of source!

CUE Always look carefully at the sentence right before the indented text. Here, you willsee phrases like "an old tale," and "according to one version," which imply thatthe story is not true. Learn to understand the subtleties of this language.

http://www.catchpenny.org/titanic.html




Source: http://www.catchpenny.org/titanic.html

7 There is a Continuum of Truth





7. There is a Continuum of Truth

Between truth and opinion liesinterpretation. The same event might be seendifferently by different sources. Thesame information might be presenteddifferently. The same statistical information canbe represented on different graphs tochange the reader's perception of thedata.

8. I Can Believe Multiple Things At Once



8. I Can Believe Multiple Things At Once

You can pick and choose what you willbelieve.

You can even believe different thingsfrom different sources -- even whenthose sources disagree with eachother! You can believe parts of what asource says, but not others, andcombine this with other beliefs. Theend result? Beliefs are an aggregate,pulled from multiple sources,

evaluated by you, and integrated withyour own personal beliefs andopinions.

Picking and Choosing from Multiple Sources



Picking and Choosing from Multiple Sources

QUESTION What is the best way to curl my hair with a curling iron?

VISIT [how to curl hair curling iron]

PROBLEM There are so many different ways I see to do it, can only one be right?

CUE It is okay to use your judgment to chose elements of different sources andpiece together an answer that works for you. Just as this is true in the “howto” areas of your life, it is true with academic sources.

Selective Fact Sharing—getting the full context



Se ect e act S a g gett g t e u co te t

QUESTION Was Former President Clinton the first president to stop caring about the civil

rights of Americans?

VISIT http://www.tentmaker.org/Quotes/conspiracy_%20nuts_theories.htm

PROBLEM Former President Clinton is often quoted as saying: “We can’t be so fixated onour desire to preserve the rights of ordinary Americans.” Doesn’t that mean hedid not care about our civil rights?

CUE If a quote is “always” being used to demonstrate a strong point, it is good tofollow it to its source. Try searching for the quote with the additional search terms[“full quote” OR context] to see if there is more to the story. Especially if there are… suggesting there is more context to the statement.

9. I Create My Own Truth: You Decide Credibility

http://www.tentmaker.org/Quotes/conspiracy_%20nuts_theories.htm

http://www.tentmaker.org/Quotes/conspiracy_%20nuts_theories.htm



y y

You are the final authority on what youdecide is true. Just because a source

or site is "credible" doesn't mean youhave to believe it. Just because asource or site is "incredible" doesn'tmean you can't believe it. In the end,what you believe and disbelieve is upto you.

Pay attention to your vocalization: "That's Weird!”



y y

QUESTION Is this webpage telling me something true? VISIT Query: [ endangered tree octopus ]lead you to webpage http://zapatopi.net/treeoctopus/

PROBLEM “My instinct told me that something was wrong with this site, but everythingseems so well-documented...” CUE Listen to yourself--if you say "That's weird!" you owe it to yourself to investigate

further.

http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=lunar+landing+hoax

http://zapatopi.net/treeoctopus/


http://www.google.com/search?sourceid=chrome&ie=UTF-8&q=lunar+landing+hoax



Source: http://zapatopi.net/treeoctopus/

10. I Can Take Some Things on Faith





Here's a deep idea: You can't personally verifyall the information presented to you, either onlineor offline. In order to craft your own truth, youhave to make assumptions, accept certain pointsof view, and know when it's okay to stop. At acertain point, you have to trust. Do you know exactly how gravity works?Do you rely on it?What about your cell phone?

Question: Where do you draw the line-of-faith?

When Do You Stop Tracking Facts Down?



QUESTION How does electricity work? VIEW http://www.pge.com/microsite/safety_esw_ngsw/esw/science/basics.ht

ml

PROBLEM PG&E has had some problems, and is probably just posting thisas some kind of advertising. Maybe I shouldn’t trust it…. CUE At a certain point, you just have to trust someone to tell yousome things that are correct. Electricity is PG&E’s business,and so they probably know what they are talking about.

http://www.pge.com/microsite/safety_esw_ngsw/esw/science/basics.html






Tools & StrategiesTechnical Skills for Checking Credibility

1. Thinking about Expertise and Motivation



Authorship analysis:Before deciding to trust a source, it is important tounderstand: • What makes the author an expert? • What motivates the author to write on this topic? • What motivates the site owner to publish it?

2. Work the Website for Background



• Look for an author name and bio at the top or bottomof the page

• Look for the copyright notice, indicating an author orpublisher

• Determine if any citations or links suggest the

author's bias

• Find the "About" page on the website (use Ctrl-F) • Parse the web address

o Especially prior to clicking on a page in your search

results o Visit the home page

3. Search It Out (background checks on entities)



• Search for the author, website, or sponsoring organization andread what others have to say about them o You may need to try the author's name in quotes:

e.g.: [ "daniel m russell“ ]

• Consult Wikipedia for background • Consult other resources appropriate to that person or entity

o Example: Better Business Bureau – for businesses Other articles written by author – for journalists / writers Organization relationships – for institutions

Example…



• Do the following query:

[wto slavery market africa ]

Click on the first link:



But check the title AND the URL



• Whenever the page title and URL are inconsistent,

this is a clue to dig more deeply.

Page title

URL ≠ page title!

Tracking backward to find owner



4. Third Party Tools



• http://www.easywhois.com/ to find out who owns the site • Alexa.com to learn site use / size / referral statistics

• Do a link: search to find who links to this site o e.g.: [ link:googleguide.com ] o Google and Yahoo! both have link: search

• Use LinkVendor's Outbound Links tool and keywords o http://www.linkvendor.com/seo-tools/outbound-

links.html

Use Alexa to get overall sense of site

http://www.easywhois.com/

http://www.linkvendor.com/seo-tools/outbound-links.html







http://www.easywhois.com/



www.alexa.com

www.ufos-aliens.co.uk

Use othertools!

http://www.alexa.com/






LinkVendor.com

- see: outbound links - see: keywords

Find out who owns the site…



• [ whois ufos-aliens.co.uk ] • www.networksolutions.com/whois/index.jsp

• www.internic.net/ -- look under “Whois Search Results”

Report for: ufos-

aliens.co.uk

http://whois.domaintools.com/ufos-aliens.co.uk

http://www.networksolutions.com/whois/index.jsp

http://www.internic.net/

http://www.internic.net/

http://www.networksolutions.com/whois/index.jsp






5. Thinking about Information Quality



Quality information sources will

usually have: • Correct grammar and spelling • Verifiable facts • Citations or links to information

sources

6. Strategies for Checking Data Quality

Read for comparison



• Read for comparison o Ask: "What would prove this wrong?"

• Fact check on Snopes.com, ProCon.org FactCheck.org and otherfact-check sites o Ask yourself, to what degree do you believe THESE sources?

• Fact Check on Google o "Average octopus length is 18 inches" Avoid confirmation bias

Search: [average octopus length] NOT: [average octopus length 18 inches]

• Search specifically for sources with citations o [democracy south america] vs.

[democracy south america bibliography OR cited] o BUT know most results will be at a higher reading level o AND remember that selection of sources can inform you about

author bias

More strategies for validating quality

http://snopes.com/

http://procon.org/

http://procon.org/

http://snopes.com/



• Search for background on News sites o News.Google.com for “newsy” topics and

people

• Search for scholarly articles that are on-topico Scholar.Google.com for technical articles

7. Thinking about Currency



When does currency matter when: • Studying the Civil War?• Deciding on privacy level settings for Facebook? • Choosing which version of a story to believe?

Learn to frame topics by how quickly their truthchanges, and take that into account in your researchprocess.

Tools and Strategies for Controlling for Currency



• Check for "last updated" information on pages you visit o Copyright notices may apply to a whole site, not a single page

• Read for comparison o Does the information in this source match others that you know are

up-to-date? • Use date range searching in Google's Left-Hand Panel to limit results

o Run your search o Click on "More Search Tools" in the Left-Hand Panel o Select a pre-set date range (e.g., "Past year") or enter a custom date

range o

Notice that once you set a date range, the date appears at the frontof the snippet

8. Ways to look for plagarism



• This is a perfect use case for double quote:o Look for a “low probability phrase or sentence” o That is, something that seems unique and NOT likely to be a common

phrase that would occur naturally

• Note that many low-quality sites completely copy contento Strategy: Look for duplicated content

• Note that many times an inaccurate story will be duplicatedfrom news-story to news-story.(Also true for low-quality email messages that spread rumors.)



• Example from:http://www.indiana.edu/~istd/examples.html

Commonphrase

Uncommonphrase

[ “first technology was theprimitive modes” ]

On Teaching Credibility: A Few Lesson ideas



Have students turn in an annotatedbibliography, assessing credibility ofeach item in the list, before they begin

the final writing of their paper orproject.

CRAP Test--One Great Acronym...

C



• Currency: o How recent is the information? o How recently has the website been updated? o Is it current enough for your topic?

• Reliability o What kind of information is included in the resource? o Is content of the resource primarily opinion? Is it balanced? o Does the creator provide references or sources for data or quotations?

• Authority o Who is the creator or author? o What are the credentials? o Who is the publisher or sponsor? o Are they reputable? o What is the publisher's interest (if any) in this information? o Are there advertisements on the website?

• Purpose/Point of View o Is this fact or opinion? o Is it biased? o Is the creator/author trying to sell you something?

Source: http://librascollaborate2009.pbworks.com/w/page/17069760/CRAP-Test

...Deserves Another (Lesson plan #2)

http://librascollaborate2009.pbworks.com/w/page/17069760/CRAP-Test






Activity: Break into groups of 2-3 people. Develop your own credibility acronym.

Source: Lincoln, Margaret. "Information Evaluation and OnlineCoursework." Knowledge Quest 38.3: 30-33.

Competitive analyses



• Divide up into teams for“competitive analyses

”

o On a given topic, write the report, BUT ALSOgive the credibility analysis.

Find more low credibility resources



• Here are few… Can your students find more? If so, how would they proceed?

• California Velcro crop -http://home.inreach.com/kumbach/velcro.html

• Boilerplate, the fake Victorian-era robot:http://www.bigredhair.com/boilerplate/intro.html o Key: check to see which of the books listed are real… None are.o Check also the Wikipedia article

• Collection of hoaxes:

http://www.museumofhoaxes.com/hoaxsites.html But remember--it is fun to use hoax sites to introduce the idea ofwebsite evaluation, but students need to learn to wrestle with moresubtle questions of quality, as well.

Even MORE search lessons plan / contents

http://home.inreach.com/kumbach/velcro.html

http://www.bigredhair.com/boilerplate/intro.html

http://www.museumofhoaxes.com/hoaxsites.html

http://www.museumofhoaxes.com/hoaxsites.html

http://www.bigredhair.com/boilerplate/intro.html

http://home.inreach.com/kumbach/velcro.html



https://sites.google.com/site/gwebsearcheducation/ aka…..

bit.ly/gwebsearcheducation

Summary of tools

1 Authorship analysis

https://sites.google.com/site/gwebsearcheducation/

https://sites.google.com/site/gwebsearcheducation/



1. Authorship analysis2. Work the website 3. Search it out 4. Third party tools:

1. Alexa2. link: 3. Whois 4. LinkVendor

5. Heuristics for assessing information quality1. Spelling / grammar2. Reading for comparison3. Factcheck methods

6. Currency

7. Comparison of texts across multiple resources8. Plagiarism detection9. Lesson plans



This lesson was developed by: Dan Russell Trent Maverick Tasha Bergson-Michelson

WebResearchCredibility Truth Trust on the Web

Documents

Transcript of WebResearchCredibility Truth Trust on the Web