PG6009Unit 2: Using the Web Effectively & Evaluating Research Resources

Module Structure

Unit 1: Research Resource Discovery

Unit 2: Using the Web Effectively & Evaluating Research Resource Results

Unit 3:Tracking Down Results &

Keeping Up-to-date

Unit 4:Managing Your Information

Unit 6: Publishing / Disseminating Your Research

Unit 5: Ethics in Using Research Information

Unit 7: Using Archives for Research in Ireland

Unit 8: Using Special Collections for Research

in Ireland

Compulsory

Assessment: 5 credits

You are required to attend Units 1-6 and to submit a Short Report (1500 words) which will be marked on a pass/fail basis, to be completed by Wednesday 17th December.

 

Note: There will no percentage mark awarded.

Unit 2: Aim and Objectives

• To provide an overview of web resources for research and networking.

• To examine the importance of critical thinking skills and to consider the use of an analytical framework within which to evaluate the information you find.

Unit 2: Learning Outcomes

On completion of this module the learner should be able to:

•Apply best-practice techniques for identifying, locating and utilising key web resources in your research area.•Identify best-practices in effective networking and the effective use of web technologies.•Apply criteria for critical appraisal. •Use an analytical approach to evaluating information.•Contextualise information in your own research area.

Part 1: Using the Web More Effectively

1. Google & other search engines, portals/gateways, repositories

2. Web Technologies for Information Exchange

- RSS, social bookmarking, blogs, social networking, research profiles etc.

Part 2: Evaluating Information Sources1. Importance of evaluation

2. Critical Analysis/Appraisal

3. Analytical framework

4. Citation analysis: an introduction

Topics

Searching the Web More Effectively: Overview

Web: Indexed Web 2.18 billion pages??

Google: uses index/database of web pages compiled from sites found by it’s spider programs. Full text of sites sent to Google index. ‘Pagerank’ retrieves based on word occurrence, proximity, location on page, links to the page, traffic etc. about 100-200 ‘ingredients’. For the most part a keyword search. See here for more

Anyone can publish: quality control?

Problem: finding relevant scholarly material

Quantity and quality of information

Tips

1. Use ‘advanced search’ on Google and other search engines

2. Use Google Scholar, Google Books & ‘more’

3. Try other search engines – not just Google

4. Don’t always use search engines, go straight to good portals and sites suitable for your topic

5. Use the ‘social web/media’ to locate, bookmark and share information

6. Always evaluate your findings for quality

1. Use Advanced Search Techniques

Won’t guarantee quality , but may help control quantity

•Consider: word order, word choice, ‘stop words’, •Use Google advanced search, operators, domain search, phrase search etc. see: Inside search and Googleguide•Remember that Google personalises your search. See ‘Verbatim’ tool.

See: Google Basic Search Video and

Google Advanced Search Video on Library website here

2. Use Google Scholar/Books & ‘more’

• Works better on-campus, or from home through the Databases A-Z (log in to your Library account)

• If no full-text, check the journal portal and the Library catalogue as usual

• See advanced search and email alerts on Google Scholar

• See also: Microsoft Academic Search as a possible alternative to Google Scholar

Practice

• Look for scholarly web material on:

Religion in the lives of Irish emigrants

3. Use Other Search Engines

• Don’t use same search engine for everything - try others (examples below). See here also:

• Examples: Exalead, Ask Jeeves, Bing ,Blekko, JURN, Scirus, Microsoft Academic Search, ScientificWebPlus, iSEEK, BASE, Yippy

4. Don’t Always Use Search Engines

Save time by going straight to sites likely to be the most useful – use gateways, catalogues and repositories.

Examples: Europeana, Euscreen , Internet Archive, H-Net, DHO Discovery, HSIS, Irish Resources in the Humanities, Digital Repository of Ireland, Academicinfo.net,  Voice of the Shuttle , Library of Congress (U.S.): digital collections, EDSITEment, Archives Hub (U.K.), Irish Archives Resource, MHRA, Social Science Research Network, Infomine, projectbamboo.org , etc. etc.

•Look at subject guides

4. (contd.)

• Search catalogues from elsewhere:

COPAC,  British Library,  Library of Congress,  The European Library,  Worldcat, Digital Public Library of America, World Digital Library, Cornucopia Also: Hathi Trust Digital Library, Project Gutenberg, OAPEN, Librophile, DOAB, ETHOS, DART-Europe E-theses Portal

• Open Repositories: CORA , RIAN, OAIster, Driver, Institutional repository search U.K. [Unit 6]

• Images: Find images for your coursework & research (guide)

Practice

Take a look at the following sites:

• Worldcat• Europeana • Internet Archive (archiv.org)• H-Net• The European Library • RIAN • OAIster

See also: 100 time-saving search-engines for serious scholars

5: Use the Social Web/Social Media

• Social web: usually free, easy to use, shared interests, interactive• Includes: Social bookmarking, Blogs, Wikis, RSS feeds, Social

networking sites, Twitter, etc.

• Why use it as a researcher?

1. Find (& organise) information that you may not find through regular sources

2. Network with others who share an interest in your research area

Research Environment

Who are ‘Researchers’?Masters, doctoral, contract researchers, early career researchers,

established academic staff, senior researchers, expertsNew Review of Information Networking (2007) 13(2): 81-99

Important trends:• Multidisciplinary research: dispersed across disciplines,

geographical and other boundaries (e.g. UCC interdisciplinary institutes)

• Public & Private research: public-private partnership, knowledge transfer, technological transfer, business-led collaborative research, accountability. See: UCC Research Support Services

• Global nature of research: what’s going on elsewhere? Partnership between universities: Asia, South America

• Importance of sharing information and building effective networks

Networking (1.)

Formal and Informal: at every stage of the scholarly communication process

• Informal: colleagues, acquaintances, friends, family, social events etc.

• Formal: professional networking groups, academic community, project teams, committees, training programmes, conferences, presentations etc.

• Why network? For information gathering - your research area- your college/university/research community- your career and professional development

Also: disseminating your ideas/research

Networking (2.)

• Your network supports you:- Group projects- Staff/team conflicts- Peer issues- ‘Managing’ your boss/supervisor/director of

studies

• Sustaining your network:- Ongoing, strategic, practiced consistently- Information sharing must be mutually rewarding

• Sharing Information: formally and informally

• Social Bookmarking: store bookmarks remotely, some allow you to share with others. Examples:

General: Evernote - Delicious – Diigo - Instapaper

Academic: CiteUlike - Mendeley - Papers - Zotero

• RSS Feeds: ‘Really Simple Syndication’ . Information comes to your reader/aggregator of choice (you don’t have to go looking for it).

E.G. Netvibes, Feedly, The Old Reader, NewsBlur , Bloglines, Feedreader, Genieo, etc.

Feeds from websites, blogs, databases, journals etc.

Note: Feed reader apps for mobile devices

• Blogs: online journal, facilitates discussion

- Create: Blogger, Livejournal, Wordpress, Typepad, tumblr

- Find: Blogsearchengine

See: 38 reasons to blog about your research

• Twitter: micro-blogging; 140 characters; quick updates. ‘Twitter and me: Using Twitter as a PhD Researcher’

• Wikis: create/edit web content; many contributors, collaborative e.g. Wikipedia

- Create: Wikispaces (basic is free), Mediawiki

- Find: wiki.com, wiki spot

• Social/Academic Networking & Profiles: create a profile, join a network e.g. Facebook, Google+, LinkedIn, Academia.edu, ResearchGate, Methodspace , Academic Room, MLA Commons

See also: Google Scholar Citations, ORCID, ResearcherID

• Other resources:

- Dropbox , Google Drive, Box.com, OneDrive: upload, share docs

- Slideshare: share presentations

- Google groups: have discussions and upload documents

- Storify

etc.

Practice

Take a look at:

• Academia research networking site• ResearchGate research networking site • Impactstory.org

http://blog.impactstory.org/impact-challenge-day-1-academia-edu/

• Feeling Better Connected’: Academics’ Use of Social Media

Part 2: Overview: Evaluating Information Sources

• Critically important for 4th level research. Quality sources: fundamental to developing a review of the literature that can be used as a foundation on which new research can be built.

• A critical approach helps determine relevance and value to your field.

• Traditionally: journals, monographs, conference proceedings, primary sources. Now new information formats e.g. blogs, wikis, discussion lists, open access journals, open repositories, preprint repositories etc.

• How can you decide if material is of sufficient quality, suitable for inclusion in your literature review?

Overview: Evaluating Information Sources

Generally we think of:

• Scholarly resources: aimed at those within the field; disseminate research within that discipline. Scholarly methods, make claims that are valid and trustworthy.

• Popular resources: aimed at a wider public/mass audience; entertain, inform, promote viewpoints, sell products & services (vested interests).

So at basic level, is it: • Factual, methodical, ‘scientific’, based on clearly referenced sources

and documentation?OR

• subjective, journalistic, personal accounts/impressions, opinion?

But: even material accepted for publication not always reliable:

E.G. 1. Research of Andrew Wakefield (‘MMR controversy’)

o The Lancet 02/28/98, Vol. 351 Issue 9103, p637 o The Lancet 06/03/04, Volume 363 Issue 9411, p750 o The Lancet 06/02/10, Volume 375 Issue9713, p445

See also:o The crash and burn of an autism guruo Booster shotso Why did the Lancet take so long?

E.G. 2. Dutch Social Psychologist Diederik Stapel

Articles in: Science, European Journal of Social Psychology, Journal of Personality and Social Psychology, Journal of Experimental Social Psychology etc.

•Diederik Stapel Under Investigation by Dutch Prosecutors•Dutch University Sacks Social Psychologist Over Faked Data •Tilburg University Press release •Dutch News article•Science journal ‘expression of concern’ and retraction•Dutch Psychologist Diederik Stapel Relinquishes His Ph.D•Stapel Investigation Website

More examples:

• ‘Plastic Fantastic’• ‘Elsevier published 6 fake journals’• ‘Merchants of Doubt’ 174.95 ORES • Woo-Suk-Hwang• Joachim Boldt• Office of Research Integrity

Critical Analysis/Appraisal

• Focussed approach: process of reflective, reasonable, rational thinking to gather, interpret and evaluate information

• Adopting an analytical and reflective mind-set at every stage of the research process will help you to

- find the best sources- evaluate the information you find there- make decisions about your research methodology- discuss new findings- weigh-up evidence and form conclusions- make recommendations and contribute effectively to

your field

i.e. your literature review becomes a solid foundation on which you can build new research, ideas, theories etc.

An Analytical Framework

An analytical approach to assessing quality/value of information:

- Relevance - Authority - Method - Objectivity - Presentation - Currency of information

Apply this checklist to all information sources that you use

1. Relevance

Why you wish to include a source; appropriate in the context of your research needs? Screen content and look for general clues before deciding whether to digest or reject:

– Books: look at title, keywords, contents, index

– Journal: look at abstract, keywords, descriptors

– Websites: look at title bar, document title, links to and from the site, author

Relevance (contd.)

Focus on:

• Level: detailed/general/simple?• Emphasis: not always obvious from keywords/abstract• Geographical: what countries/regions are included?• Context of your own research: a unique insight into an

aspect of your own research? Are ideas confirmed/refuted by the research?

N.B. save time, reduce information overload

2. Authority

• Author: someone whose opinion or testimony is accepted; affiliated, qualified expert, academic credentials OR journalist/other author/commentator. Has their research been frequently cited? Is there a ‘H index’? In what publications? Peer-reviewed? Where indexed? Impact factor? (see Citation Analysis)

For books: who is the publisher? Established academic publisher within your field, university press? etc.

• Organisation: is it commercial, non-profit, government, research/educational? Vested interest? Contactable?

Authority (contd.)

Web Sites: (see also the ‘Internet Detective’)• Look at the ‘about’ or ‘who are we’ section of the site• How well established is it and when was the site last updated?• Evidence of sponsorship?• Is it clear who wrote the content and is that person contactable?• URL will often give a clue to type of organisation/country of origin:

- Commercial company? (.com or .co.uk or .biz)- A non-profit organisation? (.org) - A government body? (.gov) - A research/educational organisation? (.org or .ac.uk

or .edu)

e.g. www.moriartytribunal.com v www.moriarty-tribunal.ie

3. Method of Production & Methodology

• Type of Publication (Rem: anyone can publish on the web)

- Peer-review procedure and instructions for authorsPeer review/Refereed/Juried:process by which an academic journal passes a paper

submitted for publication to independent experts for comments on its suitability and worth; refereeing.

e.g. British Journal of Sociology Information for Authorse.g. Addiction

- Editorial board members check journal websitee.g. British Journal of Sociology , Mind

See: Google Scholar & Junk ScienceSee also: Ulrich’s Periodicals Directory

Method of Production & Methodology (contd.)

• Research Methodology:

- Validity: tests what it aims to test? claims made that are trustworthy? Applicable to the case or circumstances?

- Reliability: Extent to which a measurement made repeatedly in identical circumstances will yield concordant results, consistent. Methods appropriate in terms of sample size, use of control groups, quantitative or qualitative survey design etc. to ensure reproducibility?

4. Objectivity:

What is the objective?

To further understanding? To contribute to the field OR news, entertainment, opinion; editorial agenda/viewpoint

Sponsors? bias/vested interests? Hidden bias, whether deliberate or not (GM foods, organic foods, climate change).

Backed up with evidence? Clearly referenced in a bibliography? Has the evidence been interpreted in a balanced/unbiased manner?

How is the information presented?

- Colour & font, general appearance - Language and writing style serious/sober OR glossy appearance - Use of diagrams and images - Structure and layout; logical?- Quality of reproduction- Advertising: minimal/substantial; target audience

5. Presentation

6. Currency

How up to date is the information/research?

• The importance of currency depends on the context within which

you plan to refer to that research in your literature review.• Factors to consider include:

- Is it clear when the information was produced?

- Does the date of the information meet my requirements?

- Is it obsolete or superseded?

Scholarly Communication Process

• “Scholarly communication is the system through which research and other scholarly writings are created, evaluated for quality, disseminated to the scholarly community, and preserved for future use.” Source: ACRL (Association of College and Research Libraries)

• A Process: From funding to the eventual dissemination of research results through formal/traditional and less formal mechanisms

Scholarly Communication

• Traditional Approach: from informal to formal with communication of research occurring at every stage:

Informal: meetings, discussions, seminars, emails, blogs, social networking sites etc.

Report on-going research at conference (conference proceedings)

Publication in an academic journal, book AND/OR completion of thesis

Indexed in research databases, catalogues, repositories

• Recent changes in scholarly communication: new opportunities and challenges e.g. self-archiving, repositories, open access publishing, digital humanities, social web

Peer-review

• ‘Peer-reviewed’/‘refereed’ – academic/scholarly

‘The process by which an academic journal passes a paper submitted for publication to independent experts for comments on it’s suitability and worth; refereeing’

• Accepted/rejected: contribution to the field/new ideas, bias/conflicts of interest, suitability for journal

• Types: Double-blind, Single-bind, Open• Future: Open online peer-review? e.g. blogs, Twitter

[See: ‘Peer-review: a guide for researchers’]

Other Periodicals

(i.e. Not peer-reviewed):

1. Popular Magazines: (written by journalists/commentators)- Newspapers- Substantive news e.g. ‘The Economist’, ‘National

Geographic’, ‘New Scientist’- Other Magazines

2. Trade Journals (specific industry, enable practitioners share market and product information within an industry) e.g. ‘Macworld’, ‘Restaurant Business’, ‘Chemical Week’. ‘Beverage World’, ‘Computerworld’ etc.

Citation Analysis (Bibliometrics)

• Number of times a paper or researcher is cited by other scholars in the field; assumes influential researchers/authors and important works cited more often. Citations can be used to develop metrics such as h-index, impact factor etc.

• Where to find these metrics:1. Web of Science (Thomson Reuters) 9,000+ peer-reviewed journals2. Scopus (Elsevier) 16,000+ peer-reviewed journals; more than 4,000 international publishers; 1996 on 3. Scimago Country & Journal Rank Database journals and country-specific scientific indicators developed from data contained in ‘Scopus.’4. Google Scholar provides links to ‘cited by’ information. ‘Publish or Perish’ software can generate metrics based on this.

Example: Number of times cited

O’Callaghan, C. and Linehan, D. (2007). "Identity, politics and conflict in dockland development in Cork, Ireland: European Capital of Culture 2005." Cities 24(4): 311-323.

1. Go to Web of Knowledge2. Pick ‘Web of Science’3. Enter the Title to locate article4. Note Number of ‘Times Cited’5. Go to ‘Scopus’6. Enter the Title to locate article7. Note ‘Cited by’5. Go to Google Scholar6. Type article title in box7. Note ‘Cited by -’

N.B Number of cites can only provide an indication of ‘quality’.

Exercise

Article Title: The second generation problematic: Rethinking democracy and civil- military relations

Author(s): Cottey, A., Edmunds, T., Forster, A.

Source: ARMED FORCES & SOCIETY 29(1) Pages: 31- 56 

Published: 2002

How many times has this paper been cited in:• Web of Science?• Scopus?• Google Scholar?

Journal Impact

• Journal Impact Factor: average number of times articles from the journal published in past two years have been cited in JCR year

Check:• ‘Journal Citation Reports’ (Journal Performance

Metrics) includes impact factors. Part of Thomson Reuter’s Web of Science

• Scimago Country & Journal Rank Database: (free) journals and country scientific indicators developed from information in ‘Scopus’

Examples: Impact Factor

1. What is the impact factor of the Journal ‘Cities’?1. Go to Web of Knowledge2. Click on ‘Select a database’3. Pick ‘Journal Citation Reports’

4. Choose ‘JCR Social Science Edition’ and ‘Search for Specific Journal’5. Click on ‘submit’6. Type ‘Cities’ in box and search for ‘Full Journal Title’

Impact factor 2013 = 1.836

2. What are the highest impact journals for History?1. Go to Web of Knowledge2. Click on ‘Select a database’3. Pick ‘Journal Citation Reports’4. Choose ‘JCR Social Science Edition’ and ‘View a Group of Journals by Subject

Category’6. Click on ‘Submit’7. Scroll down to ‘History’’8. Click on ‘Submit’9. Sort by Impact factor

Journal impact Scopus and Scimago

Scimago Country & Journal Rank Database: (free) journals and country scientific indicators developed from information in ‘Scopus’ - better for humanities and social sciences

• N.B. SJR: ‘Scimago Journal Rank’ reflects prestige of source where such citations come from. Covers a 3 year citation window.

• SNIP: ‘Source normalized impact per paper’: corrects for differences in the frequency of citation across research fields- citations given a weighting based on the total number of citations in that field. N.B. Scopus only

Exercise

• What is the 2013 impact factor for the journal ‘Applied Linguistics’?

• What is the SJR (from Scimago website)?

• What Sociology journal had the highest impact factor for 2013?

• Look for the same in Scimago

H-Index

A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np-h) papers have no more than h citations each

•highest number of papers a scientist has that have each received at least that number of citations e.g. someone with h-index 50 has written 50 papers that have each had at least 50 citations.

Where to find it:– Web of Science– Scopus– Google Scholar– Publish or Perish

Other Metrics

• InCites: uses Web of Science data to generate metrics for institutions and units within those institutions - benchmarking

• Essential Science Indicators: Institution rankings and top papers derived from Wed of Science Data ‘NATL UNIV IRELAND UNIV COLL CORK’

• Webometrics: quantifying the impact of web sites

• Altmetrics: attempt to track the impact of scholarly works in the social web

• National systems to classify journals, publishers

How many times cited?

Stapel, DA. And Koomen, W (2001).

“I, we, and the effects of others on me: How self-construal level moderates social comparison effects.” Journal of Personality and Social Psychology 80(5): 766-781.

Note: Citation data is then used to create h-index for authors, impact factor of the journal, metrics for the university etc. etc.

Bibliometrics- limitations

Useful tools, but not a measure of true quality e.g.:

• current popularity of topic & availability of article/journal: increases citations / higher IF

• IF based on average over all articles: underestimates citations of top cited articles, exaggerates number of citations of the average article

• comparison of impact factors between different fields is invalid e.g. not as relevant for literature (books citing other books)

• coverage differs in each resource (Web of Science, Scopus, Google Scholar); none is complete and reliable e.g. for IF only ISI database journals used; undercounts number of citations from other journals e.g. ‘less-developed’ countries, other languages

Bibliometrics- limitations (contd.)

• Why have the papers been cited? e.g. Review articles, Wakefield• Well over 50% of papers are never cited• What about books, book chapters, conference papers, digital

humanities etc.?• Takes time for papers to collect citations, especially in some fields• H-index is a poor metric for young researchers• H-index underestimates the importance of seminal articles• Pressure on academics to publish may lead to a drop in quality?• Google metrics are not yet reliable enough in isolation

Impact Factor- limitations (contd.)

• ‘Like nuclear energy, the impact factor is a mixed blessing.’

• ‘The use of journal impacts in evaluating individuals has its inherent dangers. In an ideal world, evaluators would read each article and make personal judgments.’

Garfield, E. (2006) ‘The History and Meaning of the Impact Factor’. JAMA, 295(1): pp 90-93 [Garfield invented scholarly citation in the 1960’s]

H-index - limitations (contd.)

‘Obviously a single number can never give more than a rough approximation to an individual’s multifaceted profile, and many other factors should be considered in combination in evaluating an individual.’

Hirsch, J.E. (2005) ‘An index to quantify an individual's scientific research output’. arXiv:physics/0508025 v5 29 Sep 2006(Hirsch proposed the H-index in 2005)

Final word:

• Can’t always rely on citation data etc. • Each researcher must make their own judgment about the

‘quality’ and suitability of any article or information source. • Depends a lot on the context i.e. how the research is to be

used or referred to in your literature review. You may need to draw on material from newspapers, conferences, web material etc. with no peer review available.

• Refer to the framework/checklist outlined above and check with your supervisor when in doubt.

• Evaluation is an art; there no perfect indicator of quality. You need to look for clues, and ultimately judge on the basis of usefulness for your research question

Slides:

www.slideshare.net/rmadden1/pg6009b

Further reading:

https://delicious.com/sifunit2