1330 mon dochart2 brock

Recommendations Improve the Search Experience

Alison Brock

www.open.ac.uk/blogs/rise

http://www.open.ac.uk/blogs/rise

Outline

Open University (OU) Context

Why use activity data?

Scope of the project

What we did

Evaluation and next steps

OU context

“The search engine on the library is not very user friendly. I had to find a specific article recommended in the text and it took several attempts to locate it.”

“The search facility is poor and doesn’t find stuff that is supposed to be there”

http://www.flickr.com/photos/james_lumb/3921968993/sizes/z/in/photostream

New search system

http://www.flickr.com/photos/jiscimages/435135071/sizes/m/in/photostream/

New generation Discovery System from EBSCO (EDS)

http://www.flickr.com/photos/davepattern/5808712333/sizes/z/in/photostream/

Could we do more?

“That recommender systems can enhance the student experience in new

generation e-resource discovery services”

Recommendations Improve the Search Experience?

Do recommendations improve the search experience?

Can you use search data to make recommendations?

Are recommendations useful in Discovery systems?

http://www.flickr.com/photos/davepattern/3473326634/sizes/z/in/photostream/

JISC funded project

February – July 2011

One of eight projects [list at http://bit.ly/gwCmNS]

JISC Activity Data Programme

http://bit.ly/gwCmNS

http://www.flickr.com/photos/zerimski/5215633183/sizes/z/in/photostream/

"Every day I wake up and ask, 'how can I flow data better, manage data better, analyse data better?"

Rollin Ford, the CIO of Wal-Mart

Why activity data?

Loans Holds

Computer bookings

Library access

Typical library activity data

Loans Holds

Computer bookings

Library access

xx xx

OU Library activity data

OU Library systems environment

Scope of our project

Search interface

Algorithms & recommender code

Activity data

EZProxy

SFX

EDS

VLE

websitebookmarklet

What data is RISE using?

• Remote host • Date/Time• OUCU• Request• Status• Size of response• Referrer• User agent• Session

So what is in the EZProxy logs?

http://www.flickr.com/photos/vixon/116447718/sizes/m/in/photostream/

\"0\"|||\"137.108.143.168\"|||20110115235421|||\“nn1234\"|||\"GET http://libezproxy.open.ac.uk:80/connect?Session=st3ShtizgtrS7tU5&url=http://search.ebscohost.com/login.aspx?direct=true&site=edslive&scope=site&type=0&cli0=FT&clv0=Y&cli1=FT1&clv1=Y&authtype=ip&group=VCStud&bquery=War%20Against%20the%20Panthers HTTP/1.1\“|||302|||0|||\http://library.open.ac.uk/\|||\"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101206 Ubuntu/10.10 (maverick) Firefox/3.6.13\"|||\"t3ShtizgtrS7tU5\"



date and time



User name



Request


RISE database

RISE database

user type | course code(s)

EZProxy

CIRCE

RISE database

People on course ‘A’ viewed resource ‘B’People who looked at resource ‘C’ also

looked at resource ‘D’

What can the data tell us?

Course recommendation

Relationship recommendation

Which are the most popular resources

This resource is being used by people studying this course

ISSNs DOI

Article information

Subject terms

But what isn’t there?

http://www.flickr.com/photos/kevharb/5466661946/sizes/z/in/photostream/


Bibliographic data matching

EZProxy

CIRCE

So how do you improve your data?

EDS Crossref

MyRecommendations

http://www.google.com/ig/directory?type=gadgets&url=library.open.ac.uk/rise/google_gadget/risesearch.xml




Bibliographic data matching

Searches in RISE

EZProxy

CIRCE

RISE

So how do you improve your data?

EDS Crossref

RISE database

People on course ‘A’ viewed resource ‘B’

People who looked at resource ‘C’ also looked at resource ‘D’

People who searched for subject ‘E’ looked at resource ‘F’

What can the data tell us?

Course recommendation

Search recommendation

Relationship recommendation

People are looking at resources on this subjectSubject data

This resource is being used by people studying this course

Resource management

Getting a recommendation

Getting a recommendation

User A Module A123

Resource BRV=14

Views

User C Module A123

Resource BRV=15

Recommended Resource BRV=16

Views +1

User C Module A123

Resource BRV=17

Rate Useful +1

User C Module A123

Resource BRV=14

Rate Not Useful -2

Resource BRV=15

Views +1

Added a privacy policy to RISE, EDS and SFX interfaces

Privacy and opt-out URL

http://library.open.ac.uk/rise/?page=privacy

Data Protection and privacy



Evaluation

Not useful22%

Quite useful22%

Very useful45%

Not sure11%

Related to records you have viewed

Survey results

Survey results

Undergraduates

Like ratings and reviews from other students

‘other people’s experiences valuable’

Which module studied?

How high a mark?

Postgraduates

Citation as a recommendation

Wary of provenance

Feed to module website

Want synonyms

Trust repository

Focus groups

First impressions of recommendations (course-related)

Asked to enter a search term. Results and recommendations explored.

Asked about relevance

Asked about preference for type of recommendation

Face to face interviews

Should we have a recommender system?

“I think it would be a very good useful feature. It would be definitely very very useful” postgraduate Maths student

I have just had a go, it was good with suggested papers that I had already found (which shows potential in my view) through Google.

“I'm afraid my first reaction is to be a bit sceptical - it presumably doesn't tell you if fellow students found the information/article useful or relevant to what they were looking for. I would hate to waste time following unproductive links laid down by others who might be failing students or think that any "lazy" students might develop poor practice by relying on what others had looked at. It sounds like a good idea but I think caution needs to be exercised. ”

Recommendations usage

1 2 3 40

200

400

600

800

1000

1200

1400

1600

1800

2000

Relationship

Course

Search

Findings and lessons learnt

•Users like recommendations ‘in principle’

•Recommendations provenance

•Interest in the search tools

•EZProxy data

•Use other data sources

•Search terms

•Need more data

•License restrictions on metadata

Findings and lessons learnt

Open Data

http://www.flickr.com/photos/okfn/6262973028/sizes/z/in/photostream/

What next?

• Google Gadget search tool• Recommendations database• MACON• Learning Analytics

http://www.flickr.com/photos/shandrew/2102808886/sizes/m/in/photostream/

What next?

Blog: www.open.ac.uk/blogs/RISECode: http://code.google.com/p/rise-project/source/browse/trunk/rise/

http://www.open.ac.uk/blogs/RISE

http://code.google.com/p/rise-project/source/browse/trunk/rise/

http://code.google.com/p/rise-project/source/browse/trunk/rise/

My thanks go to Richard Nurse and Liz Mallett of the Open University Library for giving me the use of their slides on the project for this presentation.

Any questions ???

1330 mon dochart2 brock

Technology

Transcript of 1330 mon dochart2 brock