Post on 30-May-2018
8/14/2019 JeromeDL Questionnaire
1/24
Semantic Digital Libraries Evaluation
Results of Lukasz Porwol (JeromeDL)
Pre-Evaluation Questionnaire
1. Your age
< 20
21 - 25
26 - 30
31 - 35
36 - 40
> 40
2. Your education
under-grad
BSc
MSc
PhD
Other
3. Main domain of education / research
Informatics / Computer ScienceElectronics / Telecommunications
Business / Marketing
Fine Arts / Architecture
Life Science / Social Science
Bio-Chemistry
Maths / Physics
Other
4. How often do you visit the university library? (slide the X marks on the lines below)
at leastonce aday
at leastonce aweek
at leastonce amonth
moreseldom
8/14/2019 JeromeDL Questionnaire
2/24
5. How often do you visit the university digital library? (slide the X marks on the lines below)
at leastonce aday
at leastonce aweek
at leastonce amonth
moreseldom
6. Do you know what social tagging is?
Yes
No
7. Do you use any online bookmarking system? (If yes - name up to 3 most often used.)Yes
No
8. Are you subscribed to any online social networking site? (If yes - name up to 3 most oftenused.)
Yes
No
9. Have you ever come across the term Web 2.0? Do you know what it means?
Yes
No
10. Have you ever come across the term Semantic Web? Do you know what it means?
Yes
No
Questionnaire After the Initial Task
8/14/2019 JeromeDL Questionnaire
3/24
1. What is your general first impression about the system? (slide the X marks on the linesbelow)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
2. How did you find the task of registering to the digital library? (slide the X marks on thelines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,unintuitive
intuitive, straightforward
3. How did you find the task of finding publications by students of Stefan Decker? (slide theX marks on the lines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,unintuitive intuitive, straightforward
8/14/2019 JeromeDL Questionnaire
4/24
4. How did you find the task of finding ''publications about notitio.us'' ? (slide the X markson the lines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,unintuitive
intuitive, straightforward
5. How did you find the task of bookmarking articles about notitio.us ? (slide the X marks onthe lines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,
unintuitive
intuitive, straight
forward
6. Please describe how (using which system) did you bookmark articles about notitio.us.(Please, provide a link to the bookmarking services if you used an online bookmarkingsolution, outside of the digital library.)
I used notitio.us at first, than when was searching the resourcesand when I have found article about notitio.us I have noticed that
there is embedded bookmarking system that is very usefull and I
7. Do you have any comments to this part of the evaluation?
Interface should be more intuitive and clear ...without overloadedinformation bars on the site you are watching..more graphical Iconsinstead of text
8/14/2019 JeromeDL Questionnaire
5/24
First Task
Question: What is the (autopresentative) meaning of using information technologies (IT) ineducation?
(PL: Jakie znaczenie autoprezentacyjne ma korzystanie z technologii IT w dzialaniachedukacyjnych?)
Answer:
Information technologies enables people to improve learning process thanks to fastinformation searching and access. Great impact on education improvoment has many kindof multimedia such as *.ppt presenations or just movies presenting essential informations.Thanks to IT now we can provide more custom paths of education. It also provides simpleways to present our achievements
References:
http://jeromedl.digime.name/resource/dO40R5Ox
Comment:
It was very hard to me to find any information related with this question...Every searchreturns nothing for word "autopresentative" and "autopresentation"I
Evaluation Questionnaire after the First Task
1. How did you find the task of finding information in the digital library? (slide the X markson the lines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,unintuitive
intuitive, straightforward
2. How do you find search feature in the digital library? (slide the X marks on the linesbelow)
hard to use easy to use
8/14/2019 JeromeDL Questionnaire
6/24
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
3. How do you find natural language query in JeromeDL? (slide the X marks on the linesbelow)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
4. How do you find TagsTreeMaps filtering in JeromeDL? (slide the X marks on the linesbelow)
hard to use easy to use
complex, mind boggling simple, clearly organized
8/14/2019 JeromeDL Questionnaire
7/24
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
5. How do you find Exibit filtering component in JeromeDL? (slide the X marks on the linesbelow)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
6. How do you find MBB browsing in JeromeDL? (slide the X marks on the lines below)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,
unintuitive
intuitive, straight
forward
boring interesting
8/14/2019 JeromeDL Questionnaire
8/24
ugly, unattractive attractive
useless useful, handy
7. How do you find SQE searching in JeromeDL? (slide the X marks on the lines below)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
8. Which features of JeromeDL did you like best? (select 3 at most)
Simple Search
Advanced Search
Natural Language Query Search
Filtering with TagsTreeMaps
Filtering with Exhibit
Browsing with MultiBeeBrowse
Semantic Query Expansion
9. Which features of JeromeDL did you hate most? (select 3 at most)
Simple Search
Advanced Search
8/14/2019 JeromeDL Questionnaire
9/24
Natural Language Query Search
Filtering with TagsTreeMaps
Filtering with Exhibit
Browsing with MultiBeeBrowse
Semantic Query Expansion
10. Do you have any comments to this part of the evaluation?
Evaluation Questionnaire after the Networking Task
1. How did you find the task of networking with your friends in the digital library? (slide theX marks on the lines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,unintuitive
intuitive, straightforward
2. How do you find the social networking component in JeromeDL? (slide the X marks onthe lines below)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
8/14/2019 JeromeDL Questionnaire
10/24
ugly, unattractive attractive
useless useful, handy
3. Do you have any comments to this part of the evaluation?
The social networking should be more interactive, more userfriendly, more (and intuitive) graphical icons. The socialfunctionalities should be provided (make social networking moresocial)
Second Task
Question: What (credibility) dimension are used to evaluate the credibility of theinformation on the Internet (used in the research)?(PL: Jakie skale/kryteria uzywane sa do oceny wiarygodnosci informacji Internetowej(wykorzystywane w badaniach)?)
Answer:
At first to evaluate the credibility of the iformation on the interent the same dimensions as formedia credibility were being used in the research and it was: believability, accuracy,trustworthiness, bias, and completeness of information. Then five another types ofverification strategies were used. It was accuracy, authority, objectivity, currency, andcoverage. Because perceived credibility may vary depending upon the type of informationsought and retrieved, the credibility of four types of information was assessed: news orcurrent events information, entertainment information, reference (factual) information, andcommercial or product information.
References:
http://jeromedl.digime.name/resource/RKyRXPaE
Comment:
This psychological test was much easier to me than before because i have found manyarticles for th keywords "evaluate information"
Evaluation Questionnaire after the Second Task
8/14/2019 JeromeDL Questionnaire
11/24
1. How did you find the task of finding information in the digital library? (slide the X markson the lines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,unintuitive
intuitive, straightforward
2. How do you find search feature in the digital library? (slide the X marks on the linesbelow)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
3. How do you find bookmarks sharing in JeromeDL? (slide the X marks on the lines below)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
8/14/2019 JeromeDL Questionnaire
12/24
boring interesting
ugly, unattractive attractive
useless useful, handy
4. How do you find collaborative browsing in JeromeDL? (slide the X marks on the linesbelow)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
5. How do you find annotating resources in JeromeDL? (slide the X marks on the linesbelow)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
8/14/2019 JeromeDL Questionnaire
13/24
ugly, unattractive attractive
useless useful, handy
6. How do you find ranking resources in JeromeDL? (slide the X marks on the lines below)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
7. Do you have any comments to this part of the evaluation?
When infomation searching process not iritate you, than browsingthrough digital library is a pleasure.
Third Task
Question: Which factors related to psychological cognitive processes influence thegrowth of the credibility of information from the Internet?(PL: Jakie czynniki zwiazane z psychologicznymi procesami poznawczymi wplywaja na wzrostoceny wiarygodnosci informacji internetowej?)
Answer:
To evaluate influence on credibility of the information from the internet we can use the same
8/14/2019 JeromeDL Questionnaire
14/24
factors like in the other interpersonal media. We can mark out here several groups offactors like: Source, Receiver, Message, Medium and Context. For the Source group wehave factors like: Expertise/Knowledge, Trustworthiness, Credentials, Attractivness,Simmilarity to receiver beliefs context, Likeability/Goodwill/Dynamism. Then for the Receivergroup we have factors like: Issue relevance, Motivation, Prior knowledge of the issue, Issueinvolvement, Values/beliefs/situation, Stereotypes about source or topic, Social location. Forthe Message group we have factors like: Topic/content, Internal validity/consistency,Plausibility of arguments, Supported by data or examples, Farng, Repetition, familiarity,Ordering. For the Medium group we have factors like: Organization, Usability, Presentation,Vividness. And finally for the Context group we have factors like:Distraction/"noise", Timesince message encountered. If we want our credibility of information to grow we have tomake some improvoments in presented areas to make presented factors better. For makingInternet information credibility grow we can also respond for the Internet user verificationstrategy which we can describe using questions: Check if the information is currnt? Check tosee that the information is complete an comprenhensive? Consider whether the viewsrepsesented are opinionts or facts? Seek out other sources to validate the informatio?Consider the author's goal/objectives for posting infroamtion online? Chcck to see who theauthor of the web site is? Look for n official "stamp of approval" or recommendation for
somone you know? Check to see whether the contact information for the person ororganization is provided on the sit? Verify the author's qualifications or credentials? If we fillin this expectations we will make our credibility grow. In addition, Internet information to becredible should serve as guides for the formation of appropriate Internet policy.
References:
http://jeromedl.digime.name/resource/34S3YdYbhttp://jeromedl.digime.name/resource/E37g3Undhttp://jeromedl.digime.name/resource/RKyRXPaE
Comment:
This task was pretty easy to me. I have found plenty of articles with searched subject. Thefirst article I checked was very rich (and has enough informations) I really didn't have tosearch the unswer through any other. The thing that disturb me is some problems with textinput into textarea in the middle of the sentence(letters are overwritten sometimes).
Evaluation Questionnaire after the Third Task
1. How did you find the task of finding information in the digital library? (slide the X markson the lines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,unintuitive
intuitive, straightforward
8/14/2019 JeromeDL Questionnaire
15/24
2. How do you find search feature in the digital library? (slide the X marks on the linesbelow)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
3. How do you find recommendations in resource view in JeromeDL? (slide the X marks onthe lines below)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,
unintuitive
intuitive, straight
forward
boring interesting
ugly, unattractive attractive
useless useful, handy
8/14/2019 JeromeDL Questionnaire
16/24
4. How do you find recommendations in bookmarks in JeromeDL? (slide the X marks on thelines below)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
5. Do you have any comments to this part of the evaluation?
This stage was pretty nice to me. I have found plenty of articles and
many information for searched subject. The interface works fine butshould be more userfriendly and more beautiful. I had sometechnical problems with text input into textarea whenever I wantedto insert text in the middle of the sentence or text block the letterswere overwritten sometimes.
Post-Evaluation Questionnaire
1. What is your overall impression about the DL library you used? (slide the X marks on thelines below)
hard to use easy to use
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
8/14/2019 JeromeDL Questionnaire
17/24
boring interesting
ugly, unattractive attractive
useless useful, handy
2. Which JeromeDL features did you like best? (select 3 at most)
Simple Search
Advanced SearchNatural Language Query Search
Filtering with TagsTreeMaps
Filtering with Exhibit
Browsing with MultiBeeBrowse
Semantic Query Expansion
Collaborative Browsing
Bookmarking Resources
Bookmarks Sharing (Collaborative Filtering)
Annotatin/Blogging Resources
Ranking Resources
Bookmarks Recommendations
Resource Recommendations
3. Which JeromeDL features did you like least? (select 3 at most)
Simple Search
Advanced Search
Natural Language Query Search
Filtering with TagsTreeMaps
Filtering with Exhibit
Browsing with MultiBeeBrowse
Semantic Query Expansion
Collaborative Browsing
Bookmarking Resources
8/14/2019 JeromeDL Questionnaire
18/24
Bookmarks Sharing (Collaborative Filtering)
Annotatin/Blogging Resources
Ranking Resources
Bookmarks Recommendations
Resource Recommendations
4. Which JeromeDL features did you find most useful? (select 3 at most)
Simple Search
Advanced Search
Natural Language Query Search
Filtering with TagsTreeMaps
Filtering with Exhibit
Browsing with MultiBeeBrowse
Semantic Query Expansion
Collaborative Browsing
Bookmarking Resources
Bookmarks Sharing (Collaborative Filtering)
Annotatin/Blogging Resources
Ranking Resources
Bookmarks Recommendations
Resource Recommendations
5. Which JeromeDL features did you find least useful? (select 3 at most)
Simple Search
Advanced Search
Natural Language Query Search
Filtering with TagsTreeMaps
Filtering with Exhibit
Browsing with MultiBeeBrowse
Semantic Query Expansion
Collaborative Browsing
Bookmarking Resources
Bookmarks Sharing (Collaborative Filtering)
Annotatin/Blogging Resources
8/14/2019 JeromeDL Questionnaire
19/24
Ranking Resources
Bookmarks Recommendations
Resource Recommendations
6. I think, I would perform better if only ... (complete the sentence)
If interface will be more userfriendly, intuitive and search processwill be more effective
7. What is your general opinion on semantic services in JeromeDL (TTM, MBB, Exhibit, SQE,Dynamic Collections, NLQ, recommendations) ? (slide the X marks on the lines below)
hard to use easy to use
hard to understand easy to understand
hard to execute easy to execute
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
8. What is your opinion on social services in JeromeDL (bookmarks, blog-comments,ranking, collaborative browsing) ? (slide the X marks on the lines below)
hard to use easy to use
hard to understand easy to understand
8/14/2019 JeromeDL Questionnaire
20/24
hard to execute easy to execute
complex, mind boggling simple, clearly organized
hard to master,unintuitive
intuitive, straightforward
boring interesting
ugly, unattractive attractive
useless useful, handy
9. What should be added to JeromeDL?
Some new, userfriendly interfaces should be added, especially newsearching interface. Some resources are saved as an image (evenit is still a plain text) they should be parsed somehow to enable
easy copy the text not to rewrite it what is a time wasting.
10. What could be improved in JeromeDL?
There should be more integrated and more clear userfriendly,beautiful interface with more graphics. The search process shouldbe much more effective and enable people to interact with theprocess.
11. Would you like to continue using this digital library (JeromeDL)? (Say why)
Yes
true
No
false
8/14/2019 JeromeDL Questionnaire
21/24
12. Do you have any comments to this part of the evaluation?
Evaluation is well done but it should be more concentrated onevaluation of each JeromeDL module, instead of whole JeromeDLevaluation to gain more precise information about them.
[Main Page]
8/14/2019 JeromeDL Questionnaire
22/24
Semantic Digital Libraries Evaluation
Results of Lukasz Porwol (JeromeDL)
Memory Task
Question: What (credibility) dimension are used to evaluate the credibility of theinformation on the Internet (used in the research)?(PL: Jakie skale/kryteria uzywane sa do oceny wiarygodnosci informacji Internetowej(wykorzystywane w badaniach)?)Answer:
The dimension used to evaluate the credibility was especially completeness ,believability,accuracy, trustworthiness and bias. This fission was performed by Flanagin and Metzger in2000. Also there were some other dimenssionslike : authority, objectivity, currency, andcoverage.
References:
Comment:
I still remember main subjects from the text I've read some time ago but how can iremember exact reference ? This stage of evaluation is not free of cheating. If I want to Icould search through documents I have download from JeromeDL (search through date)and don't need to have acces to JeromeDL to find an full answer to this question. Having 45min I could also make a new JeromeDL account and use it to find informations or use anyother internet source. Of course we belive that user is honest....but still it can have someinfluence on evaluation results.
Evaluation Questionnaire after the Memory Task
1. How did you find the task of recalling previously learned information? (slide the X markson the lines below)
hard to understand easy to understand
hard to execute easy to execute
hard to master,unintuitive
intuitive, straightforward
8/14/2019 JeromeDL Questionnaire
23/24
2. Which JeromeDL features did you like best? (select 3 at most)
Simple Search
Advanced SearchNatural Language Query Search
Filtering with TagsTreeMaps
Filtering with Exhibit
Browsing with MultiBeeBrowse
Semantic Query Expansion
Collaborative Browsing
Bookmarking Resources
Bookmarks Sharing (Collaborative Filtering)
Annotatin/Blogging Resources
Ranking Resources
Bookmarks Recommendations
Resource Recommendations
3. Which JeromeDL features did you like least? (select 3 at most
)Simple Search
Advanced Search
Natural Language Query Search
Filtering with TagsTreeMaps
Filtering with Exhibit
Browsing with MultiBeeBrowse
Semantic Query Expansion
Collaborative Browsing
Bookmarking Resources
Bookmarks Sharing (Collaborative Filtering)
Annotatin/Blogging Resources
Ranking Resources
Bookmarks Recommendations
Resource Recommendations
8/14/2019 JeromeDL Questionnaire
24/24
4. I think, I would perform better if only ... (complete the sentence)
5. Do you have any comments to this part of the evaluation?
[Main Page]
I had better, clear and userfr iendly interface and more effectivesearch engines.
This part was good to check how big part of informations we learnfrom the internet(using digital library) will be still in our heads aftersome perioid of time. This part is not free from cheating so it canhave some influance on overall result