CAT 400 Undergraduate Major Project -...
Transcript of CAT 400 Undergraduate Major Project -...
![Page 1: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/1.jpg)
CAT 400 Undergraduate Major Project
Analysis Review Presentation
supervisor
Dr. Gan Keng Hoon
examiners
Associate Professor Dr. Azman Bin Samsudin,
Professor Dr. Mandava Rajeswari
prepared by
Gan Kian Min110938
![Page 2: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/2.jpg)
Expert Search - The ClawProject ID
SC141521
Category
Intelligent System
![Page 3: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/3.jpg)
Introduction
![Page 4: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/4.jpg)
Project Background
Search Engine System
Focus on CS USM Academic Staffs
Academic Information
Research Field
![Page 5: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/5.jpg)
Modules
Expert Search
The Clue
The Core
The Claw
![Page 6: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/6.jpg)
The
Claw
Database
Management
Data
Collection
Database
Design
![Page 7: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/7.jpg)
Problem Statement
Data Collection Process
• Manual work
• Time Consume
• Need Automate
System
• Data Error in
automate process
Database Design &
Management
• Need an Manual
“Environment” to
control data
• Admin System
• Manage database
![Page 8: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/8.jpg)
Proposed Solutions
Automate Data
Extraction Tools
Database Management
Admin Tools
![Page 9: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/9.jpg)
Data Sources
![Page 10: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/10.jpg)
Data Extraction Tools
![Page 11: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/11.jpg)
Database Management Admin Tools
• Data Table Form
• Bulk insert, update, delete
• XML Generator
![Page 12: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/12.jpg)
System Objective
The Claw
Of
Expert Search
Identify Suitable Information
Design Expert Search Database
Develop Automate Data
Extraction Tools
Develop Database Management
Tools
![Page 13: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/13.jpg)
Benefits & Uniqueness
Uniqueness
First Search Engine System
Focus on USM CS Staff
Up-to-date Bibliography
Information
Academic Information
Reduce Data Collection
Workload
Ease Maintain Database
System
Better environment for
future development
Benefits
![Page 14: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/14.jpg)
Related Work
![Page 15: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/15.jpg)
Related Technology
Web Search
Engine
Web Crawling
• Spider bot
• Built lists
Indexing
• Store information
• Index the built lists
Search
• Search index
• Boolean operators
![Page 16: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/16.jpg)
Existing Systems
![Page 17: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/17.jpg)
Google Scholar
![Page 18: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/18.jpg)
Search all scholarly literature from one convenient place
Explore related works, citations, authors, and publications
Locate the complete document through your library or on the web
Keep up with recent developments in any area of research
Check who's citing your publications, create a public author profile
![Page 19: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/19.jpg)
DataBase systems and Logic Programming (DBLP)
![Page 20: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/20.jpg)
Bibliographic information on major computer science publications
Mission of dblp is to support computer science researchers
Providing free access to high-quality bibliographic meta-data and links to the electronic
editions of publications
![Page 21: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/21.jpg)
Stanford xSearch
![Page 22: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/22.jpg)
Provides stanford students and researchers with a single search option for multiple
online resources.
For the initial launch, 28 article databases and ejournal/ebook platforms were
available.
xSearch used to include 170 and then was expanded to 200 resources that include a
broad array of subject areas and types of materials.
![Page 23: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/23.jpg)
System Requirement & Analysis
![Page 24: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/24.jpg)
Project Scope
The Claw
Database DesignDatabase
ManagementData Collection
Determine
Purpose
Organize
information
Design
tables
Setup
tables
relationship
Apply Normalization Rule
Interface
Design
Features
Design
Insert, Update,
Delete, Search
Table Display,
Editable from
HTML
Automate data
extraction tool
XML Automate
data extraction
tool
XML Generator
tool
Database bulk
insert tool
![Page 25: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/25.jpg)
Capabilities
Back-End System
Automate Data Extraction Tools
Database Management Admin Tool
XML Generator
![Page 26: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/26.jpg)
Limitations
Data Sources
Automate Data Extraction Tools
Change of Coding & Algorithm
One tool for one source
![Page 27: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/27.jpg)
Methodology
Agile Software Development Methodology
![Page 28: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/28.jpg)
Cases, Problems and Solutions
![Page 29: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/29.jpg)
Case I
Name Ambiguity
Bibliography
Same Author, Different Citation
Group Different Citations
Identify as Same Person
![Page 30: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/30.jpg)
![Page 31: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/31.jpg)
Algorithms
The Levenshtein Distance
Spell Check
Compare
TWO
Strings
Find
Shortest
Distance
Identify
Similar
Text
Replace
Corrected
String
![Page 32: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/32.jpg)
Case II
Abstract Sources
Different Site
IEEE, Springer, Sage
Need Different Tools for each
source
Extract Abstract
![Page 33: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/33.jpg)
DBLP Sources
XML Files
IEEE
Sage
Springer Springer Extraction Tool
Sage Extraction Tool
IEEE Extraction Tool
Abstract Extraction
Bibliography Abstract
First Phase Database
(The Claw)Processed Data
![Page 34: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/34.jpg)
Use Case
![Page 35: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/35.jpg)
Class Diagram
-profile_id (PK)
-expert_name
-title
-position
-room_no
-tel_fax
-specialization
-research_interest
-dblp_url
-lecture_course (FK)
-qualified_id (FK)
-author_id (FK)
USM Expert Profile
-bib_author_id (FK)
-author_id (FK)
-profile_id (FK)
Bib Author1 1..*
-course_code (PK)
-course_title
-unit
-syllabus
-learning_outcome
-reference
Courses
-lecture_course (PK)
-course_code (FK)
-year
-semester
Lecture
-qualification_id (PK)
-qualification_name
-qualification_level
-qualification_area
Academic Qualification
-qualified_id (PK)
-qualification_id (FK)
Qualified
-journal_id (PK)
-bib_author_id (FK)
-raw_title
-dblp_key
-xml_url
-title
-journal_ref
-pages
-year
-volume
-number
-ee_url
Journal and Article
-paper_id (PK)
-bib_author_id (FK)
-raw_title
-dblp_key
-xml_url
-title
-pages
-book_title
-crossref
-ee_url
Conference and Workshop Paper
1..*
1
1..* 1
1
1..*
1..* 1
1..* 1
1..*
1
-author_id (PK)
-author_name
-author_origin
Author
1
0..1
1
1..*
-author_id (PK)
-author_am_name
Name Ambiguity 1..*
1
![Page 36: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/36.jpg)
Sequence Diagram
![Page 37: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/37.jpg)
State Diagram
Browse
[SendLinks] [Browse Sucess]
BrowseFail
[Browse Fail]
SaveSourceCode
[Send String]
IdentifyData
[Send Data]
InsertData
[End Precess]
[End Process]
![Page 38: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/38.jpg)
Conclusion
![Page 39: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/39.jpg)
Admin User
Prepared Database for The Core Module
Future Development
Manage Back-End Tasks
![Page 40: CAT 400 Undergraduate Major Project - ir.cs.usm.myir.cs.usm.my/siir/resources/expertsearch/expertsearch_theclaw_cat400.pdf · Focus on CS USM Academic Staffs Academic Information](https://reader030.fdocuments.us/reader030/viewer/2022020115/5ce2590788c99314488bc515/html5/thumbnails/40.jpg)