Project 1: Project 1: Classification Using Neural Networks Classification Using Neural Networks

2008. 9. 24

Kim, Kwonill

kikim@bi.snu.ac.kr Biointelligence laboratory

Artificial Intelligence

ContentsContents

Project outline Description on the data set Description on tools for ANN Guide to Writing Reports

Style Mandatory contents Optional contents

Submission guide / Marking scheme Demo

2(C) 2008, SNU Biointelligence Laboratory

3(C) 2008, SNU Biointelligence Laboratory

OutlineOutline

Goal Understand MLP deeper Practice researching and writing a paper

Handwritten digits problem (classification) To predict the classe labels (digits) of handwritten digit data set Using Multi Layer Perceptron (MLP) Estimating several statistics on the dataset

Data set Variation of the ‘Handwritten digit data set’

http://archive.ics.uci.edu/ml/datasets/Pen-Based+Recognition+of+Handwritten+Digits

Handwritten Digit Handwritten Digit Data Set (1/2)Data Set (1/2)

Description Digit database of 250 samples from 44 writers http://archive.ics.uci.edu/ml/datasets/Pen-

Based+Recognition+of+Handwritten+Digits 16 attributes

(xt, yt), t = 1, … , 8

0 ~ 100

Label (Class) 0, 1, 2, … , 9

4(C) 2008, SNU Biointelligence Laboratory

Handwritten Digit Handwritten Digit Data Set (2/2)Data Set (2/2)

Constitution Original data (./original) Preprocessed data (*.arff, *.csv) Use This!!

Total data (pendigits_total_set, 1099)= training data (pendigits_training, 749)+ test data (pendigits_test, 350)

Data description (pendigits.names) For WEKA (*.arff)

5(C) 2008, SNU Biointelligence Laboratory

6(C) 2008, SNU Biointelligence Laboratory

Tools for Experiments with ANN Tools for Experiments with ANN

Source codes - Choose anything!! Free software Weka (recommended) MATLAB tool box (Toolboxes Neural Network) ANN libraries (C, C++, JAVA, …)

Web sites http://www.cs.waikato.ac.nz/~ml/weka/ http://www.faqs.org/faqs/ai-faq/neural-nets/part5/

Reports StyleReports Style

English only!! Scientific journal-style

How to Write A Paper in Scientific Journal Style and Format http://abacus.bates.edu/~ganderso/biology/resources/writing/HTWsections.html

7(C) 2008, SNU Biointelligence Laboratory

 Experimental process  Section of Paper

What did I do in a nutshell?  Abstract

 What is the problem? Introduction

 How did I solve the problem?  Materials and Methods

 What did I find out?  Results

 What does it mean?  Discussion

 Who helped me out?  Acknowledgments (optional)

 Whose work did I refer to?  Literature Cited

 Extra Information Appendices (optional)

Report Contents – Mandatory (1/2)Report Contents – Mandatory (1/2)

System description Used software and running environments

Result graphs and tables

Analysis & discussion (Very Important!!)

8(C) 2008, SNU Biointelligence Laboratory

Report Contents – Mandatory (2/2)Report Contents – Mandatory (2/2)

Basic experiments Changing # of epochs (Draw learning curve)

Various # of Hidden Units

9(C) 2008, SNU Biointelligence Laboratory

# Hidden

Units

Train Test

Average Std. Dev.

Best Worst Average Std. Dev.

Best Worst

Setting 1 accuracy

Setting 2

Setting 3

Report Contents – OptionalReport Contents – Optional

Various experimental settings Normalization Learning rates Structure of MLP Feature selection Activation functions Learning algorithm …

Evaluation techniques ROC curve k-fold Crossvalidation …

10(C) 2008, SNU Biointelligence Laboratory

11(C) 2008, SNU Biointelligence Laboratory

Submission GuideSubmission Guide

Due date: Oct. 17 (Fri.) 15:00 Submit both ‘hardcopy’ and ‘email’

Hardcopy submission to the office (301-417 ) E-mail submission to kikim@bi.snu.ac.kr

Subject : [AI Project1 Report] Student number, Name

Length: report should be summarized within 12 pages. If you build a program by yourself, submit the source code with

comments

We are NOT interested in the accuracy and your programming skill, but your creativity and research ability.

If your major is not a C.S, team project with a C.S major student is possible (Use the class board to find your partner and notice the information of your team to TA(bhkim@bi.snu.ac.kr) by Oct. 1)

Marking SchemeMarking Scheme

20 points for experiment & analysis Extra 2 points for additional expriments

10 points for report 3 points for overall organization Late work

- 10% per one day Maximum 7 days

12(C) 2008, SNU Biointelligence Laboratory

ReferencesReferences

Materials about Weka Weka GUI guide (PPT) Explorer guide (PDF) Experimenter guide (PDF)

13(C) 2008, SNU Biointelligence Laboratory

WEKA DemoWEKA Demo

14(C) 2008, SNU Biointelligence Laboratory

MatlabMatlab

15(C) 2008, SNU Biointelligence Laboratory

QnAQnA

16(C) 2008, SNU Biointelligence Laboratory