Departamento de Electr onica, Telecomunica˘c~oes e Inform ...Departamento de Electr onica,...
Transcript of Departamento de Electr onica, Telecomunica˘c~oes e Inform ...Departamento de Electr onica,...
-
Departamento de Electrónica, Telecomunicações e Informática
EXPLORAÇÃO de DADOS & DATA MINING
Exercises: Classification Linear Classifiers and Non-Linear (SVM and NN) )
1. The following table describes a simple toy example (see lecture slides) with two classeswhere each object (instance) is described by two features.
attribute 1 attribute 2 class−1 1 1
1 −1 1−1 −1 1
1 1 −1
(a) Represent the data set in the space.
(b) The output y of a binary linear classifier is computed by
y = sign(wTx + w0)
where y is either 1 or −1 according to the argument of sign.• The problem is linearly separable? If so find one possible solution, e.g, find w
and w0 ≡ b which allows to assign the correct label to x.(c) Apply the perceptron learning rules (starting values w2 = w1 = w0 = 0 and learning
rate 0.5).
2. Three operators of RapidMiner are available to design linear classifiers described by thedecision surface: perceptron, linear regression and SVM.
• Use linear regression. Write the hyperplane equation• Use perceptron. What is the meaning of the parameters of the operator? Write the
hyperplane equation.
• Use SVM with linear configuration (kernel type: linear or dot). Write the hyperplaneequation. Give different values to parameter C.
3. The following table describes a simple toy example with two classes where each object(instance) is described by two features.
attribute 1 attribute 2 class−1 1 1
1 −1 1−1 −1 −1
1 1 −1
is it a linear problem? Try to solve this problem in RapidMiner
1
-
(a) SVM can be a linear and non-linear classifier. How is this achieved?
(b) Neural Networks are
• Linear classifiers with input layer and output layer with linear activation functions• Non-linear Classifiers with hidden layers with sigmoid activation functions and
output layer might have linear activation function.
4. Create a complete project to classify the data set Ripley-Set. Your project should consider
• First Phase: The evaluation of the classification by the use of cross-validation (X-validation operators).
• Second Phase: Comparison of two different classifiers (t-test).
(a) Use first only linear classifiers
(b) Use SVM in a non-linear configuration, e.g. ( kernel type: RBF). Note that twoparameters (C and γ) can be changed.
(c) Use a neural network classifier.
5. Classification into multi-classes: create a project in RapidMiner to classify the Iris dataset.
• Try the linear regression classification. How is the decision made?• How are SVM used in multiclass problems? What strategy is used in your RapidMiner
project.
• Use a multi-class perceptron.
6. With the operator generate data is possible to create artificial data which is
• Linearly separable data• Non-linearly separable data
Create data with two attributes (features) and apply the different classifiers. Is it possibleto illustrate the decision surfaces in RapidMiner?
Further Remarks
• In a SVM classifier what is a support vector?
• With Linear SVM it is possible ro compute the vector w while with non-linear SVM it isnot. What is the reason?
• What is the meaning of kernel tricky
• If a Neural Network (NN) has hidden layers only makes sense that its neurons have non-linear activation functions. Try to find a justification.
2
-
~8l-no.-d~ f~e.üc..\~--
L Q..\assJhccc\C'neL'~iJ::€ __a.oeLL~ .aenne.s CSvt\,o ~
\
(\~ ---
(,_1,-1
_ O',A" ~ óe ('" ''',\.H'", ( i - -I \ y T
"'\\ \ _L; ir:: L, '1 ,~s II _I,) I ~ :: O .l-() -1 ,,), :.
I \ \ I Ilo \ , -, 1 () I \ OI - , \- ) .\ \
-\
\ {\ \\0 \ \~_O j -'\ \ \ '\ lU I-( '\ - ~') ,- f- -
I.. =
-
~
;
I (-'I-I) I --I = .,,'.,..~ (r- -'{',')J -.)
I =-'J
,\
-
oi.
Wi:l -i.