1CSC 8520 Spring 2010. Paula Matuszek

Kinds of Machine Learning• Machine learning techniques can be grouped

into several categories, in several ways:– What kind of input or data do they work on?

– What kind of output or result do they produce?

– What gets used as a critic?

• The typical broad categories are– supervised learning

– unsupervised learning

– reinforcement learning

2CSC 8520 Spring 2010. Paula Matuszek

Supervised Learning• In supervised learning, we provide the system with

example training data and the result we want to see from those data. – each example, or training case, consists of a set of

variables or features describing one case, including the decision that should be made

– the system builds a model from the examples and uses the model to make a decision

– the critic compares the actual decision to the desired decision

– and tweaks the model to make the actual and desired decisions more similar

3CSC 8520 Spring 2010. Paula Matuszek

Supervised Learning Examples• Learn to detect spam from example spam

and non-spam email

• Decide whether a house will sell from a list of its features

• Decide the age and gender of a skeleton

• Need more!!

4CSC 8520 Spring 2010. Paula Matuszek

Supervised Learning, continued• Most commonly used machine learning

methods are based on supervised learning.

• The success of a supervised learning system depends very heavily on the examples it is given. They must be typical or representative.

• It also depends on the data or features provided, the feature space. The feature space must reflect the domain or field.

5CSC 8520 Spring 2010. Paula Matuszek

Representative Examples• The machine learning examples must accurately

reflect the field or domain that we want to learn. The examples must be typical of the ones on which we will make decisions.

• Examples– Spam or non-spam

• Good: random sample of email from the last year

• Bad: random sample of email from the last week

• Bad: email received at a gmail account

– need some good and bad examples from various fields here...

6CSC 8520 Spring 2010. Paula Matuszek

Feature Spaces• Which features to include in the examples is a

major question in developing a supervised learning system:– They should be relevant to the decision to be made– They should be (mostly) observable for every

example– They should be as much as possible independent of

one another

7CSC 8520 Spring 2010. Paula Matuszek

Relevant Examples• We want our system to look at some features and some

decision, and find the patterns which led to the decision.• This will only work if the features we give the system

are in fact related to the decision being made.• Examples:

– To decide whether a house will sell• Probably relevant: price, square footage, age

• Probably irrelevant: name of the owner, day of the week

• Most supervised systems will accept a large number of features and successfully identify the relevant ones, but if the most relevant ones aren’t included the system cannot perform well.

8CSC 8520 Spring 2010. Paula Matuszek

Unsupervised Learning• In an unsupervised learning application, we do

not give the system any a priori decisions.

• The task instead is to find similarities among the examples given and group them

• The critic is some measure of similarity among the cases in a group compared to those in a different group

• The data we provide define the kind of similarities and groupings we will find.

• Still important to have representative examples.

9CSC 8520 Spring 2010. Paula Matuszek

Unsupervised Learning• The goal in unsupervised learning is often

focused more on discovery than on specific decisions. Some examples:– do my search results have some natural

grouping? (eg, “bank” should give results related to finance and results related to rivers)

– can I identify categories or genres of books based on what people purchase?

10

CSC 8520 Spring 2010. Paula Matuszek

Reinforcement Learning• Reinforcement learning systems learn a

series of actions or decisions, rather than a single decision, based on feedback given at the end of the series.– For instance, a system learning to play a game

will make multiple moves, but the critic gives only whether the game was won or lost.

• A reinforcement learner has a goal, and carries out trial-and-error search to find the best paths toward that goal

11

CSC 8520 Spring 2010. Paula Matuszek

Reinforcement Learning• A typical reinforcement learning system is an active

agent, interacting with its environment.

• It must balance– exploration: trying different actions and sequences of

actions to discover which ones work best

– achievement: using sequences which have worked well so far

• It must also learn successful sequences of actions in an uncertain environment

• Typical current applications are in artificial intelligence and in engineering.