Non-negative Matrix Factorization with

Sparseness Constraints

Patrik O. Hoyer

HIIT Basic Research Unit

Department of Computer Science

University of HelsinkiFinland

Topics

• Introduction

• Problem overview

• Overall methodology

• Integrating NMF with Age estimation problem

• Extended NMF

• Result found for age estimation

• Conclusion

Non-negative Matrix Factorization

• Factor A = WH• A – matrix of data• m non-negative scalar variables • n measurements form the columns of A

• W – m x r matrix of “basis vectors”• H – r x n coefficient matrix• Describes how strongly each building block is

present in measurement vectors

Non-negative Matrix Factorization

Non-negative Matrix Factorization

• The image database is regarded as an n*m matrix V,each column of which contains n non-negative pixel values of one of the m facial images

• Our purpose is to construct approximate factorizations of the form,

where each column of H contains the coefficient vector ht corresponding to the measurement vector, Vt

• Written in this form, it becomes apparent that a linear data representation is simply a factorizationof the data matrix

• Given a data matrix V, the optimal choice of matrices W and H are to be defined to be those nonnegative matrices that minimize the reconstruction error between V and WH

• Various error functions have been proposed (Paatero and Tapper, 1994; Lee and Seung, 2001), perhaps the most widely

used one is eucledian distance measure

•

.

Adding sparseness constrains to NMF

• The concept of ‘sparse coding’ refers to a representational scheme where only a few units (out of alarge population) are effectively used to represent typical data vectors

• This implies most units taking values close to zero while only few take significantly non-zero values.

• On a normalized scale, the sparsest possible vector (only a single component is non-zero) should have a sparseness of one, whereas a vector with all elements equal zero should have a sparseness of zero.

• In this paper, we use a sparseness measure based on the relationship between the L1 norm andthe L2 norm:

NMF with sparseness component

• Our aim is to constrain NMF to find solutions with desired degrees of sparseness.

• What exactly should be sparse? The basis vectors W or the coefficients H? Depends on scenario

• When trying to learn useful features from a database of images, it might make sense to require bothW and H to be sparse, pointing that any given object is present in few images and affects only asmall part of the image.

Age Estimation Based on Extended Non-negative

Matrix Factorization

Ce Zhan, Wanqing Li, and Philip OgunbonaSchool of Computer Science and Software Engineering

University of Wollongong, Australia

Proposed method : Extended NMF

• Extended NMF (ENMF) impose orthogonality constraint on basis matrix W while controlling the sparseness of coefficient matrix H.

• To reduce the overlapping between basis images, different bases should be asorthogonal as possible so as to minimize the redundancy

• Denote : U=WTW

• The orthogonality constraint can be imposed by minimizing

• Maximum sparsity in the coefficient matrix makes sure that a basis componentcannot be further decomposed into more components, thus the overlapping between basis images is further re-duced.

• We want :

Extended NMF

• The objective function of the proposed method would be:

• β is a small positive constant

ENMF is defined as following optimization problem

Experimental result

Experimental result