THE GENERALIZATION ABILITY OF ONLINE ALGORITHMS

FOR DEPENDENT DATA

ABSTRACT:

We study the generalization performance of online learning algorithms trained on samples

coming from a dependent source of data. We show that the generalization error of any stable

online algorithm concentrates around its regret—an easily computable statistic of the online

performance of the algorithm—when the underlying ergodic process is - or -mixing.

We show high probability error bounds assuming the loss function is convex, and we also

establish sharp convergence rates and deviation bounds for strongly convex losses and several

linear prediction problems such as linear and logistic regression, least-squares SVM, and

boosting on dependent data.

Our results have straightforward applications to stochastic optimization with dependent data, and

our analysis requires only martingale convergence arguments; we need not rely on more

powerful statistical tools such as empirical process theory.

ECWAY TECHNOLOGIES

IEEE PROJECTS & SOFTWARE DEVELOPMENTS

OUR OFFICES @ CHENNAI / TRICHY / KARUR / ERODE / MADURAI / SALEM / COIMBATORE

CELL: +91 98949 17187, +91 875487 2111 / 3111 / 4111 / 5111 / 6111

VISIT: www.ecwayprojects.com MAIL TO: ecwaytechnologies@gmail.com