% Machine Learning
% Mort Yao
% 2017-01-19

Reading:

* Christopher Bishop. ***Pattern Recognition and Machine Learning.*** (**PRML**)
* Yaser S. Abu-Mostafa, Malik Magdon-Ismail and Hsuan-Tien Lin. ***Learning from Data: A Short Course.***
* Yevgeny Seldin. ***Machine Learning Lecture Notes.***
* Su-Yun Huang, Kuang-Yao Lee and Horng-Shing Lu. [***Lecture Notes: Statistical and Machine Learning.***](http://www.stat.sinica.edu.tw/syhuang/kern_stat_toolbox/lecture-notes.pdf)

---

# Learning Theory

Preliminaries: Basic [probability theory](/math/probability/), [statistics](/math/statistics/), [information theory](/info/), [theory of computation](/comp/) (NP-hard problems).

## Generalization Bounds

## Occam Learning

## PAC Learning

## Vapnik-Chervonenkis (VC) Theory



# Supervised Learning

## Classification

### Perceptron

### Naive Bayes Classifier

## Regression

### Simple Linear Regression: Linear Least Squares

### Bayesian Linear Regression

### Logistic Regression

### Generalized Linear Model (GLM) and LASSO

## k-Nearest Neighbors ($k$-NN)

## Kernel Methods

### Support Vector Machine (SVM)

## Decision Trees and Ensembles

### Bagging

### Boosting

### Random Forests



# Unsupervised Learning

## Clustering

### $k$-means and $k$-means++

### Mean Shift

### Expectation–Maximization (EM)

## Density Estimation

### Kernel Density Estimation (KDE)



# Online Learning Models

## Statistical Learning Model

## Adversarial Model



# Decomposition and Dimensionality Reduction Methods

## Singular Value Decomposition (SVD)

## Principal Component Analysis (PCA)

## Factor Analysis

## Independent Component Analysis (ICA)

## Nonlinear Dimensionality Reduction (NLDR) and Manifold Learning



# Anomaly Detection Methods



# Structured Prediction Methods

## Bayesian Network

## Hidden Markov Model (HMM)

## Conditional Random Field (CRF)

## Hierarchical Temporal Memory (HTM)



# Neural Networks and Deep Learning



# Reinforcement Learning