Machine Learning

Mort Yao

Reading:

Christopher Bishop. Pattern Recognition and Machine Learning. (PRML)
Yaser S. Abu-Mostafa, Malik Magdon-Ismail and Hsuan-Tien Lin. Learning from Data: A Short Course.
Yevgeny Seldin. Machine Learning Lecture Notes.
Su-Yun Huang, Kuang-Yao Lee and Horng-Shing Lu. Lecture Notes: Statistical and Machine Learning.

1 Learning Theory

Preliminaries: Basic probability theory, statistics, information theory, theory of computation (NP-hard problems).

1.1 Generalization Bounds

1.2 Occam Learning

1.3 PAC Learning

1.4 Vapnik-Chervonenkis (VC) Theory

2 Supervised Learning

2.1 Classification

2.1.1 Perceptron

2.1.2 Naive Bayes Classifier

2.2 Regression

2.2.1 Simple Linear Regression: Linear Least Squares

2.2.2 Bayesian Linear Regression

2.2.3 Logistic Regression

2.2.4 Generalized Linear Model (GLM) and LASSO

2.3 k-Nearest Neighbors (\(k\)-NN)

2.4 Kernel Methods

2.4.1 Support Vector Machine (SVM)

2.5 Decision Trees and Ensembles

2.5.1 Bagging

2.5.2 Boosting

2.5.3 Random Forests

3 Unsupervised Learning

3.1 Clustering

3.1.1 \(k\)-means and \(k\)-means++

3.1.2 Mean Shift

3.1.3 Expectation–Maximization (EM)

3.2 Density Estimation

3.2.1 Kernel Density Estimation (KDE)

4 Online Learning Models

4.1 Statistical Learning Model

4.2 Adversarial Model

5 Decomposition and Dimensionality Reduction Methods

5.1 Singular Value Decomposition (SVD)

5.2 Principal Component Analysis (PCA)

5.3 Factor Analysis

5.4 Independent Component Analysis (ICA)

5.5 Nonlinear Dimensionality Reduction (NLDR) and Manifold Learning

6 Anomaly Detection Methods

7 Structured Prediction Methods

7.1 Bayesian Network

7.2 Hidden Markov Model (HMM)

7.3 Conditional Random Field (CRF)

7.4 Hierarchical Temporal Memory (HTM)

8 Neural Networks and Deep Learning

9 Reinforcement Learning