from_scratch

ML/Optimization examples implemented from scratch.

This codebase is designed to build intuition behind commonly used functions in ML and optimization. Right now, the codebase includes examples of:

1. KMeans clustering
2. Principal Component Analysis (PCA): scratch method uses power iteration to compute singular values and eigenvectors
3. Linear Regression: linear regression over multiple independent variables. Parameter estimated determined via solving the normal equation, gradient descent, and stochastic (mini-batch) gradient descent
4. Solution to system of linear equations Ax=b using steepest descent and conjugate gradient methods
5. Logistic Regression
6. Softmax Classification with L2 Regularization
7. Linear SVC
8. Nonlinear SVC with polynomial and rbf kernels (dual problem solved with CVX)
9. Gradient Boost Regression Tree
10. Gaussian Process Regression
11. Bayesian Optimization