Some computational and statistical aspects of data science

Objectives

​

1. Provide the student with the formal mathematical details necessary to continue the systematic study of machine learning

2. Understand the details of the most computationally efficient solutions for linear regressions (if time permits neural networks as well)

3. Motivate the study of stochastic processes and their relationship with machine learning.

4. Study in detail some stochastic algorithms and their relationship with stochastic approximation.

5. Start the study of parallel aspects of data science

Syllabus

1. Linear regressions

• Review and formal definitions

• Exact solution and statistical conditions

• Over-fit and regularity

• Matrix QR decomposition

• Two-dimensional gradient method

• General gradient method

• Computational aspects of the gradient method

2. Stochastic gradient method

• Probability Complements

• Lypschitz functions and loss functions

• General results and applications

• Statistical and computational advantages

• Relationship with stochastic approximation

3. Algorithms and stochastic approximation

• Invitation to stochastic processes

• Some martingales

• Online learning

• Polya Urns

• Lyaopunov functions

• Neural Networks

​

​

​