Skip to content

YiDa.Xu ML notes

Written by


Infinity in Deep Learning

Detailed derivation of neural networks as (1) Gaussian Process using central Limit theorem (2) Neural Tangent Kernel (NTK)

Discuss Neural ODE and in particular the use of adjoint equation in Parameter training

Sinovasinovation DeeCamp

properties of Softmax, Estimating softmax without compute denominator, Probability re-parameterization: Gumbel-Max trick and REBAR algorithm

Expectation-Maximization & Matrix Capsule Networks; Determinantal Point Process & Neural Networks compression; Kalman Filter & LSTM; Model estimation & Binary classifier

Video Tutorial to these notes

  • I recorded about 20% of these notes in videos in 2015 in Mandarin (all my notes and writings are in English) You may find them on Youtube and bilibili and Youku

3D Geometry Computer vision

  • 3D Geometry Fundamentals

    Camera Models, Intrinsic and Extrinsic parameter estimation, Epipolar Geometry, 3D reconstruction, Depth Estimation

  • Recent Deep 3D Geometry based Research

    Recent research of the following topics: Single image to Camera Model estimation, Multi-Person 3D pose estimation from multi-view, GAN-based 3D pose estimation, Deep Structure-from-Motion, Deep Learning based Depth Estimation

Deep Learning

  • New Research on Softmax function

    Out-of-distribution, Neural Network Calibration, Gumbel-Max trick, Stochastic Beams Search (some of these lectures overlap with DeeCamp2019)

Reinforcement Learning

  • Reinforcement Learning Basics

    basic knowledge in reinforcement learning, Markov Decision Process, Bellman Equation and move onto Deep Q-Learning

  • Policy Gradient

    Policy Gradient Theorem, Mathematics on Trusted Region Optimization in RL, Natural Gradients on TRPO, Proximal Policy Optimization (PPO), Conjugate Gradient Algorithm

Data Science

  • Recommendation system

    collaborative filtering, Factorization Machines, Non-Negative Matrix factorisation, Multiplicative Update Rule

Probability and Statistics Background

  • Bayesian model

    revision on Bayes model include Bayesian predictive model, conditional expectation

  • Statistics Properties

    useful statistical properties to help us prove things, include Chebyshev and Markov inequality

Probabilistic Model


  • Stochastic Matrices

    stochastic matrix, Power Method Convergence Theorem, detailed balance and PageRank algorithm

Advanced Probabilistic Model

  • Determinantal Point Process

    explain the details of DPP’s marginal distribution, L-ensemble, its sampling strategy, our work in time-varying DPP

Previous article

Machine Learning formula in LaTex

Next article

Apply for a Visa credit card

Join the discussion

Leave a Reply

Your email address will not be published. Required fields are marked *