ML/DL Notes

Indices and tables

• Index

• Module Index

• Search Page

• ML
  • Basic Terms
    • Excercises
  • Decision Trees
    • Decision Tree from known features
    • Ways to split
    • Pruning
      • Prepruning
      • Postpruning
  • Perceptron
    • Voted perceptron
      • Problems
    • Average weighted perceptron
    • Perceptron proof
      • Inspiration
      • Demonstrating that \(w_{k+1} * w_*\) grows at each iteration
    • Practice
  • Optimizers
    • Good old SGD
      • LR scheduling
    • Momentum
    • Adagrad
    • Adam
    • Excercises
  • Neural Nets
    • Weight initialization
    • Layer norm
  • Loss
    • Binary Cross Entropy
    • Multiclass Cross Entropy AKA negative log likelihood loss
    • Excercises
  • Regularization
    • Why?
    • L1
    • L2
  • Other Algos
    • K-means
    • K nearest neighbor
  • Imbalanced Data
    • Subsampling
    • Weighting
  • Multiclass
    • OVA - one versus all
    • AVA - all versus all
    • Excercises
• NLP
  • Embeddings
    • What
    • Why
    • How
    • Term Document Matrix
    • Word-Word Matrix
    • TF-IDF
    • Pointwise Mutual Information
    • PPMI
    • Weighted PPMI
    • Add k PPMI
    • Bias
    • Excercises
  • Neural Embeddings
    • Skip gram with negative sampling
      • Weighted unigram sampling
    • Fasttext
    • Excercises
  • Similarity Measures
    • Cosine
  • Meta
    • Zipf’s Law
    • Evaluating similarity measures
    • Inter annotator agreement
      • Basic probability
      • Cohen’s Kappa
    • Baselines for word sense prediction
    • Closed vs Open vocab
    • Perplexity
    • Meta
    • Excercises
  • N Grams
    • Smoothing
      • Laplace Smoothing
      • Recover adjusted counts
    • Backoff
      • Stupid backoff
  • Evaluating LMs
    • BLEU
  • Beam Search
• RL
  • Policy Gradients
    • What
    • Why
    • Explanation
      • Policy Gradient Derivation
      • Reward to Go
      • Reward to Go Derivation
        • Sources
      • Baselines
        • Unbiased State Value Proof
• Math
  • Moore Penrose Pseudoinverse