Concept Wiki

Deep-dive reference pages for the algorithms and procedures behind the lessons. Every page comes with a runnable Colab notebook implementing the concept from scratch.

Agents

Agent Protocols: MCP & A2A

The two open protocols standardizing how agents connect to the world — MCP (Model Context Protocol) for agent-to-tool/data access and A2A (Agent2Agent) for agent-to-agent collaboration — their client/server architectures, message shapes, and how they compose.

11 minNotebook

Agentic Analysis: A Metrics Taxonomy

The four levels at which you measure an agentic system — single-agent correctness, per-agent profiling, multi-agent trajectory correctness, and whole-system health — plus the latency metrics that decide usability and the leading indicators that actually predict an effective agent.

15 minNotebook

LangGraph: Agents as State Graphs

Why LangGraph models an LLM agent as a stateful graph instead of a linear chain — the State/Node/Edge abstractions, conditional routing, cycles, and checkpointing — with a ReAct agent expressed as a compiled graph.

11 minNotebook

Observability for Agents

What changes when the traced system is an autonomous agent — trajectory traces with agent/tool/sub-agent spans, the agent-specific metrics that catch loops and budget burn, replay for debugging, and online evals over sampled trajectories.

12 minNotebook

Prompt Injection: Attack Taxonomy & Defenses

How direct and indirect prompt injection hijack LLM agents, why no filter fully closes the hole, and the defense-in-depth stack — privilege separation, provenance tagging, least privilege, and human-in-the-loop — that contains the blast radius instead.

14 minNotebook

Sub-Agent Types: Sequential, Parallel & Loop Agents

The three deterministic workflow-agent primitives — sequential pipelines, parallel fan-out/fan-in, and bounded loops — how they compose into hierarchies, their latency/cost/failure profiles, and when to use each instead of LLM-driven delegation.

12 minNotebook

The Agentic Project Loop

The spine every agent lesson hangs off — task definition → context & tools → orchestration → evaluation → guardrails → operations feedback. Learn to read any agent technique as a change to one slot, follow one docs-Q&A agent all the way around, and see exactly where the agentic loop parallels and departs from the ML loop.

15 minNotebook

Building With Llms

Calculus

Causal Inference

Computer Vision

Generative Models

Graphical Models

Linear Algebra

Ml Fundamentals

Ml In Practice

Ml Systems

Model Evaluation

Concordance: the c-statistic, Somers' D, Gini and Kendall's tau

Concordant, discordant and tied pairs from first principles: the c-statistic as AUC without a curve, Somers' D and Gini as rescalings of it, Kendall's tau-a and Goodman–Kruskal gamma as different tie conventions, the O(n log n) rank-sum shortcut, and what binning a score does to each.

14 minNotebook

Decile Analysis: Gains, Lift, KS and Rank Ordering

The rank-ordering table a model validation pack opens with: equal-count deciles, cumulative gains and capture rate, lift and cumulative lift, the KS statistic as the widest gap between the two cumulative curves, monotonicity breaks, and the Hosmer–Lemeshow goodness-of-fit test computed on the same bins.

14 minNotebook

Is That Difference Real? DeLong, McNemar and Paired Resampling

Statistical tests for model comparison on a shared test set: DeLong's test for correlated AUCs derived from placement values, McNemar's test on the discordant cells, the paired bootstrap and the exact within-subject permutation test, confidence intervals over p-values, and correcting for a leaderboard's worth of comparisons.

16 minNotebook

Platt Scaling and Isotonic Regression

The two workhorse post-hoc calibration maps: Platt scaling fits a logistic sigmoid on classifier scores (with regularized targets), while isotonic regression fits a monotonic step function via the Pool-Adjacent-Violators algorithm. Derivations, a worked PAV trace, and when each wins.

13 minNotebook

ROC Curve and AUC: Derivation and Rank-Sum Interpretation

Step-by-step ROC curve construction from sorted scores, why AUC equals the probability that a random positive ranks above a random negative (Wilcoxon-Mann-Whitney equivalence), PR-AUC for class-imbalanced data, and practical threshold selection criteria.

12 minNotebook

Neural Networks

Activation Functions: From Sigmoid to GELU

Why networks need non-linearity, the saturation and dead-neuron failure modes, and a worked comparison of sigmoid, tanh, ReLU and its variants, ELU, GELU, Swish, and softmax — with derivatives and a practical chooser.

13 minNotebook

Automatic Differentiation: How Autograd Actually Works

What a framework does between your forward pass and your gradients — the tape it records, the topological order it replays, the += that makes fan-out correct, and why activations dominate training memory.

16 minNotebook

Batch Normalization: Forward Pass & Running Stats

Normalize-then-rescale procedure with learnable γ/β, 4-element worked example (x=[2,4,6,8], μ_B=5, σ²_B=5), and exponential moving average running statistics for deterministic inference.

8 minNotebook

Dropout: Inverted Dropout, Regularization Mechanism, and MC Dropout

The inverted dropout algorithm and why it requires no test-time rescaling, the ensemble interpretation as training exponentially many subnetworks, spatial dropout for CNNs, the problematic interaction with batch normalization, and Monte Carlo dropout as approximate Bayesian inference for uncertainty quantification.

12 minNotebook

Gradient Descent Optimizers: SGD, Momentum, Adagrad, RMSprop & Adam

The update rules behind every deep-learning optimizer — plain SGD, momentum and Nesterov, the adaptive family (Adagrad, RMSprop, Adadelta), Adam with bias correction, and AdamW — plus when each one fails and how to choose.

14 minNotebook

GroupNorm & InstanceNorm: Batch-Independent Normalization

The shared normalize-then-rescale formula behind GroupNorm and InstanceNorm, with explicit reduction axes, a worked trace on a toy (N,C,H,W) tensor, and a comparison table against BatchNorm and LayerNorm.

12 minNotebook

Inductive Bias & Equivariance

The second thread that runs through every architecture: CNNs, RNNs, transformers, and GNNs are one idea — match the model's symmetry to the data's symmetry — in four instantiations. Learn equivariance, why the right bias buys sample efficiency, what breaks when it's wrong, and why attention is the 'remove the bias, inject position' move.

13 minNotebook

Learning Rate Schedules: Warmup, Decay & Restarts

Why a fixed learning rate is rarely optimal, and the schedules that fix it — step and exponential decay, cosine annealing, linear warmup, one-cycle, reduce-on-plateau, and SGDR warm restarts — with formulas and a practical chooser.

12 minNotebook

Softmax + Cross-Entropy Gradient

Softmax formula, 3-class numeric worked example, cross-entropy loss derivation, and full chain-rule proof of ∂L/∂z_k = p̂_k − 1[k=y] with finite-difference verification.

8 minNotebook

The Perceptron Learning Algorithm

Update rule derivation, OR-dataset convergence trace (2 mistakes, boundary x₁+x₂=0), XOR impossibility proof by contradiction (4 inequalities), and the hidden-layer fix with full forward-pass trace.

10 minNotebook

Nlp

BM25: Lexical Ranking

The probabilistic bag-of-words ranking function behind classic search — TF-IDF's successor with term-frequency saturation and document-length normalization, a full worked score, and where it beats (and loses to) dense embeddings.

12 minNotebook

Byte-Pair Encoding (BPE)

The subword tokenization algorithm behind GPT — iterative pair merging, a full worked trace, and a from-scratch implementation.

12 minNotebook

Dependency Parsing

Advanced

Recovering the grammatical structure of a sentence — dependency trees, transition-based and graph-based parsers, and neural approaches.

12 minNotebook

GloVe Training Procedure

How GloVe learns word vectors from global co-occurrence statistics — matrix construction, weighting function, and the least-squares objective.

10 minNotebook

HNSW: Hierarchical Navigable Small World Graphs

How the default ANN index works from the inside — skip-list layers over a proximity graph, the greedy-descent + ef beam search, the insertion algorithm with M/efConstruction, a worked search trace, complexity, and the delete/memory gotchas.

12 minNotebook

Text Generation Metrics: BLEU, ROUGE & METEOR

The classic n-gram-overlap metrics for scoring machine translation and summarization against a reference: BLEU's modified precision + brevity penalty, ROUGE's recall-oriented n-gram/LCS overlap, and METEOR's precision/recall harmonic mean with stemming and a fragmentation penalty — worked by hand on one candidate sentence.

13 minNotebook

Vector Databases & ANN Indexes

How vector databases search millions of embeddings in milliseconds — exact kNN vs the approximate-nearest-neighbour index families (IVF, HNSW, product quantization), the recall/latency trade-off, and metadata filtering.

11 minNotebook

Probabilistic Models

Probability

Beta-Bernoulli Conjugate Posterior

Five-step derivation showing how Beta×Bernoulli produces another Beta — with exponent collection, posterior recognition, and a numeric update from Beta(2,2) with 7 heads in 10 flips.

10 minNotebook

Hypothesis Testing: Null, Alternative & p-values

What the null and alternative hypotheses actually claim, how a test statistic turns data into a p-value, what that p-value does and does not mean, and how significance level α trades off Type I against Type II error.

10 minNotebook

Information Theory for ML: Entropy, KL Divergence, and Cross-Entropy

Shannon entropy as a measure of surprise and compressibility, KL divergence as the cost of using the wrong model, cross-entropy loss as negative log-likelihood under a categorical distribution, mutual information for feature selection, and how these quantities unify probabilistic ML losses.

14 minNotebook

MCMC Sampling: Metropolis–Hastings & Gibbs

Advanced

Why intractable posteriors need sampling, the Metropolis–Hastings acceptance ratio and detailed balance, Gibbs sampling from full conditionals, and the practical machinery — burn-in, thinning, acceptance-rate tuning, trace plots, R-hat and effective sample size.

14 minNotebook

MLE for the Gaussian

Eight-step derivation of the maximum-likelihood estimates for Gaussian mean and variance — differentiate, set to zero, solve, and interpret the bias.

12 minNotebook

Probability Distributions: A Field Guide & Where They're Used

A reference catalog of the distributions that show up in ML — Bernoulli, Binomial, Categorical, Poisson, Gaussian, Exponential, Gamma, Beta, Dirichlet, Laplace, Student-t and more — with parameters, mean/variance, the relationships between them, and the modeling job each one does.

14 minNotebook

Sampling Methods: Inverse Transform, Rejection & Importance Sampling

How to draw samples from a distribution you can only evaluate — inverse-transform sampling, rejection sampling and its acceptance rate, importance sampling for hard expectations, and variance-reduction tricks, with a bridge to MCMC.

13 minNotebook

The Bootstrap: Resampling for Standard Errors & Confidence Intervals

The plug-in principle, the core resample-with-replacement algorithm, percentile / basic / BCa confidence intervals, why ~63.2% of rows appear in each resample, when the bootstrap fails, and the block bootstrap for dependent data.

13 minNotebook

The Gaussian Integral (Normalization Constant)

Full 2-D polar-coordinate proof that ∫e^{−x²/2}dx = √(2π), followed by derivations of E[X]=μ and Var(X)=σ² for the Gaussian using substitution and integration by parts.

10 minNotebook

The Kolmogorov–Smirnov Test: Comparing Distributions

The empirical CDF, the supremum D statistic for one- and two-sample tests, how the KS distribution turns D into a p-value, a worked example, and when to reach for KS vs PSI vs Anderson–Darling for drift detection.

12 minNotebook

Probability Statistics

Recommender Systems

Reinforcement Learning

Sequence Models

Streaming

Supervised Learning

AdaBoost Algorithm

The original boosting algorithm — 6-step procedure, deriving the model weight α from exponential loss minimization, and a complete one-round worked trace with weight normalization.

10 minNotebook

Bias-Variance Decomposition

Full algebraic derivation of E[(y−f̂)²] = Bias² + Variance + σ², KNN at k=1 vs k=n, empirical estimation via repeated resampling, and a diagnosis table.

10 minNotebook

Decision Tree: Information Gain

Gini impurity, entropy, weighted child impurity — the complete split-scoring procedure with a full worked trace across all nine candidate thresholds.

12 minNotebook

Logistic Regression: Gradient Derivation

Five-link chain rule showing why the cross-entropy + sigmoid gradient collapses to ŷ − y — with a 2-point numeric worked step.

10 minNotebook

Minkowski Distance Family

Euclidean, Manhattan, and Minkowski distances defined and contrasted — with a full worked example computing all metrics for (1,2,3)→(4,6,3), and a table of the p-parameter's geometric effect.

8 minNotebook

OLS: The Normal Equation

A four-step algebraic derivation of w* = (XᵀX)⁻¹Xᵀy from first principles, with a complete 3-point hand-worked example.

12 minNotebook

Ridge & Lasso Regularization Paths

Step-by-step derivation of the Ridge closed-form solution, the Lasso's constant-gradient sparsity mechanism, and how coefficients evolve as λ sweeps from 0 to ∞.

12 minNotebook

VC Dimension

Capacity measured by shattering: the VC dimension of a hypothesis class, why a 2D linear classifier has VC dimension 3, the growth function and Sauer's lemma, and the generalization bound it yields.

11 minNotebook

Time Series

Transformers

Unsupervised Learning

Case Studies