Reference · use alongside any page
Notation and glossary
Machine learning borrows symbols from statistics and calculus. This page lists every symbol used in the series, shows how to read the main equations out loud, and defines the key terms, each linked to the page where it is introduced.
Symbols
| Symbol | Say it | Meaning | Page |
|---|---|---|---|
| x | "x" | A feature, or the input to a model | 01 |
| y | "y" | The true label or target value | 01 |
| ŷ | "y-hat" | The model's prediction; the hat means "predicted" | 01 |
| X | "capital X" | The feature matrix: n rows by d columns | 02 |
| n | "n" | Number of examples (rows) | 02 |
| d | "d" | Number of features (columns) | 02 |
| xᵢ, yᵢ | "x sub i" | The i-th example and its label | 04 |
| m, b | "m", "b" | Slope and intercept of a line | 04 |
| w, wⱼ | "w sub j" | Weights: the parameters a model learns | 07 |
| θ | "theta" | All of a model's parameters together | 01 |
| f_θ(x) | "f of x with parameters theta" | The model as a function | 01 |
| L | "L" | The loss: one number for how wrong the model is | 01 |
| Σ | "sum" | Add up the terms that follow, usually over every example | 04 |
| ∂L/∂w | "partial L partial w" | How much the loss changes when w changes slightly (the gradient) | 04 |
| α or η | "alpha" or "eta" | The learning rate (step size) | 05 |
| μ, σ | "mu", "sigma" | Mean and standard deviation, used for scaling | 05 |
| λ | "lambda" | Regularization strength | 07 |
| σ(z) | "sigma of z" | The sigmoid function; context tells you it is not the standard deviation | 08 |
| z | "z" | A raw score before the activation function | 08 |
| p | "p" | A predicted probability | 08 |
| β | "beta" | Momentum factor | 05 |
| ε | "epsilon" | A small amount; on page 14, the size of an adversarial change | 14 |
| hₜ | "h sub t" | An RNN's hidden state at step t | 13 |
| Q, K, V | "queries, keys, values" | The three inputs to attention | 13 |
| Kᵀ | "K transpose" | A matrix flipped so rows become columns | 13 |
Reading the equations
Math notation is compact, not mysterious. Each equation below is followed by a plain-English sentence you could say out loud.
ŷ = m·x + b
Linear regression (page 04). Read it as: The prediction equals the slope times x, plus the intercept.
MSE = (1/n) Σ (yᵢ − ŷᵢ)²
Mean squared error (page 04). Read it as: Average the squared differences between the true and predicted values.
w ← w − α · ∂L/∂w
Gradient descent update (page 04). Read it as: Replace each weight with itself minus the learning rate times its gradient.
x_scaled = (x − μ) / σ
Standardization (page 05). Read it as: Subtract the mean, then divide by the standard deviation.
Loss = MSE + λ Σ wⱼ²
L2 regularization (page 07). Read it as: The error plus lambda times the sum of the squared weights.
σ(z) = 1 / (1 + e⁻ᶻ)
Sigmoid (page 08). Read it as: One over one plus e to the minus z; turns any score into a value between 0 and 1.
−[y log p + (1−y) log(1−p)]
Log loss (page 08). Read it as: If the label is 1, the loss is minus log p; if 0, minus log of one minus p.
precision = TP / (TP + FP)
Precision (page 09). Read it as: Of everything flagged, the share that was really positive.
recall = TP / (TP + FN)
Recall (page 09). Read it as: Of everything really positive, the share that was flagged.
hₜ = tanh(W·hₜ₋₁ + U·xₜ + b)
RNN update (page 13). Read it as: The new hidden state comes from the old state and the current input.
Glossary
| Term | Meaning | Page |
|---|---|---|
| accuracy | Share of predictions that are correct. Misleading on imbalanced data. | 09 |
| activation function | The nonlinear function a neuron applies to its weighted sum, such as ReLU or sigmoid. | 11 |
| adversarial example | An input deliberately modified so a model misclassifies it. | 14 |
| anomaly detection | Finding examples that do not fit the normal pattern. | 10 |
| attention | A mechanism that lets each position in a sequence weigh every other position. | 13 |
| backpropagation | Computing every weight's gradient by applying the chain rule backward through a network. | 11 |
| batch size | Number of examples used for one gradient step. | 05 |
| bias (statistical) | Error from a model's overly simple assumptions; high bias means underfitting. | 06 |
| bias (neuron) | The constant b added to a neuron's weighted sum. | 11 |
| classification | Predicting a category. | 08 |
| clustering | Grouping similar examples without labels. | 10 |
| confusion matrix | A table of true positives, false positives, false negatives, and true negatives. | 09 |
| convolution | Sliding a small filter across an input and recording how well each region matches. | 12 |
| cross-validation | Training and validating k times on different splits and averaging the scores. | 09 |
| data leakage | Information from outside the training data sneaking into training, inflating scores. | 02 |
| decision boundary | The line or surface where a classifier switches from one class to another. | 08 |
| deep learning | Machine learning with neural networks of many layers. | 01 |
| drift | Changes in real-world data over time that make a deployed model less accurate. | 02 |
| dropout | Randomly switching off neurons during training to reduce overfitting. | 07 |
| early stopping | Ending training when validation error starts to rise. | 07 |
| epoch | One full pass through the training data. | 05 |
| evasion | Crafting inputs at prediction time to fool a model. | 14 |
| F1 score | The harmonic mean of precision and recall. | 09 |
| feature | A measurable input column describing each example. | 02 |
| feature map | The output of a convolution: where in the input a filter's pattern appears. | 12 |
| feature scaling | Rescaling features to similar ranges so training converges faster. | 05 |
| generalization | How well a model performs on data it was not trained on. | 06 |
| gradient | The direction and rate of steepest increase of the loss; training steps the opposite way. | 04 |
| gradient descent | Repeatedly adjusting parameters against the gradient to reduce the loss. | 04 |
| hidden layer | A layer of neurons between the input and output. | 11 |
| hyperparameter | A setting chosen before training, such as the learning rate or λ. | 07 |
| inertia | In k-means, the total squared distance from points to their cluster centers. | 10 |
| inference | Using a trained model to make predictions. | 01 |
| k-means | A clustering algorithm that alternates assigning points and moving cluster centers. | 10 |
| label | The correct answer for a training example. | 02 |
| learning rate | How large a step gradient descent takes. | 05 |
| linear regression | Predicting a number with a weighted sum of features plus a bias. | 04 |
| log loss | The loss used for classification; punishes confident wrong answers. | 08 |
| logistic regression | A linear score passed through a sigmoid to classify. | 08 |
| loss function | A formula that turns a model's errors into one number to minimize. | 04 |
| LSTM | A recurrent network with gates that help it remember over long sequences. | 13 |
| membership inference | Determining whether a specific record was in a model's training data. | 14 |
| model | A function with learned parameters that maps inputs to predictions. | 01 |
| model extraction | Copying a model by training on its answers to many queries. | 14 |
| momentum | Carrying part of the previous step into the next to speed up gradient descent. | 05 |
| neuron | A unit computing a weighted sum plus bias, followed by an activation. | 11 |
| overfitting | Memorizing training data, including noise, so the model fails on new data. | 06 |
| parameter | A number inside the model that training adjusts. | 01 |
| poisoning | Corrupting training data to change what a model learns. | 14 |
| pooling | Shrinking a feature map by summarizing blocks, for example with their maximum. | 12 |
| precision | Of predicted positives, the share that are truly positive. | 09 |
| prompt injection | Hiding instructions in text an LLM reads, to hijack its behavior. | 14 |
| recall | Of true positives, the share the model caught. | 09 |
| regression | Predicting a number. | 04 |
| regularization | Penalizing model complexity, such as large weights, to reduce overfitting. | 07 |
| reinforcement learning | Learning from rewards and penalties earned by actions. | 01 |
| ReLU | The activation max(0, z), standard for hidden layers. | 11 |
| RNN | A recurrent neural network: processes sequences while carrying a hidden state forward. | 13 |
| ROC AUC | Area under the ROC curve; how well a model ranks positives above negatives. | 09 |
| sigmoid | The S-shaped function mapping any number into the range 0 to 1. | 08 |
| softmax | Turns a list of scores into probabilities that sum to 1. | 08 |
| supervised learning | Learning from examples that have labels. | 01 |
| test set | Data held back for one final, honest evaluation. | 02 |
| threshold | The probability above which a classifier predicts the positive class. | 09 |
| training set | The data a model learns its parameters from. | 02 |
| transformer | A network architecture built from attention layers; the basis of LLMs. | 13 |
| underfitting | A model too simple to capture the pattern, with high error everywhere. | 06 |
| unsupervised learning | Finding structure in data without labels. | 10 |
| validation set | Data used during development to compare models and settings. | 02 |
| vanishing gradient | Gradients shrinking toward zero as they pass back through many layers or steps. | 13 |
| variance | Error from being too sensitive to the specific training sample; high variance means overfitting. | 06 |