AI & Machine Learning Roadmap

Master data processing, modeling, and deep learning algorithms.

Overall Progress0%

Math for ML

Explanation

Machine learning is, under the hood, mathematics: data lives in vectors and matrices, models make predictions with linear transformations, and training is an optimization problem solved with gradients. The math is not a hurdle — it is the language that makes algorithms precise and predictable. This lesson builds that language in three parts. Linear algebra (vectors, matrices, matrix multiplication, linear transformations) describes how data and models are represented. Calculus (derivatives, gradients, gradient descent) describes how models are trained. Probability and statistics (distributions, expectation, variance, hypothesis testing) describe uncertainty and how we judge results. The focus is intuition and application: what each idea means, and where it shows up in a real model.

Why Mathematics Matters in ML

Data is represented as vectors and matrices.
Model predictions are computed with matrix operations.
Training is optimization via gradients (calculus).
Uncertainty and evaluation are probability and statistics.

Vectors

A vector is an ordered list of numbers — one data point or one set of features.
Example: a house with features [size, bedrooms, age] = [120, 3, 15].
Vectors have a dimension (how many numbers) and a direction/magnitude.

Matrices and Shapes

A matrix is a 2D grid of numbers: rows and columns.
Shape is written rows × columns, e.g. (3, 2) is 3 rows, 2 columns.
A dataset is a matrix: rows are samples, columns are features.

Shape	Meaning
(n, 1)	A column vector (one feature for n samples)
(1, n)	A row vector
(n, d)	A dataset of n samples with d features

Dot Products

The dot product multiplies matching entries and sums them.
It measures how aligned two vectors are.
It is the engine of matrix multiplication and of a neuron’s weighted sum.

Matrix Multiplication

Matrix multiplication combines a matrix with a vector or matrix.
The rule: to multiply (m × n) by (n × p), the inner dimensions must match; the result is (m × p).
Each output entry is a dot product of a row with a column.

Linear Transformations

A matrix represents a linear transformation: it maps input vectors to output vectors. In a neural network, each layer multiplies its input by a weight matrix — a learned linear transformation.

Input vector x

↓

Multiply by weight matrix W

↓

Output vector Wx

Derivatives and Gradients

A derivative measures the rate of change of a function — its slope at a point.
A gradient is the vector of partial derivatives, pointing in the direction of steepest increase.
To minimize, move in the direction opposite the gradient.

Partial Derivatives

When a function has many variables, a partial derivative is the slope with respect to one variable, holding others fixed.
A model’s loss depends on many parameters, so we compute one partial per parameter.
Together they form the gradient used to update every weight.

Loss / Cost Functions

A loss function measures how wrong the model is on an example.
The cost is the average loss over the training set.
Training means finding parameters that make the cost small.

Gradient Descent

Start with some parameter values.
Compute the gradient of the cost.
Step in the opposite direction of the gradient.
Repeat until the cost stops decreasing.

Each step nudges the parameters downhill. The learning rate controls how big each step is.

Learning Rate and Local Minima

Learning rate	Effect
Too small	Converges slowly
Just right	Converges steadily
Too large	Overshoots or diverges

A local minimum is a point lower than its neighbors but not necessarily the global lowest.
Gradient descent can settle into a local minimum; good learning rates and momentum help.

Probability Foundations

A random variable is a quantity whose value depends on chance.
A probability distribution describes how likely each value is.
Probability lets ML express uncertainty, not just a single guess.

Expectation and Variance

Expectation (mean) is the long-run average of a random variable.
Variance measures how spread out the values are.
Low variance means values cluster tightly; high variance means they are scattered.

Distributions and Sampling

Distribution	Use in ML
Normal (Gaussian)	Noise, many natural measurements
Bernoulli	Binary outcomes (success/fail)
Uniform	Equal likelihood over a range

Sampling draws data from a distribution. Understanding sampling lets you estimate model behavior from limited data.

Hypothesis Testing and Significance

A hypothesis test asks whether an observed effect is real or due to chance.
Statistical significance means the result is unlikely under the null hypothesis.
Significance does not equal importance — a tiny effect can be "significant" with enough data.

Syntax

Math in Python (standard library)python

1import math2import statistics3 4# Dot product of two vectors (as lists)5a = [1, 2, 3]6b = [4, 5, 6]7dot = sum(x * y for x, y in zip(a, b))  # 328 9# Mean and variance10data = [2, 4, 4, 4, 5, 5, 7, 9]11mean = statistics.mean(data)12var = statistics.pvariance(data)

These primitives (sum, zip, statistics) are enough to implement dot products, means, and variances by hand — the foundation of NumPy.

Examples

Matrix multiplication by hand

Multiply a 2×2 matrix by a 2×1 vector.

text

1[1 2]   [5]     [1*5 + 2*6]   [17]2[3 4] × [6]  =  [3*5 + 4*6] = [39]

Output

(each result entry is a dot product of a row with the column)

Gradient descent step

One step for f(x) = x^2 at x = 4.

text

1f(x) = x^2, gradient = 2x2at x = 4: gradient = 83step with lr = 0.1: x = 4 - 0.1*8 = 3.2

Output

(moving opposite the gradient moves downhill)

Expectation of a die roll

The long-run average of a fair six-sided die.

text

1E[X] = (1+2+3+4+5+6) / 6 = 3.5

Output

(the mean of the uniform distribution over 1..6)

Key Takeaways

Data is vectors and matrices; a dataset is a matrix of samples × features.
Matrix multiplication is dot products, with a strict shape rule.
The gradient points uphill; gradient descent steps the other way.
The learning rate trades speed against stability.
Loss functions define what training minimizes.
Probability expresses uncertainty; variance and significance quantify it.

← PreviousPython for Data Science Next →Data Visualization