AI & Machine Learning Roadmap

Master data processing, modeling, and deep learning algorithms.

Overall Progress0%

Supervised Learning

Explanation

Supervised learning is learning from labeled examples. You give the algorithm many input-output pairs — features and the correct answer — and it learns a mapping from inputs to outputs that generalizes to new, unseen inputs. This lesson builds the mental model: features and labels, the difference between regression and classification, the classic algorithms (linear regression, logistic regression, decision trees, k-nearest neighbors, ensembles), and the central tension of overfitting and underfitting. You will also learn how to choose an algorithm and recognize the common mistakes. The worked examples use small, manually understandable datasets so the ideas stay clear.

What Supervised Learning Is

Inputs (features) are paired with known outputs (labels).
The model learns to predict the label for new inputs.
"Supervised" because the correct answer is provided during training.

Features and Labels

Term	Meaning	Example
Feature	Input the model sees	House size, bedrooms, location
Label (target)	Output the model predicts	Sale price
Training data	Labeled examples to learn from	Past house sales

Regression vs Classification

	Regression	Classification
Predicts	A continuous number	A category
Example	Price = $210,000	Spam or not spam
Output type	Numeric	Discrete class

Binary classification: two classes (spam / not spam).
Multiclass classification: three or more classes (cat / dog / bird).

The Supervised Learning Workflow

Collect labeled data

↓

Split train/test

↓

Choose a model

↓

Train

↓

Predict

↓

Evaluate

The loop is: prepare data, train on the training set, predict on unseen data, and evaluate how well the mapping generalizes.

Linear Regression

Fits a straight line (or plane) to the data.
Predicts a continuous value as a weighted sum of features.
Interpretable: each coefficient shows the effect of one feature.

Logistic Regression (Concept)

Despite the name, it is used for classification, not regression.
It outputs a probability between 0 and 1, then applies a threshold.
The "S" shape maps any input to a valid probability.

Decision Trees

Split data by asking a sequence of yes/no questions about features.
Each leaf is a prediction.
Easy to interpret but prone to overfitting if grown too deep.

k-Nearest Neighbors (k-NN)

Predicts by looking at the k closest training examples.
Classification: majority vote among neighbors.
Regression: average of neighbors’ values.
No explicit training; it stores data and compares at prediction time.

Support Vector Machines (Concept)

Finds the boundary that best separates classes with the largest margin.
Works well on clean, well-separated data.
Kernels let it draw non-linear boundaries.

Ensemble Methods and Random Forests

Ensembles combine many models to reduce error.
A random forest averages many decision trees trained on random subsets.
Combining weak learners often beats a single strong model.

Training vs Prediction

Training: adjust the model’s parameters to fit the training data.
Prediction: apply the learned parameters to new inputs.
A model is useful only if it generalizes from training to unseen data.

Loss / Error Concept

The loss measures how wrong a prediction is.
Training minimizes the total loss over the training set.
Different tasks use different losses (squared error for regression, cross-entropy for classification).

Overfitting and Underfitting

	Underfitting	Overfitting
Meaning	Model too simple to capture patterns	Model memorizes noise
Symptom	Poor on both train and test	Great on train, poor on test
Fix	More complexity / features	Simpler model, more data, regularization

Bias and Variance

Bias: error from oversimplifying (leads to underfitting).
Variance: error from overreacting to training data (leads to overfitting).
The goal is the sweet spot between the two.

Model Complexity

More parameters means more capacity to fit complex patterns.
Too much capacity for the data invites overfitting.
Choose complexity based on data size and the problem.

Feature Importance (Concept)

Feature importance measures how much each input contributes to predictions.
Tree-based models provide this naturally.
It helps explain models and select useful features.

Choosing an Algorithm

Situation	Good starting choice
Predict a number, want interpretability	Linear regression
Two-class prediction, want probabilities	Logistic regression
Need explainable rules	Decision tree
High accuracy on tabular data	Random forest / gradient boosting
Small, clean dataset	k-NN or SVM

Common Mistakes

Evaluating on the training set (looks good, generalizes poorly).
Leaking the label into the features.
Using accuracy on an imbalanced dataset.
Ignoring feature scale for distance-based models.
Jumping to a complex model before a simple baseline.

Examples

A tiny regression

Linear regression predicts price from size.

text

1Data: (size=100, price=200k), (150, 300k), (200, 400k)2Pattern: price = 2k per unit of size (roughly)3Predict for size=175 → about 350k

Output

(the model learned a linear relationship)

A binary classification

Classifying emails as spam or not.

text

1Features: contains "free money", sender known, link count2Label: spam (1) or not (0)3Model learns: high link count + "free money" → spam

Output

(the model maps features to a class)

Key Takeaways

Supervised learning learns a mapping from features to labels.
Regression predicts numbers; classification predicts categories.
Overfitting memorizes; underfitting oversimplifies.
Different algorithms suit different data and goals.
Always evaluate on unseen data, never on the training set.
Start with a simple baseline before a complex model.

← PreviousData Preprocessing Next →Unsupervised Learning