AI & Machine Learning Roadmap

• Deep learning is a subset of machine learning using layered neural networks.
• Classical ML often relies on hand-crafted features; deep learning learns features automatically.
• Deep learning excels with large data and raw inputs (images, text, audio).

Artificial neurons are loosely inspired by brain neurons, but the analogy is only a rough metaphor. An artificial neuron is really just a small mathematical function: a weighted sum followed by an activation. The "learning" is gradient descent on a loss function.

• Inputs: the values a neuron receives (features or previous neurons).
• Weights: how strongly each input influences the output.
• Bias: a learnable offset that shifts the neuron’s output.
• Weighted sum: z = w1*x1 + w2*x2 + ... + b.
• Activation: a = activation(z), which introduces non-linearity.

First the neuron computes a weighted sum of its inputs plus a bias, then an activation function decides how strongly to "fire".

1z = w1*x1 + w2*x2 + b2a = activation(z)

• Weights and bias are the parameters the network learns.
• The activation turns a linear combination into a richer, non-linear function.

A "deep" network simply has many hidden layers. Each layer transforms its input into a slightly more abstract representation.

Forward propagation passes data through the network from input to output, producing a prediction. No learning happens in this step.

• The loss measures how far the prediction is from the target.
• The gradient tells us how each weight should change to reduce the loss.
• From Math for ML, you know the gradient points uphill — so we step downhill.

Backpropagation is the algorithm that computes gradients for every weight in a network. It works backward from the loss, using the chain rule of calculus to attribute how much each weight contributed to the error.

1. Initialize parameters (small random values).
2. Take a batch of inputs.
3. Forward pass to produce predictions.
4. Calculate the loss.
5. Backpropagate to compute gradients.
6. Update weights (gradient descent).
7. Repeat for many epochs.
8. Evaluate on held-out data.

• An epoch is one full pass over the training data.
• A batch is the group of examples processed together in one step.
• An iteration is one weight update (one batch processed).
• Learning rate controls the size of each update (from Math for ML).

• Classification: softmax over classes → a probability per class.
• Binary: sigmoid → probability of the positive class.
• Regression: a linear output (no final activation) → a number.

• Deep networks have huge capacity and can memorize the training set.
• Dropout randomly disables neurons during training to force robustness.
• Early stopping halts training when validation loss stops improving.
• More data and weight regularization also reduce overfitting.

• More layers allow more abstract, hierarchical representations.
• Depth is not automatically better — it needs data and tuning.
• Very deep networks are harder to train (vanishing gradients).

• Training large networks requires many matrix multiplications.
• GPUs are specialized for the parallel math that neural networks need.
• Training is compute-heavy; inference (prediction) is cheaper but still needs hardware.

• Confusing parameters (learned) with hyperparameters (chosen).
• Assuming more layers always mean better.
• Evaluating only on the training set.
• Confusing epochs, batches, and iterations.
• Confusing training (learning weights) with inference (making predictions).