Generative AI & LLMs — AI & Machine Learning Roadmap

Explanation

A large language model (LLM) is a system trained to predict the next token in text. That single idea, scaled to enormous models and datasets, produces the conversational behavior people now interact with every day. Understanding what an LLM actually does — generate text token by token from learned probabilities — is the key to using it well and not being misled by it. This lesson builds the full picture: tokenization, embeddings, the Transformer, attention, pretraining and fine-tuning, decoding, retrieval (RAG), and the crucial distinction between a model and an agent. It is the most conceptual lesson in this batch. The TRY_IT is a toy next-token predictor. It is not an LLM and is clearly labeled as such.

Discriminative vs Generative Models

	Discriminative	Generative
Learns	Boundary between classes	The data distribution
Typical output	A label or score	New samples (text, images)
Example	Spam classifier	An LLM generating text

Language Modeling as Next-Token Prediction

An LLM predicts the probability of the next token given all previous tokens.
Generation is repeated prediction: choose a token, append, repeat.
This is why an LLM produces text one token at a time.

Tokens, Vocabulary, and Context

Text is split into tokens (often subwords), each mapped to an integer ID.
The vocabulary is the fixed set of tokens the model knows.
The context window is how many tokens the model can consider at once.

Embeddings and the Transformer

Each token ID is mapped to an embedding vector.
Transformer layers transform these vectors using self-attention.
Positional information is added because attention itself has no sense of order.

Self-Attention: Query, Key, Value

For each token, the model computes a query, key, and value. A token attends to others by comparing its query against their keys; the resulting attention scores weight the values.

Query: "what am I looking for?"
Key: "what do I contain?"
Value: "what do I contribute?"

Encoder, Decoder, and Decoder-Only Models

An encoder reads input into representations.
A decoder generates output from representations.
Most modern chat LLMs are decoder-only: they generate text directly.

The LLM Internal Pipeline

Text

↓

Tokenization

↓

Token IDs

↓

Embeddings

↓

Transformer layers

↓

Next-token probabilities

↓

Decoding

↓

Next token

Generation loops through this pipeline once per token. The model does not "look up an answer"; it computes a distribution over the next token each step.

Pretraining and Fine-Tuning

Pretraining: learn general language from huge text corpora (next-token prediction).
Fine-tuning: adapt the pretrained model to a task or domain with additional data.
Instruction tuning: fine-tune on instruction–response pairs so the model follows requests.

Alignment and RLHF

Alignment shapes a model toward helpful, harmless, honest behavior.
RLHF (Reinforcement Learning from Human Feedback) trains on human preferences.
Preference optimization tunes which responses people actually prefer.

Decoding: Temperature, Top-k, Top-p

Setting	Effect
Temperature	Higher = more random; lower = more deterministic
Top-k	Sample only from the k most likely tokens
Top-p	Sample from the smallest set covering p of probability mass

Decoding controls how the next token is chosen from the probability distribution. Higher temperature does not mean "smarter" — it means more random.

Training vs Inference

	Training	Inference
Input	Large dataset	User prompt tokens
Work	Optimize parameters	Forward pass + decoding
Cost	Huge (many GPUs)	Lower per request
Result	A trained model	Generated output

Prompting

A prompt is the interface through which you give a language model a task, context, and constraints. Because an LLM produces text from the prompt it is given, a well-specified prompt is how you steer its output.

Clear task: say what you want the model to do.
Context: provide the background the model needs.
Constraints: state limits (length, tone, format).
Desired output format: say how the answer should look.
Examples: show what good output looks like when helpful.

System vs User Instructions

Many conversational systems separate instructions into two roles. System instructions set standing rules for the whole conversation; user instructions contain the specific request. This gives an instruction hierarchy: system rules persist while user messages vary.

text

1system: "Return all answers as JSON."2user:   "Give me the three largest cities in Ghana."

Few-shot Prompting

Zero-shot: no examples, only the instruction.
One-shot: a single example.
Few-shot: several examples that demonstrate the expected behavior.

text

1Classify sentiment:2  "I loved it" → positive3  "It was awful" → negative4  "It was fine" → ?

Few-shot examples are most useful when the desired format or classification is not obvious, and when the model can learn the pattern from the examples alone.

Structured Output

Applications need machine-readable output, not free prose.
Structured output specifies a schema with predictable fields.
The application then validates and parses the response.
Malformed output requires retry or a fallback path.

json

1{2  "sentiment": "positive",3  "confidence": 0.914}

Tool / Function Calling

Instead of answering from memory, a model can request that an external tool be run. The model emits a structured "tool call"; the surrounding application decides whether and how to execute it, then returns the result to the model.

User request

↓

Model requests a tool

↓

Application executes

↓

Result returned

↓

Model responds

text

1get_weather(city="Accra")

Fine-Tuning vs Retrieval-Augmented Generation

	Fine-tuning	RAG
Changes	Model parameters	Adds external knowledge at inference
Best for	Style, format, domain behavior	Fresh or private facts
Example	Teach a consistent brand voice	Answer from company documents

RAG retrieves relevant documents and feeds them into the prompt so the model grounds its answer in provided facts rather than memorized parameters.

Hallucination and Grounding

Hallucination is fluent but false output.
Grounding ties answers to retrieved or provided evidence.
LLMs predict likely text; they do not guarantee facts.

LLM Evaluation

Evaluating an LLM is harder than testing a deterministic program, because there is no single "correct" output. The same question can have many valid answers, and a fluent answer can still be wrong.

Correctness: is the answer right?
Relevance: does it address the question?
Factuality: are the claims true?
Groundedness: does it stick to the provided evidence?
Instruction following: does it obey the format and constraints?
Consistency and safety: does it behave reliably and harmlessly?
Latency and cost matter for production, not just quality.

	Human evaluation	Automated evaluation
How	People judge quality	Programs score against references or rules
Strength	Captures nuance	Fast, cheap, repeatable
Weakness	Slow, costly, subjective	May miss subtle errors

Benchmark datasets give a standardized score but do not cover every task.
Task-specific evaluation (e.g., does RAG cite the right document?) matters more than a generic score.
Regression testing re-checks important cases after every model change.

Evaluation ties directly to the rest of this node: RAG answers are checked for groundedness, structured output is checked for valid, correct fields, and production monitoring watches whether quality drifts over time.

LLM vs AI Agent

An LLM generates text from a prompt.
An agent combines a model with tools, memory, planning, and external systems.
An agent may call functions, browse, or act — the LLM is one component.

Cost, Latency, and Open vs Hosted

Larger models cost more and are slower.
Hosted models are convenient; open-weight models offer control and privacy.
The trade-off is capability versus cost, latency, and data sovereignty.

Bias, Safety, and Privacy

Models reflect biases in their training data.
Safety mechanisms aim to reduce harmful output.
Sending private data to a hosted model is a privacy decision.

Common Mistakes

Assuming LLMs retrieve facts from a database.
Confusing training with inference.
Confusing RAG with fine-tuning.
Assuming higher temperature means "more intelligent".
Assuming LLM output is automatically factual.
Confusing an LLM with an agent.

Examples

Decoding a probability distribution

A hypothetical distribution over the next token.

text

1P("the") = 0.4, P("a") = 0.3, P("cat") = 0.2, P("xyzzy") = 0.12Temperature 0 → always "the" (deterministic)3Top-k=2 → sample from {"the", "a"}4Top-p=0.7 → sample from {"the", "a"}

Output

(decoding settings change which token is chosen)

Fine-tuning vs RAG for a support bot

Choosing the right approach.

text

1Need consistent tone + format → fine-tune2Need current product docs → RAG (retrieve + cite)

Output

(one changes parameters; the other adds facts at query time)

Key Takeaways

An LLM predicts the next token; generation is that process repeated.
Tokens become embeddings; Transformers refine them with self-attention.
Pretraining learns language; fine-tuning and alignment shape behavior.
Decoding (temperature, top-k, top-p) controls randomness, not intelligence.
Fine-tuning changes weights; RAG adds external knowledge at inference.
An LLM is not a database, a search engine, or an agent.