AI & Machine Learning Roadmap

Master data processing, modeling, and deep learning algorithms.

Overall Progress0%

MLOps

Explanation

MLOps is the discipline of taking machine learning from an experiment to a reliable production system. A model that works in a notebook is not a product; MLOps adds the versioning, pipelines, deployment, monitoring, and governance that keep models working in the real world over time. This lesson covers the ML lifecycle, the differences between software and ML delivery, batch versus online inference, and — most importantly — drift and monitoring, the reasons a model silently degrades after deployment. The TRY_IT simulates the deployment decision of choosing between two model versions based on accuracy, using only the standard library.

Why MLOps Exists

Models degrade as the world changes, unlike static software.
Reproducibility requires versioning code, data, and models.
Experiments must be tracked, and models must be deployable and observable.

The ML Lifecycle

Data

↓

Validation

↓

Feature preparation

↓

Training

↓

Evaluation

↓

Model registry

↓

Deployment

↓

Inference

↓

Monitoring

↓

Drift detection

↓

Retraining

This is a loop: monitoring feeds retraining, which feeds evaluation and redeployment. MLOps is engineering that closed loop.

Experimentation and Reproducibility

Track code, data, and model versions together.
Experiment tracking records parameters, metrics, and artifacts.
A result is only useful if you can reproduce it.

Model Registry and Artifacts

A model registry stores versioned models and their metadata.
Artifacts include the trained weights, preprocessing steps, and config.
The registry is the handoff point between training and deployment.

Software CI/CD vs ML CI/CD

	Software	Machine Learning
Inputs	Code	Code + data + features
Build	Compile/test	Train/evaluate
Output	Executable	Model artifact
Failure modes	Code bugs	Bugs + data + drift

ML adds data and training to the pipeline, which introduces new sources of failure that software CI/CD does not have.

Continuous Training

Retrain on a schedule or when monitoring signals degradation.
Re-evaluate before deploying; never retrain blindly.
Automate the retrain–evaluate–deploy loop with guardrails.

Batch vs Online Inference

	Batch inference	Online inference
Pattern	Predict a large set periodically	Predict a single request in real time
Latency	Not critical	Must be low
Example	Nightly churn scores	Fraud check on each transaction

Model Serving and Containers

Serving exposes a model as an API (or a batch job).
Containers package the model with its exact environment.
This solves "works on my machine" and enables scaling.

Monitoring: Data Drift

Data drift: the distribution of input features changes.
Example: customer age distribution shifts over time.
Detect it by comparing live inputs to the training distribution.

Monitoring: Concept Drift

Concept drift: the relationship between inputs and target changes.
Example: what counts as "fraudulent" changes as attackers adapt.
This is why a model can degrade even if inputs look the same.

Prediction Drift and Performance

Prediction drift: the distribution of outputs shifts.
Track model performance metrics (accuracy, precision) in production.
When performance drops below a threshold, trigger alerting and retraining.

A Drift Example: Fraud Detection

Initial good performance

↓

Behavior changes

↓

Input distribution shifts

↓

Performance drops

↓

Monitoring detects

↓

Retraining

↓

Re-evaluation

↓

Deploy or rollback

A fraud model that worked last year may fail this year because fraudsters changed tactics — that is concept drift, and monitoring is how you catch it.

Deployment Strategies

Canary: route a small fraction of traffic to the new model first.
A/B testing: compare two models on real traffic.
Shadow deployment: run the new model in parallel without serving it.
Rollback: revert to a previous model if the new one fails.

Governance and Responsible ML

Explainability: can stakeholders understand predictions?
Auditability: can you trace how a decision was made?
Security and privacy: protect data and model access.
Responsible deployment: fairness, monitoring, and accountability.

Cost and ML Technical Debt

Compute, storage, and serving all cost money.
ML technical debt accrues from entangled pipelines and hidden feedback loops.
Manage cost and debt deliberately, not as an afterthought.

Common MLOps Failures

Deploying without monitoring.
Ignoring data and concept drift.
Versioning code but not data or model artifacts.
Retraining without evaluation.
Assuming production accuracy equals development accuracy.

Examples

A real-world architecture (conceptual)

Where each stage lives in a production system.

text

1Data source → pipeline → feature processing → training2  → experiment tracking → model registry → deployment3  → model server/API → monitoring → alerting → retraining

Output

(the closed loop from data back to data)

Choosing between two model versions

A deployment decision with a threshold.

text

1model_v1 accuracy = 0.962model_v2 accuracy = 0.813threshold = 0.904v2 < threshold → keep v1 (do not deploy)

Output

(deploy only when validation passes)

Key Takeaways

MLOps closes the loop from data to deployment to monitoring and back.
Reproducibility needs code, data, and model versioning together.
Batch inference is periodic; online inference is real-time.
Data drift changes inputs; concept drift changes the input→target relationship.
Monitor performance and retrain only after re-evaluation.
Use canary/A-B/shadow deployments and keep rollback ready.

← PreviousGenerative AI & LLMs