Skip to main content

Command Palette

Search for a command to run...

Day 2 — ML Lifecycle (From Idea to Live)

Updated
5 min read
Day 2 — ML Lifecycle (From Idea to Live)
P

👋 Hello! I'm passionate about DevOps and have over 1+ years of experience in the field. I'm proficient in a variety of cutting-edge technologies and always motivated to expand my knowledge and skills. Let's connect and grow together!

SKILLS:

🔹 Languages & Runtimes: Python, Shell Scripting, HCL, YAML 🔹 Cloud Technologies: AWS, Microsoft Azure, GCP 🔹 Infrastructure Tools: Docker, Terraform, AWS CloudFormation 🔹 Other Tools: Linux, Git and GitHub Actions, Jenkins, Jira, GitLab (beginner), Docker, AWS DevOps 🔹 Web Development: HTML, CSS, Bootstrap, Python, SQL

Job & Responsibilities:

🚀 Improved development efficiency by implementing CI/CD pipelines, resulting in a 30% reduction in deployment time on the test server. 🔒 Strengthened deployment and testing reliability by utilizing Docker containers and optimizing Dockerfile, reducing development issues on the test server by 20%. ⚙️ Automated S3 bucket log creation with Shell scripting, eliminating 100% of manual search and saving 2 hours per week. 📅 Scheduled EC2 instance start/stop using Lambda functions and Event Bridge, leading to a 25% decrease in infrastructure costs. 🔧 Utilized AWS, Linux, Python, Docker, Shell scripting, Terraform, Jenkins Pipelines, and automation to streamline workflows and improve overall system performance.

I'm very detail-oriented and possess strong written and verbal communication skills. As a high performer with a possibility mindset, I strive to solve problems using efficient approaches.

Let's Connect & Grow:

If you find my profile suitable for the role you are searching for, please feel free to reach out to me at sumanprasad9766@gmail.com.

Today we go deeper into how a machine learning system actually works end-to-end, not just the model.

Think of this like running an e-commerce business.

Just like an order goes through multiple stages:

Browsing → Add to cart → Payment → Packing → Shipping → Delivery

An ML model also goes through stages. This full journey is called the ML Lifecycle.

And MLOps supports and automates every stage, just like DevOps supports software delivery.


The ML Lifecycle — Big Picture View

Here is the simple flow:

Collect Data

↓

Clean & Prepare Data

↓

Train Model

↓

Evaluate & Test Model

↓

Deploy Model

↓

Monitor Model

↓

Retrain / Improve Model

This cycle never ends, because real-world data keeps changing.


Stage 1 — Data Collection (Getting the Raw Material)

Think of data as ingredients in a kitchen.

If the ingredients are bad, the food will be bad — even if the chef is great.

Examples of Data Sources

  • Food delivery - Order history, delivery times, restaurant ratings

  • Banking - Transactions, customer info, repayment behavior

  • Hospitals - Patient records, lab test results, symptoms

  • E-commerce - Product clicks, purchases, search queries

MLOps ensures that:

  • data is collected safely

  • data pipelines run reliably

  • data is stored securely

  • data versions are tracked

Just like DevOps manages code pipelines, MLOps manages data pipelines.


Stage 2 — Data Cleaning & Preparation (Making Ingredients Usable)

Raw data is usually messy. Like vegetables with mud or rotten parts.

So we must:

  • remove incorrect values

  • fill missing data

  • remove duplicates

  • standardize formats

  • transform values

Real-world examples

  • Food delivery - Address formats differ → must be cleaned

  • Banking - Salary sometimes written as 50000 and sometimes 50k → must standardize

  • Hospitals - One system says Male/Female, another says M/F → must normalize

Without proper cleaning:

  • models get confused

  • accuracy drops

  • predictions become risky

MLOps ensures data validation checks exist so bad data never reaches the model.

This is similar to quality checks in manufacturing.


Stage 3 — Model Training (Teaching the Model)

Training is like teaching a student using past exam papers.

The model learns:

Patterns → Relationships → Predictions

Examples:

Food delivery = Learns that rain + traffic = longer time

Banking = Learns risky transaction patterns

E-commerce = Learns what customers like

Hospitals = Learns which symptoms indicate risk

MLOps ensures:

  • experiments are tracked

  • results are recorded

  • models are versioned

  • training is reproducible

This is where tools like MLflow or Weights & Biases come in.


Stage 4 — Model Evaluation (Checking the Student’s Test Performance)

We never deploy a model blindly. Just like checking exam results before promoting a student.

We test:

  • accuracy

  • fairness

  • bias

  • stability

  • performance

Real-world checks

  • Banking - Does the model discriminate against a group?

  • Hospitals - Does it miss critical patients?

  • Food delivery - Does it underestimate peak hour delays?

MLOps ensures proper testing pipelines, similar to DevOps testing pipelines.


Stage 5 — Deployment (Putting the Model into Real Use)

Deployment means releasing the model into production, where users interact with it.

Example:

  • Food delivery - ETA shown in app

  • Banking - Loan approved instantly

  • E-commerce - Product recommendations appear

  • Hospitals - Risk score shows on doctor dashboard

Deployment can be:

  • real-time (instant response)

  • batch (once a day)

  • streaming (continuous)

MLOps helps automate deployment reliably, just like DevOps CI/CD pipelines.


Stage 6 — Monitoring (Watching the Model After Release)

This is one of the biggest differences between DevOps and MLOps.

In DevOps, you monitor:

  • uptime

  • errors

  • latency

In MLOps, you also monitor:

  • accuracy

  • data drift

  • prediction quality

  • fairness

  • business KPIs

Real-world examples

  • Food delivery - Predicted ETA suddenly starts deviating

  • Banking - Fraud model starts missing scams

  • Hospitals - Model makes wrong medical predictions

MLOps triggers alerts so teams can act fast.


Stage 7 — Retraining (Continuous Improvement)

Over time, data changes. This is called concept drift.

Examples:

  • Food delivery - Traffic patterns change due to new road

  • Banking - New types of fraud appear

  • E-commerce - Customer taste trends change

  • Hospitals - New disease symptoms emerge

So models must relearn regularly.

MLOps automates:

  • detecting drift

  • retraining models

  • validating them

  • redeploying safely

This makes ML systems living systems — always learning.


How MLOps Supports Every Stage

Here’s a clean mapping:

ML StageHow MLOps Helps
Data CollectionData pipelines, validation, logging
Data CleaningAutomated quality checks
TrainingExperiment tracking & reproducibility
EvaluationTesting pipelines
DeploymentCI/CD for ML models
MonitoringDrift, accuracy, fairness checks
RetrainingAutomated model lifecycle

So while DevOps focuses mainly on code,

MLOps focuses on code + data + model + monitoring + retraining.


A Simple Story to Remember the Lifecycle

Imagine a hospital builds a model to predict heart attack risk.

  1. Collect patient data

  2. Clean mistakes

  3. Train model

  4. Test accuracy

  5. Deploy in doctor dashboard

  6. Monitor predictions

  7. Retrain as medical patterns evolve

MLOps ensures:

  • safety

  • reliability

  • traceability

  • fairness

  • compliance

This is why companies treat MLOps as a critical discipline, not an optional add-on.


Quick Recap of Day 2

By now we should clearly understand:

  • ML lifecycle is continuous

  • models are never “finished”

  • data quality matters as much as algorithms

  • monitoring and retraining are essential

  • MLOps supports every stage


Real-World Industry Case Studies

These show how big companies use ML + MLOps:

  1. Uber Michelangelo Platform

    https://www.uber.com/en-IN/blog/michelangelo-machine-learning-platform/

  2. Airbnb Machine Learning Platform

    https://medium.com/airbnb-engineering/airbnbs-ml-platform-overview-417b8d9e0c3a

  3. Netflix Recommendation & ML Pipeline

    https://netflixtechblog.com/tagged/machine-learning

These are great to demonstrate how MLOps works in production.

MLOps

Part 19 of 20

Practical MLOps series breaking down how ML systems work in production — from data pipelines to deployment, monitoring, and retraining. No buzzwords, just real-world MLOps concepts explained simply for engineers and data teams.

Up next

Day 1 - Understanding MLOps v/s DevOps

1. What Is MLOps? Think of a food delivery app like Swiggy or Zomato. They use machine learning (ML) to: predict delivery time recommend restaurants detect fraud optimize delivery routes But building an ML model once is not enough. The real ch...

More from this blog

D

DeployToCloud

405 posts

👋 Welcome to my Hashnode blog! I'm a DevOps Engineer with 2+ years of experience. Join ~5k followers and explore 320+ blogs on Python, AWS, Docker, Jenkins, Linux, and more. Let's connect & grow 🚀