MLOps for LLM: A Case Study on Dresscode

I've recently participated in the Gemma 4 challenge here on DEV.to, but fell short compared to many amazing projects. I really liked LIKAS. I encourage you all to check it; it's awesome.

Nevertheless, I liked my project, Dresscode - your AI stylist -, so I decided to go on with it. But moving from a proof of concept to a fully fledged project is not that simple. And MLOps is a crucial part of this change.

This tutorial will walk you through the complete MLOps pipeline, but I won't go into much detail and code to keep it readable.
I'll explain the steps, give real-life examples, and explain how I would do them for the production version of Dresscode (not the open-sourced code)

The classic MLOps pipelines usually consist of 7 steps

1. Data Ingestion & Engineering (data preprocessing)
2. Feature Store (feature extraction)
3. Model Training & Experimentation (CI) (training)
4. Model Registry
5. Continuous Deployment (CD) & Serving
6. Continuous Monitoring
7. The Feedback & Retraining Loop

I'll go into much detail and examples of each step, for those who don't know Dresscode, Dresscode uses AI in 2 ways:

• Visual Wardrobe Digitization (Computer Vision): The user uploads photos of their clothes, and the Gemma 4 multimodal model automatically detects and extracts a structured text list of individual clothing items found in the image.
• Context-Aware Outfit Recommendation (GenAI + Agents): The app takes the user's digitized wardrobe list and combines it with real-time environmental context. Using LLM function calling, the model dynamically fetches live weather data and the current season from an external API to curate a highly personalized, weather-appropriate outfit.

This implementation was to show the model's performance and participate in the competition. We will discuss how to apply the MLOps pipeline for these usages and optimize the performance.

Applying MLOps ensures that AI features remain accurate, reliable, cost-effective, and easy to maintain over time. Here is how that system (any ML system) moves through the MLOps pipeline:

Step 1: Data Ingestion & Engineering (The Foundation)

Think of this step as preparing the ingredients before cooking. Raw data is usually messy, scattered, and full of mistakes. If you feed bad data into an AI model, you get bad results (aka "Garbage In, Garbage Out").

• Data Ingestion: Gathering data from various sources (databases, APIs, live streams, files) and moving it to a central storage area (like a data lake).
• Data Engineering (Preprocessing): Cleaning that data. This involves removing duplicates, fixing missing values, and formatting it so the machine learning model can actually understand it.

Sample Use Cases

Here is how this step looks across three different types of systems:

1. Sensor Data Clustering (IoT / Industrial Systems)

• The Raw Data: Millions of rapid temperature and vibration readings coming from factory machines every second.
• The Preprocessing Challenge: The data is noisy, has missing time gaps (due to Wi-Fi drops), and is way too massive.
• What happens here:
  • Filtering: Smoothing out the random "noise" or spikes in the data.
  • Imputation: Filling in those missing time gaps using averages.
  • Aggregation: Downsampling the data (e.g., converting millisecond readings into 1-minute averages) so the clustering model isn't overwhelmed.

2. Image Processing (Computer Vision)

• The Raw Data: Thousands of user-uploaded photos of various sizes, formats (JPEG, PNG), and lighting conditions.
• The Preprocessing Challenge: AI models require images to be exactly the same size and format to process them mathematically.
• What happens here:
  • Resizing & Cropping: Uniformly scaling all images to a standard resolution (e.g., 224 x 224 pixels).
  • Normalization: Converting pixel colors into a consistent scale (usually between 0 and 1).
  • Augmentation (Optional): Creating variations by flipping or rotating images to give the training model more variety.

3. LLM Systems (Large Language Models / GenAI)

• The Raw Data: Massive text dumps from PDF manuals, customer support chats, and internal wikis.
• The Preprocessing Challenge: Text is unstructured, contains sensitive information, and is often too long for an LLM to read at once.
• What happens here:
  • Anonymization: Scrubbing out private data like names, phone numbers, or credit cards (PII redaction).
  • Text Cleaning: Stripping out weird HTML tags, formatting errors, or emojis if they aren't needed.
  • Chunking: Chopping long documents into smaller, bite-sized paragraphs so they can be turned into "embeddings" for the LLM to search through later.

Applying MLOps Step 1 to dresscode

Even when using off-the-shelf Foundation Models like Gemma 4 instead of training your own models from scratch, MLOps Step 1 is vital to ensure consistency, speed, and cost efficiency.

Apply to Use Case 1: Closet Image Uploads

Because users will upload images straight from their smartphones, the ingestion pipeline needs to clean up the raw files before passing them to Gemma.

• Data Ingestion: Establish an API gateway that securely uploads user photos to a cloud landing bucket (e.g., AWS S3).
• Image Downsampling & Compression: Smartphone photos can easily be 5MB to 12MB each. Passing these directly to an LLM creates massive latency and high API costs. The engineering pipeline should instantly compress images and scale them down to a standard resolution suitable for multimodal analysis.
• Format Standardization: Convert disparate uploads (HEIC from iPhones, WebP, PNG) into a single unified format (like JPEG) so Gemma receives predictable data inputs.
• EXIF Stripping: For user privacy, strip out geographic coordinates and camera metadata from the image file during ingestion.

Apply to Use Case 2: Weather & Outfit Recommendations

This use case relies heavily on text and structured API responses. The pipeline needs to handle both data streams seamlessly.

• JSON Schema Standardization (Ingestion): The external weather API will return a raw JSON payload full of data we do not need (like wind angle or air pressure). The pipeline should extract only the critical metrics (e.g., temperature, precipitation_chance, season) and format them into a clean text prompt string for the model.
• Text Normalization (Engineering): Wardrobe descriptions written by users or generated by Use Case 1 might have formatting inconsistencies. The data engineering layer should clean this up—handling missing descriptions gracefully, removing trailing spaces, and capping the list size so we do not exceed the LLM's context window.
• Caching Layer: To reduce external API costs and latencies, we can build an ingestion cache. If the same user asks for an outfit recommendation three times in one morning, the pipeline should pull the cached weather data from two hours ago instead of making a brand-new live API call.

Step 2: Feature Store & Feature Extraction (The Memory Bank)

Once the raw data is clean (from Step 1), we need to transform it into "features"—which are the specific, measurable characteristics that an AI model uses to make decisions (like a person's height, a car's speed, or a word's meaning).

• Feature Extraction: The process of turning raw data into these smart variables.
• Feature Store: A centralized library or warehouse where these features are saved, organized, and shared. Instead of recalculating features every single time a model runs (which is slow and expensive), teams can just pull them instantly from the Feature Store. It ensures that the exact same data definitions are used during both model training and real-time production.

Sample Use Cases

Here is how a Feature Store works across the three system types:

1. Sensor Data Clustering (IoT / Industrial Systems)

• The Raw Data: Cleaned, minute-by-minute temperature and vibration logs.
• The Extracted Features: Calculated metrics like vibration_standard_deviation_past_24h or temperature_spike_frequency.
• Why the Feature Store Matters: Calculating standard deviations across millions of streaming data points in real time is computationally heavy. The Feature Store calculates these rolling metrics once and serves them instantly to the clustering model to group machines into "healthy" or "failing" categories.

2. Image Processing (Computer Vision)

• The Raw Data: Resized and normalized images.
• The Extracted Features: High-level mathematical descriptions of the images, such as color histograms, edge-detection maps, or embeddings (vectors generated by an intermediate neural network layer that can be trained to represent the visual style of the clothing or the face features or any other features).
• Why the Feature Store Matters: If multiple models need to process the same image (e.g., one model to detect a dress, another to detect its color), they don't all need to re-extract the visual characteristics. They pull the pre-computed image embeddings directly from the Feature Store, drastically speeding up response times.

3. LLM Systems (Large Language Models / GenAI)

• The Raw Data: Chunked text paragraphs from documents or manuals.
• The Extracted Features: Text Embeddings (numerical vectors that capture the semantic meaning of a paragraph) and user behavioral features (e.g., user_preferred_tone_style or past_3_topics_discussed).
• Why the Feature Store Matters: In apps like Facebook, calculating the mathematical meaning of thousands of probable friends every time a user logs in would cause massive delays. The system extracts the vector embeddings for each possible friend once, stores them in the Feature Store, and instantly retrieves them when the LLM needs to suggest friends to the user.

Applying MLOps Step 2 to dresscode

Because we are using an advanced LLM and multimodal system (Gemma 4), the Feature Store won't look like a traditional database of customer table numbers. Instead, it will primarily store embeddings (mathematical vectors representing meaning) and aggregated metadata.

Apply to Use Case 1: Closet Image Uploads

When Gemma 4 processes clothing images, it interprets them by breaking them down into numbers. A Feature Store avoids repeating this heavy math.

• Offline Feature Extraction (Image Vectorization): When an image is successfully ingested, we pass it through an image embedding model to extract a visual vector (a string of numbers capturing style, pattern, and color).
• The Feature Store Strategy: Store these visual embeddings in the online section of your Feature Store (often a Vector Database like pgvector, Pinecone, or Milvus).
• Why it's needed: If a user wants to find "something similar to this blue shirt" later, or if we want to cluster their clothes by style, the app doesn't need to pass raw images to Gemma 4 again. It queries the pre-computed embeddings directly from the Feature Store in milliseconds, slashing computing costs. But since user images should be processed only once, we can simplify the pipeline by using the object description (model output) for any later clustering or processing.

Apply to Use Case 2: Weather & Outfit Recommendations

This use case relies heavily on time-sensitive features (weather changes hourly) and user historical traits.

• Real-time Feature Engineering (The Streaming Store): When the external API returns raw data (e.g., Temperature: 55F, Rain: 80%), we can transform these into categorical features in real-time, such as is_rainy = True or temperature_tier = "chilly".
• User Behavioral Features (The Batch Store): Compute historical features in batches, such as user_preferred_color_palette (based on what they wore the past 30 days) or frequently_worn_together_pairs.
• The Feature Store Strategy: The Feature Store unifies these two worlds. When the user asks for a recommendation, the system pulls the real-time weather features and combines them with the pre-calculated user preference features instantly. This clean, unified "feature vector" is what gets injected directly into the Gemma 4 context prompt, resulting in faster and vastly more accurate outfit suggestions.

Step 3: Model Training & Experimentation / CI (The AI Laboratory)

This is the phase where the AI actually learns. We take the features we prepared (from Step 2) and feed them to an algorithm so it can look for patterns and build a smart model.

• Model Training: Teaching the AI by showing it examples until it can make accurate predictions or generations on its own.
• Experimentation: Data scientists rarely get it right on the first try. They tweak settings (called hyperparameters), try different algorithms, and run hundreds of tests to find the best-performing version.
• Continuous Integration (CI): In MLOps, CI means automating this testing. Every time someone changes the training code, automated scripts run to ensure the new code doesn't break the system, tests the model's accuracy, and verifies it meets safety guidelines before moving forward.

Sample Use Cases

Here is how training and experimentation look across the three system types:

1. Sensor Data Clustering (IoT / Industrial Systems)

• The Training Process: Feeding months of historical sensor data to an unsupervised clustering algorithm to discover what a "normal machine" looks like versus a "failing machine."
• The Experimentation: Data scientists test different numbers of clusters or try different mathematical distance metrics to see which setup groups the anomalies most cleanly.
• The CI Pipeline: When new factory data architectures are introduced, the CI pipeline automatically retriggers the training script to ensure the clustering algorithm still converges correctly without throwing errors.

2. Image Processing (Computer Vision)

• The Training Process: Training a Convolutional Neural Network (CNN) on millions of tagged images so it learns to recognize the difference between the objects it should recognize.
• The Experimentation: Testing different model architectures or adjusting the "learning rate" to see which combination achieves the highest accuracy in identifying target details.
• The CI Pipeline: When a developer submits new code, the CI pipeline automatically trains a miniature version of the model on a small benchmark dataset to check for bugs and ensures it outputs the correct image classification format.

3. LLM Systems (Large Language Models / GenAI)

• The Training Process: For most companies, this rarely means training a giant LLM from scratch. Instead, it involves Fine-Tuning (taking an existing model like Gemma and training it further on a specific dataset) or optimizing prompt engineering frameworks.
• The Experimentation: Testing different system prompts, adjusting the "temperature" parameter (creativity level), or fine-tuning the model on thousands of curated fashion outfits to see which setup yields the most realistic recommendations.
• The CI Pipeline: This is often called LLM-As-A-Judge CI. Every time the prompt or model configuration changes, the CI pipeline automatically runs a test suite of standard user requests. It passes the outputs to a separate evaluation script to check for hallucinations, offensive language, or formatting errors before allowing the update to go live.

Applying MLOps Step 3 to dresscode

Because we are utilizing a pre-trained foundation model (Gemma 4) rather than training a model from scratch, "Model Training & Experimentation" in our CI pipeline focuses heavily on Prompt Engineering optimization, Prompt/Model versioning, and Automated LLM Evaluation (LLM-as-a-Judge).

Apply to Use Case 1: Closet Image Uploads

For the wardrobe digitization phase, our goal is high accuracy in clothing recognition (e.g., ensuring a jacket isn't mislabeled as a shirt) and consistent structured output formatting.

• Experimentation (Prompt & Model Tuning): We will need to test different system prompts to get Gemma 4 to output text reliably (such as a clean JSON list). We can experiment with different variants of the model (like the unified Gemma 4 12B versus smaller edge versions) to find the best balance of speed, cost, and vision accuracy.
• The CI Pipeline (Automated Evaluation): Create a "Golden Dataset" containing 100 sample photos of diverse clothing items where we already know the correct tags. Every time a developer updates the app's prompt structure or upgrades the underlying Gemma model version, the CI pipeline automatically runs those 100 photos through the new setup. It calculates a accuracy score—if the accuracy falls below the baseline threshold (e.g., 95%), the CI block fails and prevents the code from deploying.

Apply to Use Case 2: Weather & Outfit Recommendations

This agentic workflow relies on function calling. The core risk here is "hallucination" (recommending clothes the user doesn't own) or failing to execute the weather API call properly.

• Experimentation (Constraint Tuning): Data scientists will experiment with the model's Temperature setting (lower temperature, like 0.2, makes the choices less chaotic and strictly bound to the user's wardrobe list) and system guidelines to ensure the model never suggests a raincoat when it isn't raining.
• The CI Pipeline (Function Calling & Safety Verification): The CI pipeline needs to strictly mock the external weather API to test corner cases (e.g., extreme blizzard, extreme heatwaves, or API timeouts). The automated pipeline will run a suite of simulated scenarios and pass the LLM's outfit response to an "LLM Evaluation Judge" (a separate script or deterministic validator) to check:
  1. Did the model correctly trigger the API function call when given a date?
  2. Are all items in the recommended outfit strictly present in the provided wardrobe list?
  3. Is a heavy coat recommended for a 90°F day? (Failure check).

If the new prompt configuration passes all simulated weather scenarios without error, the CI pipeline automatically approves the update.

Step 4: Model Registry (The AI Vault)

Once we have successfully trained a model and proven it works (from Step 3), we don't just leave it sitting on a data scientist's laptop. We upload it to the Model Registry.

Think of the Model Registry as a secure, centralized catalog or app store for your company’s AI models. It keeps track of every model you’ve ever built, complete with its version history, who created it, what dataset it was trained on, and its exact performance scores.

Crucially, the registry manages the operational lifecycle stage of a model:

• Experimental / Staging: Testing the waters.
• Production: Live and actively serving real users.
• Archived: Safely retired, but kept for historical records or rollbacks.

Sample Use Cases

Here is how a Model Registry is utilized across the three system types:

1. Sensor Data Clustering (IoT / Industrial Systems)

• What is Registered: The clustering weights and mathematically defined cluster boundaries (e.g., Scikit-Learn or PyTorch files).
• The Registry Value: Factory equipment behavior changes depending on the season (e.g., machines run hotter in July than January). Instead of rewriting code, the operations team uses the registry to automatically swap the live model from v2.1_winter_baseline to v2.2_summer_baseline. If the new summer model misclassifies healthy machines as broken, engineers can issue a one-click rollback to the previous version.

2. Image Processing (Computer Vision)

• What is Registered: Deep learning model weights (like a YOLO or ResNet file) optimized to identify specific shapes or objects.
• The Registry Value: Imagine your image recognition team releases a new model optimized to spot car numbers on clear summer days. They register it as v3.0.0-Staging. The automated registry allows developers to test this new version on a small percentage of beta users without touching the main v2.4.0-Production model that powers the public app.

3. LLM Systems (Large Language Models / GenAI)

• What is Registered: In the LLM era, registries don't just store massive foundational model weights. They store fine-tuned adapters (like LoRA weights), specific system prompt templates, and guardrail configurations.
• The Registry Value: If you update the prompt or fine-tune an LLM to change its writing personality, you bundle those prompts and configurations together as a versioned asset in the registry. If the model starts outputting erratic responses or experiencing "hallucinations" in production, you can instantly swap the configuration package in the registry back to a stable, older blueprint.

Applying MLOps Step 4 to dresscode

Because dresscode leverages a foundation open-weights model (Gemma 4) rather than training a custom neural network from scratch, a traditional model weights registry isn't the main focus. Instead, the registry will act as a Prompt, Configuration, and Agentic Blueprint Repository.

Apply to Use Case 1: Closet Image Uploads

For visual cataloging, you are managing how a multimodal model perceives images and outputs structured text.

• What Goes in the Registry:
  • The specific variant and size of Gemma 4 you are serving (e.g., Gemma-4-12B-it-Q4_K_M.gguf for cost-efficient cloud serving or a larger 31B checkpoint).
  • The System Prompt Template that instructs Gemma 4 exactly how to format the clothing breakdown into a clean JSON array without conversational filler.
• The Registry Strategy: We bind the exact version of the Gemma model together with the exact text of the formatting prompt into a single package asset (e.g., wardrobe-digitizer:v1.2.0). If we test a new version of Gemma 4 that executes image analysis faster, we register it as v1.3.0-staging. Once it passes the quality benchmarks, we update the mutable alias pointer to production, instantly upgrading the app for all users without changing a single line of your core application code.

Apply to Use Case 2: Weather & Outfit Recommendations

This use case is highly dynamic because it manages tools (the weather API function call) and reasoning.

• What Goes in the Registry:
  • The JSON Function Call Schema (the exact code structure that tells Gemma 4 how to ask the app for the weather API data).
  • The Styling Persona Prompt (the guidelines governing the "AI Stylist" personality, rules like "never mix neon green with hot pink", and logic binding certain temperature ranges to specific clothing layers).
• The Registry Strategy: Prompts dictate agentic logic. A minor wording adjustment can drastically shift how a model selects an outfit. By putting these prompts in an official registry, you treat them as versioned code artifacts.
• Why it's needed: If a user reports that the app suddenly started recommending wool coats during an 80°F heatwave after a prompt update, we don't have to scramble to rewrite backend code or rebuild Docker containers. We go into our Model Registry UI, locate the stable outfit-generator:v2.1.0-stable snapshot, and click Rollback. The app immediately reverts to the older, safer prompt logic in milliseconds.

Step 5: Continuous Deployment & Serving (Taking It Live)

Now that our validated model is sitting safely in the Model Registry (from Step 4), it's time to put it to work in the real world.

• Model Serving: Hosting your model on a live server so apps and users can send it data and get back answers or predictions instantly. It's like turning an offline script into a live, interactive web service.
• Continuous Deployment (CD): The automated pipeline that safely transitions a model from the registry into production. Instead of engineers manually copying files to a server at midnight, the CD pipeline handles testing, scales up the necessary hardware, and rolls out the new model version smoothly without causing app downtime.

Sample Use Cases

Here is how CD and model serving work across the three system types:

1. Sensor Data Clustering (IoT / Industrial Systems)

• How it is Served: Edge or Streaming Serving. Because factory machinery runs continuously, data is streamed instantly into a lightweight runtime container hosted right on the factory floor (the edge) to avoid internet lag.
• The CD Strategy: When an updated clustering model passes all safety checks, the CD pipeline automates a Canary Deployment. It sends the new model to only 5% of the factory's machines first. If those machines report stable metrics for 24 hours, the CD system rolls out the update to the remaining 95% of the machinery automatically.

2. Image Processing (Computer Vision)

• How it is Served: Real-time API Serving. The model is packaged inside a scalable web container (like Docker) and deployed behind an API gateway on a cloud platform equipped with GPUs.
• The CD Strategy: When a new object-detection model is approved, the CD pipeline utilizes a Blue-Green Deployment. It spins up an entirely fresh server cluster running the new model ("Green"). Once the green environment is verified healthy, the load balancer instantly swaps 100% of the live user traffic away from the old server cluster ("Blue") to prevent any user disruption.

3. LLM Systems (Large Language Models / GenAI)

• How it is Served: Optimized Token-Streaming Serving. LLMs are incredibly massive, requiring specialized serving engines (like vLLM or TGI) to dynamic-batch user inputs and stream words back to users one by one rather than making them wait for the entire paragraph.
• The CD Strategy: When a prompt template or model adapter is updated, the CD pipeline manages a Shadow Deployment. The system duplicates real live user requests behind the scenes and sends them to both the old and new prompt configurations simultaneously. The app only shows the user the old model's response, but it tests how the new model performs under actual server load before turning it live.

Applying MLOps Step 5 to dresscode

Because we are using an advanced multimodal foundation model like Gemma 4, "serving" doesn't just mean hosting a script—it means managing heavy visual context inputs and optimizing rapid token streaming so the user isn't stuck staring at a loading screen.

Apply to Use Case 1: Closet Image Uploads

Users will upload multiple photos at once when setting up their virtual closet. This requires high-throughput visual processing.

• Model Serving (Asynchronous Batching): Processing high-resolution images through Gemma 4's vision matrix takes heavy GPU computing power. Instead of making a user wait on a mobile screen, serve this model via an asynchronous queue system (like Celery + Redis). The user uploads 5 photos, the app says "Analyzing your clothes...", and the photos are processed sequentially in the background.
• The CD Strategy (Blue-Green Deployment): If we decide to upgrade the underlying vision prompt package or swap from the standard Gemma 4 model to a more lightweight quantized version (like a Q4 or 8-bit precision variant) to save cloud costs, our CD pipeline should use a Blue-Green layout. It spins up a fresh, separate GPU instance running the new model variant. Once it passes health checks, traffic is instantly rerouted to it, ensuring zero app downtime for users who are actively uploading photos.

Apply to Use Case 2: Weather & Outfit Recommendations

This agentic workflow requires lightning-fast interaction because the user is waiting to see their daily outfit suggestion in real-time.

• Model Serving (Token Streaming + Multi-Token Prediction): We shouldn't make the user wait for Gemma 4 to think, we should run a function call, and draft an entire paragraph before showing the answer. Serve the recommendation pipeline using an optimized LLM engine (like vLLM or Hugging Face TGI) with Token Streaming enabled. This spits out the outfit suggestion word-by-word instantly on the UI. Additionally, take advantage of Gemma 4's built-in Multi-Token Prediction (MTP) drafters to dramatically slash latency during generation.
• The CD Strategy (Shadow Deploying Prompts): Since changing a single line in a prompt or modifying the weather API JSON tool schema can cause unexpected text outcomes, do not deploy prompt updates directly to the public. Our CD pipeline should use a Shadow Deployment. When a developer updates the stylist prompt logic, the live system duplicates actual user requests and silently feeds them to both the old stable prompt and the new shadow prompt. The system validates that the new configuration executes the API function call cleanly under real-world traffic scenarios before we flip it live.

Step 6: Continuous Monitoring (The Health Dashboard)

Just because an AI model works perfectly on day one doesn't mean it will stay that way. Real-world conditions change, and models can degrade, slow down, or become less reliable over time.

Continuous Monitoring acts like a 24/7 heart monitor for our live AI system. It tracks three core elements:

• System Performance: Are the servers running fast, or are they lagging and eating up too much memory?
• Data Drift: Is the incoming real-world data shifting drastically from the data the model was originally trained on?
• Model Performance: Is the model's accuracy dropping or becoming less reliable as time goes on?

Sample Use Cases

Here is how continuous monitoring works across the three system types:

1. Sensor Data Clustering (IoT / Industrial Systems)

• What is Monitored: Incoming sensor scales, anomaly detection frequency, and hardware latency.
• The Drift/Failure Scenario: Over time, physical factory machinery naturally wears down, or a technician installs a new brand of sensor that measures vibration in a slightly different baseline unit.
• The Monitoring Action: The system triggers an alert if the statistical distribution of the sensor values shifts drastically (Data Drift). For example, if the average temperature reading across the factory suddenly climbs 10% without a real heatwave, the monitor flags it so engineers can check if a physical sensor is broken or if the model needs a calibration adjustment.

2. Image Processing (Computer Vision)

• What is Monitored: Image dimensions, file corruptions, lighting ratios, and model confidence thresholds.
• The Drift/Failure Scenario: A smartphone manufacturer pushes a software update that changes the default photo compression format or camera color profile, causing uploaded photos to look slightly washed out to the AI.
• The Monitoring Action: The system monitors the model's internal confidence scores. If the model usually tags objects with 92% confidence, but that average suddenly slips to 61% over a single week, the monitor catches the drop and alerts the engineering team before bad tags ruin the user experience.

3. LLM Systems (Large Language Models / GenAI)

• What is Monitored: Cost per query, response length, token generation latency (Time-to-First-Token), and formatting adherence.
• The Drift/Failure Scenario: Users start using completely new vocabulary or slang that wasn't prominent in the model’s original training data, or the LLM encounters adversarial "prompt injection" attacks trying to bypass its safety guidelines.
• The Monitoring Action: The pipeline acts as an automated auditor. It scans user prompt logs and model outputs using smaller, targeted evaluation models. If it detects a spike in toxic outputs, formatting failures (like returning corrupted JSON strings), or a sudden surge in response latency, it flags the operational bottleneck instantly.

Applying MLOps Step 6 to dresscode

Because we are using Gemma 4's native multimodal capabilities and agentic tool-use, monitoring isn't just about server uptime. It is about tracking formatting failures, tool-calling failures, API latency, and LLM output quality.

Apply to Use Case 1: Closet Image Uploads

For visual cataloging, monitoring ensures Gemma 4 continues to parse image contents accurately and output predictable, structured data profiles.

• Monitoring Formatting Compliance: Since our application backend expects a strict JSON array from Gemma 4 to catalog clothing pieces, setup a monitor to track JSON Parse Error Rates. If a prompt update causes Gemma to suddenly append friendly chatter (like "Here is your list!") instead of pure JSON, the system should catch the parsing failure immediately.
• Tracking Guardrail and Confidence Dips: Monitor the length of the lists being generated. If the average number of clothing items successfully detected per image suddenly drops drastically (Data Drift), it could mean users are uploading images with poor lighting, lower resolutions, or unexpected camera angles that are tripping up Gemma's vision processing matrix.
• Latency Monitoring: Track the Time-To-First-Token (TTFT) specifically for image payloads. Visual tokens take longer to compute; if processing latency spikes, it lets us know if our GPU cluster (or cloud provider endpoint) is throttling.

Apply to Use Case 2: Weather & Outfit Recommendations

This agentic workflow relies heavily on live integrations and reasoning logic. Monitoring here preserves the user's trust in the app's styling intelligence.

• Function-Calling Error Tracking: Since Gemma 4 natively structures external tool calls, we must continuously log and monitor Function Call Success Rates. We need to catch if Gemma 4 makes a formatting error when trying to request weather data, or if the external weather API itself drops, times out, or returns bad payloads.
• Semantic Constraint Monitoring (Hallucination Tracking): Set up a lightweight, deterministic verification layer to monitor the final output text against the provided wardrobe list. If the monitor catches that Gemma 4 recommended a "yellow raincoat" but the word "raincoat" is missing from that user's specific inventory list, it flags a hallucination event. High hallucination rates signal that we need to lower the model's temperature or tighten the context constraints in the prompt registry.
• Cost & Token Usage Tracking: Monitor the total token count per outfit generation. If token volume climbs unexpectedly, it means the model is getting caught in infinite loops during its multi-step agentic planning. Because Gemma 4 supports large context windows (up to 256K tokens), unmonitored prompt expansions can quickly lead to expensive API bills.

Step 7: The Feedback & Retraining Loop (The Growth Cycle)

The final step of the MLOps pipeline closes the circle. This is where our AI system learns from its real-world mistakes, updates its knowledge base, and gets smarter over time.

Think of it as giving a student their graded test papers back. If the AI never finds out when its predictions are wrong or right, it can never improve.

• The Feedback Loop: Capturing signals from the real world. This can be explicit feedback (like a user clicking a thumbs-down button or correcting an error) or implicit feedback (like a user ignoring a recommendation entirely).
• The Retraining Loop: Feeding that new real-world data and feedback back into Step 1 of your pipeline. The system automatically packages the new data, triggers a fresh model training session, evaluates if the new version is smarter, and updates production.