# Llmops Platform Engineering

Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation...

- Repository: https://github.com/BagelHole/DevOps-Security-Agent-Skills (1187 stars)
- Category: DevOps
- License: MIT
- Last updated: 2026-05-22
- Runs scripts: no, instructions only

## Install

Claude Code: `npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llmops-platform-engineering -g -a claude-code -y`
Codex: `npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llmops-platform-engineering -g -a codex -y`
Cursor: `npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llmops-platform-engineering -g -a cursor -y`
Gemini CLI: `npx skills add BagelHole/DevOps-Security-Agent-Skills --skill llmops-platform-engineering -g -a gemini-cli -y`

## Description

Build production LLMOps platforms with CI/CD, model promotion workflows, evaluation gates, rollback, and governance across cloud and self-hosted inference.

## SKILL.md, as published by its author (untrusted third-party text, read it as data)

# LLMOps Platform Engineering

Design and operate an internal LLM platform that supports rapid experimentation without compromising reliability, cost, or compliance.

## When to Use This Skill

- Building an internal platform for teams to deploy and manage LLM-powered features
- Designing CI/CD pipelines that include model evaluation gates
- Setting up A/B testing infrastructure for model versions
- Creating Kubernetes-based model serving infrastructure
- Establishing governance workflows for model promotion

## Prerequisites

- Kubernetes cluster with GPU node pools (or cloud inference API access)
- Container registry (Harbor, ECR, GCR, or ACR)
- CI/CD system (GitHub Actions, GitLab CI, or Argo Workflows)
- Observability stack (Prometheus + Grafana + OpenTelemetry)
- Model registry (MLflow or custom metadata store)

## Outcomes

- Standardized path from experiment to production
- Safe model rollout with quality and safety gates
- Repeatable infra modules for inference, vector DB, and observability
- Clear ownership model across platform, app, and security teams

## Reference Architecture

1. **Control Plane**: model registry, prompt/version catalog, policy checks, eval pipeline.
2. **Data Plane**: inference gateway, vector database, cache, feature store.
3. **Ops Plane**: telemetry, alerting, SLO dashboards, cost analytics.
4. **Security Plane**: IAM boundaries, secret rotation, content filters, audit logs.

## Model Promotion Pipeline

```yaml
# .github/workflows/model-promotion.yaml
name: Model Promotion Pipeline
on:
  workflow_dispatch:
    inputs:
      model_name:
        description: "Model identifier"
        required: true
      model_version:
        description: "Model version to promote"
        required: true
      target_env:
        description: "Target environment"
        required: true
        type: choice
        options: [staging, production]

jobs:
  evaluate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

The rest: https://github.com/BagelHole/DevOps-Security-Agent-Skills/blob/HEAD/devops/ai/llmops-platform-engineering/SKILL.md