Project Guide | Enterprise LLM Adoption Kit

Enterprise LLM Adoption Kit

Production-grade governance toolkit for enterprise LLM deployments -- RBAC, prompt-injection detection, PII redaction, audit logging, eval harness, and multi-cloud LLM routing, all wired into a single FastAPI + React application with Kubernetes-ready infrastructure.

All rollout data and review scenarios are synthetic. The goal is a reviewable, runnable validation kit that demonstrates enterprise-grade LLM governance patterns end to end.

Technical review pack: `docs/architecture-pack.md`

Demo video: https://youtu.be/yMq03b0js0E

---

Three-Minute Proof

1. Open `docs/architecture_assets/poc_success_criteria.md` to see the production test boundary.
2. Inspect the governance path: RBAC, prompt-injection checks, PII redaction, audit logging, and eval gates.
3. Run make verify to cover backend quality, smoke checks, and frontend build.
4. Keep synthetic data and environment-gated live providers explicit in any technical review.

System Overview

Evaluation Path

• Start here: Open the governance layers, eval gate, and PoC success criteria before the full architecture.
• Local demo: Run the backend and frontend quick-start commands, then open http://localhost:8000 and http://localhost:5173.
• Checks: Run make verify; it covers backend quality, smoke checks, and frontend build.

---

Architecture Notes

• Review guide summarizes the system scope, first files to inspect, verification commands, and known boundaries.
• Quality notes lists the local checks, CI surface, and release expectations for this repository.
• Enterprise readiness notes outlines security, data, operations, integration, and handoff expectations.

Architecture

Every request passes through four governance layers before reaching the LLM. Each layer can short-circuit the request with a policy refusal, and every decision is audit-logged.

flowchart TB
    subgraph Client["Client Layer"]
        UI["React + Vite UI"]
        CLI["API / CLI Consumer"]
    end

    subgraph Gateway["API Gateway"]
        ING["TLS Ingress"] --> AUTH["JWT / OIDC Auth"]
        AUTH --> RL["Rate Limiter"]
    end

    subgraph Governance["Governance Pipeline -- 4 Layers"]
        direction TB
        RBAC["1. RBAC Enforcement<br/><i>Employee / Ops / Admin</i>"]
        INJ["2. Prompt Injection Detection<br/><i>keyword + regex heuristics</i>"]
        PII["3. PII Redaction<br/><i>email, phone, ID masking</i>"]
        SAFE["4. Safety Policy Engine<br/><i>22 regex patterns, ReDoS-safe</i>"]
        RBAC --> INJ --> PII --> SAFE
    end

    subgraph Core["Application Core"]
        RAG["RAG Retrieval<br/><i>ChromaDB + hash embeddings</i>"]
        LLM["LLM Router<br/><i>OpenAI / Ollama / Bedrock / Stub</i>"]
        TOOLS["Tool Executor<br/><i>allowlisted tools only</i>"]
        RAG --> LLM
        LLM --> TOOLS
    end

    subgraph Observability["Observability + Persistence"]
        AUDIT["Audit Logger<br/><i>SHA-256 enterprise hashing</i>"]
        PROM["Prometheus Metrics<br/><i>latency, tokens, usage, policy</i>"]
        OTEL["OpenTelemetry<br/><i>OTLP traces</i>"]
        DD["Datadog Integration"]
    end

    subgraph DataPlatform["Data Platform Integration"]
        SF["Snowflake<br/><i>eval + audit persistence</i>"]
        DB["Databricks<br/><i>MLflow + Delta Lake</i>"]
        EVAL["Eval Harness<br/><i>baseline diffs, red-team</i>"]
    end

    UI & CLI --> ING
    RL --> RBAC
    SAFE --> RAG
    LLM --> AUDIT & PROM & OTEL
    AUDIT --> SF & DB
    OTEL --> DD
    EVAL --> SF & DB

    style Governance fill:#fff3e0,stroke:#e65100
    style Observability fill:#e8f5e9,stroke:#2e7d32
    style DataPlatform fill:#e3f2fd,stroke:#1565c0

---

Tech Stack

---

One-command demo (recommended)

git clone https://github.com/KIM3310/enterprise-llm-adoption-kit.git
cd enterprise-llm-adoption-kit
make demo-local        # auto-selects Ollama if available, otherwise stub

Manual setup

## 1. Backend
make backend-install
cd app/backend && .venv/bin/python -m app

## 2. Frontend (separate terminal)
cd app/frontend
npm install && npm run dev        # starts on http://localhost:5173


## 3. Verify everything
make verify                       # syntax, deps, pytest, smoke, frontend build

Docker

cd infra && docker-compose up --build

## Backend: http://localhost:8000   Frontend: http://localhost:5173

Kubernetes

kubectl create namespace llm-adoption
kubectl apply -f infra/k8s/secret.yaml
kubectl apply -f infra/k8s/

---

Key Capabilities

---

Core API

---

Snowflake / Databricks Integration

Snowflake -- set SNOWFLAKE_ACCOUNT to activate. Stores eval results and audit logs; supports query_eval_history(), query_audit_history(), aggregate reporting.

Databricks -- set DATABRICKS_HOST to activate. MLflow experiment tracking per eval run; Delta audit tables in Unity Catalog; databricks-cli or service-principal OAuth auth.

See `.env.example` for the full configuration surface.

---

Datadog-Ready Pack

• Datadog-ready resource pack: `docs/datadog/README.md`
• Existing env hooks already reserve a Datadog integration lane in .env.example
• Current state: asset sync and OTLP wiring are prepared, but live tenant integration is intentionally disabled by default
• Best use: show how enterprise LLM governance, audit, latency, and rollout readiness would be observed in one operator-facing Datadog surface

---

Project Structure

enterprise-llm-adoption-kit/
  app/
    backend/               # FastAPI backend (RBAC, RAG, safety, audit, LLM adapters)
      app/
        main.py            # FastAPI app with governance middleware
        rbac.py            # Role-to-access-group mapping
        safety.py          # 22-pattern safety policy engine
        injection.py       # Prompt injection detection
        redaction.py       # PII redaction engine
        audit.py           # Structured audit logging with SHA-256 hashing
        auth.py            # JWT/OIDC authentication
        llm_adapter.py     # Multi-provider LLM router with circuit breaker
        rag.py             # ChromaDB RAG with RBAC-filtered retrieval
        snowflake_adapter.py
        databricks_adapter.py
        metrics.py         # Prometheus metric definitions
        telemetry.py       # OpenTelemetry instrumentation
        config.py          # 60+ env vars, type-safe parsing
    frontend/              # React + Vite UI
  evals/
    runner/                # Eval harness (run_eval, eval_gate, baseline)
    datasets/              # Test datasets (initial, red-team, Korean)
    reports/               # Generated eval reports and diffs
  infra/
    k8s/                   # Kubernetes manifests (HPA, TLS, AlertManager)
    aws/terraform/         # AWS ECS + ALB Terraform module
    gcp/terraform/         # GCP Terraform module
    docker-compose.yml     # Local multi-service orchestration
    monitoring/            # Grafana dashboard + AlertManager rules
  tests/                   # 30+ test files, 84% backend coverage
  docs/                    # Architecture, ops, blueprint, Datadog, evals docs
  scripts/                 # Demo runners, quality gates, release ops

---

Technical Proof Boundary

• Primary architecture lane: enterprise LLM governance, applied AI controls, and platform integration
• Strongest proof surface: governance pipeline, eval harness, observability surfaces, and deployment-ready backend/frontend split
• What is real here: RBAC, safety pipeline, audit logging, metrics, integration adapters, CI/CD, and deployment scaffolding
• What is bounded here: review cases and documents are synthetic, and Snowflake / Databricks / Bedrock integrations are env-gated

---

Operating Commands

• make verify runs the backend syntax/dependency checks, Python suite, smoke diagnostics, and frontend production build.
• app/backend/.venv/bin/python -m pytest tests app/backend/tests -q exercises the backend services, adapters, runtime scorecards, and UI metadata contracts.
• cd app/frontend && pnpm build produces the static frontend bundle used by the Cloudflare Pages deployment path.

---

CI/CD

GHCR Docker image published on every push to main. Security scan (pip-audit, bandit, Trivy) runs on schedule. Frontend auto-deploys to Cloudflare Pages.

---

Related Projects

For governed NL-to-SQL analytics, see Nexus-Hive. For the data pipeline layer, see lakehouse-contract-lab.

Cloud + AI Architecture

• Cloud + AI architecture blueprint
• Machine-readable architecture manifest
• Validation command: python3 scripts/validate_architecture_blueprint.py

Enterprise Productization

• Product operating model defines the product scope, trust boundary, operating checks, and service path for this repository.

System Architecture

• System architecture maps the runtime boundary, data/control flow, cloud or local deployment surface, and operating assumptions for this repository.

Service Architecture

• Service architecture defines the cloud resources, account information, usage controls, and production guardrails needed to turn this repo into a scoped service without publishing public financial assumptions.

Search And Service Surface

• Public entry: public templates, service brief samples, and architecture docs
• Paid boundary: paid workshop pack, private control tower deployment, and governance evidence bundle
• Canonical URL: https://enterprise-llm-kit.pages.dev/
• Lead capture: https://kim3310-doeon-kim-portfolio.pages.dev/?offer=enterprise-llm-adoption-kit&inquiry=private-ai-readiness-sprint#private-inquiry
• Resource route: https://kim3310-doeon-kim-portfolio.pages.dev/resources/enterprise-llm-adoption-kit/
• Commercial route: https://kim3310-doeon-kim-portfolio.pages.dev/?offer=enterprise-llm-adoption-kit#service-offers
• Machine-readable offer: docs/service-offer.json
• Search growth implementation: docs/search-growth-implementation.md
• Revenue architecture: docs/revenue-architecture.md