edgeacademyCUSTOMER ENGINEERING
MODULE 07

AI-Native Engineering and Cloudflare AI

Design useful AI applications with explicit quality, authority, privacy, cost, and recovery boundaries.

ArchitectBuildDebugCommunicate
PRACTICAL EXERCISE · MODULE 7

Design a tenant-safe runbook assistant

A fictional SaaS company wants an assistant using public docs and tenant-specific incident notes. It must never reveal another tenant's data and must handle missing evidence and provider outages.

Work through the problem

  • Draw ingestion, query, authorization, and inference flows.
  • Select candidate Workers, storage, Vectorize, Workers AI, and AI Gateway roles; justify omissions.
  • Write five evaluation cases including cross-tenant retrieval, missing evidence, stale sources, provider timeout, and a valid answer.
  • Define model and prompt versions, cache boundaries, spending limits, and fallback constraints.
  • Write a release gate and a human escalation path.

Deliverables

  • Architecture and threat-boundary diagram
  • Evaluation matrix and acceptance criteria
  • Failure policy and cost measurement plan

Self-review rubric

  • Authorization enforced outside the model
  • Retrieval and generation assessed separately
  • No unsupported provider or feature promises
  • Clear abstention, fallback, and human review behavior
Compare with the worked approach

Authenticate before retrieval, enforce tenant and document permissions, preserve source versions, and assemble bounded context. Evaluate supported inference options on the same cases. Observe requests with privacy-aware metadata, use scoped caches only where safe, and bound retries and consumption. Block release on isolation failures; use fixtures for control flow and semantic evaluations for answer quality. All business gains remain hypotheses until measured.

Sources and context