AI Feature Progressive Rollout

Problem Statement

A fintech company is launching an AI-powered fraud detection feature. The model is performing well in offline evaluation, but the consequences of false positives (blocking legitimate transactions) and false negatives (missing fraud) are severe. You need a rollout plan.

Scope

• •Design a progressive rollout strategy (shadow mode → canary → gradual rollout)
• •Define rollback criteria and procedures
• •Create a monitoring dashboard spec
• •Write the stakeholder communication plan
• •Design the human-in-the-loop escalation workflow

Evaluation Rubric

• •Rollout strategy sophistication
• •Risk identification and mitigation
• •Monitoring and alerting design
• •Stakeholder communication quality
• •Rollback plan completeness