AI-200: Developing AI Cloud Solutions on AzureDemo

Background

Fabrikam Inc. is a global retail analytics company that delivers AI-driven demand forecasting and product recommendation services to online retailers. The company is modernizing its solution to run entirely on Microsoft Azure.

The platform ingests transaction data, creates embeddings for semantic retrieval, performs vector similarity search, and returns product recommendations through containerized microservices. Developers use Python and Azure SDKs. Operations teams manage container orchestration, scaling, monitoring, and security.

The solution must satisfy strict performance, scalability, and security requirements.

Current environment

Application architecture

• The Recommendation engine is a customer-facing HTTP API that runs as a containerized Python application.
• The engine is deployed to Azure Container Apps (ACA).
• Embeddings are stored in Azure Database for PostgreSQL by using pgvector.
• Semantic retrieval uses metadata filtering together with vector similarity search.
• Azure Managed Redis is used as a caching layer.
• Front-end and API workloads are deployed to Azure Container Apps (ACA).
• Batch model retraining workloads run in Azure Kubernetes Service (AKS).

Container and CI/CD

• Container images are stored in Azure Container Registry (ACR).
• CI/CD uses ACR Tasks to build images on commit.
• ACA environments support revision management.
• AKS workloads are deployed by using Kubernetes manifest files stored in Git.

Monitoring

• Logs are collected in Azure Monitor.
• Teams inspect container logs and Kubernetes events when troubleshooting.
• Developers write KQL queries to analyze latency spikes.

Business requirements

• Customer experience: Maintain a seamless, low-latency recommendation experience for end users, even during unpredictable seasonal traffic spikes.
• Operational cost efficiency: Minimize compute costs by deallocating resources during periods of inactivity and by preventing runaway scaling costs.
• Data integrity and freshness: Ensure that product recommendations always reflect the most current catalog metadata and pricing to prevent customer dissatisfaction.
• Security and compliance: Follow a Zero Trust security model by removing long-lived credentials and centralizing the management of all sensitive secrets.
• Global scalability: Support the rapid ingestion of millions of new product embeddings daily without degrading query performance for existing retailers.

Technical requirements

• Performance: Semantic search latency must stay under 200 milliseconds at peak load.
• Database optimization: Use pgvector for embeddings and implement metadata filtering to reduce compute overhead. Configure compute and memory appropriately for vector workloads to ensure high-dimensional index residency in RAM and efficient mathematical throughput. Vector similarity calculations must be performed only against products that satisfy mandatory metadata constraints.
• Database performance: Database connections must support high concurrency with minimal latency through the implementation of connection optimization.
• Data load strategy: To ensure maximum ingestion throughput, secondary indexes must be applied only after bulk loading of embeddings is complete.
• Caching: Redis cache entries must expire automatically after 10 minutes. Implement a reactive mechanism to invalidate cache entries when metadata updates occur.
• Identity: Use managed identities for all service-to-service and service-to-database authentication. Plain-text credentials in configuration files are strictly prohibited.
• Secret management: All secrets must be stored centrally. Secrets must be rotated automatically by using a centralized lifecycle policy.
• Scaling: Use Kubernetes event-driven autoscaling (KEDA) for event-driven scaling. The Recommendation API must scale based on HTTP traffic, while batch jobs must scale based on queue length and support scale-to-zero.
• CI/CD: All images must be stored in Azure Container Registry. Use ACR Tasks to automate image builds triggered by source code commits.
• Monitoring: Use KQL to analyze performance telemetry and troubleshoot microservice connectivity failures. Inspect logs and events when troubleshooting AKS and ACA.