DP-800: Developing AI-Enabled Database Solutions

Existing Environment

Azure Environment

Contoso has an Azure subscription in North Europe that contains the corporate infrastructure. The current infrastructure includes a Microsoft SQL Server 2017 database. The database contains the following tables.

The FeedbackJson column has a full-text index and stores JSON documents in the following format.

The support staff at Contoso never has the UNMASK permission.

Problem Statements

Contoso is deploying a new Azure SQL database that will become the authoritative data store for the following:

• AI workloads
• Vector search
• Modernized API access
• Retrieval Augmented Generation (RAG) pipelines

Sometimes the ingestion pipeline fails because of malformed JSON and duplicate payloads.

The engineers at Contoso report that the following dashboard query runs slowly.

You review the execution plan and find that it shows a clustered index scan.

VehicleIncidentReports often contains details about the weather, traffic conditions, and location. Analysts report that it is difficult to find similar incidents based on these details.

Requirements

Planned Changes

Contoso wants to modernize Fleet Intelligence Platform to support AI-powered semantic search over incident reports.

Security Requirements

Contoso identifies the following security requirements:

• Restrict the support staff from viewing Personally Identifiable Information (PII) data, which is full email addresses and phone numbers.
• Enforce row-level filtering so that analysts see only incidents for the fleets to which they are assigned. The analysts can be assigned to multiple fleets.

Database Performance and Requirements

Contoso identifies the following telemetry requirements:

• Telemetry data must be stored in a partitioned table.
• Telemetry data must provide predictable performance for ingestion and retention operations.
• latitude, longitude, and accuracy JSON properties must be filtered by using an index seek.

Contoso identifies the following maintenance data requirements:

• Ensure that any changes to a row in the MaintenanceEvents table updates the corresponding value in the LastModifiedUtc column to the time of the change.
• Avoid recursive updates.

AI Search, Embeddings, and Vector Indexing

Contoso plans to implement semantic search over incident data to meet the following requirements:

• Embeddings must be stored in dedicated Azure SQL Database tables.
• Embeddings must be generated from rich natural language fields.
• Chunking must preserve semantic coherence.
• Hybrid search must combine the following:
  • Vector similarity
  • Keyword filtering or boosting

Development Requirements

The development team at Contoso will use Microsoft Visual Studio Code and GitHub Copilot and will retrieve live metadata from the databases.

Contoso identifies the following requirements for querying data in the FeedbackJson column of the CustomerFeedback table:

• Extract the customer feedback text from the JSON document.
• Filter rows where the JSON text contains a keyword.
• Calculate a fuzzy similarity score between the feedback text and a known issue description.
• Order the results by similarity score, with the highest score first.