AI-103: Developing AI Apps and Agents on Azure

By Microsoft · Question Mode

QuestionQ88

VerifiedImplement generative AI and agentic solutions

You have a web app named App1 that sends requests to a multimodal chat model deployment in a Microsoft Foundry project.

User messages can contain both text and images.

Currently, App1 embeds the image URL as plain text within the message content, so the model cannot recognize it as an image.

Traces show that the requests contain a single text message rather than a multimodal content array.

You need to send the message as a structured array that includes both the text portion and the image reference, so that the model can process the image correctly.

What should you do?

Explanation

Microsoft Foundry (Azure OpenAI) chat completion models that support vision expect the user message's 'content' field to be an array of content-part objects rather than a plain string when the message includes both text and images. Each element in the array must specify a 'type': one element with type 'text' containing the text field, and another with type 'image_url' containing an 'image_url' object with the image's URL (or base64 data URI). This structured format allows the model to distinguish text from image references and process the image correctly, as documented in Microsoft's guidance on using image and audio input in chat completions with Foundry Models.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!