AI-103: Developing AI Apps and Agents on Azure

By Microsoft · Question Mode

QuestionQ63

VerifiedImplement generative AI and agentic solutions

You are creating an agent workflow in a Microsoft Foundry project to support natural voice interactions.

The agent must receive continuous audio input, convert that input into text for reasoning, and return spoken responses to the user. The workflow must meet the following requirements:

  • Support turn-taking dynamics, where the agent can begin generating speech output before the user finishes speaking
  • Operate with low latency to maintain a conversational experience

You need to enable both speech-to-text and text-to-speech in a real-time agent interaction.

What should you do?

Explanation

Real-time speech to text continuously converts streaming audio into text with low latency as the user speaks, while text to speech synthesizes the agent's textual response into audio; used together, this pairing supports the low-latency, turn-taking conversational dynamics the scenario requires, including the agent beginning to respond before the user has fully finished speaking. Batch transcription processes recorded files asynchronously and cannot support live turn-taking, an embeddings model encodes data for similarity/retrieval rather than performing transcription or synthesis, and speech translation changes the spoken language rather than enabling a spoken response to be generated.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!