QuestionQ48

VerifiedDeploy AI-powered business solutions

Contoso, Ltd. is a high-tech manufacturing company that uses Microsoft Dynamics 365 Finance, Dynamics 365 Supply Chain Management, and Dynamics 365 Commerce for North American operations. It designs innovative products with many patents and proprietary technologies. The patents and engineering designs are closely protected secrets.

Contoso executives plan to integrate and adopt AI solutions to help scale the company for an anticipated period of rapid growth. The company has multiple legal entities and Azure subscriptions that will be used by the adopted AI solutions.

The following executives have specific responsibilities for the overall AI adoption:

  • Chief Technology Officer (CTO): Select one prebuilt AI agent for Dynamics 365 Finance, Dynamics 365 Supply Chain Management, or Dynamics 365 Commerce, and one custom Microsoft Copilot Studio AI agent, to prioritize and deploy during the initial adoption phase.
  • Chief Information Officer (CIO): Ensure appropriate security labels are assigned to data used by the AI agents.
  • Chief Financial Officer (CFO): Analyze the return on investment (ROI) for the AI agents being deployed.
  • Chief Information Security Officer (CISO): Discover and inventory AI resources for auditing.
  • Chief Executive Officer (CEO): Ensure all solutions follow industry-standard responsible AI practices.

Every AI initiative and agent will have a detailed business use case, a defined audience profile, and an estimated ROI that compares cost savings from the current process with estimated costs of the new AI solutions.

The research and development (R&D) department already has a custom Model Context Protocol (MCP) server containing comprehensive product specifications and compliance data.

The CTO has not yet chosen a prebuilt agent for Dynamics 365 Supply Chain Management and wants to view available agent templates to determine which agent provides the greatest business value. Depending on the high-priority AI agents identified, their capabilities must be previewed during a discovery meeting with the relevant business-operation stakeholders.

Contoso has identified these requirements for a custom AI agent:

  • It will use Dynamics 365 Supply Chain Management data to answer manufacturing-team questions as a low-code solution.
  • It will be available from within Microsoft Teams.
  • It must be designed to eventually connect to other agents selected according to their description.
  • Its topics will be selected based on a description of the query’s purpose, rather than a trigger phrase, to make interactions more conversational.
  • It must answer questions about product specifications by using existing technologies. The R&D department maintains the product specifications.
  • It must be integrated with, and accessible from, Dynamics 365 Supply Chain Management.
  • It must use Dynamics 365 Supply Chain Management business logic stored outside the application.

Contoso has the following analysis, reporting, and troubleshooting requirements:

  • The CISO will audit all AI solutions monthly for compliance and security.
  • The CFO will analyze all AI solutions quarterly to compare estimated ROI with actual measured efficiencies and adoption. The CFO will use the Copilot Studio agent usage estimator for this analysis.
  • The CISO wants to determine how much sensitive data was accessed in a given AI agent run and who accessed it. Excessive sensitive-data access by one user might indicate a high security risk.
  • The CTO wants to track user feedback on the quality of AI agent responses during interactions. Consistently poor feedback will initiate an escalated reengineering discussion.
  • The CEO wants a quarterly assessment of every required metric for the executives’ specific responsibilities. The assessment tools must be Microsoft-recommended and verify reliability, interpretability, fairness, and compliance.
  • The CFO wants to determine how many daily AI-agent interactions are abandoned compared with resolved conversations. Too many abandoned sessions might mean Copilot Studio credits are used inefficiently by end users.

Which Copilot Studio analytics metrics should you recommend to assist the executives with their specific responsibilities? Each metric may be used once, more than once, or not at all.

Drag & Drop
CFO concerns about Copilot Studio credit usage:
CTO concerns about poor feedback on AI agent responses:
Explanation

Effectiveness analytics track conversation outcomes, including whether conversations are resolved or abandoned; this identifies inefficient credit consumption from abandoned sessions. Satisfaction analytics capture user reactions and feedback on agent responses, which supports monitoring response-quality concerns.

Learn more

Community Discussion

No comments yet. Be the first to start the discussion!