Talking with another AI enthusiast (Ludy Bronsema), we wondered how Microsoft’s Copilot works from an infrastructure perspective and how it helps customers implement AI in different use cases. I mentioned that most of the communication from Microsoft focuses on the outcomes, not so much on the way to get there.
We left the call saying we should investigate and keep each other updated.
💡The following day, I got an idea!
📢Why not ask Copilot? Surely, Copilot should be able to tell how it can be implemented and what components and configurations it provides.
The idea of asking Copilot felt exciting, but it also made me aware that using 🤖AI isn’t fully ingrained in my way of working. It is still something I do intentionally instead of automatically.
I decided to create an imaginary use case and then ask Copilot if and how it could be implemented.
Here we start:
❓Question 1
I want to implement Microsoft Copilot in my organisation and create agents focused on the following users:
– Customer Support
– Sales
– Marketing
– Finance
These users will have access to my organisation’s data sources, while not all users will have access to all data sources.
For example, the Customer Support agent will have access to:
– FAQ,
– Product documentation (Text, video, images),
– Track and trace information,
– Live stock information.
The customer support agent will not have access to information on the internet or other external data sources.
Would Microsoft Copilot be able to support this use case? Please limit your answer to one sentence.
📢The answer from Microsoft Copilot:
‘’Yes, Microsoft Copilot can be customised to support this use case by integrating with your organisation’s data sources and limiting access based on user roles.’’
So, we are off to a good start.
❓Question 2
Can you give me a list of the services and components needed to configure this use case? Service – short description in one sentence.
📢The answer from Microsoft Copilot:
Sure, here are the services and components needed:
- Microsoft Copilot: An AI model that can generate responses based on user input and data sources.
- Data Integration Service: A service to connect and integrate your organisation’s data sources with Microsoft Copilot.
- Role-Based Access Control (RBAC): A system to manage access to data sources based on user roles.
- Natural Language Processing (NLP) Service: A service to process and understand the user’s language and context.
- User Interface (UI): A platform where users can interact with the AI agents.
- Monitoring and Analytics Service: A service to track the performance and usage of the AI agents.
- Security and Compliance Service: A service to ensure data privacy and compliance with regulations.
❓Question 3
If I was to implement this with a Microsoft only technology stack, what Microsoft services would you propose?
📢The answer from Microsoft Copilot:
Here are the Microsoft services that could be used to implement this use case:
- Microsoft Copilot: The AI model that generates responses based on user input and data sources.
- Azure Data Factory: A cloud-based data integration service that orchestrates and automates the movement and transformation of data.
- Azure Active Directory (Azure AD): Provides identity and access management services for your organisation’s data sources.
- Azure Cognitive Services: Offers a collection of APIs, SDKs, and services to help build intelligent applications without having direct AI or data science skills or knowledge.
- Microsoft Teams or Dynamics 365: Platforms where users can interact with the AI agents.
- Azure Monitor: A service that provides full-stack monitoring, collects and analyses telemetry from your cloud and on-premises environments.
- Azure Policy and Azure Security Center: Services that help you manage and enforce your organisation’s compliance standards.
❓Question 4
In the described use case, I mentioned that the Customer Support Agent would need access to multimodal Product documentation: Text, video, images. What tools do you propose to vectorise this data so it can be stored in multimodel vectorstore? Again, be specific and concise.
📢The answer from Microsoft Copilot:
For vectorising multimodal data, you could use:
- Azure Cognitive Search: For text data, it can extract, identify, and structure content.
- Azure Video Analyzer: For video data, it can extract insights through video processing.
- Azure Computer Vision: For image data, it can analyse visual content in different ways based on your needs. These services can help transform your data into a suitable format for a multimodal vector store.
❓Question 5
Zooming in the Customer Support Agent. What steps and accompanying tools should I take and use to implement this AI agent?
📢The answer from Microsoft Copilot:
Here are the steps and Microsoft tools you could use to implement the Customer Support AI agent:
1. Data Preparation: Use Azure Data Factory to gather and prepare your data sources (FAQs, product documentation, track and trace information, live stock information).
2. Data Vectorisation: Use Azure Cognitive Search for text data, Azure Video Analyzer for video data, and Azure Computer Vision for image data to vectorise and store this data in a multimodal vector store.
3. Role-Based Access Control: Set up Azure Active Directory (Azure AD) to manage access to data sources based on user roles.
4. AI Model Training: Train the Microsoft Copilot model with your prepared data.
5. Integration: Integrate the trained model with your customer support platform (like Microsoft Teams or Dynamics 365).
6. Monitoring and Improvement: Use Azure Monitor to track the performance and usage of the AI agent, and continuously improve the model based on feedback and analytics.
7. Security and Compliance: Ensure data privacy and compliance with regulations using Azure Policy and Azure Security Center.
Remember, each step involves specific configurations and settings that need to be adjusted according to your organisation’s specific needs and requirements. It’s recommended to work with a technical team or consultant familiar with these tools and processes.
❓Question 6:
I want to augment the query from the user with a prompt, so the Customer Support Agent responds appropriately. I don’t see that mentioned in the above steps. Can you explain with what Microsoft Tool I can do that? Again, be specific and concise.
📢The answer from Microsoft Copilot:
To augment the user query with a prompt, you can use Azure Bot Service in conjunction with Microsoft Copilot. Azure Bot Service allows you to manage conversation flow, handle interruptions, and dynamically change the conversation flow based on the context. This way, you can guide the AI agent to respond appropriately to user queries.
🤔Conclusion
This is where my investigative research stops. It took me less than 5 minutes and provided a ton of valuable information on implementing Microsoft Copilot and understanding how it can support organisations with RAG-based AI agents. It also confirms that Microsoft is definitely on the right track with Copilot.
❗❓But wait! I just watched the 2024 Microsoft Build Keynote recording mentioning Microsoft Copilot Studio as a key tool to create AI agents? And I don’t see that anywhere mentioned in the answer provided.
🤖🤔Let’s see if Microsoft Copilot can enlighten us.
❓❗Question 7:
Hi, I just watched the 2024 Microsoft Build keynotes with Satya Nadella, and he mentioned Microsoft Copilot Studio as a key tool for creating AI agents. And I don’t see that anywhere mentioned in the answers you provided. Can you explain why? Again, be specific and concise.
📢The answer from Microsoft Copilot:
My apologies for the oversight. Microsoft Copilot Studio, announced at Microsoft Build 2024, is indeed a key tool for creating AI agents. It allows you to create custom copilots or extend Microsoft Copilot experiences with your own enterprise data and scenarios1. The reason it wasn’t mentioned earlier is because the conversation was focused on the use of Microsoft Copilot and other Azure services. Microsoft Copilot Studio should certainly be included in the technology stack for implementing AI agents in your organisation. Thank you for bringing this to my attention.
👍👎The final conclusion
Asking Microsoft Copilot how to implement Microsoft Copilot can be a helpful way of starting your investigation. However, it’s not flawless and is best done by users with good knowledge of the technology. I bet Microsoft is more than happy to help customers and partners achieve the optimal implementation for your organisation.
💡Microsoft, when will we see a Microsoft Copilot Agent dedicated to helping organisations get the most out of their Microsoft Copilot implementation?