How to Use Data From Your AI Tools to Improve Your Transformation Programme

For Microsoft 365 Copilot deployments, the following data is available through Microsoft admin tools:

Viva Insights: aggregate and individual (where privacy settings allow) data on Copilot feature usage, time saved estimates, meeting summary adoption, email drafting adoption, and document assistance usage. Viva Insights provides both individual dashboards (visible to the user themselves) and manager/HR dashboards (aggregate data for team and function level).

Microsoft 365 admin centre: Copilot usage reports showing active users by feature, adoption trends over time, and comparison across user groups. These reports provide the deployment-level view needed for programme management.

Azure OpenAI and Azure AI usage telemetry: for organisations using Azure AI services directly (beyond just Microsoft 365 Copilot), Azure Monitor and Azure Application Insights provide rich telemetry including request volumes, latency, error rates, and content safety events.

Microsoft Copilot Dashboard (in Viva Insights): a purpose-built programme management view showing Copilot adoption metrics, feature-level usage, and time saved estimates at the programme level. This is the primary dashboard for Copilot programme managers.

AI tool usage data enables diagnosis of adoption patterns that would otherwise require expensive primary research.

Adoption segmentation: which user groups are adopting at high rates, which at low rates, and which not at all? Usage data segmented by function, seniority, and location reveals the adoption pattern that targeted intervention should address. A pattern where senior leaders are high adopters but individual contributors are not suggests the champion network is not working at the right level; a pattern where one function is high adoption and another is not suggests use case relevance, not tool quality, is the adoption variable.

Feature usage patterns: which Copilot features are being used heavily and which are barely used? Heavy meeting summary adoption with low email drafting adoption suggests that training may need to focus more on email use cases. Heavy initial usage that drops after two weeks suggests habit formation is not occurring and reinforcement interventions are needed.

Time-of-day and day-of-week usage patterns: when are employees using AI tools? Patterns of AI use concentrated in low-pressure periods (early morning, Friday afternoons) rather than in high-pressure periods (meeting-heavy days, quarter-end) may suggest employees are using AI for low-stakes experimentation but not integrating it into their most demanding work.

Usage data should directly inform change management decisions throughout the programme.

Champion network effectiveness: compare adoption rates in teams with active champions against teams without. If champion-supported teams are not showing higher adoption, the champion model or the champion quality needs to be investigated. If champion-supported teams are significantly higher, invest in expanding the champion network as a priority.

Training effectiveness: adoption rates in the four to six weeks following training, by cohort, reveal which training interventions are producing adoption and which are not. Training cohorts that show strong initial adoption followed by rapid decline indicate that reinforcement, not initial training, is the gap. Training cohorts that show no initial adoption spike may indicate training content relevance issues.

Manager impact: compare adoption rates across teams managed by different managers. High variance between similar teams managed by different managers is one of the strongest signals that manager behaviour (not team characteristics) is driving adoption. This data, handled sensitively, enables targeted manager support investment.

AI tool usage data involves employee monitoring considerations that require explicit governance.

Microsoft's privacy framework for Viva Insights and Copilot usage data gives employees control over their individual data visibility to managers while allowing aggregate views for programme management purposes. Understanding and correctly configuring these privacy settings before using usage data is a governance requirement, not an optional consideration.

Union and works council notification: in UK organisations with recognised trade unions or European works councils, using AI tool usage data for performance monitoring or programme management may trigger information and consultation obligations. HR and legal should confirm the obligations before usage data is incorporated into programme management processes.

Communicating to employees how usage data is used: employees should know that their AI tool usage data may be used for programme management purposes (understanding adoption patterns, improving training) and should not be used for individual performance assessment. Transparency about data use reduces resistance to the programme management use of usage data.