Microsoft 365 Copilot: How SMEs can measure the value of a pilot

Buy a few Copilot licences, run an introductory session and ask everyone a month later whether they are happy: this is a familiar approach, but it is not yet a credible pilot. Enthusiasm after an impressive demonstration tells you little about whether everyday work improves. Equally, one disappointing initial response does not prove that the technology is unsuitable for your company.

My recommendation for Swiss SMEs is to begin with a few recurring tasks, a small mixed group and an agreed decision at the end. After four weeks, you should know where Copilot helps, how much review remains necessary and which roles benefit from continued use. The number of assigned licences is not a useful success measure for this purpose.

This article combines Microsoft documentation checked on 25 September 2026 with an independently proposed approach. The schedule, metrics and financial examples are recommendations or explicitly hypothetical assumptions. They are neither vendor promises nor findings from a customer engagement. Product scope and available functionality must be checked in your specific tenant.

“We are testing Copilot” is too vague. Are you drafting text in chat, processing a supplied document or using assistance within a Microsoft 365 application? Which information should the tool use? Record the interface, account, licence and permitted data sources for every scenario. Otherwise, participants will compare different capabilities and eventually discuss different products as though they were identical.

Microsoft’s setup guidance describes a staged approach comprising a pilot group, wider deployment and ongoing operations. This does not mean that a successful test obliges you to equip every employee immediately. A useful pilot may conclude that only particular roles or activities benefit.

Do not begin by purchasing the largest possible licence bundle. Ask your provider to confirm which existing and additional licences the selected scenarios require, the contractual term and any other potential charges. A four-week evaluation does not automatically mean that every subscription purchased for it can be cancelled after four weeks. Agree the commercial boundaries before starting.

Choose activities with a clear input and a verifiable output. An initial draft, a structured summary or finding an answer supported by a source makes a better test than the open request “Improve our strategy”. The activities should genuinely recur several times during the pilot. Rare exceptions provide little evidence for comparison.

This table helps select scenarios; it does not promise that every capability is available under every licence. Define what lies outside the pilot as well, such as automatic sending, binding contractual advice or autonomous changes in business systems. Initially, a person with appropriate subject knowledge remains involved before the output is used.

A scenario is fully defined only when someone can decide whether the result is usable. “Sounds professional” is insufficient. Acceptance for a customer reply might require all three questions to be answered, no additional commitment, the correct salutation and a maximum of 180 words. These conditions also improve the instructions you give the AI.

According to Microsoft, Copilot respects existing access permissions. That does not mean those permissions are appropriate for the business. An excessively shared folder remains a problem. The data-readiness documentation therefore addresses sharing and protective measures. Information used in the pilot should have an identified owner and permissions that you can explain.

Microsoft also states in its privacy and security documentation that prompts, responses and data retrieved through Microsoft Graph are not used to train foundation language models. This statement does not replace an assessment of your particular processing, extensions or internal requirements. Nor is it blanket permission to use any confidential information in any scenario.

A clearly bounded set of approved information is a practical starting point. Check that it is current and exclude contradictory drafts from the test material. Participants also need to know which content they may use and how to report information that unexpectedly becomes visible. Fix access problems rather than celebrating them as impressive search capabilities.

The relevant timer does not stop when an answer appears on screen. It stops when the result is suitable for use. Include task clarification, input, waiting, review and rework. If producing a draft takes two minutes and correcting it takes twenty, an evaluation that records only the first two minutes is misleading.

First collect several comparable cases without AI assistance. Then have participants perform similarly difficult tasks with Copilot. Where practical, alternate the order and document differences in scope or complexity. Someone completing the same text for a second time may be faster simply because it is familiar. A small sample offers direction rather than scientific proof of causality.

A simple spreadsheet is sufficient: task, difficulty, time without assistance, time with assistance including review, quality assessment and a short comment. Look at the median and range instead of relying exclusively on the average. One spectacular success should not obscure ten disappointing cases. Equally, an unusual exception should not single-handedly disqualify an otherwise useful workflow.

Explain openly that you are evaluating a process rather than ranking employees. Collect only the information necessary for that purpose. Aggregated results by task are usually more helpful for a management decision than detailed profiles of individual usage. People need to be able to report a poor result without feeling that they have failed an assessment.

For planning purposes, assume ten participants, twenty working days and an average of eight minutes of net time released per person per day. Net means that subject-matter review and routine correction have already been deducted. This produces about 26.7 hours of additional monthly capacity. At assumed fully loaded costs of CHF 80 per hour, its calculated value is approximately CHF 2,133.

Every number is a deliberately chosen calculation assumption, not a current Microsoft price or a quotation. Starting effort in this example includes technical preparation, training and evaluation. Additional consulting, consumption-based services, further licences and taxes are excluded. Replace the inputs with your actual costs and avoid counting the same effort twice.

Most importantly, CHF 2,133 in capacity value does not mean CHF 2,133 less payroll expenditure. The released time becomes economically useful when it enables more customer requests to be handled, avoids overtime or brings important work forward. If the time remains unused, the financial benefit is substantially smaller. Always ask what the organisation will do with the additional capacity.

A sensitivity calculation helps prevent wishful thinking. At two rather than eight minutes per day, the capacity value is about CHF 533. Against assumed recurring costs of CHF 620, the balance becomes negative by approximately CHF 87, before starting effort. Under these assumptions, the recurring mathematical break-even point is about 2.3 minutes daily per person. Treat this as a question to test, not a guaranteed saving target.

Keep the pilot’s learning cost separate from the likely steady state. Initial training can be worthwhile even if the first month has a negative balance. However, that is a reason to state the investment transparently, not to remove it from the calculation or label theoretical capacity as cash already saved.

Microsoft’s rollout guidance recommends considering feedback, usage and business impact together. An SME can translate that into a manageable evaluation. The four weeks below are our organisational recommendation, not a vendor-mandated evaluation period.

• Week 1: Appoint an owner, select three tasks, collect baseline observations and review data permissions. Management confirms the budget and decision criteria.
• Week 2: Introduce participants using their own work. Review initial results together and document effective task instructions.
• Week 3: Repeat comparable cases. Include disappointing attempts. Use a short support session to remove specific obstacles.
• Week 4: Evaluate quality, time and costs. Decide separately for each use case whether to expand, improve a specific issue or stop.

Do not select only enthusiastic AI users. A small group with different responsibilities and experience levels is more likely to reveal where assistance is necessary. Reserve actual calendar time for training and feedback. Asking people to squeeze the pilot into an already full workload may primarily measure their lack of available time.

If too few comparable cases occur within four weeks, extend observation for that particular scenario. Do not retrospectively change the success criteria simply to make the outcome look better. Write down the unresolved question and the additional cases needed to answer it. An extension should have a purpose, not merely postpone an uncomfortable decision.

A pilot needs business ownership. IT provides access, operational support and protection; the business team assesses quality and practical value. One named person should bring these perspectives together and prepare the decision. Without that responsibility, feedback can remain scattered between support, management and a handful of motivated employees.

Agree three possible outcomes in advance. Expansion makes sense when usable results recur, value is credible and no unresolved critical protection issue remains. Targeted improvement is appropriate when a specific obstacle prevents value. Stopping is reasonable when an adequate learning period still produces no defensible advantage. Ending one use case does not mean the company has failed at AI.

When transferring a successful scenario into normal operations, include a contact person, short working instructions and periodic review. Changes to a feature, document collection or process may also change its value. The pilot decision is therefore a starting point, not a permanent certificate of success. Make it easy for employees to report that something which previously worked has become less dependable.

Keep the final decision brief enough to be used: one page with the scenarios, evidence, costs, unresolved limitations and the person accountable for the next step. A folder full of screenshots and enthusiastic comments is less useful than a clear decision about which work should change on Monday morning.

The better opening question is not “How many Copilot licences do we need?” Ask instead: “Which recurring activity do we want to improve, and how will we recognise the difference?” Three verifiable scenarios and an honest cost calculation create a stronger basis for decisions than enabling access across the organisation without a defined objective.

Start small enough to recognise mistakes and specifically enough to reach a decision. ZUMENTIQ helps connect technical prerequisites, practical acceptance and economic value.