Almost every organisation that bought assistant licences this year is reporting adoption as a percentage of seats activated, and almost every one of those numbers is meaningless. A licence assigned is an administrative act. A person who opened the tool once in March and has not returned is counted identically to the analyst who now begins every task there. The result is a set of board reports showing seventy per cent adoption alongside a finance team that cannot name a single process that changed.
A licence count tells you what you bought. It tells you nothing about whether anyone's work is different, which was the only reason to buy it
Here is what to measure instead, and how to keep the measurement honest.
The four signals that actually indicate adoption
Repeat use by the same individuals over time. Not how many people tried it — how many are still using it eight weeks later. This single number separates genuine adoption from curiosity in every deployment I have seen. Depth of use per person. Someone who uses an assistant twice a day for real work is a different phenomenon from someone who uses it twice a month. Averages across the population hide this completely; look at the distribution. Concentration by task type. Where is it actually being used — drafting, summarising, analysis, code, search? The answer tells you which capability to extend and which training to stop paying for. Whether an output reached a real destination. A draft that became a sent email, a published document, a committed change. Usage that never leaves the assistant window is practice, not production.
The measures that mislead
Licences activated. Purchasing, not behaviour. Total queries. Inflated by a small number of heavy users and by people experimenting. Rises fastest during the period when nothing is being accomplished. Self-reported time saved. Unreliable in a specific direction: people asked whether a tool their employer bought saves them time say yes. The number is not a measurement, it is a social response. Satisfaction scores. Measure how people feel about the tool, which is worth knowing and is not adoption.
Connecting usage to outcomes, carefully
The honest position is that attributing business outcomes to assistant use is very hard, because the deployment is not a controlled experiment and everything else changed too. What you can do is narrower and more useful: pick two or three processes with a measurable cycle time, establish where in those processes the assistant is used, and track the cycle time. If it moves, you have evidence about that process. If it does not, you have learned something more valuable than a survey. Resist the temptation to extrapolate. One process improving does not license a claim about organisational productivity, and the inflated claims made in the first year of this technology are the reason the second year gets scepticism.
Practical Guidance for AI Adoption Assessment
- Report eight-week retained users, not licences activated.
- Look at the usage distribution, never the average.
- Segment by task type to direct further investment.
- Count outputs that reached a destination, not sessions.
- Stop reporting self-estimated time saved.
- Instrument two or three processes with real cycle times.
- Interview the heaviest users; they know what works.
- Track who stopped, and ask why.
The Regional Angle
The first measurement problem specific to this region is language, and it is usually invisible in aggregate reporting. Assistant quality in Arabic lags English, which means adoption in a bilingual regional organisation splits along the language of the work rather than along function or seniority. A single organisational adoption figure will average a strongly adopting English-language commercial team with an Arabic-language government relations or legal function that tried it and stopped. Segment by working language before drawing any conclusion, because the remedy for low adoption differs entirely between the two groups — training for one, a better model or a different tool for the other. The second concerns what people here will tell you when you ask. Gulf workplaces combine a high proportion of expatriate staff on employment-linked residency with a strong cultural preference for affirmative responses to management enquiries, which makes voluntary self-reporting about an employer initiative particularly unreliable. Asking staff whether the assistant saves them time will produce a favourable number in almost any circumstance, and it should not be put in a board pack. Use observed behaviour, and where you do want qualitative input, get it from small structured interviews with the heavy users rather than a broad survey. The third is about the structure of regional operations and where measurement is easiest to get right. Many Gulf groups run shared service centres handling the back office for multiple entities, and those functions have real volumes, repeated tasks and measurable cycle times — precisely the conditions under which assistant impact can be observed rather than asserted. Start the instrumentation there instead of in the corporate functions where senior enthusiasm is highest, because the shared service centre will give you a defensible number in a quarter and the head office will give you anecdotes for a year.
The objection worth taking seriously
The strongest objection is that this measurement programme costs more than the information is worth. The tooling is new, the value is diffuse, and demanding rigorous evidence at this stage risks killing a capability that is obviously useful — nobody ran cycle-time studies to justify email or the spreadsheet, and organisations that insisted on proving the return before adopting them simply arrived late. Instrumenting processes, running retention cohorts and interviewing users is a research programme layered onto a tool people either use or do not. That is right about the risk of over-measuring, and the analogy to earlier general-purpose tools has force. The difference is the money and the renewal. Email was cheap and universal; assistant licences are a recurring per-seat cost at a scale that shows up in the technology budget, and someone will ask at renewal what the return was. An organisation that cannot answer will either renew on faith or cut on instinct, and both are bad decisions made blind. Note also that the recommendation here is deliberately thin — one retention number, one distribution, and two instrumented processes. That is a week of work quarterly, not a research programme, and it is the minimum required to make the renewal conversation something other than a debate about impressions.
Common Questions
What retention rate indicates real adoption?
Less important than the trend. A cohort figure that holds or rises over successive months indicates genuine embedding; one that decays after the launch period indicates curiosity, whatever its absolute level.
Should we mandate use?
No, but you should remove the obstacles the heavy users describe. Mandated usage produces usage metrics and not much else.
How do we handle people who stopped?
Ask them. Lapsed users are the most informative population in the deployment and nobody ever contacts them.
What should we expect over the next twelve months?
Expect vendors to ship better native usage analytics, mostly measuring the things that flatter the product. Expect boards to start asking for evidence rather than activation rates. Expect adoption to remain concentrated in a minority of heavy users in most organisations. And expect the credible impact claims to come from narrow instrumented processes rather than organisation-wide estimates.
AI Adoption Assessment — we replace the activation percentage with two numbers you can defend at renewal.