An AI agent's resolution rate is the share of requests the system considers resolved. To use it when choosing a vendor or deciding whether to continue a pilot, you need to know which requests count, what event qualifies as a resolution and how long the system waits before assigning an outcome.
Forty requests labelled as resolved can produce a rate of 66.7% or 80% with exactly the same outcomes. Changing the population used to calculate the percentage is enough. Vendor definitions explain why; an example with one hundred requests shows what to ask before comparing two figures.
Before comparing values, identify the metric
The denominator is the population used to calculate the ratio. It might include every request received, only those assigned to AI, or conversations in which the agent actually had an opportunity to answer. The unit matters too: one customer request can span several conversations.
In 2026, Intercom documented a change to Fin Involved: conversations in which Fin was active but had no opportunity to answer are excluded, for example because of a configured constraint. The resolution rate divides resolutions by involved conversations; the involvement rate divides those conversations by all conversations. In the cited update, the automation rate is resolutions as a share of all conversations. Excluding cases without an opportunity can therefore raise the resolution rate and lower the involvement rate, with no change in the number of resolutions, automation rate or billing. Fin metrics update.
An increase caused by this definition is insufficient evidence of better service. It may make the report better suited to a different question: how much does Fin resolve when it has an opportunity to answer?
Identify the product and account model before making the comparison. Zendesk's resolution tiers are categories the service uses to classify conversation outcomes. In that model, the reported rate includes Contained + Verified, but only Verified is billed. Contained is used when the customer asks for no clarification or human agent, gives no feedback on the answer, and the conversation does not pass the vendor's verification. Verified requires no further request for assistance and a conversation that passes this check, performed by an LLM, or large language model.
Verified does not necessarily mean the customer explicitly confirmed success. Accounts still on the pre-tier platform need to be distinguished from migrated accounts and from the separate legacy Essential feature: identify the account's actual model before choosing the applicable definition. Reporting changes, tiers and verification, identifying the account model.
For each percentage, ask for a short but complete definition:
| Field | Question to answer |
|---|---|
| Population and unit | Are requests, conversations or customers counted? Which cases are excluded? |
| Counted event | Is an answer, silence or an automated evaluation enough, or is evidence of the outcome required? |
| Window | Which event starts the clock? When does the result become observable? |
| Version | Which definition, account model and extraction date does the report describe? |
Closure, resolution and billing follow different rules
Intercom's catalogue describes an assumed resolution with an automatic closure after three minutes. Fin's pricing page for use with an external helpdesk instead specifies 24 hours after the last answer with no further interaction. These documents cover different product surfaces and purposes; they do not explain how the two timers relate operationally. A purchasing decision needs the rule that applies to the account in question. Intercom catalogue, Fin pricing and outcomes.
In Zendesk's tier model, the conversation-end window used for evaluation starts with the first email or message: 72 hours for email and two hours for messaging, with a documented extension available through support. Calling these periods hours of inactivity would change the event that starts the clock. Tier rules.
A billable outcome can also include a deliberately configured handoff to a person or workflow. Fin distinguishes these Procedure handoffs from default or workspace escalations. An outcome charge therefore cannot be read directly as a count of problems solved without human intervention. Intercom outcomes.
The same hundred cases, three different ratios
The following example uses entirely invented requests and labels. Its purpose is to check the reasoning and calculations; it does not reproduce Intercom's or Zendesk's classifiers or dashboards.
The cohort contains one hundred initial requests, each associated with one problem. AI has an opportunity to answer sixty of them. Forty receive a generic label resembling “resolved”. Separately, the dataset assigns twenty-eight successful autonomous outcomes with a complete observation window.
| Constructed quantity | Count |
|---|---|
| Initial requests, N | 100 |
| AI opportunities, E | 60 |
| “Resolved”-like labels, R | 40 |
| Successful autonomous outcomes with a complete window, G | 28 |
| Insufficient evidence within the AI scope | 4 |
| Follow-up still incomplete | 1 |
| Known non-autonomous cases within the AI scope | 27 |
The last four rows partition the sixty opportunities: 28 + 4 + 1 + 27. R is another classification of the same cases and must not be added to those rows. The forty requests outside the AI scope are not classified by this breakdown.
The ratios answer different questions:
- R/E = 40/60 = 66.7%: how many AI opportunities received the label?
- R/N = 40/100 = 40%: what share of initial requests received that label?
- G/N = 28/100 = 28%: what share received the successful autonomous outcome assigned in the example, with a complete window?
To see the effect of the scope, exclude ten opportunities that had neither label R nor outcome G. E falls to fifty: R/E rises to 80%, while R/N stays at 40% and G/N at 28%. AI coverage, E/N, falls from 60% to 50%.
The selection rule may make sense: eight of the ten excluded cases have a handoff accepted by the correct team within the example's deadline. The higher rate, however, describes a narrower population. A pilot decision also needs coverage and outcomes across all requests.
These ratios use one consistent teaching population. Fin's resolution reporting includes resolutions obtained through Custom Answers, including conversations resolved by a Procedure, whose reply is classified as a Custom Answer. Automation excludes Custom Answers and Procedure handoffs. Before combining the product's rates, check that their numerators and denominators cover the same categories. Fin catalogue definitions and conditions.
What evidence makes an outcome credible?
In the dataset, C015 has no customer confirmation but is assigned a positive outcome. C029 includes a confirmation followed by repeat contact about the same unresolved problem: the customer contacts support again. The visible signal and the evidence of the result can diverge in either direction.
A real pilot needs an advance definition of how to check the customer's goal: for example, verifying that a requested change was applied correctly. Where evidence is missing, the outcome remains unknown. The absence of further contact alone does not fill that gap. This is the same underlying problem as verifying an agent's claims: its account must match the checked result.
Time also needs an explicit measurement contract. In this example, window W is seven days from the first AI answer, a choice made for the teaching example. The cohort includes requests from 1 October and is frozen on 9 October. C060 receives its first answer only on 4 October at 11:00 UTC: its seven days end on 11 October at 11:00 UTC, so it remains under observation at the freeze date. Waiting seven days after collecting the requests does not give every request the same follow-up.
Repeat contacts are counted by initial request. Six reopenings and seven new conversations concern ten requests, because three appear in both groups: 6 + 7 − 3 = 10. The ratio 10/40 = 25% is the share of requests initially labelled R with repeat contact about the still-unresolved problem within W. It differs from Intercom's native recontact metrics, which have their own windows and denominators. Recontact metrics.
The ledger to request for a pilot
An auditable comparison keeps one row per initial request and links conversations and repeat contacts to that row. Defining the problem and joining records across channels takes work: uncertain links need review, alongside exporting the IDs.
| Information | What it lets you check |
|---|---|
| Request ID, customer/problem and linked conversations | Counting unit and repeat contacts about the same problem |
| Inclusion in the AI scope and reason for exclusion | Coverage and case selection |
| Native label and its version | Reconstruction of the vendor's report |
| Goal, evidence of the outcome and need for human intervention | Autonomous, non-autonomous or unknown outcome |
| First AI answer, end of window and freeze date | Complete or still-open follow-up |
| Team, handoff request and acceptance | Acceptance by the correct team within the agreed deadline |
Handoff acceptance needs to be measured separately from autonomous resolution. In the example, fifteen requests need a handoff; twelve reach the correct team and are accepted within 24 hours of the handoff request. The ratio 12/15 = 80% measures that acceptance, rather than the final resolution of the problem. This deadline is also a choice made for the example.
The export must include the denominator as well as billed events. Zendesk's documentation describes separate usage views and analytics reports; before reconstructing the rate, check which categories and periods the export available on the account contains. Usage, analytics export.
Before deciding whether to continue the pilot, agree on the criteria and sample selection. Have a subset of outcomes reviewed without showing the reviewer the vendor's label, recording disagreements, missing evidence and uncertain links. Keep checks on that sample separate from targeted checks on unknown outcomes and high-risk requests.
Alongside software usage, count the time spent configuring sources, correcting answers, assessing outcomes, handling escalations and repeat contacts, and maintaining the integration. A useful cost comparison concerns the work needed to achieve the service outcome. If a difference between percentages disappears when populations and criteria are aligned, it no longer supports the original ranking.
Reproduce the calculations
The two attachments contain the complete synthetic dataset and the program to recalculate it. Download them into the same folder; copy the Python block from the second document into a file named recalculate-support.py. You need Python 3.10 or later and the standard library.
From the folder containing the dataset and program:
python3 recalculate-support.py support-cohort.json --self-checkThe expected output reports N=100, E=60, R=40, G=28, four cases with insufficient evidence, one incomplete follow-up, ten requests with repeat contact, and twelve handoffs accepted by the correct team out of fifteen. The package's nine checks cover this arithmetic and the constructed labels, including duplicate repeat contacts and windows still open. The package was run with Python 3.15.2.
The distinction between denominators, resolution and deflection has also been discussed by Communicate. The contribution proposed here is to bring these questions together in an auditable ledger before deciding on a pilot.