What happens after a software evaluation?

By the company's next interview, about ten months later on median, about half of evaluations had reached a new-tool outcome: a product selected, being implemented or in use. About a quarter of those were not among the options named at the first interview. Others were still open, and some companies were still on their previous tool with no confirmed switch. The evaluation's stage and who answered the follow-up were both associated with the outcome.

Based on verified interviews with the marketing, ecommerce, CRM and data leaders who run software evaluations, at DTC and enterprise brands. Buyers are anonymized before publication; vendor names and views are reported as given. No vendor paid to appear or could edit this page.

About half
Evaluations that had produced a new tool, chosen, being implemented or live, by the same company's next interview, about ten months later on median.
1 in 4
Of those new tools, the share that was not among the options the buyer named at the first interview.
2×
How often an evaluation was reported still open when the same person was interviewed both times, against a different person at the same company: about 30% against 15%.

What happens after a company evaluates software?

Alium interviews some companies more than once. This page sets the evaluations buyers described at one interview, whether talking to vendors, comparing options, piloting or making the business case, against the same company's next interview, about ten months later on median. It covers evaluations active at the first interview where the follow-up said enough to classify what had happened since; evaluations the second interview did not address are left out. About half, 53%, had reached a new-tool outcome: an option named at the first interview was live or being implemented in 37% of cases, one had been selected but not started in 3%, and a product not named at first had been selected, was being implemented or was in use in 12%, a few of them in-house builds. Another 23% were still being evaluated, 22% still had the incumbent running with no active evaluation reported, and 2% had shelved a project for a capability they did not yet have. These are statuses at the next available interview, not final outcomes: a selected product may never go live, and an open search may close later. Nor are they vendor win rates, migration success rates or the share of all evaluations that end in a purchase.

Status of evaluations at the next interview with the same company, about ten months later on median. The three left-hand segments are evaluations where a new product had been selected, was being implemented or was in use, not all completed implementations. Each company contributes at most one evaluation per category of software, evaluations without enough follow-up information are left out, and shares are rounded. The stage and respondent rows are cuts of the same followed evaluations, not separate samples: the two stage rows and the two interviewee rows split them two ways, and the few already decided at the first interview sit in neither stage row.

The six outcomes

Each evaluation lands in exactly one of six stages at the second interview, taken in the order below: the first that fits is the one recorded. They are what the two interviews say, not a record of contracts.

The six outcomes of an evaluation at the same company's next interview, with the share of all evaluations in each. Rounded, so the column sums to 99.
Outcome What it means here Share
Live or implementing A tool the buyer named as an option at the first interview is in use, or the migration onto it is under way.37%
Selected, not live An option named at the first interview has been chosen or signed, and implementation has not started.3%
Off the shortlist The company chose, is implementing or runs a tool that was not among the options named at the first interview, or built its own. In-house builds are a few of these.12%
Still evaluating The buyer says the search is still open: still looking, an RFP under way, a pilot running with no decision, or waiting on budget or approval.23%
Kept the incumbent The company's existing tool was still in operation at the follow-up, and no replacement was reported as selected, being implemented or live. Some buyers said outright they had decided against replacing it; in other cases continued use was observed without confirmation that the evaluation had ended.22%
Dropped There was no existing tool, and the need has been shelved or is no longer wanted.2%

Two distinctions matter here. Evaluating counted only when a buyer actively described considering software at the first interview; a passing interest or a product only the interviewer raised did not count. And kept the incumbent does not always mean an affirmative decision to stop: in about six in ten such cases the follow-up showed the old software still running but did not mention the evaluation, which may include searches still under way or decisions the respondent did not know about. The category describes the recorded status at follow-up more reliably than a final retention decision. None of these shares is a vendor win rate, a churn rate or a final outcome.

Does the shortlist pick the winner?

Mostly. Most new-tool outcomes were a product already named at the first interview; about a quarter were absent from the options recorded then. That is a comparison with the options named at the first interview, not necessarily the company's full formal shortlist: a product may have been considered internally without being mentioned, or entered the search later. It shows the first interview did not capture every eventual destination, not that every unmentioned product was a late entrant.

The options named early are less predictive. When the first interview caught the evaluation at the research stage, with interest and a few names but no substantive vendor talks, about 18% later produced a tool not named then, and 29% still had the incumbent running at follow-up. Where the buyer was already talking to vendors, comparing named options or piloting, those figures were about 7% and 16%, and the share with a named option live or being implemented rose from 28% to 44%. The share reaching any new-tool outcome was about the same in both groups, at around half. That is consistent with an early list being less settled, but it may also reflect how completely buyers described their alternatives, and it does not show that a formal shortlist was overturned each time.

Two buyer histories illustrate how a company can end up choosing a product it didn't name in its earlier interview. A mid-sized services company's digital and technology chief built a shortlist of personalization vendors from research, sent out a capabilities list, and named VWO as the favorite. By the next interview the company had switched off a personalization module in its content platform to save hosting cost, “and we put in place of it AB Tasty, an A/B test tool with some personalization capabilities.” A global footwear brand had spent two years in conversations with Dynamic Yield, including “some kind of piloting, proof of concept, demos, but leadership is hesitant to pull the trigger.” At the next interview, a different person explained that the brand needed “another tool for more localized sorting, site merchandising, and recommendation. That's why we moved to Algolia.”

Pilots go with more advanced outcomes. Evaluations already in a pilot at the first interview more often had a named option live or being implemented at follow-up: about seven in ten, against fewer than four in ten for evaluations with vendor conversations but no pilot. That is an association with the stage at the first interview, not a measured effect of piloting: companies that pilot may already have stronger intent, clearer requirements, funding or approval. And the footwear brand above shows that extensive testing does not guarantee the company picks the vendor it piloted.

This is what buyers in general found. What fits your stack?A free tech stack review, built from hundreds of verified buyer interviews.
Get my stack review →

What keeps an evaluation open?

About a quarter of evaluations were still open at the next interview. The median interval was about ten months across the whole analysis and about nine among the evaluations still open. When buyers say why, the reason is more often the company's own calendar than the vendors. The search has been queued behind another project, the budget or the approval has not arrived, or the company is waiting for the incumbent's contract to end. An ecommerce replatform is the project other evaluations wait for more than once.

A small footwear brand had held calls with Smile.io and called it the front-runner for a loyalty program. Six months later the program was still unbought: “So we're still in the market looking for loyalty solutions. Like I know I said Smile.io, but I'm still open to looking for others.” The reason was sequencing, not the vendor: “We really wanted to get the loyalty program, but that's shifted because of the whole product recommendations.”

Evaluations observed after longer intervals had more often progressed. Where the second interview came more than a year after the first, about 46% had a named option live or being implemented and 14% were still open; within a year, those figures were 34% and 27%. These are different groups of evaluations with different intervals, not the same evaluations watched over time, so they do not show a standard buying-cycle length, and time alone is not shown to explain the difference.

When does a company keep the tool it has?

Among evaluations to replace an existing product, about a third were classified at follow-up as incumbent still in use with no confirmed new tool. Some of those buyers said they had chosen to keep it or that the move was called off; others were inferred from continued use. So the figure is not the share of replacements definitively abandoned in favor of the existing tool. Evaluations for a capability the company did not have yet reached a new-tool outcome more often, about 65% of the time against 48% for replacements, and they are almost the only place a project is dropped outright. Replacing a tool means beating a system that already works, a migration, and a contract with a date on it.

When buyers explain staying, the reasons are mostly about the move rather than the alternative: the engineering time a migration would take, the revenue work it would displace, a budget that would rather keep a cheap tool, an incumbent that improved in the meantime, or a renewal that arrived before a decision did. Sometimes the decision is made above the evaluator.

The clearest case is a DTC fashion brand's retention director, who had been meeting Iterable as a replacement for the company's email platform at the first interview. At the second: “Yeah, we actually almost signed with Iterable last year. Had the contract, had a better price, had the whole thing. The leadership team decided not to sign it at the last moment.” The company was still on its incumbent platform, and the buyer still wanted to replace it.

Why does the answer depend on who you ask?

The reported outcome differs a lot by whether the same person took part both times. When the same person was interviewed twice, about 30% of evaluations were still open and 17% had the incumbent still running; when a different person at the company answered, it was 15% and 28%. The share reaching a new-tool outcome was similar in both, and the contrast between open and incumbent held when only follow-ups within a year of the first interview are compared. That is consistent with respondent continuity affecting how status gets reported; it does not show that the choice of respondent caused the difference, or that one account is always more accurate.

One possible explanation, ours rather than something buyers state, is that a new respondent may know what software is running without knowing whether an earlier evaluation is still live. Another is that different people define an active evaluation differently, or see different teams' decisions. A different respondent also left the original evaluation unaddressed about a quarter of the time, against about 6% for the same person; those cases are excluded, which may affect how the two groups are made up. So treat the 22% with the incumbent still running with caution: some of it may be unresolved evaluations nobody described as active, especially when the respondent changed. The interviews do not show how many were misclassified or which group is closer to the real status, and the same-person figure of 17% is a useful comparison, not a verified correction.

Does company size change the outcome?

Little observed difference. About half of evaluations had produced a new tool at every size of company. The largest companies show slightly more evaluations with the incumbent still running and slightly fewer still open, but the gaps are within what these sample sizes would produce by chance.

Where evaluations stood at the next interview, by the company's current headcount. Each band rests on a few dozen to about a hundred evaluations, so read the gaps rather than the decimals.
Company size Produced a new tool Still evaluating Kept the incumbent
Under 1,000 employees 53%25%20%
1,000 to 9,999 53%23%21%
10,000 or more 53%18%28%

The counter-current: going live is not the end of the decision

A new-tool outcome does not establish implementation success or lasting satisfaction. A specialty outdoor retailer's marketing head named a session-recording tool as one of two platforms to bring on. By the next interview, within a year, it was in place, and the buyer was weighing cutting or downgrading it because it was delivering too little value for what it cost: “That's honestly one that we're looking at just cutting from the stack.” In the buyer's account, much of the cost came from the setup: a former team member had pushed to record every session, and the buyer now calls it “just an underutilized tool for us.” The outcomes on this page record reported decision and implementation status, not sustained adoption, return on investment or retention.

Want a review of your own stack?

Get my review →

What this means if you are running an evaluation

These are our recommendations: implications of the histories above, not a tested process for raising purchase or implementation success. The follow-ups show how decisions evolve, not which practices cause a purchase or whether the tool meets the company's needs. Five questions to settle:

What not to read into this

The shares are not a close rate for any vendor, not the share of all evaluations in the market, and not a churn rate for any category. They describe evaluations Alium caught in one interview and could follow to the same company's next one, excluding the ones the second interview did not address. Still evaluating is not a failure: many of those evaluations may close after the second interview. The stage of an evaluation is what the buyer described at the first interview, not an audit of the process. Kept the incumbent includes explicit decisions and cases inferred from continued use; because follow-up respondents differed in what they knew, it may include unresolved searches, so do not read its percentage as a confirmed retention rate. And the pilot, stage and incumbent comparisons are associations between groups of companies, not effects: they do not show that piloting or contacting vendors earlier would change any one company's outcome.

Common questions

What usually happens after a company evaluates new software?

In Alium's interviews, about half of evaluations had reached a new-tool outcome by the same company's next interview, about ten months later on median: an option named at the first interview live or being implemented in about 37% of cases, one chosen but not yet started in 3%, and a product not named at first in 12%, a few of them in-house builds. Another 23% were still being evaluated, 22% still had the incumbent running with no active evaluation reported, and 2% had been shelved. These are statuses at the next interview, not final outcomes, and they cover evaluations the second interview addressed, not every evaluation a company starts.

How often do companies buy a tool that wasn't on their shortlist?

About one in four of the new tools chosen in these evaluations was not among the options the buyer named at the first interview, a few of them in-house builds. That is a comparison with the options named then, not necessarily a full formal shortlist. It happened most when the first interview caught the evaluation early, at the stage of interest and research: about 18% of those evaluations later produced a tool not named then, against about 7% where the buyer was already talking to vendors or running a pilot.

How often does a software evaluation end with the company keeping its current tool?

Where an existing tool did the job, about a third of evaluations to replace it still had that tool in use, with no confirmed replacement, at the next interview. Across all evaluations it was about one in five. Most of those cases were inferred rather than stated: the old tool was still running and the evaluation was not mentioned again, so the share is a recorded status, not a confirmed retention rate. When a different person answered the second time, incumbent outcomes were more common and still-open ones less common, so some of that share may be evaluations a new respondent did not know were still open.

Does running a pilot make a software purchase more likely?

Evaluations that were already in a pilot at the first interview had a named option live or being implemented about seven times in ten at the next interview, against fewer than four in ten for evaluations that were talking to vendors without a pilot. That is an association, not proof that pilots cause purchases: a company that agrees to a pilot may already have stronger intent, clearer requirements, funding or approval. A pilot can also stall: one footwear brand piloted and held demos with a personalization vendor over two years and then moved to a different vendor.

Do larger companies take longer to finish a software evaluation?

In these interviews there was little observed difference by company size. About half of evaluations had produced a new tool by the next interview at companies under 1,000 employees, between 1,000 and 10,000, and over 10,000. The largest companies showed slightly more evaluations with the existing tool still in use and slightly fewer still open, but the gap is small enough to be noise at these sample sizes.

Related reading: for where a shortlist comes from, see how software buyers find vendors; for when owners start looking at all, at what rating shopping begins; for what a first purchase asks that a replacement does not, replacing or buying first; and for the tools that stay despite poor ratings, why buyers keep software they rate poorly.

A tech stack review, powered by the buyers who've been there

What matters most to you, a shortlist for your stack, and the next steps to take — built from hundreds of verified buyer interviews.

Get my stack review

Free · no password · your answers are anonymized before they inform research like this.

Want to come back to this?We'll email you a link to this page, so it's easy to find later or forward to your team.
Methodology. Alium conducts verified interviews with software buyers — the people who select and operate these platforms. This page follows companies that were interviewed twice, at least six months apart. Every case where a buyer at the first interview described evaluating a tool for a need — talking to vendors, a shortlist, a pilot, a business case or a stated plan to replace or add a tool — was read alongside the same company's next interview and coded by hand into one of six outcomes, from the buyers' own words. Products named only by the interviewer, sales and advertising channels, tools the company already ran with no change in view, and passing wishes were set aside as not evaluations. Only evaluations active at the first interview are included, and only where the follow-up said enough to classify a later status; evaluations the second interview did not address, about one in six, are excluded from the shares, and each company counts at most once per category of software, using its earliest evaluation. Statuses are those at the next available interview, not final outcomes. The stage of an evaluation is what the buyer described at the first interview, not an audit of the process. A check of a sample of codes against the transcripts changed fewer than one in ten. Company size is the company's current headcount. Buyer identities are verified at interview time and anonymized before publication; vendor names are reported as given. No vendor paid to appear or was able to edit this page.