Most organizations treat a pilot as a test they hope to pass. The pilot works, a headline number goes into the deck, and the conversation moves straight to how fast the program can roll out. That habit is expensive. A pilot that only proves demand has answered the easiest question. The harder ones, about value, economics, and operations, are often still open when the investment decision is made.

A better way to think about it: a successful pilot isn't proof that you're ready to scale. It's evidence of what you still need to solve.

Demand is not value

Early usage numbers for a new digital health service can look extraordinary. When a service starts from a small base, the multiples can be very large. At Anthem, the Sydney Care pilot with 1.9 million members produced a 50x increase in virtual care utilization. That is a meaningful signal. It is not, on its own, a business case.

A demand signal tells you people will try something. Value requires knowing whether they come back, whether the care they receive replaces something more expensive or adds cost on top of it, and whether the result holds across a broader, less motivated population. What made the Sydney Care evidence useful was that it covered the chain from downloads to engagement to conversion to care, not just the top of it.

A compelling metric is not an investable case

Executives who fund a scale-up are buying a forecast. One impressive number doesn't give them one. They need a set of linked measures that explain how activity turns into financial impact, and an honest account of what the pilot couldn't show. Before I'd call a pilot investable, I'd want to see:

  • The full conversion chain, from awareness to sustained use, with the weakest link identified.
  • The population, including who was in the pilot and how the scaled population will differ.
  • The comparison, meaning what would have happened anyway, through a control group or a credible baseline.
  • The cost to serve at scale, not at pilot volume.
  • The financial measure the enterprise actually manages to. For a health plan, that is cost per member per month, not downloads.

The Sydney Care pilot became part of the case for an $8.1 million launch investment in Virtual Primary Care. The multiple got attention. The evidence across the whole chain is what made the case.

Pilot economics are not operating economics

Pilots run under conditions that rarely survive scale. Volumes are smaller, teams pay closer attention, and edge cases are rare enough to handle by hand. Scale exposes what a pilot hides.

Virtual Primary Care launched under the pressure of COVID, which meant it was effectively tested in a live environment. When demand arrived, registration lagged: 46% of members registered, compared with 80% who installed and opened the app. Part of the cause was one-time passcodes failing under server load. It was my first launch in a live environment, and no one on the team, including Product and IT, had asked whether the infrastructure would hold at that volume. I should have asked. We fixed passcode delivery with the vendor, moved required legal disclaimers from registration to virtual care enrollment, and registration reached 80%.

The lesson I took from it is simple. Before scaling, test the operational assumptions the pilot never exercised: system capacity, vendor limits, support load, and compliance steps. These rarely appear in a pilot readout, and they are exactly what determines whether scaled results match the pilot.

The best pilots don't produce the biggest multiple. They leave leadership with the fewest unknowns.

Marketing performance is not enterprise performance

Marketing metrics are necessary, but they are not what the enterprise funds. Downloads, registration, and engagement show that the acquisition system works. The enterprise question is cost and outcomes.

When Virtual Primary Care moved into Sydney Health, the member app people already used, and scaled to Anthem's 12.1 million eligible members, the outcome that mattered was measured at the program level: $4 in savings per member per month. That is a program result, produced by care delivery, operations, product, and marketing together. It should be reported that way, and marketing leaders build more credibility by saying so than by claiming it.

Scaling through an app members already used also points to a broader principle. Distribution is part of the scale decision. Asking millions of members to adopt one more standalone app is a very different bet from adding a capability where they already are.

Five questions before you fund the scale-up

  1. What would have to be true for this to work at ten times the size, and which of those things did the pilot actually test?
  2. Are we looking at demand, or at value measured in the enterprise's own financial terms?
  3. Which link in the conversion chain is weakest, and is it a marketing, product, or operational problem?
  4. What breaks at volume: infrastructure, vendors, support, or compliance?
  5. Who owns the outcome after scale, and how will it be measured and reported?

A pilot that surfaces a problem you can fix before scale is worth more than one that only confirms what you hoped. Fund the evidence, not the excitement.