Does an AI shopping assistant actually work? Measure it with a holdout

Every AI shopping assistant claims a lift. Most of them count any order the widget so much as brushed against and call it “assisted revenue.” The dashboard looks great and means almost nothing. If you want to know whether an assistant actually works, you have to measure it the way a clinical trial measures a treatment: against a control group.

The problem with last-click credit

Attribution that gives the assistant credit for every order it touched can't separate causedsales from ones that would have happened anyway. Without a control, it's a story, not a measurement, and it's always flattering.

Run a holdout

Cervito's attribution and proof runs a randomized holdout: a percentage of your visitors never see the widget. Their order rate is the control; everyone else's is the test. The difference is a two-proportion z-test, the same method a trial uses, so the conversion lift you report is causal, with a confidence interval you can point at. (In our demo store, the illustrative figures are a +23% lift at 95% significance.)

Five models, so you pick the lens you trust

The same shopper journey is scored five ways (last-widget, linear, time-decay, first-touch, and position-based) over a 30-day window built for slower gifting and considered purchases. You run the model you believe in, not the one that flatters us most. When they disagree, the dashboard says so.

Tie it back to what you changed

Because measurement is causal, you can attribute the lift to specific moves: a new campaignthat leaned Sarah into a promoted collection, a coaching note that changed how she answers. That's the difference between a metric and a lever.

The fastest way to see this on your own catalog is a 30-minute demo with the founder, a Google Meet call where we open Sarah on your store. Book a time, or write to [email protected]. You're usually live within a day.