New Versus Returning Customers: Compare Disputes Without Bias

Returning customers can appear safer simply because they buy different products, place smaller orders, or have had more time to reveal a purchase history. A useful comparison freezes customer status when each purchase happened, gives both groups the same opportunity to develop disputes, and then compares like purchases. The result should inform an operational investigation, not create a permanent judgment about individual shoppers.

Start with a precise question: among comparable purchases made during a defined period, did orders from customers with an earlier purchase produce different observed dispute proportions? That question is narrower, and more useful, than asking whether loyal customers are trustworthy.

Freeze the label at the purchase

Define returning as having at least one qualifying earlier purchase before the current order timestamp. State what qualifies: for example, a completed paid purchase, excluding test transactions. Apply that rule consistently to both groups. An earlier order placed minutes before the current order still needs a documented treatment; do not silently decide based on whether the later purchase was disputed.

Do not use today's lifetime order count to relabel yesterday's first purchase. A shopper who buys again next month was still new when making the first purchase. Using future behavior would select people who continued purchasing into the returning group and distort the question.

Record an unknown category when the available order history begins after the customer's relationship with the store. A migrated customer database, missing guest-order linkage, or incomplete historical export can prevent a reliable label. Unknown is useful information about the analysis; forcing it into new creates false precision.

Use your merchant reporting records for purchase context. Shopify Analytics is an official starting point for available reports; the cohort definition here is an independent analytical choice.

Build comparisons within shared exposure

Choose purchase weeks with equal follow-up and separate large differences in product family, basket value, and fulfillment model. The purpose is to prevent customer status from serving as a substitute for a more direct operational explanation.

A manageable first pass uses two product families and two basket bands. Avoid dozens of tiny cells. If only returning customers buy a certain replenishment product, there is no comparable new-customer population for that product. Report this absence rather than inventing a match.

Keep the unit consistent. If the analysis uses paid orders, count orders with at least one linked dispute episode, and describe the result as an internal order-cohort proportion. Do not present it as the processor's monitoring rate. Consult the separate Shopify chargeback monitoring guidance for the official monitoring view.

Work through a mix-driven difference

Consider this hypothetical mature purchase cohort. Each order belongs to one product group, and every order has the same follow-up period.

New customers have eight disputed orders among 1,000 orders, or 0.8%. Returning customers have thirteen among 2,000, or 0.65%. The unadjusted figures suggest a difference.

Within standard products, both groups have a 0.5% observed proportion. Within premium products, both have 2%. Customer status adds no observed distinction inside either product group. The apparent advantage arises because returning customers buy a greater share of standard products.

Apply a common hypothetical mix of 80% standard and 20% premium orders. Both groups then have an adjusted proportion of 0.8%: 80% multiplied by 0.5%, plus 20% multiplied by 2%. This does not prove the groups are identical. It shows that this dataset's headline difference is explained by the chosen product mix.

If basket bands reveal a remaining difference, repeat the calculation within those bands. Preserve the unadjusted results alongside adjusted ones so readers can see what changed and why.

Turn the result into a limited decision

Write a conclusion that distinguishes observation, adjustment, and next action. A reusable format is: “For purchases from [period], observed through [date], the raw difference was [result]. After using a shared [product and basket] mix, the difference was [result]. Coverage excluded [unmatched or unknown groups]. We will investigate [specific operational condition] before changing [decision].”

Review a few underlying purchases from the cells driving the result. Ask whether premium items require a different expectation, longer preparation, or a more complex receiving process. These questions address the purchase journey rather than assuming a customer label explains the outcome.

Assign an owner to resolve missing history and a separate owner to investigate the relevant product experience. Schedule the next comparison when another cohort has equivalent follow-up. A conclusion that changes with the next few cases should remain provisional. The useful outcome is a clearer operational question, supported by a comparison that does not borrow information from the future.

Use Lower Chargeback for operational dispute review while comparing customer cohorts in your own tools.

Review your disputes

Related reading in this collection:

• How to Calculate Your Shopify Chargeback Rate: Count and Value Explained
• Are International Orders Riskier? Audit the Comparison First
• Why a Falling Sales Month Can Distort Your Dispute Dashboard