Metrics and diagnostics

Which Products Drive Disputes? A Fair SKU-Level Analysis

Rack of blank tshirts.
Photo: Matthew Henry / Burst

A disputed basket can contain several products, but that does not mean every product caused the dispute. Before ranking SKUs, decide whether the analysis measures product presence in disputed orders or assigns a verified cause to a particular item. Those are different questions and require different evidence.

Use an order-based exposure table for the first question and a separate investigated-cause field for the second. Keep mixed baskets visible. Otherwise, a commonly bundled accessory can appear responsible for problems originating elsewhere, and product totals can count the same disputed order several times.

Choose the unit before building the product table

Define the purchase population and observation period, then identify distinct orders containing each SKU. For a product-presence measure, an order contributes once to a SKU even if it contains several units of that item. The numerator is the distinct orders containing that SKU that meet your defined dispute condition.

That product-presence view allows overlap between SKUs. Its rows are useful individually but are not additive across the store. State this beside the table. If the audience expects product rows to sum to the total disputed-order count, use a separate allocation method and explain its assumptions.

Shopify's sales-report documentation provides the commercial reporting context and warns that report structure can create several rows for an order. Build the analytical unit deliberately rather than treating every exported row as a separate purchase.

Use stable product and variant identifiers alongside readable names. A renamed product should not silently become a different exposure group, while genuinely different variants should not be combined solely because their titles look similar.

Work through the mixed-basket problem

Consider this hypothetical population, observed over the same defined order-age window:

Basket composition Distinct orders Orders with a dispute
Product A only 500 5
Product B only 500 10
Products A and B together 100 8
Store total 1,100 23

Product A appears in 600 orders, including 13 disputed orders. Product B appears in 600 orders, including 18 disputed orders. Adding those product-level disputed counts produces 31 because the eight mixed-basket orders are present in both rows. The store still has 23 distinct disputed orders.

The basket view reveals another useful fact: the hypothetical mixed group has eight disputed orders out of 100, compared with five out of 500 for A-only and ten out of 500 for B-only. That pattern warrants investigation into the combined purchase experience. It does not identify either item as the cause.

Possible explanations include a promise attached to the combined offer, a fulfillment path used only for mixed baskets or a different customer population. The analytical next step is to examine those differences, not to declare the higher-presence SKU defective.

Choose an allocation only when the question needs one

If management needs an additive workload allocation, one transparent convention is to split each mixed-basket disputed order equally across its two products. In the hypothetical example, A receives five single-product cases plus four allocated cases, and B receives ten plus four. The allocated total is nine plus 14, which reconciles to 23.

Label those figures “allocated disputed orders” or another equally explicit term. They are not counts of independently verified product-caused disputes. An equal split is a convention for reporting responsibility or workload, not evidence about causation.

Other questions can require a different allocation, such as assigning an investigated primary cause when the case record supports it. Preserve an unresolved category where the information is insufficient. Do not allocate financial loss by item price and then present the result as the amount actually disputed for that item unless the payment record supports that interpretation.

Keep product presence, analytical allocation and verified cause as separate fields. That design lets the team answer different questions without changing the underlying meaning of a single number every time the report is reused.

Compare like-for-like exposure before acting

Review product differences alongside basket composition, purchase age, fulfillment route and relevant offer version. A new SKU with little observation time should not be compared directly with a mature product history. A product sold mostly in mixed baskets should not inherit a causal conclusion from a simple presence ranking.

Keep the provider's monitoring population separate from this internal product analysis. Shopify's chargeback monitoring guidance supplies the account-health context. Your product-presence percentages answer a merchandising investigation question and are not substitutes for the provider's official measure.

Before sharing the result, verify that the distinct store total reconciles, mixed-basket overlap is disclosed, unlinked cases remain visible and the observation window is consistent. Include a short note describing the allocation method if one is used.

The resulting decision might be to review a combined offer, inspect a particular fulfillment route or investigate a variant-specific complaint. Keep the decision proportionate to the evidence. Product analysis is useful when it narrows the next question accurately; it becomes misleading when the table's row label is treated as proof of what caused the case.

Review disputed orders in Lower Chargeback before investigating product exposure in your own reports.

Review your disputes

Related reading in this collection: