Metrics and diagnostics
Are International Orders Riskier? Audit the Comparison First
An international-versus-domestic dispute comparison is meaningful only after you define what international means and test whether the two groups contain comparable purchases. Destination, billing country, currency, and fulfillment origin describe different parts of a transaction. Combining them into one label can turn a shipping problem or product-mix change into an unsupported conclusion about a market.
The immediate goal is an evidence package for a market decision: a defined geography, comparable exposure, clear uncertainty, and an explanation of what remains unresolved. Country-level restrictions should not be the automatic result of a small or poorly matched sample.
Choose the geography that fits the question
If the question concerns the delivery experience, group by the destination country recorded for the purchase. If the question concerns currency presentation, use checkout currency as a separate dimension. A buyer's nationality is neither necessary nor an appropriate substitute for these operational facts.
Write the definition in the report header. For example: “International means delivery outside the merchant's primary fulfillment country, based on the destination assigned when the order was accepted.” Specify how local warehouses, digital purchases, and changed destinations are treated. Digital products may have no relevant physical destination and should not be forced into a shipping comparison.
Check which sales dimensions you can reconstruct from actual records using Shopify's sales-report documentation. Do not assume one report contains both the payment population and every needed dispute event.
Retain an unknown-destination group. Excluding unknowns silently is especially misleading if older international orders have more incomplete records than domestic purchases.
Separate product mix and observation age
Compare purchases made in the same period and observed for an equivalent length of time. International orders sold last month should not be contrasted with domestic orders followed for six months. Record the age of each cohort at extraction and revisit immature groups later.
Next inspect product family, basket size, delivery model, and promised delivery range. These are candidates for explaining a difference; they are not reasons to erase inconvenient cases. Present raw totals before adjustments and show how much exposure remains after matching.
Do not match on a fact that occurs after purchase if doing so removes the very issue under investigation. For example, limiting both groups to successfully delivered orders would discard nonreceipt failures and answer a different question. You can examine delivered orders separately, but label that conditional population clearly.
For currency, compare counts without converting monetary units. When comparing amounts, keep native currencies separate or document a consistent conversion date and rate source. A larger average amount expressed in a different currency is not a larger economic exposure until the units are aligned.
Use a transparent hypothetical audit
Suppose a hypothetical store has 4,000 domestic orders with twenty disputed orders and 1,000 international orders with ten. All purchases have equal follow-up. The raw internal proportions are 0.5% and 1%.
The product breakdown changes the interpretation:
| Destination group | Standard product orders / disputes | Made-to-order orders / disputes |
|---|---|---|
| Domestic | 3,600 / 9 | 400 / 11 |
| International | 600 / 1 | 400 / 9 |
Made-to-order products account for 10% of domestic exposure and 40% of international exposure. Their observed dispute proportions are higher than standard products in both destinations. Within each product group, international orders do not show a higher observed proportion in this example.
That does not establish that international delivery is harmless. It shows that the overall difference does not isolate destination. The next useful investigation concerns the made-to-order promise, including whether preparation estimates and specification changes were understood consistently.
The table should include the count of unmatched orders if, for example, a product is sold exclusively overseas. Those purchases can be investigated on their own merits. They cannot support a direct domestic comparison where no counterpart exists.
Set the evidence required for a market decision
Prepare a one-page decision record with five entries: the geography definition; raw and comparable counts; observation age; dominant reason categories; and competing explanations. Attach the exact extraction date so the result can be reproduced after new cases arrive.
Use the processor's current monitoring view independently. Shopify's chargeback monitoring documentation describes that separate operational reference; your destination audit should not replace it.
A limited decision might be to review one international product promise, improve ownership of a particular delivery exception, or collect another mature cohort. State the signal that would justify revisiting the decision. For example, repeated nonreceipt complaints within a matched delivery model would warrant a focused logistics investigation.
If the data remains sparse, publish counts and say that the comparison cannot yet distinguish a stable destination effect from random variation and product mix. That statement still enables action on actual customer problems. It prevents a weak aggregate comparison from becoming a sweeping judgment about buyers or an unsupported market exit.
Review current dispute activity before investigating international order cohorts separately.
Related reading in this collection: