Data analysis
Support and unpaid outreach data
Separate support quality, attributed sales, and mature holdouts.
Three sources cover daily support performance, unpaid-order outreach attribution, and mature holdout comparison. Operational queues and configuration remain in Customer Service performance and Unpaid-order outreach.
All tables contain fictional sample data for explaining calculations, not real Shop performance or experiment conclusions. Sources have different dates and denominators; they are not one unified funnel.
#Daily support performance
Track workload, response, escalations, and service quality. Rows summarize a Shop by America/Los_Angeles date, including active/new conversations, inbound/outbound messages, escalations and resolution, rating samples, satisfaction, first response, and support-guided GMV.
Summarize Brightloom US workload, escalation rate, satisfaction, and first response by day. Retain rating sample counts and state guided-GMV availability.
#Sample summary data
| Date (Los Angeles) | Active conversations | Escalated | Escalation rate | Rated | Satisfied | Satisfaction | Avg first response (s) |
|---|---|---|---|---|---|---|---|
| 2026-10-01 | 100 | 10 | 10.0% | 20 | 18 | 90.0% | 24 |
| 2026-10-20 | 120 | 18 | 15.0% | 10 | 8 | 80.0% | 30 |
Row one's escalation rate is 10 ÷ 100; satisfaction is 18 ÷ 20, not 18 ÷ 100. Combined satisfaction is 26 ÷ 30 ≈ 86.67%, not an average of 90% and 80%. First-response averages also need actual first-response sample counts as weights.
#Sample mature-sales measures
| Date | Currency | Support-guided GMV | Complete 7-day rolling average |
|---|---|---|---|
| 2026-10-01 | USD | 500.00 | 480.00 |
| 2026-10-20 | USD | — | — |
For a query on October 21, guided GMV for the latest seven complete Shop-local days is unavailable, not zero. The rolling average requires seven consecutive daily values in one currency. Guided GMV is platform attribution, not proven incremental support revenue.
#Unpaid-order outreach summary
Track unpaid journeys through eligibility, successful sending, and subsequent attributed payment. Group by Shop-local order creation date. Repeated messages do not multiply reached orders or attributed sales.
Analyze 30-day outreach results with distinct orders and message counts. Use reached orders as the payment-rate denominator, separate currencies, and do not claim incrementality.
#Sample summary data
| Order date | Tracked unpaid orders | Eligible | Reached orders | Sent messages | Paid after outreach | Attributed payment rate | Associated GMV (USD) |
|---|---|---|---|---|---|---|---|
| 2026-10-13 | 100 | 80 | 60 | 90 | 12 | 20.0% | 480.00 |
| 2026-10-14 | 70 | 50 | 40 | 55 | 4 | 10.0% | 160.00 |
Combined, 100 orders were reached through 145 messages, with 16 attributed payments: 16 ÷ 100 = 16%, not 16 ÷ 145 or the simple daily-rate average of 15%. Tracked unpaid orders are neither all Shop orders nor the current unpaid backlog.
Ordinary reporting needs no experiment and supports up to 90 days per query. Recent cohorts can still change. Missing warehouse coverage is not zero conversion. Post-send association does not prove buyers would otherwise never have paid.
#Mature holdout comparison
Compare payment rates after random assignment to no outreach or production outreach. Query mature samples, paid counts, rates, and an unadjusted rate difference by experiment version and Shop-local order-creation cohort.
Compare mature control and production samples by experiment version. Keep sample sizes, do not merge versions, and do not announce significance from this simplified table.
#Sample summary data
| Experiment | Control mature | Control paid | Control rate | Production mature | Production paid | Production rate | Difference |
|---|---|---|---|---|---|---|---|
| DEMO-E01 | 200 | 20 | 10.0% | 200 | 26 | 13.0% | +3.0 pp |
| DEMO-E02 | 100 | 12 | 12.0% | 100 | 13 | 13.0% | +1.0 pp |
This mature randomized comparison includes natural payments and orders receiving no message, not just successfully sent orders. Outcome windows run from creation: 48 hours for Mexico and one hour for other supported regions.
Queries support up to 90 days and keep experiment versions separate. Production samples exclude exploratory B/C variants during multi-configuration experiments. These differences have no confidence interval and are not the formal stratified estimate. Use the experiment page for full comparisons and significance conclusions.
Return to Business data or read Interpreting results.