# Testing Plan — Review & Edit: Substitution Structure

## Study overview
**Study name:** Review & Edit — Substitution Structure
**Date:** 2026-05-18
**Researcher:** Roy Hsia
**Number of participants:** 9 (3 per variant)
**Platform/tool:** UserTesting.com — unmoderated, mobile (375px)

## Research questions
1. Which structure helps customers manage their substitution preferences with the least effort?
2. Which structure best communicates which items will and won't be replaced?
3. Do customers understand which items are ineligible for substitution (e.g. alcohol)?

## Hypotheses
H1: A tabbed structure lets users locate and change a preference with the fewest navigation steps.
H2: Users comprehend the count of items that will be replaced more accurately when counts are persistently visible.
H3: System-oriented labels ("eligible / ineligible") are harder to understand than intent-oriented labels ("will be replaced / won't be replaced").

## Tasks

**Scenario:** Imagine you placed a grocery delivery order. Some items may be out of stock, so the store could substitute them with something similar. You're on the "Substitutions" screen to review and adjust which items can be replaced. Look at the page as you normally would. There are no right or wrong answers — please think out loud and share your thoughts as you respond to each question.

| # | Task | Type | Correct answer |
|---|------|------|----------------|
| 1 | Please don't tap or click anything yet. Take a moment to look at this page and tell me what stands out to you. | Open-ended | — |
| 2 | How many of your items will be replaced with a substitute if they're out of stock? | Count | 6 |
| 3 | Which of your items will NOT be replaced if they're out of stock? | Open-ended | The 2 "don't replace" items (milk, eggs) |
| 4 | Find "Lipton Noodle Soup Mix" and tell me whether it will be replaced. | Task | Yes — it's in the items that will be replaced |
| 5 | You decide you don't want your milk replaced. Show me how you would do that. | Task | Move milk to "Don't replace" |
| 6 | Some items can't be substituted at all. Can you find which ones, and tell me why? | Open-ended | Beer + pain reliever — alcohol/medication are ineligible |
| 7 | In your own words, what does this screen let you do? | Open-ended | — |
| 8 | How confident do you feel that your substitution preferences are set correctly? | Rating (1 not at all confident – 5 very confident) | — |
| 9 | How clear was this screen overall? | Rating (1 not clear at all – 5 very clear) | — |

## Success metrics
| Metric | Target |
|--------|--------|
| Task 2 (count comprehension) correct | ≥ 70% |
| Task 4 (locate a specific item) success | ≥ 80% |
| Task 6 (identify ineligible items) correct | ≥ 70% |
| Avg confidence (Task 8) | ≥ 4 / 5 |
| Avg clarity (Task 9) | ≥ 4 / 5 |

## Participant criteria
- Has used online grocery delivery in the last 3 months
- Mix of substitution-experienced and first-time users
- Mobile users

## Item set (identical across all variants)
- **Will be replaced (6):** Lipton Noodle Soup Mix, Sam's Choice Mahi-Mahi Fillets, Ajax Ultra Dish Soap, Honey Nut Cheerios, Organic Bananas, Roma Tomatoes
- **Don't replace (2):** Great Value Whole Milk, Eggland's Best Large Eggs
- **Ineligible — cannot be substituted (2):** Coors Light 6 pack (alcohol), Bayer Low Dose Aspirin (medication)

## Notes
Between-subjects: each participant sees ONE structure (Tab / Accordion / Filter). Item set is identical across variants.