199 lines
18 KiB
Markdown
199 lines
18 KiB
Markdown
# Freshyo E2E sanity — results (run 2026-10-10 16:30, final)
|
||
|
||
Companion to `docs/e2e-sanity-test-plan.md`. Maestro MCP drove the iOS simulator
|
||
(`B1F98A94-…`, `com.freshyotrial.app` / `in.freshyo.adminui`); authoritative assertions were
|
||
made over tRPC (sanctioned by the plan). Screenshots: `/tmp/freshyo-e2e/*.jpg`.
|
||
Maestro viewer used: `http://127.0.0.1:10000/`.
|
||
|
||
Scope actually executed: the plan's **recommended daily sanity subset** (20 cases).
|
||
Every case below with a status was *executed in this run*; the remaining plan cases are
|
||
marked **NOT RUN (this run)** — the 13:10 report (`e2e-sanity-test-results-2026-10-10-1310.md`)
|
||
remains the record for those.
|
||
|
||
## Environment
|
||
|
||
| Item | Value |
|
||
|---|---|
|
||
| Backend | `wrangler dev --config wrangler.dev.toml` @ `:8787` (workerd PID 96583, started 15:15:48) |
|
||
| Metro | `:8081` (pre-existing, untouched) |
|
||
| Web | `:4174` customer · `:4175` superadmin · `:4176` admin |
|
||
| GPS | `16.7489,77.9856` → region 1, bound 1 |
|
||
| Baseline (before) | regions: only 1 (Mahabubnagar); bound 1 charge 20 / threshold 300 / min 300; flash off; versions 2/2/2; 20 products; staff `shafi` (id 6); customer 388 `9676651496` |
|
||
| Credentials used | superadmin `superadmin`/`superadmiN123`; staff `shafi`/`shafI123`; customer 388; (created) staff B `e2eadminB1702`; (created) fresh customer `9000011715` |
|
||
|
||
## Harness notes discovered (add to `maestro-mcp-agent-guidelines.md`)
|
||
|
||
1. **`launchApp: {clearState: true}` is a no-op on iOS** — the session survived; log out through
|
||
the UI (Me `90%,92%` → gear `89%,21%` → swipe `50%,80%→50%,30%` → `Logout` `50%,78%`).
|
||
2. **Pushed routes have no tab bar** (cart, checkout, login walls, order success, search results):
|
||
tab taps are ignored; leave with the back chevron `7%,7%` / `Go Back` / `Continue Shopping`.
|
||
3. **MCP requests die at 60 s** — long chained flows abort mid-way. Either keep flows short or raise
|
||
`timeout` in `.pi/mcp.json` (not done here).
|
||
4. **The region sheet stops responding after first use** — the sheet's list query shows an endless
|
||
top-center spinner; `stopApp`/`launchApp` clears it. Cold launch re-resolves GPS and restores the
|
||
GPS region (that is how the run returned from region B to A).
|
||
5. **Cart item removal asks for confirmation** ("Remove Item … Cancel / Remove").
|
||
6. `yaml` flows need `appId: com.freshyotrial.app` + `---`, and admin flows need the app's own
|
||
`appId: in.freshyo.adminui`; the repo's `tests/e2e` testIDs all work on iOS (name/store/type/
|
||
variant fields), but **typing into those fields on iOS routes text into the wrong input**
|
||
(keyboard focus) — creation was therefore done through the same tRPC mutations the UI calls,
|
||
as the 13:10 run did.
|
||
7. QR/percentage coordinates that shifted this run: coupon row 61% (not 51%), flash `Checkout`
|
||
72%,77%, `Place Order` 50%,70%, `Get in 1 Hour` 74%,75%.
|
||
8. Customer home is a single a11y blob (`txt: ""`), so evidence = screenshots + blob substring checks.
|
||
9. An "Open in Freshyo?" alert did not appear this run (no deep links used).
|
||
|
||
---
|
||
|
||
## Results — the 20-case daily subset
|
||
|
||
### A. Regions, bounds, configuration
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| RG-01 | **PASS** | Created region 2 "E2E Testnagar 1702" + bound (priority 1) + admin `e2eadminB1702` through the same APIs the UI calls (`superadmin.regions.create`, `superadmin.staff.create`). `common.listRegions` (used by `apps/user-ui/src/components/RegionSelector.tsx`) returns both regions with `effectiveBoundId`; the app's Delivery Region sheet lists the new region with its description; after switching, the region loads with **no undefined amounts** — ₹500 threshold from bound 3, its own stores/catalog (`/tmp/freshyo-e2e/RG-06-region-sheet4.jpg`, `RG-06-region-B-home.jpg`). |
|
||
| RG-03 | **PASS** | Cold launch **signed out** → `DELIVER TO Mahabubnagar`, catalog (Popular rail with prices) and `₹300 more for FREE Delivery` with **no login** (`RG-03-signedout-home-relaunch.jpg`); signed-in repeat identical plus `UPCOMING ORDER #ORD2466` and `₹235 • 1 Item / ₹65 more` (235+65=300 ✓) (`RG-03-signedin-home2.jpg`). Resolved once, no flip. |
|
||
| RG-06 | **PARTIAL → effectively PASS** | A: fixture products/stores visible; B: same search → "No products found", B's own store/item/threshold visible, cart cleared on switch; API isolation: admin B → A product 152 "Product not found", admin A → B product 153 "Product not found", admin B stores/slots contain only B's. **The B→A leg could not be completed through the sheet** (harness: sheet hang) — it was exercised only indirectly via cold launch re-resolution, so the "browse A → B → A" sequence is one leg short of the plan's wording. |
|
||
|
||
### B. Authentication
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| AU-01 | **PASS** | Signed out: browse/search/product detail work (`AU-01-search-chicken-signedout.jpg`, `AU-01-product-detail-signedout.jpg`); Cart → "Please log in / Log in to view your cart and continue to checkout." + Log In/Go Back (`AU-01-cart-login-wall.jpg`); Me → "Please log in / Log in to view your profile, orders and more."; no forced redirects; tabs usable after leaving the wall. |
|
||
| AU-02 | **PASS** | Item added while signed out (₹235 • 1 Item, ₹65 more) → cart wall → Log In → `user-login-mobile` 9676651496 → `user-login-other-ways` (password path, **no OTP**) → `user-login-password` → Login → returned to the Scheduled Delivery Cart with the line intact, address "Shafi" (`AU-02-after-login-return.jpg`). |
|
||
|
||
### C. Stores, products, variants
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| PD-01 | **PASS** (creation via API; results via UI) | Store 11 with image; product 152 with SKUs 153 (500 g ₹199/249) and 154 (1 kg ₹349/399, flash ₹299), genuine uploaded images (presigned PUT + claim), unit features; slot 2033 with both SKUs. Customer: both variants with correct price/MRP/unit/images; add-dialog offers **only** the assigned slot; item added; store detail "E2E Store 1657 · 2 ITEMS" (`PD-01-customer-search-after-cache.jpg`, `PD-01-SL-slot-dialog-options.jpg`, `PD-01-store-detail.jpg`). Cache verified directly: `products.json` v-4 contains the fixture; versions bumped 2→4. **Caveat:** admin-app form creation could not be driven on iOS (keyboard focus) — see harness note 6. |
|
||
| PD-02 | **PASS** (API) | Cross-region product/store/slot lookups all denied; admin B list/slots contain only B entities (see RG-06). |
|
||
| PD-05 | **PASS** | `toggleOutOfStock{152}` → both SKUs UNAVAILABLE in search + red "Out of Stock" badge on the cart line while the other line stays valid (`PD-05-sku-oos-customer.jpg`, `PD-05-cart-with-oos-line.jpg`); `placeOrder` with SKU 153 → **"Product 153 is unavailable"**, with 154 → **"Product 154 is unavailable"** (product-level OOS). Stock restored. |
|
||
|
||
### D. Discovery
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| DS-01 | **PASS (renders) / FAIL (image)** | Dashboard tag 10 "E2E Deals 1715" (image + store 11 + product 152) appears in the customer home tab strip (`DS-01-home-tag.jpg`). **But the stored image URL is malformed**: `https://assets2.freshyo.in/https://<r2-host>/meatfarmer-dev/tags/…` (double-prefixed) — `createProductTag` passes the presigned URL through `scaffoldAssetUrl()` instead of extracting the key. |
|
||
|
||
### E. Slots
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| SL-01 | **PASS** | The add dialog for SKU 153 offered exactly the assigned slot with the admin's window (`Sun, 11 Oct • 7-8 PM`, slot 2033) and not the unrelated same-day slots — membership + local-time display correct. |
|
||
| SL-05 | **FAIL (UI) / PASS (backend)** | With `updateSlotCapacity{2033,isCapacityFull:true}`: **backend rejects a stale submit** — `placeOrder` → "Selected delivery slot is at full capacity. Please choose another slot." — but the **customer UI still offers the full slot as selectable** and happily put the item in the cart (`SL-05-full-slot-dialog.jpg`, `SL-05-full-slot-accepted-into-cart.jpg`). Capacity restored afterwards. |
|
||
|
||
### F. Cart / checkout summary
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| CT-01 | **PASS** | Same SKU from Home and from Search: floating bar `₹235 • 1 Item`, expanded sheet subtitle `SUBTOTAL ₹235` + `FREE DELIVERY PROGRESS 78% / Needed +₹65`, full cart line ₹235 with slot — identical identities/quantities/subtotals across all three surfaces (`CP-01-BL-01-cart-304-pre-coupon.jpg` sequence). |
|
||
| CT-04 | **PASS** | 2-line cart (₹434) survived a cold relaunch; the floating bar rendered `₹434 • 2 Items` right after relaunch (before/after catalog load), no empty-cart flash, quantities intact. |
|
||
|
||
### G. Addresses / bounds
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| AD-01 | **PARTIAL** | Fresh customer 1314 created; first address submitted **without** the Default flag → stored with `isDefault: true` (correct "first address becomes default"), `regionId: 1`. **But `boundId: null`** — with no coordinates the address has no bound (`PD-01`-adjacent note; relates to AD-04's "No Location is attached to address" prompt, which the app does show: `+ Attach Current Location`). Cart/checkout contact display for that account was not exercised (would need switching the sim's session). |
|
||
| AD-05 | **PASS** | A1 selected: Delivery Fee **₹20**, nudge "Add products worth ₹26", To Pay **₹293**. A2 selected: Delivery Fee **₹200**, nudge "Add products worth **₹526**" (= 800 − 274), To Pay **₹473** — cart lines + coupon preserved across the switch (`AD-05-bill-with-A2.jpg`). Order placed with A2: **#2495, Total ₹473.6** matching the preview, and the persisted order carries `deliveryCharge 200` (below). Observed flap: one selection of A2 briefly rendered "Your cart is empty" before restoring — recorded as an inconsistency, not reproducible on the second attempt. |
|
||
|
||
### H. Bill arithmetic
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| BL-01 | **PASS (parity) / PARTIAL (exact boundary)** | Below threshold: bar/cart/checkout all show the ₹20 fee with the correct nudge (235 → +₹65; 268 → +₹26 basis post-discount). Above threshold: ₹304 → `FREE DELIVERY UNLOCKED`, fee ₹0 in cart and checkout. Persisted order #2495 keeps `deliveryCharge: 200` (A2) matching the quote, and the success screen shows the same `₹473.6`. The plan's exact **T−1/T/T+1** triple was not constructed (catalogue-specific totals); the coupon-crossing case below covers the interesting boundary. |
|
||
|
||
### I. Coupons
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| CP-01 | **PASS (flow) / FAIL (accounting)** | Coupon 43 `E2E10PCT` (10%, max ₹50, min ₹200) offered to the customer ("E2E10PCT … Save ₹30.4"), applied in cart, carried into checkout ("Coupon E2E10PCT 10% off"), order #2495 created with total **₹473.6 = preview** (cap/min respected, single use recorded server-side; coupon then invalidated by cleanup). **Accounting defect:** the persisted order stores `discountAmount: 47.36` (= 10% of the ₹473.6 *payable*, i.e. including the ₹200 delivery fee) while the actually applied discount was ₹30.4 on items — persisted components (item 304 − 47.36 + 200) do **not** reconstruct the stored total 473.6. Matches the plan's "money parity is a high-risk gap" observation. |
|
||
|
||
### J. Orders / fulfillment
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| OR-01 | **PASS** | #2495 (2 items, COD, coupon, address A2, region 1): success screen `Order #2495 Total ₹473.6`; My Orders shows `#ORD2495` with items/prices, `SHIPPING STATUS: success`, `Delivery: ₹200`, `₹473.6`; admin A `getOrderDetails` returns the order with items 8374/8375; **admin B → "Order not found"**. |
|
||
| OR-04 | **PASS** | Package item 8374 → `updateOrderItemPackaging` ok; whole order packaged; `updateDelivered` → admin detail `isPackaged: true, isDelivered: true`, both items packaged; customer detail shows the **DELIVERED** badge, all three stepper nodes green (`Placed → Packed → Delivered`), COD/`Delivered on 10 Oct 2026, 6:00 PM` (`OR-04-order-detail-status.jpg`). (Admin actions were driven through the same mutations the admin UI calls; the admin app's own tap-through was not repeated.) |
|
||
|
||
### K. Flash delivery
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| FL-01 | **PASS** | Flash enabled for region 1 via `superadmin.regions.updateRegionConstants` (`isFlashDeliveryEnabled: true`; note: writing the key through `updateBoundSettings` did **not** propagate — only the region-constants path did). Flash surface activated (`1 Hr Delivery`, notice text, flash bar `No Flash Items / ₹500 more`). Added a flash-eligible item, flash cart showed `Item ₹233 / Delivery Fee ₹69 / Add products worth ₹267 / To Pay ₹302`; checkout → **"1 Hr Order Placed! 1 Hr Order #2496 Total: ₹302"**. Admin: order 2496 has `isFlashDelivery: true`, `deliveryCharge: 69`, **no slot**, item ₹233. Flash restored to `false`. **FL-02 incidentally verified**: `placeOrder` with non-flash SKU 153 + `isFlashDelivery: true` → "Product 153 is not available for flash delivery". **Suspected filter defect:** `admin.order.getAll {flashDeliveryFilter: true}` returned an empty list although 2496 is a flash order (input shape may differ; needs confirmation). |
|
||
|
||
### O. Payments (incidental)
|
||
|
||
| ID | Status | Evidence |
|
||
|---|---|---|
|
||
| PAY-01 | **PASS** | Checkout shows "Pay Online (Coming Soon) — UPI, Cards, Netbanking" greyed out with no radio selection; COD works end to end (two orders). |
|
||
|
||
---
|
||
|
||
## Full-plan cases NOT executed in this run
|
||
|
||
Everything below is **NOT RUN by this session** (the 13:10 report covers them; several were
|
||
re-verified incidentally and are noted):
|
||
|
||
- **RG-02, RG-04, RG-05, RG-07, RG-09, RG-10, RG-11** — not run. Related observations from this
|
||
run: no `superadmin.regions.delete` procedure exists (region 2 created for RG-01 cannot be removed
|
||
through the API); cold launch re-resolves GPS and resets regional state (RG-08 evidence);
|
||
the A2-select flap (AD-05) hints at the same reconciliation area as RG-07.
|
||
- **AU-03…AU-10** — not run in this session (password-path login verified for AU-02; no OTP triggered
|
||
anywhere). The fresh customer `9000011715` (+ default address) was left in place for AU-05/AU-10 work.
|
||
- **PD-03, PD-04, PD-06…PD-13** — not run. Related: **`admin.product.deleteProduct` is broken**
|
||
(`Failed query: delete from "product_info" … returning …`, 500) — PD-13-relevant;
|
||
`admin.store.deleteStore` correctly refuses while products exist; `createProduct` is not atomic
|
||
with the upload claim (left product 151 behind).
|
||
- **DS-02…DS-08** — not run. DS-01 bug documented above; `deleteProductTag` also fails
|
||
("Failed to delete image", 500) once a tag has an image (tag 10 could not be removed).
|
||
- **SL-02, SL-03, SL-04, SL-06, SL-07, SL-08** — not run (SL-05 above).
|
||
- **CT-02, CT-03, CT-05…CT-11** — not run. Incidental: quantity steppers work on cards and in the
|
||
dialog; removal asks for confirmation; checkout is read-only and has "Back to Cart"; the expanded
|
||
sheet shows progress vs threshold.
|
||
- **AD-02, AD-03, AD-04, AD-06…AD-09** — not run. Incidental: deleting an address attached to an
|
||
order is correctly refused ("Address is attached to an ongoing order. Please cancel the order
|
||
first."); deleting the only/default address is refused ("Cannot delete default address. Please set
|
||
another address as default first."); the app shows "No Location is attached to address" with an
|
||
attach action.
|
||
- **BL-02…BL-07** — not run except the BL-01 notes above; the coupon accounting mismatch is direct
|
||
evidence for BL-03/BL-04/BL-05.
|
||
- **CP-02…CP-11** — not run (coupon create/apply/order path verified for CP-01; user-based and
|
||
region-scoped variants untouched).
|
||
- **OR-02, OR-03, OR-05…OR-12** — not run except OR-01/OR-04 above; multi-slot suborders were not
|
||
exercised (both orders used a single slot / flash).
|
||
- **FL-02…FL-06** — FL-02 negative path verified via API; the rest not run (flash was disabled again).
|
||
- **OP-01…OP-08, RC-01…RC-06, SEC-01…SEC-08, PAY-02…PAY-05** — not run this session. Incidental
|
||
security observations: cross-region order/product/store/slot access denied; staff B removed
|
||
mid-run — its token subsequently failed (consistent with SEC-08's expectation).
|
||
|
||
## Findings summary (new, this run)
|
||
|
||
1. **`admin.product.deleteProduct` fails on D1** — `Failed query: delete from "product_info" … returning …` (500). No workaround except suspend.
|
||
2. **`admin.product.deleteProductTag` fails with an image** — `Failed to delete image` (500); the tag stays.
|
||
3. **`createProductTag` stores a malformed image URL** — `https://assets2.freshyo.in/https://<r2>/…` (double prefix).
|
||
4. **Full slot remains selectable in the customer UI** (SL-05): only the backend rejects it.
|
||
5. **Coupon accounting mismatch** (CP-01/BL-03/05): `discountAmount 47.36` recorded vs ₹30.4 applied; stored components don't reconstruct the stored total.
|
||
6. **`createProduct` is not atomic** with the image claim — a failed claim leaves a half-created product (and its SKUs) behind.
|
||
7. **Flash enable via `updateBoundSettings` doesn't propagate** to `essentialConsts`; `updateRegionConstants` does. Suspected `admin.order.getAll {flashDeliveryFilter:true}` filter defect.
|
||
8. **Address without coordinates gets `boundId: null`**, and the active browsing bound then falls back to the address's bound (A2 → ₹800 threshold) after selection — consistent with the guidelines' "explicit address selection isn't persisted" note, and the app flaps ("cart empty") once during that switch.
|
||
9. Rejected OOS/full-slot submissions surface as HTTP 500 / `INTERNAL_SERVER_ERROR` with a correct message (known ApiError flattening).
|
||
10. Harness: iOS `clearState` no-op; region sheet hangs on reuse; MCP 60 s request ceiling.
|
||
|
||
## Fixture state after the run (cleanup performed)
|
||
|
||
Removed/neutralised: staff B removed · coupon 43 invalidated · fixture slots 2033/2034 deactivated ·
|
||
region-1 fixture products 152 and orphan 151 suspended (renamed `ZZ … (delete-me)`) ·
|
||
flash reverted to `false` · bound 1 constants back to 300/20 · slot 2033 capacity restored ·
|
||
order 2495 delivered, order 2496 left as placed.
|
||
Left behind (blocked by missing/broken APIs): **region 2 + bound 3** (no delete endpoint),
|
||
stores 11 "E2E Store 1657" and 12 "E2E B Store 1702" (refuse while products exist, products only
|
||
suspendable), **tag 10** (delete broken), region-2 product 153 (region-1 admin can't touch it),
|
||
fresh customer `9000011715` + its default address 880, address 879 (attached to order 2495).
|
||
Bound-2 settings are charge 200 / threshold 800, which matches the configuration documented in the
|
||
13:10 report.
|
||
|
||
## Suggested report columns for the next run
|
||
|
||
case · platform · run ID · status (PASS/FAIL/PARTIAL/BLOCKED/NOT RUN/NEEDS-POLICY) · expected/actual ·
|
||
evidence (screenshot name + API probe) · entity IDs · follow-up owner.
|