59 KiB
Freshyo cross-app E2E sanity and regression plan
Status: proposed checklist from code inspection. Partially executed since — see
docs/test reports/e2e-sanity-test-results-2026-10-10-1310.md and
docs/test reports/e2e-sanity-test-results-2026-10-10-1650.md, with the defect register in
docs/test reports/e2e-errors-2026-10-10.md and per-defect detail in
docs/test reports/e2e-findings-2026-10-10.md.
Scope and approach
This plan follows the customer-facing and operational flows in apps/admin-ui,
apps/admin-web, apps/user-ui, apps/web-ui, apps/superadmin-ui, the vendor
page in apps/fallback-ui, the backend tRPC routers, SQLite helpers, schema,
regional cache generation, and existing E2E flows. It is a workflow review, not
a claim that every source/generated/dependency file has been read line by line.
The important E2E chain is:
Superadmin configuration → admin changes → DB/cache publication → customer discovery → cart/checkout → persisted order → admin/vendor fulfillment → customer status.
A success toast alone is not proof. Check the result in the other app, and for money/ownership tests also check authoritative API data. Do not mock the backend in these E2E journeys. Mocked screen/component tests remain a separate layer.
Assertion rules (apply to every case)
- Money — for any case that touches a total, assert the identity
items − discount + fees = payableon the persisted order payload, and again on the order-detail response the customer/admin actually reads (not only the cart preview or the success screen). - Cache/config — before asserting a served value, record the DB value and the served version/timestamp so "stale" can be told apart from "wrong". A stale read inside the publication deadline is not a defect; a stale read after it is.
- Mutations — assert the absence of unintended writes (row counts, name lookups, sibling rows), not only the response body or a toast.
- Failures — a request that fails must leave no partial write; a request that is merely slow must not leave any screen in a permanent loading state.
- UI defects — before filing one, confirm from the screenshot/hierarchy that the expected screen was really open (a bottom sheet dims the background; no backdrop means it never opened).
Fixture lifecycle (confirmed policies)
- Regions cannot be deleted (by design). Reuse one long-lived test region per environment, or neutralise a throwaway one by renaming it and shrinking its bound polygon. Do not plan cleanup around region deletion.
- Bounds are deletable (
superadmin.regions.deleteBound) but are refused when it is the region's only bound or when addresses still reference it. A region must always keep at least one bound. - Deleting a product is a soft delete: it disappears from catalog/cart/slots/availability while its rows remain, so order history keeps the original product name and billed amount, and the name becomes reusable.
- An address may exist without a bound and is then served by the active bound for the region (see
G/BLcases); missing coordinates are an address-quality warning, not a bound error.
Test fixtures
- Dedicated staging backend, DB, KV, assets, queues and notification destinations. Never point mutation, account-deletion, notification or payment tests at production.
- Two regions, A and B, each with its own admin and store. Customer identity is global; catalog, slots, orders and operational access are region-sensitive.
- A has two bounds A1/A2 with priorities 1/2 and deliberately different charges. B has B1. Include an overlapping polygon and a point outside every polygon.
- Customers U1/U2 plus a fresh customer with no addresses or orders. Use disposable accounts for suspension, password changes, first-order and deletion cases.
- Product P with two SKU variants; product Q; an offer; a combo with a combo-only component; one regular-only SKU; one flash SKU with a distinct flash price.
- Two active scheduled slots S1/S2 with future freeze times, plus an inactive slot, a frozen slot and a full-capacity slot. Assign SKUs explicitly.
- Unique run prefix for product/tag/coupon/snippet names. Product and tag duplicate checks are currently global; do not assume the same name is allowed in B.
- Freeze/delivery times relative to run start with generous margins. Separately exercise midnight/IST transitions; do not rely on hardcoded dates or slot IDs.
How to run this plan eventually
- All login flows use password authentication only. OTP login, OTP delivery and OTP verification are out of scope; do not trigger SMS as part of test setup.
- Provision test customers with passwords and read credentials from test environment variables, not hardcoded flow files.
- Maestro: admin/customer native apps; Playwright or Maestro Chromium: web apps. A coordinator should carry region/product/SKU/slot/order IDs between app flows.
- Use UI setup when testing creation itself. For independent checkout/edge cases, use a sanctioned staging fixture API so every test does not recreate all data.
- Reusable flows: sign in, select region, create product, create slot, add address, add item, place COD order, find order by ID, cancel disposable order.
- Prefer stable
testID/accessible labels, including dynamic IDs based on entity IDs. Avoid percentage taps and matching the first product/address in a list. - Form entry: type one field at a time with the repo pattern
(
tapOn→inputText→hideKeyboard→scrollUntilVisible→tapOn), as intests/e2e/add-product.yaml. Never type into a field that may sit behind the keyboard; verify each field's value before the next. - Stalled vs failed requests: catalogue/consts requests are capped at 90 seconds. A stalled request must surface as an error/retry within that bound — an eternal spinner is a FAIL, and the report should say whether the request was still pending. Retrying after an ambiguous failure must not create a duplicate.
- Wait for business state with a bounded deadline. Catalog regeneration is
debounced by roughly 30 seconds (
CacheCreator), and constants polling is 60 seconds; allow a configurable staging deadline, initially 120 seconds. Refresh discovery/catalog when appropriate; do not replace checks with sleeps. - Run publication assertions on both a fresh customer session and an already-open session. Do not clear customer storage before a persistence/recovery test.
- Record run ID, input fixtures, expected/actual totals, entity IDs, screenshots, and failure step. Harness errors are BLOCKED, not PASS or proven app defects.
- Cleanup only this run's disposable data using approved fixture tooling. Keep screenshots and failing fixtures for diagnosis. Do not run migrations as setup.
Recommended daily sanity subset
Start with these 24 cases, reusing safe baseline fixtures where possible:
RG-01, RG-03, RG-06, AU-01, AU-02, PD-01, PD-02, PD-05,
DS-01, SL-01, SL-05, CT-01, CT-04, AD-01, AD-05, BL-01,
CP-01, OR-01, OR-04, FL-01, plus one case from each of the newer
high-yield classes: P-01 (write atomicity), Q-01 (config/cache propagation),
R-01 (4xx contract), S-01 (post-delete order history).
These cover startup, authentication, publication, region isolation, availability, cart persistence, addresses/bounds, money, coupons, COD, fulfillment and flash — and now also the failure-atomicity, cache-propagation, error-contract and destructive-lifecycle classes where real defects have been found. Run the full checklist before release; mandatory API security checks should also be a release gate. High-risk financial cases deserve priority even if slower.
A. Regions, bounds and configuration
| ID | Steps / journey | Required result |
|---|---|---|
| RG-01 | Superadmin creates region A with a valid bound → assigns an admin → customer opens region selector and selects A. | A is discoverable; configuration/cache loads without undefined amounts or broken asset requests. An empty catalog has an intentional empty state. |
| RG-02 | Try creating a region without bounds; add malformed GeoJSON or duplicate bound priorities. | Validation explains the error; no half-created region/configuration remains. |
| RG-03 | Fresh customer launch with GPS inside A, first signed out and then signed in. | A resolves once; catalog and delivery threshold load without needing login. |
| RG-04 | Deny GPS, return an outside-service point, or make resolution time out; repeat with/without a saved region. | Saved region survives when location is unusable; otherwise a usable manual chooser appears, not an endless spinner. |
| RG-05 | Delay GPS resolution → manually select B before the response → let A's GPS response arrive. | Late startup response does not undo the customer's manual choice. |
| RG-06 | Create distinct products/stores/slots/tags/banners in A/B → browse A, then B, then A. | Each region shows only its own content; no cached content/prices from the previous region leak. |
| RG-07 | Add regular and flash items in A → request switch to B → cancel, then repeat and confirm. | Cancel preserves A and both carts; confirm clears both carts and address/slot selections and loads B. |
| RG-08 | Keep the app open → move GPS from A to B → later cold-launch from B. | No continuous automatic switching while open; cold launch performs the one-shot resolution and resets regional state if changing region. |
| RG-09 | Give U1 addresses in A1/A2 → sign in/browse A; remove an eligible nondefault address and refresh; repeat with no A addresses. | Active browsing bound is lowest numeric priority among U1's A address bounds, else A's effective bound. B addresses do not participate. |
| RG-10 | Superadmin changes A1/A2 priorities, then edits A1 charge; separately edit A's support/welcome/flash settings and global app version. | Effective bound changes as specified; bound edit affects that bound, region edit affects A's bounds, global edit affects all regions; B's regional settings stay unchanged. |
| RG-11 | Resolve GPS in overlapping bounds and on a polygon edge → compare with active-bound selection from addresses. | Geo-resolution follows numeric priority within a region; browsing prices follow address/effective-bound rules, not an accidental GPS-bound override. Define cross-region overlap policy separately. |
B. Authentication and account lifecycle
| ID | Steps / journey | Required result |
|---|---|---|
| AU-01 | Signed-out customer browses/searches/products → opens Cart and Me. | Public browsing works; Cart and Me show login walls without forced redirects; other tabs remain usable. |
| AU-02 | From Cart wall, use password login → repeat from Me wall. | Correct credentials sign in and return to the originating screen; cart contents survive the sign-in. No OTP is sent by the password shortcut. |
| AU-03 | Submit wrong password, malformed mobile and unaccepted terms → correct them and retry. | Clear validation/auth error; no authenticated state from a failure; retry succeeds. |
| AU-04 | Password-login using each supported identifier (mobile/email) on a password-provisioned customer; try an unknown identifier and then valid credentials. | Supported identifiers authenticate the same account; unknown credentials fail cleanly; no SMS is triggered. |
| AU-05 | Register a disposable customer → try duplicate mobile/email and invalid inputs → sign in using supported identifiers. | Registration persists exactly once; duplicates/errors do not create incomplete accounts. |
| AU-06 | Sign in → cold-launch → log out → sign in as U2. | Session persistence is correct; logout restores walls; U2 cannot see U1's addresses/orders/coupons or submit with U1's checkout selections. Explicitly decide local cart behavior on account switch. |
| AU-07 | Edit name/profile fields and profile image → relaunch → inspect customer in admin. | Saved values/images agree across profile and admin customer details. |
| AU-08 | Password-authenticated disposable customer changes password → signs out → tries old password → signs in with new password. | New password works, old password fails, and the customer can recover from the failed attempt without losing account data. |
| AU-09 | Admin suspends U1 while U1 has a populated cart → U1 attempts order → admin unsuspends → retry. | Suspended user cannot place order, including with an existing token; unsuspension restores allowed behavior. |
| AU-10 | Disposable customer requests account deletion with wrong mobile, then correct mobile → relaunch/re-authenticate. | Wrong confirmation is rejected; successful deletion logs out and leaves no access to the deleted account. Verify order-retention policy without damaging shared fixtures. |
C. Stores, products, variants, offers and combos
| ID | Steps / journey | Required result |
|---|---|---|
| PD-01 | Admin A creates a store with image → creates P with two SKUs/images/features/prices → creates active future slot with those SKUs → customer selects A and searches P. | Correct variants, price, unit/features, store, images and delivery slot appear; item can be added. Also inspect store detail, not only Home. |
| PD-02 | With P published in A, customer selects B; admin B opens product/store/slot lists and selectors. | P/A's store and slot do not appear anywhere in B. |
| PD-03 | Admin edits P's name/descriptions/features/variant image → customer refreshes list and detail. | Latest content is consistent in Home/search/store/detail; replaced images do not stay stale or break other variants. |
| PD-04 | Create duplicate product name, missing SKU features, missing store or nonpositive price → then submit valid product. | Invalid save is rejected without partial product/SKU rows; valid retry creates exactly one product. |
| PD-05 | Put P in customer cart → admin opens the product's Stock dialog and marks one variant of multi-SKU P out of stock (leaving the other in stock) → customer refreshes → admin restores it. | Unavailable state propagates for that variant only; the sibling variant stays in stock and orderable; the product list shows it as Partial, not In or Out; restoration works without reinstalling. |
| PD-06 | Admin suspends or soft-deletes one SKU of multi-SKU P → browse/search/detail and existing cart. | Removed SKU is no longer purchasable; sibling SKU remains usable; no phantom payable cart line or crash. |
| PD-07 | Admin adds a third SKU → assigns it to a slot → later removes that SKU. | New variant is discoverable and orderable; removal does not corrupt old orders or remaining variants. |
| PD-08 | Admin changes regular/MRP/flash prices via Prices Overview → customer with P already in cart refreshes. | Availability-only publication updates card/detail/cart prices; flash uses flash price; B is unchanged. Prices are addressed by skuId (renamed 2026-10-10); a request with no price fields is rejected, and an unrelated SKU/product is never touched. |
| PD-09 | Admin toggles P's Offer flag → customer opens Offers tab and detail → remove flag. | SKU appears/disappears in Offers correctly, with working Add to Cart and price. The order-again route is currently this Offers screen. |
| PD-10 | Admin creates combo C with component SKUs → schedules C → customer opens Combos, detail and purchases C. | Correct components/features/images and combo price; ordered line is the intended combo SKU, not duplicate component charges. |
| PD-11 | Mark a component combo-only → publish C → search Home/store/Offers and open C. | The combo-only SKU must be absent from every customer list surface (Home, search, Offers, store pages, Combos strip) while still being served inside its parent combo. Decision 2026-10-10: backend enforcement at placeOrder is not required — a direct API call is the only way to reach one, and that is accepted. |
| PD-12 | Make combo component unavailable/suspended/deleted → inspect C and attempt checkout; restore component. | Agreed combo availability policy is enforced consistently, without silently selling invalid components. This policy needs explicit confirmation. |
| PD-13 | Admin edits store name/image → customer refreshes Stores and P detail; try deleting a store with products. | Store changes propagate; deletion is either safely rejected or follows an explicit dependency policy, never orphaning a broken customer catalog. |
D. Discovery, tags, groups, ordering and banners
| ID | Steps / journey | Required result |
|---|---|---|
| DS-01 | Admin creates dashboard tag with image, P/Q and related store → customer opens its Home section. | Correct category/image/products/store links; no unrelated regional SKUs. |
| DS-02 | Edit tag membership and product order → customer refreshes → delete tag. | Membership/order propagate; deleted tag disappears without hiding products from unrelated sections. |
| DS-03 | Admin reorders dashboard tags (tagsOrder) in A → browse A and B. |
A follows its saved order; B retains its own order. No global tag-order override. |
| DS-04 | Admin sets Popular Items and All Items Order → refresh customer Home. | Configured visible SKUs appear in intended order; removed/unavailable IDs do not crash the section. |
| DS-05 | Create product group containing P/Q → attach group to slot → edit group's membership → inspect customer slot view. | Intended group-to-slot behavior holds for additions/removals. Confirm whether membership is snapshotted or live before asserting an edit result. |
| DS-06 | Create banner with image/P/Q → first inspect while inactive → activate → tap it. | Inactive banner is not presented as active; active banner routes to intended products/content with matching image. |
| DS-07 | Edit banner image/link/order → deactivate or delete → customer refreshes. | New content/navigation/order propagate; removed/inactive banners no longer produce broken navigable content. Verify serial/order and active-flag semantics. |
| DS-08 | Search exact/partial product and SKU names, no-match text, and a B-only product while in A; navigate result to detail and add. | Results and details are region-correct; no-match state is useful; selected result is the SKU actually added. |
E. Scheduled slots and capacity
| ID | Steps / journey | Required result |
|---|---|---|
| SL-01 | Admin creates active S1 with future freeze/delivery and P/Q → customer opens slot view and add-item dialog. | Only assigned SKUs offer S1; shown delivery/close times match admin values in local time. |
| SL-02 | Admin adds/removes a SKU on S1 and edits delivery/freeze time → customer refreshes. | Membership and timing update on all entry points; removed SKU cannot retain a purchasable S1 assignment. |
| SL-03 | Create inactive, past-delivery and past-freeze slots → customer checks dialog/cart/slot view. | They cannot be selected for a new order; deliberate existing-order history still renders. |
| SL-04 | Add P to S1 → let freeze time pass before checkout → refresh or submit directly from stale checkout. | Customer is prompted to choose a valid alternative or blocked; backend must not accept a closed slot merely because the UI was stale. |
| SL-05 | Admin marks S1 full → customer checks P's slot choices; repeat while customer's checkout is already open. | Full slot is removed/disabled and stale submission is rejected without creating an order. |
| SL-06 | Admin reopens S1 capacity → customer refreshes and places order. | S1 becomes selectable and orderable again. |
| SL-07 | P has S1/S2 → add using S1 → make S1 invalid → relaunch or open cart. | Valid fallback is selected consistently without deleting P or changing its quantity; if none remain, line is unavailable and not charged. |
| SL-08 | Create slot across midnight/IST boundary → compare admin, customer selection, checkout and saved order. | Exact date/instant is preserved, not yesterday/tomorrow due to timezone conversion. |
F. Cart, quantities and checkout summary
| ID | Steps / journey | Required result |
|---|---|---|
| CT-01 | Add P from Home → add Q from Search/Store → inspect floating bar, expanded bar and full Cart. | Same SKU identities/quantities/subtotals everywhere; no duplicate or missing line from entry-point differences. |
| CT-02 | Add existing P again, then use Update Item → inspect quantity and totals. | Duplicate add merges by SKU as intended; explicit update sets quantity instead of accidentally adding twice. |
| CT-03 | Increase/decrease quantity in dialog, floating bar and cart; test product increment step and zero removal/cancel-removal. | Consistent allowed quantities and totals; zero removes only after confirmed UI behavior. Fractional increments need an explicit contract because order input currently requires integers. |
| CT-04 | Add multiple lines → cold-launch without clearing storage → open Cart before/after catalog loads. | Lines appear automatically as stores load; no permanent empty cart, manual-refresh dependency or lost quantities. |
| CT-05 | Assign P/Q different slots → change cart slot → leave/re-enter Cart and go to Checkout. | Slot selection remains as promised by product policy; no silent reversion or extra quantity from reconciliation. |
| CT-06 | Keep both regular and flash carts populated → edit/remove one → navigate back and forth. | Carts remain independent; no conversion, duplicated additions or unexpected deletion of the other cart. |
| CT-07 | Admin makes one of several cart items unavailable → customer proceeds; repeat with all unavailable. | Partial checkout clearly identifies actual ordered/charged items; all-invalid cart blocks checkout. Verify whether remaining invalid lines should survive successful checkout. |
| CT-08 | Enter instructions on Cart → select address/coupon → go to Checkout → Back to Cart → edit instructions → return. | Instructions survive and the latest value is submitted; Checkout is read-only for items/slots/address/coupon/instructions. |
| CT-09 | Inspect Cart layout → Checkout layout → use Back to Cart. | Address selector is above Bill Details, instructions editor is on Cart, hidden “Go to 1 Hr Delivery” switcher stays absent; Checkout shows grouped summary and valid back navigation. |
| CT-10 | Rapidly add two different SKUs and repeatedly tap quantity controls → reload Cart. | No lost storage updates, duplicate local IDs, negative quantities or stale floating count. |
| CT-11 | Change quantity/coupon/slot on Cart after visiting Checkout → reopen Checkout; try stale/direct checkout URL. | Latest valid selection is summarized and submitted; malformed/stale route parameters do not order a different or empty cart. |
G. Addresses, location attachment and selected-bound rules
| ID | Steps / journey | Required result |
|---|---|---|
| AD-01 | Fresh customer adds first address while browsing A, without checking Default → inspect Me and Cart. | Address belongs to A and becomes default; Cart can select it and Checkout shows correct contact/address. |
| AD-02 | Add second address → make it default → edit contact/address without sending isDefault, and with an explicit value → relaunch. |
Exactly one default persists; an omitted isDefault must leave the flag untouched, an explicit false clears it and an explicit true moves the default; edits never leave the user with zero defaults or select an unrelated address. |
| AD-03 | Submit missing fields, bad phone/pincode and no region → correct and save. | Clear 4xx validation for each field — phone = exactly 10 digits starting 6/7/8/9, pincode = exactly 6 digits (rule 2026-10-10); no partial saved address; retry succeeds once. |
| AD-04 | Add address without coordinates → inspect “No Location is attached to address” → attach GPS location; repeat denying permission and using a supported Maps link. | Missing-location notice is accurate; successful attachment persists; denied/invalid-link path is recoverable without overwriting address data. Define whether missing coordinates block ordering. |
| AD-05 | U1 has A1/A2 addresses → select A2 on Cart → compare fee before/after selection → place order. | Selected address's bound overrides browsing active bound; catalog remains A; saved order total agrees with A2 config. |
| AD-06 | While browsing A, explicitly choose B as a new address's region; separately edit an existing B address while browsing A. | Address region and bound are coherent; editing preserves/derives the correct bound, never borrowing A's current browsing bound for B. |
| AD-07 | A cart populated, default address in B → load address selector → explicitly tap B address → cancel, then confirm. | No automatic foreign-default selection; cancel retains A/cart; confirm switches B, clears both carts and returns Home. |
| AD-08 | Delete nondefault unused address → try the only address, the default, and an address attached to an ongoing order → cancel/finish the order and retry. | Allowed deletion persists. Protected deletions must explain how to proceed (4xx + message), the only address can never be deleted, and the reason is surfaced to the customer in a BottomDialog rather than a silent failure. |
| AD-09 | Add/edit/delete address whose bound controls active-bound choice → refresh discovery and relaunch. | Active bound recalculates and prices/fees update without stale user/address cache. Selected-address override remains internally consistent. |
H. Bill arithmetic and bound-specific fees
Use different minRegularOrderValue and freeDeliveryThreshold values, distinct
regular/flash prices, and at least one two-slot cart. Equal fixture values can hide bugs.
| ID | Steps / journey | Required result |
|---|---|---|
| BL-01 | Set bound delivery threshold T and fee D → order subtotal T−1, T and T+1 without coupon. | Same fee rule in floating bar, Cart, Checkout, success, persisted order and admin details; exact threshold boundary is correct. |
| BL-02 | Set regular minimum M different from free threshold T → try totals below M, between M/T and above both. | Policy settled 2026-10-10: there is no minimum order value — orders of any amount must be accepted, and minOrder must not be conflated with the free-delivery threshold. Assert that a small order below the free-delivery threshold still places and is charged the fee. |
| BL-03 | Coupon lowers subtotal from above T to below T → repeat with post-coupon total exactly T. | Agreed pre/post-discount fee policy holds on client and backend, including boundary. |
| BL-04 | P/Q in S1/S2, paid delivery and coupon → place order → sum all generated order totals and fees. | One checkout creates expected slot suborders; total fee is charged only once, coupon is allocated correctly, and the summed total equals displayed To Pay. (Accepted 2026-10-10: sibling sub-orders show their own payable total and are not required to itemise the prorated discount; only the coupon-holding sub-order reconciles item/discount/fee.) |
| BL-05 | Use noninteger prices/percentage discounts and several quantities → compare previews and persisted amounts. Include a capped percentage coupon and a multi-slot split (the cap is prorated per suborder). | Consistent rounding/currency precision; truncation in labels does not conceal overcharge or contradictory success totals; the discount reported by the order-detail endpoint equals the discount actually deducted for the coupon-holding suborder. |
| BL-06 | Select A1 then A2, with visibly different charges/thresholds → repeat regular and flash checkout, then send a mismatched client boundId. |
The delivery address's bound is authoritative for fees/thresholds; the client boundId is used only when the address has no bound (then the region's effective bound). Mismatched client ids cannot change the charge; no undefined/NaN/negative totals. |
| BL-07 | Customer opens Checkout → admin changes SKU price or selected-bound fee → submit. | No silent difference between quoted and authoritative total; agreed reprice/refresh/confirmation policy is followed. |
I. Coupons and offers on orders
| ID | Steps / journey | Required result |
|---|---|---|
| CP-01 | Admin A creates general percentage coupon → eligible customer adds P → applies coupon → places COD order → admin checks total. | Correct discount and cap; saved total equals preview; usage recorded once. |
| CP-02 | Create flat coupon with/without max cap → test small/large carts → remove coupon. | Discount never creates negative payable amount; removal restores correct subtotal/fee. |
| CP-03 | Create coupon with minimum spend → test below/exactly/above; change quantities after selection. | Eligibility recalculates; invalid selected coupon cannot silently stay applied or bypass backend validation. |
| CP-04 | Create U1-specific coupon → compare U1/U2 eligible and My Coupons screens → place U1 order. | Only U1 can see/apply it; usage belongs to U1. Direct U2 application must also be denied (API check). |
| CP-05 | Create a coupon that the admin form let you scope to a SKU → cart with that SKU and cart with other SKUs → place each. | Policy: coupons are not product/SKU-based. A coupon discounts the whole cart; any SKU scoping sent to the admin API is ignored at pricing time. Assert the discount applies regardless of SKUs. |
| CP-06 | Set maximum uses per customer to one → place order → try second use and two simultaneous first uses. | Limit is enforced, including concurrent requests; one multi-slot checkout counts as one use as intended. |
| CP-07 | Create first-order-only coupon → fresh customer uses it → tries again; separately cancel first order. | Eligibility matches defined first-order/cancellation policy, not inconsistent tracker/order-count behavior. |
| CP-08 | Admin expires/invalidates/edits coupon while customer has it selected at Checkout → submit. | Invalid stale coupon is rejected; changed valid coupon cannot produce unnoticed quote mismatch. |
| CP-09 | Admin creates reserved coupon → U1 redeems secret code in My Coupons → applies personal coupon → U2 retries secret. | Redemption produces correct personal coupon; one-time secret cannot be claimed by both customers. Test simultaneous redemption too. |
| CP-10 | Redeem malformed, expired or first-order-ineligible reserved code → then valid code. | Correct rejection without burning valid codes or leaving half-created coupons; valid retry works. validTill is the single deadline for both claiming the secret and spending the coupon. |
| CP-11 | Create coupon in A → customer browses B and attempts redemption/application. | Policy: coupons are not region-based (global). A coupon created in A must be visible and applicable in B. (Only user scoping is real: U1-only coupons must still be denied to U2.) |
J. Orders, fulfillment, cancellation and after-order edits
| ID | Steps / journey | Required result |
|---|---|---|
| OR-01 | Customer orders P/Q in A using COD/address/instructions/coupon → checks success and My Orders → admin A finds same order ID → admin B searches it. | Matching SKUs/quantities/slot/address/notes/coupon/total in customer and A admin; B cannot access it. Ordered cart is cleared according to agreed policy. |
| OR-02 | Order items in S1/S2 → check success, history and admin lists. | Each generated slot order is discoverable, correctly grouped and dated; success does not obscure the second order. Total/fee parity follows BL-04. |
| OR-03 | Place order → edit instructions from customer order menu → admin refreshes; admin adds internal notes. | User notes propagate; admin notes remain correctly distinguished from customer instructions. The customer path rejects editing a delivered/cancelled order; the admin path may annotate delivered orders (business decision 2026-10-10). |
| OR-04 | Admin packages one item then all items, verifies packaging and marks Delivered → customer refreshes tracking. | Item and aggregate packaging state agree across admin lists/details; customer sees correct fulfillment/delivery status. |
| OR-05 | Admin reverses an accidental packaged/delivered toggle where supported → refresh both apps. | Supported reversal is consistent; forbidden transitions show clear errors. No cancelled order resurrected accidentally. |
| OR-06 | Customer cancels pending COD order with reason → admin list/detail and vendor page refresh. | Cancelled status/reason/actor persist; cancelled order is not fulfillable or included in vendor production totals; COD refund is not shown as cash owed. |
| OR-07 | Admin cancels pending order with reason → customer history/detail and next-order glimpse refresh. | Cancellation is attributed to admin and visible to customer; next-order summary no longer treats it as upcoming. |
| OR-08 | Try blank reason, second cancellation and cancellation after Delivered; cancel one of two slot suborders. | Invalid cancellations reject without duplicate refund records; other suborder stays intact. |
| OR-09 | Admin removes delivery charge from order → customer/admin details refresh. | Saved payable total and fee update together, exactly once; unrelated suborders stay unchanged. |
| OR-10 | Place order → admin changes prices → verify old order unchanged → run explicit Rebalance on chosen slot(s). | Normal catalog edits do not reprice historical orders. Explicit rebalance reprices item prices only and freezes the historic delivery fee and the discount actually charged (policy 2026-10-10); delivered/cancelled orders are skipped. After rebalancing each touched order still reconciles as items(priced) − frozen discount + frozen fee = stored total. |
| OR-11 | Create many A/B orders → exercise admin filters and pagination and customer My Orders pagination. | No missing/duplicate A orders due to B's high IDs; filters match actual statuses; customer sees own orders across regions with region labels. |
| OR-12 | Submit Place Order once (and verify the in-app re-entrancy guard) → then replay two concurrent API submissions; note the residual exposure on ambiguous failure/retry. | Exactly one order group per user action; the app's submittingRef + disabled button must prevent a double-tap. Decision 2026-10-10: server-side idempotency is explicitly not required — concurrent/malicious API submissions may create two orders; the retry-after-ambiguous-failure path (RC-04) is the known residual exposure. |
K. Flash delivery
| ID | Steps / journey | Required result |
|---|---|---|
| FL-01 | Enable flash in A → create flash-eligible SKU with price different from regular → customer adds to flash cart → selects A address → places COD order → admin opens Flash orders. | Flash price/fees are consistent, order has flash flag and no scheduled slot, delivery summary is correct and only flash cart is cleared. |
| FL-02 | Regular-only, out-of-stock and suspended SKUs → browse/add via flash entry points. | Noneligible items cannot be ordered through Flash or a regular dialog shortcut. |
| FL-03 | Disable flash in A while B remains enabled → customer refreshes A/B; submit stale A flash checkout. | Correct regional UI availability; backend rejects disabled flash orders, B remains available. |
| FL-04 | Admin removes flash eligibility or changes flash price while SKU is in flash cart. | Eligibility/price refresh everywhere; stale order cannot silently sell a disallowed SKU or use regular price unexpectedly. |
| FL-05 | Use coupons with minimum/cap on flash subtotal near free-delivery threshold. | Eligibility and discounts use the intended flash price basis; all bill surfaces and saved order agree. |
| FL-06 | Trigger staging flash enable/disable scheduled jobs → inspect all configured regions and customer app. | Expected schedule and regional values publish to KV; UI transitions without restart. This is a scheduled-job integration test, not a real-time daily wait. |
L. Vendor, delivery operations, support and notifications
| ID | Steps / journey | Required result |
|---|---|---|
| OP-01 | Admin creates slot-specific snippet for P → customer orders P+Q → vendor opens snippet URL. | Only attached SKU lines from matching noncancelled slot orders appear; quantities/summary agree; Q is not exposed by P's snippet. |
| OP-02 | Vendor toggles P packaging → admin refreshes order details/list → reverse toggle. | Same order-item packaging state persists; other vendor/customer items remain unchanged. |
| OP-03 | Create permanent snippet → vendor selects S1/S2 → expire/delete a slot-specific snippet and retry old link. | OUT OF SCOPE (2026-10-10): vendor snippets are not a current concern. Kept for traceability; not executed. (The permanent-mode metadata lookup failing is dropped with it.) |
| OP-04 | Admin edits delivery sequence and address map coordinates → reopen delivery view and order detail. | Sequence/map changes persist and stay within region; operational coordinate override does not silently mutate customer's region/bound identity. |
| OP-05 | Customer raises complaint on own order with photo → admin A views and resolves → customer views response. | Correct order/customer/photo/body/response/status across apps; admin B cannot resolve A complaint. Repeat with general complaint without an order. |
| OP-06 | Customer creates product review/rating/photo → admin opens matching review and responds → customer refreshes detail. | OUT OF SCOPE (2026-10-10): the review feature is no longer maintained. Kept for traceability; not executed. (Any review findings — e.g. createReview failing for SKU-split products — are dropped with the feature.) |
| OP-07 | Admin records customer incident/negativity note → reopen customer details/order operational display. | Incident belongs to intended customer/order, score persists and regional access follows policy; do not assume it automatically suspends the user. |
| OP-08 | In isolated staging destinations, send targeted notification, place order for admin push/deep link, cancel for Telegram, and resolve complaint for customer push. | Correct payload/recipient/navigation; unrelated customers receive no targeted push. Verify intended regional recipient policy. Include queue retries and physical-device receipt separately from queue acceptance. |
M. Recovery, cross-platform and configuration robustness
| ID | Steps / journey | Required result |
|---|---|---|
| RC-01 | Customer already open while admin rapidly edits product/price/slot in A and B → wait/refetch discovery and catalog. | Final changes publish for both regions; one pending region does not overwrite another; clients do not mix catalog/availability/slot versions. |
| RC-02 | Cold-launch with saved cart/config and slow catalog/availability/slot responses; temporarily fail asset requests then restore network. Also stall (accept but never answer) one catalogue endpoint. | Recoverable loading/error state, no permanent zero-priced items, cart deletion or endless spinner; a stalled request must fail within the 90 s request cap and clear the loading indicator. No offline order submission. |
| RC-03 | In controlled staging, remove a bound's KV payload or return corrupt local config → customer opens region. | Backend rebuild/backfill and client cache recovery work; foreign bound's cached config is never reused. |
| RC-04 | Disconnect during product/image/address save and before/during order request → reconnect and inspect stored result before retry. | Error is recoverable; no half-uploaded published image, duplicate address/product/order, or false success; retrying an ambiguous failure reports the existing record instead of creating a second one. Distinguish precommit from postcommit request loss. |
| RC-05 | Run critical customer/admin flows on iOS and Android; admin-web/customer-web in Chromium, including refresh/direct links/back. | Same business results with platform-specific storage, permissions, pickers and routing. Web is a separate implementation, not assumed equivalent because APIs are shared. |
| RC-06 | Superadmin changes supported internal version/store URLs → open older/newer app builds; cancel image/location permissions and retry. | Update behavior matches version policy and OS URL; permission denial is recoverable. Verify regional support/welcome text rather than hardcoded Mahabubnagar copy in B. |
N. Mandatory API-backed security and invariant checks
These supplement the UI journeys. UI visibility alone cannot prove authorization. Use staging requests with distinct identities and verify both response and absence of unintended writes. Expected denials below are requirements, not claims about the present implementation.
| ID | Steps / journey | Required result |
|---|---|---|
| SEC-01 | Customer token/anonymous request calls staff CRUD; staff token calls superadmin region/staff/global-config APIs; try legacy admin global-config mutation too. | Unauthorized identities are denied; hiding an admin screen is not the access control. |
| SEC-02 | Admin B supplies A IDs to product/store/slot/tag/group/banner/coupon/snippet/order/complaint reads and mutations; mix A/B relation IDs in an otherwise valid save. | No cross-region reads/mutations/associations; rejected mixed save leaves no partial state. |
| SEC-03 | U2 supplies U1 address/order/complaint/payment IDs to read/edit/cancel/delete operations. | Ownership is enforced, not only authentication; U1's data is unchanged. |
| SEC-04 | Order request mixes A region with B SKU/slot/bound/address, or a SKU not assigned to that slot. Also test a global product (regionId null). | All submitted identities/relations are validated server-side; no foreign or incoherent order accepted (4xx). A global product is valid in every region and may be attached to any region's slot; a product of another region is rejected, and a scheduled order's SKU must still be carried by the chosen slot. |
| SEC-05 | Submit empty items, zero/negative/fractional quantities, invalid/null regular slot, deleted/suspended/OOS SKU, frozen/inactive slot and full slot. | Backend enforces actual purchasability/quantity/slot rules; bypassing UI guards cannot create an invalid order. |
| SEC-06 | U2 applies U1 coupon directly; apply expired/invalid/foreign-region/SKU-ineligible coupon; race usage and reserved redemption. | User scoping and one-time limits are authoritative and atomic; expired codes are rejected. Global coupons are expected to apply in any region and to any SKU (policy), so those two probes assert success, not rejection. |
| SEC-07 | Vendor request edits packaging using an unrelated item ID; expired snippet queries another slot; anonymous caller tries same mutation without valid snippet authority. | Vendor access is constrained to authorized snippet/region/items. A public vendor URL is not authority to edit every order item. |
| SEC-08 | Reuse revoked/deleted staff identity/token → attempt region-bound mutation; change staff region in superadmin then retry old session. | Access reflects current staff assignment/existence, not a stale token's region. |
O. Payment/refund readiness (not current COD sanity)
PaymentAndOrderComponent currently disables online payment as “Coming Soon”;
RazorpayPaymentService methods are commented-out stubs. Do not report live
online payment/refund journeys as supported or passing today.
| ID | Steps / journey | Required result |
|---|---|---|
| PAY-01 | Current customer Checkout attempts selecting online payment. | It remains clearly disabled; COD works without any gateway call. |
| PAY-02 | Once implemented, single- and multi-slot sandbox online checkout → verify callback → inspect customer/admin orders. | Correct discounted group amount and INR minor units; one payment covers intended suborders, all become paid exactly once. |
| PAY-03 | Sandbox declines/cancels/times out payment → retry, double callback and late success/webhook. | No duplicate payment/order, accurate pending/failed/success state, consistent cart and retry behavior. |
| PAY-04 | Paid customer order cancellation → admin partial/full refund → gateway settles → customer views status; retry refund. | Amount never exceeds collected payment, authorization enforced and no duplicate refund; COD does not enter online-refund flow. |
| PAY-05 | Forged signature, another customer's merchant order ID, duplicate callback and mismatched amount/currency. | Backend rejects forged/foreign payment, verifies authoritative ownership/amount and handles callbacks idempotently. |
P. Write-path integrity, atomicity and idempotency
Every case here asserts the absence of unintended writes on failure, using row counts, name lookups and sibling rows — not just the response. These come from defects where a request failed after committing.
| ID | Steps / journey | Required result |
|---|---|---|
| P-01 | createProduct where one image URL is valid and the other was never uploaded (previously: 500 "Failed to claim upload URL" with the product already stored). |
4xx naming the offending key; no product/SKU/market-stats rows exist (verify by name lookup and counts); retrying with the same name does not report "already exists". |
| P-02 | updateProduct with a bad image while also deleting an old image — try it in both newImageUrls and skus[].images, and also try reusing another product's image key. |
Rejected before any change: old images still present, name/prices/variants unchanged. Only genuinely new image references (keys the product does not already own) need a pending upload; the product's own images may be re-sent. |
| P-03 | Bad upload URL on tag create, tag update, review create and review response. | Target row absent (tag/review not created, tag not modified) and the old image cleanup has not run. |
| P-04 | Double-submit each admin create form (rapid double tap) and retry after an ambiguous 5xx. | Exactly one record; the second attempt returns a clean conflict/validation error, never a duplicate. The message must be right and the status must be 4xx (an ApiError must not surface as HTTP 500 — see A19). |
| P-05 | Kill the client mid-save (background the app, force-quit, or abort the request). | Either nothing was written or the record is complete — never a half-created product/tag/slot/address; the UI recovers on relaunch. |
| P-06 | Delete a tag/group/banner whose stored image cannot be cleaned up (missing or malformed object key). | The DB delete still happens (cleanup failure is logged, not fatal); no undeletable row remains. |
Q. Config and cache propagation (same region)
Publication of a write to the customer app is a first-class assertion. Record the served version/timestamp and the DB value so stale and wrong can be distinguished; the staging deadline is 120 s.
| ID | Steps / journey | Required result |
|---|---|---|
| Q-01 | Admin marks a future slot full (and later clears it) → customer slot list, product next-delivery and checkout slot picker. | The change reaches every surface inside the deadline; the served catalogue version advances; the slot cannot be selected anywhere while full, and reappears after clearing. |
| Q-02 | Write a region-scoped key through updateRegionConstants (must take effect) and through the bound-settings endpoint (must be rejected). |
Correct endpoint changes the effective value; wrong endpoint returns 4xx naming the right endpoint and stores no stray row; the resolved payload contains the value from its own table. |
| Q-03 | Change a bound's charge/threshold → floating bar, cart, checkout and the next order. | All surfaces and the next order use the new fee after the deadline; a stale value after the deadline is a FAIL with the version evidence attached. |
| Q-04 | Create/edit a tag image (fresh upload) → customer tag surface and the tag list. | The stored image reference is a key/normalised URL: no double asset-domain prefix, no %3F… leftovers; the image request returns 200; deleting the tag succeeds. |
R. API contract enforcement
UI filtering is not protection. These cases call the API directly and assert the shape of failures.
| ID | Steps / journey | Required result |
|---|---|---|
| R-01 | Trigger each business rejection directly: out-of-stock SKU, full slot, delete store with products, delete default address, delete the region's only bound, address attached to an ongoing order. | 4xx with the specific message (not 500/INTERNAL_SERVER_ERROR); the resource is unchanged. |
| R-02 | Send unknown keys, out-of-scope keys, wrong-type values and foreign ids to product/tag/slot/order/const mutations. | 400 that names the offending key/id; no partial write and no misleading success: true. |
| R-03 | Multi-SKU product P: mark one variant out of stock via the API, then order it and its sibling. | Only the targeted variant is refused; the sibling still orders; the persisted product state matches the per-variant call. |
| R-04 | Repeat identical deletes/toggles/cancellations twice. | Idempotent or a consistent 404/409 (never a 500, duplicate side effect, or resurrection of the row). |
S. Destructive lifecycle and post-delete semantics
| ID | Steps / journey | Required result |
|---|---|---|
| S-01 | Create product P, order it (COD), then admin deletes P → customer catalog/cart/slot/availability and the historical order detail. | P disappears from every live surface; the old order still shows the original product name and billed amount; P's name can be reused for a new product; deleting again is a clean 404/conflict. |
| S-02 | Delete a tag with products/images, a group with members, a banner with images, and a slot assigned to a product. | Either the children are handled (cascade) or the response is a clear 4xx naming the dependency; no orphan child rows remain in any case. |
| S-03 | Delete the only bound of a region, then delete a bound that an address references. | Both refused with a clear message; the region keeps taking orders and the existing address/bound still resolves. |
| S-04 | Attempt region deletion (superadmin) and then neutralise a throwaway region (rename + shrink its bound polygon). | Region deletion is not available (by design); the neutralised region stops capturing addresses and its customers are unaffected. |
| S-05 | Suspend and soft-delete a product and a variant, then reopen past orders, invoices and admin order detail. | Historical orders render unchanged (names, amounts, packaging state); only live surfaces change. |
| S-06 | Delete a store that still has products; delete a product's last remaining variant; delete an address used by an open order. | Each refused with the specific blocker and no partial delete of related rows. |
T. Permanent regression seeds for fixed defects
One permanent row per fixed defect so a regression is caught by the daily subset, not by chance.
| ID | Steps / journey | Required result |
|---|---|---|
| T-01 | Delete a product that has order history, then open that order. | Soft delete semantics: hidden from live surfaces, original name/amount still rendered, name reusable. |
| T-02 | Create a tag with a freshly uploaded image, read it back, then delete the tag. | Normalised image reference and a successful delete (no "Failed to delete image"). |
| T-03 | createProduct with a never-uploaded image. |
4xx and no product/SKU rows (formerly a 500 with an orphan). |
| T-04 | Write a region-scoped const key through the bound endpoint. | 4xx naming the correct endpoint (formerly success: true with no effect). |
| T-05 | Mark one variant of a multi-SKU product out of stock. | Sibling stays orderable (formerly both flipped). |
| T-06 | Request the slot list for region A. | Only region A's future, orderable slots are returned (formerly every region, including past ones). |
| T-07 | Stall one catalogue/consts request. | Fails within the 90 s cap and clears the loading state (formerly an eternal spinner). |
| T-08 | After a region switch, open the region sheet from the home bar. | Sheet opens with the region list (formerly a spinner/no-op path). |
Code observations that should influence priority
These are static observations, not executed bug reports. Where desired behavior is not yet defined, agree the contract rather than write a test that endorses a bug.
- Money parity is a high-risk gap. Cart uses
freeDeliveryThresholdagainst post-couponfinalTotal;placeOrderUtilusesminRegularOrderValuefor regular delivery against pre-discounttotalAmount. Backend percentage discount receives a suborder total including its delivery fee. SeeBL-01–BL-04:apps/user-ui/components/cart-page.tsxandapps/backend/src/trpc/apis/user-apis/apis/order.ts. - Global-config restriction must be tested at API level.
admin.const.updateGlobalConstantsstill usesprotectedProcedurein the inspected code. That common guard permits either customer or staff identity; removing the UI is not enough. SeeSEC-01andapps/backend/src/trpc/apis/admin-apis/apis/const.ts/trpc-index.ts. - Order validation needs negative requests. The inspected order path fetches
SKU by ID and checks slot capacity/flash eligibility, but does not visibly
validate every submitted region/SKU/slot relation, frozen state, suspension or
OOS flag. See
SEC-04/SEC-05; do not assume UI filtering is protection. - Address identity can drift.
AddressFormsubmits current browsing bound even when the form selects another region or edits a foreign-region address. SeeAD-06andapps/user-ui/src/components/AddressForm.tsx. - Coupon UI and backend eligibility differ. User coupon retrieval is not
region-filtered; checkout validation checks expiry/minimum/usage/first-order,
but does not visibly enforce all user/product/region restrictions. See
CP-04/CP-05/CP-11/SEC-06and the user coupon/order DB helpers. - Vendor authority needs a release gate. Packaging mutation is public and
accepts an item ID without snippet code; the selected-slot snippet query does
not visibly enforce expiration like the other snippet query. Permanent-link
UI initially calls a metadata endpoint that requires a slot. See
OP-03andSEC-07,vendor-snippets.tsand fallbackvendor-order-list.tsx. - Persistence tests must use current code. Older cart documentation describes
mount-only remove/re-add reconciliation; current
useReconcileCartSlotspreserves lines and updates their slot. Do not derive expected behavior from historical docs alone. SeeCT-04/SL-07, currentcart-query-hooks.tsx. - Existing E2E is a starting point, not complete coverage.
tests/e2e/add-product.yaml,add-slot-with-product.yamlanduser-order.yamlcover part of the main chain. The user-order flow assumes older auth/address placement (automatic login route and checkout address selection), and needs alignment with current Cart/Me login walls and read-only Checkout. Add regional selection and persisted-order/admin assertions. - Writes that fail after committing. Several endpoints perform the DB write first and a follow-up step
(upload-URL claim, image cleanup, KV mirror) afterwards, so a failing follow-up used to abort the request
with the row already stored (product, tag, review). Pre-validate, then write, then treat follow-ups as
best-effort — see section
P. - "Delete" means different things per entity. Product deletes are soft (rows kept for order history and
the name freed for reuse); tags/groups/banners/slots delete rows but must not be blocked by image cleanup;
bounds delete but never the last one and never while referenced; regions cannot be deleted at all. Cases
must assert the agreed semantics, not a generic "it is gone" — see
S. - Config writes can be silently ignored. Each constant is resolved from its own table, but the write
endpoints accept any key, so a wrong-scope write returns success and changes nothing (and leaves a stray
row). Scope validation now rejects those; keep a case for it — see
Q-02/R-02. - Request lifetime matters as much as correctness. Catalogue/consts requests have no natural bound, and a
request that hangs leaves loading states (e.g. the home refresh indicator) alive indefinitely. A 90 s
AbortController cap is in place on the customer app; assertions must distinguish stalled from failed —
see
RC-02/T-07. - Stock granularity is per SKU per region. The all-variants toggle cannot express one variant being out
and destroys a partial state; the admin app now chooses variants in a stock dialog. Assert per-variant
behaviour and the sibling staying orderable — see
PD-05/R-03.
Policies settled
-
Error status contract (2026-10-10): an
ApiError(msg, status)thrown in any procedure now reaches clients with the real status (400/403/404/409). Business rules thrown as plainErrorinside the db-helper packages are the one remaining exception (e.g. coupon validation) and still surface as 500. (assert these; do not re-litigate) -
Regions are not deletable — by product decision. Bounds are deletable, refused when it is the region's only bound or when addresses reference it.
-
An address may exist without a bound and is then served by the active bound: the least-priority bound among the customer's addresses in that region, else the region's effective (lowest-priority) bound. Missing coordinates are an address-quality warning, not a bound error.
-
Product deletion is a soft delete — the product leaves live surfaces but its rows stay, so order history keeps the original name and billed amount, and the name becomes reusable.
-
Money basis — the coupon discounts the item subtotal; the free-delivery threshold is decided on the post-discount subtotal; the delivery fee is added after the discount. Order-detail components must reconstruct the paid amount (see
BL-04/BL-05). -
Constant scope — every key belongs to one table (global/region/bound/legacy); writes through the wrong endpoint are rejected, never silently accepted.
-
Slot listing — the customer slot list is region-scoped and future-only (a frozen slot is not orderable).
-
Request cap — the customer app aborts catalogue/consts requests after 90 seconds so a stalled request surfaces instead of hanging.
Policies still open before turning every row into a hard assertion
- Is
minRegularOrderValuea hard order minimum or only informational? Which subtotal controls free delivery: pre- or post-coupon? - Which SKU-level coupons are exclusive, and can coupons cross regions?
- Should unavailable lines survive a successful partial checkout? Should cart instructions reset after success/region switch/logout? Current shared notes store is not explicitly cleared in those paths.
- Are fractional quantity steps supported? Dialog/cart stepping and order input validation must agree.
- Should combo availability depend on all components, and are slot-group memberships live or snapshots?
- Which orders may be explicitly rebalanced, and how do fees, prior coupons and multi-slot allocations survive repricing?
- Do cancelled first orders restore first-order coupon eligibility?
- Are customer/admin notifications regional or intentionally global? Current broadcast/admin-notify paths should not be assumed regional by UI labels.
- How are overlapping regions handled (legacy null region/bound addresses are now settled above)?
Keep these as explicitly named policy checks; do not silently mark ambiguous behavior PASS. Recommended report columns: case, platform, run ID, status (PASS/FAIL/BLOCKED/NOT RUN), expected/actual, evidence, entity IDs, follow-up.