How we prove it works

1Testing Goal

Prove that orders move correctly from intake to dispatch, that stock figures stay exact and explainable, that bots never act beyond their level, and that nothing that works today breaks.

2Critical User Journeys

Airline: New order (Qantas-style PO, MEL) → invoice no. → appears on Production Planning → print → confirm → allocate → schedule (Interstate · MEL · Roadmaster) → dispatch → live stock reduced.

Non-airline: Phone order → availability check → delivery date → invoice no. → print → confirm → one line short → Inform Alex → Stock IN → allocate → schedule (Non-airline) → dispatch.

The release must not ship if either journey is broken.

3Regression — existing app (run every release)

• Login works; wrong password shows error; logout ends access
• Stock search + product card + Stock IN/OUT (units and boxes); OUT > stock blocked
• Products filters (search, category, subcategory) and admin add/edit/delete/set stock
• Sync from XLSM: upload → preview → apply → numbers match the file; discard changes nothing
• Production Planning: edit cell autosaves, colours, add column/week, save/load plan, print
• Tartlet Orders: as-at date, date sort ↑↓, colour picker, totals and NEED TO MAKE, save/load, print
• Users: add/edit/delete; last admin cannot be deleted; change password
• Export stock and transactions open in Excel
• After deploy: existing users can still log in with their existing passwords

4Orders — intake

• Airline order with 3 lines saves; status NEW; order no. generated
• Cartons entry converts using pack size
• Missing customer / dispatch date / lines → clear message, nothing saved
• Duplicate PO for same customer → warning
• Bespoke free-text line saves; cannot be allocated until linked
• Attachment: PDF uploads; .php or > 10 MB rejected; file not reachable by direct URL
• Non-airline availability check: green/amber/red correct; suggested date sensible

5Invoice, print, confirm

• Correct Xero label per entity (SkyCrest vs Annie Makes Cakes)
• Cannot pass INVOICED without invoice no. (admin override needs reason)
• Kitchen ticket prints on one A4 page for ≤ 25 lines
• Confirmation text includes lines, date, method; marking sent stamps time and user

6Packing & allocation

• Allocate full: available-to-promise drops; live stock unchanged
• Partial allocation shows remaining short
• Two orders competing for the same stock: second sees reduced availability
• Double-click Allocate does not double-allocate
• Packing role can allocate but cannot edit orders or customers

7Production planner link

• Airline lines appear in the right port/date column, summed per product
• Editing the order updates the planner; cancelling removes it
• Linked cells cannot be typed over; manual (unlinked) cells still work
• Non-airline short appears on Alex's missing-items list

8Schedule & dispatch

• Tomorrow's schedule groups: Interstate (by port), Sydney, Non-airline
• Booking type/ref saves; print and .xlsx correct
• Dispatch creates one out transaction per line with order no. in the note; live stock = before − dispatched
• Dispatch blocked when not fully allocated; admin partial dispatch needs reason
• Undo dispatch creates in transactions; original rows remain
• Failure mid-dispatch rolls back everything (no half-dispatched order)

9Agents (rule-based)

• All bots disabled on first deploy
• Kill switch stops a bot within one cron cycle
• L2 bot creates a proposal only — no stock/order change until Approve
• Reject requires a reason; same proposal not re-raised
• Approve records decided_by and time
• Dispatch Scheduler at 14:00 lists tomorrow's orders and "not ready" ones correctly
• Reconciler flags a deliberately negative stock item
• Digest shows the right items for office, packing, Alex
• Two overlapping cron runs do not both execute (lock)
• --agent= cannot be triggered from a browser URL

9a. Dashboard & analytics

• Login lands on the dashboard for each role with the right panels
• Tiles match the Orders / Packing / Dispatch pages exactly
• Seen is recorded only after ≥ 10 s visible; switching tab before that does not count
• Admin "seen today" board correct; non-rostered / on-leave staff shown as "off"
• Analytics totals for Jul 2025–Apr 2026 match AIRLINE_ANALYSIS.md (216 POs, 5,262 ctns)

9b. Hermes Reminder Bot

• Staff who has seen the dashboard gets nothing
• Not seen: step 1 message at +15 min, SMS at +30, call at +45 (only if call_allowed), manager at +60
• Reply "OK" or pressing 1 on the call stops the ladder and is logged
• Max one call per person per day; nothing outside 06:00–18:00, weekends or leave
• Unknown numbers / non-staff never contacted
• Wrong or missing bearer token → API returns 401; API exposes no prices or customer data
• Hermes down for 30 min → admin dashboard shows heartbeat alert
• Kill switch stops reminders within one cycle

10AI Order Intake

• dnata and Gate Gourmet PO templates: 100 % of 20 sample POs parsed correctly (PO no., kitchen, codes, cartons)
• 20 past non-airline orders (PDF PO, HTML PO, email text): ≥ 90 % of lines correct (code + qty)
• Low-confidence fields highlighted; unknown products flagged, not guessed silently
• Nonsense / spam email → no draft or a draft clearly marked "not an order"
• API failure → email stays in queue, retried later, error on Admin → Agents
• API key not present in any web-served file or page source

11Responsive & accessibility

• Packing and Dispatch pages usable on a 768 px tablet with touch
• No horizontal overflow on Orders list at 1024 px
• All inputs labelled; keyboard can reach buttons; status not shown by colour alone (chips have text)

12Security

• POST without CSRF token rejected
• User role cannot reach admin-only actions by URL
• stock.db, includes/, backups and attachments return 403/404 over HTTP
• All output escaped (try <script> in customer name and notes)

13Release Blockers

Do not release if: login or existing passwords break · stock figures change without a transaction · dispatch removes wrong quantities · a bot changes stock/orders without approval · existing Plan / Tartlet / Sync pages regress · stock.db not backed up before deploy.

14Test Result Format

Test · Expected · Actual · Device/browser · Steps · Screenshot · Severity · Status