Hands-on testing
Testing standard
See what we have tested, what we have not tested, and what a product must pass before we publish test results.
Current status
No catalog product has completed the physical test protocol yet. Current product pages use public product details that we checked. They are not hands-on reviews. Recommendation receipts use checked manufacturer and authorized-seller records, stated cost assumptions, and versioned matching rules. An untested product page must say that it uses checked sources and is not a hands-on review.
The Lab publishes our dated weekly price checks and analysis of published product details. A price check observes the market, not the machine. An analysis calculates from the source record. Neither is a physical test. The planned physical test steps, funding queue, coverage, and data format are recorded at /lab/protocol. Testing starts when the queue has enough funding.
We publish a test claim only when we physically test that exact product and record the test. A manufacturer claim, owner story, review roundup, or estimated score cannot become an “our tests show” statement.
Software checks
Find my setup and Fix a shot use fixed software rules. They do not measure taste or equipment. Automated tests check fit exclusions, diagnosis order, undoing a follow-up change when it does not help, old saved data, and bad saved data. Browser tests check complete flows, page reloads, keyboard use, privacy-safe measurement fields, and supported mobile widths.
Passing software tests shows that the published rules run consistently. It does not show that a cup improved, that a puck channeled, or that every machine and coffee will respond the same way.
What a physical test must include
Before a page receives an observed physical test label, its test record must complete the current pre-registered Lab protocol and include:
- Retail U.S. production unit, firmware if applicable, purchase or loan status, and test dates.
- Room conditions, water composition, dry coffee amount, basket, grinder, and warm-up definition.
- At least three repeated timing or measurement runs, with raw values retained.
- Stored and operating dimensions, including portafilter, reservoir, hopper, cord, and steam-wand travel.
- Ready time, hands-on time, one-drink time, two-drink time, and cleanup time reported separately.
- Sound measurements with device, distance, room background, and operating task documented.
- Energy readings with meter and cycle boundaries documented when energy is claimed.
- Photographs or video sufficient to identify the exact unit and reproduce the test steps.
What each label means
- Checked product detail
- Public primary or authorized-retailer sources checked. This does not claim hands-on performance.
- Dated price check
- A dated first-party check of an advertised price. It observes the market, not the machine, and is not a performance claim.
- Observed physical test
- We used a named unit under a published protocol and list the test date and limits.
- Measured
- A numeric observation lists the tool, method, units, and repeated runs.
- Long-term
- The exact unit has a published ownership period and maintenance log. Short reviews do not qualify.
Samples and paid relationships
If a manufacturer supplies or discounts a unit, we show that fact near the result. Brands may correct objective product facts before publication, but they do not receive score approval, copy approval, or a promise of coverage. Affiliate availability cannot turn a product into a test winner. Read the editorial policy and affiliate disclosure.