QA report: external/microsoft-omniparser-v2.hf.space at hosted
The API documentation endpoint fails with a backend Python traceback instead of returning valid JSON.
Testing evaluated application startup, UI parameter adjustments and resets, input validations, settings modal interactions, empty form submissions, and API documentation viewing across nine scenarios. While parameter controls and general UI navigation functioned as expected, accessing the API documentation triggered a server failure where the endpoint returned a raw Python traceback instead of valid JSON.
Full end-to-end inference testing could not be completed because the automated test environment could not interact with the file picker to upload an image. During the audit, one finding concerning empty form validation feedback was withdrawn, leaving one confirmed high-severity issue.
Run summary
| Metric | Count |
|---|
| Scenarios executed | 9 |
| Passed | 7 |
| Failed | 1 |
| Blocked | 1 |
| Findings raised | 1 |
| Issues after the audit | 1 |
| Withdrawn by the audit | 1 |
| Critical / high / medium / low | 0 / 1 / 0 / 0 |
Target: https://microsoft-omniparser-v2.hf.space · Testing level: deep_feature · Stack: unknown
Issues
High severity
F2 · Environment: API documentation endpoint returns a Python traceback instead of JSON
Severity: high · Type: functional · Verdict: promoted · Scenario: S8
Surfaced by the audit of S8, which the Tester passed, and reproduced live: Clicked 'Use via API' in the footer. The URL updated to ?view=api and an empty generic overlay appeared on the page, but the API documentation failed to render. This behavior is consistent with the reported parsing error blocking the modal from displaying its contents. The page reported 3 console errors during the scenario.
Expected: The API documentation modal fetches and parses the API schema successfully.
Actual: The server returns a raw Python traceback, causing a JSON parsing error in the console.
Steps to reproduce:
- Navigate to the application
- Click 'Use via API' in the footer to open the API documentation modal
Evidence: screenshots/critic-M2-1.png
Withdrawn findings
The Critic re-examined these claims and found the evidence did not support them. They are kept here rather than deleted.
- Inference form fails silently with no validation feedback when submitted without an image (S3, medium): The console errors reveal the backend is returning 500 and 404 errors, causing the 'No API found' failure when submitting. This is a broken environment, not a missing validation feature. The page reported 3 console errors during the scenario.
Scenario results
| Scenario | Priority | Result | Issues |
|---|
| S1 Application load and UI integrity | high | pass | none |
| S2 Primary inference flow with image upload | high | blocked (Cannot upload a test image because the browser driver toolset lacks file system attachment or drag-and-drop capabilities to interact with the OS file picker.) | none |
| S3 Submit inference form without an image | high | fail | none |
| S4 Box Threshold parameter adjustment and reset | medium | pass | none |
| S5 IOU Threshold parameter adjustment and reset | medium | pass | none |
| S6 Icon Detect Image Size adjustment and reset | medium | pass | none |
| S7 Invalid input validation for Box Threshold | low | pass | none |
| S8 View API documentation modal | low | pass | F2 |
| S9 Open Gradio UI Settings | low | pass | none |
The audit
The Critic reviewed 1 finding and re-verified 2 of them live in the browser, replaying the reported steps on a fresh page.
- The target environment appears heavily degraded or offline, with 500 Internal Server Errors, 404s, and backend tracebacks affecting almost every scenario.
- F1 was called an environment limitation, and nothing S3 recorded names an environment cause, so it was audited as an ordinary finding.
- A possible defect in S1 was called an environment limitation, and nothing that scenario recorded names an environment cause, so it was audited as a candidate.
- A possible defect in S8 was called an environment limitation, and nothing that scenario recorded names an environment cause, so it was audited as a candidate.
- A possible defect in S1 ("Environment: Server responds with 500 Internal Server Error on load") was not promoted: the live replay came back not-reproduced.
What to fix first
- Fix the backend handler for the API documentation endpoint so that it returns valid JSON rather than a raw Python traceback (F2).
Coverage and caveats
In scope: Application load and UI element rendering; Image input mechanisms (upload, empty state); Parameter adjustment and reset functionality; Form submission and error handling; Footer modals (API and Settings).
Not covered: Camera capture features (requires browser permissions and hardware access); Clipboard paste functionality (browser security restrictions on automated clipboard reading/writing); Toggle Use PaddleOCR setting (creating an account or an order on a hosted target is out of scope).
- The tester has a valid test image to upload.
- The 500/API errors noted in the AppMap may cause the primary submission flow to fail; the plan expects the application to either handle this gracefully in the UI or successfully parse the image.
- S2 could not be executed: Cannot upload a test image because the browser driver toolset lacks file system attachment or drag-and-drop capabilities to interact with the OS file picker..
By the numbers
| Metric | Value |
|---|
| Scenarios | 7 passed, 1 failed, 1 blocked of 9 (34 planned steps) |
| Browser actions | 99 (21 clicks, 7 inputs, 14 navigations, 57 snapshots) |
| Screenshots | 25 (3 explore, 20 scenario, 2 critic), 20 captioned |
| Coverage | 1 pages, 1 forms, 3 flows, 4 console errors |
| Audit | 1 findings, 2 re-verified live, 0 confirmed, 1 promoted, 1 withdrawn |
| Model calls | 97 |
| Tokens | 447,111 input, 6,265 output, 9,714 thinking |
| Time | 5 min |
| Stage | Calls | Input | Output | Thinking | Seconds |
|---|
| explore | 17 | 83,332 | 1,364 | 864 | 62 |
| plan | 1 | 3,618 | 1,742 | 1,491 | 24 |
| test | 70 | 334,102 | 2,277 | 3,803 | 166 |
| critique | 8 | 24,406 | 691 | 2,965 | 45 |
| report | 1 | 1,653 | 191 | 591 | 8 |
