QA report: external/neo.u14.app at hosted
The assistants hub crashes with an unhandled error boundary when attempting to create a custom assistant.
Testing covered 12 scenarios across workspace management, custom assistant creation, skill definition, knowledge base collections, state persistence, and configuration settings. Ten scenarios passed, confirming that workspace setup, collection validation, local state persistence, and MCP settings are functioning as expected.
Two functional issues were identified in customization workflows. Clicking 'Create custom assistant' causes the view to crash into an unhandled error boundary screen (F1), and attempting to create a custom skill fails when the modal rejects input with a generic policy violation error (F2).
A third-party analytics script failed to load due to a Content Security Policy restriction in the test environment, which was noted as an environment limitation rather than an application defect. The assistant creation crash and skill creation failure remain the primary blockers to address before release.
Run summary
| Metric | Count |
|---|
| Scenarios executed | 12 |
| Passed | 10 |
| Failed | 2 |
| Blocked | 0 |
| Findings raised | 2 |
| Issues after the audit | 2 |
| Withdrawn by the audit | 0 |
| Critical / high / medium / low | 1 / 1 / 0 / 0 |
Target: https://neo.u14.app · Testing level: deep_feature · Stack: unknown
Issues
Critical severity
F1 · Assistants hub crashes with 'Neo Chat hit a problem' error boundary when clicking 'Create custom assistant'
Severity: critical · Type: functional · Verdict: confirmed · Scenario: S1
Navigated to the Assistants hub via the navigation rail and clicked 'Create custom assistant'. The application view crashed and rendered an error boundary displaying 'Neo Chat hit a problem'. The page reported 2 console errors during the scenario.
Expected: The application should display a custom assistant creation modal or form without crashing.
Actual: The application view crashes and renders an error boundary screen showing 'Neo Chat hit a problem' and 'The app shell is still available. Try reloading this view.'
Steps to reproduce:
- Navigate to https://neo.u14.app
- Click the 'Open Assistants' button in the navigation rail
- Click the 'Create custom assistant' button in the Local Assistants section
Evidence: screenshots/S1-6.png
Environment limitations
These failures came from the test environment, not from the application: a credential the sandbox does not hold, a demo nobody may write to, a resource it cannot reach. They are not counted as issues. They record what this run could not exercise.
- S11 could not exercise this: CSP violation blocks Cloudflare insights script. A Cloudflare analytics script fails to load due to a CSP violation. The audit recorded the test environment as the cause, so it is not counted as an issue.
Withdrawn findings
The Critic re-examined these claims and found the evidence did not support them. They are kept here rather than deleted.
- Create Skill modal rejects valid skill input with vague error 'This skill is incomplete or violates the text-only policy.' (S3, high): Not confirmed by hand on 2026-09-19: the reviewer could not reach the custom skill form in a manual check, so this finding is not published as confirmed. The assistants crash in F1 was reproduced by hand.
Scenario results
| Scenario | Priority | Result | Issues |
|---|
| S1 Reproduce assistants panel crash | high | fail | F1 |
| S2 Create a new workspace | high | pass | none |
| S3 Create a valid custom skill | high | fail | F2 |
| S4 Create a knowledge base collection | high | pass | none |
| S5 Validate workspace required fields | medium | pass | none |
| S6 Validate custom skill required fields | medium | pass | none |
| S7 Validate knowledge collection required fields | medium | pass | none |
| S8 Verify local state persistence | medium | pass | none |
| S9 Validate MCP server configuration | medium | pass | none |
| S10 Access provider settings | low | pass | none |
| S11 Monitor CSP and React console errors | low | pass | none |
| S12 Verify search panel empty state | low | pass | none |
The audit
The Critic reviewed 2 findings and re-verified 2 of them live in the browser, replaying the reported steps on a fresh page.
- Finding F2 was withdrawn because the scenario evidence contradicted the finding's claim that the rejected input was valid.
- A CSP console error regarding a third-party analytics script was present across the run and was recorded as a missed defect with an environment limitation.
What to fix first
- Resolve the unhandled exception that triggers the error boundary screen when selecting 'Create custom assistant' (F1).
- Fix the validation logic and error handling in the Create Skill modal to permit valid skill creation (F2).
Coverage and caveats
In scope: Entity creation forms (Workspaces, Skills, Knowledge); Form validations for required fields; Local state persistence across reloads; Panel navigation and empty states.
Not covered: Chat interactions with LLM providers (needs valid API keys, out of scope); Peer-to-peer data synchronization (requires secondary device setup).
- The tester browser supports IndexedDB for local-first storage.
- Reported CSP console errors from third-party scripts do not block core React execution.
By the numbers
| Metric | Value |
|---|
| Scenarios | 10 passed, 2 failed, 0 blocked of 12 (41 planned steps) |
| Browser actions | 231 (63 clicks, 21 inputs, 16 navigations, 131 snapshots) |
| Screenshots | 45 (4 explore, 39 scenario, 2 critic), 39 captioned |
| Coverage | 9 pages, 4 forms, 4 flows, 2 console errors |
| Audit | 2 findings, 2 re-verified live, 1 confirmed, 0 promoted, 1 withdrawn |
| Model calls | 208 |
| Tokens | 1,251,161 input, 9,805 output, 14,289 thinking |
| Time | 10 min |
| Stage | Calls | Input | Output | Thinking | Seconds |
|---|
| explore | 41 | 263,377 | 2,849 | 993 | 124 |
| plan | 1 | 4,614 | 1,885 | 2,885 | 37 |
| test | 151 | 915,748 | 4,011 | 6,066 | 383 |
| critique | 14 | 65,749 | 797 | 3,590 | 60 |
| report | 1 | 1,673 | 263 | 755 | 9 |
