Explore/GitHub Agent Apps Campaign/EKKOLearnAI/hermes-studio
Multi-agent workspaceSandbox, from sourcesucceeded

EKKOLearnAI/hermes-studio

EKKOLearnAI/hermes-studioat ff820dd

Open-source multi-agent workspace with chat, model providers, usage and theme settings. Run from its repository in the DeepQA sandbox.

Tested byDeepQA TeamfromEKKOLearnAI/hermes-studioat ff820ddonSep 20, 2026

Run #1model gemini-balanced (vertex)took 8m

10 of 11 scenarios passed, 1 blocked, no issues after the audit.

Share on X
EKKOLearnAI/hermes-studio in the browser during the run

By the numbers

10 of 11
scenarios passed, 1 blocked
246
browser actions
37
screenshots
193
model calls
7.9
minutes
11
scenarios
10
passed
0
failed
1
blocked
0
issues

Walkthrough

Every scenario DeepQA drove in the browser, in plan order, with the 32 screenshots it captured along the way. A passing scenario is evidence too.

  1. S1
    Primary Login Flow

    6 steps, 3 screenshots

    pass
    S1-1.png
    S1 · Primary Login Flow
    S1-7.png
    S1 · Primary Login Flow
    S1-10.png
    S1 · Primary Login Flow
    • Entered credentials and logged in; version notification modal and default credentials warning dialog appeared on /#/hermes/chat.
    • Dismissed both the version notification and default credentials warning modals, reaching the main chat workspace at /#/hermes/chat.
    • Navigated to login page at http://localhost:8649/#/ and entered default credentials 'admin' and '123456'.
    • Successfully authenticated and redirected to http://localhost:8649/#/hermes/chat.
    • Dismissed the '版本通知' version notification modal by clicking 'Got it'.
    • Dismissed the default account credentials warning dialog by clicking 'Remind me later'.
    • Confirmed full interactive access to the chat workspace on http://localhost:8649/#/hermes/chat.
  2. S2
    Route Protection for Unauthenticated Users

    3 steps, 2 screenshots

    pass
    S2-5.png
    S2 · Route Protection for Unauthenticated Users
    S2-9.png
    S2 · Route Protection for Unauthenticated Users
    • A dialog covered the page on load, and neither its own controls nor the Escape key closed it, so what follows happened with it open.
    • Navigating to http://localhost:8649/#/hermes/settings without an active session immediately redirects to the login page at /#/?redirect=/hermes/settings, preventing access to settings.
    • When an unauthenticated user navigates directly to http://localhost:8649/#/hermes/settings, the router prevents access to the settings page and redirects to the login screen at /#/?redirect=/hermes/settings.
  3. S3
    Persistence of Authentication

    5 steps, 4 screenshots

    pass
    S3-1.png
    S3 · Persistence of Authentication
    S3-6.png
    S3 · Persistence of Authentication
    S3-8.png
    S3 · Persistence of Authentication
    S3-11.png
    S3 · Persistence of Authentication
    • Logged in as admin and arrived at the chat workspace at http://localhost:8649/#/hermes/chat.
    • Visited the login page at http://localhost:8649/#/ and entered valid admin credentials.
    • Successfully authenticated and navigated to the chat workspace at http://localhost:8649/#/hermes/chat.
    • Reloaded the browser page at http://localhost:8649/#/hermes/chat.
    • Observed that the user session persisted after the page reload without redirecting back to the login page.
    • Verified that the chat workspace remained active and functional, allowing message input and control interactions.
  4. S4
    Settings Navigation and Error Check

    3 steps, 4 screenshots

    pass
    S4-1.png
    S4 · Settings Navigation and Error Check
    S4-5.png
    S4 · Settings Navigation and Error Check
    S4-8.png
    S4 · Settings Navigation and Error Check
    S4-11.png
    S4 · Settings Navigation and Error Check
    • A dialog covered the page on load, and neither its own controls nor the Escape key closed it, so what follows happened with it open.
    • Navigated to /hermes/settings and verified the Settings page loaded with account management and settings options without errors.
    • Navigated to /hermes/logs and verified server log entries rendered properly without errors.
    • Navigated to http://localhost:8649/#/hermes/chat and verified the primary chat interface loaded.
    • Navigated to http://localhost:8649/#/hermes/settings and confirmed settings configurations rendered properly without errors.
    • Navigated to http://localhost:8649/#/hermes/logs and confirmed system logs rendered correctly with search and filter options.
    • Navigated to http://localhost:8649/#/hermes/usage and confirmed the Usage Statistics page loaded cleanly with range selectors and content without any 502 Bad Gateway errors or blank screens.
  5. S5
    Login Validation - Invalid Credentials

    4 steps, 3 screenshots

    pass
    S5-4.png
    S5 · Login Validation - Invalid Credentials
    S5-7.png
    S5 · Login Validation - Invalid Credentials
    S5-10.png
    S5 · Login Validation - Invalid Credentials
    • A dialog covered the page on load, and neither its own controls nor the Escape key closed it, so what follows happened with it open.
    • Signed out and reached the login page with Username and Password fields.
    • Submitted invalid credentials ('admin' / 'wrongpassword') and verified the error message 'Invalid username or password' is displayed and login is rejected.
    • Attempting to log in with invalid credentials ('admin' and 'wrongpassword') correctly failed and displayed the error message 'Invalid username or password' without granting access.
  6. S6
    Login Validation - Empty Submission

    3 steps, 2 screenshots

    pass
    S6-1.png
    S6 · Login Validation - Empty Submission
    S6-3.png
    S6 · Login Validation - Empty Submission
    • Initial login page loaded with empty Username and Password fields.
    • Clicked Login with empty username and password fields; validation message 'Please enter username and password' appeared and submission was blocked.
    • The login form properly validates empty fields upon clicking Login, displaying the error message 'Please enter username and password' and preventing form submission.
  7. S7
    Model Provider - Cancel Addition

    3 steps, 3 screenshots

    pass
    S7-8.png
    S7 · Model Provider - Cancel Addition
    S7-10.png
    S7 · Model Provider - Cancel Addition
    S7-14.png
    S7 · Model Provider - Cancel Addition
    • Navigated to the Models management page showing the default provider list.
    • Clicked 'Add Provider' and verified the Add Provider modal opened with Cancel and Add buttons.
    • Dismissed the Add Provider dialog via Cancel and Close buttons; verified modal closed without creating any blank or extra provider entry.
    • Navigated to http://localhost:8649/#/hermes/models after authenticating.
    • Clicked 'Add Provider' button to open the provider addition modal dialog.
    • Dismissed the Add Provider modal by clicking 'Cancel' and verified it closed cleanly.
    • Re-opened and closed via the 'close' button to verify dismissing works reliably.
    • Verified that no blank, duplicate, or incomplete provider was added to the models list.
  8. S8
    Theme Customization - Adjust Colors

    3 steps, 3 screenshots

    pass
    S8-2.png
    S8 · Theme Customization - Adjust Colors
    S8-4.png
    S8 · Theme Customization - Adjust Colors
    S8-11.png
    S8 · Theme Customization - Adjust Colors
    • A dialog covered the page on load, and neither its own controls nor the Escape key closed it, so what follows happened with it open.
    • Navigated to the Theme settings page with font size, text color, selection color inputs and live preview.
    • Adjusted font size to 18px, text color to #ff0000, and selection color to #0055ff; inputs and reset buttons updated immediately.
    • Navigated to http://localhost:8649/#/hermes/theme successfully.
    • Adjusted the font size from 14 to 18px in the Theme settings.
    • Adjusted the text color to #ff0000 and selection color to #0055ff, observing immediate UI feedback and corresponding Reset buttons appearing for both.
    • No client-side errors were observed and the theme customization controls functioned as expected.
  9. S9
    Theme Customization - Restore Defaults

    4 steps, 3 screenshots

    pass
    S9-3.png
    S9 · Theme Customization - Restore Defaults
    S9-8.png
    S9 · Theme Customization - Restore Defaults
    S9-10.png
    S9 · Theme Customization - Restore Defaults
    • A dialog covered the page on load, and neither its own controls nor the Escape key closed it, so what follows happened with it open.
    • Navigated to Theme settings page showing current theme settings and Restore defaults button.
    • Adjusted theme settings to non-default values: font size 20px, text color #00ff00, selection color #ff00ff.
    • Clicked Restore defaults; verified text size reverted to 14px, text color to #1a1a1a, and selection color to #333333 immediately.
    • Navigated to http://localhost:8649/#/hermes/theme.
    • Adjusted text size (to 20px), text color (to #00ff00), and selection color (to #ff00ff).
    • Clicked 'Restore defaults' button.
    • Verified the application immediately reverted font size to 14px, text color to #1a1a1a, and selection color to #333333 without breaking UI layout.
  10. S10
    Version Preview - Prepare Code

    3 steps, 3 screenshots

    pass
    S10-3.png
    S10 · Version Preview - Prepare Code
    S10-6.png
    S10 · Version Preview - Prepare Code
    S10-8.png
    S10 · Version Preview - Prepare Code
    • A dialog covered the page on load, and neither its own controls nor the Escape key closed it, so what follows happened with it open.
    • Navigated to Version Preview page, showing Prepare Code button and workspace status details.
    • Clicked Prepare Code button; the button transitioned to disabled and Action Log updated with '[timestamp] prepare requested: main'.
    • Preparation completed successfully: Current Tag updated to 'main', Repository Ready changed to 'Yes', and 'Install Dependencies' became available.
    • Navigated to http://localhost:8649/#/hermes/version-preview and clicked Prepare Code.
    • The Action Log immediately updated with '[timestamp] prepare requested: main' and the Prepare Code button disabled during processing.
    • Upon completion, Repository Ready transitioned to 'Yes', Current Tag changed to 'main', and the Install Dependencies button became enabled.
  11. S11
    Theme Customization - Upload Background

    4 steps, 2 screenshots

    blocked
    S11-2.png
    S11 · Theme Customization - Upload Background
    S11-5.png
    S11 · Theme Customization - Upload Background
    • A dialog covered the page on load, and neither its own controls nor the Escape key closed it, so what follows happened with it open.
    • Navigated to the Theme page showing Appearance settings and Background image section with an Upload image button.
    • Successfully navigated to the Theme page at http://localhost:8649/#/hermes/theme.
    • Observed the Theme settings interface including Appearance controls and the Background image section.
    • Clicked the 'Upload image' button which triggers the native browser file picker dialog, but no file upload tool is available in the test harness to supply a file.

Issues

No finding survived the audit. Nothing to fix from this run.

Critic audit

An adversarial second pass over every finding before it reaches the report.

0
findings reviewed
1
re-verified live
0
withdrawn
    • Scenario S1 reports successfully dismissing the initial dialogs, but S2 and several subsequent scenarios report an unclosable dialog covering the page, suggesting either an intermittent UI defect or a flaky harness interaction.
    • A possible defect in S2 ("Unclosable dialog obstructs page content") was not promoted: the live replay came back inconclusive.

    Report

    QA report: EKKOLearnAI/hermes-studio at ff820dd

    All evaluated authentication, settings, and theme customization flows passed with no confirmed defects.

    Testing covered core application features including authentication flows, credential validation, route protection, session persistence, settings navigation, model provider configuration, and theme customization.

    Ten of the eleven scenarios passed cleanly. One scenario, testing theme background image uploads, was blocked because the automated browser environment cannot interact with native file picker dialogs.

    No application defects were confirmed during the run, indicating stable baseline behavior across the tested functionality.

    Run summary
    MetricCount
    Scenarios executed11
    Passed10
    Failed0
    Blocked1
    Findings raised0
    Issues after the audit0
    Withdrawn by the audit0
    Critical / high / medium / low0 / 0 / 0 / 0

    Target: http://localhost:8649 · Testing level: deep_feature · Stack: node / npm (vite)

    Issues

    No issues survived the audit.

    Scenario results
    ScenarioPriorityResultIssues
    S1 Primary Login Flowhighpassnone
    S2 Route Protection for Unauthenticated Usershighpassnone
    S3 Persistence of Authenticationhighpassnone
    S4 Settings Navigation and Error Checkhighpassnone
    S5 Login Validation - Invalid Credentialsmediumpassnone
    S6 Login Validation - Empty Submissionmediumpassnone
    S7 Model Provider - Cancel Additionmediumpassnone
    S8 Theme Customization - Adjust Colorsmediumpassnone
    S9 Theme Customization - Restore Defaultsmediumpassnone
    S10 Version Preview - Prepare Codemediumpassnone
    S11 Theme Customization - Upload Backgroundlowblocked (The automated browser environment lacks a file upload capability to interact with the native file chooser dialog opened by the Upload image button.)none
    The audit

    The Critic reviewed 0 findings and re-verified 1 of them live in the browser, replaying the reported steps on a fresh page.

    • Scenario S1 reports successfully dismissing the initial dialogs, but S2 and several subsequent scenarios report an unclosable dialog covering the page, suggesting either an intermittent UI defect or a flaky harness interaction.
    • A possible defect in S2 ("Unclosable dialog obstructs page content") was not promoted: the live replay came back inconclusive.
    Coverage and caveats

    In scope: Primary authentication flow and route protection; Settings navigation and stability; Theme customization features; Model provider modal interactions; Version preview code preparation.

    Not covered: External preview ports 8650 and 8651 (noted as inactive in exploration); Deep creation of model providers (save/submit button for modal was not explicitly mapped).

    • The default credentials (admin / 123456) are active and valid for the test environment.
    • The initial login modals ('版本通知' and default credentials warning) can be dismissed using standard browser actions (like Esc or clicking a visible close icon) since specific dismiss buttons were not mapped.
    • The 'Upload image button' triggers a standard file selection dialog that the automated browser can interact with.
    • S11 could not be executed: The automated browser environment lacks a file upload capability to interact with the native file chooser dialog opened by the Upload image button..
    By the numbers
    MetricValue
    Scenarios10 passed, 0 failed, 1 blocked of 11 (41 planned steps)
    Browser actions246 (58 clicks, 27 inputs, 30 navigations, 131 snapshots)
    Screenshots37 (4 explore, 32 scenario, 1 critic), 32 captioned
    Coverage12 pages, 1 forms, 3 flows, 1 console errors
    Audit0 findings, 1 re-verified live, 0 confirmed, 0 promoted, 0 withdrawn
    Model calls193
    Tokens967,066 input, 9,185 output, 14,372 thinking
    Time8 min
    StageCallsInputOutputThinkingSeconds
    explore41241,4482,6961,01297
    plan13,7831,8763,30739
    test141692,0444,0636,717289
    critique928,3754203,16744
    report11,4161301693

    Run log

    stagecallstokenstime
    Explore41245.2k1m 37s
    Plan19k39s
    Test141702.8k4m 49s
    Critique932k44s
    Report11.7k3s
    Total193990.6k7m 52s
    Intake
    Explore
    Plan
    Test
    Critique
    Report
    • 00:59:12Zexploreexplore started
    • 01:07:33ZexploreExplored / (3 controls, 1 forms)
    • 01:07:33ZexploreMapped 12 pages, 1 forms, 3 flows in 40 turns.
    • 01:07:33Zexploreexplore completed in 97s.
    • 01:07:33Zplanplan started
    • 01:07:33ZplanPlanned 11 scenarios (4 high, 6 medium, 1 low).
    • 01:07:33Zplanplan completed in 39s.
    • 01:07:33Ztesttest started
    • 01:07:33ZtestS1 executed (pass)
    • 01:07:33ZtestS2 executed (pass)
    • 01:07:33ZtestS3 executed (pass)
    • 01:07:33ZtestS4 executed (pass)
    • 01:07:33ZtestS5 executed (pass)
    • 01:07:33ZtestS6 executed (pass)
    • 01:07:33ZtestS7 executed (pass)
    • 01:07:33ZtestS8 executed (pass)
    • 01:07:33ZtestS9 executed (pass)
    • 01:07:33ZtestS10 executed (pass)
    • 01:07:33ZtestS11 executed (blocked)
    • 01:07:33ZtestExecuted 11 scenarios: 10 passed, 0 failed, 1 blocked, 0 findings.
    • 01:07:33Ztesttest completed in 289s.
    • 01:07:33Zcritiquecritique started
    • 01:07:33ZcritiqueReviewed 0 findings; 1 possible defect spotted in passed scenarios.
    • 01:07:33ZcritiqueRe-verified a possible defect in S2: inconclusive.
    • 01:07:33ZcritiqueAudit complete: 0 confirmed, 0 withdrawn, 0 promoted, 1 re-verified live.
    • 01:07:33Zcritiquecritique completed in 44s.
    • 01:07:33Zreportreport started
    • 01:07:33ZreportReported 0 issues (0 critical, 0 high, 0 medium, 0 low) from 0 findings.
    • 01:07:33Zreportreport completed in 3s.

    Put an agent team on your next pull request.

    Connect a repo, dispatch a Run, and read an audited, evidence-backed report the same day.