QA report: external/multisig-toolkit.mystenlabs.com at hosted
Core multisig and offline signing workflows are operational, with one minor user experience defect exposing raw exception text.
Testing covered 12 scenarios focusing on offline signing, multisig address generation, signature combination, transaction execution, and signature analysis. All 12 scenarios passed their functional criteria.
One low-severity user experience issue was identified during invalid byte previewing (F1). When malformed base64 input is provided, the application exposes an unformatted raw JavaScript runtime exception in the user interface instead of a sanitized, user-friendly error message.
Coverage had two environment limitations. Sandbox network constraints encountered CORS blocks when attempting live transaction broadcasting to the RPC node, and restricted clipboard permissions prevented verification of automatic URL copying.
Run summary
| Metric | Count |
|---|
| Scenarios executed | 12 |
| Passed | 12 |
| Failed | 0 |
| Blocked | 0 |
| Findings raised | 0 |
| Issues after the audit | 1 |
| Withdrawn by the audit | 0 |
| Critical / high / medium / low | 0 / 0 / 0 / 1 |
Target: https://multisig-toolkit.mystenlabs.com/ · Testing level: deep_feature · Stack: unknown
Issues
Low severity
F1 · Raw JavaScript exception exposed in UI for invalid base64 input
Severity: low · Type: ux · Verdict: promoted · Scenario: S2
Surfaced by the audit of S2, which the Tester passed, and reproduced live: I entered the non-base64 string 'non-base64-string' into the Transaction Bytes input at /offline-signer, then clicked 'Preview Effects'. An 'Error' heading appeared with the raw JavaScript error: \"Failed to execute 'atob' on 'Window': The string to be decoded is not correctly encoded.\"
Expected: The application displays a user-friendly validation message indicating the input must be valid base64.
Actual: The application catches and displays a raw JavaScript exception ('Failed to execute atob...') directly to the user.
Steps to reproduce:
- Navigate to /offline-signer.
- Enter a non-base64 string into the Transaction Bytes input.
- Click 'Preview Effects'.
Evidence: screenshots/critic-M1-1.png
Environment limitations
These failures came from the test environment, not from the application: a credential the sandbox does not hold, a demo nobody may write to, a resource it cannot reach. They are not counted as issues. They record what this run could not exercise.
- S6 could not exercise this: Transaction broadcast fails due to CORS policy on the RPC node. The fetch request is blocked by CORS policy, leading to a network error. The audit recorded the test environment as the cause, so it is not counted as an issue.
- S9 could not exercise this: Clipboard write permission denied when generating reusable URL. The copy action fails with a console error because the test environment denies clipboard write permissions. The audit recorded the test environment as the cause, so it is not counted as an issue.
Scenario results
| Scenario | Priority | Result | Issues |
|---|
| S1 Offline Signer - Connect and Sign | high | pass | none |
| S2 Offline Signer - Preview Invalid Bytes | high | pass | F1 |
| S3 MultiSig Address - Create | high | pass | none |
| S4 Combine Signatures - Aggregate | high | pass | none |
| S5 Signature Analyzer - Decode | high | pass | none |
| S6 Execute Transaction - Broadcast | high | pass | none |
| S7 MultiSig Address - Multiple PubKeys | medium | pass | none |
| S8 MultiSig Address - Empty Public Key | medium | pass | none |
| S9 Combine Signatures - Reusable URL | medium | pass | none |
| S10 Combine Signatures - Required Threshold | medium | pass | none |
| S11 Signature Analyzer - Invalid Base64 | medium | pass | none |
| S12 Execute Transaction - Empty Fields | medium | pass | none |
The audit
The Critic reviewed 0 findings and ran 1 live replay in the browser, each on a fresh page.
- Scenario S1 contradicts itself, first claiming the initial warning modal could not be closed with its controls or Escape, then later stating it was successfully dismissed.
What to fix first
- F1: Sanitize base64 decoding errors and present a user-friendly validation message instead of raw JavaScript exceptions.
Coverage and caveats
In scope: Offline Signer flows including wallet connection and dry-runs; MultiSig Address generation with single and multiple public keys; Signature combination and reusable URL generation; Signature Analyzer decoding capabilities; Transaction execution and broadcasting on testnet; Form validation across all tool pages.
Not covered: Mainnet broadcasting (test wallet only supports testnet, out of scope); Help page rendering (lowest priority, omitted to stay within scenario cap).
- The unclosable 'Warning' dialog mentioned in the notes does not block programmatic interaction with the underlying form elements, as the Explorer successfully recorded them.
- Dummy base64 strings used for transaction and signature bytes will be validated by the application or the RPC, and a graceful error message is considered a success condition for those inputs.
- S10's page network select showed Mainnet at the time it ran. Combine Signatures is an offline tool, it only aggregates signatures the user pastes in and never reaches the RPC, so its result does not depend on the network selection, and the scenario's pass verdict stands regardless.
By the numbers
| Metric | Value |
|---|
| Scenarios | 12 passed, 0 failed, 0 blocked of 12 (52 planned steps) |
| Browser actions | 189 (37 clicks, 44 inputs, 18 navigations, 90 snapshots) |
| Screenshots | 39 (4 explore, 34 scenario, 1 critic), 34 captioned |
| Coverage | 6 pages, 5 forms, 5 flows, 0 console errors |
| Audit | 0 findings, 1 re-verified live, 0 confirmed, 1 promoted, 0 withdrawn |
| Model calls | 191 |
| Tokens | 998,710 input, 11,627 output, 18,568 thinking |
| Time | 8 min |
| Wallet | 0 transactions, 0 signatures, 0 refusals on chain sui:testnet |
| Stage | Calls | Input | Output | Thinking | Seconds |
|---|
| explore | 29 | 142,059 | 3,149 | 1,973 | 84 |
| plan | 1 | 5,201 | 2,294 | 2,838 | 37 |
| test | 153 | 831,734 | 5,150 | 9,994 | 326 |
| critique | 7 | 18,112 | 827 | 3,382 | 42 |
| report | 1 | 1,604 | 207 | 381 | 5 |