DeepQAdeQA · web3 QA network

Any worker. Any engine.
Verified delivery,
settled on-chain.

The web3 network side of DeepQA: app owners post a QA plan, any worker runs it with any engine inside a pinned sandbox, the platform verifies at beta, settlement is on-chain. Building toward the Arc testnet beta.

Join the testnet betatestnet betadeepqa.app/deqa
36.7%

Heavy blind macro F1. Under the paper's judge 30.4%, above the published best of 26.4%, although the confidence interval overlaps it.

95

confirmed defects across three open-source apps in 98 scenarios, every one re-verified live by the Critic

21

passing JobEscrow contract tests, including a stateful conservation invariant

Built on

  • ArcArc
  • Robinhood ChainRobinhood Chain
  • SolanaSolana
  • USDCUSDC
  • USDTUSDT

Arc is Circle's EVM-compatible chain with USDC as the native gas token. DeepQA already tests Arc apps in place: see the Arc Campaign.

USDC and USDT, supported stable settlement assets. On Arc, USDC is the native gas token. Beta runs one valueless testU token.

01 The gap

QA is scarce and expensive, and one engine is one point of view.

01

Apps ship faster than they are tested

A prompt becomes a deployed app the same day. The published best on WebTestBench is 26.4% macro F1.

02

Agent QA works, but it is centralized

DeepQA scores above that best, interval overlapping, on one vendor's servers with one engine.

03

Coverage needs its own reward

Bounties pay for findings. Who pays for agreed coverage when the app passes?

The missing piece: a portable QA plan, sandboxed execution, and evidence another engine can check.

02 Where deQA starts

The DeepQA engine: six stages, one adversarial Critic.

Same pipeline behind the managed service and behind deQA's plan builder and verifier.

scroll sideways for the full diagram

re-verifies every finding live, verdict binding01Intakeguided02ExploreExplorer03PlanStrategist04TestTester05CritiqueCritic06ReportReporterdeterministic gate: schema and coverage validators between stages

The adversarial Critic

Audits every verdict, re-verifies live, promotes misses, never deletes a refuted finding. Binding.

Engine levels

Fast, Audited, Heavy. Only Heavy, with frozen prompts, is ever quoted.

Deterministic gates

Validators between stages. Every run records its exact model configuration.

03 Evidence

Measured engine results. Network validation comes next.

Held-out macro F1, Heavy engine, 10 never-tuned apps

25%
50%
36.7%
51.8%
30.4%
26.4%

Heavy, blind

Heavy, audited

Heavy, paper's judge

Paper's best (GPT-5.1)

WebTestBench, prompts frozen. 30.4% under the paper's judge is above the published 26.4%, although the confidence interval overlaps it.

Real-app campaign, August 2026

3

open-source apps: Restful Booker Platform, Vikunja, InvoiceShelf

98

scenarios run end to end in a real browser

95

confirmed defects after the Critic's live re-verification

1

genuine security vulnerability, disclosed privately

Six criticals on the demo booking app, none on the mature task manager. F1 understates what the engine does on real software.

04 The thesis

Same plan, same judge, different brain.

Blind macro F1 on the same 10 held-out apps

25%
50%
36.7%
23.2%
13.5 points

Heavy engine, mixed strong backbone

Same harness, Sonnet 4.5 in every agent

Prompts byte-identical, same plan format, same judge. Only the backbone changed.

One engine is one point of view

A second engine can expose different defects and refute different false alarms.

Diversity is the thesis, not a measured result

M2's two-engine comparison on one fixture is the first measurement: finding overlap, acceptance agreement, cost, replay failures.

So the network runs many brains on one plan, comparable through one acceptance policy and one engine manifest per report.

05 deQA

One plan, many workers, verified and settled on-chain.

  1. 01

    Portable plan

    deterministic partexplorer partversionedthe job unit

  2. 02

    Any engine, sandboxed

    ClaudeClaudeGeminiGeminiOpenAIOpenAILLocal model
    pinned containersame evidence schema

  3. 03

    Verified delivery

    4 checks3 outcomespay follows coverage

  4. 04

    On-chain settlement

    deposit-first escrowstake and bondsfrozen termsno protocol token

DeepQA managed service at deepqa.app

The managed side. Every job on the shared engine and managed servers, within a budget cap.

deQA network at deepqa.app/deqa

The network side. Same engine for planning and verification, workers execute, own servers and contracts.

The network

One plan, many brains.

Decentralized execution: the platform verifies at beta, independent verifiers join at M3b.

scroll sideways for the full diagram

JobEscrowjob ids, hashes, escrowArc testnet, EVM betaUSDC as the gas tokenSolana at M3asame invariants on devnetSettlement assetsUSDC, USDT, testURequesterFixture app, milestone 1RequesterFixture app, milestone 1RequesterStaging app, milestone 2Claudeworker 01Geminiworker 02OpenAIworker 03LLocal modelworker 04AOwn agentworker 05Claudeworker 06Geminiworker 07LLocal modelworker 08OpenAIworker 09PlatformDeepQA engine verifies at beta,verifiers join at M3b
Contract: post, claim, submit, settlePrivate: claim-scoped details and report uploadThe Requester authorizes, the platform posts. Evidence by hash.

How it works

One job, end to end.

scroll sideways for the full diagram

  1. 01

    Prepare

    Requester + platform

    Deposit, pick a fixture and preset, review plan and quote.

  2. 02

    Authorize + post

    Requester → platform → chain

    Requester signs, platform posts, terms lock.

  3. 03

    Claim + execute

    Worker

    Claim with a bond, fetch the plan, run in the sandbox.

  4. 04

    Upload + anchor

    Worker → platform + chain

    Upload artifacts, sign and anchor the report hash.

  5. 05

    Verify delivery

    Platform at beta

    Binding, coverage, replay sample, every finding re-verified.

  6. 06

    Settle

    Anyone → chain

    After the dispute window: reward, fees, collateral credited.

Pay follows accepted coverage, not defect counts. Retries never pay twice.

Acceptance policy v1

Four checks, three outcomes, pay follows coverage.

  1. 01

    Binding

    signed by the claiming worker

  2. 02

    Coverage

    every required scenario ran

  3. 03

    Replay sample

    a seeded sample re-run by the platform

  4. 04

    Critic re-verification

    every finding re-run live

  • accepted

    all four pass

    reward and bond to the Worker, fees earned

  • non-delivery

    unbound, missing coverage, fabrication

    bond slashed, reward returned, verify fee retained

  • inconclusive

    target drift, outage, contradiction

    everything returned, verify fee retained, nobody slashed

Slashable at beta: missing coverage, fabricated evidence, no submission. Nothing else. A failing app or a zero-finding report is never non-delivery.

06 Architecture

On-chain, off-chain, and who signs what.

Settlement

Deposit-first escrow, frozen terms, no protocol token.

scroll sideways for the full diagram

Fund

Requester deposit

free balance

Worker stake

eligibility, no yield

post
claim

Lock

Job escrow

reward + both fees, terms frozen

Claim bond

locked from stake

settle

Credit

Worker

reward on accepted

Platform

earned verify fee

Treasury

protocol fee on accepted

Requester

refund per outcome

  • ArcArc
  • SolanaSolana
  • tUtestU
  • USDCUSDC
  • USDTUSDT

USDC and USDT, supported stable settlement assets. On Arc, USDC is the native gas token. Beta runs one valueless testU token.

Collateral returns to its owner unless slashed under the policy. Tokens leave only through withdraw and unstake.

What the chain holds

What the chain holds, and what it never sees.

On-chain: JobEscrow

Arc

Arc testnet

  • job id

    one per posted job

  • plan hash

    what the worker agreed to run

  • report hash

    anchored at submit

  • outcome and score

    written at verify

  • escrow, stake, collateral

    per token

  • frozen terms

    every change emits an event

Off-chain: deQA platform

private object store and Postgres index, the job's parties only

  • the plan

    deterministic and explorer parts

  • evidence

    screenshots and state per step

  • the report

    results, findings, engine manifest

  • the assessment

    replay results and critique

  • target credentials

    never on-chain

  • worker record

    deliveries, findings, replay agreement

A hash proves integrity, not truthful execution. Truthfulness comes from replay and the Critic.

Chain adapter

One interface in shared, EVM first, Solana next.

deqa CLI (Rust)
backend api
web viewer

shared: chain adapter interface

depositwithdrawstakeunstakepostclaimsubmitverifysettledisputeresolvetimeoutevents

equivalent economic invariants and outcomes, normalized outputs

EVM: contracts/evm, Foundry

Ethereum, EIP-712
Arc

Arc testnet, primary

Milestone 1, Circle's EVM-compatible chain

Robinhood Chain

Robinhood Chain, secondary EVM target

built on Arbitrum Orbit

JobEscrowEIP-712 authorizationthree outcomespermissionless timeouts

Solana: contracts/solana, Anchor

Solana

Milestone 3a: the same flow on Solana devnet

same invariantssame three outcomesstub today

Two packages with equivalent economic invariants, not identical low-level events.

JobEscrow state machine

Every open state has a deadline anyone can enforce.

scroll sideways for the full diagram

WorkerWorkerPlatformacceptedRequester, no claim yetRequester or Worker, bond lockedAdmin resolvesAnyone, non-delivery or inconclusiveAdmin, non-delivery or inconclusiveclaim deadlinesubmit deadlineverify deadline, no faultresolve deadline, no faultPostedClaimedSubmittedVerifiedSettledCancelledDisputedRefundedRequesterWorkerPlatformAdmin, multisig before mainnetAnyone, after a deadline

Terminal states: Settled, Refunded, Cancelled. Every deadline is fixed when the state is entered.

Identity and keys

Who signs what.

  • Requester wallet

    deposits, withdraws, the EIP-712 post authorization, disputes

  • Worker owner wallet

    holds stake and deposits

  • Worker execution signer

    claims, submits, signs reports, settles

  • Worker credential key

    X25519, decrypts target credentials for a claimed job

  • Platform signer

    calls post and verify, rotatable

  • Admin, Treasury

    resolves disputes, single key on testnet, multisig before mainnet

Ethereum, EIP-712

EIP-712 typed authorization. The Requester signs chain, contract, action, account, nonce, expiry and terms. Verified on-chain at post, nonce consumed.

Beta: the execution signer is a dedicated capped testnet wallet. No payment key enters the runtime container.

Deployment view

Two repos, four places where code runs.

scroll sideways for the full diagram

Worker machine

Linux x86_64 or macOS arm64, Docker

deqa CLI (Rust)

keys, config, signing

runtime container

pinned browser and engine stages, no payment key

engine endpoint

Claude, Gemini, OpenAI, or a local model

Requester machine

deqa CLI, one EVM wallet

submit, approve, status, dispute

deQA server

own deployment, separate from the managed service

api

intake, quote, post, verify, indexer

web

entry page, private job viewer

engine

a pinned DeepQA runner

fixture apps

open-source apps with synthetic data

database

jobs, claims, outcomes

private object store

reports and evidence, parties only

Chain

Arc

public RPC

Arc testnet

JobEscrow, USDC as the gas token, testU faucet

Solana devnet (M3a)

same invariants

Solana

two repos

open: contracts, CLI, schemas. private: web, api, engine

The open half is published before mainnet. Nothing open depends on the private half.

07 Three roles

Requester, Worker, Verifier.

Requester

M1 fixtures, M2 staging apps

App owners who want agreed coverage. One EVM wallet, deposit first, approve the quote.

deqa wallet depositdeqa job submitdeqa job quotedeqa job approvedeqa job statusdeqa job disputedeqa wallet withdraw

How to join

Teams with staging apps and synthetic data to put under test at M2.

Write to contact@matrixlabs.org

Worker

CLI scaffold today, testnet at M1

Anyone running the deqa CLI with any engine in the pinned sandbox. Stake buys claim eligibility only.

deqa identity initdeqa wallet stakedeqa healthdeqa work listdeqa work claimdeqa work rundeqa work uploaddeqa work submitdeqa work settle

How to join

Operators who will run a worker on a second machine with their own engine.

Write to contact@matrixlabs.org

Verifier

M3b

Independent staked nodes that run the acceptance protocol, selected by a rule tested for Sybil, conflict, bribery, and liveness.

Verifier tooling ships with M3b.

How to join

Research partners for M3b: verifier selection, independence, correlated failure.

Write to contact@matrixlabs.org

Trust model, stated plainly

Distributed execution at beta. Independent verifiers at M3b.

Distributed at beta

  • ✓Execution: any worker, any engine, own machine, pinned sandbox
  • ✓Evidence signed by the worker and anchored by hash
  • ✓Escrow, stake, bonds, settlement enforced by the contract
  • ✓Every timeout permissionless, no party can strand funds

Centralized at beta

  • Intake, pre-scan, plan build on the platform engine
  • Acceptance and assessment computed and signed by the platform
  • Disputes: admin key on testnet, multisig before mainnet
  • A compromised platform could approve a false report even when every hash is valid

What changes, and when

  • M3b: independent staked Verifiers run the acceptance protocol
  • Mainnet gate: platform-only adjudication is not acceptable
  • Contracts, CLI, schemas open source before mainnet

Decentralized here means distributed execution with platform-verified delivery, until M3b ships.

08 Workers

Any engine through one CLI, always inside one sandbox.

A Claude Code or Codex worker, step by step.

scroll sideways for the full diagram

  1. 01

    deqa health

    checks keys, chain, engine, sandbox

  2. 02

    deqa commands

    reads the capability contract

  3. 03

    work list

    read-only discovery

  4. 04

    work claim

    explicit claim, then the plan

  5. 05

    work run

    inside the pinned sandbox

  6. 06

    work submit

    signed report, hash anchored

  7. 07

    work resume

    if interrupted

  8. 08

    work settle

    after the dispute window

Same source for both harnesses: the Claude Code skill is the file, the Codex plugin reads it.

deqa CLI, recorded output, abridged
❯ deqa --help
deQA: decentralized QA jobs for any worker, any engine
 
Commands:
  identity  Execution signer and credential key
  wallet    Deposit, withdraw, stake, unstake, balance
  config    chain, token, engine, backend, caps
  job       Requester side: submit, quote, approve, status, dispute
  work      Worker side: list, claim, fetch, run, upload, submit, resume, settle
  health    Test backend, chain, engine endpoint, runtime image, keys
  commands  Print the capability contract for agent plugins

Bring your engine

ClaudeClaudeGeminiGeminiOpenAIOpenAILLocal model

Every report carries a worker-declared engine manifest.

Or bring your agent

Claude Code and Codex plugins read the capability contract.

The sandbox is not optional

Pinned container, no payment key, egress to target and engine only.

09 Roadmap

M1 to M4.

The beta design is closed and the contract is tested. The network is not launched, and no job has settled on a testnet yet.

scroll sideways for the full diagram

  1. M1

    Testnet flow

    nextArctU

    one job settles end to end on Arc testnet

  2. M2

    One outside operator

    planned

    a stranger completes a job with their own engine

  3. M3a

    Solana

    plannedSolana

    the same flow on Solana devnet

  4. M3b

    Independent verifiers

    planned

    verification leaves the platform

  5. M3c

    Economics

    planned

    a published cost model

  6. M4

    Mainnet

    plannedUSDCUSDT

    the first paid job

Built today

A tested contract, a CLI with agent plugins, a proven engine.

21

passing JobEscrow contract tests, recorded from the open contracts repo

JobEscrow contract

Deposits, stake, frozen terms, three outcomes, timeouts for every state.

Rust deqa CLI

Requester and worker commands, health check, machine-readable capability contract.

Agent plugins

Claude Code and Codex plugins from one shared source.

deQA, the web3 QA network

Many brains.
One plan.
Verified delivery.

Looking for testnet workers and requesters, and chain and research partners for the integration gates and M3b.

DeepQA team with ROSS @ MRC, University of Alberta, and UPB under Prof. Nicolae Goga.

Join the testnet betacontact@matrixlabs.orgGemini and OpenAI icons by Icons8

Put an agent team on your next pull request.

Connect a repo, dispatch a Run, and read an audited, evidence-backed report the same day.

Explore live resultsRequest early access