AI in testing automation is two loops running at once, not one clever model

AI in testing automation at the Assrt code level is two concurrent loops sharing one browser. Loop A is the foreground agent running your #Case scenarios using tools defined at agent.ts lines 16 through 196: navigate, snapshot, click, type_text, evaluate, http_request, wait_for_stable, and so on. Loop B is the discovery worker, implemented in queueDiscoverPage at agent.ts lines 555 through 562 and generateDiscoveryCases at lines 585 through 618. Every time Loop A navigates to a new URL, Loop B pushes that URL onto a pending queue, opens up to MAX_CONCURRENT_DISCOVERIES = 3 parallel LLM calls with a dedicated DISCOVERY_SYSTEM_PROMPT, and streams 1 to 2 new #Case drafts per page as discovered_cases_chunk server-sent events. The whole process caps at MAX_DISCOVERED_PAGES = 20 per run. Category pages on mabl, testRigor, Functionize, and TestGrid describe generic categories like self-healing tests, visual validation, and autonomous generation. None of them show the two-loop architecture, the bounded concurrency, or the separate system prompt that makes this safe to run while your main test is still in flight.

Because the two loops need different output shapes. The main SYSTEM_PROMPT at agent.ts lines 198 through 254 pushes the driver to call tools, assert, and close out scenarios. If you handed that prompt to a freshly spotted URL with no context, the model would immediately try to click things on a page it is not yet attached to. The DISCOVERY_SYSTEM_PROMPT at agent.ts lines 256 through 267 is the opposite shape: it forbids login and signup cases, forbids CSS and responsive and performance cases, caps output at 1 to 2 cases, caps each case at 3 to 4 actions, and demands references to actual visible buttons and inputs. The worker reads the accessibility tree (truncated to 4000 characters at agent.ts line 586), produces clean #Case text, and never touches the live browser. Separating the prompts is the reason the two loops don't step on each other.

Three hard limits, all visible in agent.ts. First, MAX_CONCURRENT_DISCOVERIES = 3 at line 269 caps how many discovery LLM calls run at once; the fourth is queued. Second, MAX_DISCOVERED_PAGES = 20 at line 270 caps the total number of pages ever discovered in a single run; anything past that gets dropped on the floor by queueDiscoverPage. Third, flushDiscovery at lines 564 through 583 checks browserBusy before grabbing the snapshot, so the worker never fights the main agent for the single browser tab. The main loop sets browserBusy = true before every tool call and false after (lines 754 and 1042), which makes flushDiscovery yield while any foreground action is in flight. That is why you can run this on a long multi-step scenario without the discovery queue either saturating your API limits or interleaving mid-click.

They stream back to the client as discovered_cases_chunk events with incremental text, then a single discovered_cases_complete event when the generation finishes. The driver UI receives these over the same SSE channel as the main test output. Independently, each discovered page also fires a page_discovered event that carries a URL, a (mostly empty) title, and a base64 JPEG screenshot, all at agent.ts line 574. In the web app, the side panel shows a list of pages the agent has seen, each with a preview image and a streaming block of generated #Case text. You can save those drafts, edit them, re-run them as fresh scenarios. The test you actually asked for is never blocked on any of this: the discoveries run as background tasks with activeDiscoveries tracked separately from browserBusy.

Not in the current default, but the architecture supports it with a trivial change because the tool schemas are translated once at agent.ts lines 277 through 301. Both providers go through the same TOOLS list, and GEMINI_FUNCTION_DECLARATIONS auto-derives Gemini function declarations from the Anthropic input_schema. The current constructor stores a single provider (lines 354 through 367), so both loops use the same one, but splitting them is a 10-line change: instantiate a second Anthropic or GoogleGenAI client solely for the DISCOVERY_SYSTEM_PROMPT path, and route generateDiscoveryCases through it. The interesting implication is that the tool schema is not the coupling point. You can run a slow deep model on discovery (generates better test ideas) and a fast cheap model on driving (cheaper per click), because the contract between them is the accessibility tree and nothing else.

GPT writing a plan from a prompt is guessing. The Assrt discovery worker is grounded in the actual accessibility tree of the actual page, captured live from the actual browser that the main agent is driving. At agent.ts line 571 the worker calls this.browser.snapshot() to pull the Playwright MCP accessibility tree, then feeds that text plus a base64 JPEG into the LLM alongside the DISCOVERY_SYSTEM_PROMPT. The cases it emits reference elements that are actually on the page because they came from a real DOM read, not from your description of the page. The DISCOVERY_SYSTEM_PROMPT explicitly requires 'Reference ACTUAL buttons/links/inputs visible on the page' (line 266). This is the difference between a generated test plan that reads like marketing and a generated test case that a downstream agent can actually execute in the same browser session.

claude-haiku-4-5-20251001, set at agent.ts line 9. Haiku 4.5 is the right pick for a tool-use loop that is heavy on fast decisions and light on long-form generation. The driver's max_tokens is 4096 per call (line 715), which is plenty for a tool-use response and nowhere near the reasoning budget of a coding model. The discovery worker uses max_tokens = 1024 (line 608) because its output is always a short block of #Case Markdown. With both loops on Haiku 4.5, a typical 10-step scenario that discovers 5 new pages runs under two dollars of LLM spend at list price. The same scenario on a closed AI QA platform at $7,500 a month of seat licensing is the comparison the SERP does not make.

Yes. The whole agent is a single TypeScript file, agent.ts in the assrt-mcp repo, 1,087 lines long. The discovery worker specifically lives between lines 256 and 618. Turning off discovery entirely is a one-liner: either set MAX_DISCOVERED_PAGES = 0, or comment out the queueDiscoverPage call at line 457 (inside run()) and line 775 (inside the navigate tool handler). Tightening it is just as easy: lower MAX_CONCURRENT_DISCOVERIES to 1 to serialize, or widen the SKIP_URL_PATTERNS regex list at line 271 to ignore more paths. The point of shipping this as open source instead of a SaaS is that you can read every knob, move every bound, and keep the tests that come out of it as your own Markdown files at /tmp/assrt/scenario.md.