For the complete documentation index, see llms.txt. This page is also available as Markdown.

Manual Crawler

Record Trajectories Manually

Manual Crawler allows you to manually navigate through a website and create trajectories (sequences of actions) that can be tested and replayed. This is useful for creating specific test scenarios or exploring complex user flows.

Browser Session Selection (Optional)

Select an existing browser session to use for authenticated manual crawling

How to use:

  • Click the dropdown to see available browser sessions.

  • Select a session if you want to crawl authenticated areas.

  • Leave empty for unauthenticated crawling.

Using Manual Crawler

  1. Start the crawler:

    • Click Start Manual Crawler.

    • A proxied browser window opens on the SANDBOX agent. This may take a few seconds.

    • If the browser window does not open automatically, click Open proxied browser from the active crawler card.

  2. Understanding the Toolbar:

The browser window includes a toolbar with these controls:

  • Status Indicator (Green/Yellow Dot):

    • Green: System is waiting for your action

    • Yellow: System is processing your last action

    • Red: Error occurred

  • Status Text:

    • Shows current state: "Waiting for action", "Processing...", "Loading page..."

  • Start Recording:

    • Starts recording a new trajectory from the current page.

    • Use this when you want to capture a menu, form, or multi-step flow that automated discovery missed.

  • Reset:

    • Stops the current trajectory recording and resets back to the starting point.

  • Done:

    • Saves the trajectories you created and ends the manual crawler session.

  • Abort:

    • Stops the manual crawler without saving the current in-progress trajectory.

Last updated