Browser Action

The browser action drives a real Chrome through chromedp : it opens pages, reads and types into them, waits for elements, and takes screenshots. Chrome or Chromium has to be installed where Probe runs.

It is an external action, published in mozership/probe-browser ; it was built into Probe up to v1.21.0. Probe downloads it the first time a workflow uses it, and runs the executable whose SHA-256 the action.yml at the pinned commit names. It needs Probe v1.21.0 or later. The notes of each release  start with the uses line to copy.

The action cannot tell what a page will do, so it keeps to no guard: under --read-only or --allow-host, a step that uses it is refused unless --allow-action names it.

Basic Syntax

A browser step lists what to do under actions, in order. Every step starts a fresh browser, runs its actions, and closes it, so cookies and a login do not carry over to the next step on their own: put a whole flow in one step, or save the login with save_storage_state and start the next step’s browser from it, as described in Carrying a login over.

steps:
  - name: Read the heading
    uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
    with:
      actions:
        - name: navigate
          url: "{{vars.url}}/"
        - name: text
          id: heading
          selector: h1
    test: res.code == 0 && res.results.heading == "Welcome"

Parameters

These go directly under with.

Starting Chrome takes part of timeout, often a second or more, so a short limit can run out before the first page opens.

Actions

Each entry of actions names its action in name and takes these fields:

A wait_text takes at most one of equals, contains and matches, and no other action takes them. An empty contains or matches is refused too, since any text would pass it. A workflow that sets them otherwise is refused before the browser starts.

An action that looks for an element waits until the element appears, and the step fails when timeout runs out first. Most of them also wait for it to be visible.

Navigation and waiting

These open a page or hold the step until the page is in the state the next action needs.

Reading the page

These put a string in res.results, under id or the action’s name.

Interacting

These act on the page the way a user would. They return nothing.

Screenshots

These save an image file and put its path in res.filepaths, under id or the action’s name. The file is named for its format, .png or .jpg.

Storage state

Response Object

When every action succeeds, the step sees:

When an action fails there is no response to test: the step fails with an action error, as described in Error Handling.

Examples

Each example is one step, since a step’s actions share one browser and the next step starts a new one. Carrying a login over is the exception: it has two steps, as it hands the login from one to the next.

Logging in

A login and the check that it worked have to share one browser, so they are one step.

secrets:
  - PASSWORD
vars:
  url: "{{APP_URL}}"
  user: "{{USERNAME}}"
  password: "{{PASSWORD}}"
 
jobs:
- name: Sign in
  steps:
  - name: Log in and reach the dashboard
    uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
    with:
      timeout: 30s
      actions:
        - name: navigate
          url: "{{vars.url}}/login"
        - name: type
          selector: "#username"
          value: "{{vars.user}}"
        - name: type
          selector: "#password"
          value: "{{vars.password}}"
        - name: click
          selector: "button[type='submit']"
        - name: wait_visible
          selector: "#dashboard"
        - name: text
          id: greeting
          selector: "#dashboard h1"
    test: res.code == 0 && res.results.greeting contains vars.user

Reading several values

Give each reading action an id so the results do not replace each other.

  - name: Product page
    uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
    with:
      actions:
        - name: navigate
          url: "{{vars.url}}/products/42"
        - name: text
          id: title
          selector: "h1"
        - name: text
          id: price
          selector: ".price"
        - name: get_attribute
          id: image
          selector: "img.product"
          attribute: [src]
    test: |
      res.code == 0 &&
      res.results.title != "" &&
      res.results.price startsWith "$"
    outputs:
      image_url: res.results.image

Waiting for a text

A page that fetches its data shows it a moment after it loads, so a text read as soon as the element is visible can get Loading and fail the test. A wait_text with equals, contains or matches reads the text again until it is the one expected.

  - name: Order list
    uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
    with:
      timeout: 30s
      actions:
        - name: navigate
          url: "{{vars.url}}/orders"
        - name: wait_text
          id: status
          selector: "#status"
          matches: '^\d+ orders$'
    test: res.code == 0
    outputs:
      status: res.results.status

When the step’s timeout runs out first, the step fails with an action error that gives the last text read, and the page is saved, as described in Page at failure:

wait_text "#status": the text was "Loading", want it to match "^\\d+ orders$": context deadline exceeded (page at failure: ...)

Carrying a login over

A step that signs in can save the login with save_storage_state, and the steps after it start their browsers signed in by naming the file in storage_state, rather than each signing in again.

  - name: Sign in
    id: login
    uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
    with:
      timeout: 30s
      actions:
        - name: navigate
          url: "{{vars.url}}/login"
        - name: type
          selector: "#username"
          value: "{{vars.user}}"
        - name: type
          selector: "#password"
          value: "{{vars.password}}"
        - name: click
          selector: "button[type='submit']"
        - name: wait_visible
          selector: "#dashboard"
        - name: save_storage_state
          id: state
    test: res.code == 0
    outputs:
      state: res.filepaths.state
 
  - name: Orders
    uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
    with:
      storage_state: "{{outputs.login.state}}"
      actions:
        - name: navigate
          url: "{{vars.url}}/orders"
        - name: wait_text
          selector: "h1"
          equals: "Orders"
    test: res.code == 0

The file holds the cookies of every site the browser has, HttpOnly ones included, and the localStorage of the origin of the page open when it is saved; to keep the localStorage of another origin, save the state while a page of it is open. It has the shape of Playwright’s storage state , so that either tool reads what the other writes.

Before the first action, a browser given storage_state takes its cookies, and opens a page of each origin with items in it to write the localStorage there. Those pages are answered with an empty one inside the browser, so they reach no server and run none of the app’s scripts, and the browser is back on about:blank when the first action runs. A file that is missing or is not a storage state fails the step before the browser starts, as when the step that writes it failed.

The cookies sign in as the user, so the file is readable only by the user who ran Probe. Without path it goes to the temporary directory, which keeps it out of the repository; with path, keep it out of version control, and out of the files a CI run uploads.

Checking the console

A page can break behind what it shows, such as a click handler that throws. res.console_errors lists what the page wrote to its console as an error while the actions ran: each call of console.error, each console.assert that failed, and each exception or promise rejection nothing caught. Each has these fields:

The browser’s own messages, such as a resource that failed to load, are not among them, nor are logs and warnings. The errors are read from the page’s tab, so those of a popup, or of an iframe from another site, are not either.

To fail the step on any error, test that the list is empty:

  - name: Checkout page
    uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
    with:
      actions:
        - name: navigate
          url: "{{vars.url}}/checkout"
        - name: click
          selector: "#pay"
        - name: wait_visible
          selector: "#receipt"
    test: res.code == 0 && len(res.console_errors) == 0
    echo: "{{res.console_errors}}"

When an action fails, the failure message names the first few of them, as one is often why it failed:

... context deadline exceeded (console errors: "Uncaught TypeError: Cannot read properties of null (reading 'total') (http://localhost:8090/checkout.js:42:17)") (page at failure: ...)

Keeping a screenshot

The path of the saved image is in res.filepaths, ready to print or pass on.

  - name: Checkout page
    uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
    with:
      window_w: 1280
      window_h: 800
      actions:
        - name: navigate
          url: "{{vars.url}}/checkout"
        - name: full_screenshot
          id: page
    test: res.code == 0
    echo: "Saved {{res.filepaths.page}}"

Error Handling

An action that cannot finish makes the whole step fail with an action error, which exits the run with status 3. A selector that matches nothing is not an empty result: wait_visible, text, click and the other actions that look for an element keep waiting until the step’s timeout. A navigation fails as soon as the browser reports a network error, such as a refused connection, while an address that never answers also waits until timeout. In every case there is no res to test, and the failure message carries the browser’s error, along with the errors the page wrote to its console, as Checking the console shows.

Page at failure

When the actions fail, Probe saves the page as it was at that moment: a full-page PNG screenshot and the HTML of the document, under one shared name such as probe-browser-failure-1947177907.png and .html. The failure message names both files along with the page’s URL:

... context deadline exceeded (page at failure: url http://localhost:8090/, screenshot /tmp/probe-browser-failure-1947177907.png, html /tmp/probe-browser-failure-1947177907.html)

The same message is the failure’s message in the files --report writes. A test that evaluates to false is not an action failure, so the page is not saved then. To have it saved when a text is not the one expected, check the text with a wait_text and equals, contains or matches rather than in test, or add a capture_screenshot action to keep the page in any case.

The files go to the system’s temporary directory unless evidence_dir names another. The directory is created when needed, which suits uploading it as a CI artifact. The page can show private data, so the files are readable only by the user who ran Probe:

- name: Checkout page
  uses: github.com/mozership/probe-browser@634baa8dff2b273ed326996fec4d91c2dba9d38b # v0.2.0
  with:
    evidence_dir: out/browser
    actions:
    - name: navigate
      url: "{{vars.url}}/checkout"
    - name: wait_visible
      selector: "#pay"

The browser is kept for up to 10 seconds after the timeout so that the page can still be read. If it cannot be, for example because the browser never started or the page was still loading when time ran out, the message says so and nothing is saved.

Running in CI

A CI runner needs a browser and a little more time than a laptop.

• Install Chrome or Chromium on the runner.
• Keep headless: true, or provide a display such as Xvfb when a window is needed.
• Chrome is started with --no-sandbox, as containers usually require, so point it only at sites you trust.
• Give timeout room for Chrome to start, which is slower on a cold runner.