S3 Action

The S3 action reads and writes objects in S3 and in storage that speaks its protocol, such as MinIO, Cloudflare R2 or Versity Gateway, so a workflow can check what an API stored.

It is an external action, published in mozership/probe-s3 . Probe downloads it the first time a workflow uses it, and runs the executable whose SHA-256 the action.yml at the pinned commit names. It needs Probe v1.21.0 or later.

Basic Syntax

A workflow pins the action by a full commit SHA. The notes of each release  start with the action and its commit to copy. The examples here give it a name once, under actions, which needs Probe v1.24.0 or later; with an earlier one, write the action in full in each uses.

name: Upload
actions:
  s3: github.com/mozership/probe-s3@8d2a120ffe6f2860229f5b6171a3477ed0f07b6d # v0.1.0
jobs:
  - name: upload
    steps:
      - name: Upload an avatar through the API
        id: upload
        uses: http
        with:
          url: https://api.example.com
          post: /avatars
          multipart:
            image:
              file: ./fixtures/logo.png
        test: res.code == 201
        outputs:
          key: res.body.key
 
      - name: The object is in the bucket, as it was sent
        uses: s3
        with:
          bucket: example-avatars
          region: ap-northeast-1
          head: "{{outputs.upload.key}}"
        test: res.code == 200 && res.content_type == "image/png" && res.metadata.owner == "42"

A step does one operation on one bucket. It cannot make or remove a bucket.

Operations

A step names its operation by one of these keys, and has exactly one of them.

Parameters

A key of with that is not one of these fails the step before anything is sent. Its action.yml declares them as params, so probe check reports such a key with its line. A parameter given to an operation that does not take it, such as body with get, fails the step the same way.

path_style defaults to true with an endpoint, as MinIO and the like are served under one name, and to false on AWS, except for a bucket with a dot in its name, which the certificate of *.s3.<region>.amazonaws.com does not cover.

A list returns one page: at most max_keys objects, with res.truncated telling whether the storage has more. Multipart uploads are not supported.

Credentials

When with has none of access_key_id, secret_access_key and session_token, the action reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN from the environment Probe runs in. Without credentials in either place, the request is sent unsigned, as a public bucket takes it. The action reads no profile, no instance metadata and no other source of credentials, so it connects to nothing but the storage.

Leave the credentials to the environment where you can. A secret access key written in with is never in the result or a report, and from v1.24.0 Probe hides the values of secret_access_key and session_token in --verbose output and the logs of actions too. With an earlier Probe, list the variable under secrets when you pass one through with.

Response Object

A get and a head add what the headers tell of the object.

A get adds the content.

res.sha256 and res.size are of the whole object however large it is, so a binary or large object is checked by them, against the req.sha256 of the step that wrote it or a digest you know.

A put adds res.etag. A list adds these.

Once the storage answers, what it answered is a result, so a test can assert on a 404 for a key that should be gone or a 403 for a bucket that should be closed. Only a request that gets no answer, such as a refused connection or a timeout, fails the step as an error.

Examples

Against AWS, with the credentials of the environment:

steps:
  - name: The export is there and not empty
    uses: s3
    with:
      bucket: example-exports
      region: ap-northeast-1
      list: "daily/{{vars.today}}/"
    test: "res.code == 200 && res.count > 0 && all(res.objects, #.size > 0)"

Against MinIO, writing a fixture and reading it back. The job’s defaults, keyed by the name the workflow gave the action, keep the endpoint, the bucket and the credentials out of each step:

jobs:
  - name: fixtures
    defaults:
      s3:
        endpoint: http://localhost:9000
        bucket: fixtures
        access_key_id: "{{vars.minio_user}}"
        secret_access_key: "{{vars.minio_password}}"
    steps:
      - name: Put a fixture
        id: put
        uses: s3
        with:
          put: config/app.json
          body:
            enabled: true
        test: res.code == 200
        outputs:
          sha256: req.sha256
 
      - name: Get it
        uses: s3
        with:
          get: config/app.json
        test: res.body.enabled == true && res.sha256 == outputs.put.sha256
 
      - name: A key that must not be there
        uses: s3
        with:
          head: config/removed.json
        test: res.code == 404

Redirects

On AWS, a bucket that is not in the region the request was signed for is read from the region AWS names in its answer. A Location the storage answers with is followed too, up to 5 times, when it stays within the storage: under amazonaws.com, or the host of endpoint and the names under it. Each request is signed again for where it goes. A redirect to anywhere else is not followed, and is the result.

Under a Guard

The action keeps to the guard of the run, and its action.yml declares guard: [read-only, allow-host], so Probe runs it under either without --allow-action. A step it refuses fails with the kind refused.

• Under --read-only, only get, head and list are run. A put or a delete is refused before anything is sent, and before file is read.
• Under --allow-host, every host a request goes to must be one the run allows: the host the bucket is addressed at, which on AWS is <bucket>.s3.<region>.amazonaws.com, or s3.<region>.amazonaws.com with path_style, and the host of each redirect. A URL without a port is taken at the port of its scheme: 80 for http and 443 for https. --allow-host '*.amazonaws.com' allows a bucket in whichever region it is.

See Guard.

See Also

• External Actions - How Probe resolves and checks an external action
• HTTP - Requests over plain HTTP
• Redis - Commands on a Redis or Valkey server