Testing a Seige app with Playwright and CI

What a test runs against

Seige has no test mode, mock client or sandbox issuer yet. A test uses the same SDK and the same sign-in as your users: the app opens the Seige window, the test signs in and approves there, and the data is encrypted and stored in a real Seige account.

Point the app at the Seige environment under test with the issuer option, and read it from your build configuration so tests and production use the same code:

const seige = createSeige({
  appName: "Habits",
  // For example http://localhost:8795/oauth for a local Seige environment.
  issuer: import.meta.env.VITE_SEIGE_ISSUER || undefined,
});

Without issuer, the SDK uses https://account.seige.ai/oauth. Each website origin is its own app, so a test server on http://localhost:5173 and one on another port are different apps with different user identifiers. Use a fixed port for tests.

Test accounts

On seige.ai, account creation is open to invited usernames. Create a dedicated test account by hand, keep its username and password in your CI secrets, and use it only for tests. Do not use a personal account: tests write to it.

On a Seige environment whose sign-up page is open, a test run can create a throwaway account first. The sign-up page is /create-account on the account origin:

// global-setup.js
export async function createAccount(browser, accountOrigin, username, password) {
  const page = await browser.newPage();
  await page.goto(`${accountOrigin}/create-account`);
  await page.fill("#username", username);
  await page.fill("#password", password);
  await page.fill("#confirm-password", password);
  await page.getByRole("button", { name: "Create account" }).click();
  await page.waitForURL(/\/app/);
  await page.close();
}

Usernames are 3 to 64 characters of a–z, 0–9 and hyphens, and passwords need at least 16 characters. Seige cannot reset passwords, so a lost test password means a new test account.

An account stays signed in to Seige on up to 20 browsers at a time, and every fresh browser profile that signs in counts as one. Signing in on another browser signs out the browser that signed in longest ago, so a long test run that uses a fresh profile for each test can sign out a profile an earlier test kept open. Keep the profiles of one run few, or reuse a saved profile with Playwright’s storageState. Settings, Sign-in & security lists the signed-in browsers.

Drive the Seige window

The app's first call that needs the user, or seige.auth.signIn(), opens the Seige window as a popup. Trigger it from a click in the page, as a user would; a call made without a click shows the “Continue with Seige” card first. In Playwright, wait for the popup with context.waitForEvent("page").

The window shows the sign-in form when this browser context has no Seige session, then the approval screen. After a sign-in in the same context, later windows go straight to the approval screen. These are the current labels and identifiers:

Screen	Control	Selector
Sign-in	Username field	`#username`
Sign-in	Password field	`#password`
Sign-in	Submit button	Button named “Sign in” (exact)
Approval	Heading “Connect to <app name>?”	`#consent-title`
Approval	Decline	Button named “Cancel”
Approval	Approve	One button, whose name depends on the account’s state: “Create folder and connect” (first approval, new folder), “Allow on this browser” (the app already uses a folder on another browser), “Allow connection”, or “Trust application and allow access” (a folder of the same name already holds data)

These labels are not a versioned interface and can change between Seige releases. Keep them in one helper so a change is a one-line fix. Match the approve button by all of its names, because the same test account sees a different one on its first and later runs.

After approval, the window closes itself and the app's call continues. Selecting Cancel makes the call return ACCESS_DENIED; closing the window returns SIGN_IN_CANCELLED. Both are worth a test.

// seige-window.js
import { expect } from "@playwright/test";

const APPROVE = /^(Create folder and connect|Allow on this browser|Allow connection|Trust application and allow access)$/;

export async function throughSeigeWindow(page, click, { username, password, action = "approve" }) {
  // Fail fast when the click opens no window, for example when the app
  // returned an error first.
  const opened = page.context().waitForEvent("page", { timeout: 10_000 });
  await click();
  const popup = await opened;
  const signIn = popup.locator("#username");
  const consent = popup.locator("#consent-title");
  await expect(signIn.or(consent)).toBeVisible();
  if (await signIn.isVisible()) {
    await signIn.fill(username);
    await popup.fill("#password", password);
    await popup.getByRole("button", { name: "Sign in", exact: true }).click();
  }
  await expect(consent).toBeVisible();
  if (action === "close") return popup.close();
  const name = action === "cancel" ? "Cancel" : APPROVE;
  await popup.getByRole("button", { name }).click();
  await popup.waitForEvent("close");
}

Wait for writes before checking them

A seige.from() write resolves after Seige has the change when Seige is reachable, and a seige.files.put() resolves once the file is stored in this browser. seige.sync.status() reports synced before the first call, so it is not a wait condition. Before a test reloads, opens another profile or ends, have the app await its write and then seige.sync.flush(), which resolves when every change saved in this browser has reached Seige.

Expose that state to the test. A simple way is an attribute the app sets while any save is in progress. Count the saves, so a save that finishes while another is still running does not clear the flag:

let saving = 0;

async function save(row) {
  saving++;
  document.body.dataset.saving = "true";
  try {
    const { error } = await seige.from("habits").insert(row);
    if (error) return showError(error);
    const flushed = await seige.sync.flush();
    if (flushed.error) showError(flushed.error);
  } finally {
    if (--saving === 0) delete document.body.dataset.saving;
  }
}

The test then waits with await expect(page.locator("body")).not.toHaveAttribute("data-saving", "true").

After seige.close(), every call on that client returns OPERATION_FAILED with message client_closed; create a new client to continue. At the end of a test, call seige.close() in the page, for example through a test-only window.seige, then close the page before the browser context. A browser hands the database's writer lease back after each published write and when its page closes. A context closed while a write was still in progress can leave the lease held for up to 30 seconds, and the next test's first write waits for it.

Prove the data is in Seige

A local read can come from this browser's encrypted copy. To show that a write reached Seige, read it back from a second, empty browser context signed in to the same account:

import { test, expect } from "@playwright/test";
import { throughSeigeWindow } from "./seige-window.js";

const account = { username: process.env.SEIGE_TEST_USER, password: process.env.SEIGE_TEST_PASSWORD };

test("a habit saved in one browser is read back in another", async ({ browser }) => {
  const name = `Stretch ${Date.now()}`;

  const first = await browser.newContext();
  const a = await first.newPage();
  await a.goto("/");
  await throughSeigeWindow(a, () => a.click("#sign-in"), account);
  await a.fill("#habit-name", name);
  await a.click("#add");
  await expect(a.locator("body")).not.toHaveAttribute("data-saving", "true");
  await a.evaluate(() => window.seige.close());
  await a.close();
  await first.close();

  const second = await browser.newContext();
  const b = await second.newPage();
  await b.goto("/");
  await throughSeigeWindow(b, () => b.click("#sign-in"), account);
  await expect(b.getByText(name)).toBeVisible();
  await second.close();
});

Each new browser context is a new installation of the app, so the approval screen appears again and shows “Allow on this browser”. Each installation is listed under Connected apps in the test account; disconnect old ones there from time to time, since a folder accepts a limited number of connected browsers.

Run the tests in CI

A reachable Seige environment, set as the app's issuer, and a test account on it, from CI secrets or created by the run where sign-up is open.
A browser that Playwright can drive, with pop-ups allowed, which is the default in Playwright. Seige's own browser tests of the SDK run in Chromium.
The app served on a fixed origin, for example with Playwright's webServer option, over http://localhost or HTTPS.
One test worker per test account. Tests that share an account share its database, and parallel writers wait for each other's writer lease.
Some headroom for rate limits. Seige limits each IP address to 120 sign-in requests and 6,000 data requests per minute; over the limit, calls return RATE_LIMITED. Sign in once per test file where you can, rather than once per test.
Logging of error.code, error.message and error.hint from the app to the console, so failures in CI say what went wrong. The browser console also shows one failed 400 request at each sign-in, for the DPoP nonce; it is expected.

AI calls count against the test account's hourly AI limits (20 requests per app and 100 per account) and its AI allowance. Keep AI tests few, or run them separately.

Testing your app