healix

CI Python versions License

A pure-Python library that crawls a website, recognizes and classifies every page it finds, extracts every element with maximum raw detail into JSON, and generates self-healing Selenium/Playwright automation scripts. It is built to plug into external orchestration platforms through an SDK, a CLI, and webhooks.

Status: 2.0. The driver abstraction (Playwright and Selenium), iframe/shadow-DOM traversal, stable-ID normalization, page discovery with its manifest, rule-based page classification, extraction to per-page JSON, and the SDK, CLI, and event/webhook surface are implemented and tested, and so are automatic login (username/password, SSO, session expiry), self-healing (weighted, threshold-gated, persisted, audited) and the optional SAP UI5 and Salesforce platform adapters. Script generation (ScriptGenerator, healix generate) and healix doctor are built too. See Known limitations for what it does not do, and CHANGELOG.md for the release notes.

Features

Available now:

Install

pip install "healix[all]"        # or healix[playwright], healix[selenium]
playwright install chromium      # for the Playwright backend
healix doctor                    # check that this machine is ready

From a checkout, for development:

git clone https://github.com/nikhilvdev/healix.git
cd healix
python -m venv .venv
source .venv/bin/activate
pip install -e ".[playwright]"
playwright install chromium

Requires Python 3.10+. This also installs the healix command. Runtime dependencies are logquill (logging), python-dotenv (the CLI loads .env) and jinja2 (script generation). The browser libraries are optional extras: healix[playwright] (the default backend; then playwright install chromium) and healix[selenium] (needs Chrome installed; Selenium finds the matching driver itself). healix[all] installs both and healix[postgres], the psycopg driver only needed to keep fingerprints in PostgreSQL.

Quickstart

Crawl a site with the SDK

Write a run config (safe to commit — see Run configuration):

{
  "base_url": "https://quotes.toscrape.com/",
  "crawl": {
    "discovery": { "max_pages": 4 },
    "extraction": { "output_path": "./output/" }
  }
}

from healix import Crawler

run = Crawler("run_config.json", on_event=print).discover_and_extract()
print(run.pages_extracted, "of", run.pages_discovered, "pages ->", run.manifest_path)

on_event is called with one dict per event as the crawl progresses (see Events). run.pages are the manifest entries; the JSON for each is under output/pages/.

Crawl a site with the CLI

healix crawl --config run_config.json --webhook-url https://hooks.example.com/healix

run bebd30b63d35: 4 of 4 pages extracted (discovery max_pages_reached)
manifest: output/manifest.json

The webhook received the same events on_event would have — here, four page_discovered, four page_extracted, and one run_complete:

{"event": "run_complete", "run_id": "bebd30b63d35", "timestamp": "2026-09-19T18:00:14.862Z", "data": {"pages_discovered": 4, "pages_extracted": 4, "platform_detected": null, "manifest_path": "output/manifest.json"}}

Extract every element on a page

from healix.driver.playwright_adapter import PlaywrightDriverAdapter

with PlaywrightDriverAdapter() as driver:
    driver.navigate("https://example.com")
    for el in driver.get_elements():
        print(el.tag, el.css_selector, el.text_content)

html html None
body html > body None
div html > body > div None
h1 html > body > div > h1 Example Domain
p html > body > div > p:nth-of-type(1) This domain is for use in documentation examples ...
p html > body > div > p:nth-of-type(2) None
a html > body > div > p:nth-of-type(2) > a Learn more

get_elements() returns Element objects — see Element schema. Pass your own Page (PlaywrightDriverAdapter(page)) to embed Healix in a Playwright session you already manage; the adapter then leaves the browser lifecycle to you.

Find, click, and write

import json

from healix.driver.playwright_adapter import PlaywrightDriverAdapter
from healix.healing.fingerprint import Fingerprint

with PlaywrightDriverAdapter() as driver:
    driver.navigate("https://example.com")

    link = driver.find(
        Fingerprint(
            page_url="https://example.com", element_role="link", tag="a", text_content="Learn more"
        )
    )
    print(json.dumps(link.to_dict(), indent=2))

    driver.click(link)  # also accepts a Fingerprint directly
    print(driver.current_url)  # https://www.iana.org/help/example-domains
    # driver.write("hello", into=field)

find raises ElementNotFoundError when no locator resolves to exactly one element. It does not guess; repairing a fingerprint that has drifted is the job of the healer.

Discover every page on a site

from healix.discovery.crawler import DiscoveryConfig, DiscoveryCrawler
from healix.driver.playwright_adapter import PlaywrightDriverAdapter

with PlaywrightDriverAdapter() as driver:
    crawler = DiscoveryCrawler(driver, DiscoveryConfig(max_pages=50))
    manifest = crawler.discover(
        ["https://example.com/"], run_id="demo", manifest_path="output/manifest.json"
    )

print(manifest.pages_discovered, manifest.discovery_status)
for page in manifest.pages:
    print(page.status, page.page_type, page.url)

1 complete
pending unknown https://example.com/

(example.com links only to another domain, so there is nothing else in scope.)

Classify a page

from healix.classification import classify, classify_page
from healix.driver.playwright_adapter import PlaywrightDriverAdapter

with PlaywrightDriverAdapter() as driver:
    driver.navigate("https://github.com/login")
    elements = driver.get_elements()

    print(classify(elements, driver.current_url))  # login

    result = classify_page(elements, driver.current_url)
    print(result.page_type, result.confidence)  # login 1.0
    print(
        result.signals["login"]
    )  # ['password_input', 'submit_control', 'few_fields', 'login_cue']

DiscoveryCrawler runs this on every page it visits, so each manifest entry already has a page_type.

Extract every discovered page to JSON

from healix.discovery.crawler import DiscoveryConfig, DiscoveryCrawler
from healix.driver.playwright_adapter import PlaywrightDriverAdapter
from healix.extraction import ElementExtractor, ExtractionConfig

with PlaywrightDriverAdapter() as driver:
    # Discovery: find the pages (writes output/manifest.json)
    DiscoveryCrawler(driver, DiscoveryConfig(max_pages=5)).discover(
        ["https://quotes.toscrape.com/"], run_id="demo", manifest_path="output/manifest.json"
    )

    # Extraction: extract each one, one at a time, into output/pages/
    manifest = ElementExtractor(driver, ExtractionConfig(output_path="output")).extract(
        "output/manifest.json"
    )

print(manifest.pages_extracted, "of", manifest.pages_discovered, "extracted")
for page in manifest.pages:
    print(page.status, page.page_type, page.output_file)

4 of 4 extracted
extracted list pages/0001-quotes.toscrape.com.json
extracted login pages/0002-quotes.toscrape.com-login.json
extracted unknown pages/0003-quotes.toscrape.com-author-albert-einstein.json
extracted detail pages/0004-quotes.toscrape.com-tag-change-page-1.json

Resume an interrupted run

Progress is saved after every page. With the SDK or CLI, re-run with the same run_id (Crawler(config, run_id="…"), healix crawl --run-id …); see Resuming. Working at the lower level, just run the extractor again over the same manifest:

with PlaywrightDriverAdapter() as driver:
    ElementExtractor(driver, ExtractionConfig(output_path="output")).extract("output/manifest.json")

Pages already extracted are skipped, failed pages are retried, and a page marked extracted whose output file has gone missing is extracted again. Low-level access is on Manifest: remaining_pages(), mark_extracted(), mark_failed(), save().

Run configuration

One JSON file describes a crawl. It is safe to commit: it never holds secrets.

{
  "mode": "guided | autonomous",
  "base_url": "https://example.com",
  "start_url": "https://example.com/login",
  "crawl": {
    "discovery": {
      "domain_scope": "same_domain",
      "max_pages": 50,
      "dedupe_by": "url_normalized_and_structural_hash",
      "template_sample_size": 3,
      "click_discovery": false
    },
    "extraction": {
      "sequence": "one_by_one",
      "output_format": "json",
      "output_path": "./output/",
      "iframe_traversal": true,
      "platform_detection": "auto"
    }
  },
  "backend": "playwright | selenium",
  "roles": ["admin", "standard"]
}
Key Meaning
mode guided: crawl starts at start_url (and base_url, if given). autonomous: only base_url; the crawl discovers from scratch. Optional — inferred as guided when start_url is present
base_url, start_url Absolute http(s) URLs. Guided mode needs start_url; autonomous needs base_url
crawl.discovery See Discovery config
crawl.extraction See Extraction config
roles Optional list of user role names (never credentials): crawl once per role and compare. See Multi-role runs
backend playwright (default) or selenium. Everything else in the config means the same on both. A backend whose library is not installed raises BackendUnavailableError saying which extra to install

Everything is optional except the start point. Unknown keys are errors, not silently ignored, so a typo can’t quietly change a crawl.

Secrets never go here. Any key that looks like a credential (password, secret, token, api_key, credential…, username) is rejected at any depth, with a message pointing at .env. Put credentials and the webhook secret in the environment or a git-ignored .env (see .env.example).

SDK

from healix import Crawler, Extractor
   
Crawler(config, *, run_id=None, on_event=None, webhook_url=None, webhook_secret=None, webhook_outbox=None, driver=None, headless=True, credentials=None, auto_login=True, fingerprint_store=None, fingerprint_mode="keep") .discover() finds pages and writes the manifest. .discover_and_extract() then extracts each one
RoleCrawler(config, *, run_id=None, only=None, on_event=None, webhook_url=None, webhook_secret=None, webhook_outbox=None, headless=True, credentials=None, auto_login=True, fingerprint_store=None, fingerprint_mode="keep") For a config with roles: .discover() / .discover_and_extract() crawl once per role and return a MultiRoleRun (.runs, .diff, .diff_path) — see Multi-role runs. Crawler refuses such a config
Extractor(config=None, *, on_event=None, webhook_url=None, webhook_secret=None, webhook_outbox=None, driver=None, headless=True, credentials=None, auto_login=True, fingerprint_store=None, fingerprint_mode="keep", role=None) .extract(manifest) extracts a Manifest or a path to one, resumably. role picks whose credentials to use; it defaults to the manifest’s own
ScriptGenerator(source, *, base_dir=None, fingerprint_db=None, record_fingerprints=True, max_elements_per_page=200, on_event=None, webhook_url=None, ...) .to_playwright(style="pom") / .to_selenium(style="pom") / .generate(backend, style) build a script and return a GeneratedScript — see Script generation
Healer(store=None, *, driver=None, threshold=0.5, ambiguity_margin=0.05, run_id=None, on_event=None, webhook_url=None, ...) Records fingerprints and finds elements again after the page changes — see Self-healing

config is a path to a run-config JSON, a dict, or a RunConfig. Both calls return a Run: run.run_id, run.pages (manifest entries), run.pages_discovered, run.pages_extracted, run.pages_failed, run.platform_detected, run.discovery_status, run.blocked_on_auth, run.manifest_path, run.summary().

Resuming a run

run_id names a run. Re-running with the same run_id over the same output directory resumes it: finished discovery is reused, and extraction continues from the first page that is not extracted (failed pages are retried). The events you receive describe the work done by that call — a resume does not re-announce pages discovered earlier.

Without a run_id, a new one is generated and kept on crawler.run_id. If the output directory already holds a run, that is a RunConflictError rather than a silent overwrite — pass the run’s id to resume it, or choose another output path.

CLI

healix crawl   --config run_config.json [--output DIR] [--run-id ID] [--discover-only] [--role NAME]...
               [--webhook-url URL [--webhook-outbox PATH]] [--headed] [--no-login]
               [--fingerprint-db PATH] [--json] [--log-level LEVEL]
healix extract --manifest output/manifest.json [--config run_config.json] [--role NAME]
               [--webhook-url URL [--webhook-outbox PATH]] [--headed] [--no-login]
               [--fingerprint-db PATH] [--json] [--log-level LEVEL]
healix generate --input output/manifest.json [--backend playwright|selenium]
               [--style pom|test|action] [--output DIR] [--fingerprint-db PATH] [--no-record]
               [--webhook-url URL [--webhook-outbox PATH]] [--json] [--log-level LEVEL]
healix flush-events --outbox PATH --webhook-url URL [--timeout SECONDS] [--retry-rejected] [--json]
healix doctor [--launch] [--json]
healix --version          # also: python -m healix
Exit status Meaning
0 Success
1 Runtime error (browser failure, backend unavailable, …); for doctor, no usable backend
2 Usage or configuration error, including a run conflict
3 The run finished, but some pages failed (--run-id the same id to retry them)
4 Blocked on authentication: a login page was reached but no credentials are set
130 Interrupted. Progress is saved; the message says how to resume

Events

Every event has the same envelope, whether it arrives through on_event or a webhook:

{"event": "page_extracted", "run_id": "bebd30b63d35", "timestamp": "2026-09-19T18:00:09.658Z", "data": {"url": "…", "page_type": "list", "element_count": 136, "output_file": "pages/0001-quotes.toscrape.com.json"}}

data has exactly these fields for each event type — no more, no fewer. make_event validates it, so the contract can’t drift.

Event data contains
page_discovered url, page_type, structural_hash
page_extracted url, page_type, element_count, output_file
element_healed element_key, old_locator, new_locator, strategy_used, confidence_score, page_url
script_generated backend, style, file_path, element_count
run_complete pages_discovered, pages_extracted, platform_detected, manifest_path
login_failed url, reason, screenshot_ref
roles_compared roles, diff_path, differences

In a multi-role run every event also carries "role" in the envelope, next to event and run_id, saying which role it happened under. A run without roles has no such key, so its payloads are exactly what they always were. roles_compared is emitted once, after the last role, and its differences is the number of page and element differences in roles-diff.json.

page_type in page_discovered is provisional (classified during discovery); in page_extracted it is final. A login_failed reason is one of mfa_required, timeout, selector_not_found, auth_rejected.

Emitted today: page_discovered (once per new manifest entry), page_extracted (once per page written), login_failed (once per failed login — see Authentication), and run_complete (last). element_healed is emitted by Healer (see Self-healing), and script_generated by ScriptGenerator and healix generate (once per script; file_path is null when the script was returned but not written to disk).

A login_failed url never carries a query string (SSO redirect URLs hold state and code), and screenshot_ref is a path relative to the output directory, or null.

Webhooks

--webhook-url (or webhook_url=) POSTs each event’s JSON, in order, from a background thread — a slow endpoint never stalls the crawl.

Durable delivery

Add --webhook-outbox PATH (or webhook_outbox= on Crawler, Extractor, Healer and ScriptGenerator) and every event is written to a SQLite file before it is sent, and removed only once the receiver has accepted it:

healix crawl --config run_config.json --webhook-url https://hooks.example.com/healix \
             --webhook-outbox events.db

Authentication

Login is not a configuration step. When discovery or extraction lands on a page the classifier calls login, Healix fills it in and carries on — one credential per crawl. To crawl as several users and compare what each can reach, see Multi-role runs.

Credentials come from the environment, never from the run config:

# .env  (git-ignored; see .env.example)
WEBLIB_LOGIN_USERNAME=alice@example.com
WEBLIB_LOGIN_PASSWORD=...

The CLI loads .env from the current directory. In the SDK, call dotenv.load_dotenv() or pass Crawler(config, credentials=Credentials(username, password)).

What it handles

What it will not do

The failure modes are fixed, and none of them can hang or loop:

login_failed reason When
mfa_required A one-time-code / second-factor prompt appeared
auth_rejected The site refused the credentials, sent the form back, or the session keeps ending
timeout The form was submitted but nothing changed before the deadline
selector_not_found No usable login form: no password field, no submit control, or the page was not one it is willing to fill in (see below)

A failure also saves a screenshot to <output>/screenshots/login-failed-N.png and reports it in the event. It may show the username you typed (never the password, which the browser masks) — treat output/ as sensitive.

Blocked on auth

With no credentials set, reaching a login page does not fail the login — the run is flagged blocked_on_auth (in the manifest, on Run, in --json output; the CLI exits 4), one warning is logged, and the pages behind the login are recorded as failed with requires authentication, and no credentials are set. No login_failed event is raised: the event’s reasons describe a login that was attempted. Set the two variables and re-run with the same run_id; a blocked run is discovered again, since what is behind the login was never seen.

Where credentials are typed

Credentials are typed only on in-scope pages, or on pages whose URL host or path looks like an OAuth/SSO endpoint (/oauth/…, /authorize, /saml, /sso, okta.com, accounts.google.com, and so on). The query string never counts — anyone can append ?client_id= to a URL. An unexpected redirect to some other login page is refused, and nothing is typed.

Before typing, the handler also checks the form itself: exactly one password field. A registration page (two password fields) that was misclassified as a login is left alone.

Limit of that guard. It stops accidental credential entry. It is not a defence against a hostile site: an application chooses its own identity provider, so a site you point Healix at and give credentials to can send them anywhere a real SSO flow could. Only use credentials for sites you trust with them — the same trust you place in typing them into the site’s own login form.

Secrets stay out of everything

The credentials are never logged, never put in an event, and never written to disk; a Credentials object prints as Credentials(username='***', password='***'). URLs in logs and in login_failed events drop their query string. A test crawls with a distinctive password and checks that neither it nor the username appears in any log record, event, summary, manifest, or output file.

Multi-role runs

A site shows different things to different people. List role names in the run config, give each its own credentials in the environment, and Healix crawls the site once as each role and tells you what differs:

{ "base_url": "https://app.example.com", "roles": ["admin", "standard", "anonymous"] }

# .env  (git-ignored). Each role's variables carry its name in capitals.
WEBLIB_LOGIN_USERNAME_ADMIN=admin@example.com
WEBLIB_LOGIN_PASSWORD_ADMIN=...
WEBLIB_LOGIN_USERNAME_STANDARD=sam@example.com
WEBLIB_LOGIN_PASSWORD_STANDARD=...
# "anonymous" is reserved: it never logs in, and needs nothing.

healix crawl --config run_config.json

run 3f9a1c2b7d10: 3 role(s)
  admin: 14 of 14 pages extracted (discovery complete)
  standard: 9 of 9 pages extracted (discovery complete)
  anonymous: 2 of 2 pages extracted (discovery complete)
compared admin, standard, anonymous: 12 page difference(s), 7 element difference(s)
diff: output/roles-diff.json
output: output/

The diff

roles-diff.json compares every role that has a result:

{
  "roles": ["admin", "standard"],
  "summary": { "pages_reached": {"admin": 14, "standard": 9}, "pages_in_all_roles": 9,
               "page_differences": 5, "element_differences": 3,
               "only": {"admin": {"pages": 5, "elements": 3}, "standard": {"pages": 0, "elements": 0}} },
  "page_differences":    [ {"url": "https://app.example.com/admin", "roles": ["admin"], "missing": ["standard"]} ],
  "element_differences": [ {"url": "https://app.example.com/reports", "element": "button:export-csv",
                            "roles": ["admin"], "missing": ["standard"]} ]
}

What roles do and do not do

Self-healing

A script that finds #user-4471 breaks the day the id becomes user-9032. Healix records a fingerprint of every actionable element — everything extraction captured about it — and when a locator stops matching, it finds the element again by resemblance, then repairs the fingerprint.

from healix import Healer

with Healer("healix.db") as healer:
    healer.learn("https://shop.example.com/login")            # record the baseline once

    # ... later, after the page has changed ...
    healer.write("alice", "https://shop.example.com/login", "textbox:login-username")
    healer.click("https://shop.example.com/login", "button:login-submit")

Fingerprints usually come from a crawl: healix crawl --config run_config.json --fingerprint-db healix.db, or Crawler(config, fingerprint_store="healix.db"). Each is stored under (page_url, element_role). A role is a stable name built from the element’s most durable identity — textbox:login-username, button:sign-in, link:forgot-password — with repeats numbered #2, #3. Recording keeps an existing baseline (fingerprint_mode="keep"); pass "refresh" to overwrite it after an intended redesign.

How an element is found

  1. Primary locators, in priority order: stable test attributes (data-testid, data-test, data-qa, data-cy, …) → id → name → aria-label → css → xpath → normalized id → text. The first that matches exactly one element wins. A test id, id, name, or aria-label is trusted when it is the first locator tried. Any other match is verified against the fingerprint and rejected if it doesn’t resemble the stored element — a locator that matches something isn’t necessarily matching the element.
  2. Scoring. If no locator survives, every element with the same tag is scored against the fingerprint.
  3. The gate. The best candidate must reach the confidence threshold (default 0.5, configurable), and must not be a near-tie with the runner-up (ambiguity_margin, default 0.05). A best-but-weak or ambiguous match is rejected with ElementNotHealedError — never silently used. A page drawn on a canvas raises UnsupportedRenderingError instead (see below).

How candidates are scored

The score is a weighted comparison, not a flat one. Stable signals count for far more than volatile ones:

Signal Weight Signal Weight
stable test attribute 0.25 other attributes (type, href, …) 0.08
name 0.12 DOM path (ancestor tags) 0.08
visible text 0.12 id (raw, or same after normalizing) 0.04
nearby label text 0.12 classes 0.04
aria-label 0.10 parent · sibling index 0.03 · 0.02

Three rules decide how they combine:

One exception, for a locator’s own hit. Damping keeps a search honest, but it also means an element with little to identify it (an image-only link, an icon-only button: no text, label, name or test id) can never reach 0.5, even when it is exactly the element that was recorded. A page that had not changed at all would then fail. (A real book shop found this: every product image link, and a bare disabled text box on another site, failed on an unchanged page.) So when the css or xpath locator resolves to a single element, it is accepted if the live element provides everything the fingerprint has to offer and all of it agrees (raw similarity of at least 0.9, or your threshold if that is higher). An element that has lost the test id, name or text it was recorded with is not accepted this way, and neither is one whose href, text, label or class differs: those are weak matches, rejected as before. What this cannot catch is a look-alike that differs in nothing Healix can see, so an element with nothing but its position to go on is trusted at that position. It applies only to checking a locator’s hit, never to choosing among candidates.

A scored heal’s healed element log record (at info level) lists each signal’s similarity, and Score.explain() returns the same, so a surprising heal is debuggable.

What it does on a real page

One URL, changing between “deploys” (from the integration tests; roles are the username and password fields and the submit button):

The page changes to… Result Confidence
only generated ids regenerated found exactly by the test id — nothing to heal —
ids, classes regenerated; test ids stripped healed via name (fields), text (button); churn 0.87 · 0.87 · 0.66
heavy refactor: new wrappers, ids, classes; no test ids, names or aria-labels healed by weighted score (fields), text (button); churn 0.54 · 0.51 · 0.58
a field’s label becomes “Email address or mobile number” healed via name, flagged regression 0.75
the form is removed; only a search box remains refused — best candidate 0.13 —
two indistinguishable text boxes refused — below threshold (or ambiguous) 0.42
the UI is drawn on a <canvas> refused with UnsupportedRenderingError —

Note the heavy refactor: the healed fields clear the threshold by a hair (0.51, 0.54). The gate is real, and a redesign that also changes labels or types will not pass it.

The healing history

Every heal replaces the fingerprint and appends a record to the history, in one transaction, so the fix survives future runs and the store never holds one without the other. A record carries the old and new fingerprints, the old and new locators, the strategy, the confidence, and which fields changed — and a verdict on what kind of change it was:

healer.history(change_kind="regression")    # heal records, newest first
healer.report()                              # per element: heals, churn, regressions

healer.report() ranks elements by how often they heal — the flaky locators worth stabilizing.

Events

Each heal emits an element_healed event through on_event / webhook_url — the same payloads as every other event:

{"event": "element_healed", "run_id": "demo", "timestamp": "2026-09-19T18:59:00.860Z", "data": {"element_key": "https://shop.example.com/login#textbox:login-username", "old_locator": "stable_attr:data-testid=[data-testid=\"login-username\"]", "new_locator": "id=[id=\"f_8d2a\"]", "strategy_used": "name", "confidence_score": 0.873, "page_url": "https://shop.example.com/login"}}

strategy_used is the locator strategy that matched, or weighted_score for a scored heal.

The store

Two backends implement the same FingerprintStore interface (get, put, put_if_absent, fingerprints, apply_heal, history, close), and one contract test suite runs against both:

  Use Opened with
SQLite (default) one file, no extra dependency (stdlib sqlite3) a path: Healer("healix.db")
PostgreSQL fingerprints shared across machines or CI runs a URL: Healer("postgresql://user:pass@host:5432/db")

The path or URL is accepted everywhere a store is: Healer(...), Crawler(config, fingerprint_store=...), and healix crawl --fingerprint-db .... You can also pass your own FingerprintStore.

pip install "healix[postgres]"      # adds the psycopg driver
healix crawl --config run_config.json --fingerprint-db postgresql://user:pass@host:5432/healix

The canvas boundary

Canvas-rendered UI is unsupported, and the healer fails cleanly on it. Some Dynamics 365 canvas apps and legacy Java-applet Oracle Forms draw their UI instead of building it from DOM elements, so there is nothing for any locator to find. When healing fails on a page that is essentially one canvas (or plugin object) with almost no DOM controls, Healer raises UnsupportedRenderingError with an explanation — not a low-confidence guess.

Limits

Script generation

ScriptGenerator turns the extracted pages of a crawl into a script you can run, for Playwright or Selenium:

from healix import Crawler, ScriptGenerator

run = Crawler("run_config.json").discover_and_extract()
script = ScriptGenerator(run).to_playwright(style="pom", output="./scripts")
print(script.file_path, script.page_count, "pages,", script.element_count, "elements")

healix generate --input ./output/manifest.json --backend playwright --style pom

ScriptGenerator(run.pages) works too; the page files are then read from ./output (or base_dir=). to_playwright/to_selenium return a GeneratedScript (.source, .file_path, .page_count, .element_count, .skipped, .truncated); pass output= (a directory, or a .py path) to write it.

One generator, three styles, two backends. style picks what is written; backend picks which browser the script drives.

Style File What it is
pom pages_<backend>.py A class per page with a method per element (fill_username, click_sign_in, locate_…), and a session() that opens a browser
test test_healix_<backend>.py A pytest module: one test per page that checks the address and that every element is found, visible and (if it was) enabled. It types and clicks nothing, so it is safe against a live site
action actions_<backend>.py A plain script: per page, fill the inputs and click the submit button. No assertions

For example, an action script for a login page and an order form:

def run_login(healer: Healer) -> None:
    """The login page."""
    url = "http://shop.example.test/login"
    healer.driver.navigate(url)
    healer.write(os.environ["WEBLIB_LOGIN_USERNAME"], url, "textbox:login-username", navigate=False)
    healer.write(os.environ["WEBLIB_LOGIN_PASSWORD"], url, "textbox:password", navigate=False)
    healer.click(url, "button:sign-in", navigate=False)

Self-healing comes from Healer. Every step names an element by its stable role (textbox:login-username), not by a locator, so when the page changes the script resolves it the way Healer does — through the next locator that still matches, else a weighted, threshold-gated score — and records the fix. That needs the fingerprints: generating records them into --fingerprint-db (default healix.db) unless you pass --no-record because the crawl already did (healix crawl --fingerprint-db) — in which case pass that same --fingerprint-db, because a script looks in healix.db unless told otherwise. generate warns (GeneratedScript.warnings) when a local SQLite database has no fingerprints for the pages it scripted; a Postgres URL is not connected to just to check. The generated code calls healix.driver, so it does not import playwright or selenium itself, and it needs healix installed to run.

Secrets never go into a script. The value for a login page’s username and password fields is read from WEBLIB_LOGIN_USERNAME / WEBLIB_LOGIN_PASSWORD (a .env is loaded). Nothing is copied from the page’s own values. A postgresql:// --fingerprint-db is not written into the file (it carries a password): the script reads HEALIX_FINGERPRINT_DB instead. Page text is quoted as data, so a hostile page cannot inject code into a script, and the output is deterministic — the same manifest gives the same file.

Only pages whose status is extracted are used; pages left out are listed in .skipped with the reason, and a page with more than max_elements_per_page (default 200) actionable elements is cut off, counted in .truncated.

What generated scripts do not do

Validated on real sites

The fixture sites in the test-suite were written alongside Healix, so they cannot say whether it works on pages nobody wrote for it. Before 2.0, every kind of script was generated for six public sites built to be scraped and automated, and the generated tests were run against the live sites on both backends (each crawl bounded to at most 14 pages):

Site Pages crawled Generated tests passing (Playwright and Selenium)
quotes.toscrape.com 4 4 of 4
quotes.toscrape.com/js/ (rendered by script) 5 5 of 5
books.toscrape.com 2 distinct structures 2 of 2 (about 500 elements each, all found again)
the-internet.herokuapp.com 13 9 to 11 of 11, depending on which randomised page misbehaves that load
demo.playwright.dev/todomvc (single-page app) 1 1 of 1
saucedemo.com (logs in with the demo login it publishes) 2 1 of 2

The failures are the sites’ doing, and are what Healix should do:

What validation found, and fixed: on an unchanged page, the generated tests failed for every element with little to identify it (each book’s image link; a bare disabled text box). The scorer’s evidence damping capped such an element below the threshold even when it was exactly the one recorded. It is fixed (see How candidates are scored), and a fixture with the same shapes now guards it. Both backends also found the same pages and the same element counts on all five sites they were compared on, except where the site itself is non-deterministic or Chrome behaves differently (a 401 page, and http links Chrome upgrades to https).

It also measured the Playwright quiet-window wait (settle_quiet_ms) on three client-rendered sites: the pages and the element counts were identical with and without it, and it cost 15 to 45 percent more time. That is why it is still off by default.

Re-run it yourself, against the live sites, whenever you change how pages are read or elements are found:

HEALIX_REAL_SITES=1 pytest tests/real_sites -v

It is opt-in (network, several minutes), so CI does not run it, and a site changing can fail it. These are practice sites, not the enterprise applications Healix is meant for: a large real application is still the test that has not been run.

Checking the Salesforce adapter against your org

Nothing in the suite can load a Lightning org, so the adapter is unverified until someone runs it on one. A free Developer Edition org is enough, and nothing is changed in it:

  1. Put a login in .env (WEBLIB_LOGIN_USERNAME, WEBLIB_LOGIN_PASSWORD) and crawl a few pages: healix crawl --config run.json with "start_url" at your …lightning.force.com home and "max_pages": 5.
  2. Open output/manifest.json. "platform_detected": "salesforce_lwc" means the adapter saw the Lightning globals. null means it did not.
  3. Open a page file. Elements inside Lightning components should carry a platform_signal of the form {"platform": "salesforce_lwc", "component": "lightning-input", "is_component_host": false, "aura_attributes": {…}}, and element_counts.in_shadow_root should be above zero.
  4. If it does not detect Lightning, or a component tag is wrong, open an issue with the page’s platform_signal values (never with credentials or record data). The generic pipeline works either way: an adapter only ever adds a signal.

How it works

Healix crawls in two stages rather than as a depth-limited breadth-first walk.

Stage 1 — discovery                        Stage 2 — extraction
───────────────────                        ────────────────────
walk links, no depth cutoff                walk the manifest one page at a time
dedupe by URL + structural hash            navigate → wait → extract full element JSON
stop at max_pages or empty frontier        → write output → mark "extracted" → next
        │                                              ▲
        └────────────► manifest.json ──────────────────┘
                       (resume any run_id from the first non-extracted page)

Extraction is sequential by default so fingerprint-store writes stay ordered and resumability stays simple.

The Driver abstraction

class Driver(ABC):
    current_url: str  # property

    def navigate(self, url: str) -> None: ...
    def find(self, fingerprint: Fingerprint) -> Element: ...
    def click(self, target: Element | Fingerprint) -> None: ...
    def write(self, text: str, into: Element | Fingerprint) -> None: ...
    def get_elements(self, *, iframe_traversal: bool = True) -> list[Element]: ...
    def get_frames(self) -> list[Frame]: ...  # recursive, same-origin
    def screenshot(self) -> bytes: ...

Extraction, discovery, healing, classification, and generation never import playwright or selenium; they go through Driver. There are two adapters, and the rest of Healix cannot tell them apart: PlaywrightDriverAdapter (the default) and SeleniumDriverAdapter (healix.driver.selenium_adapter, browser="chrome", "firefox" or "edge"; Chrome and Firefox are both run through Healix’s own Selenium tests in CI (the Firefox job informs but does not block a merge or a release), and Edge is untested). Both take platform_adapters= (see Platform adapters), and both accept an existing Page / WebDriver to embed in a session you manage.

navigate() waits for the load event and then, best-effort, up to settle_timeout_ms (default 3000) for the page to go quiet so client-rendered pages have content before it is read. Set it to 0 to skip that wait. quiet_ms (or crawl.extraction.settle_quiet_ms in the run config) sets how long the page must stay unchanged, on both backends; see below.

Selenium differs from Playwright in three ways, and the adapter makes up for each:

  Playwright Selenium adapter
Waiting for the page Network idle. Off by default, quiet_ms adds a second wait for no new resources or elements, for pages that render from a timer or a script of their own after network idle No “network idle” exists in WebDriver, so it waits for load, then for no new resources or elements for quiet_ms (default 500). A request still in flight is invisible, so an API that answers more slowly than quiet_ms can be missed: raise quiet_ms for slow back ends
Clicking and typing Waits until the element is actionable Retries for up to action_timeout_ms (default 5000) while the element is missing, covered or not yet interactable. An ambiguous selector is reported at once, never retried
A page that will not load goto raises Some browsers show an error page and report success; the adapter detects it and raises, so a failed load is never read as a page

Shadow roots and frames are read by the same in-page scripts on both backends, so a page gives the same elements either way: the test-suite compares them, and every locator’s result, side by side. Selenium is not asked to walk shadow roots one getShadowRoot() call at a time; the in-page walk is one round trip per frame and, like Playwright, sees open shadow roots only.

Platform adapters

Thin, optional, and additive only. The generic pipeline — iframe traversal, shadow DOM piercing, stable-id normalization — always runs, whatever platform a page is built with. An adapter can only add an element’s platform_signal; if it does not fire, or throws, extraction is unchanged.

Adapter Fires when platform_signal
sap_ui5 window.sap.ui exists {"platform": "sap_ui5", "control_id": "__xmlview0--saveButton", "control_type": "sap.m.Button", "is_control_root": true} for an element inside a UI5 control, found through sap.ui.getCore().byId() (or Element.getElementById() on UI5 versions without getCore)
salesforce_lwc $A or Aura exists, or an element carries data-aura-rendered-by / data-aura-class {"platform": "salesforce_lwc", "component": "lightning-input", "is_component_host": false, "aura_attributes": {"data-aura-rendered-by": "1:0"}} — the Lightning component the element belongs to (its own tag, or the shadow host that rendered it) and its data-aura-* attributes

An element outside any control or component has platform_signal: null. The run’s manifest records platform_detected (the first platform that put a signal on an element). Turn it all off with "platform_detection": "off" in crawl.extraction or platform_adapters=() on a driver.

Limits, stated plainly: the signal is recorded on the element and its fingerprint, but the healer’s scorer does not weight it. The SAP adapter was checked by hand against a real OpenUI5 runtime; the suite checks both adapters against stubs shaped like the real APIs, since real SAPUI5 and Salesforce orgs are not something a test-suite can load. The Salesforce adapter has not been run against a real org, so it is not claimed as supported. Checking it against yours takes a few minutes. Custom adapters (healix.platform_adapters.PlatformAdapter, two snippets of JavaScript) must be valid JavaScript: it runs in the page next to the collector.

Iframes, shadow DOM, and stable IDs

These three are generic core capabilities, not per-vendor code.

Capability Behavior
Iframes The frame tree is walked recursively. Same-origin frames are merged into the page’s elements, each tagged with its iframe_path. Cross-origin frames (and everything beneath them) are listed by get_frames() with same_origin=False and skipped; skipped_frames() says which frames a read left out and why, and the page output records them
Shadow DOM Open shadow roots are pierced recursively. css_selector crosses the boundary with a descendant combinator; shadow_path lists each host’s selector, outermost first
Stable IDs Volatile id segments are replaced with placeholders; both id and id_normalized are stored, and find falls back to the normalized form when the raw id no longer resolves
Raw id Normalized
user-4471 user-{n}
pt1:r1:0:soc1::content pt{n}:r{n}:{n}:soc{n}::content
row-123e4567-e89b-12d3-a456-426614174000 row-{uuid}
btn_a3f9c2d81b btn_{hex}
login-form login-form

Element schema

Every element captures maximum raw detail at extraction time. This is real output for the “Learn more” link on example.com:

{
  "tag": "a",
  "id": null,
  "id_normalized": null,
  "name": null,
  "classes": [],
  "attributes": { "href": "https://iana.org/domains/example" },
  "text_content": "Learn more",
  "computed": {
    "visible": true,
    "enabled": true,
    "bounding_box": { "x": 256, "y": 186.078125, "width": 82, "height": 18 },
    "checked": null,
    "selected": null,
    "readonly": null,
    "required": null,
    "focused": false,
    "position": "static",
    "z_index": null,
    "href": "https://iana.org/domains/example"
  },
  "xpath": "/html[1]/body[1]/div[1]/p[2]/a[1]",
  "css_selector": "html > body > div > p:nth-of-type(2) > a",
  "iframe_path": ["main"],
  "platform_signal": null,
  "dom_context": {
    "parent_tag": "p",
    "parent_id": null,
    "sibling_index": 0,
    "nearby_label_text": null,
    "tag_path": ["html", "body", "div", "p", "a"]
  },
  "shadow_path": []
}

Notes on individual fields:

Discovery and the manifest

DiscoveryCrawler(driver, config).discover(start_urls, run_id=None, manifest_path=None) visits every distinct in-scope page reachable from the start URLs and returns a Manifest. Links are read from Driver.get_elements(), so anchors inside same-origin iframes and open shadow roots are found too.

Config

DiscoveryConfig is the crawl.discovery block of the run config:

Key Default Meaning
domain_scope "same_domain" same_domain (host match, ignoring a leading www.) or same_origin (scheme + host + port)
max_pages 50 Safety ceiling on page visits (navigations), not manifest entries
dedupe_by "url_normalized_and_structural_hash" Or "url_normalized" to keep every distinct URL
template_sample_size 3 Once this many URLs of one path template (/product/1, /product/2, …) have been visited, further ones are deferred until all other pages are visited — never dropped. 0 disables
click_discovery false Also click buttons and script links that have no <a href>, to find pages only script can reach. See Click-through discovery
max_clicks_per_page 15 Most clicks on any one page
max_clicks 100 Most clicks in the whole run
click_deny [] Extra words that mark a control as never to be clicked (added to the built-in list)

template_sample_size exists so a family of look-alike pages can’t use up max_pages ahead of distinct pages. With 200 product links and max_pages=50, /about still gets visited.

Click-through discovery

Some navigation is only script: a <button> that calls history.pushState, a <div role="button"> that sets location, an <a href="#"> with a click handler. There is no link to follow, so ordinary discovery cannot see the page behind it. Turn on crawl.discovery.click_discovery and Healix clicks such controls and notes where the page ends up:

{ "crawl": { "discovery": { "click_discovery": true, "max_clicks": 50 } } }

It is off by default, and a run without it behaves exactly as before. Clicking things on a live site can do harm, so what it does not do matters more than what it does:

Each click starts from a fresh load of the page, so one click cannot change what the next finds. That makes a run with many clicks slow, and a navigation bar that repeats on every page is clicked on every page: keep the caps tight and add the noisy controls to click_deny.

A page found this way says so in the manifest ("discovered_via": "click", "discovered_from": "<the page whose button led here>"), and the manifest records what happened overall:

"click_discovery": { "clicks": 12, "pages_found": 4, "skipped_unsafe": 5, "blocked_writes": 1 }

skipped_unsafe is how many controls were left alone for being unsafe, and blocked_writes how many requests the guard stopped. A non-zero blocked_writes means a button you did not expect to write did try to. Both keys are left out of the manifest when the option is off.

A found route is later extracted from a fresh navigation like every other page, so it has to load when opened directly. A single-page app whose server only answers / will record such a route as failed, which is honest: the page cannot be reached without the click. Routes in the URL hash (#/team) work, since normalize_url keeps them.

How pages are deduplicated

Manifest format

{
  "run_id": "demo",
  "start_urls": ["https://example.com/"],
  "discovery_status": "complete",
  "pages_discovered": 1,
  "pages_extracted": 0,
  "platform_detected": null,
  "blocked_on_auth": false,
  "pages": [
    {
      "url": "https://example.com/",
      "page_type": "unknown",
      "structural_hash": "c260c62ac032aa1d",
      "status": "pending",
      "output_file": null,
      "variant_urls": [],
      "error": null
    }
  ]
}

Manifest writes are atomic (temp file + rename), so a crash never leaves a truncated file.

Extraction and output

ElementExtractor(driver, config).extract(manifest) is the second stage. For each page in the manifest, in order, it navigates, waits for load, extracts every element at full detail, writes the page’s JSON, marks the manifest entry extracted, and moves on. It is sequential by design — it keeps fingerprint-store writes ordered and makes resume simple — so don’t parallelize it without revisiting that.

manifest is a Manifest or a path to one. Progress is saved to <output_path>/manifest.json (or manifest_path=) after every page. The call returns the updated Manifest.

Config

ExtractionConfig is the crawl.extraction block of the run config:

Key Default Meaning
sequence "one_by_one" The only supported value today
output_format "json" The only supported value today
output_path "./output/" Where manifest.json and pages/ go
iframe_traversal true Merge same-origin frames’ elements in; false reads the main frame only (open shadow roots are still pierced)
platform_detection "auto" auto: the platform adapters may add platform_signal, and the manifest records platform_detected. off: neither
settle_quiet_ms null How long a page must stay unchanged (no new resources, no new elements) before it is read. null keeps each backend’s own behaviour: Selenium waits 500 ms, Playwright waits for network idle only. Set it (for example 2000) for client-rendered sites that render after network idle. It applies to discovery too, and is bounded by the driver’s settle_timeout_ms. See The Driver abstraction

Output layout

output/
  manifest.json
  pages/
    0001-example.com.json
    0002-example.com-orders-42.json

A page’s output_file in the manifest is relative to output_path. File names use the page’s 1-based position in the manifest (unique and stable) plus a readable slug of its URL. Files are written atomically.

Page JSON

{
  "schema_version": 1,
  "run_id": "demo",
  "url": "https://quotes.toscrape.com/login",
  "page_type": "login",
  "structural_hash": "3fa4f7fde11f3d63",
  "captured_at": "2026-09-19T17:38:50.626Z",
  "element_counts": {
    "total": 28,
    "visible": 27,
    "in_shadow_root": 0,
    "by_tag": { "a": 4, "body": 1, "div": 9, "footer": 1, "form": 1, "h1": 1, "html": 1, "input": 4, "label": 2, "p": 3, "span": 1 },
    "by_frame": { "main": 28 }
  },
  "elements": [ "…each one in the raw element schema above…" ]
}

final_url is added only when the page redirected. skipped_frames is added only when a frame’s elements are missing from elements: a list of {"path": ["main", "ads"], "url": "…", "reason": "…"} with reason one of cross_origin (the frame has another origin, so a script cannot read it), inside_cross_origin_frame (it has the page’s origin, but sits inside a cross-origin frame, so it cannot be reached either) and unreadable (it could be reached but reading it failed, for example because it navigated away mid-read). A page without the key had nothing skipped. page_type and structural_hash are from the fresh extraction — if the structure changed since discovery, that is logged at info. by_frame keys are the iframe_path joined with /. Only the structural representative of a group of same-template pages is extracted; the others are listed under its variant_urls.

Failures, resume, and events

Output files are sensitive. They record page markup as found: attribute values, link URLs (which can carry tokens), and hidden-input values such as CSRF tokens. Only a password field’s markup value is redacted. output/ is git-ignored by default; keep it that way.

Page classification

classify(elements, url="") labels a page with one of nine types, or "unknown". It is pure element-count and attribute heuristics over the Element list and the URL — no LLM calls, no network, no randomness — so the same page always gets the same answer. Only visible elements vote, so hidden templates and collapsed panels don’t skew a result.

Type Detection heuristic
login A single password input plus a submit control and few other fields, with a sign-in cue in the URL or a heading; or an OAuth/SSO redirect URL, or an app sign-in page that offers only SSO (an SSO button plus a /login-style URL or a “Sign in” heading). Sign-up cues count against it
dashboard Several KPI/summary widgets and charts (canvas, chart-library markup), few inputs, an overview/dashboard cue
list Repeating row structures (table rows, cards) outside navigation, plus pagination controls
detail Single-entity display: one h1, key/value pairs (dl, read-only fields), text content, no repeating rows, an entity-shaped URL such as /product/123
form High input-field-to-text ratio, several fields, and a submit/save control
search A search input, filter controls (selects, checkboxes), and a results area — a header search box alone is not enough
checkout Payment-field patterns (cc-number, CVC, expiry…), multi-step indicators, checkout cues
nav_shell Mostly navigation links, low input density, little prose
modal A dialog (role="dialog", aria-modal, <dialog open>) or a large high-z-index fixed overlay that dominates the page

How a type is chosen

Each type has a rule that awards weighted signals (weights sum to 1.0) and a few penalties. A page gets a type when its score reaches 0.5. When several types score within 0.15 of the best, the most specific wins, in this order: modal, login, checkout, search, form, list, dashboard, detail, nav_shell. That is how a login form (structurally also a small form) is called a login, and a results page (also a form and a list) is called a search. If nothing reaches 0.5 the answer is "unknown" — an honest “no rule matched” rather than a weak guess.

classify_page returns the evidence — every type’s score and the signals that fired — so a surprising label is debuggable, and confidence lets callers ignore low-confidence labels:

result = classify_page(elements, url)  # a product grid with a header search box
result.page_type  # "list"
result.confidence  # 0.9
result.scores  # {"login": 0.15, "dashboard": 0.15, "list": 0.9, "detail": 0.1, "form": 0.45,
#  "search": 0.0, "checkout": 0.0, "nav_shell": 0.7, "modal": 0.0}
result.signals["list"]  # ["repeating_rows", "pagination", "few_fields"]
result.signals["nav_shell"]  # [..., "-repeating_content_rows"]  (a "-" prefix is a penalty)

Accuracy and limits

These are heuristics, not a guarantee. The rules are exercised against realistic rendered pages for every type and for the look-alikes that trip naive rules (a registration form with a password field, a header search box on a product grid, a cookie banner on an article), and spot-checked against public sites — login pages on GitHub and two practice sites, list pages on Hacker News and two scraping sandboxes, a DuckDuckGo results page. Expect misses on unusual layouts.

Logging

Healix logs through logquill: every record is a short constant message plus structured metadata, rendered as one JSON line —

{"timestamp":"2026-09-19T17:24:53.728Z","level":"INFO","logger":"healix.discovery.crawler","message":"discovery finished","meta":{"run_id":"demo","status":"complete","visits":1,"pages_discovered":1}}

By default only WARN and above are shown, on stderr (stdout stays free for program output). Turn up the detail with the environment or in code:

HEALIX_LOG_LEVEL=debug python my_crawl.py

from logquill import FileTransport
from healix.log import configure_logging

configure_logging(level="info")  # just change the level
configure_logging(transports=[FileTransport("healix.log")])  # send records elsewhere
configure_logging(transports=[])  # silence Healix entirely

Levels: info records discovery start and finish, debug adds every discovered page, every classification with its scores, redirects, and skipped frames; warn reports pages that failed to load and frames that couldn’t be read. Any logquill transport or plugin works — see the logquill docs.

In your own code around Healix, from healix.log import get_logger gives you a logger under the same configuration. configure_logging updates every Healix logger, including ones created before it was called.

Design decisions

The full picture of how the package is built (layers, data flow, the healing algorithm, script generation, extension points) is in ARCHITECTURE.md.

These are fixed unless explicitly reopened:

Known limitations

Roadmap

Milestone Scope Status
1 Driver ABC, Playwright adapter, iframe + shadow DOM traversal, ID normalization ✅ Done
2 Discovery, manifest, dedup, per-page status ✅ Done
3 Rule-based page classification (login, dashboard, list, detail, form, search, checkout, nav_shell, modal), and logquill-based logging ✅ Done
4 Sequential extraction — one JSON file per page, resumable ✅ Done
5 SDK (Crawler, Extractor), CLI (crawl, extract), event schema and webhooks ✅ Done
6 Auto-detected login with .env credentials, SSO, MFA abort, mid-crawl re-login ✅ Done
7 Self-healing: fingerprints, weighted scorer, confidence threshold, persistent store and history ✅ Done (SQLite and PostgreSQL stores)
8 Selenium adapter, SAP UI5 and Salesforce LWC platform adapters ✅ Done
— Script generation: ScriptGenerator, healix generate, the script_generated event ✅ Done
9 Packaging (extras, .env.example, MIT licence, README), healix doctor, PyPI release ✅ Done
10 Release gate, opt-in Playwright quiet window, skipped-frame reporting, Firefox in CI ✅ Done (1.1)
11 Durable webhook delivery: an outbox, healix flush-events, a delivery id ✅ Done
12 Opt-in click-through discovery with a write guard ✅ Done
13 Multi-role runs and the comparison between roles ✅ Done
14 Validation on real sites, the locator-hit fix, release 2.0 ✅ Done. The Salesforce adapter is still unverified against a real org

Releases go out from a version tag (v*) through the release workflow, which publishes to PyPI with trusted publishing. Before anything is built it checks that the tag matches the version in pyproject.toml and healix.__version__ and that the changelog has an entry for it, and it runs the full CI suite on the tagged commit; a failure in any of them stops the release.

API reference

Every public class and function has a docstring; the reference is generated with pdoc and published at nikhilvdev.github.io/healix on every push to main. To build it locally:

pip install -e ".[docs]"
pdoc --docformat google healix

Development

python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev,hooks]"
playwright install chromium
pre-commit install

ruff check .
ruff format --check .
mypy
pytest

The browser tests run real headless Chromium against small fixture sites served over local HTTP (including a second origin for the cross-origin cases); they skip themselves if Playwright or its browsers aren’t installed. Most of them run once per backend, so Selenium tests need Chrome. The generation tests build scripts from a real crawl and run them, on both backends. Set HEALIX_TEST_SELENIUM_BROWSER=firefox to run the Selenium tests on Firefox (CI has a non-blocking job that does); asking for a browser by name means a launch failure fails the test instead of skipping it.

The PostgreSQL store tests need a real server. Set HEALIX_TEST_POSTGRES_URL to point at one (CI does, with a service container), or just have Docker running: the tests start a throwaway postgres:16-alpine container and remove it afterwards. With neither, they skip.

See CONTRIBUTING.md for the PR workflow, the Code of Conduct for community standards, and SECURITY.md for how to report a vulnerability.

License

MIT

This site is open source. Improve this page.