Every command accepts --api <base-url> to run on a remote webx serve/webxd instead of locally.
| Command |
What it does |
scrape <url> |
URL → clean markdown + metadata. Render, actions, fit, chunks, PII redaction, a11y refs |
batch <file> |
Scrape many URLs concurrently; --api makes it a durable server job |
map <domain> |
Sitemap URL discovery |
cdx <pattern> |
Common Crawl index — enumerate URLs without touching the host |
wayback <url> |
Wayback Machine captures — list, or extract --at <ts> |
archive <url>… |
Raw fetches → WARC/1.0 (ISO 28500, replayweb.page-compatible) |
| Command |
What it does |
search <q> |
15-provider metasearch — weighted RRF + BM25 + domain priors |
ask <q> |
Search → read top results → cited excerpts in one shot |
research <q> |
Deep research → cited markdown report (--schema for JSON) |
verify <claim> |
Fact-check → supported|refuted|unclear + confidence |
eval |
Search/extraction quality on the bundled dev set (hit rate + MRR) |
doctor |
Probe every provider — liveness, latency, config errors |
| Command |
What it does |
index <domain> |
Crawl → local SQLite FTS5 (+ embeddings with WEBX_EMBED_MODEL) |
query <q> |
Index-only search — offline; --semantic adds neural fusion |
seed |
Index ~25 curated docs domains |
similar <url> |
Term-signature near-neighbors — index or --web |
diff <url> |
Live vs indexed line-diff — conditional GET (304-aware) |
watch <url> |
Change monitor — --every, --once, --exec, --semantic |
llms <domain> |
Whole-site llms.txt from the index |
| Command |
What it does |
extract <url> |
Page → schema’d JSON — deterministic CSS/JSON-LD or LLM |
mcp |
MCP server — stdio, or POST /mcp on a running server (17 tools) |
skill |
Emit a SKILL.md teaching agent harnesses how to use webx |
botkey |
Web Bot Auth signing key + self-hostable JWKS |
| Command |
What it does |
serve |
HTTP API — all routes, tenancy, jobs, /v1+/v2 Firecrawl compat |
jobs / cancel |
List/inspect/cancel async jobs — local or --api remote |
install <engine> |
Install Chrome for Testing or Lightpanda |
license |
keygen / gen / check — signed license tooling |
completion |
Shell autocompletion (bash/zsh/fish/powershell) |
help |
Help on any command |
