Metadata-Version: 2.4
Name: abstract_toolserver
Version: 0.0.50
Summary: The abstract_* ecosystem exposed as an API-callable AI toolset: functions become self-describing Flask endpoints with /endpoints discovery and ?help.
Home-page: https://github.com/AbstractEndeavors/abstract-toolserver
Author: putkoff
Author-email: partners@abstractendeavors.com
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: server
Requires-Dist: abstract_flask; extra == "server"
Requires-Dist: abstract_utilities; extra == "server"
Provides-Extra: files
Requires-Dist: abstract_flask; extra == "files"
Requires-Dist: abstract_utilities; extra == "files"
Requires-Dist: abstract_search; extra == "files"
Requires-Dist: abstract_paths; extra == "files"
Provides-Extra: web
Requires-Dist: abstract_flask; extra == "web"
Requires-Dist: abstract_utilities; extra == "web"
Requires-Dist: abstract_webtools; extra == "web"
Provides-Extra: media
Requires-Dist: abstract_flask; extra == "media"
Requires-Dist: abstract_utilities; extra == "media"
Requires-Dist: abstract_ocr; extra == "media"
Requires-Dist: abstract_pandas; extra == "media"
Requires-Dist: media_intelligence; extra == "media"
Provides-Extra: db
Requires-Dist: abstract_flask; extra == "db"
Requires-Dist: abstract_utilities; extra == "db"
Requires-Dist: abstract_database; extra == "db"
Provides-Extra: ui
Requires-Dist: abstract_flask; extra == "ui"
Requires-Dist: abstract_utilities; extra == "ui"
Requires-Dist: abstract_clicks; extra == "ui"
Requires-Dist: abstract_windows; extra == "ui"
Provides-Extra: claude
Requires-Dist: abstract_flask; extra == "claude"
Requires-Dist: abstract_utilities; extra == "claude"
Requires-Dist: abstract_claude; extra == "claude"
Provides-Extra: gpt
Requires-Dist: abstract_flask; extra == "gpt"
Requires-Dist: abstract_utilities; extra == "gpt"
Requires-Dist: abstract_gpt; extra == "gpt"
Provides-Extra: fileshare
Requires-Dist: abstract_flask; extra == "fileshare"
Requires-Dist: abstract_utilities; extra == "fileshare"
Requires-Dist: abstract_logins; extra == "fileshare"
Requires-Dist: abstract_securefiles[relay]; extra == "fileshare"
Requires-Dist: abstract_database; extra == "fileshare"
Requires-Dist: gunicorn; extra == "fileshare"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-python
Dynamic: summary

# abstract_toolserver

The `abstract_*` ecosystem exposed as an **API-callable AI toolset** — every tool
is a plain Python function turned into a self-describing HTTP endpoint by
[`abstract_flask`](https://github.com/AbstractEndeavors/abstract_flask). A portable
tool layer any model (Claude, hugpy, …) can drive over HTTP instead of being
bound to one runtime's tool harness.

## Part of the hugpy orbit

```
                    ┌──────────────────────────── hugpy (fleet) ───────────────────────────┐
                    │ central + workers: platform · engine · fleet · server · media · …     │
                    │ OpenAI-compatible /v1 — every local model, incl. B (Qwen3-Coder-Next) │
                    └───────▲───────────────────────▲──────────────────────────▲───────────┘
                            │ inference             │ inference                │ B reductions
   ┌────────────────────────┴──┐   ┌────────────────┴──────────┐   ┌───────────┴───────────────┐
   │ hugpy-station             │   │ hugpy-agent               │   │ abstract-toolserver       │
   │ desktop + headless console│──▶│ agent runtime · TUI ·     │◀─▶│ comms · ledgers · boards ·│
   │ tmux seats per locus      │   │ OpenCode/qwen seats       │   │ exchanges · MCP · b_ask   │
   └────────────┬──────────────┘   └────────────┬──────────────┘   └───────────▲───────────────┘
                │ keeper/codex seats            │ --serve                      │ tools (MCP/HTTP)
   ┌────────────▼──────────────┐   ┌────────────▼──────────────┐               │
   │ abstract-gpt (Codex seat) │   │ abstract-claude serve ────┼───────────────┘
   │ abstract-claude (Claude)  │   │  └ abstract-serve-core    │
   └───────────────────────────┘   └───────────────────────────┘
          everything ships through abstract-pypit → PyPI (+ GitHub)
```

| Package | Role | PyPI |
|---|---|---|
| **hugpy** (14 lockstep dists) | the self-hosted LLM fleet: central, workers, engine, media, server | [hugpy](https://pypi.org/project/hugpy/) |
| **hugpy-station** | Electron desktop + headless backend; tmux seats, prompt composer, loop/bug scan | deb via central install links |
| **hugpy-agent** | agent runtime on the fleet; `hugpy-agent tui` over abstract-claude serve | [hugpy-agent](https://pypi.org/project/hugpy-agent/) |
| **abstract-claude** | Claude Code launch/session/rollover + `abstract-claude serve` (roles keeper/chat/worker/local) | [abstract-claude](https://pypi.org/project/abstract-claude/) |
| **abstract-serve-core** | the HTTP routes `abstract-claude serve` actually runs (queue, relay, rollover sweeps) | [abstract-serve-core](https://pypi.org/project/abstract-serve-core/) |
| **abstract-gpt** | Codex/ChatGPT seat counterpart of abstract-claude | [abstract-gpt](https://pypi.org/project/abstract-gpt/) |
| **abstract-toolserver** | one tool service per host: comms, ledgers, boards, exchanges, MCP bridge, B on call | [abstract-toolserver](https://pypi.org/project/abstract-toolserver/) |
| **abstract-pypit** | one-command publisher: bump → build → PyPI → GitHub push | [abstract-pypit](https://pypi.org/project/abstract-pypit/) |

## Run it

```bash
pip install abstract_toolserver          # + the extras you want to expose
python -m abstract_toolserver            # HOST/PORT/DEBUG from TOOLSERVER_* env
```

```python
from abstract_toolserver import get_toolserver_app
app = get_toolserver_app()               # a normal Flask/WSGI app
```

## Self-describing surface

The app auto-mounts introspection endpoints (from `abstract_flask`):

| Endpoint | What it gives an LLM |
|---|---|
| `GET /prefixes` | the tool categories (`/fs`, `/db`, `/ui`, …) |
| `GET /endpoints` | every tool as `{endpoint, url, methods}` |
| `GET /<cat>/<tool>?help=true` | that tool's signature/help |

Call a tool with JSON; unknown keys are pruned to the function signature, and the
reply is `{"result": ...}` (or `{"error": ...}`). Discover-and-dispatch from a
client is already provided by `abstract_apis.make_endpoint_call`.

```bash
curl -s localhost:5000/fs/count_tokens -d '{"text":"hello world"}'
# {"result": 2}
curl -s localhost:5000/db/schema           # {"result": {table: [cols...]}}
curl -s 'localhost:5000/db/query?help=true'
```

## Tool categories

| Prefix | Tools | Backend |
|---|---|---|
| `/fs` | search, read_span, extract, read_file, write_file, read_json, find_keys, find_paths, glob, imports, find_content | abstract_search, abstract_utilities, abstract_paths |
| `/text` | count_tokens, chunk, detect_language | abstract_utilities |
| `/web` | text, links, attributes | abstract_webtools |
| `/media` | ocr_image, pdf_text, summarize, keywords, transcribe | abstract_ocr, abstract_pandas, media_intelligence |
| `/ai` | query | abstract_ai |
| `/db` | tables, schema, columns, fetch, **query** | abstract_database |
| `/sys` | run_cmd | stdlib (gated) |
| `/ui` | capture, monitors, ocr, windows, **click_verify** | abstract_clicks, abstract_windows |
| `/browser` | shot, go, locate, **click**, type, key, scroll, read, console_save — Firefox in a libvirt guest driven by screen only (`vm=`, default `ubuntu-desktop`) | abstract_clicks (backends.qemu, vision, macros.firefox) + `/vl` fleet |
| `/loci` | list, **pointers** (the hugpy-station distribution feed), register, archive | central locus registry (Postgres) |
| `/handoff` | request, list, claim, station (seat-API probe) | jump-in seats via hugpy-station |
| `/session` | identity, **pull**, spin, list, release | pull a live Claude Code session into a hugpy-station seat |
| `/instructions` | tree, read, add | composition guides plus create-only caller contributions |
| `/b` | **ask** — B (the fleet's local model) reduces text / a ledger / a file to the context a question needs | hugpy fleet `/v1/chat/completions` |
| `/comms` | ping, send, inbox, poll, claim, ack, reply, delivered | central board rows + live push to serves |
| `/ledger` | put, get, list, template — structured handoff state per `locus/task` | Postgres |
| `/todo`, `/board` | add, batch, update, done, remove, list · board list/summary | Postgres (scoped write path) |
| `/exchange`, `/assess` | record, ingest_transcript, archive, sessions, usage · rolling state, focus, roll | Postgres + `rolling_reduce` |
| `/canvas`, `/issue`, `/vl`, `/vm`, `/image`, `/claude`, `/gpt` | design docs · issue memory · fleet vision models · VMs · images · Claude/Codex seat config (from abstract-claude / abstract-gpt when installed) | various |

## Operating instructions

Tool schemas describe individual calls; the built-in instruction tree documents
how calls compose into repeatable workflows:

- `GET /tools/toolserver/` — MCP configuration, discovery, runtime-neutral
  channel comms, and optional runtime-specific wake-up adapters.
- `GET /instructions/ae/solcatcher/` — Solcatcher-specific entry point.
- `GET /instructions/a-brain/alpha/` — Alpha's capability-channel instructions.
- MCP: `instructions_tree`, then `instructions_read`, through `ts_call`.

Authenticated MCP callers may create a new document with `instructions_add` at
an `instructions/...` path. Creation is durable and immediately readable, but
MCP intentionally exposes no update or delete tool. Those operations remain
local to server administration, and built-in documents are immutable.

## Safety gates

Backends load lazily, so the server boots on a headless box and a missing backend
errors only when its tool is called. Beyond that:

- **`/db/query`** — read-only gate: rejects anything that isn't a single
  `SELECT`/`WITH`, blocks stacked statements and data-modifying keywords. Use
  `/db/fetch` (identifier-composed, params-not-SQL) as the default read path.
- **`/sys/run_cmd`** — disabled unless `TOOLSERVER_CMD_ALLOWLIST=ls,grep,…` is set;
  only allowlisted binaries run.
- **`/fs/read_file` · `/fs/write_file`** — local-only; the underlying SSH/remote
  kwargs are never exposed at the boundary.
- **`/session/pull`** — the agent's own MCP bridge (`abstract-claude mcp`) fills the calling
  session's identity (session id, user@host, cwd); the toolserver registers machine + session
  loci, records a `pull` handoff, and asks the station (`HANDOFF_SPAWN_URL`) to seat
  `claude --resume <id> --fork-session` there. Without a resume-capable station it stays
  pending (`/session/spin` retries) — never a silent fresh seat. A pulled session carries its
  OWN name in the station (`name=` → tmux seat **and** session locus; default `sess-<id8>`).
- **default loci** — the station service user (`vm_mgr`) and the host are always in the
  `/loci/pointers` distribution: seeded once into the registry, re-merged into the feed even
  before the table exists (`TOOLSERVER_DEFAULT_LOCI`, `TOOLSERVER_STATION_USER`).
- **`/ui/click_verify`** — the click→observe→**verify** loop: locate (text or image
  template) → click → re-capture → report whether the screen (or a `region`)
  changed. The half most tool APIs lack.

## Authentication — the operator token is the ONLY gate

Every route requires `TOOLSERVER_OPERATOR_TOKEN`, sent as `X-Operator-Token: <token>`
or `Authorization: Bearer <token>` (the MCP bridge sends both). There is **no IP
allow-list and no loopback bypass**: a LAN, WireGuard or `127.0.0.1` caller without the
header gets `401 {"error":"unauthorized"}` exactly like the public internet (operator
ruling 2026-09-29 — the nginx `allow 192.168.x/deny all` block that used to front
`toolserver.hugpy.ai` was a second, redundant gate and is gone). With the env var unset
the server fails **closed** (every gated route 401s; startup logs an error).

Open by design (they carry their own credential or expose nothing):

| Path | Why |
|---|---|
| `GET /healthz` | liveness probe, `{"ok": true}` only |
| `GET /endpoints?access=<TOOLSERVER_ENDPOINTS_TOKEN>` | read-only catalog capability for a browser link |
| `/ch/<id>?t=<token>` | shareable comms link — per-channel token (channels.py) |
| `/clients/heartbeat` · `/clients/work` · `/clients/result` | per-client token (clients.py) |
| `GET /` `/console` `/ui` | the static console page where the operator types the token (wsgi.py) |

`TOOLSERVER_REQUIRE_TOKEN` (the old opt-in blueprint gate) is deprecated: parsed,
logged as ignored, never enforced.

## Configuration

| Env var | Purpose |
|---|---|
| `TOOLSERVER_OPERATOR_TOKEN` | **required** — the only access gate (see Authentication) |
| `TOOLSERVER_ENDPOINTS_TOKEN` | optional read-only capability for `GET /endpoints?access=` |
| `TOOLSERVER_HOST` / `TOOLSERVER_PORT` / `TOOLSERVER_DEBUG` | bind + debug |
| `TOOLSERVER_CMD_ALLOWLIST` | comma-separated binaries `/sys/run_cmd` may run |
| `SOLCATCHER_POSTGRESQL_*` | DB connection (via abstract_database) |
| `HANDOFF_SPAWN_URL` / `HANDOFF_SPAWN_TOKEN` | hugpy-station seat API (`/api/handoff/spawn`) + its X-Console-Token |
| `TOOLSERVER_DEFAULT_LOCI` | `name=user@host[:port][\|goal],…` — loci every station inherits (default: `vm_mgr` + this login on this host) |
| `TOOLSERVER_STATION_USER` | station service user for the fallback default locus (default `vm_mgr`) |

## canvas.* — per-locus ◳ design / flow documents (2026-09-03)

The station's ◳ canvas tab (⬚ design = `wireframe.v1`, ⋔ flow = `flow.v1`) and
every seat share ONE copy per (locus, kind) in the `canvas` table:

- `POST /canvas/get  {locus, kind}` → `{state|null, rev, by, note, updated}`
- `POST /canvas/put  {locus, kind, state, by?, note?, notify?}` — whole document,
  validated fail-closed, stored verbatim; a flow's `rev` bumps on every changed
  put; `notify=true` also posts a `[canvas]` high-priority request on the locus
  board (a deliberate hand-off — never for autosave).
- `POST /canvas/list {locus?}` → which loci hold which kinds (no bodies).

Writes fire on the `locus_change` bus (table `canvas`, id = kind) so open
drawers reload live. Through `abstract-claude mcp` these are the Claude Code
tools `canvas_get` / `canvas_put` / `canvas_list`.

## B on call — `b_ask` (2026-10-02)

`b_ask(question, text= | ledger_locus=,ledger_task= | path=, model=)` sends the
source plus the question to **B**, the fleet's local model, and returns
`{answer, excerpts[], model, source, chars, clipped}`. Excerpts are verbatim
passages from the source; B is told never to invent content. This is how an
agent reads an oversized ledger or file without loading it whole.

- Model: `TOOLSERVER_B_MODEL` → `HANDOFF_POLISH_MODEL` → `HANDOFF_JUDGE_MODEL`
  → default `Qwen3-Coder-Next-GGUF`. Transport: `HUGPY_BASE` +
  `HUGPY_API_KEY`, the same fleet JSON chat the roller uses.
- `TOOLSERVER_B_MAX_CHARS` (48000) caps the source sent; `clipped: true` says
  it was cut. `TOOLSERVER_B_TIMEOUT` (120 s).
- Latency: ~18 s warm on Coder-Next for a 33 KB file; the first call after
  an idle period includes the model load (~100 s).
- Ledgers are read straight from the DB, so a `b_ask` over a ledger is never
  hit by the MCP result governor below.

## Comms

`comms_ping` writes a durable `[ping]` board row, then (0.0.47+) makes a
best-effort live push to the target locus's registered `pointer.serve_url`
(`POST <serve>/api/session/message`, `to=keeper`, `wait=false`, 5 s, never
blocks; `result.push` reports `pushed to …`). The lookup ignores the
registry's archive status. `from_`/`to` ids must match
`[a-z0-9][a-z0-9_-]{0,62}` — no colons. `kind=message` rows are drained into
the target Station's mail tab and auto-closed.

## Ledgers, handoffs, boards

- **Ledgers** (`ledger_put/get/list/template`) are the handoff state of record
  per `locus/task`: goal, rulings, decisions, world state, in-flight step,
  open questions, pointers. A put replaces the whole doc; a sections-only put
  fails (500).
- **Handoffs** (`handoff_request/pull/claim`) hold the seat handoff; the old
  file-pointer handoffs are retired.
- **Boards**: `todo_update` refuses unknown ids (`no todo <id>`); board-item
  format is checked per `TOOLSERVER_BOARD_FORMAT=warn|enforce`.
- `instructions_add` needs both `path` and `text`.

## Rolling state

`rolling_reduce.py` is **deterministic** — no model call. It folds primed
exchange rows into `objective / done / in_progress / blockers / next_steps /
key_paths / open_questions / init_prompt` (each list capped at 30). Since
0.0.48, tool-failure blockers expire once newer turns arrive, prose blockers
drop when the objective changes, and nothing expires on an empty slice.
Station's banner shows `blockers[0..3]` of `/api/frontier/state`. An optional
model polish (`HANDOFF_POLISH=1`, `HANDOFF_POLISH_MODEL`, ≤30 s, skipped if
busy) and the legacy judge fold (`HANDOFF_REDUCER=judge`) remain available.

## MCP bridge and the result governor

`abstract-toolserver-mcp` exposes every tool as `mcp__toolserver__<name>`;
`ts_call` reaches any tool by name, including ones added after the MCP
client loaded its tool list. `session_message` accepts the cross-locus
target `<locus>:keeper`.

Every MCP result over `AC_MCP_RESULT_MAX_LINES` (200) or
`AC_MCP_RESULT_MAX_BYTES` (16384) is cut to 75 % head + 25 % tail, the full
text is archived (`exchange_archive_get locus=… source=mcp-result:<tool>:<ts>`),
and a `[truncated: …]` marker is appended. `AC_MCP_GOVERNOR=0` disables it;
`AC_MCP_GOVERNOR_EXEMPT` lists tools never cut (default: the `fs_read_*`
tools and `exchange_archive_get`). Prefer `b_ask` over reading a truncated
result.

## Deployment on a host

One Flask service per host: unit `7004_hugpy_toolserver` (gunicorn as
`vm_mgr`, `127.0.0.1:7004`) running an **editable install of the dev tree**,
so a service restart picks up source edits without a publish. Discovery
writes `endpoint.json`. Token resolution order: `HUGPY_TOOLSERVER_TOKEN` →
`TOOLSERVER_OPERATOR_TOKEN` → `TOOLSERVER_TOKEN` → `HUGPY_OPERATOR_TOKEN` →
`STATION_CONSOLE_TOOLSERVER_TOKEN` → env files. Auth fails closed when no
token is set; `/ch/` channels and per-client token paths are exempt; the
`/vms/` page uses cookie auth.

## Attention-worthy

- The MCP tool list is fixed when a client session starts: a new tool needs a
  service restart **and** a new session to appear as `mcp__toolserver__*`
  (use `ts_call` meanwhile).
- `ledger_get` returns the doc twice (`doc` + `sections`), so a 17 KB ledger
  trips the 16 KB governor — read big ledgers through `b_ask`.
- The roller's background `exchange_ingest_transcript` logs `no transcript at
  (none found)` when a locus has no transcript; harmless.
- Fleet `usage.prompt_tokens` can under-report on the hugpy route.

