
doc-maintenance
Audits README, SPEC, and PRODUCT docs against recent git history to detect drift and opens minimal PR-ready accuracy fixes.
Lift+0ppscreen tier · directional
90
Trustsecurity · quality · license · upkeep
Scan Saferisk 0/100
LicenseMITserved via MCP
Skill-matched tasks (generated, public + held-out) · glm-5.3-flash@cf
+0pp95% CI [0, 0]unverified · screen tier
trigger rate 100% · 12 trials
A/B outputs — median trial per arm
Same task, same model: without the skill (left) vs with it (right).
doc-maintenance--m1
baseline · 1.00
Yes — two things changed in the log that affect the README:
1. **Version is stale.** The package was bumped to 2.2.0, but the README pins 2.1.0.
- Edit: `pip install coolpkg==2.1.0` → `pip install coolpkg==2.2.0`
2. **The `--env` example is now misleading.** The default changed to `dev`, so readers may assume `--env prod` is the default behavior. The command itself still works (explicit flags override the default), but a one-line clarification avoids confusion:
- Edit: keep `coolpkg init --env prod` and add a note, e.g.:
> `coolpkg init` defaults to `--env dev`; pass `--env prod` to initialize for production.
Minimal diff summary:
```diff
- pip install coolpkg==2.1.0
+ pip install coolpkg==2.2.0
coolpkg init --env prod
+ # Note: default env is now `dev`; use --env prod for production.
```
If you want the README to reflect the *new* default instead, you could change the example to plain `coolpkg init` — but that depends on whether your Quick Start is meant to demo the default or the production path.with skill · 1.00
Yes — both commits affect the Quick Start. Two minimal edits: 1. **Version bump:** `pip install coolpkg==2.1.0` → `pip install coolpkg==2.2.0` 2. **Default env:** `coolpkg init --env prod` → `coolpkg init --env dev` (or just `coolpkg init`, since `dev` is now the default — pick whichever matches the surrounding style; if the README previously showed the flag to be explicit, keep `--env dev`) Nothing else in the log suggests drift (no new features, removals, or structural changes). After committing, update your review cursor (`.doc-review-cursor`) to `3f2a1b9` so the next audit starts from there.
doc-maintenance--m2
baseline · 1.00
Based on the recent commits, the docs have drifted from the implementation. Here's what needs fixing:
## SPEC edits
1. **Endpoint path**: Change `POST /api/v1/items` → `POST /api/v1/products` (per the rename commit).
2. **Status code**: Change `201` → `200` (per the "return 200 instead of 201 on create" commit).
## PRODUCT doc edits
1. **Example curl**: Update the URL from `/api/v1/items` → `/api/v1/products` so it matches the renamed endpoint and the SPEC.
## Suggested updated SPEC line
```
POST /api/v1/products accepts JSON {name, price} and returns 200 with {id, name, price}.
```
Note: the response body shape `{id, name, price}` is unaffected by the commits, so no change needed there. If the 200-on-create behavior was unintentional (REST convention favors 201), you may instead want to revert the fix commit — but as written, the docs should match the current behavior.with skill · 1.00
Based on the recent commits, here's the drift and the minimal edits needed:
## SPEC (`doc/SPEC.md`)
Two factual inaccuracies:
1. **Endpoint path**: `POST /api/v1/items` → `POST /api/v1/products` (from `feat: rename /api/v1/items to /api/v1/products` — a breaking rename)
2. **Status code**: `201` → `200` (from `fix: return 200 instead of 201 on create` — a fix that changes documented behavior)
The request/response body shape (`{name, price}` → `{id, name, price}`) is unaffected.
## PRODUCT doc (`doc/PRODUCT.md`)
1. **Example curl**: update the URL from `/api/v1/items` to `/api/v1/products`
2. **Check for a status-code mention**: if the example or surrounding text references `201 Created`, change it to `200 OK` to match the SPEC fix
## Notes
- The `docs: update PRODUCT doc examples` commit apparently missed the `/api/v1/items` reference, so it's stale despite being a docs commit.
- Keep edits minimal — fix only these facts, no reformatting or rewording.
- Follow-up worth flagging: grep the repo for any other lingering `/api/v1/items` references (README, tests, client SDKs) since the rename commit may have left more stale paths.doc-maintenance--m3
baseline · 0.85
## Documentation Drift Audit **README — Python version range (drifted)** - Commit `chore: drop Python 3.8 support` means "3.8–3.11" is stale. - Edit: "Supports Python 3.9–3.11, requires Redis 6.2+" **SPEC — feature flags path (drifted)** - Commit `feat: read feature flags from env var FEATURE_FLAGS_PATH` makes the hardcoded path wrong (or at least no longer authoritative). - Edit: "Feature flags are read from the path set in `FEATURE_FLAGS_PATH` (default: `/etc/feature_flags.yaml`)." — adjust the default to match the code. **PRODUCT — scheduler interval (drifted)** - Commit `fix: scheduler interval changed to 10 minutes` contradicts "every 5 minutes." - Edit: "The scheduler runs every 10 minutes." All three docs are stale relative to the last three commits; each needs the one-line update above.
with skill · 1.00
## Audit Results **Commits scanned:** 3 — all notable (1 breaking, 1 feature, 1 behavior-changing fix). ### Drift found | Doc | Claim | Status | Trigger commit | |---|---|---|---| | README | "Supports Python 3.8-3.11" | ❌ Drift — 3.8 dropped | `chore: drop Python 3.8 support` | | SPEC | "Flags read from /etc/feature_flags.yaml at startup" | ❌ Drift — path now configurable via `FEATURE_FLAGS_PATH` | `feat: read feature flags from env var FEATURE_FLAGS_PATH` | | PRODUCT | "Scheduler runs every 5 minutes" | ❌ Drift — now 10 minutes | `fix: scheduler interval changed to 10 minutes` | ### PR-ready minimal edits **README.md** ```diff - Supports Python 3.8-3.11, requires Redis 6.2+ + Supports Python 3.9-3.11, requires Redis 6.2+ ``` **doc/SPEC.md** ```diff - Feature flags are read from /etc/feature_flags.yaml at startup. + Feature flags are read from the path specified by the FEATURE_FLAGS_PATH env var at startup (defaults to /etc/feature_flags.yaml). ``` *(If the commit didn't specify a default, drop the parenthetical — verify against the diff.)* **doc/PRODUCT.md** ```diff - The scheduler runs every 5 minutes. + The scheduler runs every 10 minutes. ``` ### Notes - All three fixes are factual one-liners; no rewrites, no new sections. - Follow-up: if the README has a version-support table elsewhere, update it consistently. - After merge, update `.doc-review-cursor` to the new HEAD so the next audit starts from there.
doc-maintenance--m4
baseline · 0.85
Based on the commits vs. current docs, here's what's out of sync:
## Documentation updates needed
1. **README login example is stale** — `acme login` no longer works without a token; the `--token` flag was added (`feat: add --token flag to login`, `docs: update README with token example`).
2. **SPEC references a removed endpoint** — `GET /v1/status` was removed (`refactor: remove /v1/status`) and replaced by `GET /v1/health` (`feat: add /v1/health endpoint`). Update the SPEC to document `/v1/health` and its response shape.
3. **Batch upload behavior changed** — uploads are now chunked (`fix: batch uploads now chunked`). If any docs describe upload limits or payload size, note the chunking behavior; the PRODUCT claim "supports batch uploads" is still accurate but worth a doc note on chunking.
## New README login example
```bash
pip install acme-cli
acme login --token <your-api-token>
```
(Optionally show where to obtain the token, e.g. "Get your token from the Acme dashboard → API settings.")
## Updated SPEC excerpt
```
GET /v1/health
Response: {"status":"ok"}
```
(Adjust the response body if the health endpoint returns a different shape — the commit history doesn't specify it.)with skill · 1.00
## Change summary (from the 5 commits)
- **FEATURE:** `--token` flag added to `acme login` (README already updated via `docs:` commit — verify it matches)
- **FEATURE:** `/v1/health` endpoint added
- **BREAKING:** `/v1/status` removed
- **FIX:** batch uploads now chunked (behavior change worth a one-word note, not a rewrite)
## Required doc updates
**README** — the login quickstart is stale:
- Current: `pip install acme-cli` then `acme login` (no token) — must show the token flag. (A `docs:` commit claims this was updated, so confirm the snippet below actually landed; if not, apply it.)
**SPEC** — two fixes:
1. Remove/replace the `GET /v1/status` → `{"status":"ok"}` entry — that endpoint no longer exists.
2. Add the new endpoint: `GET /v1/health` (document its actual response shape from the code — don't guess it returns `{"status":"ok"}`).
**PRODUCT** — "Acme CLI supports batch uploads" is still true but now incomplete/misleading: uploads are chunked. Update to e.g. "Acme CLI supports batch uploads (large batches are automatically chunked)."
## New README login example
```bash
pip install acme-cli
acme login --token <your-api-token>
```
## Notes
- The `refactor: remove /v1/status` commit is classified **breaking** despite the `refactor:` prefix — check the diff for any other removed public surface while you're in there.
- Follow-up (not this pass): if token auth changed auth flows materially (env vars, token storage), the README may need a short auth note — flag it in the PR description rather than adding a new section now.
- After merging, update `.doc-review-cursor` to the new HEAD.Limitations
- Agent noise: single-model results vary run to run; the confidence interval reflects task-level variance.
- Screen tier (1 trial/arm) is a directional gate, not a verified claim.
- Results are for the SkillzBench Reference Agent on glm-5.3-flash@cf and do not necessarily transfer to other agents or models.
- Task coverage: Skill-matched tasks (generated, public + held-out) only; a skill can shine elsewhere.