understand-diff
Analyzes git diffs or pull requests against a project knowledge graph to explain what changed, which components are affected, and what the risks are.
Pass rate with the skill minus without, same tasks, same model.
- Content Lift forced arm
- +25pp [0, +75]
- Trigger rate
- 100%
- Trials
- 12 · 4 tasks
- Model
- GLM-5.3-flash
- Measured
- 2026-10-15
Content Lift = the skill's instructions injected into the system prompt, so it measures what the content does once loaded, apart from whether the agent chooses to load it. Method →
Embed this score · live badge for your README
The badge shows the real benchmark number and updates itself when we re-bench — including the honest states (“did not trigger”, “not benched yet”). All badge options →
[](https://skillzbench.com/skills/understand-diff)[](https://skillzbench.com/skills/understand-diff)<a href="https://skillzbench.com/skills/understand-diff"><img src="https://skillzbench.com/api/badge/understand-diff.svg" alt="SkillzBench benchmark" height="20"></a>A/B outputs — median trial per arm
Chat harness (v2): the agent's final message without the skill (left) vs with it (right).
understand-diff--m1
What changed
apply_discount in price.py now rounds its result to 2 decimal places before returning, instead of returning the raw float.
# before
return price * (1 - percent / 100)
# after
return round(price * (1 - percent / 100), 2)
Potential issues
-
Precision loss for downstream callers — anything that relied on the exact unrounded value (e.g., further chained discounts, unit tests with exact float assertions, or aggregating many line items) will now see slightly different numbers. Tests comparing with
==on floats are the most likely to break. -
Rounding drift on aggregation — rounding per-item then summing can differ from summing then rounding (penny-level discrepancies). If totals are computed from multiple discounted prices, they may no longer match a total computed from the raw values.
-
Banker's rounding — Python's
round()uses round-half-to-even, soround(2.675, 2)can give2.67due to float representation. If financial correctness matters,decimal.Decimalwith explicitROUND_HALF_UPis safer. -
Type/contract change — the return is still a float, so no signature change, but callers expecting "exact math result" semantics now get a display-rounded value. Worth checking the docstring/comments reflect the new behavior.
Quick checks
- Run the existing tests for
price.pyand anything importing it. - Grep for callers that sum or chain
apply_discountresults. - If this is for money display, consider rounding only at the presentation layer instead.
Overall it's a low-risk, sensible change for currency display — the main risk is exact-value assertions in tests and penny-level aggregation drift.
What changed: In price.py, apply_discount now rounds its result to 2 decimal places before returning:
return round(price * (1 - percent / 100), 2)
Purpose: Likely to avoid floating-point artifacts (e.g., 19.999999999) in displayed/charged prices.
Could it break anything?
- Callers expecting exact math — any code that compares results with
==against unrounded values (e.g.,apply_discount(100, 10) == 90.0still passes, butapply_discount(33.33, 15) == 28.3305would now fail). Tests asserting full-precision values will break. - Accumulation drift — if discounts are applied repeatedly or summed across many items, rounding each step compounds error vs. rounding only at the end. Financial code usually rounds only at final settlement.
- Banker's rounding — Python's
round()uses round-half-to-even:round(2.675, 2)→2.67, not2.68. If you expect "round half up" (common in pricing), results may differ by a cent. For money,decimal.DecimalwithROUND_HALF_UPis safer. - Float precision caveat —
round()on a float can still be off by a cent due to binary representation;Decimalavoids this entirely.
Risk: Low blast radius (one line, pure function), but review all callers/tests of apply_discount for exact-value assertions and multi-step discount chains. If this feeds billing or invoicing, consider switching to Decimal.
understand-diff--m2
This PR filters out inactive users from two database queries:
Changes:
api.py—get_user(user_id)now only returns users whereactive = 1, so fetching a single user by ID will fail/return nothing for deactivated accounts.admin.py—list_users()now only lists active users, hiding deactivated accounts from the admin listing.
Affected components:
- The user lookup API endpoint (single-user fetch)
- The admin user listing view
- Anything downstream that relies on these queries (e.g., clients fetching inactive users' data, or admin tooling that needs to view/reactivate deactivated accounts — that functionality would effectively be lost unless handled elsewhere).
No knowledge graph is available in this environment, so here's a direct analysis of the diff:
What changed
Both changes add an active = 1 filter to user queries — i.e., inactive/deactivated users are now excluded from results.
api.py—get_user(user_id): The single-user lookup now requiresactive = 1. Previously it returned any user by ID; now it returns nothing (or an error/404, depending on handling) for inactive users.admin.py—list_users(): The user listing now only returns active users, hiding deactivated accounts from the list.
Affected components
get_userAPI endpoint — any client fetching a deactivated user's profile will now get an empty/missing result instead of data. Authentication, session resolution, or anything else that callsget_userby ID could be impacted if it needs to look up inactive users (e.g., to show "account deactivated" messages).list_usersadmin view — admins can no longer see or manage inactive accounts through this listing (no way to reactivate users if this is the only listing path).- Downstream consumers of these functions (pagination counts, user caches, exports) may see changed result sets.
Risks to review
- Behavioral change, not just a filter:
get_userreturning nothing for inactive users may break flows that expect a user record (e.g., login, token validation). Consider whether callers need to distinguish "not found" from "inactive." - Admin functionality loss: If admins need to view/reactivate deactivated users, this removes that capability — consider an
?include_inactiveflag or a separate admin query. - No migration or schema change involved; risk is moderate and concentrated in API contract changes.
understand-diff--m3
Summary
The diff moves the cache lookup outside the lock in get_or_set, so reads no longer acquire the lock.
What's affected
- Read path performance: cache hits are now lock-free — likely the intent of the change (reduced contention under high read load).
- Write path:
compute()still runs inside the lock, so expensive computations remain serialized.
Risks
-
Race / duplicate compute (thundering herd): Two threads can both miss the cache, then serialize on the lock and each call
compute(). Previously the double-checked locking pattern guaranteedcompute()ran at most once per key. Now it can run multiple times concurrently — a correctness/perf regression ifcompute()is expensive or non-idempotent (e.g., API calls, DB writes, counters). -
Torn/stale reads without synchronization: In CPython, dict get/set are effectively atomic under the GIL, so this is mostly safe there. But on GIL-free Python (3.13+ free-threading), PyPy, or if
cacheis swapped/rebuilt at runtime, an unsynchronized read could observe a partially-populated state. -
Lost semantic guarantees: If callers relied on
get_or_setas a "compute exactly once" primitive (memoization of side-effectful work), that contract is silently broken.
Recommendations
- Keep double-checked locking: check inside the lock again before calling
compute():
This preserves lock-free fast-path reads while preventing duplicate computes.if key in cache: return cache[key] with lock: if key in cache: # re-check return cache[key] value = compute() cache[key] = value return value - Alternatively use
functools.lru_cache/ a per-key lock if duplicate computes must be fully prevented. - Add a test covering concurrent
get_or_seton the same cold key (assertcomputecall count == 1).
No knowledge graph is available in this environment, so here's a direct analysis of the diff.
What changed
cache.py — get_or_set(key, compute) was modified:
Before: the entire check-then-compute sequence ran inside the lock (double-checked locking, fully serialized).
After: the cache lookup (if key in cache) was moved outside the lock; only compute() and the write happen under the lock.
Affected components
- Everything that calls
get_or_set— any code relying on it for deduplicated, expensive computation (DB queries, API calls, heavy transforms). - Any consumers relying on its atomicity/consistency guarantees.
Risks
- Race condition / duplicate compute (thundering herd): Two threads can both miss the cache, both acquire the lock sequentially, and both run
compute(). The second result overwrites the first. This defeats the function's core purpose under contention. - Duplicate side effects: If
compute()is non-idempotent (writes to DB, sends emails, charges money), it can now execute multiple times concurrently — potentially a correctness/financial bug, not just a perf one. - Stale overwrite: A slower thread's older computed value can overwrite a newer one written after it started.
- Unsynchronized read: Reading
cacheoutside the lock is only safe ifcacheis a dict and the GIL makeskey in cacheatomic; it's fragile and non-portable (e.g., alternative interpreters, or ifcachebecomes a non-thread-safe structure). - Lock no longer protects the invariant that a key's value corresponds to exactly one compute call.
Likely intent & better fix
The change was probably meant to avoid holding the lock during expensive compute() (a real problem — long lock holds serialize all callers). But the correct pattern is double-checked locking:
if key in cache:
return cache[key]
with lock:
if key in cache: # re-check inside lock
return cache[key]
value = compute()
cache[key] = value
return value
This still holds the lock during compute (or use a per-key lock / future-based memoization to avoid that), but guarantees single computation.
Risk level: Medium–High — safe only if compute() is pure, idempotent, cheap, and duplicate execution is acceptable; risky otherwise. Review all get_or_set call sites for side effects before merging.
understand-diff--m4
Summary of Changes
auth.py — create_token now embeds an admin claim in the JWT, sourced from user.get("admin", False).
config.py — Replaces a hardcoded JWT secret with os.environ["JWT_SECRET"].
database.py — get_user now selects only id, role, admin instead of SELECT *.
Affected Components
- Token issuance (auth)
- JWT verification anywhere the payload is decoded (any code reading claims)
- User lookup queries and anything depending on the full user row
- Deployment/config (new required env var)
Risks
-
Hard startup failure:
os.environ["JWT_SECRET"]raisesKeyErrorif unset. Preferos.environ.get("JWT_SECRET")with a fail-fast check and clear error message, and update deployment configs/secrets management. -
Secret rotation invalidates all tokens: Switching from the old hardcoded secret means every existing JWT becomes invalid — all users are logged out at deploy time. Plan a rollout window or support dual-secret verification temporarily.
-
Privilege escalation via
adminclaim: Theadminclaim is now part of the token. Verify that downstream authorization checks read the server-side role, not just the token claim — and that theusers.admincolumn can't be self-modified (e.g., via mass assignment or an unvalidated update endpoint). If any endpoint lets users write toadmin, this change is a serious vulnerability. -
Silent default to
False:user.get("admin", False)masks schema drift. If the column is missing or named differently, admins silently lose privileges. Sincedatabase.pyexplicitly selectsadmin, a missing column will raise a DB error instead — good, but ensure the migration adding/confirming that column ships with this PR. -
Query contract change: Callers of
get_userthat relied on other fields fromSELECT *will now break (likely withKeyError/AttributeError). Audit all call sites. On the plus side, narrowing the query reduces accidental data leakage into tokens. -
Algorithm unchanged (HS256): Fine, but consider confirming key length/entropy requirements for the new secret, and whether RS256/asymmetric signing is warranted if tokens are verified by multiple services.
Recommendations
- Add a migration for the
admincolumn if it doesn't exist. - Add tests: token contains
adminclaim; missing env var fails loudly; existing tokens rejected gracefully. - Grep for all JWT decode sites to confirm how
adminis consumed. - Document the new
JWT_SECRETrequirement in README/deployment docs.
PR Review: JWT admin claim + secret externalization
Note: no project knowledge graph (.ua/knowledge-graph.json) was available to trace callers automatically, so this impact analysis is derived from the diff itself.
What changed
| File | Change |
|---|---|
auth.py | create_token now embeds an admin claim (user.get("admin", False)) in the JWT payload |
config.py | SECRET replaced hardcoded value with os.environ["JWT_SECRET"] |
database.py | get_user narrowed from SELECT * to explicit id, role, admin columns |
Affected components
- Token verification / auth middleware — anything that decodes tokens must now handle (or deliberately ignore) the new
adminclaim. If authorization logic reads claims, it needs updating in the same PR. - All callers of
get_user— this is the biggest blast radius. Any code that used fields fromSELECT *(e.g., `email
- Results are for the SkillzBench reference agent on GLM-5.3-flash. Switch models above — a skill can help one model and not another.
- Screen tier (1 trial per arm) is directional. Verified needs a full-tier run (≥ 8 tasks, ≥ 3 trials per arm) with the CI excluding zero.
- Task coverage: 4 generated tasks for this skill's category; a skill can shine elsewhere.