diff --git a/README.md b/README.md index 704b81a..b1c4f0d 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,12 @@ directly. fix works or freezes the whole app in a way that looks nothing like the cause. - Knowing **which tools to reach for and where each one lies** — including the ones that only exist as a GUI, so you ask for a human instead of silently substituting a weaker method. +- Making a patched build **stay** patched: neutralising version checks, forced-upgrade dialogs and + self-update installers so the work cannot be switched off remotely — and recognising the + hot-update/remote-config channel that can quietly undo it without any version change. +- Separating a **client-side sign-in gate** (patchable) from an **account-scoped resource** (empty + because the server has nothing to answer with), and knowing that forging a session produces a state + worse than being signed out. - Keeping a **long task honest**: a live record, graded conclusions, calibrated timeouts, and bounded waits, so progress is not lost and the same mistake is not made twice. - Avoiding the specific mistakes that produce an APK that builds perfectly and dies at @@ -65,6 +71,15 @@ references/ loaded on demand, one topic each long-task-discipline.md live record, conclusion grading, drift control, timeout and wait calibration, deliverable-form drift, handover ad-removal.md ad taxonomy, wrapper mapping, callback trap, global gates, verification + updates-and-forced-upgrade.md keeping a patched build alive: locating the version check, the + two-layer patch (no-op the routine, neutralise the comparison), what not + to touch (manifest version, installer permission, host blocking), + self-update and hot-update/remote-config channels, verifying that no + version request is issued at all + account-gates.md sign-in walls, forced phone binding, guest mode: telling a client-side + gate (patchable) apart from an account-scoped resource (not), why + fabricating a session is worse than staying signed out, and the + unavoidable session loss after a reinstall signature-derived-keys.md when the app's own signing certificate is used as key material: detection greps, why offline extraction is unreliable, the hardcode-then-verify procedure diff --git a/SKILL.md b/SKILL.md index 49d6a65..37aa723 100644 --- a/SKILL.md +++ b/SKILL.md @@ -66,6 +66,17 @@ Answer these before touching a tool. Every one of them changes the whole plan. mechanism actually fires** (the signal and the tombstone split them apart), and **neutralise by returning, never by making it not return** — a spinning stub freezes the process and produces a symptom that looks nothing like the cause. +12. **Will this build still be usable in a week?** → `references/updates-and-forced-upgrade.md` + If the app has any version check, upgrade prompt, or self-update path, an unpatched build can be + turned off remotely or replaced by the official package. This is one or two edits and it decides + whether the work is durable — do it as part of the build, not as a follow-up. Also check for a + **hot-update / remote-config** channel, which can restore behaviour you removed without any version + change at all. +13. **Does the request touch sign-in or phone binding — "no login required", "skip binding", "guest ok"?** + → `references/account-gates.md`. The whole difficulty here is separating a **client-side gate** + (patchable) from an **account-scoped resource** (the screen is empty because the server has no + account to answer for — not patchable). Classify first; and never fabricate a session to satisfy a + gate, which produces a state worse than being signed out. ## The workflow, end to end @@ -76,6 +87,8 @@ Answer these before touching a tool. Every one of them changes the whole plan. 4. **Decide the patch layer** — client SDK call / client rendering / client data consumption / server contract. See the table in `references/ad-removal.md`. 5. **Patch surgically** — `references/dex-patching.md`. Prefer **dexlib2 method-level rewriting** over whole-tree smali round-trip. Whole-tree round-trip damages R8-optimized dex in ways that only show up at runtime. 6. **Repack and sign** — `references/repack-and-sign.md`. **Do not strip the whole `META-INF/`.** This single mistake destroys otherwise-correct builds. +6b. **Neutralise the update path — before you call the build done.** If the app checks for updates at all, add the two-layer patch (`references/updates-and-forced-upgrade.md`): no-op the update routine's entry, and force the version comparison to its "no update" side. A build that can be switched off or replaced remotely is not a deliverable, and this costs minutes here versus a rebuild later. Do the same for any **remote-config or hot-update** channel that could restore the behaviour you removed. +6c. **Handle account gates only after classifying them** — if the request mentions sign-in or binding, apply `references/account-gates.md` and state plainly which guarded screens become usable and which stay empty because their content is account-scoped. 7. **Verify on device** — `references/environment.md` + `references/verification.md`. Check: launches, the changed behavior actually changed, nothing unrelated broke, and **the app reaches its normal UI with no blocking dialog**. First prove the artifact actually changed on the device -- a package manager reporting success does not prove an interposed confirmation was accepted (P18). Capture continuously for the first ~20 seconds after launch, **and look at the captures** — sampling gaps are how a blocking modal goes unseen (P20), and a burst of images that were never inspected is not evidence. If the accessibility tree is empty, the image is the primary evidence rather than a fallback. 8. **Log what you learned** — if a failure cost you more than thirty minutes, add it to `references/pitfalls.md`. That file is the most valuable artifact in this skill. @@ -109,6 +122,8 @@ Load only what the current step needs. | `references/native-tamper-and-suicide.md` | The process dies on its own (no Java stack, or a native crash that looks like a bug); you are about to neutralise a `kill`/`exit`/`abort` path; or a hardened library's sections/function boundaries look wrong | | `references/toolchain.md` | Choosing or invoking tools, something is not installed, a tool's output smells wrong, or you need to know which tools exist only as a GUI | | `references/ad-removal.md` | Task involves ads, trackers, sponsored cards, splash/interstitial/reward | +| `references/updates-and-forced-upgrade.md` | The patched build must **keep working over time**; the app has any version check, forced-upgrade dialog, self-update installer, or hot-update/resource channel. Load this for essentially every build you intend to ship. | +| `references/account-gates.md` | Task mentions "no login required", "don't force sign-in", "skip phone binding", "guest mode"; or a screen/feature is unreachable signed-out. Also load before promising that an account-scoped screen will show anything | | `references/membership-and-limits.md` | Task involves VIP, subscription, paid content, unlock, "fully cracked" | | `references/server-api.md` | The behavior is decided by a response, or you need to know if a patch can even matter | | `references/dex-patching.md` | Any actual editing of dex/smali, choosing a patch layer, choosing a tool | diff --git a/references/account-gates.md b/references/account-gates.md new file mode 100644 index 0000000..0026ab1 --- /dev/null +++ b/references/account-gates.md @@ -0,0 +1,160 @@ +# Account gates: sign-in walls, forced binding, guest mode + +Load this when the task mentions "no login required", "don't force me to sign in", "skip phone +binding", "guest mode", or when a screen is unreachable without an account — and also **before** +promising that any account- or membership-derived feature will work offline. + +This file is mostly about **telling apart two things that look identical in the UI and are completely +different in reality**: + +- a **client-side gate** — code that refuses to proceed. Patchable. +- an **account-scoped resource** — the screen is empty because the server sent nothing, because the + request was made without a valid identity. **Not** patchable. + +Confusing the two is how a task ends up "done" while the user stares at an empty page. + +## Step 1: classify what you are looking at + +| Symptom | Most likely cause | Can a client patch help? | +|---|---|---| +| A dismissible dialog / bottom sheet asking to sign in, then the app works | client-side gate | **yes** | +| A full-screen wall with no dismiss, app unusable | client-side gate | **yes** | +| Screen renders its frame but the list is empty and a "log in to see" hint is drawn | **account-scoped data** | no — the data does not exist locally | +| A specific action (comment, favourite, follow, download) refuses | client-side gate | usually yes | +| The action succeeds locally but reverts on restart / never reaches the server | **server-side identity** | no | +| "Binding required" before an action | client-side gate | **yes** | +| An entitlement appears then disappears | server-side authority | no | + +**The deciding question is never "is there a login screen?" — it is "does the thing I want exist +without an identity?"** + +Ask it before patching: + +- If the content is **fetched per-account** (subscriptions, favourites, reading history, purchased + items), a patch changes the gate and nothing else. The list stays empty because the server was asked + for *this account's* items and there is no account. +- If the content is **public** and merely hidden behind a prompt (a reader, a catalogue, a settings + page), the patch is real and the feature becomes usable. + +Reading the code tells you which: follow the "sign in" prompt to what it guards, then look at whether +that guarded path issues a request carrying a token, or touches only local state. + +## Step 2: find the convergence point + +Sign-in checks are usually concentrated, because a UI cannot meaningfully enforce login at thirty +call sites. Look for a small number of helpers with names in this family: + +``` +checkLogin* ensureLogin* requireLogin* needLogin* isLogined* isLoggedIn* +checkBind* ensureBind* requireBind* needBind* +showLogin* openLogin* loginSheet* loginDialog* +``` + +and, on the state side, a **login-state field or stream** that every helper reads +(`isLogin`, `loginedStream`, an `Rx`, an auth-state enum, a credential store wrapper). + +**Rule: patch the predicate, not the prompt.** Making the sheet not appear is cosmetic — the caller +still treats the answer as "not signed in" and takes its else-branch, which is usually where the real +feature got skipped. The durable fix is to make the *verdict* come back as "signed in / no sign-in +needed", so the caller takes its happy path. + +Two useful shapes, both better than editing the dialog: + +| Target | Change | Effect | +|---|---|---| +| the gate predicate used by many callers | return "ok / already signed in" unconditionally | every caller proceeds; no new code path | +| the "should I prompt?" decision inside the helper | return "nothing to do" | the prompt never appears and callers proceed | + +Prefer the first when callers branch on the result; prefer the second when the helper is fire-and-forget. + +**Count the callers before choosing** (`scripts/find_refs.py`). A predicate with one caller is a +surgical target; a predicate with forty callers is a behavioural change you must reason about — it will +also release flows you were not asked to touch (settings, purchase, upload). That may be fine, but it is +a decision, not a side effect. + +### Forced binding specifically + +Phone binding is the same pattern with its own helper and usually **two** state checks in series +(bound-phone? and email-or-other-recovery?), because either satisfies the requirement. Patch the +combined verdict, not the first condition — a patch that only satisfies one branch still prompts on the +other. + +Watch for a **"skip"/"later" path the app already has**, and for **which actions the binding actually +gates**. Binding is often required for a subset (password change, deletion, payment) while the rest of +the app is unaffected; releasing it globally changes what the app believes about account safety. + +## Step 3: what NOT to do + +These are the tempting moves that produce a worse artifact than doing nothing: + +- **Do not fabricate a session.** Writing a dummy token, a fake credential blob, or a placeholder + identity into the credential store makes the client *believe* it is signed in. It then calls + identity-scoped endpoints with garbage, and the app enters a state that is **worse than signed out**: + screens show an authentication-failure message, or silently return empty data with no explanation. + A signed-out app that says "please sign in" is better than one that claims a session and fails. + *(If you touch stored credentials at all, keep a backup and expect to have to restore it.)* +- **Do not delete the sign-in UI.** Removing the entry point does not change the verdict; it only makes + the failure unexplainable when a caller does branch on it. Keep the entry point working — it is + normally an explicit requirement ("signing in must still be possible"). +- **Do not release gates that protect server-side operations.** If the action's result is an + entitlement the server must grant (a purchase, a quota, a subscription), releasing the client gate + only lies to the UI. See `references/membership-and-limits.md`. +- **Do not assume "guest can browse" means "guest can do everything".** Establish per feature which of + the three states you are in: public, gate-only, or identity-required. + +## Step 4: the session-loss problem (this one is unavoidable) + +Any change to the signing identity — including a plain reinstall of *your own* build — can invalidate +stored credentials, because they may be sealed with a key held by the platform keystore rather than by +the app. + +**What you observe:** the app opens fine, then one subsystem reports it cannot decrypt or unwrap +something, or the user is silently signed out. + +**What it means:** the app's private data was carried across an uninstall/reinstall (or restored from a +backup), but the encryption key was destroyed with the app. The ciphertext is now unopenable by anyone. + +**What to do:** treat "the user signs in once on the new build" as the expected, documented outcome — not +as a defect to fix. Put it in the delivery notes in one line, together with the fact that a later +upgrade over the *same* signing key can be installed without uninstalling, and therefore will not lose +the session. + +**Do not** chase this into the keystore. Restoring the old key is not part of packaging a patched build, +and any workaround that keeps the *old* ciphertext alive while the key is gone is a dead end. + +Related and worth knowing: an app may keep the same logical account in **two places** — a plain +preference blob and a sealed store. After a reinstall the plain one survives and the sealed one does +not, which shows up as an account that is half-present (identity fields populated, token empty). That +half-state is a useful diagnostic: it tells you the app reads its identity from the sealed store, which +is exactly the thing you must not try to forge. + +## Step 5: verify + +The claim to support is narrow and should be stated narrowly. + +1. **The prompt does not appear** on a cold start and on the screens that used to show it. +2. **The feature behind it actually works**, not merely "no longer blocked": open the reader, the + catalogue, the settings page — whatever the gate guarded — and confirm real content or real function. +3. **The sign-in entry still exists and is usable.** Signing in with a real account must still work; that + is usually part of the request, and it is the thing that proves you patched a gate rather than broke + authentication. +4. **Signed-in behaviour still works** for a real account, so you can tell the difference between + "released the gate" and "broke the auth path". +5. **No fake-session artefact is left behind** — check that you did not write anything into the + credential store or preferences. If you did, remove it and re-verify from a clean install. + +State the result as: *"the gate no longer blocks X, from a signed-out state; **the content behind it is +public / the content behind it is per-account and therefore still empty when signed out**."* That second +clause is what stops a client patch from being mistaken for an account. + +## Reporting template + +``` +Gate: +Class: client-side gate / account-scoped data (not patchable) +Patch: +Signed-out state: +Signed-in state: +Not achievable: +Session note: sign in once after install; upgrade over the same key keeps the session +``` diff --git a/references/updates-and-forced-upgrade.md b/references/updates-and-forced-upgrade.md new file mode 100644 index 0000000..e8a6971 --- /dev/null +++ b/references/updates-and-forced-upgrade.md @@ -0,0 +1,180 @@ +# Updates and forced upgrade + +Load this whenever the deliverable is a patched build that must **keep working over time** — which is +almost always. A build that is correct on the day you ship it and dead a week later has failed, and the +cause is nearly always an update path you did not neutralise. + +This is the file that decides whether a patch is *durable*, and it is cheap: the whole job is usually +one or two edits. + +## Why this is not optional + +Three independent failure modes, all of which present as "the mod stopped working" with no obvious link +to updates: + +1. **Forced upgrade.** The app checks a version endpoint, sees a newer build, and blocks its UI behind a + non-dismissible dialog until the user installs the official (unpatched) package. Your patch is gone. +2. **Self-update install.** The app downloads an APK and hands it to the package installer. Because the + downloaded build is signed differently, the install either fails or — worse — succeeds and replaces + your work. +3. **Silent "compatibility" downgrade.** The server decides the client version is too old and starts + omitting fields or returning an error page. No local code changed, so nothing local looks broken. + +A fourth case is a **resource / hot-update channel**: the app fetches a bundle (JS, a Dart/RN patch, a +config blob, a plugin APK) that can reintroduce the behaviour you removed. That is a separate code path +from version checking — handle it explicitly if the app has one (see §hot-update below). + +## Step 1: locate the check + +There is usually exactly **one** update service. Find it before patching anything else. + +**From the client's own vocabulary** (works on any runtime): + +``` +update / upgrade / new version / check version / force update +版本更新 / 检查更新 / 发现新版本 / 立即更新 / 强制更新 +``` + +Search those in the app's string table and in class names; on heavy runtimes search the string pool, +remembering its encoding (see `framework-runtimes.md` §string encoding — searching only one encoding +misses half your hits, and "nothing found" is usually the encoding, not an absence). + +**Signals that a version check exists even if you cannot read the strings:** + +- A network call early in startup whose response contains a version number, a download URL, and + sometimes a `force` / `must` / `minimum` flag. +- A stored "last update check time" preference (throttling a check implies one exists). +- The app ships the package installer path or requests `REQUEST_INSTALL_PACKAGES`. +- A permissions/manifest entry for an "unknown sources" install intent. + +**Also look for the *coupling*.** Update checks are frequently run from the same startup routine as ad +loading and splash dismissal, and a blocking update dialog often **suspends those siblings** — you may +find logging that literally says so ("update dialog showing, skipping X"). That tells you both where the +check lives and that your patch must not leave the dialog state machine half-finished. + +## Step 2: classify the shape + +| Shape | How it behaves | Where to cut | +|---|---|---| +| **Optional update** | a dismissible prompt with a download button | suppressing the dialog is enough | +| **Forced update** | non-dismissible; blocks the app | must be cut at the decision, not the dialog | +| **Silent version gate** | no dialog; features degrade or the server 400s | client-side is *not* fixable — see §the honest answer | +| **Self-update installer** | downloads and installs an APK | cut the check, and ideally the install path too | +| **Hot-update / resource patch** | fetches a bundle that can restore behaviour | separate route; treat as a data channel | + +Assume **forced** until you have evidence otherwise, because that is the case that costs the user the +most when missed. + +## Step 3: patch the decision, not the dialog + +**Prefer the earliest safe exit over every downstream surface.** Supressing the dialog only hides it; +the download may still start, and a later code path may still act on the "new version" verdict. + +Two layers, in this order: + +**Layer 1 — make the check itself a no-op.** Replace the entry of the update routine so it returns +immediately. Nothing is requested, nothing is compared, no dialog state is created. This is the root fix +and it removes the network call entirely, which also makes verification trivial (Step 4). + +**Layer 2 — neutralise the comparison.** Somewhere the client compares the server version against its +own and branches on the result. Force that branch to the "already current" side. This catches any path +that reaches the comparison without going through the entry you patched (a second caller, a manual +"check for updates" button, a settings screen). + +Layer 2 alone is a valid minimal fix if Layer 1 is awkward; Layer 1 alone is valid if you have confirmed +a single entry. **Doing both is cheap insurance and is what makes the claim "this build cannot be +upgraded into uselessness" defensible.** + +``` +target routine: (...) -> return immediately (no request, no state) +target branch: result -> force the "no update" side +``` + +**Do not** patch the dialog itself as your primary fix, and **do not** patch the comparison to lie in +the *other* direction (claiming an update when there is none) — that produces exactly the blocking +dialog you are trying to avoid. + +### What not to touch + +- **Do not rewrite the app's own version number** in the manifest or in a version string. It is the + cheapest-looking fix and it breaks things you are not looking at: server-side feature negotiation, + analytics, cached asset keys, and the app's own self-comparison. Change the *decision*, not the + identity. +- **Do not strip the installer permission** unless you have checked nothing else uses it. +- **Do not block the update hostname at the transport layer.** That is the same class of mistake as + blocking ad hosts (`pitfalls.md` P5): it fails offline/online transitions, needs a helper on the + machine, and is not a property of the artifact. It is also not required — the check is local code. + +## Step 4: verify + +Update paths are checked once at launch, so verification is a single clean cold start: + +1. **Clear logs, cold start, and grep for the update vocabulary.** Expect **zero** hits. Not "quieter" — + absent. If the routine still runs, the string will still appear. +2. **Watch the screen through startup.** The failure mode is a modal, so a sample you did not look at is + not evidence (`environment.md` §look at the screen). A blocking dialog is unmistakable and it will be + on screen for a long time. +3. **Check the network.** With the check disabled there should be **no request** to the version/update + endpoint at startup. This is the strongest signal: it distinguishes "dialog suppressed" from "check + never happened". +4. **Exercise the manual path** if the app has a "check for updates" entry in settings — that is the + caller Layer 2 exists for. +5. **Confirm what you disabled is only updates.** The update routine sometimes shares a bootstrap with + other startup work. If you no-op'd a routine rather than its decision, verify that splash dismissal, + first-run setup, and the ad/consent flow still complete. + +**Two-sample rule for the durability claim.** If you are asked whether the build *survives* a server-side +version bump, you cannot test that directly. What you can honestly claim is: *(a)* no version request is +issued at startup, and *(b)* the comparison no longer reaches its "newer" branch. State it in those terms +instead of asserting the future. + +## Step 5: hot-update and resource channels + +If the app can fetch a bundle at runtime, version checking is not the only way your change can be undone. + +- **What to look for**: a startup fetch of a signed bundle/zip/config with a version or hash, a local + unpack directory, a dynamic plugin loader, or a JS/Dart/RN patch channel. +- **Why it matters**: even with updates disabled, a remotely fetched config can re-enable a feature, and + a fetched bundle can overwrite patched logic in the parts it owns. +- **Handle it like a data channel, not like an update**: identify what is fetched, whether the patched + behaviour is inside that payload, and cut the consumption point if it is. If the patched logic lives in + the shipped artifact and the fetched payload cannot override it, say so and move on — that is a + finding, not a gap. + +Note the flip side: a **server-issued configuration** is the same category. If the thing you removed was +driven by remote config, the client patch holds only while the server keeps sending that config — see +`references/ad-removal.md` §server-driven and `references/membership-and-limits.md` for the general +rule that *client-side enforcement can be patched, server-side authority cannot*. + +## The honest answer for silent version gates + +If the app degrades because the **server** decides the client is too old — no local dialog, just missing +data or errors on some screens — then there is no client-side fix. The only honest report is: + +- the version the client reports to the server (with evidence of where it is read), +- the endpoint that makes the decision (with the observed response), +- and a clear statement that this is server-side authority. + +Do not paper over it by forcing the version string at random read sites. If you do change a reported +version as an experiment, label it experimental, and remember that a version number is used in more +places than the one you are looking at. + +## Delivery note: always state what was disabled + +Update suppression changes behaviour a user can observe ("it never tells me about new versions"). Put it +in the delivery notes, in one line, together with anything else you neutralised. If the app also has a +legitimate reason to update (security fixes), say that the build will not pick them up automatically. + +## Checklist + +- [ ] Update vocabulary searched in **both** string encodings, in the right layer for the runtime +- [ ] The single update entry point identified (or its absence established, with evidence) +- [ ] Shape classified: optional / forced / silent / installer / hot-update +- [ ] Decision-level patch applied, not just the dialog +- [ ] Comparison branch neutralised as a second layer +- [ ] Manifest version and identity left alone +- [ ] Cold start: **zero** update log lines, no blocking modal, **no version request on the wire** +- [ ] Manual "check for updates" entry exercised +- [ ] Startup siblings (splash dismissal, ads, consent, first-run) still complete +- [ ] Hot-update / remote-config channel considered explicitly +- [ ] Delivery note states what is now disabled diff --git a/scripts/dex_classdiff.py b/scripts/dex_classdiff.py index c2cac1d..243cb22 100644 --- a/scripts/dex_classdiff.py +++ b/scripts/dex_classdiff.py @@ -9,12 +9,13 @@ IncompatibleClassChangeError ("Found interface X, but class was expected") at ru Important limitation: this check compares tables only. It cannot see code-item damage — see references/patch-audit.md for the checks that can. - -Usage: python dex_classdiff.py [filter-substring] """ +import argparse import struct import sys +ACC_INTERFACE = 0x200 + def uleb128(data, off): result = 0 @@ -30,26 +31,22 @@ def uleb128(data, off): def read_strings(data): - off = struct.unpack(' 3 else None - A = class_map(a) - B = class_map(b) - print('A classes=%d B classes=%d' % (len(A), len(B))) + ap = argparse.ArgumentParser( + description='Compare two dex files at class_defs level (names + access_flags). ' + 'Proves a reassembly did not damage interface/class relationships.') + ap.add_argument('a', help='first dex (e.g. the original)') + ap.add_argument('b', help='second dex (e.g. the rebuilt one)') + ap.add_argument('filter', nargs='?', default=None, + help='only report names containing this substring') + ap.add_argument('--max-list', type=int, default=10, + help='how many A-only / B-only names to print (default 10)') + ap.add_argument('--max-flags', type=int, default=25, + help='how many access_flags differences to print (default 25)') + ap.add_argument('--quiet', action='store_true', + help='print only the summary counts') + args = ap.parse_args() + + A = class_map(args.a) + B = class_map(args.b) only_a = sorted(set(A) - set(B)) only_b = sorted(set(B) - set(A)) - print('only_in_A=%d only_in_B=%d' % (len(only_a), len(only_b))) - for n in only_a[:10]: - print(' A-only:', n) - for n in only_b[:10]: - print(' B-only:', n) diff_iface = [] diff_flags = [] for name in sorted(set(A) & set(B)): fa, fb = A[name], B[name] - ia, ib = bool(fa & 0x200), bool(fb & 0x200) - if ia != ib: - diff_iface.append((name, hex(fa), hex(fb))) + if bool(fa & ACC_INTERFACE) != bool(fb & ACC_INTERFACE): + diff_iface.append((name, fa, fb)) elif fa != fb: - diff_flags.append((name, hex(fa), hex(fb))) + diff_flags.append((name, fa, fb)) + + print('A classes=%d B classes=%d' % (len(A), len(B))) + print('only_in_A=%d only_in_B=%d' % (len(only_a), len(only_b))) + if not args.quiet: + for n in only_a[:args.max_list]: + print(' A-only:', n) + for n in only_b[:args.max_list]: + print(' B-only:', n) print('\n*** ACC_INTERFACE mismatch: %d ***' % len(diff_iface)) - for name, fa, fb in (diff_iface if not filt else [x for x in diff_iface if filt in x[0]]): - print(' %-70s A=%s B=%s' % (name, fa, fb)) + if not args.quiet: + for name, fa, fb in diff_iface: + if args.filter and args.filter not in name: + continue + print(' %-70s A=%s B=%s' % (name, hex(fa), hex(fb))) + print('\naccess_flags diff (same interface-ness): %d' % len(diff_flags)) - shown = 0 - for name, fa, fb in diff_flags: - if filt and filt not in name: - continue - if shown < 25: - print(' %-70s A=%s B=%s' % (name, fa, fb)) + if not args.quiet: + shown = 0 + for name, fa, fb in diff_flags: + if args.filter and args.filter not in name: + continue + if shown >= args.max_flags: + print(' ... (%d more suppressed)' % (len(diff_flags) - shown)) + break + print(' %-70s A=%s B=%s' % (name, hex(fa), hex(fb))) shown += 1 + # Exit non-zero when a structural difference exists, so a caller can gate on it. + verdict = (len(only_a) == 0 and len(only_b) == 0 and len(diff_iface) == 0) + print('\nverdict: %s' % ('class tables identical' + if verdict else 'STRUCTURAL DIFFERENCE — inspect above')) + return 0 if verdict else 1 + if __name__ == '__main__': - main() + sys.exit(main()) diff --git a/scripts/dex_strings.py b/scripts/dex_strings.py index b7ac8ef..75c9a52 100644 --- a/scripts/dex_strings.py +++ b/scripts/dex_strings.py @@ -29,6 +29,8 @@ Notes *uses* that feature. Confirm with runtime behavior before acting. * `--urls` output is the fastest way to see an app's whole API surface and its third-party endpoints at once. +* A dex may be a multi-dex set: run this per directory, not per file, to see which + split holds a given marker. """ import argparse import glob @@ -38,7 +40,6 @@ import struct import sys URL_RE = re.compile(rb'https?://[A-Za-z0-9\.\-_:/%\.\?=&~#]{4,200}') -ASCII_RE = re.compile(rb'[\x20-\x7e]{4,200}') def uleb128(data, off): @@ -55,7 +56,7 @@ def uleb128(data, off): def dex_strings(path): - """Yield the dex string table in table order.""" + """Yield the dex string table in table order, as bytes.""" data = open(path, 'rb').read() if data[:4] != b'dex\n': return @@ -74,18 +75,28 @@ def iter_dex(target): def main(): - ap = argparse.ArgumentParser(add_help=True) - ap.add_argument('target') - ap.add_argument('--urls', action='store_true') - ap.add_argument('--strings', action='store_true') - ap.add_argument('--classes', metavar='PREFIX') - ap.add_argument('--find', metavar='REGEX') - ap.add_argument('--per-file', action='store_true') - ap.add_argument('--diff', nargs=2, metavar=('A', 'B')) - ap.add_argument('--min', type=int, default=6) - ap.add_argument('--max', type=int, default=200) + ap = argparse.ArgumentParser( + description='Extract strings / URLs / class descriptors / vendor markers from dex files ' + 'without a decompiler.') + ap.add_argument('target', nargs='?', default=None, + help='a .dex file or a directory containing .dex files (omit when using --diff)') + ap.add_argument('--urls', action='store_true', help='print only http(s) URLs') + ap.add_argument('--strings', action='store_true', help='dump raw string table entries') + ap.add_argument('--classes', metavar='PREFIX', + help='print only entries starting with this prefix (e.g. Lcom/example/app/)') + ap.add_argument('--find', metavar='REGEX', + help='only keep entries matching this regex (applied to the decoded text)') + ap.add_argument('--per-file', action='store_true', + help='with --find: report one line per dex with a hit count, instead of each string') + ap.add_argument('--diff', nargs=2, metavar=('A', 'B'), + help='compare two dex files string tables') + ap.add_argument('--min', type=int, default=6, help='minimum string length (default 6)') + ap.add_argument('--max', type=int, default=200, help='maximum string length (default 200)') + ap.add_argument('--max-print', type=int, default=5000, + help='stop printing after this many lines (default 5000; use 0 for unlimited)') a = ap.parse_args() + # --diff mode needs no target if a.diff: A = set(dex_strings(a.diff[0])) B = set(dex_strings(a.diff[1])) @@ -100,24 +111,32 @@ def main(): print(' +', x.decode('utf-8', 'replace')) return 0 - pat = re.compile(a.find.encode()) if a.find else None + if not a.target: + ap.error('target is required unless --diff is used') + + # Two patterns: the per-file mode scans raw bytes (fast), the per-string mode + # matches decoded text. Keeping both avoids the bytes/str mismatch that silently + # breaks one of the two paths. + pat_text = re.compile(a.find) if a.find else None + pat_bytes = re.compile(a.find.encode('utf-8')) if a.find else None files = iter_dex(a.target) if a.find and a.per_file: for fp in files: data = open(fp, 'rb').read() - hits = len(pat.findall(data)) + hits = len(pat_bytes.findall(data)) if hits: print('%-22s hits=%d' % (os.path.basename(fp), hits)) return 0 + printed = 0 seen = set() for fp in files: for raw in dex_strings(fp): s = raw.decode('utf-8', 'replace') if not (a.min <= len(s) <= a.max): continue - if pat and not pat.search(s): + if pat_text and not pat_text.search(s): continue if a.classes and not s.startswith(a.classes): continue @@ -126,18 +145,20 @@ def main(): if s in seen: continue seen.add(s) - if a.urls: - print(s) - elif a.classes: + if a.max_print and printed >= a.max_print: + print('[note] output cap reached (%d). Use --max-print 0 to lift.' % a.max_print) + return 0 + if a.classes: print('%-18s %s' % (os.path.basename(fp), s)) else: - print('%s' % s) + print(s) + printed += 1 if a.urls and not seen: - # fall back to raw byte scan (covers URLs stored outside the string table) + # fall back to a raw byte scan: covers URLs stored outside the string table print('[note] no URLs in string tables; raw scan:') for fp in files: - for m in set(URL_RE.findall(open(fp, 'rb').read())): + for m in sorted(set(URL_RE.findall(open(fp, 'rb').read()))): print(os.path.basename(fp), m.decode('utf-8', 'replace')) return 0