Add deliverable-form discipline, execution identity, and four reusable scripts
Generalized from a long hardened-target engagement. Additions target the failure
modes that cost the most time and are not yet covered; nothing here is sample-specific.
New discipline
- SKILL.md: a third override rule -- deliver in the environment the request requires.
A root-only or live-instrumentation result can look like success while failing the
actual requirement (unrooted, installable, self-contained). Also adds two classify
questions: what form the deliverable must take, and which architecture/library is
actually executing.
- long-task-discipline.md: "the most expensive drift" -- a constraint table (privilege,
modification form, ABI, network, persistence, distribution), the checkpoint question,
and how to report a privileged workaround as a fallback rather than a deliverable.
Plus "bound every wait": timeouts on every command, budgets calibrated from
measurement so slow and hung are distinguishable, and sampling an observable you can
inspect instead of sleeping blind.
- environment.md: look at the screen instead of driving blind; which of the two screen
evidence kinds is trustworthy (an empty accessibility tree is common for
runtime-rendered UI, where the image is primary rather than a fallback); emulator
console control and adb recovery; preflight before every experiment block.
- native-and-so.md: what the device claims vs what the package manager chose vs what is
actually mapped; translation layers and their consequences for hooks and native
checks; fat-APK ABI selection.
- runtime-data.md: when the app rewrites your edit (server refresh vs local
regeneration), why chmod/chown cannot work because the app deletes and recreates the
file, and the immutable-attribute fix; decoding a value that only looks encrypted.
- framework-runtimes.md: string encoding traps (a UTF-8 scan of a UTF-16 snapshot
returns nothing and reads as "the strings were stripped"), and non-ASCII obfuscated
identifiers -- resolve by shape, not by name.
- pitfalls.md: P23 the fix lives in a file the app rewrites; P24 the artifact changed
but the wrong one is executing; P25 "search found nothing" treated as "not there".
New scripts (parameterized, path-agnostic, all with explicit timeouts)
- preflight.py environment check: device, root, ABI/translation, clock skew, leftover
proxy/forwards, dead device server, chosen ABI, free space
- lib_map.py what is actually mapped into a live process: path, base, architecture
from the ELF header, and whether it came from the APK or was
materialized at runtime
- blob_decode.py search (don't guess) the framing of a stored value, then re-encode the
edited payload with the same parameters
- snap.py bounded burst screenshots + control-tree capture, with a stall detector
and an explicit verdict on whether the tree is usable
All four were exercised against a live device while writing: preflight correctly
flags multi-device ambiguity and detected the ARM translation layer; lib_map read real
ELF headers and identified the translation shim; blob_decode round-tripped a
rotation+deflate+base64 blob at realistic size in ~0.2s; snap detected a stalled screen
across three hashes.
This commit is contained in:
inliver233
@@ -13,13 +13,21 @@ directly.
|
||||
|
||||
- Deciding **fast** whether a request is even achievable client-side, instead of
|
||||
burning hours on a paywall that is enforced by a server.
|
||||
- Deciding **what form the deliverable must take** before any work starts — an
|
||||
unrooted, self-contained artifact is a different problem from "make it work on this
|
||||
machine", and confusing the two is the most expensive drift in this domain.
|
||||
- Choosing the **safest patch layer** for a given change, and avoiding the layers that
|
||||
break the app.
|
||||
- Separating **your own mistakes from the app's or the server's problems** — a
|
||||
feature-scoped failure (login, registration, payment) is often a TLS/certificate issue on
|
||||
one code path, not a consequence of the patch you just built.
|
||||
one code path, not a consequence of the patch you just built. Device state, a dead device
|
||||
server and clock drift masquerade the same way.
|
||||
- Establishing **which architecture and which library are actually executing**, rather
|
||||
than trusting what the manifest ships or what the device claims.
|
||||
- Working through **packed/hardened targets**: identifying the packer, unpacking, and turning a
|
||||
memory dump back into a patched, installable APK.
|
||||
- Keeping a **long task honest**: a live record, graded conclusions, calibrated timeouts, and
|
||||
bounded waits, so progress is not lost and the same mistake is not made twice.
|
||||
- Avoiding the specific mistakes that produce an APK that builds perfectly and dies at
|
||||
runtime.
|
||||
|
||||
@@ -31,10 +39,13 @@ references/ loaded on demand, one topic each
|
||||
recon.md identify packer, SDKs, code location, tamper checks; unpacking
|
||||
packers.md hardened targets: rejection signals, measuring the validation
|
||||
boundary with single-variable tests, choosing a native host
|
||||
framework-runtimes.md Flutter / React Native / Unity: which layer owns the UI
|
||||
framework-runtimes.md Flutter / React Native / Unity: which layer owns the UI, and how to
|
||||
find logic when there are no symbols (string encoding traps)
|
||||
native-and-so.md .so hosts, DT_NEEDED vs JNI_OnLoad, relocation limits,
|
||||
relocation-free bootstrapping, replacing Java methods natively
|
||||
long-task-discipline.md live record, conclusion grading, drift control, handover
|
||||
relocation-free bootstrapping, replacing Java methods natively,
|
||||
and which ABI/library is *actually loaded and executing*
|
||||
long-task-discipline.md live record, conclusion grading, drift control, timeout and
|
||||
wait calibration, deliverable-form drift, handover
|
||||
ad-removal.md ad taxonomy, wrapper mapping, callback trap, global gates, verification
|
||||
membership-and-limits.md server vs client authority; what is and is not patchable
|
||||
server-api.md probe an app's API; prove who owns the gate
|
||||
@@ -42,9 +53,11 @@ references/ loaded on demand, one topic each
|
||||
third-party-builds.md auditing a "cracked"/"modded" APK before trusting it
|
||||
dex-patching.md patch-layer table + dexlib2 technique in depth
|
||||
repack-and-sign.md repack rules, unpack-and-repack, signing, post-install hazards
|
||||
runtime-data.md DataStore / SharedPreferences / SQLite / protobuf
|
||||
runtime-data.md DataStore / SharedPreferences / SQLite / protobuf; when the app
|
||||
rewrites your edit, and decoding a value that looks encrypted
|
||||
dynamic-frida.md Frida setup, version pinning, the four-layer probe, hook strategy
|
||||
environment.md device/emulator setup, ADB, UI automation, offline devices, log signals
|
||||
environment.md device/emulator setup, root, ADB, offline devices, log signals,
|
||||
emulator console control and recovery, preflight, look-at-the-screen
|
||||
verification.md the claim ladder; what "done" means
|
||||
pitfalls.md the failure catalogue -- read before building
|
||||
scripts/ parameterized, path-agnostic
|
||||
@@ -65,6 +78,15 @@ scripts/ parameterized, path-agnostic
|
||||
frida_probe.js four-layer runtime probe (app net layer + OkHttp + java.net + exceptions)
|
||||
run_probe.py inject the probe, stream it to a log file, stay resident
|
||||
tls_check.py strict certificate check for one or more hosts
|
||||
preflight.py environment check before every experiment block (device, root,
|
||||
ABI/translation, clock skew, leftover proxy/forwards, dead server)
|
||||
lib_map.py what is *actually mapped* into a live process: per-library path,
|
||||
base, architecture, and whether it came from the APK or was
|
||||
materialized at runtime
|
||||
blob_decode.py search, don't guess, the framing of a stored value
|
||||
(base64/hex x rotation x deflate); re-encode the edited payload
|
||||
snap.py bounded burst screenshots + control-tree capture with a stall
|
||||
detector, and a verdict on whether the tree is usable at all
|
||||
```
|
||||
|
||||
## Install
|
||||
|
||||
@@ -7,10 +7,11 @@ description: Reverse engineer, debloat, de-ad, patch, or re-sign Android APKs, a
|
||||
|
||||
Goal of this skill: get to a **verified, installable, still-working artifact** fast, and avoid the whole class of mistakes that silently destroy an APK.
|
||||
|
||||
Two rules override everything else in this skill:
|
||||
Three rules override everything else in this skill:
|
||||
|
||||
1. **Never ship an unverified APK.** "It assembles" is not "it works". Install it, launch it, exercise the feature you changed, on a real device or a close emulator.
|
||||
2. **Know which layer owns the behavior before you patch.** Ads, paywalls, and feature gates live in different places. Patching the wrong layer either does nothing or breaks the app. Classify first (see step 2 below), patch second.
|
||||
1. **Deliver it in the environment the request actually requires.** "It works" is not the goal; "it works under the stated constraint" is. A result obtained with root, or with live instrumentation, or with a host proxy, frequently does **not** satisfy a request for an installable artifact that works on a normal device — and it is easy to present that result as a finished one. Write the constraint down as a testable sentence in the first minutes and re-read it at every checkpoint. If you cannot meet it, say so and label any privileged workaround as a fallback, never as the deliverable. Full treatment: `references/long-task-discipline.md` §the most expensive drift.
|
||||
2. **Never ship an unverified APK.** "It assembles" is not "it works". Install it, launch it, exercise the feature you changed, on a real device or a close emulator.
|
||||
3. **Know which layer owns the behavior before you patch.** Ads, paywalls, and feature gates live in different places. Patching the wrong layer either does nothing or breaks the app. Classify first (see step 2 below), patch second.
|
||||
|
||||
## Start here: classify the target in seven questions
|
||||
|
||||
@@ -27,13 +28,23 @@ Answer these before touching a tool. Every one of them changes the whole plan.
|
||||
3. **Is the app's own code in plain dex, or moved to native/Flutter/Unity?**
|
||||
Plain dex → you can patch. Flutter (`libflutter.so` + `libapp.so`) / Unity (`libil2cpp.so`) / pure native → different toolchain entirely. See `references/recon.md` §code-location and `references/framework-runtimes.md`.
|
||||
**Runtime check (cheap -- do it before committing to a layer):** hook the obvious Java classes for the UI you care about, then reproduce that UI. If those hooks fire, the behavior is Java-owned. If they fire **zero times** while the UI is plainly on screen, the behavior is drawn by the runtime or by native code, and a dex-only plan will stall. Do not keep hunting in dex after a zero-hit probe -- that is the most expensive wrong turn in this skill's history.
|
||||
4. **Does the app verify its own signature, or does the server?**
|
||||
4. **What must the deliverable be able to do?** Write the answer as a testable sentence before
|
||||
planning anything, then re-read it at every checkpoint. This is the drift guard, and the drift it
|
||||
guards against is the most expensive one in this skill: a runtime-only result (a data edit, a live
|
||||
hook, a blocked hostname, a host proxy) can look like success while failing the actual requirement.
|
||||
The axes that decide it: **privilege** (unrooted?), **modification form** (a rebuilt, installable
|
||||
artifact, or is live instrumentation acceptable?), **ABI/device class**, **network** (must it work
|
||||
online?), **persistence** (survives restart / upgrade / fresh install?), **distribution** (must the
|
||||
shipped file be self-contained?). → `references/long-task-discipline.md` §the most expensive drift.
|
||||
5. **Does the app verify its own signature, or does the server?**
|
||||
App-side → you must bypass it. Server-side → re-signing silently breaks the app later. See `references/repack-and-sign.md` and `references/server-api.md`.
|
||||
5. **What is your device situation?** → `references/environment.md`
|
||||
Rooted real device (best), emulator with root, or no device (static only). Also: this determines whether Frida is usable.
|
||||
6. **Is one *specific feature* failing at runtime — login, registration, payment, an API-backed screen — while the rest of the app works?**
|
||||
6. **What is your device situation?** → `references/environment.md`
|
||||
Rooted real device (best), emulator with root, or no device (static only). Also: this determines whether Frida is usable. **Run `scripts/preflight.py` before your first experiment**, and again whenever a failure surprises you — device state, a dead device server, a leftover proxy, and clock drift all masquerade as a broken patch (`pitfalls.md` P9).
|
||||
7. **Which architecture is actually executing?** → `references/native-and-so.md` §Cross-architecture
|
||||
`getprop` reports what the device claims and `primaryCpuAbi` reports what the package manager chose — neither is what is running. Only the live mapping is ground truth (`scripts/lib_map.py`). If the library you meant to patch is not mapped, a translator is in play, or the ABI differs from your assumption, that changes the plan more than any patch will.
|
||||
8. **Is one *specific feature* failing at runtime — login, registration, payment, an API-backed screen — while the rest of the app works?**
|
||||
→ `references/tls-and-cert.md`. A feature-scoped network failure is very often a **TLS/certificate problem on one code path**, not a consequence of your patch. The app can even carry two independent trust chains, so "other requests work" proves nothing. Rule this out in minutes before hunting for a signature check.
|
||||
7. **Was the input a build you did not produce** (a "cracked"/"modded" APK circulating online)?
|
||||
9. **Was the input a build you did not produce** (a "cracked"/"modded" APK circulating online)?
|
||||
→ `references/third-party-builds.md`. Audit it before adopting it: such builds are frequently re-protected (sometimes with *more* layers than the original) and may carry injected components or endpoints. Never use one as a patching workbench.
|
||||
|
||||
**Long-task rule:** if this is likely to run long, open `references/long-task-discipline.md` now
|
||||
@@ -43,13 +54,13 @@ Answer these before touching a tool. Every one of them changes the whole plan.
|
||||
|
||||
## The workflow, end to end
|
||||
|
||||
1. **Recon** — `references/recon.md`. Manifest, package name, version, ABI, dex count, packer, embedded SDKs, where the app's own code lives. Ten minutes here saves hours. **If it is packed, unpack before anything else** (`references/recon.md` §unpacking): you cannot patch code you cannot read, the encrypted payload lengths tell you which dumped dex is the original, and a memory dump must be de-duplicated by hash and structurally validated before any of it is trusted.
|
||||
1. **Preflight, then Recon** — `scripts/preflight.py` before anything else if a device is involved (it takes seconds and prevents a whole class of false conclusions), then `references/recon.md`. Manifest, package name, version, ABI, dex count, packer, embedded SDKs, where the app's own code lives. Ten minutes here saves hours. **If it is packed, unpack before anything else** (`references/recon.md` §unpacking): you cannot patch code you cannot read, the encrypted payload lengths tell you which dumped dex is the original, and a memory dump must be de-duplicated by hash and structurally validated before any of it is trusted.
|
||||
2. **Extract strings and endpoints** — build a picture of the app's API surface and SDK inventory from the dex string tables. No decompiler needed for this, and it is fast. Scripts: `scripts/dex_strings.py`.
|
||||
3. **Trace to the owning class** — find the class that wraps the behavior (the app almost always wraps third-party SDKs in one helper). Reverse-lookup instructions: `references/dex-patching.md` §finding-the-call-site.
|
||||
4. **Decide the patch layer** — client SDK call / client rendering / client data consumption / server contract. See the table in `references/ad-removal.md`.
|
||||
5. **Patch surgically** — `references/dex-patching.md`. Prefer **dexlib2 method-level rewriting** over whole-tree smali round-trip. Whole-tree round-trip damages R8-optimized dex in ways that only show up at runtime.
|
||||
6. **Repack and sign** — `references/repack-and-sign.md`. **Do not strip the whole `META-INF/`.** This single mistake destroys otherwise-correct builds.
|
||||
7. **Verify on device** — `references/environment.md` + `references/verification.md`. Check: launches, the changed behavior actually changed, nothing unrelated broke, and **the app reaches its normal UI with no blocking dialog**. First prove the artifact actually changed on the device -- a package manager reporting success does not prove an interposed confirmation was accepted (P18). Capture continuously for the first ~20 seconds after launch; sampling gaps are how a blocking modal goes unseen (P20).
|
||||
7. **Verify on device** — `references/environment.md` + `references/verification.md`. Check: launches, the changed behavior actually changed, nothing unrelated broke, and **the app reaches its normal UI with no blocking dialog**. First prove the artifact actually changed on the device -- a package manager reporting success does not prove an interposed confirmation was accepted (P18). Capture continuously for the first ~20 seconds after launch, **and look at the captures** — sampling gaps are how a blocking modal goes unseen (P20), and a burst of images that were never inspected is not evidence. If the accessibility tree is empty, the image is the primary evidence rather than a fallback.
|
||||
8. **Log what you learned** — if a failure cost you more than thirty minutes, add it to `references/pitfalls.md`. That file is the most valuable artifact in this skill.
|
||||
|
||||
## Non-negotiable constraints
|
||||
@@ -59,6 +70,8 @@ Answer these before touching a tool. Every one of them changes the whole plan.
|
||||
- **Verify structure after every dex edit.** `scripts/dex_classdiff.py` must report zero differences in class set and access flags for classes you did not intend to change.
|
||||
- **Do not patch a method that is widely shared.** Before patching any helper, count its callers (`scripts/find_refs.py`). A `Long.valueOf` wrapper with 30 callers is not an ad-specific hook.
|
||||
- **Do not make an API fail to suppress a UI element.** A 404/400 on an endpoint that other features depend on takes the whole screen down with it. Suppress at the data-consumption or render layer instead.
|
||||
- **Look before you conclude — and look while you wait.** Execute, capture, and **inspect**; do not drive and sleep blind. A screen that is actually looked at answers in one step what coordinate-guessing cannot answer in five: the layout moved, a different dialog is up, a countdown is frozen, the text on screen says exactly why. Where the thing you are waiting on is visible, a sample you can inspect beats a duration you hoped was right, and byte-identical samples mean nothing is going to change. `scripts/snap.py`; `references/environment.md` §look at the screen.
|
||||
- **Put a timeout on every command, and calibrate it from measurement.** An unbounded call turns a stall into "the task stopped making progress", which is indistinguishable from slow work and costs hours silently. Time the operation once, record it, then derive the bound from it — that is what makes slow and hung distinguishable. A deadline that passes is a measurement, not a verdict. `references/long-task-discipline.md` §bound every wait.
|
||||
- **Every claim needs evidence.** "Probably", "should be", "in theory" are not findings. Either you observed it, or you label it unverified.
|
||||
- **"Done" means the user-visible outcome**, not an internal signal. A blocking dialog still on screen means the task is not done, however many errors disappeared from the log. Absence of a log line is absence of evidence, never evidence of success.
|
||||
- **Do not discard a route on compound evidence.** If a failure followed two simultaneous changes, the attribution is a hypothesis, not a finding. Re-run it single-variable before writing the route off -- mis-attributed failures have removed viable approaches for a long time.
|
||||
@@ -74,19 +87,19 @@ Load only what the current step needs.
|
||||
| `references/recon.md` | Starting any new sample; identifying packer, SDKs, code location, ABI |
|
||||
| `references/packers.md` | The app is packed/hardened, or an edit makes it die before your code runs. Also load before discarding any route as "blocked by the shell" |
|
||||
| `references/framework-runtimes.md` | The UI is not native (Flutter / React Native / Unity / Cordova), or Java-layer hooks fire zero times while the UI clearly works |
|
||||
| `references/native-and-so.md` | Patching in a `.so`, needing code to run before the app's own code, or hand-built native payloads that crash inside the linker |
|
||||
| `references/native-and-so.md` | Patching in a `.so`, needing code to run before the app's own code, hand-built native payloads that crash inside the linker, or **deciding which library/ABI is actually loaded and executing** |
|
||||
| `references/ad-removal.md` | Task involves ads, trackers, sponsored cards, splash/interstitial/reward |
|
||||
| `references/membership-and-limits.md` | Task involves VIP, subscription, paid content, unlock, "fully cracked" |
|
||||
| `references/server-api.md` | The behavior is decided by a response, or you need to know if a patch can even matter |
|
||||
| `references/dex-patching.md` | Any actual editing of dex/smali, choosing a patch layer, choosing a tool |
|
||||
| `references/repack-and-sign.md` | Rebuilding, signing, installing, or a repacked app misbehaves |
|
||||
| `references/runtime-data.md` | Local state matters: DataStore, SharedPreferences, SQLite, protobuf caches, tokens |
|
||||
| `references/runtime-data.md` | Local state matters: DataStore, SharedPreferences, SQLite, protobuf caches, tokens — **or your data edit keeps being reverted, or a stored value looks encrypted** |
|
||||
| `references/dynamic-frida.md` | Frida setup, hooking strategy, tracing caller chains, finding the real call site |
|
||||
| `references/environment.md` | Device/emulator setup, root, ADB, networking, offline devices |
|
||||
| `references/environment.md` | Device/emulator setup, root, ADB, networking, offline devices, emulator console control and recovery, **the preflight check to run before every experiment block** |
|
||||
| `references/verification.md` | Defining what "done" means; building the evidence chain |
|
||||
| `references/tls-and-cert.md` | One feature fails at runtime (login, registration, payment, an API-backed screen) while the rest of the app works |
|
||||
| `references/third-party-builds.md` | The input is a "cracked"/"modded" build you did not produce — audit it before trusting it |
|
||||
| `references/long-task-discipline.md` | The task will run long, or you are resuming one. Live record, conclusion grading, drift checkpoints, handover |
|
||||
| `references/long-task-discipline.md` | The task will run long, or you are resuming one. Live record, conclusion grading, drift checkpoints, **deliverable-form drift (rooted-only vs shippable)**, bound-your-waits, handover |
|
||||
| `references/pitfalls.md` | Always worth a skim before building. This is the failure catalogue. |
|
||||
|
||||
## Script index
|
||||
@@ -111,3 +124,7 @@ All scripts are parameterized and path-agnostic; pass paths explicitly. Run `--h
|
||||
| `scripts/frida_probe.js` | Four-layer runtime probe: app network layer + OkHttp + java.net + swallowed exception messages |
|
||||
| `scripts/run_probe.py` | Inject a probe, stream it to a timestamped log file, stay resident while you operate the app |
|
||||
| `scripts/tls_check.py` | Strict certificate check for one or more hosts (expired / wrong host / untrusted CA) |
|
||||
| `scripts/preflight.py` | Read-only environment check before every experiment block: device, root, ABI/translation, clock skew, leftover proxy/forwards, dead device server. Run this before blaming a patch. |
|
||||
| `scripts/lib_map.py` | What is **actually mapped** into a live process: per-library path, base, architecture (`ELF e_machine`), and classification (system / from-APK / runtime-materialized). Answers "which library and which ABI is really executing". |
|
||||
| `scripts/blob_decode.py` | Decode an opaque stored value by searching the parameter space (base64/base64url/hex × rotation × deflate/zlib/gzip) instead of guessing, then re-encode an edited payload with the same parameters. |
|
||||
| `scripts/snap.py` | Bounded burst screenshot + control-tree capture, with a stall detector and an explicit verdict on whether the accessibility tree is usable at all. Use it so you *look* at the screen instead of driving blind. |
|
||||
|
||||
+130
-6
@@ -65,11 +65,86 @@ Notes:
|
||||
- Some SDKs bypass the system proxy entirely (many ad SDKs do). Proxy logs therefore **undercount** traffic — do not conclude "no ad traffic" from proxy logs alone; confirm with a runtime DNS hook.
|
||||
- Clean up when done: `settings put global http_proxy :0`.
|
||||
|
||||
## Preflight — run this before every experiment block
|
||||
|
||||
`scripts/preflight.py` checks, read-only, everything that silently fakes a failure: device
|
||||
reachability, whether more than one device is attached without an explicit serial, root, the
|
||||
runtime translation layer, device/host clock drift, a leftover device-wide proxy setting, stale
|
||||
port forwards, a dead device server, the ABI the package manager actually chose, and the free
|
||||
space left for installs.
|
||||
|
||||
```bash
|
||||
python scripts/preflight.py --pkg <app.package> --expect-root
|
||||
python scripts/preflight.py --cleanup # also clears a leftover proxy and stale forwards
|
||||
```
|
||||
|
||||
**The rule it exists to enforce:** *do not attribute a failure to your patch while preflight is
|
||||
dirty.* A surprising fraction of "my change broke it" is a device that was already in a bad state.
|
||||
Costing thirty seconds here is cheaper than costing several rounds to a wrong conclusion, and the
|
||||
wrong conclusion is the one that gets written down and trusted later.
|
||||
|
||||
## Emulator notes
|
||||
|
||||
- Verify the app can install at all. INSTALL failures on emulators are common (ABI, min SDK, vendor checks) and are usually not related to your patch.
|
||||
- Snapshot/rollback is the main advantage — use it to A/B two builds quickly.
|
||||
- Never quote emulator behavior as proof for a device-only question (and vice versa).
|
||||
Emulators are the right place to iterate and the wrong place to conclude.
|
||||
|
||||
- **Verify the app can install at all.** INSTALL failures on emulators are common (ABI, min SDK,
|
||||
vendor checks, a stale copy with a different signature) and are usually **not** related to your
|
||||
patch. Get a clean install of the *unmodified* build working before you change anything.
|
||||
- **Snapshot/rollback is the main advantage** — use it to A/B two builds quickly, and to get back
|
||||
to a known-good state after a destructive experiment.
|
||||
- **Never quote emulator behavior as proof for a device-only question** (and vice versa). An
|
||||
emulator that runs the app is not evidence about a physical ARM device.
|
||||
- **Most vendors ship a console binary that is far more reliable than the GUI.** Learn yours early;
|
||||
it is what you will need when the GUI is unresponsive and adb is already down.
|
||||
|
||||
| Vendor | Typical console binary | Useful verbs |
|
||||
|---|---|---|
|
||||
| LDPlayer | `dnconsole.exe` (next to the player exe) | `list2`, `launch --index N`, `reboot --index N`, `quitall` |
|
||||
| MuMu | `MuMuManager.exe` | `info -v all`, `control -v N launch`, `control -v N shutdown` |
|
||||
| AVD / Android Emulator | `emulator.exe` | `-list-avds`, `@<avd>`, `-no-window` |
|
||||
| Genymotion | `gmtool` | `admin list`, `admin start` |
|
||||
|
||||
- **Recovery order when `adb devices` goes empty.** Do these in order; the first two are the ones
|
||||
people skip:
|
||||
|
||||
1. `adb kill-server && adb start-server` — clears a wedged host daemon.
|
||||
2. Re-run `adb devices`. If still empty, check whether the **emulator process is actually alive**
|
||||
(`tasklist` / `ps`). A running process with **no listening port** means the VM never finished
|
||||
booting its adb bridge — restarting the *device* is the only fix.
|
||||
3. Restart the instance through the vendor console (`reboot --index N`). Restarting via the
|
||||
player binary's "launch" verb frequently leaves you with a process and still no adb.
|
||||
4. Only then consider that something is wrong with the host.
|
||||
|
||||
Symptom worth memorising: **the emulator process exists but nothing is listening on the adb
|
||||
port.** Distinguishing "not running" from "running but not booted" is what makes step 2 worth
|
||||
doing — they have different fixes and only one of them needs the GUI.
|
||||
|
||||
- **Root is a per-vendor setting, not a given.** Most emulators expose it as a toggle in their
|
||||
settings ("root permission" / "ROOT 权限"); some ship a separate rooted image. Always verify
|
||||
rather than assume:
|
||||
|
||||
```bash
|
||||
adb -s <serial> shell "su -c id" # want: uid=0(root)
|
||||
```
|
||||
|
||||
A rooted emulator is usually the fastest environment for everything in this skill, which makes it
|
||||
easy to forget that it is also the *least representative* one.
|
||||
|
||||
- **Running two instances.** Different serials make two emulators genuinely disjoint, which is
|
||||
useful for holding a clean control (original build, untouched state) alongside your workbench.
|
||||
Keep them labelled, always pass `--serial`, and **do not run the same experiment on both** — the
|
||||
value of a second device is independence, and duplicating work destroys it. Operate one at a
|
||||
time; the other is a reference point, not extra throughput.
|
||||
|
||||
- **Apps can detect the emulator** and change behaviour or refuse to run. If the app behaves
|
||||
differently here than on hardware, that is a finding about the emulator, not about the app. Note
|
||||
it and move the question to a real device.
|
||||
|
||||
## Which architecture is actually executing
|
||||
|
||||
Device selection and architecture are the same question. `getprop ro.product.cpu.abi` reports what
|
||||
the device claims; it does **not** report what is executing. Read `native-and-so.md` §Cross-architecture
|
||||
before you choose which library to patch, and `scripts/lib_map.py` to see the live truth.
|
||||
|
||||
## Install / reinstall
|
||||
|
||||
@@ -110,15 +185,64 @@ If the control still does not react, bypass the UI path entirely — `am start -
|
||||
|
||||
`input text` additionally mangles or drops non-ASCII input. For CJK text, either install an ADB-driven helper IME or set the field from a runtime call rather than typing.
|
||||
|
||||
### Screen evidence without pixels
|
||||
### Look at the screen — do not drive and wait blind
|
||||
|
||||
`screencap` returns a 0-byte file on some ROMs. First try writing on-device and pulling:
|
||||
The most expensive habit in device work is: tap a coordinate, sleep, tap again, sleep, conclude
|
||||
something about the app. A screen that is *looked at* answers in one step what coordinate-guessing
|
||||
cannot answer in five — the layout shifted, a different dialog came up, a countdown is frozen, the
|
||||
button is disabled, the text on screen says exactly why.
|
||||
|
||||
**Treat a look as a routine step, not a debugging last resort.** Capture:
|
||||
|
||||
- **immediately before** anything time-dependent, so you know the starting state;
|
||||
- **during** a wait, at intervals — state changes are the information, and a single sample at the end
|
||||
cannot distinguish "it progressed" from "it never moved";
|
||||
- **at every decision point**, before choosing the next action;
|
||||
- **on any surprise**, before forming a theory about it.
|
||||
|
||||
`scripts/snap.py` does this with sane bounds, and tells you which kind of evidence you actually got:
|
||||
|
||||
```bash
|
||||
python scripts/snap.py --out shots --tag before
|
||||
python scripts/snap.py --out shots --tag waiting --count 6 --interval 3
|
||||
```
|
||||
|
||||
**Use the stall detector.** If consecutive samples are byte-identical, nothing is happening and more
|
||||
waiting cannot help. Stop, and go find out why — that is a different investigation from waiting
|
||||
longer.
|
||||
|
||||
### Two kinds of screen evidence, and which one is trustworthy here
|
||||
|
||||
| Evidence | What it gives you | When it is the right one |
|
||||
|---|---|---|
|
||||
| **the image** | exactly what is rendered: layout, which dialog, disabled states, drawn text | always available; the **only** evidence for runtime-drawn UI |
|
||||
| **the control tree** (`uiautomator dump`) | precise `bounds`, exact text, diffable | only when it actually has content |
|
||||
|
||||
The tree is more convenient *when it exists* — it gives you tap coordinates and text you can diff. But
|
||||
**verify it has content before planning around it**:
|
||||
|
||||
```bash
|
||||
adb -s <serial> shell "su -c 'uiautomator dump /sdcard/ui.xml'"
|
||||
adb -s <serial> pull /sdcard/ui.xml
|
||||
grep -c '<node' ui.xml # 0 nodes -> the tree is useless for this screen
|
||||
```
|
||||
|
||||
**A runtime-rendered UI frequently exposes no real controls at all.** Cross-platform runtimes, web
|
||||
views and canvas-drawn surfaces often produce an empty (or text-free) tree, which is why a plan built
|
||||
on "read the bounds out of the XML" stalls on exactly those apps — and also why tapping a control that
|
||||
"should be there" silently does nothing. When the tree is empty, the image is the **primary** evidence,
|
||||
not a fallback, and you read it directly rather than trying to derive coordinates from a tree that does
|
||||
not exist.
|
||||
|
||||
`screencap` itself returns a 0-byte file on some ROMs. Write on-device and pull instead:
|
||||
|
||||
```bash
|
||||
adb -s <serial> shell "su -c 'screencap -p /sdcard/x.png'" && adb -s <serial> pull /sdcard/x.png
|
||||
```
|
||||
|
||||
If that is also empty, do not fight it: `uiautomator dump`'s XML is **better** evidence — text, diffable, and it records the real control tree and its contents. (If the dump fails with `could not get idle state`, retry once; a paused animation is usually the cause.)
|
||||
If that is also empty, do not fight it — but note that **a 0-byte capture is not evidence the screen is
|
||||
blank** (`pitfalls.md` P20). And if `uiautomator dump` fails with `could not get idle state`, retry once;
|
||||
a paused animation is usually the cause.
|
||||
|
||||
### Verify form input by reading it back
|
||||
|
||||
|
||||
@@ -72,11 +72,37 @@ the snapshot rather than by pattern-matching machine code.
|
||||
|
||||
- **Strings first.** User-visible text (dialog bodies, feature labels) is the cheapest anchor in any
|
||||
runtime. Find it, then find what references it.
|
||||
- **Search them in the encoding the runtime actually uses, or you will conclude they do not exist.**
|
||||
This is the single most common false negative in cross-platform work. A runtime snapshot does not
|
||||
necessarily store text as UTF-8; UTF-16 (little-endian on these targets) is common and entirely
|
||||
valid. A UTF-8 search of such a snapshot returns **zero hits**, which reads as "the strings were
|
||||
stripped / encrypted, this route is dead" — and that conclusion is wrong.
|
||||
|
||||
```python
|
||||
blob = open('libapp.so', 'rb').read() # or whichever artifact holds the snapshot
|
||||
for phrase in ('<feature label>', '<dialog title>'):
|
||||
print(phrase, 'utf-8:', blob.find(phrase.encode('utf-8')),
|
||||
'utf-16le:', blob.find(phrase.encode('utf-16-le')))
|
||||
```
|
||||
|
||||
Try both, and try a short distinctive substring rather than a long phrase.
|
||||
- **A phrase that is absent may simply never exist as one literal.** UI text is often assembled from
|
||||
fragments or templates, so "the whole sentence" can be missing while both halves are present.
|
||||
Search the shortest distinctive token, and expect the interesting anchor to be a *label* rather
|
||||
than a sentence.
|
||||
- **Landing on a string tells you where the text lives, not which code decided to show it.** Treat the
|
||||
hit as an anchor for reference-hunting, not as the answer.
|
||||
- **Compare two builds.** The same feature in a slightly different version often reveals the code path.
|
||||
- **Watch the boundary, not the interior.** For cross-platform apps it is usually far cheaper to observe
|
||||
what crosses between layers than to reverse the interior of the runtime.
|
||||
- **Do not assume the obfuscated names are stable or meaningful.** They are not, and treating them as
|
||||
identities leads to conclusions that break on the next build.
|
||||
- **Obfuscated identifiers can be non-ASCII and invisible.** Renaming passes can replace class and
|
||||
member names with characters outside ASCII (combining marks, variant selectors and similar). If a
|
||||
class exists at runtime but "cannot be found" by the name you read from a decompiler, suspect
|
||||
encoding or normalisation in your tooling rather than a missing class. Resolve such members by
|
||||
**shape** — parameter counts, types, interface implementation — instead of by name, and prefer
|
||||
runtime enumeration of loaded classes over guessing identifiers.
|
||||
|
||||
## What this changes about your plan
|
||||
|
||||
|
||||
@@ -83,6 +83,116 @@ Do these cheaply, and only when they can change a decision — not as ceremony.
|
||||
- **When resuming**: read the record first, and treat anything not in it as unknown, even if it feels
|
||||
familiar.
|
||||
|
||||
## The most expensive drift: solving it in an environment the deliverable will never see
|
||||
|
||||
There is one drift in this domain that costs more than all the others, because it produces a
|
||||
result that looks like success and is not one.
|
||||
|
||||
**A runtime-only result is easy to obtain and easy to mistake for a finished artifact.**
|
||||
|
||||
Editing a data file, hooking a live process, blocking a hostname, holding a proxy open — each of
|
||||
these can make the app behave correctly *on the machine where you did it*, with no repackaging
|
||||
required. It is often the fastest path to a visible win. It is also frequently **not the
|
||||
deliverable at all**, because the requirement was never "make it work here".
|
||||
|
||||
The constraint axes that decide this — check every one against the original request, and write
|
||||
down the answer as a testable sentence before you start:
|
||||
|
||||
| Axis | The question | Why it silently invalidates work |
|
||||
|---|---|---|
|
||||
| **Privilege** | must it run **unrooted**? | a rooted-only result cannot be given to a normal user at all |
|
||||
| **Modification form** | must it be a **rebuilt/installable artifact**, or is a live-instrumentation result acceptable? | hooks and data edits do not ship inside an APK |
|
||||
| **ABI / device class** | which ABI, which device family? | an emulator-only or x86-only result is not evidence for a physical ARM device |
|
||||
| **Network** | must it work **online**? | a fix that depends on being offline or on a host proxy fails the moment it is used normally |
|
||||
| **Persistence** | must it survive restart, upgrade, and a fresh install? | in-memory and session-scoped state evaporates |
|
||||
| **Distribution** | does the *shipped file* have to be self-contained? | anything that needs a helper on the machine is not a shippable artifact |
|
||||
|
||||
**Checkpoint question, asked at the same moments as the drift checks above:**
|
||||
|
||||
> *If I handed over exactly what exists right now, would it satisfy the constraint sentence I wrote
|
||||
> at the start?*
|
||||
|
||||
If the answer is no, you have made real progress on a **component**, and that is worth stating as
|
||||
such — but it is not the task. Do not let "it works" stand in for "it works under the constraint".
|
||||
|
||||
**What to do instead of over-claiming.** Split the result into two explicitly labelled parts:
|
||||
|
||||
1. **What is achievable inside the constraint**, and how far along it is.
|
||||
2. **The unconstrained workaround** (needs root / needs a host / needs a proxy), stated as a
|
||||
deliberate fallback, with its requirements made obvious to the reader.
|
||||
|
||||
A privileged workaround is genuinely useful — it can be the difference between using the app and
|
||||
not. It becomes a problem only when it is presented as the deliverable. Report it as a fallback,
|
||||
keep the constrained goal open, and say plainly which one you have.
|
||||
|
||||
**Also watch the inverse**: if the constraint is "must be a rebuilt artifact", do not let a working
|
||||
runtime result quietly close the investigation. Use it for what it is good for — it proves the
|
||||
mechanism and identifies the exact code or data to change — then port that finding into the
|
||||
artifact. The runtime result is the map, not the destination.
|
||||
|
||||
## Bound every wait
|
||||
|
||||
A long task stalls in ways that produce no information and consume the most valuable resource you
|
||||
have: wall-clock time and the next person's patience. Three habits prevent almost all of it.
|
||||
|
||||
**1. Every command has a timeout, and its absence is the bug.** A helper that shells out without a
|
||||
timeout can hang forever, and the failure presents as "the task stopped making progress" rather
|
||||
than "this call blocked". Pass an explicit timeout on every subprocess, every HTTP request, every
|
||||
device call. When one expires, the result is **unknown**, not failed — record it that way and
|
||||
re-check state before retrying.
|
||||
|
||||
**2. Calibrate the expected duration from measurement, not from a guess.** A timeout only means
|
||||
something if it is set relative to how long the operation *should* take, because that is what makes
|
||||
"slow" and "hung" distinguishable. Guessed budgets are either so short that healthy work gets killed,
|
||||
or so long that a stall looks like patience.
|
||||
|
||||
So: **measure once, write it down, then derive the bound.**
|
||||
|
||||
| Operation class | How to bound it |
|
||||
|---|---|
|
||||
| host-side tool (`baksmali`, `apksigner`, a compiler) | seconds to low minutes; time it once, then set roughly 3–5× observed |
|
||||
| device shell call (`adb shell …`) | seconds — **except** the first `su` after a reboot, which can prompt and block |
|
||||
| install / push of a large artifact | minutes; scales with size, not with the app's complexity |
|
||||
| app cold start to first frame | tens of seconds — **longer with a packer**, which does real work before your code runs |
|
||||
| a step needing a human or an external service | **unbounded in principle** — do not wait at all (see P22) |
|
||||
| waiting for an on-screen state change | bound it *and* sample it (point 3) |
|
||||
|
||||
Record the observed durations in the live record. They are exactly the kind of mechanical fact that
|
||||
gets re-derived painfully after a context loss, and having them turns a later timeout into an
|
||||
interpretable signal instead of a mystery.
|
||||
|
||||
**Exceeding the expected window is itself the finding.** It tells you something about state — the
|
||||
device is wedged, the app never reached that phase, the action never happened — and that is a
|
||||
different investigation from waiting longer. Re-check state instead of extending the deadline.
|
||||
|
||||
**3. Every wait has a deadline, an observable to sample, and a look.** "Wait for the operation to
|
||||
finish" is not a plan. Name the observable, how often you will sample it, how long you will sample
|
||||
before giving up, and what you conclude on expiry:
|
||||
|
||||
```
|
||||
waiting for: <observable> e.g. the overlay is gone / the counter incremented
|
||||
sample: <interval + how> e.g. every 15 s: screenshot + current window focus
|
||||
give up at: <deadline> e.g. 150 s
|
||||
on expiry: <what I conclude> e.g. "did not complete within the window" -- not "cannot work"
|
||||
```
|
||||
|
||||
**Sample with your eyes rather than sleeping blind.** When the step is gated on something visible — a
|
||||
screen, a dialog, a progress indicator — capture it and **look at it** instead of sleeping through the
|
||||
interval. A fixed sleep either wastes time or measures the wrong instant; a look tells you what
|
||||
actually happened, and byte-identical consecutive samples tell you nothing is going to change
|
||||
(`environment.md` §look at the screen). This is the cheapest way to avoid spending rounds on a state
|
||||
that was never going to arrive.
|
||||
|
||||
Three rules that follow:
|
||||
|
||||
- **A deadline that passes is a measurement, not a verdict.** "Still not finished after N seconds"
|
||||
tells you about latency and reliability. It does not tell you the approach is impossible, and
|
||||
writing it down as impossible is how a viable route gets discarded.
|
||||
- **Never busy-poll a long job while you have independent work.** Start it, do the other thing, and
|
||||
collect it when it settles. Polling wastes the same resource the task is already spending.
|
||||
- **Prefer a bounded observable over a fixed sleep.** Poll the state you actually care about, with a
|
||||
cap — and prefer a sample you can inspect over a duration you hope is right.
|
||||
|
||||
## Handover
|
||||
|
||||
A handover is the record plus three things, written for someone with **no** memory of the task:
|
||||
|
||||
@@ -101,12 +101,82 @@ Stripped and obfuscated libraries still expose structure:
|
||||
|
||||
## Cross-architecture notes
|
||||
|
||||
- **arm64** is the common case on modern devices; prefer it and make sure the host library you pick
|
||||
actually exists for that ABI.
|
||||
- **x86_64** appears on emulators. Shells and payloads are ABI-specific: a file built for one ABI will
|
||||
not be loaded on the other, and the failure mode looks like a missing library rather than a mismatch.
|
||||
- Keep your build artifacts per-ABI and label them. Mixing them is a common source of "it crashed for no
|
||||
reason" when the same APK behaves differently on an emulator versus a device.
|
||||
Architecture is not background information here — it decides **which artifact you must edit** and
|
||||
**whether your hook can observe anything at all**. Get it from the runtime, not from the manifest.
|
||||
|
||||
### What the device claims vs what is executing
|
||||
|
||||
Three different answers, and they disagree more often than people expect:
|
||||
|
||||
| Source | Answers | Command |
|
||||
|---|---|---|
|
||||
| The device | which ABIs the system supports | `getprop ro.product.cpu.abi` and `ro.product.cpu.abilist` |
|
||||
| The package manager | **which ABI it chose for this app on this device** | `dumpsys package <pkg> \| grep primaryCpuAbi` |
|
||||
| The live process | **what is actually mapped right now** | `scripts/lib_map.py --pkg <pkg>` |
|
||||
|
||||
Only the third is ground truth. The first two are predictions, and they are wrong exactly when it
|
||||
matters: an emulator whose primary ABI is `x86_64` can be running an `arm64-v8a`-only app through a
|
||||
translator, and a fat APK can have the package manager pick an ABI you did not assume.
|
||||
|
||||
**Consequence: if the library you patched does not appear in the live mapping, your change cannot
|
||||
matter.** That is a plan problem, and no amount of re-patching will fix it.
|
||||
|
||||
### Translation layers change what you are observing
|
||||
|
||||
When an ARM-only app runs on an x86 host, a translator (Intel Houdini, `libndk_translation`, a
|
||||
`native_bridge` in general) is executing the guest code. Detect it from the maps — look for
|
||||
translator marker libraries and for app libraries whose architecture differs from the host's
|
||||
primary ABI (`lib_map.py` reports both).
|
||||
|
||||
What it changes:
|
||||
|
||||
- **Timing.** Translation is slower and less predictable. Anything you measured about latency is
|
||||
about the translator, not the code.
|
||||
- **Native integrity and anti-tamper checks can behave differently** under translation, in either
|
||||
direction — a check that fails on hardware may pass here, and vice versa. A pass under translation
|
||||
is not evidence of a pass on hardware.
|
||||
- **Hook behaviour.** Intercepting translated code is not the same as intercepting native code:
|
||||
address spaces, calling conventions, and what the host sees at a syscall boundary all differ. If a
|
||||
native hook reports nothing while the feature plainly runs, suspect translation before suspecting
|
||||
your script.
|
||||
- **Syscall-level observation shows the host's view.** A file or network call made from the guest
|
||||
may look different on the host side than it would natively.
|
||||
|
||||
**Rule: verify the artifact on the ABI the user will actually run.** An emulator-only result is a
|
||||
mid-task checkpoint, never the final claim (`verification.md`).
|
||||
|
||||
### Fat APKs: patching the ABI that never loads
|
||||
|
||||
An APK can ship `lib/arm64-v8a/`, `lib/armeabi-v7a/`, `lib/x86_64/`… The package manager extracts
|
||||
**one** of them into the install-time native library directory, and only that one is loaded.
|
||||
|
||||
- Check `primaryCpuAbi` and the live maps before editing a `.so`. Editing `arm64-v8a` while the
|
||||
device loads `armeabi-v7a` produces a build that is byte-different and behaviorally identical.
|
||||
- If the deliverable must work for an unknown user, remember their device may select a different
|
||||
ABI than yours. Either patch every ABI present, or state which ABI your artifact targets.
|
||||
- If the app's own libraries are written at *runtime* rather than extracted from the APK, the live
|
||||
paths will not be inside the APK at all (`lib_map.py` marks these `materialized`). Those belong to
|
||||
whatever produced them, and a repacked APK will not carry them in that location.
|
||||
|
||||
### Payloads and tooling are per-ABI too
|
||||
|
||||
- A native payload, an injected library, or a shellcode blob built for one ABI **will not load** on
|
||||
another. The failure usually surfaces as a *missing library* or a generic linker error rather than
|
||||
a clean "wrong architecture" message — do not read it as "my payload is broken".
|
||||
- **Pointer size differs** between 32- and 64-bit targets. A script or struct layout that assumes
|
||||
8-byte words misbehaves silently on a 32-bit target.
|
||||
- Keep build artifacts per-ABI and label them with the ABI in the filename. Mixing them up is a
|
||||
reliable source of "it crashed for no reason" when the same APK behaves differently on an emulator
|
||||
and on a device.
|
||||
|
||||
### Quick decision list
|
||||
|
||||
1. `scripts/lib_map.py --pkg <pkg> --app-only` → which app libraries are loaded, from where, and at
|
||||
what architecture.
|
||||
2. If a translator is present and the task needs reliable native behaviour, move to a matching
|
||||
device.
|
||||
3. If the library you meant to patch is absent, stop — pick the right library first.
|
||||
4. Only then start editing.
|
||||
|
||||
## Verification
|
||||
|
||||
|
||||
@@ -493,3 +493,113 @@ an impression of work being done.
|
||||
- Detect stalls by change, not by elapsed time: if N consecutive samples are identical, break out.
|
||||
- Prefer driving the prompt to completion over waiting it out — the same prompt usually recurs, so
|
||||
automating it once pays back immediately.
|
||||
|
||||
---
|
||||
|
||||
## P23. The fix lives in a file the app rewrites
|
||||
|
||||
**Symptom**
|
||||
You change a value in the app's data, the write succeeds, the file reads back correctly — and after
|
||||
the next launch the value is back to what it was. Often byte for byte identical, which makes it look
|
||||
like nothing happened at all.
|
||||
|
||||
**Root cause**
|
||||
The stored value is a **cache, not a source of truth**. Either the app re-fetches it and re-persists
|
||||
it, or it rewrites the file from its own defaults on every start. Your edit was never authoritative.
|
||||
|
||||
Compounding it: the intuitive way to protect the file — a restrictive mode or a changed owner — **does
|
||||
not work**, because the app does not open-and-write the existing file. It **deletes the file and
|
||||
creates a new one**, and a new file is created with the app's own mode and owner. Nothing is inherited,
|
||||
so `chmod`/`chown` are silently ineffective.
|
||||
|
||||
**Why it is hard to see**
|
||||
Verification is usually done immediately after writing, while the file is still correct. The rewrite
|
||||
only happens on the next start, which is one step further along than you looked. And the "fix" that
|
||||
seems obviously right (tighten permissions) fails without any error.
|
||||
|
||||
**Do instead**
|
||||
- **Verify after a relaunch, not after the write.** The write succeeding is not the finding; surviving
|
||||
a restart is.
|
||||
- **Distinguish the two causes with one offline launch.** If the value survives with the network down,
|
||||
it came from the server. If it does not, the app is regenerating it locally — and then a data edit is
|
||||
the wrong layer entirely; patch the read path instead.
|
||||
- To make a data edit stick, use the **immutable attribute** and confirm it took effect:
|
||||
```bash
|
||||
su -c "chattr +i <file>"; su -c "lsattr <file>" # expect the 'i' flag
|
||||
```
|
||||
It is enforced by the filesystem against the delete itself, which is why it holds where permissions
|
||||
do not. Undo with `chattr -i`.
|
||||
- Then **exercise the feature**, not just the value. A blocked write the app depends on can make it
|
||||
misbehave; "the file still has my value" is not "the app still works".
|
||||
- Remember a lock is **device state, not artifact state** — it does not travel with an APK. Record it
|
||||
as an environment requirement. The only form that ships is a code patch (`runtime-data.md`).
|
||||
|
||||
---
|
||||
|
||||
## P24. The artifact changed, but the wrong one is executing
|
||||
|
||||
**Symptom**
|
||||
The build differs from the original, the pipeline reports success, the file on disk is genuinely
|
||||
modified — and the app behaves exactly as before. Or a native hook reports nothing while the feature
|
||||
plainly runs.
|
||||
|
||||
**Root cause**
|
||||
Something other than your edit is being used at runtime:
|
||||
|
||||
- **Wrong ABI.** A fat APK ships several `lib/<abi>/` directories; the package manager extracts and
|
||||
loads **one**. Editing `arm64-v8a` while the device loads `armeabi-v7a` produces a byte-different,
|
||||
behaviorally identical build.
|
||||
- **The library is not from the APK at all.** Some libraries are written into the app's data directory
|
||||
at runtime rather than extracted from the package. Patching the APK copy changes a file nobody loads.
|
||||
- **Multiple processes.** The work was done in, or the check lives in, a different process than the one
|
||||
you are observing.
|
||||
- **A stale install.** The package manager reported success for a request that did not replace what is
|
||||
on disk (`P18`).
|
||||
|
||||
**Why it is hard to see**
|
||||
Every local indicator agrees: the diff is non-empty, the build is signed, the install returned success.
|
||||
Nothing in the *build* pipeline can detect this, because the build is fine. The contradiction only
|
||||
exists at runtime.
|
||||
|
||||
**Do instead**
|
||||
- **Ask the running process what it loaded**, before editing: `scripts/lib_map.py --pkg <pkg>`.
|
||||
Libraries whose path is not under the installed APK's lib directory were materialized at runtime
|
||||
and belong to whatever produced them.
|
||||
- Confirm the ABI the package manager actually chose (`dumpsys package <pkg> | grep primaryCpuAbi`)
|
||||
rather than the one you assumed from the manifest.
|
||||
- **If the library you patched is not in the live mapping, stop.** No amount of re-patching helps; the
|
||||
plan is wrong.
|
||||
- Treat a behaviorally identical rebuild as **positive evidence that your edit is not being loaded**,
|
||||
not as "the change had no effect". Those are different conclusions and only one of them is actionable.
|
||||
|
||||
---
|
||||
|
||||
## P25. "The search found nothing" treated as "the data is not there"
|
||||
|
||||
**Symptom**
|
||||
You scan an artifact for a known-present value — a UI label, a marker string, an endpoint — get zero
|
||||
hits, and conclude the content is stripped, encrypted, or otherwise unavailable. A route gets written
|
||||
off on that basis.
|
||||
|
||||
**Root cause**
|
||||
The search used the wrong representation. A byte scan for UTF-8 text returns nothing against content
|
||||
that is stored as UTF-16, or compressed, or framed inside a container, or split across fragments. The
|
||||
data is present; the needle was encoded differently from the haystack.
|
||||
|
||||
**Why it is hard to see**
|
||||
"Zero results" is a clean, confident-looking output. It feels like a measurement, so it gets recorded
|
||||
as a finding, and findings propagate into the plan.
|
||||
|
||||
**Do instead**
|
||||
- **Before concluding absence, search more than one encoding.** UTF-8 and UTF-16LE will between them
|
||||
cover most text storage:
|
||||
```python
|
||||
blob.find(needle.encode('utf-8')), blob.find(needle.encode('utf-16-le'))
|
||||
```
|
||||
- **Search the shortest distinctive fragment.** Text is often assembled from pieces or templates, so a
|
||||
full sentence can be absent while its parts are present.
|
||||
- **Try the value without its framing.** A hit rate of zero is also the expected result for content
|
||||
that is inside a compressed or encoded container — decode the container first (`runtime-data.md`,
|
||||
`scripts/blob_decode.py`).
|
||||
- **State a negative result with its scope**: "no UTF-8 or UTF-16LE literal match in this artifact"
|
||||
is a finding. "The string does not exist" is a guess wearing a finding's clothes.
|
||||
|
||||
@@ -105,8 +105,85 @@ If the effect must ship inside an APK, move it into code. Patterns, in order of
|
||||
|
||||
Reference case: a promo popup was gated by `<key>_expires_at`. Writing a large value via DataStore suppressed it on the test device, but a fresh install lost it. The durable fix replaced the dedicated map-lambda that produces the value so it always yields a far-future timestamp — a single-purpose class, safe to patch, no effect on the shared serialization helpers.
|
||||
|
||||
## When the app rewrites your edit
|
||||
|
||||
A very common sequence: you change the value, relaunch, and the value is **back** — often byte for
|
||||
byte identical to what it was before. That is not a failed write. It is the app re-establishing the
|
||||
value from a source of truth you have not touched yet.
|
||||
|
||||
Two causes look identical and need different fixes:
|
||||
|
||||
| Cause | How to tell | Fix |
|
||||
|---|---|---|
|
||||
| The app **re-fetches from the server** and re-persists | take the app offline and relaunch: does your value survive? | lock the file, block the refresh request, or move the change into code |
|
||||
| The app **rewrites the file on every start** from its own defaults | your value does not survive even with the network down | the value is not authoritative — patch the read path instead |
|
||||
|
||||
Do the offline test first. It costs one launch and separates the two cases.
|
||||
|
||||
### Making a data edit stick (in order of durability)
|
||||
|
||||
1. **Immutable file attribute** — the reliable, reversible way to stop a rewrite:
|
||||
```bash
|
||||
su -c "chattr +i <path/to/prefs.xml>"
|
||||
su -c "lsattr <path/to/prefs.xml>" # expect the 'i' flag, e.g. -----i-------
|
||||
# to undo:
|
||||
su -c "chattr -i <path/to/prefs.xml>"
|
||||
```
|
||||
Verify with `lsattr`, not by assuming the command worked.
|
||||
|
||||
**Why permissions do not achieve this.** Tightening the mode or changing the owner looks like the
|
||||
obvious move and does not work, because the app does not open-and-write the existing file — it
|
||||
**deletes the file and creates a new one**. The new file is created by the app, with the app's own
|
||||
mode and owner, so a restrictive `chmod` or a `chown root` is simply not inherited. The immutable
|
||||
attribute is enforced by the filesystem against the delete itself, which is why it holds.
|
||||
|
||||
2. **Keep the app offline** so the refresh source is unreachable. Effective for a test, and
|
||||
sometimes acceptable in use — but be explicit about it, because a fix that only works offline
|
||||
fails the "works under normal conditions" constraint. See the deliverable-drift section in
|
||||
`long-task-discipline.md` before presenting this as a result.
|
||||
|
||||
3. **Block the refresh request** at the runtime or network layer — durable while your instrumentation
|
||||
is running, and requires you to have identified the endpoint (`references/server-api.md`).
|
||||
|
||||
4. **Move the change into code** — patch the read path or the decision. This is the only form that
|
||||
ships inside an APK (`§Making the fix durable` above).
|
||||
|
||||
### Two cautions about locking
|
||||
|
||||
- **Verify the feature, not the value.** Blocking a write the app depends on can make it misbehave
|
||||
or fail loudly. After locking, exercise the feature you care about — "the file still has my value"
|
||||
is not the same as "the app still works".
|
||||
- **A lock is device state; it does not travel with an APK.** If the recipe requires it, record it as
|
||||
an environment requirement, so nobody later mistakes it for something baked into the artifact.
|
||||
|
||||
## Encoded values: do not assume "encrypted"
|
||||
|
||||
Preference values that carry server configuration are frequently stored as a single opaque string
|
||||
rather than readable XML. Before concluding the value is encrypted, check whether it is merely
|
||||
**framed**: a common shape is `base64( rotate( deflate( json ) ) )`, where the cyclic rotation exists
|
||||
precisely so a naive decode fails.
|
||||
|
||||
`scripts/blob_decode.py` searches the parameter space instead of guessing it — outer encoding
|
||||
(base64 / base64url / hex), rotation offset, and compression type — and reports every combination
|
||||
that yields a structured document, plus the exact parameters to re-encode your edited payload:
|
||||
|
||||
```bash
|
||||
# pull the value straight out of a preferences XML and decode it
|
||||
python scripts/blob_decode.py --prefs-xml prefs.xml --name <key> # (see --help for exact name)
|
||||
|
||||
# after editing the decoded payload, rebuild the value with the winning parameters
|
||||
python scripts/blob_decode.py --encode --file decoded.bin --outer base64 --inner raw --cut <N>
|
||||
```
|
||||
|
||||
The rotation search is cheap — a few tens of thousands of candidates resolve in well under a second
|
||||
— so there is no reason to guess. **Distinguish framing from real encryption before spending time
|
||||
on a key:** a framed blob becomes structured the moment you remove the framing, whereas a keyed blob
|
||||
stays random-looking. If it stays random, treat it as opaque and move to the code path that consumes
|
||||
it.
|
||||
|
||||
## Cautions
|
||||
|
||||
- Editing data while the app is running is unreliable: in-memory caches win.
|
||||
- The server may overwrite your value on next sync. If the app re-fetches and re-persists the authoritative value, a data edit is temporary by design — which is exactly why the durable version belongs in code.
|
||||
- Editing data while the app is running is unreliable: in-memory caches win. Force-stop, edit, then launch.
|
||||
- The server may overwrite your value on next sync — see *When the app rewrites your edit* above.
|
||||
- Do not edit a token you do not own and expect it to be accepted; tokens are validated server-side (`references/server-api.md`).
|
||||
- A restored data directory needs ownership fixed again after every reinstall, because the uid increments.
|
||||
|
||||
@@ -0,0 +1,276 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Decode (and re-encode) an opaque single-value blob cached by an app.
|
||||
|
||||
Many apps cache server-issued configuration as ONE opaque string inside a
|
||||
preferences file or a key-value store. The encoding is usually some stack of:
|
||||
outer: base64, base64url, or hex
|
||||
transform: a cyclic rotation of the byte stream (so the compressed stream
|
||||
does not begin at its header), sometimes preceded by a short
|
||||
header to skip
|
||||
inner: raw deflate, zlib, or gzip
|
||||
|
||||
You do not have to guess the parameters. This script searches them and reports
|
||||
every combination that decodes to something structured, then can re-encode your
|
||||
edited payload with the same parameters so it can be written back.
|
||||
|
||||
Why rotation matters: a plain base64->inflate attempt fails on these blobs, and
|
||||
the failure looks like "this value is encrypted". Exhaustively trying cyclic
|
||||
shifts is cheap and frequently succeeds, which turns an opaque blob into an
|
||||
editable JSON/XML document.
|
||||
|
||||
Usage:
|
||||
# decode the value of a key inside a prefs XML
|
||||
python blob_decode.py --prefs-xml prefs.xml --name flutter.remoteConfig
|
||||
|
||||
# decode a blob stored in its own file (or piped on stdin)
|
||||
python blob_decode.py --file value.txt
|
||||
cat value.txt | python blob_decode.py
|
||||
|
||||
# keep searching harder (slower) and write the best result out
|
||||
python blob_decode.py --file value.txt --max-skip 64 --out decoded.bin
|
||||
|
||||
# re-encode an edited payload with the parameters that worked
|
||||
python blob_decode.py --encode --file decoded.bin --outer base64 --inner raw --cut 15435 --out value.txt
|
||||
|
||||
Notes:
|
||||
* Output is written to --out; nothing on device is touched.
|
||||
* The winning parameters are printed as a copy-pasteable line for --encode.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import base64
|
||||
import binascii
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
import zlib
|
||||
|
||||
INNERS = [("raw", -15), ("zlib", 15), ("gzip", 31)]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# outer encodings
|
||||
# --------------------------------------------------------------------------- #
|
||||
def outer_decode(text: str, which: str) -> bytes | None:
|
||||
t = text.strip().replace("\n", "").replace("\r", "").replace(" ", "")
|
||||
try:
|
||||
if which == "base64":
|
||||
return base64.b64decode(t + "=" * (-len(t) % 4))
|
||||
if which == "base64url":
|
||||
return base64.urlsafe_b64decode(t + "=" * (-len(t) % 4))
|
||||
if which == "hex":
|
||||
return binascii.unhexlify(re.sub(r"[^0-9a-fA-F]", "", t))
|
||||
except Exception:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def outer_encode(data: bytes, which: str) -> str:
|
||||
if which == "base64":
|
||||
return base64.b64encode(data).decode("ascii")
|
||||
if which == "base64url":
|
||||
return base64.urlsafe_b64encode(data).decode("ascii")
|
||||
if which == "hex":
|
||||
return binascii.hexlify(data).decode("ascii")
|
||||
raise ValueError("unknown outer encoding: %s" % which)
|
||||
|
||||
|
||||
def sniff_outer(text: str) -> list[str]:
|
||||
"""Cheap guess first so the common case is instant."""
|
||||
t = text.strip()
|
||||
guesses = []
|
||||
if re.fullmatch(r"[0-9a-fA-F\s]+", t) and len(t) >= 16:
|
||||
guesses.append("hex")
|
||||
if re.fullmatch(r"[A-Za-z0-9+/\s=]+", t):
|
||||
guesses.append("base64")
|
||||
if re.fullmatch(r"[A-Za-z0-9\-_\s=]+", t):
|
||||
guesses.append("base64url")
|
||||
# always keep base64 last-resort, then everything else
|
||||
for w in ("base64", "base64url", "hex"):
|
||||
if w not in guesses:
|
||||
guesses.append(w)
|
||||
return list(dict.fromkeys(guesses))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# inner transforms
|
||||
# --------------------------------------------------------------------------- #
|
||||
def try_inflate(body: bytes, max_out: int | None = None) -> tuple[str, bytes] | None:
|
||||
best = None
|
||||
for name, wbits in INNERS:
|
||||
try:
|
||||
if max_out is None:
|
||||
out = zlib.decompress(body, wbits)
|
||||
else:
|
||||
d = zlib.decompressobj(wbits)
|
||||
out = d.decompress(body, max_out)
|
||||
if out:
|
||||
if best is None or len(out) > len(best[1]):
|
||||
best = (name, out)
|
||||
except Exception:
|
||||
continue
|
||||
return best
|
||||
|
||||
|
||||
def inflate_raw(data: bytes) -> bytes:
|
||||
co = zlib.compressobj(9, zlib.DEFLATED, -15)
|
||||
return co.compress(data) + co.flush()
|
||||
|
||||
|
||||
def score(out: bytes) -> float:
|
||||
"""How much does this look like a real document we can edit?"""
|
||||
if not out:
|
||||
return -1.0
|
||||
printable = sum(1 for b in out[:4096] if 32 <= b < 127 or b in (9, 10, 13))
|
||||
ratio = printable / min(len(out), 4096)
|
||||
s = ratio * 10.0
|
||||
head = out[:64].lstrip()
|
||||
if head[:1] in (b"{", b"[", b"<", b"#"):
|
||||
s += 6.0
|
||||
if b'"' in out[:512] or b"=" in out[:512]:
|
||||
s += 2.0
|
||||
if len(out) < 32:
|
||||
s -= 3.0
|
||||
return s
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
def search(blob: bytes, max_skip: int, log) -> list[dict]:
|
||||
hits: list[dict] = []
|
||||
n = len(blob)
|
||||
t0 = time.time()
|
||||
|
||||
for skip in range(0, min(max_skip, n) + 1):
|
||||
base = blob[skip:]
|
||||
m = len(base)
|
||||
if m < 8:
|
||||
break
|
||||
for k in range(m):
|
||||
cand = base[k:] + base[:k]
|
||||
got = try_inflate(cand, max_out=8_000_000)
|
||||
if got:
|
||||
name, out = got
|
||||
hits.append({
|
||||
"skip": skip, "cut": k, "inner": name, "size": len(out),
|
||||
"score": round(score(out), 2), "data": out,
|
||||
})
|
||||
if hits and log and skip == 0:
|
||||
log(" first hit found with skip=0 after %.1fs" % (time.time() - t0))
|
||||
if log and skip and skip % 16 == 0 and time.time() - t0 > 30:
|
||||
log(" ... skip=%d, %.0fs elapsed" % (skip, time.time() - t0))
|
||||
log(" search finished in %.1fs (%d candidate(s))" % (time.time() - t0, len(hits)))
|
||||
return hits
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
src = ap.add_mutually_exclusive_group()
|
||||
src.add_argument("--file", help="file containing the encoded value")
|
||||
src.add_argument("--prefs-xml", help="a preferences XML file to pull the value out of")
|
||||
ap.add_argument("--name", help="key name when using --prefs-xml")
|
||||
ap.add_argument("--outer", choices=["base64", "base64url", "hex", "auto"], default="auto")
|
||||
ap.add_argument("--max-skip", type=int, default=0,
|
||||
help="also try dropping 0..N leading bytes before the rotation search (slower)")
|
||||
ap.add_argument("--min-size", type=int, default=200, help="ignore results smaller than this")
|
||||
ap.add_argument("--top", type=int, default=5, help="how many candidates to print")
|
||||
ap.add_argument("--out", help="write the best decoded result here")
|
||||
|
||||
ap.add_argument("--encode", action="store_true", help="re-encode instead of decoding")
|
||||
ap.add_argument("--inner", choices=[n for n, _ in INNERS], default="raw")
|
||||
ap.add_argument("--cut", type=int, default=0, help="rotation used for write-back")
|
||||
a = ap.parse_args()
|
||||
|
||||
def log(msg: str) -> None:
|
||||
print(msg, file=sys.stderr)
|
||||
|
||||
# ------------------------------------------------------------------ encode
|
||||
if a.encode:
|
||||
if not a.file:
|
||||
log("--encode requires --file (the payload to encode)")
|
||||
return 2
|
||||
payload = open(a.file, "rb").read()
|
||||
body = inflate_raw(payload) if a.inner == "raw" else zlib.compress(payload, 9)
|
||||
if a.inner == "zlib":
|
||||
body = zlib.compress(payload, 9)
|
||||
cut = a.cut % max(len(body), 1)
|
||||
rotated = body[-cut:] + body[:-cut] if cut else body
|
||||
enc = outer_encode(rotated, "base64" if a.outer == "auto" else a.outer)
|
||||
if a.out:
|
||||
open(a.out, "w", encoding="utf-8", newline="\n").write(enc)
|
||||
log("wrote %d chars to %s" % (len(enc), a.out))
|
||||
else:
|
||||
print(enc)
|
||||
log("write-back parameters: outer=%s inner=%s cut=%d"
|
||||
% ("base64" if a.outer == "auto" else a.outer, a.inner, cut))
|
||||
return 0
|
||||
|
||||
# ------------------------------------------------------------------ input
|
||||
if a.prefs_xml:
|
||||
if not a.name:
|
||||
log("--prefs-xml requires --name <key>")
|
||||
return 2
|
||||
xml = open(a.prefs_xml, encoding="utf-8", errors="replace").read()
|
||||
m = re.search(r'name="%s"[^>]*>([^<]*)<' % re.escape(a.name), xml)
|
||||
if not m:
|
||||
log("key not found in XML: %s" % a.name)
|
||||
log("keys present: %s" % ", ".join(re.findall(r'name="([^"]+)"', xml)[:40]))
|
||||
return 2
|
||||
text = m.group(1)
|
||||
log("value length: %d" % len(text))
|
||||
elif a.file:
|
||||
text = open(a.file, encoding="utf-8", errors="replace").read()
|
||||
else:
|
||||
text = sys.stdin.read()
|
||||
|
||||
if not text.strip():
|
||||
log("empty input")
|
||||
return 2
|
||||
|
||||
# ------------------------------------------------------------------ decode
|
||||
candidates = [a.outer] if a.outer != "auto" else sniff_outer(text)
|
||||
for which in candidates:
|
||||
blob = outer_decode(text, which)
|
||||
if blob is None or len(blob) < 8:
|
||||
log("outer=%s -> not decodable, skipping" % which)
|
||||
continue
|
||||
log("outer=%s -> %d bytes; searching rotation (and skip<=%d)" % (which, len(blob), a.max_skip))
|
||||
hits = [h for h in search(blob, a.max_skip, log) if h["size"] >= a.min_size]
|
||||
if not hits:
|
||||
continue
|
||||
hits.sort(key=lambda h: (-h["score"], -h["size"]))
|
||||
print("=" * 66)
|
||||
print("outer=%s candidates=%d" % (which, len(hits)))
|
||||
for h in hits[:a.top]:
|
||||
print(" score=%-6s skip=%-4d cut=%-7d inner=%-5s size=%d"
|
||||
% (h["score"], h["skip"], h["cut"], h["inner"], h["size"]))
|
||||
best = hits[0]
|
||||
print("-" * 66)
|
||||
print("write-back line:")
|
||||
print(" python blob_decode.py --encode --file <edited_payload> --outer %s --inner %s --cut %d"
|
||||
% (which, best["inner"], best["cut"]))
|
||||
print("-" * 66)
|
||||
preview = best["data"][:1200]
|
||||
try:
|
||||
print(preview.decode("utf-8"))
|
||||
except UnicodeDecodeError:
|
||||
print(preview.decode("latin1"))
|
||||
print("(non-UTF-8; first 64 bytes: %s)" % best["data"][:64].hex())
|
||||
if a.out:
|
||||
open(a.out, "wb").write(best["data"])
|
||||
log("wrote %d bytes to %s" % (len(best["data"]), a.out))
|
||||
return 0
|
||||
|
||||
print("no decodable candidate found.", file=sys.stderr)
|
||||
print("Next steps:", file=sys.stderr)
|
||||
print(" * raise --max-skip, or pass --outer explicitly", file=sys.stderr)
|
||||
print(" * the blob may be genuinely encrypted (a keyed cipher) rather than merely", file=sys.stderr)
|
||||
print(" encoded -- confirm before spending time on it: a keyed blob should look", file=sys.stderr)
|
||||
print(" random, whereas an encoded one still has structure once the framing is", file=sys.stderr)
|
||||
print(" removed. See references/runtime-data.md.", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,218 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Report which native libraries are ACTUALLY mapped into a live process.
|
||||
|
||||
`getprop ro.product.cpu.abi` tells you what the device claims to be. It does not
|
||||
tell you what is executing. Those differ more often than people expect:
|
||||
|
||||
* an x86_64 emulator running an app whose only native libraries are arm64,
|
||||
because an ARM translation layer is present;
|
||||
* a hardened app whose real libraries are materialized into its data
|
||||
directory at runtime, so the paths you see are not the paths inside the APK;
|
||||
* a fat APK where the package manager picked a different ABI than you assumed,
|
||||
so you patch a `.so` that never loads.
|
||||
|
||||
This script reads the live mapping table and reports, per library: the path, the
|
||||
base address, and the architecture of the actual ELF on disk. It classifies each
|
||||
path so the interesting ones stand out (system / from-APK / runtime-materialized).
|
||||
|
||||
Requires root to read another process's maps.
|
||||
|
||||
Usage:
|
||||
python lib_map.py --pkg com.example.app
|
||||
python lib_map.py --pkg com.example.app --app-only
|
||||
python lib_map.py --pid 1234 --all
|
||||
python lib_map.py --pkg com.example.app --json
|
||||
python lib_map.py --pkg com.example.app --grep ssl
|
||||
|
||||
Reading it:
|
||||
* A library whose path sits under the app's data dir (not under the APK's
|
||||
lib/<abi> dir) was written at runtime. That is a packer/dropper fingerprint
|
||||
and it will not be present in a repacked APK -- patch or replace with care.
|
||||
* If app libraries are 64-bit ARM but the device primary ABI is x86_64,
|
||||
a translator is in play. Timing and some native checks will differ.
|
||||
* If the library you patched does not appear here at all, your patch cannot
|
||||
matter. That is a plan problem, not a patch problem.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
|
||||
E_MACHINE = {
|
||||
0x03: "x86", 0x3E: "x86_64", 0x28: "arm", 0xB7: "aarch64",
|
||||
0x08: "mips", 0xF3: "riscv", 0x16: "s390", 0x15: "ppc",
|
||||
}
|
||||
TRANSLATOR_MARKERS = ("houdini", "libndk", "native_bridge", "libntv", "armtrans",
|
||||
"libhoudini", "ndk_translation", "libarm", "exagear", "box64")
|
||||
|
||||
|
||||
def run(cmd: list[str], timeout: int = 60) -> str:
|
||||
try:
|
||||
p = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=timeout)
|
||||
return (p.stdout or "") + (p.stderr or "")
|
||||
except Exception as e:
|
||||
return "ERROR: %s" % e
|
||||
|
||||
|
||||
def classify(path: str, pkg: str | None) -> str:
|
||||
if path.startswith(("/system/", "/apex/", "/vendor/", "/product/", "/odm/", "/system_ext/")):
|
||||
return "system"
|
||||
if re.search(r"/data/app/", path):
|
||||
return "apk-lib"
|
||||
if pkg and (("/data/data/%s" % pkg) in path or ("/data/user/0/%s" % pkg) in path):
|
||||
return "materialized"
|
||||
if "/data/" in path:
|
||||
return "data-other"
|
||||
return "other"
|
||||
|
||||
|
||||
def read_elf_arch(sh_fn, path: str) -> str:
|
||||
"""Read the ELF header on the device and name the machine. '?' means the read failed."""
|
||||
hdr = sh_fn("dd if='%s' bs=20 count=1 2>/dev/null | od -An -tx1 2>/dev/null" % path)
|
||||
hexs = re.sub(r"[^0-9a-fA-F]", "", hdr)
|
||||
if len(hexs) < 40:
|
||||
return "?"
|
||||
b = bytes.fromhex(hexs[:40])
|
||||
if b[:4] != b"\x7fELF":
|
||||
return "not-elf"
|
||||
cls = {1: 32, 2: 64}.get(b[4], 0)
|
||||
little = b[5] == 1
|
||||
mach = int.from_bytes(b[18:20], "little" if little else "big")
|
||||
if mach in E_MACHINE:
|
||||
return E_MACHINE[mach] # the machine name already implies the width
|
||||
return "0x%02x/%db" % (mach, cls or 0)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
tgt = ap.add_mutually_exclusive_group(required=True)
|
||||
tgt.add_argument("--pkg", help="target package name")
|
||||
tgt.add_argument("--pid", type=int, help="target pid")
|
||||
ap.add_argument("--adb", default=os.environ.get("ADB", "adb"))
|
||||
ap.add_argument("--serial", default=None)
|
||||
ap.add_argument("--all", action="store_true", help="include system libraries")
|
||||
ap.add_argument("--app-only", action="store_true", help="only APK libs and runtime-materialized libs")
|
||||
ap.add_argument("--grep", default=None, help="only paths matching this regex")
|
||||
ap.add_argument("--no-arch", action="store_true", help="skip reading ELF headers (faster)")
|
||||
ap.add_argument("--max-libs", type=int, default=120, help="cap how many libraries get an arch probe")
|
||||
ap.add_argument("--json", action="store_true", dest="as_json")
|
||||
a = ap.parse_args()
|
||||
|
||||
def sh(cmd: str, timeout: int = 60) -> str:
|
||||
base = [a.adb] + (["-s", a.serial] if a.serial else []) + ["shell", cmd]
|
||||
return run(base, timeout)
|
||||
|
||||
def su(cmd: str, timeout: int = 90) -> str:
|
||||
base = [a.adb] + (["-s", a.serial] if a.serial else []) + ["shell", 'su -c "%s"' % cmd]
|
||||
return run(base, timeout)
|
||||
|
||||
pid = a.pid
|
||||
if a.pkg and not pid:
|
||||
out = sh("pidof %s" % a.pkg).strip()
|
||||
if not out:
|
||||
print("no running process for %s -- launch it and let it settle first" % a.pkg, file=sys.stderr)
|
||||
return 2
|
||||
pid = int(out.split()[0])
|
||||
|
||||
maps = su("cat /proc/%d/maps" % pid)
|
||||
if not maps.strip() or "No such file" in maps:
|
||||
print("cannot read /proc/%d/maps (root required, or the process is gone)" % pid, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
entries: dict[str, dict] = {}
|
||||
for line in maps.splitlines():
|
||||
m = re.match(r"^([0-9a-f]+)-([0-9a-f]+)\s+(\S+)\s+([0-9a-f]+)\s+\S+\s+\S+\s*(.*)$", line)
|
||||
if not m:
|
||||
continue
|
||||
start, end, perms, off, path = m.groups()
|
||||
path = path.strip()
|
||||
if not path or path.startswith("["):
|
||||
continue
|
||||
if not path.endswith(".so") and ".so." not in path:
|
||||
continue
|
||||
e = entries.setdefault(path, {"path": path, "perms": set(), "base": start, "count": 0})
|
||||
e["perms"].add(perms)
|
||||
e["count"] += 1
|
||||
if perms.startswith("r-x") or perms.startswith("r--"):
|
||||
e["base"] = min(e["base"], start)
|
||||
|
||||
dev_abi = sh("getprop ro.product.cpu.abi").strip()
|
||||
abilist = sh("getprop ro.product.cpu.abilist").strip()
|
||||
pkg = a.pkg
|
||||
if not pkg:
|
||||
cmdline = su("cat /proc/%d/cmdline" % pid).split("\x00")[0].strip()
|
||||
pkg = cmdline or None
|
||||
|
||||
rows = []
|
||||
probed = 0
|
||||
for path, e in sorted(entries.items()):
|
||||
kind = classify(path, pkg)
|
||||
if a.app_only and kind not in ("apk-lib", "materialized"):
|
||||
continue
|
||||
# --grep is an explicit narrow request, so honour it across all kinds
|
||||
if not a.all and not a.app_only and not a.grep and kind == "system":
|
||||
continue
|
||||
if a.grep and not re.search(a.grep, path):
|
||||
continue
|
||||
rows.append({"path": path, "base": e["base"], "kind": kind,
|
||||
"perms": "/".join(sorted(e["perms"])), "maps": e["count"], "arch": "?"})
|
||||
|
||||
if not a.no_arch:
|
||||
for r in rows[:a.max_libs]:
|
||||
r["arch"] = read_elf_arch(su, r["path"])
|
||||
probed += 1
|
||||
|
||||
# translation markers
|
||||
markers = sorted({p for p in entries if any(t in p.lower() for t in TRANSLATOR_MARKERS)})
|
||||
app_archs = sorted({r["arch"] for r in rows if r["kind"] in ("apk-lib", "materialized") and r["arch"] not in ("?", "not-elf")})
|
||||
host_is_x86 = "x86" in dev_abi
|
||||
translated = bool(markers) or (host_is_x86 and any(x.startswith(("arm", "aarch64")) for x in app_archs))
|
||||
|
||||
if a.as_json:
|
||||
print(json.dumps({"pid": pid, "pkg": pkg, "device_abi": dev_abi, "abilist": abilist,
|
||||
"translation_suspected": translated, "translator_markers": markers,
|
||||
"libs": rows}, indent=2, ensure_ascii=False))
|
||||
return 0
|
||||
|
||||
print("pid=%s pkg=%s" % (pid, pkg or "?"))
|
||||
print("device primary abi=%s abilist=[%s]" % (dev_abi, abilist))
|
||||
print("%-13s %-9s %-6s %-6s %s" % ("arch", "kind", "perms", "maps", "path"))
|
||||
print("-" * 100)
|
||||
for r in rows:
|
||||
print("%-13s %-9s %-6s %-6d %s" % (r["arch"], r["kind"], r["perms"], r["maps"], r["path"]))
|
||||
print("-" * 100)
|
||||
print("mapped libraries shown: %d" % len(rows))
|
||||
if not a.no_arch and probed < len(rows):
|
||||
print("(architecture probed for the first %d; raise --max-libs to cover the rest -- '?' means not probed)"
|
||||
% probed)
|
||||
|
||||
if markers:
|
||||
print("\nTRANSLATOR MARKERS PRESENT:")
|
||||
for mk in markers:
|
||||
print(" " + mk)
|
||||
print(" -> native ARM code is being translated. Timing differs from a real ARM device,")
|
||||
print(" and native integrity checks may behave differently. Verify the final artifact")
|
||||
print(" on the ABI the user actually runs.")
|
||||
if app_archs:
|
||||
print("\napp library architectures: %s" % ", ".join(app_archs))
|
||||
if translated:
|
||||
print("translation suspected: YES")
|
||||
mat = [r["path"] for r in rows if r["kind"] == "materialized"]
|
||||
if mat:
|
||||
print("\nRUNTIME-MATERIALIZED libraries (not inside the APK):")
|
||||
for p in mat[:20]:
|
||||
print(" " + p)
|
||||
print(" -> written at runtime by the app or its shell. A repacked APK will not contain")
|
||||
print(" them in this location; account for that before assuming your patch is loaded.")
|
||||
if not rows:
|
||||
print("\nnothing matched. Widen with --all, or check that the process is the one that draws the UI.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,250 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Preflight: prove the environment is sane BEFORE you blame your patch.
|
||||
|
||||
Run this at the start of every experiment block, and again whenever something
|
||||
fails in a way you did not expect. A large share of "my patch broke the app"
|
||||
turns out to be device state, a dead device server, a leftover proxy setting, or
|
||||
a clock skew -- all of which look exactly like a broken artifact.
|
||||
|
||||
It is deliberately read-only: it changes nothing except optional cleanup of a
|
||||
device-wide proxy setting and port forwards it created itself.
|
||||
|
||||
Usage:
|
||||
python preflight.py
|
||||
python preflight.py --serial <serial> --pkg com.example.app
|
||||
python preflight.py --pkg com.example.app --expect-root
|
||||
python preflight.py --cleanup # remove proxy + frida port forwards it finds
|
||||
python preflight.py --json # machine-readable summary
|
||||
|
||||
Exit code is 0 when no BLOCKER was found, 1 otherwise. WARN never fails the run.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
BLOCKER, WARN, OK, INFO = "BLOCKER", "WARN", "OK", "INFO"
|
||||
|
||||
|
||||
class Report:
|
||||
def __init__(self) -> None:
|
||||
self.rows: list[tuple[str, str, str, str]] = []
|
||||
|
||||
def add(self, level: str, area: str, detail: str, fix: str = "") -> None:
|
||||
self.rows.append((level, area, detail, fix))
|
||||
|
||||
def worst(self) -> str:
|
||||
for lv in (BLOCKER, WARN, OK, INFO):
|
||||
if any(r[0] == lv for r in self.rows):
|
||||
return lv
|
||||
return OK
|
||||
|
||||
|
||||
def run(cmd: list[str], timeout: int = 60) -> tuple[int, str]:
|
||||
try:
|
||||
p = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=timeout)
|
||||
return p.returncode, (p.stdout or "") + (p.stderr or "")
|
||||
except FileNotFoundError:
|
||||
return 127, "not found: %s" % cmd[0]
|
||||
except subprocess.TimeoutExpired:
|
||||
return 124, "TIMEOUT after %ss: %s" % (timeout, " ".join(cmd[:4]))
|
||||
except Exception as e: # pragma: no cover - defensive
|
||||
return 1, "error: %s" % e
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--adb", default=os.environ.get("ADB", "adb"), help="adb executable (default: $ADB or 'adb')")
|
||||
ap.add_argument("--serial", default=None, help="device serial; required when more than one is attached")
|
||||
ap.add_argument("--pkg", default=None, help="target package to check (optional)")
|
||||
ap.add_argument("--expect-root", action="store_true", help="treat missing root as a BLOCKER")
|
||||
ap.add_argument("--cleanup", action="store_true", help="remove leftover proxy setting and frida forwards")
|
||||
ap.add_argument("--json", action="store_true", dest="as_json")
|
||||
a = ap.parse_args()
|
||||
|
||||
r = Report()
|
||||
ADB = a.adb
|
||||
|
||||
# ---- host tools -------------------------------------------------------
|
||||
for tool, why in (
|
||||
("java", "dexlib2 patcher, apksigner"),
|
||||
("python", "all scripts"),
|
||||
):
|
||||
found = shutil.which(tool)
|
||||
r.add(OK if found else WARN, "host:" + tool, found or "not on PATH", "" if found else "install it if you need " + why)
|
||||
for tool in ("apksigner", "zipalign", "aapt", "aapt2"):
|
||||
found = shutil.which(tool)
|
||||
r.add(OK if found else INFO, "host:" + tool, found or "not on PATH",
|
||||
"" if found else "only needed for signing/manifest work; build-tools on PATH helps")
|
||||
|
||||
# ---- adb + device selection ------------------------------------------
|
||||
if shutil.which(ADB) is None and not os.path.exists(ADB):
|
||||
r.add(BLOCKER, "adb", "executable not found: %s" % ADB, "pass --adb <path> or set $ADB")
|
||||
return finish(r, a.as_json)
|
||||
|
||||
rc, out = run([ADB, "devices", "-l"])
|
||||
if rc != 0:
|
||||
r.add(BLOCKER, "adb", "adb devices failed: %s" % out.strip()[:160], "start the adb server; check USB/emulator")
|
||||
return finish(r, a.as_json)
|
||||
|
||||
devices = []
|
||||
for line in out.splitlines()[1:]:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
parts = line.split()
|
||||
if len(parts) >= 2:
|
||||
devices.append((parts[0], parts[1], line))
|
||||
online = [d for d in devices if d[1] == "device"]
|
||||
|
||||
if not devices:
|
||||
r.add(BLOCKER, "device", "no device attached", "boot the emulator/connect the device, then re-run")
|
||||
return finish(r, a.as_json)
|
||||
if len(online) > 1 and not a.serial:
|
||||
r.add(BLOCKER, "device", "%d devices online but no --serial given" % len(online),
|
||||
"pass --serial explicitly, or adb will pick one at random and you will debug the wrong target")
|
||||
if not online:
|
||||
bad = ", ".join("%s(%s)" % (d[0], d[1]) for d in devices)
|
||||
r.add(BLOCKER, "device", "attached but not usable: %s" % bad,
|
||||
"unauthorized -> accept the USB debugging prompt; offline -> reconnect/reboot the emulator")
|
||||
return finish(r, a.as_json)
|
||||
|
||||
serial = a.serial or online[0][0]
|
||||
r.add(OK, "device", "using %s (of %d online)" % (serial, len(online)), "")
|
||||
|
||||
def sh(cmd: str, timeout: int = 60) -> str:
|
||||
return run([ADB, "-s", serial, "shell", cmd], timeout=timeout)[1]
|
||||
|
||||
def su(cmd: str, timeout: int = 90) -> str:
|
||||
return run([ADB, "-s", serial, "shell", 'su -c "%s"' % cmd], timeout=timeout)[1]
|
||||
|
||||
# ---- device facts -----------------------------------------------------
|
||||
abi = sh("getprop ro.product.cpu.abi").strip()
|
||||
abilist = sh("getprop ro.product.cpu.abilist").strip()
|
||||
rel = sh("getprop ro.build.version.release").strip()
|
||||
sdk = sh("getprop ro.build.version.sdk").strip()
|
||||
model = sh("getprop ro.product.model").strip()
|
||||
sec = sh("getenforce").strip()
|
||||
|
||||
r.add(OK, "device:os", "Android %s (sdk %s) %s" % (rel, sdk, model), "")
|
||||
r.add(OK, "device:abi", "%s abilist=[%s]" % (abi, abilist), "")
|
||||
r.add(OK if sec.lower() in ("permissive", "") else INFO, "device:selinux", sec or "?", "")
|
||||
|
||||
# translation layer / emulator hints -- these change how native code behaves
|
||||
trans = []
|
||||
for probe in ("ro.dalvik.vm.native.bridge", "ro.enable.native.bridge.exec", "ro.boot.native_bridge"):
|
||||
v = sh("getprop " + probe).strip()
|
||||
if v and v != "0":
|
||||
trans.append("%s=%s" % (probe, v))
|
||||
r.add(WARN if trans else OK, "device:translation",
|
||||
"; ".join(trans) if trans else "no ARM translation declared",
|
||||
"native arm libraries run through a translator here: timing differs and some native checks misbehave" if trans else "")
|
||||
if re.search(r"^1$", sh("getprop ro.kernel.qemu").strip()):
|
||||
r.add(INFO, "device:type", "emulator (ro.kernel.qemu=1)",
|
||||
"final verification belongs on the real ABI the user will run")
|
||||
|
||||
# ---- root -------------------------------------------------------------
|
||||
idout = su("id")
|
||||
has_root = "uid=0" in idout
|
||||
lvl = OK if has_root else (BLOCKER if a.expect_root else WARN)
|
||||
r.add(lvl, "root", "available" if has_root else "NOT available (%s)" % idout.strip()[:60],
|
||||
"" if has_root else "anything beyond static analysis needs root: data dirs, Frida, file locks")
|
||||
|
||||
# ---- clock (silently breaks TLS work) --------------------------------
|
||||
dev_epoch = sh("date +%s").strip()
|
||||
if dev_epoch.isdigit():
|
||||
drift = abs(int(dev_epoch) - int(__import__("time").time()))
|
||||
r.add(OK if drift < 120 else WARN, "clock",
|
||||
"device/host drift %ds" % drift,
|
||||
"" if drift < 120 else "an expired/wrong certificate can be nothing but this clock; fix before TLS triage")
|
||||
|
||||
# ---- leftover state that fakes a failure -----------------------------
|
||||
proxy = sh("settings get global http_proxy").strip()
|
||||
if proxy and proxy not in ("null", ":0"):
|
||||
r.add(WARN, "state:proxy", "device-wide http_proxy=%s" % proxy,
|
||||
"a leftover proxy makes every request fail; clear with: settings put global http_proxy :0")
|
||||
if a.cleanup:
|
||||
sh("settings put global http_proxy :0")
|
||||
r.add(INFO, "state:proxy", "cleared (--cleanup)", "")
|
||||
else:
|
||||
r.add(OK, "state:proxy", "not set", "")
|
||||
|
||||
rc_f, fwd = run([ADB, "-s", serial, "forward", "--list"])
|
||||
if fwd.strip():
|
||||
r.add(INFO, "state:forwards", fwd.strip().replace("\n", " | ")[:200],
|
||||
"stale forwards point frida at the wrong place; 'adb forward --remove-all' resets them")
|
||||
if a.cleanup:
|
||||
run([ADB, "-s", serial, "forward", "--remove-all"])
|
||||
r.add(INFO, "state:forwards", "removed (--cleanup)", "")
|
||||
|
||||
# ---- frida server reachability ---------------------------------------
|
||||
srv = su("pgrep -f frida-server || pgrep -f kwork")
|
||||
if srv.strip().isdigit() or srv.strip().split("\n")[0].strip().isdigit():
|
||||
r.add(OK, "frida:server", "device server process alive (pid %s)" % srv.strip().split("\n")[0].strip(), "")
|
||||
else:
|
||||
r.add(INFO, "frida:server", "no device server process found",
|
||||
"only needed for dynamic work; see references/dynamic-frida.md (name it out of obvious paths)")
|
||||
|
||||
# ---- target package ---------------------------------------------------
|
||||
if a.pkg:
|
||||
pm = sh("pm list packages | grep -F %s" % a.pkg)
|
||||
installed = ("package:" + a.pkg) in pm or a.pkg in pm
|
||||
r.add(OK if installed else INFO, "target", "installed" if installed else "not installed", "")
|
||||
if installed:
|
||||
path = sh("pm path %s" % a.pkg).strip()
|
||||
uid = su("dumpsys package %s | grep -m1 userId=" % a.pkg).strip()
|
||||
r.add(INFO, "target:apk", path[:160] or "?", "")
|
||||
r.add(INFO, "target:uid", uid or "?", "re-check after every reinstall: uid increments")
|
||||
# the native library dir the package manager will use for this device
|
||||
nat = sh("dumpsys package %s | grep -m3 primaryCpuAbi" % a.pkg).strip()
|
||||
if nat:
|
||||
r.add(INFO, "target:abi", nat.replace("\n", " | ")[:160],
|
||||
"this is the ABI the package manager chose for THIS device")
|
||||
maps = su("cat /proc/$(pidof %s | awk '{print $1}')/maps 2>/dev/null | head -1" % a.pkg)
|
||||
if maps.strip():
|
||||
r.add(INFO, "target:running", "process is running", "prefer a clean cold start before measuring anything")
|
||||
else:
|
||||
r.add(OK, "target:running", "not running", "")
|
||||
|
||||
# ---- free space -------------------------------------------------------
|
||||
# Report by device node, not by the mount-point column: a block device can be
|
||||
# mounted at more than one path (an ARM translation shim does exactly this on
|
||||
# some emulators), and `df` will then label /data with an unrelated path.
|
||||
space = ""
|
||||
for line in sh("df -h /data 2>/dev/null").splitlines()[1:]:
|
||||
cols = line.split()
|
||||
if len(cols) >= 5:
|
||||
space = "%s: %s available of %s (%s used)" % (cols[0], cols[3], cols[1], cols[4])
|
||||
break
|
||||
if space:
|
||||
r.add(INFO, "device:space", space, "")
|
||||
|
||||
return finish(r, a.as_json)
|
||||
|
||||
|
||||
def finish(r: Report, as_json: bool) -> int:
|
||||
if as_json:
|
||||
print(json.dumps({"worst": r.worst(),
|
||||
"rows": [dict(level=l, area=ar, detail=d, fix=f) for l, ar, d, f in r.rows]},
|
||||
indent=2, ensure_ascii=False))
|
||||
return 1 if r.worst() == BLOCKER else 0
|
||||
|
||||
width = max(len(x[1]) for x in r.rows) if r.rows else 10
|
||||
for level, area, detail, fix in r.rows:
|
||||
print("%-8s %-*s %s" % ("[" + level + "]", width, area, detail))
|
||||
if fix:
|
||||
print(" " * 9 + "%-*s -> %s" % (width, "", fix))
|
||||
print("\nworst: %s" % r.worst())
|
||||
if r.worst() == BLOCKER:
|
||||
print("Fix every BLOCKER before running another experiment. Do not attribute a failure to your patch yet.")
|
||||
return 1
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+203
@@ -0,0 +1,203 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Capture what is actually on screen -- and say whether the control tree is usable.
|
||||
|
||||
Why this exists
|
||||
---------------
|
||||
Driving a UI blind (tap a coordinate, wait, tap again) is the single most
|
||||
expensive habit in device work. A look at the screen resolves in one step what
|
||||
coordinate guessing cannot resolve in five: the layout moved, a different dialog
|
||||
is up, a countdown is frozen, a button is disabled, text says why.
|
||||
|
||||
Two independent kinds of evidence, and they are not interchangeable:
|
||||
|
||||
* the **image** -- always available, always truthful about what is rendered.
|
||||
It is the only evidence for UI drawn by a runtime or a web view, where no
|
||||
real controls exist.
|
||||
* the **control tree** (`uiautomator dump`) -- precise, diffable, gives exact
|
||||
bounds and text. Frequently EMPTY for canvas/webview/cross-platform UI.
|
||||
|
||||
This script grabs both, reports the size and whether the tree had content, so
|
||||
you know which one is actually informative before you rely on it. It never
|
||||
touches app state.
|
||||
|
||||
Usage
|
||||
-----
|
||||
# one look
|
||||
python snap.py --out shots --tag before
|
||||
|
||||
# watch a wait: 6 samples, 3 s apart (bounded: hard cap of 20 samples)
|
||||
python snap.py --out shots --tag waiting --count 6 --interval 3
|
||||
|
||||
# tree only, no images
|
||||
python snap.py --out shots --tag tree --tree-only
|
||||
|
||||
# when nothing is on screen, capture and it will say so rather than guess
|
||||
python snap.py --out shots --tag check --serial <serial>
|
||||
|
||||
Reading the output
|
||||
------------------
|
||||
* `bytes=0` after both capture paths -> the device refused; see the note in the
|
||||
output rather than concluding the screen is blank.
|
||||
* `tree=EMPTY` -> do not plan around accessibility bounds. Use the image and
|
||||
reason from pixels, or drive the app through a non-UI path.
|
||||
* Identical hashes across samples -> nothing is changing. Stop waiting and go
|
||||
find out why (this is the stall detector).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import os
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
|
||||
MAX_SAMPLES = 20 # hard cap so a typo cannot produce an unbounded loop
|
||||
|
||||
|
||||
def run(cmd: list[str], timeout: int) -> tuple[int, str, str]:
|
||||
try:
|
||||
p = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=timeout)
|
||||
return p.returncode, p.stdout or "", p.stderr or ""
|
||||
except subprocess.TimeoutExpired:
|
||||
return 124, "", "TIMEOUT after %ss: %s" % (timeout, " ".join(cmd[:4]))
|
||||
except FileNotFoundError:
|
||||
return 127, "", "not found: %s" % cmd[0]
|
||||
|
||||
|
||||
class Dev:
|
||||
def __init__(self, adb: str, serial: str | None):
|
||||
self.adb = adb
|
||||
self.base = [adb] + (["-s", serial] if serial else [])
|
||||
self.serial = serial
|
||||
|
||||
def sh(self, cmd: str, timeout: int = 60) -> str:
|
||||
return run(self.base + ["shell", cmd], timeout)[1]
|
||||
|
||||
def su(self, cmd: str, timeout: int = 90) -> str:
|
||||
return run(self.base + ["shell", 'su -c "%s"' % cmd], timeout)[1]
|
||||
|
||||
def pull(self, remote: str, local: str, timeout: int = 120) -> bool:
|
||||
rc, _, err = run(self.base + ["pull", remote, local], timeout)
|
||||
return rc == 0 and os.path.exists(local) and os.path.getsize(local) > 0
|
||||
|
||||
|
||||
def tree_stats(dev: Dev, workdir: str) -> tuple[str, int, int]:
|
||||
"""Return (status, node_count, text_count). status in {OK, EMPTY, FAILED}."""
|
||||
xml = "/data/local/tmp/_snap_ui.xml"
|
||||
dev.sh("rm -f " + xml)
|
||||
out = dev.su("uiautomator dump %s" % xml, timeout=90)
|
||||
if "dumped to" not in out and "UI hierchary" not in out and os.sep not in out:
|
||||
# some builds print nothing useful; fall through to the file check anyway
|
||||
pass
|
||||
local = os.path.join(workdir, "_snap_ui.xml")
|
||||
if not dev.pull(xml, local, timeout=120):
|
||||
return "FAILED", 0, 0
|
||||
try:
|
||||
data = open(local, encoding="utf-8", errors="replace").read()
|
||||
except Exception:
|
||||
return "FAILED", 0, 0
|
||||
nodes = len(re.findall(r"<node", data))
|
||||
texts = len([t for t in re.findall(r'text="([^"]*)"', data) if t.strip()])
|
||||
if nodes == 0:
|
||||
return "EMPTY", 0, 0
|
||||
# A tree with nodes but almost no text is usually a shell with no real widgets.
|
||||
return ("OK" if texts else "SPARSE"), nodes, texts
|
||||
|
||||
|
||||
def capture_image(dev: Dev, workdir: str, name: str) -> tuple[int, str]:
|
||||
remote = "/data/local/tmp/%s.png" % name
|
||||
dev.su("rm -f " + remote)
|
||||
# on-device write then pull: the streaming form returns 0 bytes on some ROMs
|
||||
dev.su("screencap -p %s" % remote, timeout=90)
|
||||
local = os.path.join(workdir, name + ".png")
|
||||
if dev.pull(remote, local, timeout=120):
|
||||
n = os.path.getsize(local)
|
||||
if n > 0:
|
||||
return n, hashlib.sha256(open(local, "rb").read()).hexdigest()[:12]
|
||||
# fallback: stream to host
|
||||
rc, out, err = run(dev.base + ["exec-out", "screencap -p"], timeout=120)
|
||||
if rc == 0:
|
||||
data = out.encode("latin1", "ignore") if isinstance(out, str) else b""
|
||||
if data:
|
||||
open(local, "wb").write(data)
|
||||
return len(data), hashlib.sha256(data).hexdigest()[:12]
|
||||
return 0, ""
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--adb", default=os.environ.get("ADB", "adb"))
|
||||
ap.add_argument("--serial", default=None, help="device serial; required when several are attached")
|
||||
ap.add_argument("--out", default="snapshots", help="output directory (created if missing)")
|
||||
ap.add_argument("--tag", default="snap", help="label used in filenames")
|
||||
ap.add_argument("--count", type=int, default=1, help="samples to take (capped at %d)" % MAX_SAMPLES)
|
||||
ap.add_argument("--interval", type=float, default=2.0, help="seconds between samples")
|
||||
ap.add_argument("--tree-only", action="store_true", help="skip images; only read the control tree")
|
||||
ap.add_argument("--image-only", action="store_true", help="skip the control tree")
|
||||
a = ap.parse_args()
|
||||
|
||||
if a.count > MAX_SAMPLES:
|
||||
print("count capped at %d (asked for %d)" % (MAX_SAMPLES, a.count), file=sys.stderr)
|
||||
a.count = MAX_SAMPLES
|
||||
|
||||
workdir = os.path.abspath(a.out)
|
||||
os.makedirs(workdir, exist_ok=True)
|
||||
dev = Dev(a.adb, a.serial)
|
||||
|
||||
# cheap reachability gate with a timeout, so we fail fast and informatively
|
||||
rc, out, err = run(dev.base + ["shell", "echo __ok__"], timeout=30)
|
||||
if "__ok__" not in out:
|
||||
print("device not answering: %s%s" % (out.strip()[:120], (" / " + err.strip()[:120]) if err else ""),
|
||||
file=sys.stderr)
|
||||
print("check `adb devices`; if the emulator process is alive but nothing is listening on the adb port,",
|
||||
file=sys.stderr)
|
||||
print("restart the instance through the vendor console (see references/environment.md).", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
tree_status = "skipped"
|
||||
nodes = texts = 0
|
||||
if not a.image_only:
|
||||
tree_status, nodes, texts = tree_stats(dev, workdir)
|
||||
print("control tree: %s (nodes=%d, non-empty text=%d)" % (tree_status, nodes, texts))
|
||||
if tree_status in ("EMPTY", "FAILED"):
|
||||
print(" -> the accessibility tree is not usable here. Do NOT plan around bounds from it.")
|
||||
print(" The image is the primary evidence for this UI. Cross-platform runtimes, web views")
|
||||
print(" and canvas-drawn UI frequently expose no real controls at all.")
|
||||
elif tree_status == "SPARSE":
|
||||
print(" -> tree exists but carries almost no text; treat it as unreliable and read the image.")
|
||||
|
||||
prev_hash = None
|
||||
same_run = 0
|
||||
for i in range(a.count):
|
||||
tag = a.tag if a.count == 1 else "%s_%02d" % (a.tag, i)
|
||||
if a.tree_only:
|
||||
print("%-14s tree-only sample" % tag)
|
||||
else:
|
||||
size, digest = capture_image(dev, workdir, tag)
|
||||
if size == 0:
|
||||
print("%-14s bytes=0 -> device refused the capture. Not evidence that the screen is blank." % tag)
|
||||
else:
|
||||
note = ""
|
||||
if digest == prev_hash:
|
||||
same_run += 1
|
||||
note = " (identical to previous -> nothing changed)"
|
||||
else:
|
||||
same_run = 0
|
||||
prev_hash = digest
|
||||
print("%-14s bytes=%-8d sha=%s%s" % (tag, size, digest, note))
|
||||
print("%-14s %s" % ("", os.path.join(workdir, tag + ".png")))
|
||||
if same_run >= 2:
|
||||
print(" -> %d identical samples in a row. This is a stall, not a slow operation." % (same_run + 1))
|
||||
print(" Stop waiting: inspect the image, or check the app is still alive and in the foreground.")
|
||||
if i < a.count - 1:
|
||||
time.sleep(max(0.0, a.interval))
|
||||
|
||||
print("\nNow LOOK at the image(s) before deciding the next action.")
|
||||
print("Driving blind and waiting is how rounds get wasted; the screen usually states the reason.")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user