diff --git a/README.md b/README.md index f072d84..f8d94c5 100644 --- a/README.md +++ b/README.md @@ -117,7 +117,10 @@ references/ loaded on demand, one topic each find logic when there are no symbols (string encoding traps) dart-aot.md Dart AOT in depth: version pinning and building a matching decompiler, the object pool and reference indexes, register/boolean conventions, - the three signatures that identify business logic, locating, patching + the three signatures that identify business logic, locating, patching. + Begins with the snapshot-decoding front end it depends on (aotopsy or + blutter) because the pool listing is an input, not something this skill + produces itself native-and-so.md .so hosts, DT_NEEDED vs JNI_OnLoad, relocation limits, relocation-free bootstrapping, replacing Java methods natively, and which ABI/library is *actually loaded and executing* @@ -318,6 +321,17 @@ React Native/Hermes bytecode internals, and defeating a server-side authority ar covered, and the skill is written to say so and stop rather than apply the nearest documented procedure to a target it was not written for. +Two qualifications that the Coverage section states in full and that belong here too: + +- **Flutter/Dart AOT analysis has a dependency.** The workflow begins at a pool listing + (`pp.txt`-class output). Producing that needs a snapshot-decoding decompiler — aotopsy (a static + binary, no toolchain) or blutter (built from source, ~80 s) — and this repository does not contain + one. It is named as a prerequisite rather than left implicit. +- **Not every claim in this repository has a run behind it.** `docs/verification-jiongnew/` records + what was actually measured, on which target, and with which independent cross-check; anything not + covered there is documented from experience and should be read as *inferred*, per this skill's own + claim ladder. + ## Repository maintenance Three tools live at the root and are not part of the installed skill: @@ -329,3 +343,9 @@ check_refs.py every cross-reference that names a section of another document reaches a real heading in that document build_scripts.py audit for machine-specific leftovers (absolute paths, credentials) ``` + +`docs/verification-jiongnew/` is not part of the installed skill either. It is the evidence record +for one measurement pass against a real target: what each script actually did, which independent +method confirmed it, which defects were found, and which scenarios the target could not exercise. +It exists so the **Coverage** claims in `SKILL.md` can be checked against runs instead of trusted, +and so the gaps are written down where the next person will find them. diff --git a/docs/verification-jiongnew/FINDINGS.md b/docs/verification-jiongnew/FINDINGS.md new file mode 100644 index 0000000..f9ee540 --- /dev/null +++ b/docs/verification-jiongnew/FINDINGS.md @@ -0,0 +1,179 @@ +# Findings about the skill, from measuring it + +Everything here is a result about **the skill**, not about the target. Each item names the evidence +that produced it and its strength. + +--- + +## F1 — The Coverage claim was partly aspirational, and the pass identified exactly which part + +`SKILL.md`'s Coverage section lists as *covered by verified mechanisms*: + +> Flutter / Dart AOT: pinning the engine version, decoding the object pool, locating and patching +> business logic inside `libapp.so` (`dart-aot.md`). + +Measured against a real Dart 3.6.0 `libapp.so`, that sentence splits into three claims of very +different standing: + +| Sub-claim | Standing after this pass | +|---|---| +| pinning the engine version | **verified** — the banner route (§1) is correct and 3.6.0 is confirmed three ways | +| decoding the object pool | **false as written** — the kit cannot do it; there is no resolver | +| locating business logic inside `libapp.so` | **true, but conditional** — it works, on top of an external snapshot dumper that the skill neither ships nor installs | + +The worked example in `TOOL-VERDICTS.md` shows both halves at once: the chain +`"showRewardAdProvider"` → `pp+0x23548` → ref site `0x7a9464` → `ShowRewardAdFamily.call_2f28ac +@ 0x7a93ec` succeeded, and step 1 was performed by blutter, not by the skill. + +The reason is structural, not a shortage of effort: `dart_pool_strings.py` reports **file offsets** +while `pp.txt` and `dart_pprefs.py` work in **pool offsets**. Over the 4,241 strings present in both +spaces there are **4,237 distinct deltas** (min −1,361,429, max +277,654). The mapping is a property +of the reconstructed pool and is not a constant, so no amount of care inside the script recovers it. + +Capabilities the kit does not have, stated plainly: snapshot/object-pool deserialisation; symbol and +name recovery; a real call graph (the `--build-index` index holds raw `B`/`BL` targets only); a +decompiler; a function boundary/size table. + +--- + +## F2 — Two scripts fail in the way that reads as a target finding (the P26 class) + +The handover singled out `pitfalls.md` P26: *"a self-built analysis script fails in a way that reads +as a target finding."* This pass produced two live instances of exactly that, which is why the P26 +warning earns its place. + +**`elf_plt.py` returned a confident false negative.** It reports 3,437 stubs where `.rela.plt` holds +3,438, and answers `0x587060 -> not a resolved stub` for an entry that is a legitimate stub for +`dl_iterate_phdr`, with a real caller at `0x576a2c`. Nothing in the output indicates a problem. For +the script's stated purpose — deciding whether a hardening symbol exists in a library — the wrong +answer is indistinguishable from the right one. + +**`find_refs.py` returns a clean empty result for input it cannot read.** Its docstring advertises +"a smali tree or a directory of dex files"; `iter_files()` accepts only `.smali`. Given the dex +directory it printed `[total refs] 0`, exited **0**, and emitted no warning. This is worse than a +crash: it is the exact input to "nothing references this, safe to patch", produced by the very tool +recommended for judging blast radius before patching. + +Both, plus `dart_pprefs.py`'s silently dropped double constants, share one property: **the failure is +one-sided and quiet.** None of the three reports reduced confidence in its own output. + +--- + +## F3 — Defects found, ranked by consequence + +| # | Script | Defect | Consequence | Status | +|---|---|---|---|---| +| 1 | `find_refs.py` | `iter_files()` matches only `.smali`; dex dirs yield 0 results, no error, exit 0 | silent false "no references" on the tool whose job is blast-radius | **cause identified, not fixed** | +| 2 | `elf_plt.py` | loop bound `off < fsz - 16` misses the final stub slot | false negative for one real symbol per library, conditionally | **fixed on a copy, re-verified** | +| 3 | `dart_pprefs.py` | only the GP register file is decoded; `ldr dD,[x27,#imm]` (`0xFD400000`) and all double constants lost | under-reports pool references (302 displacements / 1,802 sites here), which breaks the "count refs to judge importance" advice in `dart-aot.md` §9/§10 — one site went `[0 refs]` → `[79 refs]` | **fixed on a copy; agrees with capstone 0/0** | +| 4 | `so_constpatch.py` | rebuild drops the zip `extra` field, destroying 4 KB alignment | for `extractNativeLibs=false` APKs the output will not load; masked here because this target is DEFLATED + `extractNativeLibs=true` | **cause identified, proven by fixture; not fixed** | +| 5 | `so_constpatch.py` | `--diff --name-regions` compares file offset with vaddr | feature completely inert on libraries where `p_vaddr != p_offset` (2 of 36 here), with no warning | cause identified | +| 6 | `so_constpatch.py` | `external_attr` rewritten on 1982/1982 entries | metadata-based integrity comparison detects the rebuild | cause identified | +| 7 | `dexutil.py` | switch payloads decoded as instructions; `decode()` does not resolve operands | caused a spurious "desync likely" verdict on `saveToLocal` | cause identified | +| 8 | `so_constpatch.py` | "N byte(s) changed" is the overwrite window, not bytes changed | overstates change count (13 reported vs 7 actual) | cause identified | +| 9 | `lib_map.py` | `?` means both "read failed" and "not probed" | a real read failure is invisible | cause identified | +| 10 | `dart_pool_strings.py` | TSV escaping is lossy for LF/CR/TAB | 7 of 18,363 rows ambiguous; 19 entries genuinely contain those bytes | cause identified | +| 11 | `elf_plt.py` | `size = 16 if ... else 16` | dead code | cause identified | +| 12 | `dex_strings.py` | `--per-file` prints only files with hits | can be misread as "only this dex was scanned" | cause identified | + +Items 1-3 are the ones that changed a conclusion during this pass rather than merely being noticed. + +--- + +## F4 — Documentation contradictions + +Independently found while mapping the skill and while using it. The executable cross-reference +checkers (`check_refs.py`, `check_repo.py`) pass, so these are invisible to the repo's own gates. + +| # | Location | Problem | +|---|---|---| +| 1 | `dart-aot.md` §2 | Claims blutter's `asm/` has *no instructions* — "Do not plan around it." **Wrong.** `asm/windmill_ad_plugin/src/ads/reward/reward.dart` is 1,327 lines carrying class layouts, signatures, a full instruction listing, pool annotations and resolved call targets. It is the richest artifact in the chain and the doc says to ignore it. | +| 2 | `dart-aot.md` §2 | Budgets "tens of minutes" for the first blutter build; measured **≈78 s** (~30× pessimistic). Also demands a very recent C++ compiler; **MSVC 19.34 (Nov 2022) suffices**. | +| 3 | `dart-aot.md` §7 | Presents two-byte hit chaining as an acceptance test. On this sample, 9,840 two-byte CJK candidates yield 2,081 chained runs, all garbage. | +| 4 | `signature-derived-keys.md:61` | Documents `python scripts/sig_probe.py --live --pkg ` — the script has no `--pkg`; it takes `--live PKG`. Line 54 and `pitfalls.md` P27 give the correct form. | +| 5 | `dart-aot.md:204` | `dart_disasm.py ... --index callers.json --build-index` omits the required argument value; argparse errors out. | +| 6 | `byte-level-patching.md:186` | Says different-length strings "need `dex_strpatch`-class tooling", while `dex_strpatch.py` requires `len(new) == len(old)` — it is precisely the tool that cannot do this. Contradicts `dex-patching.md:123`. | +| 7 | Coverage vs `toolchain.md` | Coverage lists Unity/IL2CPP logic recovery as **not covered**, but `toolchain.md` Tier 4 names concrete tooling without pointing back at that disclaimer, so a reader of `toolchain.md` alone would take IL2CPP for a supported route. | +| 8 | Script count | `SKILL.md`/`README.md` index 35 rows, which is 34 scripts plus the `scripts/dexpatch/` directory row. | +| 9 | `dart_pool_strings.py` docstring | Says two-byte auto-detection is unreliable (confirmed here); `dart-aot.md` §7 treats UTF-16LE hits as a reliable rule. | +| 10 | `verification.md` rung 7 | Re-downloading a distributed file to compare hashes has no landing place in `SKILL.md`'s six "done" criteria, so the two sections disagree on strength. | + +Item 1 is the most damaging because it steers a reader away from the best available evidence. + +--- + +## F5 — "verified mechanisms" left no reproducible trace in the repo + +The Coverage preamble frames its list as *verified mechanisms*, and `references/long-task-discipline.md` +defines **Observed** as *"reproduced it, with the exact command and output"*. The repository contains +no sample, no fixture, no captured log or screenshot, no patch record, no test and no CI, and its +`README.md` states it holds no target-specific data. + +So the strongest support the repository itself can exhibit for those claims is *Inferred*. This is +**not** an accusation that the claims are false — several were confirmed here on the first attempt, +and `dart-aot.md` §1 proved exactly right. It is a statement about evidentiary standing: the repo +asks its readers to label their own claims and does not leave itself a way to meet its own bar. + +The concrete measurement figures that *are* reproducible are the 14 sample-scoped numbers listed in +the skill map (e.g. dex 4.32 MB → 7.73 MB; `.RSA` 1199 B vs 777 B at runtime; a log tag going `8 → 0`). + +--- + +## F6 — Boundary evidence: where the skill's method held, and where it ran out + +This is the observation the pass was built to produce, so it is stated as a sequence rather than a +verdict. + +**Held, unchanged:** manifest and packer triage; multi-dex string/class recon at 27.6 MB across four +dex files (`dex_strings.py`, `dexutil.py`); native ELF import mapping at scale (36,974 relocations); +live-process library mapping; equal-length `.so` constant patching; the launch-timeline methodology +(`coldstart.py`); the P26 discipline itself — checking whether a tool was ever able to answer before +reading its silence as a finding, which is what turned the blutter crash from a false "unusable" into +a corrected "works after a relink". + +**Ran out:** from the Dart snapshot inward. Everything past `pp.txt` is fine; getting *to* `pp.txt` +from the binary requires a container resolver the kit does not have. That is one dependency, not a +subject area. + +**Not exercised at all:** packers, code virtualization, custom linkers, integrity-check redirection, +tamper-triggered suicide. This target has none of those, and the unmodified app's own startup failure +also removes the repack loop those scenarios need. + +--- + +## F7 — A trap in this specific environment that will produce false confirmations + +`androguard` 4.x (the version installed here) has **four independent reference-analysis failures on +this sample**: `get_xref_from()` returns 0 for everything, including `AdLoader.loadAd()V` which has +20+ callers by other means; `get_ref()` raises; a full `get_operands()` sweep returns 0 strings. + +Anyone who reaches for androguard to double-check a "zero references" conclusion will get **zero, and +therefore agreement**. The zero-reference result in `TARGET-FACTS.md` does not come from androguard; +it comes from `dexutil.py` plus raw-byte re-checks of the bodies that would not decode cleanly. This +is worth a line in the environment documentation because it is a tool that lies in the direction of +the hypothesis being tested. + +--- + +## F8 — Reconciling the two failures: a genuine but unproven loader signal + +Recorded as a boundary between what was established and what was guessed, because collapsing them +would be its own error. + +The launch does show non-standard startup plumbing (observed): + +- `E LoaderLog: 10026 / 101004` on the device, `10020 / 101003` on the emulator — an SDK-side + loader logging numeric status codes +- `E Instrumentation: Uninitialized ActivityThread, likely app-created Instrumentation, disabling + AppComponentFactory` — the process is not taking the ordinary `ActivityThread` path +- `E m.tingfeng.too: Unknown bits set in runtime_flags: 0x40000000` + +And yet the dex-side packer verdict is clean: no hardening library in the native set, an ordinary +`application android:name` (`app.video.guoguo.GApplication`), no known packer markers. Those two +observations sit in tension. + +**Not resolved.** The `LoaderLog` emitter was not attributed to a specific class — the string exists +in `classes2.dex` and `classes3.dex` but the classes that declare it are third-party, and this pass +did not trace the call chain. Whether this plumbing is a benign logging SDK, part of the Flutter +plugin bootstrap, or a light-weight loader contributing to the abort is **unverified**. Stating it +as established would be exactly the kind of overreach this report is trying to document. diff --git a/docs/verification-jiongnew/README.md b/docs/verification-jiongnew/README.md new file mode 100644 index 0000000..989e377 --- /dev/null +++ b/docs/verification-jiongnew/README.md @@ -0,0 +1,29 @@ +# jiongnew.apk 实测记录 + +This directory is the **evidence record for the `apk-reverse` skill measured against a real +target**, not target-specific data the skill ships for reuse. It exists so the Coverage +statement in `SKILL.md` can be checked against the runs behind it, and so a later reader can +tell which claims were *observed*, which were *inferred*, and which are still *unverified*. + +Target: `jiongnew.apk` — Flutter AOT, 4 dex, 36 native libraries, no packer. +sha256 `0C2B063E6D1A3BD05777611ABF2F1A64056E290E4F6728C1B12DC202747DC645`, 74,394,920 B, 1,982 entries. + +The APK itself is **not** in this repository (`.gitignore` excludes `*.apk`). + +## Strength labels + +Every claim below carries one of three labels, used the same way `references/verification.md` +uses them: + +- **observed** — reproduced here, with the exact command and its output. +- **inferred** — follows from observed facts, but the step itself was not executed. +- **unverified** — assumed or reported by a tool and not independently confirmed. + +## Index + +| File | Covers | +|---|---| +| `TOOL-VERDICTS.md` | One verdict per tested script, with the independent cross-check behind it | +| `TARGET-FACTS.md` | What the target is: manifest, dex, natives, assets, versions, and what it does at launch | +| `FINDINGS.md` | Findings about the skill itself: defects, contradictions, boundary evidence | +| `REPO-DECISIONS.md` | What changed in the repo as a result, and what deliberately did not | diff --git a/docs/verification-jiongnew/REPO-DECISIONS.md b/docs/verification-jiongnew/REPO-DECISIONS.md new file mode 100644 index 0000000..f5522f2 --- /dev/null +++ b/docs/verification-jiongnew/REPO-DECISIONS.md @@ -0,0 +1,161 @@ +# Repository decisions, and the evidence behind them + +The pass was framed as a choice between two ways to split the skill. The measurement rejected the +framing, and this file records why, then states exactly what changed in the repository. + +--- + +## The decision: **no second skill.** Fix skill A instead. + +This is the "暂不建" outcome the brief explicitly permits, and it is supported by the evidence rather +than by preference. + +### Why the framing did not survive contact with the target + +The two candidate splits were: + +- **by entry point** — everything reached from an APK (including Flutter/IL2CPP) stays in A; only + standalone ELF goes to B. Cost: A keeps growing. +- **by logic location** — A keeps only logic genuinely in dex; Flutter, IL2CPP, hardened dex, native + and standalone ELF all move out. Cost: four references (~962 lines) relocate. + +`jiongnew` is exactly the case that was supposed to discriminate between them: Flutter (machine-code +side) whose entry point is an APK. It produced a third answer. + +**The problem is not where the content lives. It is one missing dependency.** Everything the kit does +*after* a snapshot dumper has produced `pp.txt` works correctly — the reference index, the +disassembly windows, the caller index, all verified against capstone. The single thing it cannot do +is get *from the binary to* `pp.txt`, because it has no snapshot container resolver, and that gap is +not bridgeable from inside the scripts (4,241 shared strings, 4,237 distinct deltas, no constant). + +A second skill would relocate Flutter documentation without adding the capability that is missing. +The gap would travel with the documents. Worse, it would split the Coverage claim in two and leave +the user of either skill less able to tell what actually works. + +### The supporting evidence, in order of weight + +1. **The kit works where it was tested.** `dart_disasm.py` decoded identically to capstone (32/32, + 96/96); the caller index re-derived independently agrees with symmetric difference **0**; + `dart_pool_strings.py` was byte-exact on 18,356 of 18,363 rows with all 7 residuals explained as + TSV escaping, and 12/12 random offsets re-located from raw bytes. These are not the results of a + tool that needs replacing. +2. **The blocker is a front end, and free front ends exist.** aotopsy ran the full pipeline out of + the box — 30,586 functions, 39,202 pool entries, 175,120 call edges, 95.7% BLR annotation — with + **no toolchain**: pure Go, no compiler, no Dart SDK. That is the "decoding the object pool" + capability, delivered as a binary. +3. **A split would contradict the Coverage section's own structure.** Coverage already lists Flutter + under *covered by verified mechanisms* and Unity/IL2CPP under *not covered*. The real defect is + that the Flutter line overstates what ships. The fix for an overstated claim is to correct the + claim, not to relocate it. +4. **The cost of the split lands on the wrong people.** Moving `dart-aot.md`, + `native-and-so.md`, `native-tamper-and-suicide.md` and `code-virtualization-and-custom-linkers.md` + would break 162 cross-references that currently resolve, and would move material that this pass + could not exercise at all (packers, VMP) on the strength of evidence gathered about something + else. + +### What the second skill *would* have been, stated so the decision can be revisited + +If a future pass ever does need one, the boundary the evidence points at is **"standalone machine-code +targets with no APK"** — a plain `.so`/ELF analysed on its own. That is the split the entry-point +option described, and it remains untested: this target is an APK, so no evidence for or against it +was produced here. The pass therefore declines to build it and says so plainly rather than inventing +a skeleton. + +--- + +## Changes made + +### `references/dart-aot.md` + +| Change | Reason | +|---|---| +| Documented a **pinned front end** (aotopsy v1.6.0, sha256 `5a90bcf2…fdc1`; Dart 2.10–3.13; ARM64 + x86_64) as the entry point for the workflow, with blutter as the alternative when a compiled Dart VM is acceptable | The workflow's step 1 was not performable with what the skill shipped. Measured: the kit cannot go from a string to a pool offset. | +| Corrected the build budget: **≈78 s measured**, not "tens of minutes" | Measured end to end, log timestamps 04:45:59 → 04:47:17 | +| Corrected the compiler requirement: **MSVC 19.34 (Nov 2022) suffices** | Verified by compiling and running a `std::format` probe with it | +| Removed the claim that `asm/` contains no instructions | Measured false: the file carries class layouts, signatures, a full instruction listing, pool annotations and resolved call targets | +| Recorded the version-probe trap: `aotopsy doctor` labels the snapshot **3.6.2**, the engine banner and snapshot hash say **3.6.0**, and 3.6.0 is correct | Three independent facts agree on 3.6.0; aotopsy's number is a structural profile label and must not be passed to a VM-compiling tool | +| Noted that the two-byte chaining test in §7 did not hold here | 9,840 two-byte CJK candidates → 2,081 chained runs, all junk | + +### `references/references-index` / script defects + +`dart_pprefs.py` is fixed in the repository. The defect — only the GP register file decoded, so +`ldr dD,[x27,#imm]` (`0xFD400000`) and an entire class of double constants were dropped — was +re-verified on the patched script against capstone over the whole `.text`: **0 displacements and 0 +sites of difference**, against 302 and 1,802 before. + +`find_refs.py`, `elf_plt.py` and the `so_constpatch.py` alignment defect are **documented but not +fixed**. The repository's rule is that nothing ships unrun, and the fixes for these were not +exercised against the real target in this pass: + +- `find_refs.py` — accepting `.dex` input means deciding how to read dex without baksmali/jadx + present, which is a design decision, not a one-line patch. +- `elf_plt.py` — the three-bound fix was verified on a **copy** (`work/native/tools/elf_plt_fixed.py`), + not in the repository, so the repository version is untouched. +- `so_constpatch.py` — the alignment fix needs a padding implementation and a zipalign story. + +Each is recorded in `docs/verification-jiongnew/FINDINGS.md` with its root cause and the fix sketch, +which is the honest form: the knowledge is preserved and the unrun code does not enter the tree. + +### `SKILL.md` — Coverage + +Rewritten so the Flutter line states what was measured: + +- **In coverage:** Flutter/Dart AOT analysis *given a snapshot dump* — pool-reference counting, + disassembly windows, caller indexing and patching — all verified on a real Dart 3.6.0 + `libapp.so`, with the front end named as a documented dependency. +- **Explicit new dependency line:** the kit does not decode the object pool; it requires a snapshot + dumper (aotopsy pinned, or blutter built from source) and says so instead of implying otherwise. +- **Unchanged:** Unity/IL2CPP, React Native/Hermes, iOS, defeating a server-side authority, and the + general-unpacker/anti-detection arms race all remain uncovered. +- **Added:** an explicit statement that packer, VMP and integrity-check scenarios were **not + exercised** by this pass, so no claim is upgraded on their behalf. + +### `SKILL.md` — the three hand-off points + +Written from the perspective that a hand-off is a pointer, and that duplicating content across two +places is how the two copies drift. Each one says what the other side owns. + +1. **JNI** — Java declaring a native method ↔ its `.so` implementation. Static linkage shows as + `Java_com_pkg_Class_method` and disappears under R8 short names; dynamic registration leaves the + symbol table empty and must be found at `RegisterNatives`. This target ships the static form + (`FlutterJNI.loadLibrary` appears in `classes3.dex`) but the pass did not trace a JNI boundary + end to end, so the section carries the **unverified** label. +2. **Hardening** — dex-side observation of a packer implies a native-side shell implementation. + **Not exercised here**: this target has no packer. Recorded as a pointer with that stated. +3. **Existing boundary documents** — `native-and-so.md` and `native-tamper-and-suicide.md` read + native anomalies *from the APK side* (immediate SIGSEGV after a repack, Java claiming the check + passed while the process dies). That belongs in A; deep native work is the other side. The new + section points each way rather than repeating either. + +### `docs/verification-jiongnew/` + +New directory holding this pass's evidence record. `README.md` states the folder is a *measurement +record, not shipped target data*, and carries the strength-label convention. It is referenced from +`SKILL.md` so a reader can check the Coverage claims against the runs behind them. + +The APK is **not** in the repository (`.gitignore` already excludes `*.apk`) and no credential or +token appears in any file. + +### `README.md` + +Updated for the directory tree, the install command and the division of labour between the skills, +plus a pointer to the verification record. + +--- + +## Verification + +All four gates were re-run against the final tree and their real output is in +`docs/verification-jiongnew/VERIFY-OUTPUT.md`: + +- `python -B check_repo.py` +- `python -B check_refs.py` +- `python -B build_scripts.py . ` +- `npx skills add newliver666/apk-reverse --list` + +Two behaviours of the checkers worth knowing, both established by experiment during this pass: +`check_repo.py` and `check_refs.py` have **no argparse**, so `--help` is silently ignored and the +full check runs; and running `check_repo.py` with `PYTHONIOENCODING=utf-8` set produces a spurious +failure (exit 1, one script reported as crashed) because its `subprocess.run(..., text=True)` does +not pin an encoding while the child emits UTF-8 — the parent then decodes as cp936 and the streams +come back empty. diff --git a/docs/verification-jiongnew/TARGET-FACTS.md b/docs/verification-jiongnew/TARGET-FACTS.md new file mode 100644 index 0000000..bb917fa --- /dev/null +++ b/docs/verification-jiongnew/TARGET-FACTS.md @@ -0,0 +1,284 @@ +# Target facts — `jiongnew.apk` + +Everything here is **observed** unless labelled otherwise. Method: read-only analysis of the APK +(`zipfile` + `androguard` 4.x + direct dex structure parsing) plus install/launch on two devices. +Nothing in this file comes from the handover document — where the handover and the measurements +disagree, the disagreement is recorded. + +``` +sha256 0C2B063E6D1A3BD05777611ABF2F1A64056E290E4F6728C1B12DC202747DC645 +size 74,394,920 B (70.95 MiB) +entries 1,982 (zip testzip() == None) +``` + +## Manifest + +Read with androguard's AXML parser. The handover document warned that hand-reading the binary AXML +gets the class-name prefix wrong; that warning held up — the parser is the reason these values are +trustworthy rather than the string scrape the handover was reduced to. + +| Field | Value | +|---|---| +| `package` | `com.tingfeng.tool` | +| `application android:name` | **`app.video.guoguo.GApplication`** | +| `main activity` | `app.video.guoguo.SplashActivity` | +| `versionCode` / `versionName` | `84` / `1.5.8.0` | +| `minSdk` / `targetSdk` / `compileSdk` | `24` / `35` / `34` | +| `extractNativeLibs` | `true` | +| `usesCleartextTraffic` | `true` | +| `debuggable` | absent (release build) | +| `allowBackup` | `false` | +| application tag count | 1 | + +Two questions the handover explicitly left open, now closed: + +1. **`application android:name` is `app.video.guoguo.GApplication` — not a packer stub.** + This is an ordinary app class. The handover's string scrape only surfaced a 穿山甲/pangle stub + and could not rule out a shell; the parsed manifest rules it out. Combined with the absence of + hardening-library names in the native list, this target has **no packer on the dex side** + (observed) — though see `FINDINGS.md` for the *loader* signal that muddies this. +2. **The app does not run.** It aborts during startup on every attempt, on two different devices. + Detail below; this is the single most consequential fact about the target. + +Counts: 115 activities, 14 services, 2 receivers, 18 providers, 28 permissions. +All non-app components belong to bundled ad SDKs (kwad/ksad, qq.e, baidu mobads, beizi, sigmob, +meishu, bytedance openlive, windmill) plus Flutter plugins (`io.flutter.plugins.*`, `urllauncher`, +`imagepicker`) and Sentry (`io.sentry.android.core.*`). + +## Dex + +| File | Bytes | Compressed | +|---|---|---| +| `classes.dex` | 8,636,420 | 3,243,680 | +| `classes2.dex` | 7,910,800 | 2,918,337 | +| `classes3.dex` | 7,067,828 | 2,503,205 | +| `classes4.dex` | 4,032,596 | 1,509,375 | +| **total** | **27,647,644** | 10,174,597 | + +Per-dex string-table sizes measured directly from the header (`string_ids_size` at `0x38`): +63,267 / 55,279 / 46,693 / 27,549. + +**Cross-check against the handover:** the four dex sizes match the handover's table to the byte. +Also present at APK root and not mentioned in the handover: `DebugProbesKt.bin`, +`kotlin-tooling-metadata.json`, `androidsupportmultidexversion.txt` — a Kotlin app. + +## Native libraries + +36 libraries, 18 per ABI (`arm64-v8a` + `armeabi-v7a`), total 92,097,048 B uncompressed. The +handover's 36/18+18 count reproduces exactly. + +Largest: `lib/armeabi-v7a/libapp.so` 13,730,388 · `lib/arm64-v8a/libapp.so` 12,256,160 · +`libjingle_peerconnection_so.so` (WebRTC/声网) 11,377,944 / 6,536,952 · `libflutter.so` +10,814,192 / 7,614,304 · `libcore.so` 6,723,088 / 4,937,444 · `libloader.so` 6,221,896 / 4,540,264 · +`libalivcffmpeg.so` 6,136,472 / 5,092,604 · `libsaasCorePlayer.so` 4,885,216 / 3,440,932 · +`libsentry.so` 681,784 / 440,816 · `liblua-core.so` 384,472 / 265,796 · +`libplt-base.so` 373,056 / 245,956 · `libsaasDownloader.so` 354,496 / 267,916 · +`libdevInfo.so` 280,632 / 153,228 · `libti-monitor.so` 115,872 / 66,324 · +`libsgcore.so` 92,360 / 112,500 · `libsentry-android.so` 16,832 / 12,080 · +`libzeus_direct_dex.so` 14,160 / 9,756 · `libpangleflipped.so` 10,064 / 9,756 · +`libdatastore_shared_counter.so` 7,112 / 4,416. + +Packer-name scan (rglob over `libjiagu`, `libDexHelper`, `libnesec`, `libmobisec`, `libsecexe`, +`libshella`, `libsgmain`, `libtup`, `libprotectClass`, `libexec.so`, `libexecmain`, `StubApp`, +`libnativehelper`, `libapktool`): **zero hits**. Consistent with the handover. + +`libloader.so` is the interesting one — 6.2 MB with the largest `.dynsym` in the set, and the +reason it was the right sample for `elf_plt.py`. See `TOOL-VERDICTS.md`. + +## Flutter / Dart + +Pinned from the engine banner, which is the authoritative source and does not depend on symbols: + +``` +lib/arm64-v8a/libflutter.so @ 0x1fe035: + 3.6.0 (stable) (Thu Dec 5 07:46:24 2024 -0800) on "android_arm64" +``` + +Snapshot identity, from blutter's own `extract_dart_info`: + +``` +Dart 3.6.0 +Snapshot f956f595844a2f845a55707faaaa51e4 +Target arm64 / android +Flags product no-code_comments no-dwarf_stack_traces_mode + dedup_instructions no-tsan no-msan compressed-pointers +``` + +So: **Dart 3.6.0, compiled with compressed pointers, in a PRODUCT build with code comments and +Dwarf stack traces stripped.** The `product` flag matters — it is the reason field and local names +are absent at the format level, not merely hard to recover. + +`libapp.so` is stripped: `.dynsym` is 144 bytes. Its ELF header is `EM_AARCH64` (e_machine `0xB7`) +for the arm64 ABI and `EM_ARM` (`0x28`) for the v7a one. + +No `assets/flutter_assets/version` file exists (21 entries under `flutter_assets/`), so the usual +cheap version probe fails here — the banner above is the reliable route. + +**Discrepancy worth recording:** `aotopsy doctor` reports **Dart 3.6.2** for the same +`libapp.so`, while the engine banner and the snapshot hash both say 3.6.0. See `FINDINGS.md`. + +## Assets + +| Asset | Bytes | Note | +|---|---|---| +| `assets/1274036544` | 6,632,594 | 穿山甲/pangle plugin container (`ZEUS_PLUGIN_PANGLE` in manifest) | +| `assets/gdt_plugin/gdtadv2.jar` | 2,238,863 | 腾讯优量汇 plugin (contains dex) | +| `assets/bdxadsdk.jar` | 1,302,180 | 百度 ad SDK | +| `assets/flutter_assets/assets/jiong_white.zip` | 986,623 | theme resources | +| `assets/flutter_assets/assets/jiong_black.zip` | 986,413 | theme resources | +| `assets/flutter_assets/NOTICES.Z` | 123,708 | Flutter licence bundle | +| `assets/reward.json` | 16,404 | ← see below | +| `assets/feed.json` | 16,251 | ← | +| `assets/splash.json` | 10,843 | ← | +| `assets/intertitial.json` | 10,592 | ← note the spelling | +| `assets/banner.json` | 9,151 | ← | +| `assets/draw.json` | 4,644 | **not in the handover inventory** | +| `assets/fullscreen.json` | 4,201 | **not in the handover inventory** | +| `assets/ksad_idc.json` | 327 | Kuaishou IDC list | +| `assets/dexopt/baseline.prof`, `.profm` | 4,235 / 385 | Kotlin/Android baseline profiles | + +Also `assets/flutter_assets/packages/luavm/lua/*.lua` (10 Lua files: dkjson, http, url, ftp, ltn12, +smtp, socket, tp, headers, mbox, mime) and `liblua-core.so` — a third logic surface beside dex and +Dart. Not pursued in this pass. + +The handover listed 5 ad-config JSONs; the measurement finds **7**. The two additions +(`draw.json`, `fullscreen.json`) are the same shape as the other five. + +**The ad-config JSONs are not ad configuration.** They are test fixtures bundled inside the +美数/meishu aggregation SDK, and their read path is dead code in this build. The load-bearing parts: + +- Reader: `classes3.dex` → `Lcom/meishu/sdk/core/utils/TestToolUtil;->getTest{Splash,Banner, + Intertitial,Fullscreen,Reward,Draw,Feed}Json(Landroid/content/Context;Z)Ljava/lang/String;` + → `Context.getAssets()` + `const-string/jumbo "splash.json"`. Seven methods, seven filenames. (observed) +- The only callers are `TestToolUtil.getJsonByType` and `AdLoader.getTestData()`, and **neither has + a single `invoke` referencing it anywhere in the four dex** (coverage: 175,829 method bodies with + code; the 362 bodies that decoded uncleanly were rechecked by raw byte search for the target + `method_idx` — zero hits). (observed) +- Dart cannot be the reader: `intertitial` (the distinctive misspelling) has **zero** hits in + `libapp.so` in both UTF-8 and UTF-16LE, so the Dart side cannot even form the filename. The files + also sit in `assets/` rather than `assets/flutter_assets/` and are absent from `AssetManifest.json`, + so `rootBundle` structurally cannot reach them. (observed) +- Content self-identifies as fixture data: `reward.title="测试激励视频"`, `feed.title="测试"`, + `splash.clk_area="测试测试测试"`, `appstore_id="1234567"`, monitor hosts contain `-demo`. (observed) +- **Runtime confirmation of the dead-path conclusion** (run independently by the lead after the + static analysis): `getExternalCacheDir()/.adConfig/jsonConfig/` does not exist, and no + `*adConfig*` / `*jsonConfig*` path exists anywhere under either the external or the private data + directory. The only directory the app did create under `cache/` is `sentry/`. (observed) +- No on/off field exists in any of the seven files: matching + `enable|show|switch|open|close|disable|isOpen|isPlay|visible|frequency|interval` yields + **zero** candidates. (observed) + +Consequence: deleting the seven JSONs is safe and pointless — no caller, so no dependency, so no +effect on ads. The handover's hypothesis that "the convergence point is probably the config +consumption layer" is **refuted** for this asset family. The real convergence point is a +two-layer aggregation boundary (Dart `windmill_ad_plugin` ↔ `com.windmill.windmill_ad_plugin` +MethodChannel at `WindmillAd.createAdInstance`, channel names +`com.windmill/{splash,reward,interstitial,banner,native}`; below that +`com.meishu.sdk.core.loader.AdLoader.loadAd` → `https://sdk.1rtb.net/sdk/req_ad` fanning out to +CSJ/GDT/KS/BAIDU/OPPO/HW/TopOn). (observed for the mechanism, inferred for its completeness) + +## Launch behaviour — the app does not start + +`observed`. This is the fact that constrains everything else. + +Installed and launched on the Android 11/arm64 physical device: + +``` +adb -s SSBYPJKFNVEU6PGA install -r -t jiongnew.apk -> Success (4.1 s) + codePath = /data/app/~~fDojk_Ku0C15jUkrmP36vA==/com.tingfeng.tool-gncxaby8OxxhG9E1TuF3aw== + primaryCpuAbi = arm64-v8a, versionCode 84, targetSdk 35 +adb -s SSBYPJKFNVEU6PGA shell am start -n com.tingfeng.tool/app.video.guoguo.SplashActivity +``` + +Timeline on the device (logcat): + +``` +04:41:59.217 Start proc 20512 for pre-top-activity {...SplashActivity} +04:41:59.280 E LoaderLog: 10026 / 101004 +04:41:59.307 E Instrumentation: Uninitialized ActivityThread, likely app-created + Instrumentation, disabling AppComponentFactory +04:41:59.765 I InputDispatcher: Focus entered window ... SplashActivity <- window shown +04:42:00.297 F libc: FORTIFY: pthread_mutex_lock called on a destroyed mutex (0x7260f77b30) +04:42:00.301 F OpenGLRenderer: Failed to set damage region ... EGL_NOT_INITIALIZED +04:42:00.352 F libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 20512 + (m.tingfeng.tool), pid 20512 +04:42:00.409 E DEBUG: failed to open directory /proc/20512/fd: Permission denied +04:42:00.410 F crash_dump64: failed to attach to thread 20512, already traced by 0 () +04:42:00.411 W tombstoned: crash socket received short read of length 0 (expected 12) +04:42:00.444 I Zygote: Process 20512 exited cleanly (0) +04:42:00.453 I ActivityManager: Process com.tingfeng.tool (pid 20512) has died: fg TOP +``` + +Survival time ≈ 1.1 s from window focus to abort, ~1.3 s from process start. + +**Environmental control (single variable changed):** the same APK on the other attached device — +`emulator-5554`, x86_64, SDK 28, a different ROM and a different ABI — reproduces the identical +signature: + +``` +05:42:54.501 E LoaderLog: 10020 / 101003 +05:42:55.547 F libc: FORTIFY: pthread_mutex_lock called on a destroyed mutex (0x7639156ff120) +05:42:55.855 I Zygote: Process 16507 exited cleanly (0) +``` + +Same loader line, same FORTIFY abort, ≈1.04 s vs ≈1.07 s. Two devices, two ABIs, two Android +versions, same death. This **rules out** target-specific-to-one-ROM behaviour and rules out a pure +root/Magisk detection as the sole cause. The crash is a property of the app, not of the device. + +**The crash channel is owned by a third party.** Three independent signals say so: + +1. `crash_dump64` cannot attach — `already traced by 0 ()`. Something else already holds the + thread via ptrace. +2. No tombstone is written. `/data/tombstones/` newest file is `tombstone_23` from 03:54, while + this crash happened at 04:42; `tombstoned` logs `crash socket received short read of length 0`. +3. `tombstoned` receives the request for a *different* pid than the one that died + (`received crash request for pid 23771` while the dying pid was 23813), and `crash_dump64` + can no longer find the threads (`failed to attach to thread ...: No such process`). + +The mechanism is visible in the emulator log immediately before the abort: Sentry's NDK layer is +live and installing itself — + +``` +05:42:54.877 V libnb: enter native_bridge2_getTrampoline + Java_io_sentry_ndk_NativeScope_nativeSetTrace, trampoline_addr 0x76391344a080 +05:42:54.878 V libnb: ...Java_io_sentry_ndk_NativeScope_nativeAddBreadcrumb, trampoline_addr ... +``` + +Sentry's native crash handler replaces the signal handlers; the platform handler then cannot +attach and the ordinary evidence trail (tombstone, backtrace, `AndroidRuntime` fatal block) is +destroyed. `libsentry.so` / `libsentry-android.so` are in the APK and +`io.sentry.android.core.SentryInitProvider` is a declared provider. (observed) + +**Root cause of the abort itself: not localized.** What is established is the abort site's *shape* +— a `pthread_mutex_lock` on an already-destroyed mutex, on the **main** thread (tid == pid), i.e. a +native lifecycle/teardown ordering bug reached from the startup path, not the fault of an +obfuscation trick or a deliberate anti-tamper abort. Attempts to get a symbolized stack were made +and failed for stated reasons: + +- Platform route: impossible. `crash_dump` is pre-empted (above); no tombstone exists to read. +- `setenforce 0` to relax SELinux: rejected — `setenforce: Couldn't set enforcing status to '0': + Invalid argument`. Permissive mode was not obtainable on this ROM. +- Frida spawn-gating (the documented workaround when platform crash handling is unavailable): + the device carries a disguised frida-server (`/data/local/tmp/odm_service`, listening + `127.0.0.1:27099`). `frida-ps -H 127.0.0.1:27099` enumerates processes, but `spawn` fails with + `Failed to spawn: connection closed`, so the pre-signal-handler window cannot be opened through + that server. **unverified** whether a plain frida-server would succeed where this one does not. + +So the honest statement is: *the app aborts deterministically in native code during startup, on the +main thread, and the platform's ability to show us where has been taken away by a bundled crash +reporter.* The `LoaderLog` line and the app-created `Instrumentation` indicate non-standard startup +plumbing is also in play, but their contribution to the abort is **inferred, not established** — +see `FINDINGS.md` for why that distinction was kept rather than collapsed. + +## What this target can and cannot exercise + +Kept explicit, because the useful output of this pass is partly a list of what was *not* testable. + +| Exercisable | Not exercisable | +|---|---| +| Flutter/Dart AOT analysis (Dart 3.6.0, object pool, call graph) | `native_crash.py`'s "artificially constructed crash" verdict — the crash here is a real internal fault; there is no decoy to detect | +| Multi-dex scale (27.6 MB over 4 files) | `so_constpatch.py`'s "redirect the checker library" scenario — there is **no integrity-check library** in this target | +| `elf_plt.py` at scale (`libloader.so`, 36,974 relocations) | Every hardening scenario in `packers.md` / `code-virtualization-and-custom-linkers.md` — no packer | +| `lib_map.py` against a live process | A **durable** live process at all — the 1.1 s window is the constraint | +| `dart_*` scripts on a real stripped `libapp.so` | Any repack/install verification loop — the unmodified app already fails to start, so there is no working baseline to regress against | +| Config-driven ad analysis (and its refutation) | — | diff --git a/docs/verification-jiongnew/TOOL-VERDICTS.md b/docs/verification-jiongnew/TOOL-VERDICTS.md new file mode 100644 index 0000000..8bd6e6f --- /dev/null +++ b/docs/verification-jiongnew/TOOL-VERDICTS.md @@ -0,0 +1,509 @@ +# Tool verdicts — measured on `jiongnew.apk` + +One verdict per script, drawn from the closed set: + +| Verdict | Means | +|---|---| +| **verified on real target** | Ran here, and its output was confirmed by an independent method that does not share code with it | +| **broken — fixed** | Was wrong; the defect is localized and a corrected copy was run and re-verified | +| **broken — cause identified, not fixed** | Was wrong; root cause established, no fix attempted | +| **unusable here because …** | Correct diagnosis, but this environment/target cannot exercise it | +| **not applicable to this sample** | The target lacks the feature the tool addresses | + +The distinction that governs the whole table: **a script that runs without error is not a script +that is right.** Every "verified" below names the independent cross-check, because "it didn't +crash" was not accepted as evidence anywhere in this pass. Equally, a tool failure was never read +as "the target does not contain that thing" — each failure was first checked against the question +of whether the tool was ever able to answer it. + +--- + +## Summary + +| Script | Verdict | +|---|---| +| `elf_plt.py` | **broken — fixed** (core stub→symbol mapping verified on real target; two defects, one proven to cause a false negative) | +| `lib_map.py` | **verified on real target** | +| `so_constpatch.py` | **verified on real target** for equal-length rewriting; **broken — cause identified, not fixed** for one sealed serious defect (zip alignment) | +| `dex_strings.py` | **verified on real target** (multi-dex traversal confirmed) | +| `dexutil.py` | **verified on real target** (used as the primary workhorse for the ad-chain analysis) | +| `find_refs.py` | **broken — cause identified, not fixed**, and it fails *silently* | +| `dart_pool_strings.py` | **verified on real target** | +| `dart_pprefs.py` | **broken — fixed** (defect proven; corrected copy agrees with capstone to zero) | +| `dart_disasm.py` | **verified on real target** | +| `coldstart.py` | **verified on real target** (the run produced the launch-crash finding) | +| `native_crash.py` | **not applicable to this sample** (no artificially constructed crash; see `TARGET-FACTS.md`) | +| `so_constpatch.py` redirected-checker scenario | **not applicable** — no integrity-check library exists in this target | +| All packer / VMP tooling | **not applicable** — no packer in this target | +| blutter (external) | **works** — build ≈78 s, transient first-run `0xC0000005` not reproducible after relink | +| aotopsy (external) | **works out of the box** — full pipeline on the same `libapp.so` + +Two of these verdicts matter more than the rest: `elf_plt.py` produced a **false negative** on a +symbol that exists and is called, and `find_refs.py` returns a clean "no references" answer for +input it cannot read at all. Both are the failure mode that looks like a finding. + +--- + +## `elf_plt.py` — broken — fixed + +**Claim under test:** walks `PT_LOAD`/`PT_DYNAMIC` directly, without depending on the section +table, maps PLT stubs to imported symbols, and diffs two `.so` files byte-wise. + +**Sample:** `lib/arm64-v8a/libloader.so` (6,221,896 B, largest `.dynsym` in the APK). + +**Run:** + +``` +python scripts/elf_plt.py work/native/samples/arm64-v8a_libloader.so + exit 0 790 ms 3440 lines + arch=aarch64 exec seg: vaddr=0x0 file=0x0 size=0x587070 + imported relocations=36974 stubs resolved=3437 +``` + +**Independent cross-check — four paths, none sharing code with the script:** + +1. **Relocation total.** `readelf -d` (pyelftools) reads `RELASZ=804864`, `PLTRELSZ=82512`, + `RELAENT=24`. `804864/24 = 33536` (`.rela.dyn`) + `82512/24 = 3438` (`.rela.plt`) `= 36974`. + Matches the script's `imported relocations=36974`. +2. **Symbol names, item by item.** pyelftools parses `.rela.plt` + `.dynsym` independently + (3,438 entries, all `R_AARCH64_JUMP_SLOT=1026`). Against the script's 3,437 output lines: + `name mismatches: 0`. The script's GOT set is a subset of the `.rela.plt` GOT set, with an + intersection of **0** against `.rela.dyn` — i.e. it does not mistake local relocations for + imports. Zero `?` names. +3. **lief 1.0.0** independently reports `pltgot_relocations = 3438`, equal to the pyelftools set, + with zero name disagreements. +4. **capstone 5.0.7 real disassembly** of all 3,437 reported stub addresses, 16 bytes each, with the + GOT recomputed from the decoded instructions: `shape_violations: 0` (every stub is + `adrp x16 / ldr x17,[x16,#off] / add x16,x16,#off / br x17`) and + `got_recompute_mismatch: 0`. + +Conclusion: the four hand-written bitfield decoders (`_decode_adrp`, `_decode_ldr_unsigned`, +`_decode_add_imm`, `_decode_br`) are **correct**, not merely non-crashing. + +### Defect 1 — loop bound, proven to cause a false negative + +`.rela.plt` has **3,438** entries; the script reports **3,437**. The arithmetic: + +- first `.plt` stub `0x579990` + 3438 × 16 = **0x587070**, exactly the executable segment end + (`size=0x587070`) +- the source reads `while off < fsz - 16`, so the largest offset examined is `0x58705c`; the final + slot `0x587060..0x58706f` is never checked. `fsz-16` would need to be `fsz-15`; the correct form + is `off + 16 <= fsz`. + +The skipped slot is a legitimate stub, proven by independent disassembly: + +``` +0x587060 adrp x16, #0x5c8000 +0x587064 ldr x17, [x16, #0x108] +0x587068 add x16, x16, #0x108 +0x58706c br x17 +recomputed GOT = 0x5c8108 -> .rela.plt entry for that GOT: dl_iterate_phdr +BL/B callers of the skipped stub: 1 ['0x576a2c'] +``` + +And the script therefore answers wrongly, with a confident negative: + +``` +$ elf_plt.py libloader.so --query 0x587060 +0x587060 -> not a resolved stub <-- FALSE NEGATIVE +``` + +This is the most important single result of the native pass. The script does not error, its output +looks complete, and for its stated purpose — *"which stub points at the hardening symbol I want to +neutralise"* — it reports that a real, called symbol (`dl_iterate_phdr`) does not exist in the +library. That is precisely the failure mode the script's own docstring warns about. + +**Fix**, applied to a copy and re-verified — three isomorphic bounds: + +``` +_stubs_x86: while i < fsz - 6 -> while i + 6 <= fsz +_stubs_arm64: while off < fsz - 16 -> while off + 16 <= fsz +find_branch_callers: while off < fsz - 4 -> while off + 4 <= fsz +``` + +After the fix: `stubs resolved=3438`, equal to the `.rela.plt` entry count. The diff against the +original output is **exactly two lines** — the header count and the single new +`0x00587060 -> dl_iterate_phdr (GOT 0x5c8108)` — with all other 3,437 lines byte-identical. +`--query 0x587060` now resolves correctly. + +### Defect 2 — `--diff --name-regions` compares a file offset against a virtual address + +`cmd_diff`'s `describe(off)` receives a **file offset** and compares it against stub **vaddrs** +(`va <= off < va + size`). Those coincide only when `p_vaddr == p_offset`. Two of the 36 libraries +in this target violate that: + +``` +lib/arm64-v8a/libflutter.so exec PT_LOAD vaddr=0x444880 offset=0x434880 (delta 0x10000) +lib/armeabi-v7a/libflutter.so exec PT_LOAD vaddr=0x277500 offset=0x267500 (delta 0x10000) +``` + +Demonstrated by mutating a byte inside the `abort` stub (stub vaddr `0x9f6050`, file offset +`0x9e6050`) and diffing: + +``` +$ elf_plt.py libflutter.so --diff flutter.mutated.so --name-regions + differing bytes=1 in 1 run(s) + off=0x9e6053 len=1 A: b0 + B: b1 + <-- the "<- PLT stub for abort" annotation is missing +``` + +Plain `--diff` and `--diff --name-regions` produce **identical** output, i.e. the feature is +completely inert on these two libraries and reports nothing about being inert. + +**Fix:** map the file offset back through `PT_LOAD` before comparing. + +**Also:** `size = 16 if elf_a.machine == EM_AARCH64 else 16` — both branches identical; dead code. + +**Credit where due:** the script's defensive warning fires correctly on a zero-stub target. +`libapp.so` (Dart AOT, 144-byte `.dynsym`) reports `stubs resolved=0` **and** prints an explicit +"do not read this as 'the library imports nothing'" message. That is exactly the right instinct — +it is unfortunate that the scanner then violates that principle itself, one slot early. + +--- + +## `lib_map.py` — verified on real target + +**Claim under test:** reads a live process's mapping table and reports which libraries are actually +loaded, at what base, with what real architecture. + +**Run:** `--pkg com.tingfeng.tool --serial SSBYPJKFNVEU6PGA`, exit 0, 635 ms, pid 27064. +Reported three actually-loaded app libraries — `libloader.so` (3 segments), +`libsentry-android.so` (2), `libsentry.so` (3). + +**Independent cross-check:** + +- `su -c cat /proc/27064/maps` parsed with an independent parser: path set, segment counts and + permission strings agree with **zero** differences. +- `od` read of the ELF header gives `b7 00` = AArch64, matching the script's architecture column. +- Scale check against `systemui`: **318 libraries**, compared item by item — path, segment count + and permissions all zero-difference; `--json` base addresses agree **318/318** with independent + computation. +- Translation-layer detection fires correctly on the emulator + (`/system/lib64/libhoudini.so` → `translation suspected: YES`). + +**Improvement, not a defect:** `?` in the architecture column is two-valued in the docs — +`read_elf_arch`'s docstring says `?` = read failed; the CLI's trailing note says `?` = not probed. +All 8 libraries printed as `?` were independently read and are AArch64, i.e. they exist and simply +were not probed. A genuine read failure is indistinguishable from "not probed". + +**Anomaly resolved as not-a-bug:** one crosscheck showed `libsentry.so 3 vs 4` segments. Four +snapshots established this as runtime mapping change (it is 3 segments at the instant of launch), +not a script defect. + +**The real constraint on this tool here:** the target lives ~1.1 s, so `lib_map.py` must run inside +that window. That is also why the app's own libraries are limited to three — `libapp.so` and +`libflutter.so` had not been loaded yet. + +--- + +## `so_constpatch.py` — verified (core) with one sealed serious defect + +**Claim under test:** equal-length rewriting of string constants inside a `.so`. + +**Verified by:** length unchanged at 6,221,896 bytes, sha256 changed, and an independent byte-wise +diff showing changes at exactly one site (`0x81593`, 7 bytes). `readelf -d` confirms the change +landed at the ELF semantic layer: the `NEEDED` entry changes from `libandroid.so` to +`libfoo0000.so`. Guards behave as designed — unequal-length input exits 1 and writes no file; `-o` +pointing at the input exits 2; non-isolated operation refuses by default. On a real APK copy: 3.5 s, +**all 1,982 entries preserved**, order preserved, only the target entry's CRC changed, original APK +sha256 unchanged before and after. All patching was done on copies; no sample was mutated. + +### Defect — zip alignment destroyed on rebuild (sealed, not fixed) + +The rebuild discards the local header `extra` field, which breaks 4 KB alignment. Measured on a +fixture: `data_offset 4096 (mod4096=0, extra_len=4031)` becomes `65 (mod4096=65, extra_len=0)`. + +**Why this is serious despite not firing here:** for an APK with `extractNativeLibs=false`, `.so` +entries must be `STORED` and 4 KB-aligned or the library will not load. This target cannot show the +bug — all 36 native libraries are `method=8` (DEFLATED) and `extractNativeLibs=true` (confirmed in +both the manifest and by the runtime path `/data/app/.../lib/arm64/` that `lib_map.py` observed). +So the defect is **masked by this sample and proven by a fixture**. That asymmetry is exactly why +"it worked on our target" is not a verification. + +**Fix sketch:** copy `extra`/`external_attr`/`internal_attr`/`create_system`/`comment` from the +source `ZipInfo` and implement padding; and state in the help text that a rebuild must be +re-zipaligned before signing. + +### Defect — entry metadata reset + +`external_attr` is rewritten on **1982/1982** entries (e.g. `2175008768 (0x81A40000)` → +`25165824 (0x01800000)`), and `create_system` changes from 3 (Unix) to 0 on 77 entries. Usually +harmless for installation, but it changes extracted permission bits and makes any +zip-metadata-based integrity comparison immediately detect the rebuild. + +### Defect — the change count is misleading + +`do_replace` prints `'%d byte(s) changed' % (len(applied) * len(old))`, which is the **overwrite +window length**, not the number of bytes that actually changed. Two measured examples: +`libandroid.so → libfoo0000.so` prints 13 but changes 7; `libandroid → libandroiX` prints 10 but +changes 1. For an attribution/audit tool this invites overstatement. + +### Minor + +Rebuilding changes the APK's own bytes (+2.2 MB on the sample, different deflate level). Worth a +note in the output that output size may differ from input. + +--- + +## `dex_strings.py` — verified on real target + +**The multi-dex question the handover flagged is answered: yes, directory mode walks all four dex +files.** `iter_dex()` globs `*.dex` for a directory; empirically +`--find 'json' --per-file` emits four lines (`classes.dex`, `2`, `3`, `4`), while single-file mode +emits one. + +Two traps worth knowing: + +- `--per-file` prints **only files that had a hit**. An output containing just `classes3.dex` does + not mean only `classes3.dex` was scanned. This is the shape of output most likely to be + misread as "only one dex contains this". +- `--per-file` scans **raw bytes** rather than the string table (source lines 124-130), so hits for + broad patterns include coincidental matches. + +Fast: 0.27 s over the full 27.6 MB. + +## `dexutil.py` — verified on real target + +The workhorse of the ad-chain analysis: symbol resolution, branch-target back-annotation, and a +`ended cleanly` verdict per method. Coverage: 175,829 method bodies with code, 213,180 `method_id`s. +It is the tool whose output the zero-reference conclusion rests on. + +Two defects, both found by working at this scale: + +- `decode()` does not resolve operands — string and method indices must be decoded by the caller. +- **Switch payloads are decoded as instructions**, contradicting the docstring's claim that + `ident 1..3` are recognized. This is the root cause of `saveToLocal` being reported as + "desync likely". + +## `find_refs.py` — broken — cause identified, not fixed, and it fails silently + +The docstring says it accepts "a smali tree or a directory of dex files". It does not: +`iter_files()` yields only files ending in `.smali`. Given a dex directory it returns **0 files, no +error, exit code 0**. + +This is the worst shape a defect can take in this particular tool, because `find_refs.py` is the +script recommended for judging blast radius *before* patching. Its silent zero is exactly the +input to a confident wrong conclusion — "nothing references this, safe to patch" — and the exit +code corroborates it. Because no baksmali/jadx is installed on this machine, it was **completely +unusable** for this target, and the analysis had to be built on `dexutil.py` instead. + +**Correction to a related assumption in this environment:** `readelf` on `PATH` resolves to +pyelftools' `readelf.py` and is *not* binutils — `--dyn-syms` and `-W` are rejected as unrecognized +arguments (exit 2). Likewise MinGW `objdump`/`nm` (binutils 2.28) reject every aarch64 ELF with +`File format not recognized`. Aarch64 cross-checks in this pass therefore used the **pyelftools, +lief and capstone library APIs**, not those CLIs. There is no `xxd`; hex work was done in Python. + +--- + +## Dart / Flutter scripts + +Verified facts, each independently cross-checked rather than accepted on exit code: + +**`dart_pool_strings.py` — verified on real target.** 18,363 rows recovered. Byte-exact re-check of +tag + payload: **18,356 exact, 7 mismatched**, and all 7 mismatches are a single known escape +ambiguity (`\n`/`\r`/`\t` written as the two characters backslash+n, indistinguishable in a TSV from +a literal backslash followed by n — 19 recovered entries genuinely contain LF/CR/TAB). 12 rows +sampled at random were independently re-located from the raw bytes; **all 12 reproduced**, with tag +at `tag_off` and payload at `tag_off + 1` exactly. Chain violations among run≥2 rows: **0**. + +**`dart_pprefs.py` — broken — fixed.** The first run produced a real defect: **302 displacements and +1,802 reference sites missing**. Correcting the mask makes it agree with capstone exactly: + +``` + repo (before) repo (fixed) capstone +displacements 31,843 32,145 32,145 +reference sites 97,571 99,373 99,373 +only-in-capstone 302 0 — +only-in-repo 0 0 — +``` + +**Root cause — corrected after deeper analysis, and worth recording because the first hypothesis +was wrong.** The obvious guess was an `x27` addressing mask. That is not it: `LDUR` with `Rn=x27` +occurs **zero** times in this snapshot. The actual cause is that the scanner decodes only the +**general-purpose register file** — `ldr dD, [x27, #imm]` (encoding `0xFD400000`) was never matched +at all, so an entire class of **double constants** loaded from the object pool was silently +dropped. Measured recall 98.19%, precision 100%; after the fix, capstone and the script agree +across the whole `.text` with 0/0. + +Why this is the expensive kind of bug: it breaks precisely the judgement the documentation tells +you to rely on in `dart-aot.md` §9/§10 — *count the references to decide whether a pool entry +matters*. At one measured site the count goes from `[0 refs]` to `[79 refs]`. A reader following +that advice would conclude a live, heavily-referenced object was dead. It affects all Dart 3.x +arm64 snapshots, and the failure is one-sided and silent. + +**`dart_disasm.py` — verified on real target.** Decoded output compared against capstone on the same +ranges: **32/32 and 96/96 instructions identical**, `VERDICT: identical decode`. The independently +re-derived caller index also agrees exactly: `callers.json targets=59904 sites=219181`, independent +re-derivation `targets=59904 sites=219181`, **symmetric difference of call sites: 0**. All B/BL +words in range confirmed by capstone: 27,615/27,615. + +**`dart_disasm.py` — verified on real target.** Decoded output compared against capstone on the same +ranges: **32/32 and 96/96 instructions identical**, `VERDICT: identical decode`. The independently +re-derived caller index also agrees exactly: `callers.json targets=59904 sites=219181`, independent +re-derivation `targets=59904 sites=219181`, **symmetric difference of call sites: 0**. + +**`dart_pool_strings.py` two-byte mode — unusable here because detection is unreliable.** With +`--two-byte`: 936,136 candidates, 57,049 kept (34,216 UTF-16LE). The script's own warning says the +detection is unreliable, and the independent measurement confirms it: 168,700 independent UTF-16LE +CJK runs, and of 9,840 two-byte CJK candidates only 2,081 form chained runs ≥2 — the rest are +coincidental. Sample output is noise (`誋讋讋讋讋`). The one-byte default is the right default. + +--- + +## `coldstart.py` — verified on real target + +Used exactly as documented — `--serial`, `--pkg`, `--activity`, `--duration`, `--interval`, +`--expect-activity`. It behaved as advertised and produced the finding that governs this whole +report: the app reaches a splash window and then aborts. The script's existence is why the launch +failure was caught as a *measurement* (timestamped foreground-activity and logcat timeline) rather +than as a vague impression, and its `--expect-activity` design directly addresses the failure it +names in its own docstring — a vendor installer window being captured and read as the app. + +--- + +## External toolchains + +Both Dart AOT analyzers were tested, and they behave differently enough that the difference is +itself a finding. + +### blutter — **works; the first-run crash is not reproducible** + +`blutter` compiles and embeds a matching Dart VM, so it needs one build per Dart version. Building +from source is **mandatory** — the repository's `bin/` is gitignored and it publishes **no releases**, +so there is no prebuilt Dart 3.6.0 executable to fetch. (Naming, for anyone searching: +`bin/blutter_dartvm3.6.0_android_arm64.exe`, not `_windows_x64`.) + +Measured cost, which corrects the documentation: + +| Step | Time | +|---|---| +| `git clone --depth 1 worawit/blutter` (HEAD `4a60ac6`) | seconds | +| `python scripts\init_env_win.py` (ICU 73.2 + capstone 4.0.2) | **7,049 ms** | +| sparse clone `dart-lang/sdk` tag `3.6.0` | seconds | +| cmake + ninja, 22 targets (unity build via `dartvm_create_srclist.py`) | — | +| link + install `bin/blutter_dartvm3.6.0_android_arm64.exe` | — | +| **whole pipeline, wall clock** | **≈ 78 s** | + +`dart-aot.md` §2 budgets "**tens of minutes** for the first build". Reality here is **≈78 s**, +roughly **30× pessimistic**. The toolchain requirement is also overstated: the README asks for a +very recent C++ compiler with C++20 `` and the doc repeats "VS 2022 required", but +**MSVC 19.34 (VS 17.4.3, Nov 2022) is sufficient** — verified by compiling and running a +`std::format` probe with it. + +**The crash, and a correction to the first reading.** Run 1 (`blutter.py` under `vcvars64`) built the +VM and linked the exe, then died with exit `3221225477` = **0xC0000005 ACCESS_VIOLATION** and **no +output at all**. Runs 2-4 (invoking the installed exe directly, on `libapp.so`, `libflutter.so` and a +nonexistent path) reproduced it in ~89 ms each — always after argument parsing succeeded (a bad ABI +still prints `ELF: Support only 64 bits` and exits 0) and before the first stdout line, pointing at +`DartApp app{ libappPath.c_str() }` (`blutter/src/main.cpp:29`). + +Adding two `std::cerr` traces and relaunching `ninja` (2 targets: recompile `main.cpp`, relink) +produced an executable that **ran to completion, exit 0** (`libapp is loaded at 0x1dfe6d60000 / +Dart heap at 0x1e000000000`) — and then the **originally installed binary also ran clean**: 3/3 runs, +exit 0, 5,899 / 5,862 / 5,928 ms, each producing `pp.txt` (2,639,894 B), `objs.txt` (648,382 B), +`asm/`, and `blutter_frida.js`. + +So the verdict is **not** "unusable here". The root cause of the transient fault is **not identified** +and it did not reproduce after the relink, including for the unchanged binary; the best-supported +reading is a first-run/environment artifact rather than a property of this snapshot or of Dart 3.6.0. +Cleared suspects: `std::format` works on MSVC 19.34; capstone 4.0.2 and ICU 73.2 DLLs sit beside the +exe; `--no-analysis` is unnecessary because the full path works. The practical rule this yields: +**budget one retry, and if it still faults, relink before concluding anything about the sample** — +here a tool crash did not mean the target lacked anything. + +### aotopsy — **works out of the box** + +`aotopsy_v1.6.0_windows_amd64.zip`, 2,994,902 B, sha256 +`5a90bcf27f4aaa281ae393bf3038b6e1e2325386fe61c1a18ed089390983fdc1` (matches the release manifest). +Unpack and run — **no compiler, no Dart SDK, no toolchain**, because it parses the snapshot format +directly in Go. + +| | result | +|---|---| +| `doctor` | Support **OK**; compressed 4-byte pointers; hash `f956f595844a2f845a55707faaaa51e4`; the same 9 feature flags blutter derived | +| full pipeline | exit 0 — **30,586 functions**, 6,133 class layouts, **39,202 pool entries (38,274 resolved)**, **175,120 call edges** (18,718 BLR, **95.7% annotated**), 20,927 string refs, 556 signal findings | +| artifacts | `functions.jsonl`, `classes.jsonl`, `call_edges.jsonl`, `string_refs.jsonl`, `dispatch_table.jsonl`, `evidence.jsonl`, `asm/` (16,046 files), a `--decompile` pseudocode mode, `_debug strings --xref` | + +**The Dart version discrepancy, resolved.** aotopsy `doctor` says **3.6.2**; blutter derives **3.6.0** +from the engine banner; the banner reads `3.6.0 (stable) (Thu Dec 5 07:46:24 2024 -0800)`; and there +is **no `3.6.2` byte sequence anywhere in `libflutter.so` or `libapp.so`**. + +Judgement: **3.6.0 is the correct SDK version (observed)** — three independent facts agree (byte +search, blutter's parser, and the fact that the Dart 3.6.0 runtime blutter built accepts the snapshot +hash). aotopsy's `3.6.2` is its own **structural profile label (inferred)**: its README states version +detection is structure-based because the snapshot carries a git-derived hash rather than a version +number, so the label names the newest profile matching that hash. Impact on support is nil, but +**`aotopsy doctor`'s number must not be fed to a VM-compiling tool as an SDK tag.** The banner route +taught in `dart-aot.md` §1 is correct and is the best-validated instruction in that document. + +--- + +## End-to-end: can the kit locate one specific piece of business logic? + +**Yes — with an external snapshot dumper in front of it.** This is the question the whole pass was +built to answer, so the worked example is recorded in full. + +| # | Tool | Result | +|---|---|---| +| 1 | blutter `pp.txt` | `[pp+0x23548] String: "showRewardAdProvider"` | +| 2 | `dart_pprefs.py --lookup pp_refs_fixed.json 0x23548` | **1 ref site: `0x7a9464`** | +| 3 | `dart_disasm.py libapp.so --pp bout5/pp.txt --refs pp_refs_fixed.json 0x7a9464` | annotated window (below) | +| 4 | `dart_disasm.py --index callers.json 0x7a93ec` | **9 callers** | +| 5 | independent confirmation | `aotopsy _debug strings --find rewardAd --xref` → `used in: ShowRewardAdFamily.call_2f28ac @ 0x7a93ec (pool load @ 0x7a9464, pool[18087])` | + +Step 3 resolves the site completely, including the enclosing Dart closure and its source file: + +``` +7a943c: add x1, x27, #0x23, lsl #12 +7a9440: ldr x1, [x1, #0x540] ; pp+0x23540 AnonymousClosure: (0x7a9508), + ; of [package:guoguo/pages/video/video_page.dart] ShowRewardAdProvider +7a9454: add x1, x27, #0x23, lsl #12 +7a9458: ldr x1, [x1, #0x430] ; pp+0x23430 Obj!ShowRewardAdFamily@b487a1 : { [8 refs] +``` + +Two analyzers built by unrelated projects agree on the function name, the address and the pool index +— which is what makes this a verification rather than a coincidence. + +**But the kit cannot take step 1 by itself.** Reaching a pool offset from a string requires resolving +the snapshot's internal container, and the kit has no such resolver: 4,241 strings share a name across +4,237 distinct deltas with no constant bridge between the string table and the object pool. blutter +(or aotopsy) supplies that front end. Everything downstream of it — the cross-reference count, the +disassembly window, the caller index — the kit does do, and correctly. + +### Documentation defect that matters more than the crash + +`dart-aot.md` §2 says of blutter's `asm/`: *"per-class declarations — instructions — **there are none +here**. Do not plan around it."* That is **wrong** for this blutter HEAD on Dart 3.6.0. +`bout5/asm/windmill_ad_plugin/src/ads/reward/reward.dart` is 1,327 lines / 71,099 B and contains class +layouts, function signatures **and a full instruction listing with pool annotations and resolved call +targets**: + +``` +// 0x923b3c: r0 = LinkedHashMap.from() +// 0x923b3c: bl #0x60165c ; [dart:collection] LinkedHashMap::LinkedHashMap.from +``` + +`asm/` is the single richest artifact in the chain, and the document tells the reader to ignore it. +(Secondary nit: `asm/` is one file per **library URI**, mirroring the package path, not per class.) + +## Not applicable to this sample — stated, not stretched + +- **`native_crash.py`'s "artificially constructed crash" verdict.** The tool exists to decide + whether a crash was *manufactured* as an anti-tamper response. This target's crash is a genuine + internal mutex-lifecycle fault that occurs on the unmodified APK, before any modification, on two + devices. There is no decoy to detect and no tamper event to attribute, so the judgement the tool + makes was never exercised. **Not applicable — no artificial crash exists here.** +- **`so_constpatch.py`'s "redirect the checker library" use case.** Requires an integrity-check + library to subvert. This target has **none** — no packer, no integrity checker, no `.so` that + validates the APK. The redirection capability is real (see the verified `NEEDED` rewrite) but its + purpose-built scenario does not exist here. +- **`packers.md` / `code-virtualization-and-custom-linkers.md` in full.** No packer: zero hits for + every known hardening-library name, and the parsed manifest shows an ordinary application class + rather than a shell stub. **Unverified in this pass** — nothing here should be read as evidence + either way about how those documents perform. +- **`native-tamper-and-suicide.md`'s judgement chain.** It reads *native anomalies observed from the + APK side* (immediate SIGSEGV after a repack, the Java layer saying the check passed while the + process dies). This pass never produced a repack to observe, because **the unmodified app does not + start**, so there is no working baseline to regress from. The document's scenarios remain + unexercised. diff --git a/docs/verification-jiongnew/VERIFY-OUTPUT.md b/docs/verification-jiongnew/VERIFY-OUTPUT.md new file mode 100644 index 0000000..b7a958c --- /dev/null +++ b/docs/verification-jiongnew/VERIFY-OUTPUT.md @@ -0,0 +1,198 @@ +# Verification output + +The four repository gates, run against the tree **after** the changes described in +`REPO-DECISIONS.md`. The text below is the captured output, not a restatement of it — a table +saying "passed" is exactly the kind of claim this repository asks people not to make. + +Environment: Python 3.14.0 on Windows 11, `npx` 11.6.2, repository at commit +`46812ec` plus the working-tree changes listed in `REPO-DECISIONS.md`. + +| # | Command | Exit | Result | +|---|---|---|---| +| 1 | `python -B check_repo.py` | 0 | 0 problems | +| 2 | `python -B check_refs.py` | 0 | 0 dangling | +| 3 | `python -B build_scripts.py . ` | 0 | 0 findings | +| 4 | `npx skills add newliver666/apk-reverse --list` | 0 | Found 1 skill | + +--- + +## 1. python -B check_repo.py + +Every skill discovered, frontmatter valid, each script runnable, documented paths resolve, README paths explicit and existing. + +Exit code: **0**. Output (47 lines): + +``` +== skills discovered == + skills/apk-reverse + +== frontmatter == + 1 checked + +== apk-reverse: python syntax + --help == + apk_diff.py ok + blob_decode.py ok + coldstart.py ok + dart_disasm.py ok + dart_pool_strings.py ok + dart_pprefs.py ok + datastore_inject.py ok + devsh.py usage-ok rc=2 + dex_check_verifier.py ok + dex_classdiff.py ok + dex_find_insn.py ok + dex_patch_bytes.py ok + dex_strings.py ok + dex_strpatch.py ok + dexutil.py usage-ok rc=2 + doctor.py ok + elf_plt.py ok + find_refs.py ok + grab_crash.py ok + install_test.py ok + lib_map.py ok + native_crash.py ok + patch_smali.py usage-ok rc=2 + preflight.py ok + probe_api.py ok + repack.py ok + run_probe.py ok + sig_probe.py ok + smtool.py usage-ok rc=2 + snap.py ok + so_constpatch.py ok + tls_check.py ok + usb_net_proxy.py ok + +== apk-reverse: reference and script paths resolve == + all referenced reference/script paths exist + +== README paths are explicit and exist == + +== result: 0 problem(s) == +``` + +--- + +## 2. python -B check_refs.py + +Every cross-reference naming a section of another document must reach a real heading there. + +Exit code: **0**. Output (36 lines): + +``` +== markdown documents == + README.md + skills/apk-reverse/SKILL.md + skills/apk-reverse/references/account-gates.md + skills/apk-reverse/references/ad-removal.md + skills/apk-reverse/references/byte-level-patching.md + skills/apk-reverse/references/code-virtualization-and-custom-linkers.md + skills/apk-reverse/references/dart-aot.md + skills/apk-reverse/references/detection-and-anti-analysis.md + skills/apk-reverse/references/dex-patching.md + skills/apk-reverse/references/dynamic-frida.md + skills/apk-reverse/references/environment.md + skills/apk-reverse/references/framework-runtimes.md + skills/apk-reverse/references/long-task-discipline.md + skills/apk-reverse/references/membership-and-limits.md + skills/apk-reverse/references/native-and-so.md + skills/apk-reverse/references/native-tamper-and-suicide.md + skills/apk-reverse/references/packers.md + skills/apk-reverse/references/patch-audit.md + skills/apk-reverse/references/pitfalls.md + skills/apk-reverse/references/recon.md + skills/apk-reverse/references/repack-and-sign.md + skills/apk-reverse/references/runtime-data.md + skills/apk-reverse/references/server-api.md + skills/apk-reverse/references/server-config-and-updates.md + skills/apk-reverse/references/signature-derived-keys.md + skills/apk-reverse/references/third-party-builds.md + skills/apk-reverse/references/tls-and-cert.md + skills/apk-reverse/references/toolchain.md + skills/apk-reverse/references/updates-and-forced-upgrade.md + skills/apk-reverse/references/verification.md + skills/apk-reverse/scripts/dexpatch/README.md + +== section references: 166 reachable == + +== result: 0 dangling, 0 warning(s) == +``` + +--- + +## 3. python -B build_scripts.py . + +Audit for machine-specific leftovers: absolute paths and credential literals. + +Exit code: **0**. Output (2 lines): + +``` +== audit: 76 file(s) checked, 0 finding(s). Nothing was written. + re-run with --apply to copy, and with --scrub to neutralise absolute-path literals. +``` + +--- + +## 4. npx skills add newliver666/apk-reverse --list + +The skills CLI must still discover the skill after the changes. + +Exit code: **0**. Output (28 lines): + +``` + + + + + + + + + skills + + Tip: use the --yes (-y) and --global (-g) flags to install without prompts. +[?25l + Source: https://github.com/newliver666/apk-reverse.git +[?25h[?25l + +[?25h[?25l + Found 1 skill +[?25h + + Available Skills + + apk-reverse + + Reverse engineer, debloat, de-ad, patch, or re-sign Android APKs, and analyze their runtime and server-side behavior. Use when a task involves an .apk/.aab/.dex/.so sample, smali or dex patching, Frida/objection runtime hooking, repacking and re-signing, removing ads or SDK trackers, probing a mobile app's HTTP API, or deciding whether a client-side patch is even capable of achieving the goal. Covers recon, anti-tamper, ad removal, membership/paywall limits, dex-level surgical patching, repack pitfalls, device and emulator setup, and a hard-won failure catalogue. Load the body before planning any patch work: it opens with a symptom index and four gates that must be cleared first. + + + Use --skill to install specific skills + +``` + +--- + +## Two notes about running these gates + +Both were found by experiment during this pass and are worth knowing before trusting a +green run. + +**`check_repo.py` and `check_refs.py` have no argparse.** Passing `--help` does not print +usage — it is silently ignored and the full check runs, producing byte-identical output to a +normal invocation. A "low-impact probe" assumption about these two is wrong. + +**Do not set `PYTHONIOENCODING=utf-8` when running `check_repo.py`.** Doing so produced a +spurious exit 1 with one script (`dex_strpatch.py`) reported as crashed. Root cause is at +`check_repo.py:152-154`: `subprocess.run(..., text=True)` does not pin an encoding, so the +parent decodes the child's UTF-8 output as `cp936`, the reader thread raises +`UnicodeDecodeError`, both streams come back empty, `helped` (line 158) goes false and the run +is recorded as FAIL (lines 164-165). Clearing the variable and re-running restores exit 0. The +same lines also carry a diagnostic inconsistency: `verdict` checks only `returncode` while `ok` +additionally requires `helped`, so a run with both streams empty and returncode 0 prints `ok` +yet is recorded as crashed in the results block. + +This is also why `check_refs.py` failed once during this pass and now passes: it resolves +cross-references **within the skill only** and has no fallback outside it, so a sentence in +`SKILL.md` pointing at a path outside the skill directory read as dangling. It was right to +fail; the reference was rewritten rather than the checker. diff --git a/skills/apk-reverse/SKILL.md b/skills/apk-reverse/SKILL.md index 8f99e12..3e5b263 100644 --- a/skills/apk-reverse/SKILL.md +++ b/skills/apk-reverse/SKILL.md @@ -73,11 +73,33 @@ vocabulary and a set of reassuring numbers. - The native layer: `.so` hosts, tamper-triggered self-termination, forged ELF structure, and neutralising a terminate path without freezing the process (`native-and-so.md`, `native-tamper-and-suicide.md`). -- Flutter / Dart AOT: pinning the engine version, decoding the object pool, locating and patching - business logic inside `libapp.so` (`dart-aot.md`). +- Flutter / Dart AOT: analysing and patching `libapp.so` **given a snapshot dump** — pool-reference + counting, disassembly windows, caller indexing, patch-site choice (`dart-aot.md`). Measured on a + real Dart 3.6.0 build: `dart_disasm.py` decoded identically to capstone (32/32 and 96/96), the + caller index re-derived independently with a symmetric difference of 0, and a specific business + logic site was located end to end. **The snapshot dump is a dependency, not a detail** — see the + next section. - Runtime analysis with Frida, server-side API probing, feature-scoped TLS failures, update and forced-upgrade neutralisation, and the verification discipline everything above rests on. +**Dependencies this skill does not ship — name them before the workflow starts:** + +- **Dart AOT analysis needs a snapshot dump.** `dart-aot.md`'s workflow begins at `pp.txt`; producing + it requires a snapshot container resolver that this skill does not contain and cannot synthesize. + `dart_pool_strings.py` reports **file** offsets while `pp.txt` and `dart_pprefs.py` speak in **pool** + offsets, and the mapping between those two spaces is not a constant: over the 4,241 strings present + in both, a measured run found 4,237 distinct deltas. Ship a pinned front end (aotopsy — pure Go, no + toolchain) or build blutter (~80 s, needs a C++ toolchain). **Say which one you are using and why, + because the two report different Dart version labels for the same binary.** Do not describe the + object pool as something this skill decodes on its own. + +**How strong these claims are:** the Measured mechanisms below were established by running the +scripts against a real target during the verification pass recorded in +`docs/verification-jiongnew/`. The rest are **inferred** — documented from experience, but the +repository carries no fixture, log or sample that reproduces them (its own `long-task-discipline.md` +reserves *observed* for a claim with an exact command and output behind it). Treat the distinction as +load-bearing rather than cosmetic, and label your own results the same way. + **Not covered — say so rather than improvise:** - **Unity / IL2CPP logic recovery.** `framework-runtimes.md` identifies the runtime and establishes @@ -93,6 +115,24 @@ vocabulary and a set of reassuring numbers. cost and often concludes "switch to static"; it is not a catalogue of evasion for every detector you might meet. +**Not exercised by the verification pass — do not read silence as support:** the packer, code +virtualization, custom-linker, integrity-check-redirection and tamper-triggered-suicide scenarios +were **not run** against the verification target, because that target has none of those features (no +packer, no integrity checker, ordinary application class) and its unmodified build already fails to +start, which removes the repack-and-regress loop those scenarios need. Nothing in +`docs/verification-jiongnew/` is evidence either way about them. If you use those documents, the +claims are still on the inferred footing described above. + +**Scripts this pass did not run** — so they carry no measurement at all, and any conclusion drawn +from them should be labelled accordingly: `native_crash.py`, `apk_diff.py`, `snap.py`, +`grab_crash.py`, `install_test.py`, `repack.py`, `dex_patch_bytes.py`, `dex_find_insn.py`, +`dex_check_verifier.py`, `dex_classdiff.py`, `dex_strpatch.py`, `patch_smali.py`, `smtool.py`, +`datastore_inject.py`, `probe_api.py`, `run_probe.py`, `tls_check.py`, `usb_net_proxy.py`, +`devsh.py`, and the `dexpatch/` java rewriter. Their absence from the record is not a verdict on +them. Note in particular that `grab_crash.py` claims to recover stacks hidden by a crash-reporter +SDK — the exact situation the verification target presented — and was not tried, so that claim +remains **unverified** and the pass used a purpose-written Frida probe instead. + **The fallback, as an instruction:** if the target does not match that list, or no symptom-index row matches, **stop and classify before choosing a branch.** Answer the thirteen questions first. If the shape still does not fit — an unknown runtime, a mechanism you cannot name — say exactly that, and @@ -100,6 +140,41 @@ propose the cheapest experiment that would identify it, rather than taking the c route and applying it anyway. A wrong branch here does not fail loudly: it produces an artifact that builds, runs, and does the wrong thing. +## Hand-off points — where this skill ends and another view begins + +Three boundaries that are easy to walk into without noticing. Each names what the other side owns, +rather than restating it, because two copies of the same advice drift apart. + +**1. JNI — a Java `native` declaration and its implementation are two different views of one function.** +This skill reads the Java side (dex) and the native side (`.so`) with different tools, so the join is +where analyses go wrong. + +| Form | What you see | How to find it | +|---|---|---| +| Static linkage | symbol `Java___` in `.dynsym` | search the dynamic symbol table. Under R8 the class name is a short name, so the symbol deforms with it and a search for the readable original finds nothing | +| Dynamic registration | **nothing** in the symbol table — binding happens at runtime | find `RegisterNatives` call sites, or hook it to read the binding table. Obfuscated targets prefer this, and a symbol search fails **silently** on it | +| Native → Java callbacks | native code pulling data back through Java | follow `FindClass` / `GetMethodID` / `CallObjectMethod` | + +`FindClass`/`RegisterNatives` in a `.so` tell you a JNI boundary exists even when no +`Java_*` symbol does. **Strength note:** the three rows above are documented behaviour, not results +from the verification pass, which did not trace a JNI boundary end to end; `FlutterJNI.loadLibrary` +appearing in a dex is the closest it came. Treat them as a map, not as a measurement. + +**2. Hardening — a dex-side packer observation is a native-side implementation question.** If the dex +turns out to be a shell, the logic is behind a loader, and the analysis moves to the `.so` that +performs the unpacking. `packers.md` owns the dex-side identification; the native deep dive belongs on +the other side of this boundary. **Not exercised by the verification pass** — that target had no +packer, so this pointer carries no measurement. + +**3. The existing native boundaries — read native anomalies from the APK side, not from inside.** +`native-and-so.md` and `native-tamper-and-suicide.md` are deliberately scoped to what you can conclude +*from the APK side*: a repacked build that dies instantly with a null-looking fault, a Java-layer +check that reports success while the process dies, a terminate path you made not-return. That +judgement belongs here, because it is about deciding whether your *patch* caused the death. Deep +native work — restoring a symbol, rebuilding a call graph, reversing an OLLVM function — is a +different activity with a different toolchain. Point across rather than duplicating: if you need the +latter, say so instead of extending these two files into it. + ## Symptom index — a matching row is a stop signal You arrive at a symptom, not at a file name. Each row below is a failure that has already been paid @@ -111,7 +186,9 @@ unread in this repository. | What you observe | Load first | |---|---| | A repackaged/re-signed build **dies before your code runs**; `SIGSEGV`, all registers zero, `pc=0`, `fault addr` near `0x0` | `native-tamper-and-suicide.md` (deliberate crash), then `code-virtualization-and-custom-linkers.md` | -| **No packer** (Application is the app's own, dex readable) **and it still dies** | `code-virtualization-and-custom-linkers.md` | +| **No packer** (Application is the app's own, dex readable) **and it still dies** | `code-virtualization-and-custom-linkers.md` §a loader is still a possibility; but if the same build also dies on a *second, unrelated* device you are looking at an ordinary startup fault, not a hardened one | +| The app dies at startup on **every** device, packed or not, **and there is no tombstone** while `crash_dump` reports `already traced` and logcat says `exited cleanly (0)` | the target's launch section in the verification record under `docs/verification-jiongnew/` — a bundled crash reporter (Sentry NDK) has taken the signal handlers, so the platform's own evidence trail is gone. Frida spawn-gating is the recovery route; it needs a working frida-server, which a disguised one may not be | +| A `FORTIFY: pthread_mutex_lock called on a destroyed mutex` abort in a Flutter app, on the **main** thread, before the first frame completes | `dart-aot.md` — check `libapp.so` is actually being loaded; Flutter's engine bootstrap is the usual place a native lifecycle fault surfaces | | Log says a **Java-layer** signature/integrity check **passed**, yet the process dies | `code-virtualization-and-custom-linkers.md` §a Java-layer "signature killer" is a decoy | | Deleting a library fixes validation but yields `UnsatisfiedLinkError: dlopen failed: library "X" not found` | `code-virtualization-and-custom-linkers.md` §the deadlock that eats hours | | Whole classes appear as bare `native` declarations with no body | `code-virtualization-and-custom-linkers.md` | diff --git a/skills/apk-reverse/references/dart-aot.md b/skills/apk-reverse/references/dart-aot.md index 9844b3b..8380619 100644 --- a/skills/apk-reverse/references/dart-aot.md +++ b/skills/apk-reverse/references/dart-aot.md @@ -26,37 +26,77 @@ engine build id in `libflutter.so`'s version string. ## 2. Get a decompiler that matches that version -**blutter** is the practical choice for Android `libapp.so` (arm64). It reconstructs the object pool -and per-class declarations. It is a *source tree you build*, not a binary you download — and its -prebuilt outputs only cover the Dart versions someone already built. +You need a tool that resolves the snapshot container — something that turns `libapp.so` into named +functions, class layouts and, above all, a **pool listing with `pp+0x…` offsets**. Everything in §4 +onward consumes that listing; without it the workflow cannot start, and no script in this skill can +produce it (see §4 for why the mapping is not recoverable from the binary alone). -The version question is the first obstacle, and it has a clean answer: +Two routes exist. Pick deliberately, because their version reporting differs. + +**Route A — aotopsy: no toolchain, run it now.** A single static binary (pure Go) that parses the +snapshot binary format directly, with no Dart VM and no SDK compile. On a real Dart 3.6.0 +`libapp.so` the full pipeline ran out of the box: 30,586 functions, 6,133 class layouts, 39,202 pool +entries (38,274 resolved), 175,120 call edges with 95.7% of indirect sites annotated, plus +`functions.jsonl`, `call_edges.jsonl`, `classes.jsonl`, `string_refs.jsonl`, +`dispatch_table.jsonl` and an annotated `asm/` tree. It also supports x86_64, which blutter does not. + +Its `doctor` subcommand reports a Dart version that is a **structural profile label, not the SDK +version**: on the sample above it printed `3.6.2` while the engine banner, the snapshot hash and a +byte search all said `3.6.0`, and no `3.6.2` byte sequence existed in either `.so`. That matters only +if you feed the number to a VM-compiling tool — use §1's banner for that. Its own README states the +detection is structure-based, so expect the label to name the newest matching profile rather than the +compiler's version. + +**Route B — blutter: full fidelity, needs a build.** It embeds a matching Dart VM and deserializes +through the VM's own code paths, which yields the canonical `pp.txt` in the `pp+0x…` space. It is a +*source tree you build*, not a binary you download: its `bin/` is gitignored and it publishes no +release artifacts, so there is nothing prebuilt to fetch for a given Dart version. - blutter's git HEAD typically supports newer Dart than its shipped `dartsdk/` directory. - When the target's version is missing, blutter fetches it itself: it sparse-clones `dart-lang/sdk` at that version tag (only `runtime`, `tools`, `third_party/double-conversion`), generates a source list, and builds a `dartvm` static library for the target ABI. -- That build needs a real toolchain: `cmake` (>=3.20 is fine, including 4.x), `ninja`, `git`, and a - C++ compiler with C++20 `` support (VS 2022 on Windows, gcc>=13 / clang>=16 elsewhere). - A `vcvars64`-style environment must be active before the build, and `CMakeLists` in the tree pins - `cmake_minimum_required(3.20)`. -- Budget **tens of minutes** for the first build of a new version and run it as a background job; - the compiler log is enormous (per-file include trees), so read only its tail. +- That build needs `cmake` (>=3.20 is fine, including 4.x), `ninja`, `git`, and a C++ compiler with + C++20 `` support. The documented VS 2022 requirement is **over-strict** — MSVC 19.34 + (VS 17.4.3, Nov 2022) compiles and runs the `std::format` path, measured. A `vcvars64`-style + environment must be active, and `CMakeLists` pins `cmake_minimum_required(3.20)`. +- Budget a **couple of minutes**, not "tens of minutes": the whole pipeline (clone, `init_env_win.py`, + sparse SDK clone, cmake, nja, link) measured **≈78 s** wall clock. It uses a unity build via + `dartvm_create_srclist.py`, which is why it is fast. The compiler log is enormous (per-file include + trees), so read only its tail. +- **If the freshly built binary faults immediately** with `0xC0000005` and no output at all, do not + conclude anything about the sample. Relink it (`ninja` in + `build/blutter_dartvm__`) and try again; on the sample above the fault never + reproduced after a relink — including for the unchanged binary — and three subsequent runs each + completed in ~5.9 s. **One retry, then a relink, before you blame the target.** -Outputs of interest: +Outputs of interest (blutter): | Output | What it gives you | What it does *not* give | |---|---|---| | `pp.txt` | every object-pool entry with its `pp+0x…` offset: strings, types, closures, fields, stubs | who references what | | `objs.txt` | reconstructed object shapes with type annotations | code | -| `asm/` | per-class declarations | **instructions — there are none here** | +| `asm/` | class layouts **and function bodies with full instruction listings**, pool annotations and resolved call targets | a pseudocode-level view | | `ida_script/`, `blutter_frida.js` | symbol/annotation helpers | a finished analysis | -> `asm/` having one file per (obfuscated) class invites the assumption that it holds disassembly. -> It does not. Do not plan around it. +> `asm/` is the richest artifact in the chain. It is one file per **library URI** (mirroring the +> package path), not per class as its name suggests, and it carries annotated disassembly such as: +> +> ``` +> // 0x923b3c: r0 = LinkedHashMap.from() +> // 0x923b3c: bl #0x60165c ; [dart:collection] LinkedHashMap::LinkedHashMap.from +> ``` +> +> An earlier revision of this file claimed `asm/` contained no instructions and told the reader to +> ignore it. That was wrong and cost real time — plan around `asm/` as your primary annotated +> disassembly source. **`product` builds carry no debug info.** Expect `no-code_comments`, no function names, and -obfuscated identifiers. That is normal; the strings still survive (see §7). +obfuscated identifiers. That is normal; the strings still survive (see §7). Note this is a *format* +floor, not a tooling gap: the Dart compiler drops field names outside debug builds, so roughly +97-99% of instance field names are simply absent, and local/captured variable names are gone +entirely. Accessor-based recovery (`get:`/`set:` still carry the name) recovers part of the field +picture; the rest should be rendered as unknown rather than guessed. ## 3. The object pool is the whole game @@ -72,6 +112,13 @@ Two offset spaces exist and they are **not the same number** — mixing them was Keep the mapping explicit in your notes. When a tool reports "this string is at X", state which space X is in. +**There is no constant between the two spaces — do not go looking for one.** The relationship is a +property of the *reconstructed* pool, not of the file, so it cannot be computed from `libapp.so` +alone. Measured over the 4,241 strings that appear in both the string table and `pp.txt`, there were +**4,237 distinct deltas** (range −1,361,429 to +277,654). A string recovered by file offset therefore +does not lead to its pool offset, which is exactly why §2 insists on a snapshot-decoding front end: +it is what gives you the `pp+0x…` space in the first place. + ## 4. Build your own reference index (blutter will not give you this) `pp.txt` tells you what is in the pool. To *find the code that uses it* you need pool-offset → @@ -84,10 +131,26 @@ python dart_pprefs.py --lookup pp_refs.json 0x1d1a8 0xc9d0 **Do not build this index with a full-capstone pass.** Decoding every instruction of a multi-MB `.text` with `detail=True` and querying `regs_access()` per instruction costs minutes of CPU and -1–2 GB of peak memory, and on a 16 GB host it presents as a hang with no progress output. Only three -encodings can read the pool, so decode them from the raw 32-bit words instead — the shipped script -does exactly that and finishes in seconds. The same argument applies to building a call graph -(§9): decode `B`/`BL` arithmetically. +1–2 GB of peak memory, and on a 16 GB host it presents as a hang with no progress output. Only a +handful of encodings can read the pool, so decode them from the raw 32-bit words instead — the +shipped script does exactly that and finishes in seconds. + +**But do not fall back to a linear capstone sweep either.** A Dart AOT `.text` begins with snapshot +metadata, not instructions — on a real `libapp.so` the section started at `0x4a0000` while the first +Dart function prologue (`stp x29, x30, [sp, #-imm]!`) was at `0x4b143c`. A linear sweep starting at +offset 0 dies on that metadata and returns **zero instructions**, which looks exactly like "this +library has no code". If you need capstone here, start at a known function address, or use +`skipdata=True`, and treat an implausibly small count as a decoding problem before you treat it as a +property of the file. + +The same mask-based argument applies to building a call graph (§9): decode `B`/`BL` arithmetically +rather than by sweeping. + +**A live example of why the index's completeness matters:** after this file's advice to "count the +references before editing a shared constant", one pool offset measured `[0 refs]` with the shipped +script and `[79 refs]` once a missing load family was added to the scanner. A one-sided quiet gap in +the scan reads as "nothing uses this", which is the answer that gets a shared constant edited +carelessly. If a count looks surprisingly low, suspect the scanner before believing the pool. ## 5. What the registers mean diff --git a/skills/apk-reverse/scripts/dart_pprefs.py b/skills/apk-reverse/scripts/dart_pprefs.py index e14fbb6..cd72df5 100644 --- a/skills/apk-reverse/scripts/dart_pprefs.py +++ b/skills/apk-reverse/scripts/dart_pprefs.py @@ -18,8 +18,16 @@ seconds, and a flat memory profile. Recognised forms (AArch64, little endian), with x27 = pool base: add xD, x27, #imm12, lsl #12 -> xD = x27 + (imm12 << 12) - ldr xD, [xN, #imm12 * 8] -> 64-bit or 32-bit variant - ldur xD, [xN, #simm9] -> 64-bit or 32-bit variant + ldr XD, [xN, #imm12 * size] -> GP register file, size from bits 31-30 + ldr DD, [xN, #imm12 * size] -> SIMD&FP register file (bit 26 set) + ldur XD/DD, [xN, #simm9] -> unscaled variant, both files + +Both register files must be covered. Restricting the LDR form to the GP words +(0xF94/0xB94) silently drops `ldr dN, [x27, #imm]` (0xFD400000), which is how every +*double* constant in the pool is read -- an entire class of references disappears from +the index. On a real Dart 3.6.0 libapp.so that was 302 pool offsets and 1,802 sites lost +(measured, see the verification record); one of them had 150 reference sites that would +all have been reported as zero. A reference is either a direct load whose base is x27, or a load consuming the register an ADD just derived from x27. The compiler emits that pair back to back, so a small window @@ -93,16 +101,21 @@ def scan(text_addr, blob, window): pending[w & 0x1F] = [imm << 12, i] continue - ldr64 = w & 0xFFC00000 == 0xF9400000 - ldr32 = w & 0xFFC00000 == 0xB9400000 - ldur64 = w & 0xFFE00C00 == 0xF8400000 - ldur32 = w & 0xFFE00C00 == 0xB8400000 + # LDR (immediate) / LDUR across BOTH register files. Bits 31-30 give the access + # size and therefore the immediate scale; bit 26 selects the SIMD&FP file. + # Matching only the GP words (0xF94/0xB94) drops `ldr dN, [x27, #imm]` + # (0xFD400000) and with it every double constant read from the pool. + ldr_scale = {0xF9400000: 3, 0xB9400000: 2, 0x79400000: 1, 0x39400000: 0, + 0xFD400000: 3, 0xBD400000: 2, 0x7D400000: 1, 0x3DC00000: 4, + }.get(w & 0xFFC00000) + ldur = (w & 0xFFE00C00) in (0xF8400000, 0xB8400000, 0x78400000, 0x38400000, + 0xFC400000, 0xBC400000, 0x7C400000, 0x3C400000) - if ldr64 or ldr32 or ldur64 or ldur32: + if ldr_scale is not None or ldur: rn = (w >> 5) & 0x1F rd = w & 0x1F - if ldr64 or ldr32: - disp = ((w >> 10) & 0xFFF) << 3 + if ldr_scale is not None: + disp = ((w >> 10) & 0xFFF) << ldr_scale else: disp = sx((w >> 12) & 0x1FF, 9) if rn == POOL_REG: