Close the byte-patching gap, add the server-config path, fix two script crashes

Worked end-to-end on a real unobfuscated commercial app (single-dex, no packer, server-driven launch screen) and folded back everything that cost time. Net +17 files, no new dependencies, no sample-specific identifiers. New references byte-level-patching.md The technique that was missing entirely. Equal-length in-place edits beat method rebuilding (measured on a real sample: method rewrite grew debug_info 924 B -> 22.8 KB and the dex 4.32 MB -> 7.73 MB), but four failure modes are silent: same-length is not enough, branch polarity, verifier legality, and stale dex header fields. Covers locating an instruction's exact offset (why listings and byte-pattern guesses both fail), the instruction width table and the two traps that desynchronise a decode, neutralise-a-branch vs redirect-it, and the header's signature-then-checksum order. server-config-and-updates.md The most common shape of "ad" in a modern app, and the one usually mis-diagnosed: the server supplies UI the client renders. The two-layer fetch (local fallback then remote override) that proves it, finding the config DTOs through the field names data classes keep, why you patch the decision and not the data, deciding the scope of "remove", and remote re-enable / cached config durability. New scripts dexutil.py dependency-free dex reader: structural walk, exact instruction decode, dex header recompute/verify in the correct order, branch-target and operand helpers dex_find_insn.py locate an instruction by decoded semantics, print its exact byte offset with context and BOTH sides of any branch dex_patch_bytes.py equal-length byte patches from a JSON spec: semantic match, polarity pin via expect_next, equal-length enforcement, verifier check, header recompute, re-decode to prove it landed dex_check_verifier.py does any conditional branch target a move-result (bypassing its producer)? Compares builds and separates pre-existing findings from regressions the patch introduced coldstart.py cold-launch capture: timed screenshot burst + logcat signals + installed-build facts + launch timing, and a warning when the foreground activity is not the app check_repo.py maintenance check: every script parses and answers --help, every referenced path resolves, every file is documented Fixes found while running the above repack.py wrote archives with a plain zipfile writer, which cannot satisfy the Android R+ install gate: resources.arsc must be STORED *and* 4-byte aligned, and a zip extra record cannot express a 1-3 byte pad (a stored filler entry is inserted instead). Also fixes the central-directory pointer and adds an alignment gate to the output. usb_net_proxy.py, dex_strpatch.py crashed on --help with a traceback instead of printing usage. Updated toolchain.md ddc added (single-binary dex->Java with query subcommands, with measured timings and its real limits); "not on PATH is not not-installed" with the search commands; which signer to use and why jarsigner breaks alignment repack-and-sign.md alignment requirement and its arithmetic; dex header recompute; jarsigner hazard; vendor install interception ([N] with no symbolic constant -> root pm install); the installer can keep owning the screen after an install recon.md resolve package identity from a tool, not from hand-read AXML strings (a wrong package sends all later device queries nowhere); snapshot the config surface during recon long-task-discipline.md captures you never looked at are not evidence (incl. the consistent-but-wrong-window variant); long-context decay and the record shape that survives it dynamic-frida.md when live attach cannot work at all: static instrumentation, static read, and self-recorded evidence, with the control build caveat each one carries ad-removal.md server-driven config is usually the EASIEST case, not the hardest; a three-way zero result is a positive finding; scope note on local brand splash vs promotional content pitfalls.md P37 stale dex header checksum, P38 [-124] alignment, P39 bare-numeric install failure, P40 wrong-window evidence, P41 inverted branch polarity, P42 decode desynchronisation SKILL.md, README.md symptom index, reference/script indexes and requirements updated for all of the above
2026-09-20 02:05:41 +08:00
parent d1349b3cfa
commit a552157f1b
20 changed files with 3179 additions and 52 deletions
@@ -1032,3 +1032,255 @@ complete, and it costs a few lines to parse. Do not reverse-decode backwards loo
 — on aarch64 almost any 4-byte window decodes as *something*, so a naive scanner reports hundreds
 of fictional entry points — and prologue pattern-matching yields a plausible set with no
 completeness guarantee.
+
+---
+
+## P37. A byte patch lands, and the app dies with `Bad checksum` and a missing normal class
+
+**Symptom**
+
+After editing a dex by a few bytes and repacking, the app fails to start, and logcat shows an
+ordinary class failing to resolve — the Application class, or a small AndroidX component:
+
+```
+W/<pkg>: Failure to verify dex file '.../base.apk': Bad checksum (eacdc11c, expected 6456b6b5)
+E/LoadedApk: java.lang.ClassNotFoundException: Didn't find class "<AppClass>" on path: ...
+E/AndroidRuntime: FATAL EXCEPTION: main
+    java.lang.RuntimeException: Unable to instantiate application <AppClass>: java.lang.ClassNotFoundException: ...
+```
+
+**Root cause**
+
+Every dex header carries two integrity fields that cover the rest of the file, and **they must be
+recomputed in a specific order**:
+
+```
+bytes 12..32 = sha1(data[32:])        # signature — computed FIRST
+bytes  8..12 = adler32(data[12:])     # checksum — covers the signature, so LAST
+```
+
+Computing them in the reverse order leaves the adler32 taken while the signature field was still
+zeroed, so the header never verifies. Note also that in the log line above the **real** adler32 is
+printed as "expected" and the header's stale value as the computed one, which reads backwards and
+sends you looking at the wrong number.
+
+Two properties make this expensive:
+
+- **Android may still start the process**, falling back to interpreting the dex instead of using a
+  verified/optimized image. So the failure is not "rejected", it is a *different* failure, and it
+  appears as a class-resolution problem in a component unrelated to your edit.
+- **Some producers ship a dex whose signature field is all zeros.** On such a file the wrong order
+  is self-consistent, so the bug stays invisible until the first edit — and then looks like your
+  edit caused it.
+
+**Do instead**
+
+Recompute both fields on every dex you touch, in the order above, and assert the result:
+
+```
+adler32(data[12:]) == header_checksum   and   sha1(data[32:]) == header_signature
+```
+
+`scripts/dexutil.py` provides `fix_dex_header()` (correct order) and `verify_dex_header()`;
+`scripts/dex_patch_bytes.py` runs both and refuses to write a header that does not self-verify.
+When you see `Bad checksum`, check the header before investigating the class it named.
+
+---
+
+## P38. `[-124]` on install after a repack: `resources.arsc` must be STORED *and* aligned
+
+**Symptom**
+
+An APK that was previously installed fine, rebuilt with only a dex change, is refused:
+
+```
+Failure [-124: Failed parse during installPackageLI: Targeting R+ (version 30 and above)
+requires the resources.arsc of installed APKs to be stored uncompressed and aligned
+on a 4-byte boundary]
+```
+
+**Root cause**
+
+Two independent requirements are compressed into that one sentence: the entry must be **STORED**,
+and its **data offset** must be divisible by 4. A `zipfile`-based repack can satisfy the first and
+still fail the second, because Python's zip writer gives you no control over entry offsets.
+
+An aggravating factor: this is exactly the class of defect a *signing* step can introduce.
+`jarsigner` recompresses entries as a side effect of adding the v1 JAR signature, so an archive that
+was correctly aligned before signing is not aligned after — and `jarsigner -verify` still reports
+success. If a build that installed a minute ago now returns `[-124]`, suspect the signer first.
+
+**Do instead**
+
+- Write the archive aligned **while writing it** (`scripts/repack.py` emits local headers itself and
+  pads the local extra field), rather than repairing alignment afterwards.
+- Know the padding arithmetic trap: a zip extra area is a sequence of `(id, size, payload)` records,
+  so its minimum useful length is 4 bytes — **a required pad of 1-3 bytes cannot be expressed**.
+  Insert a stored filler entry of exactly that size instead.
+- Sign with **`apksigner`** (v1+v2+v3), which appends the signature block without reordering the zip.
+- Verify by reading the archive, not the tool's exit code: for each gated entry, parse the local
+  header, compute `header_offset + 30 + namelen + extralen`, and check `compress_type == 0` and
+  `offset % 4 == 0`.
+
+---
+
+## P39. Install fails with a bare numeric code and no `INSTALL_FAILED_*` constant
+
+**Symptom**
+
+```
+Performing Streamed Install
+adb.exe: failed to install app.apk: Failure [-99]
+```
+
+The APK installs fine through the device's own file manager, and the same build installs on another
+device. There is no symbolic `INSTALL_FAILED_*` name, so there is nothing to look up.
+
+**Root cause**
+
+Some OEM ROMs route `adb install` through their own security/verification service. The failure is
+**not about your APK**; a device process is declining to accept an ADB-initiated install. The device
+log names the real actor:
+
+```
+ColorPackageInstallInterceptManager: <VENDOR>_ADB_INSTALL_CANCEL ... packageName=<pkg>
+```
+
+**Do instead**
+
+Recognise the shape: numeric-only failure, no symbolic constant, installs fine via the device UI,
+and a vendor package-installer/security process in logcat. Then bypass ADB's install path with root:
+
+```bash
+adb -s <serial> push app.apk /data/local/tmp/app.apk
+adb -s <serial> shell "su -c 'pm install -r -g -d /data/local/tmp/app.apk'"
+```
+
+Do **not** start rebuilding the APK, and do not remove permissions or components to "make it
+installable" — the artifact was never the problem. Search the device log for the install attempt
+before touching the build again.
+
+---
+
+## P40. The screen you captured is not your app
+
+**Symptom**
+
+A launch capture sequence shows a consistent, plausible screen, and conclusions are drawn from it.
+Later, the app is discovered never to have been in the foreground at all. The captured frames show,
+for example, a vendor package-installer confirmation bearing a package name from an unrelated
+earlier attempt.
+
+**Root cause**
+
+An install that went through a UI prompt or an OEM interception can leave the **installer window as
+the foreground activity indefinitely**. From then on:
+
+- `am start` on your app appears to do nothing; the other window owns the display.
+- Screenshots of your supposed launch show the installer instead.
+- `am start -W` — which waits for a first frame — can block past any reasonable timeout, because
+  that frame never arrives. This presents as "the tool hung", not as "the wrong window is up".
+
+The insidious part is that the frames are **mutually consistent**, which feels like corroboration.
+Consistency is not corroboration when every frame shares the same blind spot.
+
+**Do instead**
+
+- Check the foreground before trusting any capture, and again at the end of the sequence:
+  ```bash
+  adb -s <serial> shell "dumpsys activity activities | grep -m1 ResumedActivity"
+  ```
+  If the component is not your package, the frames are evidence about something else.
+- Clear the stray window (`am force-stop <installer pkg>`, or a HOME key event) and re-capture.
+- Prefer a launcher-driven capture over `am start -W`: derive the time line from captures plus
+  logcat instead of blocking on a first frame.
+- Treat byte-identical consecutive frames as a **finding** (a hang, a dialog awaiting input), not as
+  a capture artefact.
+
+`scripts/coldstart.py` performs the foreground check and warns; see also
+`long-task-discipline.md` §captures you never looked at are not evidence.
+
+---
+
+## P41. A patched branch does the opposite of what was intended, and starts cleanly
+
+**Symptom**
+
+The patch targets a boolean config gate. The build installs, launches, and never crashes — and the
+behaviour is exactly inverted: the thing that was supposed to be suppressed now appears every time,
+or vice versa. Nothing in the logs indicates a problem.
+
+**Root cause**
+
+The **polarity of the branch was read from the field name instead of from the control flow.**
+Field names describe intent, not branch layout.
+
+```
+0x151304  iget-boolean v11, v0 -> Config.enabled
+0x151308  if-nez v11, :far         ; enabled == false jumps AWAY
+0x15130c  invoke ...startMain()    ; fall-through: straight into the app
+0x151314  :far  iget v11, v0 -> Config.duration   ; the "show it" path
+```
+
+Here `enabled == true` is the value that **skips** the promo — the opposite of the literal reading.
+A patch that "enables the skip" forces the promo to display on every launch.
+
+The second, equally quiet variant: redirecting a conditional branch (`if-*` -> `goto`) to force one
+side. That introduces a new control-flow edge, which can land on a `move-result*` and make the class
+fail to load with `VerifyError` — a class-load failure that does not always surface as a crash
+dialog.
+
+**Do instead**
+
+- **Decode both sides before editing**, and write one line naming what each one does. If you cannot
+  describe the fall-through and the target, you are not ready to patch.
+- Prefer **neutralising the branch** (`if-*` -> `nop` pair) over redirecting it. Removing an edge is
+  safe; adding one is not.
+- Pin the polarity in the patch specification itself: assert which instruction must immediately
+  follow the branch. `scripts/dex_patch_bytes.py` fails the patch if `expect_next` does not hold,
+  which makes this mistake impossible to commit silently.
+- Audit verifier legality after the edit instead of trusting that it launched
+  (`scripts/dex_check_verifier.py`).
+
+---
+
+## P42. A decode desynchronises, and every offset after that point is wrong
+
+**Symptom**
+
+An instruction you can see in a smali listing is not found by your own decoder, or is reported at an
+offset that does not match the disassembler. Sometimes the decode still produces plausible-looking
+instructions, just shifted, so "no match" is reported for something that is definitely present.
+
+**Root cause**
+
+**One wrong instruction width desynchronises everything after it.** Common offenders, each with its
+own trap:
+
+- `0x32`-`0x3D` (`if-test` 22t / `if-testz` 21t) are **2** code units, not 1. Treating them as 1
+  unit invents a fake second instruction at every branch.
+- `0x1A` (`const-string/jumbo`) appears in real toolchains as a **4-byte** form (op, register,
+  uint16 string index), not the 6-byte 31c shape its format name suggests. Counting it as 3 units
+  shifts the rest of the method by one unit per occurrence.
+- `0x28` is `goto` (10t, **1** unit); `0x29` is `goto/16` and `0x2A` is `goto/32`.
+- A `nop` payload is encoded `00 <ident> <size>` with `ident` in 1..3; a plain `00 00` is an ordinary
+  one-unit `nop`. Treating every `00` as a payload swallows the following instruction.
+- dalvik encodes a distant conditional jump as `if-*` **plus** a separate `goto`, not a single
+  instruction.
+
+**Why it is hard to see**
+
+The decoder reports what it decoded. A shifted stream looks like a method that simply does not
+contain the instruction you want, which reads as a finding about the target — the same failure shape
+as P25/P26/P33, and it removes viable patch sites for free.
+
+**Do instead**
+
+- **Assert the walk ends exactly on `insns_off + insns_size*2`.** If it overshoots or undershoots,
+  the width table is wrong somewhere and every offset derived from that method is suspect. One
+  comparison catches all of the above.
+- **Cross-check register numbers.** If the method's `registers` count is 12 and the listing mentions
+  `v13`, the decode has drifted.
+- **Cross-check one known instruction** against a disassembler before trusting offsets you derived.
+- Build the width table from the format groups in `references/byte-level-patching.md` rather than
+  from memory, and keep the width logic in one place so a fix applies everywhere.