Broaden skill with unpacking, TLS triage, third-party build auditing, runtime probe
references: recon gains packer identification + unpacking workflow (payload/plaintext size mapping, dump validation, hash de-duplication, business-dex heuristic); repack-and-sign gains the de-shelled repack pipeline, why apktool over binary AXML, and the apksigner scheme-selection caveat; new tls-and-cert documents feature-scoped network failures, dual trust chains and the fix template; new third-party-builds covers auditing a cracked/modded APK before trusting it; dynamic-frida gains version pinning, remote-device path, the four-layer probe and hook-never-fires triage; environment gains UI automation pitfalls; ad-removal gains the single convergence point, dynamic-plugin SDKs and the global timestamp gate; pitfalls gains P13-P17; verification gains rung 7 and the three-signal ad check. scripts: new frida_probe.js (four-layer runtime probe), run_probe.py and tls_check.py; repack.py fixes two crash-on-default bugs and a misleading META-INF note; datastore_inject.py, PatchMethodExample.java and build_scripts.py lose hardcoded values - the example class name did not match its file and could not compile.
This commit is contained in:
inliver233
@@ -15,6 +15,11 @@ directly.
|
|||||||
burning hours on a paywall that is enforced by a server.
|
burning hours on a paywall that is enforced by a server.
|
||||||
- Choosing the **safest patch layer** for a given change, and avoiding the layers that
|
- Choosing the **safest patch layer** for a given change, and avoiding the layers that
|
||||||
break the app.
|
break the app.
|
||||||
|
- Separating **your own mistakes from the app's or the server's problems** — a
|
||||||
|
feature-scoped failure (login, registration, payment) is often a TLS/certificate issue on
|
||||||
|
one code path, not a consequence of the patch you just built.
|
||||||
|
- Working through **packed/hardened targets**: identifying the packer, unpacking, and turning a
|
||||||
|
memory dump back into a patched, installable APK.
|
||||||
- Avoiding the specific mistakes that produce an APK that builds perfectly and dies at
|
- Avoiding the specific mistakes that produce an APK that builds perfectly and dies at
|
||||||
runtime.
|
runtime.
|
||||||
|
|
||||||
@@ -23,15 +28,17 @@ directly.
|
|||||||
```
|
```
|
||||||
SKILL.md decision tree, workflow, hard constraints, indexes
|
SKILL.md decision tree, workflow, hard constraints, indexes
|
||||||
references/ loaded on demand, one topic each
|
references/ loaded on demand, one topic each
|
||||||
recon.md identify packer, SDKs, code location, tamper checks
|
recon.md identify packer, SDKs, code location, tamper checks; unpacking
|
||||||
ad-removal.md ad taxonomy, wrapper mapping, callback trap, verification
|
ad-removal.md ad taxonomy, wrapper mapping, callback trap, global gates, verification
|
||||||
membership-and-limits.md server vs client authority; what is and is not patchable
|
membership-and-limits.md server vs client authority; what is and is not patchable
|
||||||
server-api.md probe an app's API; prove who owns the gate
|
server-api.md probe an app's API; prove who owns the gate
|
||||||
|
tls-and-cert.md feature-scoped network failures: expired certs, dual trust chains
|
||||||
|
third-party-builds.md auditing a "cracked"/"modded" APK before trusting it
|
||||||
dex-patching.md patch-layer table + dexlib2 technique in depth
|
dex-patching.md patch-layer table + dexlib2 technique in depth
|
||||||
repack-and-sign.md repack rules, signing, install, post-install hazards
|
repack-and-sign.md repack rules, unpack-and-repack, signing, post-install hazards
|
||||||
runtime-data.md DataStore / SharedPreferences / SQLite / protobuf
|
runtime-data.md DataStore / SharedPreferences / SQLite / protobuf
|
||||||
dynamic-frida.md Frida setup, hooking strategy, anti-instrumentation
|
dynamic-frida.md Frida setup, version pinning, the four-layer probe, hook strategy
|
||||||
environment.md device/emulator setup, ADB, offline devices, log signals
|
environment.md device/emulator setup, ADB, UI automation, offline devices, log signals
|
||||||
verification.md the claim ladder; what "done" means
|
verification.md the claim ladder; what "done" means
|
||||||
pitfalls.md the failure catalogue -- read before building
|
pitfalls.md the failure catalogue -- read before building
|
||||||
scripts/ parameterized, path-agnostic
|
scripts/ parameterized, path-agnostic
|
||||||
@@ -42,13 +49,16 @@ scripts/ parameterized, path-agnostic
|
|||||||
dex_classdiff.py prove a dex edit was surgical
|
dex_classdiff.py prove a dex edit was surgical
|
||||||
dex_strings.py strings/URLs/SDK markers without a decompiler
|
dex_strings.py strings/URLs/SDK markers without a decompiler
|
||||||
find_refs.py count callers of a method before patching it
|
find_refs.py count callers of a method before patching it
|
||||||
repack.py rebuild APK, strip only signatures, sign
|
repack.py rebuild APK, strip only signatures, sign, verify
|
||||||
devsh.py quoting-safe ADB shell helper
|
devsh.py quoting-safe ADB shell helper
|
||||||
usb_net_proxy.py give an offline device network over USB
|
usb_net_proxy.py give an offline device network over USB
|
||||||
datastore_inject.py encode/inject AndroidX DataStore preferences safely
|
datastore_inject.py encode/inject AndroidX DataStore preferences safely
|
||||||
probe_api.py probe an HTTP API with the right headers
|
probe_api.py probe an HTTP API with the right headers
|
||||||
grab_crash.py recover stacks hidden by a crash-reporter SDK
|
grab_crash.py recover stacks hidden by a crash-reporter SDK
|
||||||
install_test.py install + launch health check with logcat signal scan
|
install_test.py install + launch health check with logcat signal scan
|
||||||
|
frida_probe.js four-layer runtime probe (app net layer + OkHttp + java.net + exceptions)
|
||||||
|
run_probe.py inject the probe, stream it to a log file, stay resident
|
||||||
|
tls_check.py strict certificate check for one or more hosts
|
||||||
```
|
```
|
||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ Two rules override everything else in this skill:
|
|||||||
1. **Never ship an unverified APK.** "It assembles" is not "it works". Install it, launch it, exercise the feature you changed, on a real device or a close emulator.
|
1. **Never ship an unverified APK.** "It assembles" is not "it works". Install it, launch it, exercise the feature you changed, on a real device or a close emulator.
|
||||||
2. **Know which layer owns the behavior before you patch.** Ads, paywalls, and feature gates live in different places. Patching the wrong layer either does nothing or breaks the app. Classify first (see step 2 below), patch second.
|
2. **Know which layer owns the behavior before you patch.** Ads, paywalls, and feature gates live in different places. Patching the wrong layer either does nothing or breaks the app. Classify first (see step 2 below), patch second.
|
||||||
|
|
||||||
## Start here: classify the target in five questions
|
## Start here: classify the target in seven questions
|
||||||
|
|
||||||
Answer these before touching a tool. Every one of them changes the whole plan.
|
Answer these before touching a tool. Every one of them changes the whole plan.
|
||||||
|
|
||||||
@@ -30,10 +30,14 @@ Answer these before touching a tool. Every one of them changes the whole plan.
|
|||||||
App-side → you must bypass it. Server-side → re-signing silently breaks the app later. See `references/repack-and-sign.md` and `references/server-api.md`.
|
App-side → you must bypass it. Server-side → re-signing silently breaks the app later. See `references/repack-and-sign.md` and `references/server-api.md`.
|
||||||
5. **What is your device situation?** → `references/environment.md`
|
5. **What is your device situation?** → `references/environment.md`
|
||||||
Rooted real device (best), emulator with root, or no device (static only). Also: this determines whether Frida is usable.
|
Rooted real device (best), emulator with root, or no device (static only). Also: this determines whether Frida is usable.
|
||||||
|
6. **Is one *specific feature* failing at runtime — login, registration, payment, an API-backed screen — while the rest of the app works?**
|
||||||
|
→ `references/tls-and-cert.md`. A feature-scoped network failure is very often a **TLS/certificate problem on one code path**, not a consequence of your patch. The app can even carry two independent trust chains, so "other requests work" proves nothing. Rule this out in minutes before hunting for a signature check.
|
||||||
|
7. **Was the input a build you did not produce** (a "cracked"/"modded" APK circulating online)?
|
||||||
|
→ `references/third-party-builds.md`. Audit it before adopting it: such builds are frequently re-protected (sometimes with *more* layers than the original) and may carry injected components or endpoints. Never use one as a patching workbench.
|
||||||
|
|
||||||
## The workflow, end to end
|
## The workflow, end to end
|
||||||
|
|
||||||
1. **Recon** — `references/recon.md`. Manifest, package name, version, ABI, dex count, packer, embedded SDKs, where the app's own code lives. Ten minutes here saves hours.
|
1. **Recon** — `references/recon.md`. Manifest, package name, version, ABI, dex count, packer, embedded SDKs, where the app's own code lives. Ten minutes here saves hours. **If it is packed, unpack before anything else** (`references/recon.md` §unpacking): you cannot patch code you cannot read, the encrypted payload lengths tell you which dumped dex is the original, and a memory dump must be de-duplicated by hash and structurally validated before any of it is trusted.
|
||||||
2. **Extract strings and endpoints** — build a picture of the app's API surface and SDK inventory from the dex string tables. No decompiler needed for this, and it is fast. Scripts: `scripts/dex_strings.py`.
|
2. **Extract strings and endpoints** — build a picture of the app's API surface and SDK inventory from the dex string tables. No decompiler needed for this, and it is fast. Scripts: `scripts/dex_strings.py`.
|
||||||
3. **Trace to the owning class** — find the class that wraps the behavior (the app almost always wraps third-party SDKs in one helper). Reverse-lookup instructions: `references/dex-patching.md` §finding-the-call-site.
|
3. **Trace to the owning class** — find the class that wraps the behavior (the app almost always wraps third-party SDKs in one helper). Reverse-lookup instructions: `references/dex-patching.md` §finding-the-call-site.
|
||||||
4. **Decide the patch layer** — client SDK call / client rendering / client data consumption / server contract. See the table in `references/ad-removal.md`.
|
4. **Decide the patch layer** — client SDK call / client rendering / client data consumption / server contract. See the table in `references/ad-removal.md`.
|
||||||
@@ -50,6 +54,7 @@ Answer these before touching a tool. Every one of them changes the whole plan.
|
|||||||
- **Do not patch a method that is widely shared.** Before patching any helper, count its callers (`scripts/find_refs.py`). A `Long.valueOf` wrapper with 30 callers is not an ad-specific hook.
|
- **Do not patch a method that is widely shared.** Before patching any helper, count its callers (`scripts/find_refs.py`). A `Long.valueOf` wrapper with 30 callers is not an ad-specific hook.
|
||||||
- **Do not make an API fail to suppress a UI element.** A 404/400 on an endpoint that other features depend on takes the whole screen down with it. Suppress at the data-consumption or render layer instead.
|
- **Do not make an API fail to suppress a UI element.** A 404/400 on an endpoint that other features depend on takes the whole screen down with it. Suppress at the data-consumption or render layer instead.
|
||||||
- **Every claim needs evidence.** "Probably", "should be", "in theory" are not findings. Either you observed it, or you label it unverified.
|
- **Every claim needs evidence.** "Probably", "should be", "in theory" are not findings. Either you observed it, or you label it unverified.
|
||||||
|
- **Attribute a failure to the right layer before patching again.** When something stops working after a rebuild, first check whether the **unmodified original** fails the same way on the same device and network. Feature-scoped network failures in particular are frequently the app's own TLS/certificate problem (`references/tls-and-cert.md`); chasing a signature check that does not exist burns hours.
|
||||||
|
|
||||||
## Reference index
|
## Reference index
|
||||||
|
|
||||||
@@ -67,6 +72,8 @@ Load only what the current step needs.
|
|||||||
| `references/dynamic-frida.md` | Frida setup, hooking strategy, tracing caller chains, finding the real call site |
|
| `references/dynamic-frida.md` | Frida setup, hooking strategy, tracing caller chains, finding the real call site |
|
||||||
| `references/environment.md` | Device/emulator setup, root, ADB, networking, offline devices |
|
| `references/environment.md` | Device/emulator setup, root, ADB, networking, offline devices |
|
||||||
| `references/verification.md` | Defining what "done" means; building the evidence chain |
|
| `references/verification.md` | Defining what "done" means; building the evidence chain |
|
||||||
|
| `references/tls-and-cert.md` | One feature fails at runtime (login, registration, payment, an API-backed screen) while the rest of the app works |
|
||||||
|
| `references/third-party-builds.md` | The input is a "cracked"/"modded" build you did not produce — audit it before trusting it |
|
||||||
| `references/pitfalls.md` | Always worth a skim before building. This is the failure catalogue. |
|
| `references/pitfalls.md` | Always worth a skim before building. This is the failure catalogue. |
|
||||||
|
|
||||||
## Script index
|
## Script index
|
||||||
@@ -88,3 +95,6 @@ All scripts are parameterized and path-agnostic; pass paths explicitly. Run `--h
|
|||||||
| `scripts/datastore_inject.py` | Encode/inject AndroidX DataStore preferences (protobuf) safely |
|
| `scripts/datastore_inject.py` | Encode/inject AndroidX DataStore preferences (protobuf) safely |
|
||||||
| `scripts/probe_api.py` | Probe an app's HTTP API with correct headers, report status/shape |
|
| `scripts/probe_api.py` | Probe an app's HTTP API with correct headers, report status/shape |
|
||||||
| `scripts/install_test.py` | Install a build and run a launch/health check with logcat signal extraction |
|
| `scripts/install_test.py` | Install a build and run a launch/health check with logcat signal extraction |
|
||||||
|
| `scripts/frida_probe.js` | Four-layer runtime probe: app network layer + OkHttp + java.net + swallowed exception messages |
|
||||||
|
| `scripts/run_probe.py` | Inject a probe, stream it to a timestamped log file, stay resident while you operate the app |
|
||||||
|
| `scripts/tls_check.py` | Strict certificate check for one or more hosts (expired / wrong host / untrusted CA) |
|
||||||
|
|||||||
+167
-63
@@ -1,84 +1,188 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
"""One-shot builder: copy the battle-tested tools from the project workspace into
|
"""Sync a script directory to a publish location, and audit it for machine-specific leftovers.
|
||||||
the skill, stripping machine-specific hardcoding.
|
|
||||||
|
|
||||||
Run from anywhere: python build_scripts.py <project_work_dir> <skill_scripts_dir>
|
WHY THIS EXISTS
|
||||||
|
---------------
|
||||||
|
An Agent Skill gets copied across machines and across runtimes (Claude Code, Codex,
|
||||||
|
DeepSeek Harness). Anything machine-specific that survives the copy becomes a
|
||||||
|
landmine: a baked-in JDK path, a keystore password, a home directory that only
|
||||||
|
exists on the author's box. This tool does two things:
|
||||||
|
|
||||||
|
1. AUDIT (default, read-only): report every absolute-path literal and every
|
||||||
|
credential-looking constant, with file and line number.
|
||||||
|
2. SYNC (--apply): copy matching files to the destination, optionally after
|
||||||
|
neutralising the absolute-path literals it found (--scrub).
|
||||||
|
|
||||||
|
The audit is the default on purpose: a silent copy is exactly how the leftovers
|
||||||
|
travel from the author's machine into someone else's first run.
|
||||||
|
|
||||||
|
USAGE
|
||||||
|
-----
|
||||||
|
python build_scripts.py scripts2 out # audit only, nothing written
|
||||||
|
python build_scripts.py scripts out --apply # copy
|
||||||
|
python build_scripts.py scripts out --apply --scrub # copy, neutralising abs paths
|
||||||
|
python build_scripts.py scripts out --pattern '*.py' --pattern '*.js'
|
||||||
|
|
||||||
|
Exit codes: 0 = clean, 2 = findings reported (audit mode), 1 = usage or IO error.
|
||||||
|
|
||||||
|
NOTES
|
||||||
|
-----
|
||||||
|
* A finding is a heuristic, not a verdict: `/data/local/tmp/...` is a legitimate
|
||||||
|
Android device path and is deliberately NOT reported; a drive-letter path is.
|
||||||
|
* --scrub rewrites `'<abs path>'` literals into `''`, which keeps the file
|
||||||
|
syntactically valid but leaves the decision (and the fix) to a human. It never
|
||||||
|
runs unless --apply is also given.
|
||||||
"""
|
"""
|
||||||
|
import argparse
|
||||||
|
import fnmatch
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
import shutil
|
import shutil
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
SRC = sys.argv[1] if len(sys.argv) > 1 else r'E:\ciyuan\work'
|
DEFAULT_PATTERNS = ('*.py', '*.js', '*.java', '*.md', '*.txt')
|
||||||
DST = sys.argv[2] if len(sys.argv) > 2 else r'E:\ciyuan\apk-reverse\scripts'
|
|
||||||
|
|
||||||
# file -> (new name, list of (pattern, replacement))
|
# Absolute-path literals that belong to one particular machine.
|
||||||
COPY = {
|
# Deliberately excluded: /data/... (an Android device path, not a host path).
|
||||||
r'tools\dex_strpatch.py': ('dex_strpatch.py', []),
|
WINDOWS_LITERAL = re.compile(r"(?P<prefix>(?:r|u|b)?r?)(?P<quote>['\"])"
|
||||||
r'tools\dex_classdiff.py': ('dex_classdiff.py', []),
|
r"(?P<path>[A-Za-z]:[\\/][^'\"]*)(?P=quote)")
|
||||||
r'tools\patch_smali.py': ('patch_smali.py', []),
|
POSIX_LITERAL = re.compile(r"(?P<prefix>(?:r|u|b)?r?)(?P<quote>['\"])"
|
||||||
r'tools\repack.py': ('repack.py', [
|
r"(?P<path>/(?:home|Users|mnt)/[^'\"]*)(?P=quote)")
|
||||||
# strip machine-specific defaults; make them explicit args
|
|
||||||
(re.compile(r'KS_PATH\s*=\s*r?["\'][^"\']*["\']'), 'KS_PATH = None # pass --ks on the command line'),
|
# Any drive-letter or foreign-home path outside a quoted literal as well.
|
||||||
(re.compile(r"workdir'?,?\s*default=r?['\"][^'\"]*['\"]"), "workdir', default=None"),
|
# Tuned to avoid the two false-positive families that made an earlier version of
|
||||||
]),
|
# this audit useless: a URL scheme ("https://" contains "s:" + "/") and a Python
|
||||||
r'tools\cyc_proxy.py': ('usb_net_proxy.py', []),
|
# escape sequence ("is:\n" contains "s:" + "\"). Hence: the previous character must
|
||||||
r'tools\mk_video_prefs.py': ('datastore_inject.py', []),
|
# not be alphanumeric, a drive letter must be followed by a real backslash (not an
|
||||||
r'tools\probe_adverts.py': ('probe_api.py', []),
|
# escape letter, not a second separator), and the POSIX form must start a path.
|
||||||
r'tools\grab_crash.py': ('grab_crash.py', []),
|
BARE_ABS = re.compile(
|
||||||
}
|
r"(?<![A-Za-z0-9])(?:"
|
||||||
|
r"[A-Za-z]:\\(?![ntr0bxu'\"\\])\S{2,}"
|
||||||
|
r"|/(?:home|Users|mnt)/[A-Za-z0-9_.\-]{2,}"
|
||||||
|
r")")
|
||||||
|
|
||||||
|
CREDENTIAL_HINT = re.compile(
|
||||||
|
r"(?i)(BEGIN [A-Z ]*PRIVATE KEY|storepass\s*=\s*['\"][^'\"]|"
|
||||||
|
r"keypass\s*=\s*['\"][^'\"]|password\s*=\s*['\"][^'\"]|"
|
||||||
|
r"api[_-]?key\s*=\s*['\"][^'\"]|secret\s*=\s*['\"][^'\"]{6,})")
|
||||||
|
|
||||||
|
|
||||||
def clean(text):
|
def collect(src, patterns):
|
||||||
"""Remove absolute paths that point at one particular machine."""
|
"""Return [(absolute_path, relative_path)] for every matching file."""
|
||||||
text = re.sub(r"r?'[A-Za-z]:\\\\[^'\"]*'", "''", text)
|
out = []
|
||||||
text = re.sub(r'"[A-Za-z]:\\\\[^"]*"', '""', text)
|
for root, _dirs, files in os.walk(src):
|
||||||
return text
|
for name in sorted(files):
|
||||||
|
if any(fnmatch.fnmatch(name, p) for p in patterns):
|
||||||
|
full = os.path.join(root, name)
|
||||||
|
out.append((full, os.path.relpath(full, src)))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def audit(path, rel):
|
||||||
|
"""Return [(kind, lineno, detail)] for one file."""
|
||||||
|
findings = []
|
||||||
|
try:
|
||||||
|
with open(path, encoding='utf-8', errors='replace') as fh:
|
||||||
|
lines = fh.readlines()
|
||||||
|
except OSError as exc:
|
||||||
|
return [('io-error', 0, str(exc))]
|
||||||
|
|
||||||
|
for lineno, line in enumerate(lines, 1):
|
||||||
|
for match in BARE_ABS.finditer(line):
|
||||||
|
findings.append(('abs-path', lineno, match.group(0)[:90]))
|
||||||
|
if CREDENTIAL_HINT.search(line):
|
||||||
|
findings.append(('credential', lineno, line.strip()[:110]))
|
||||||
|
return findings
|
||||||
|
|
||||||
|
|
||||||
|
def scrub(text):
|
||||||
|
"""Neutralise absolute-path literals; returns (new_text, count)."""
|
||||||
|
total = 0
|
||||||
|
|
||||||
|
def repl(_match):
|
||||||
|
return "''"
|
||||||
|
|
||||||
|
for pattern in (WINDOWS_LITERAL, POSIX_LITERAL):
|
||||||
|
text, count = pattern.subn(repl, text)
|
||||||
|
total += count
|
||||||
|
return text, total
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
os.makedirs(DST, exist_ok=True)
|
ap = argparse.ArgumentParser(
|
||||||
os.makedirs(os.path.join(DST, 'dexpatch'), exist_ok=True)
|
description='Sync a script directory to a publish location and audit it for '
|
||||||
for rel, (new, subs) in COPY.items():
|
'machine-specific leftovers (absolute paths, hardcoded credentials).')
|
||||||
src = os.path.join(SRC, rel)
|
ap.add_argument('src', help='source directory to publish from')
|
||||||
if not os.path.isfile(src):
|
ap.add_argument('dst', help='destination directory to publish into')
|
||||||
print('[skip missing] %s' % rel)
|
ap.add_argument('--pattern', action='append', default=list(DEFAULT_PATTERNS),
|
||||||
|
help='glob to include, repeatable (default: %s)'
|
||||||
|
% ' '.join(DEFAULT_PATTERNS))
|
||||||
|
ap.add_argument('--apply', action='store_true',
|
||||||
|
help='actually copy files (default is audit only)')
|
||||||
|
ap.add_argument('--scrub', action='store_true',
|
||||||
|
help='with --apply, neutralise absolute-path literals in the copies')
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
if not os.path.isdir(args.src):
|
||||||
|
print('error: source directory not found: %s' % args.src)
|
||||||
|
return 1
|
||||||
|
|
||||||
|
files = collect(args.src, args.pattern)
|
||||||
|
if not files:
|
||||||
|
print('error: no files matched %s in %s'
|
||||||
|
% (', '.join(args.pattern), args.src))
|
||||||
|
return 1
|
||||||
|
|
||||||
|
total_findings = 0
|
||||||
|
scrubbed = 0
|
||||||
|
copied = 0
|
||||||
|
|
||||||
|
for full, rel in files:
|
||||||
|
findings = audit(full, rel)
|
||||||
|
abs_findings = [f for f in findings if f[0] == 'abs-path']
|
||||||
|
cred_findings = [f for f in findings if f[0] == 'credential']
|
||||||
|
other = [f for f in findings if f[0] not in ('abs-path', 'credential')]
|
||||||
|
|
||||||
|
if findings:
|
||||||
|
print('-- %s' % rel)
|
||||||
|
for kind, lineno, detail in abs_findings + cred_findings + other:
|
||||||
|
print(' [%s] line %d: %s' % (kind, lineno, detail))
|
||||||
|
total_findings += len(findings)
|
||||||
|
|
||||||
|
if not args.apply:
|
||||||
continue
|
continue
|
||||||
t = open(src, encoding='utf-8', errors='replace').read()
|
|
||||||
for pat, rep in subs:
|
|
||||||
t = pat.sub(rep, t)
|
|
||||||
out = os.path.join(DST, new)
|
|
||||||
with open(out, 'w', encoding='utf-8', newline='\n') as f:
|
|
||||||
f.write(t)
|
|
||||||
print('[copied] %-24s -> %s' % (rel, new))
|
|
||||||
|
|
||||||
# dexlib2 patcher sources
|
target = os.path.join(args.dst, rel)
|
||||||
for rel, new in [(r'tools\dexpatch\PatchMethod.java', 'PatchMethod.java'),
|
os.makedirs(os.path.dirname(target) or '.', exist_ok=True)
|
||||||
(r'tools\dexpatch\PatchClasses7v2.java', 'PatchMethodExample.java')]:
|
|
||||||
src = os.path.join(SRC, rel)
|
|
||||||
if os.path.isfile(src):
|
|
||||||
t = open(src, encoding='utf-8', errors='replace').read()
|
|
||||||
with open(os.path.join(DST, 'dexpatch', new), 'w',
|
|
||||||
encoding='utf-8', newline='\n') as f:
|
|
||||||
f.write(t)
|
|
||||||
print('[copied] %-24s -> dexpatch/%s' % (rel, new))
|
|
||||||
else:
|
|
||||||
print('[skip missing] %s' % rel)
|
|
||||||
|
|
||||||
# cp.txt content -> smali_cp.txt template
|
if args.scrub and (abs_findings or cred_findings):
|
||||||
cp = os.path.join(SRC, 'tools', 'dexpatch', 'cp.txt')
|
with open(full, encoding='utf-8', errors='replace') as fh:
|
||||||
if os.path.isfile(cp):
|
text = fh.read()
|
||||||
raw = open(cp, encoding='utf-8').read().strip()
|
new_text, count = scrub(text)
|
||||||
jars = [j for j in raw.split(';') if j.strip()]
|
if count:
|
||||||
with open(os.path.join(DST, 'smali_cp.txt'), 'w', encoding='utf-8', newline='\n') as f:
|
with open(target, 'w', encoding='utf-8', newline='\n') as fh:
|
||||||
f.write('# One jar path per line. Adjust the directory to your own tools.\n')
|
fh.write(new_text)
|
||||||
f.write('# Needed: smali baksmali dexlib2 util antlr-runtime stringtemplate '
|
scrubbed += count
|
||||||
'jcommander guava\n')
|
print(' -> copied with %d absolute-path literal(s) neutralised' % count)
|
||||||
for j in jars:
|
copied += 1
|
||||||
f.write(os.path.basename(j) + '\n')
|
continue
|
||||||
print('[wrote] smali_cp.txt (%d jars)' % len(jars))
|
|
||||||
|
shutil.copy2(full, target)
|
||||||
|
copied += 1
|
||||||
|
|
||||||
|
print('')
|
||||||
|
if not args.apply:
|
||||||
|
print('== audit: %d file(s) checked, %d finding(s). Nothing was written.'
|
||||||
|
% (len(files), total_findings))
|
||||||
|
print(' re-run with --apply to copy, and with --scrub to neutralise '
|
||||||
|
'absolute-path literals.')
|
||||||
|
return 2 if total_findings else 0
|
||||||
|
|
||||||
|
print('== sync: %d file(s) copied to %s (%d literal(s) scrubbed), %d finding(s) reported.'
|
||||||
|
% (copied, args.dst, scrubbed, total_findings))
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
main()
|
sys.exit(main())
|
||||||
|
|||||||
+87
-39
@@ -1,53 +1,75 @@
|
|||||||
# Ad removal
|
# Ad removal
|
||||||
|
|
||||||
Goal: ads stop appearing, and **nothing else breaks**. Read `references/pitfalls.md` P5 and P6 before patching anything here.
|
Goal: ads stop appearing, and **nothing else breaks**. Read `pitfalls.md` P5 and P6 before patching anything here.
|
||||||
|
|
||||||
## Step 1: identify which ads exist and who owns them
|
## Step 1: enumerate what "ads" means in this app
|
||||||
|
|
||||||
Do not assume "ads" is one thing. Enumerate first.
|
Do not assume "ads" is one thing. Enumerate first.
|
||||||
|
|
||||||
**SDK-integrated ads** (the app calls an SDK):
|
**SDK-integrated ads** (the app calls an SDK):
|
||||||
- Search dex strings for SDK markers: `openadsdk`, `TTAdSdk`, `TTAdNative`, `Pangle`, `pangolin`, `com.qq.e`, `GDTAd`, `gdt_plugin`, `anythink`, `ATSDK`, `ATRewardVideoAd`, `bdxadsdk`, `sigmob`, `ksad`, `mobads`, `beizi`.
|
- Search dex strings for SDK markers: `openadsdk`, `TTAdSdk`, `TTAdNative`, `Pangle`, `pangolin`, `com.qq.e`, `GDTAd`, `gdt_plugin`, `anythink`, `ATSDK`, `ATRewardVideoAd`, `bdxadsdk`, `sigmob`, `ksad`, `mobads`, `beizi`.
|
||||||
- **Aggregators** (AnyThink, TopOn, and similar) are wrappers: the visible ad networks underneath are *their* adapters, not app code. Patching an individual network's class is usually pointless — the aggregator still runs and still reaches the network.
|
- **Aggregators** (AnyThink, TopOn, and similar) are wrappers: the ad networks underneath are *their* adapters, not app code. Patching an individual network's class is usually pointless; the aggregator still runs and still reaches the network.
|
||||||
- The app almost always funnels every call through **one wrapper class**. Find it; that is your patch surface.
|
- The app almost always funnels every call into **one wrapper class**. That class is the patch surface (Step 2).
|
||||||
|
|
||||||
|
**Dynamic-plugin ad SDKs** (the SDK is not in the APK). Aggregators increasingly ship as a runtime plugin: the app downloads an APK on first launch into its private directory and loads it through a plugin classloader.
|
||||||
|
|
||||||
|
```
|
||||||
|
/data/data/<app.package>/files/<sdk>_p/<plugin.pkg>/version-*/apk/base-1.apk
|
||||||
|
/data/data/<app.package>/files/<sdk>_p/<plugin.pkg>/version-*/lib/<abi>/*
|
||||||
|
```
|
||||||
|
|
||||||
|
- Consequence: **deleting bundled `assets/`, `lib/*.so`, or the SDK's classes from the dex does not kill this SDK.** The loader re-downloads on the next launch, and a half-deleted plugin is worse than an intact one.
|
||||||
|
- The kill switch is the **init gate** (Step 2). Killing init also prevents the working directory from ever being created, which is the strongest verification signal available (Step 4).
|
||||||
|
- The plugin may also be bundled. Two copies still means one init gate; patch that, not the payload.
|
||||||
|
|
||||||
**Server-driven ads / sponsored content** (the server returns the ad, the client renders it):
|
**Server-driven ads / sponsored content** (the server returns the ad, the client renders it):
|
||||||
- Look for endpoints like `/adverts`, `/adv`, `/banner`, `/config`, and DTOs named `Advertisement*`, `Advert*`, `Banner*`, `Promotion*`.
|
- Look for endpoints like `/adverts`, `/adv`, `/banner`, `/config`, and DTOs named `Advertisement*`, `Advert*`, `Banner*`, `Promotion*`.
|
||||||
- The client renders whatever the list contains. If the list is empty there is nothing to show — but see `references/pitfalls.md` P5 on **how not** to empty it.
|
- The client renders whatever the list contains. If the list is empty there is nothing to show, but see `pitfalls.md` P5 on **how not** to empty it.
|
||||||
- Sponsored cards that look like content (a VPN promo, a network-accelerator card, a third-party product) are usually exactly this.
|
- Sponsored cards that look like content (a VPN promo, a network-accelerator card, a third-party product) are usually exactly this.
|
||||||
|
|
||||||
**Legit content that looks like an ad.** Verify before acting. A `/adverts?position=banner` response containing anime titles and poster images is the home-page carousel, not an advertisement. Removing it removes real functionality.
|
**Legit content that looks like an ad.** Verify before acting. A `/adverts?position=banner` response containing anime titles and poster images is the home-page carousel, not an advertisement. Removing it removes real functionality.
|
||||||
|
|
||||||
## Step 2: map the wrapper
|
## Step 2: find the single convergence point
|
||||||
|
|
||||||
Find the app's own helper and enumerate its public surface. Typical shape:
|
Almost every ad-enabled app wraps one SDK in a **single singleton helper**: `init`, `loadSplash`, `loadBanner`, `loadFullScreen`, `loadReward` all live in the same class. Find that class, and patch nothing else if you can.
|
||||||
|
|
||||||
```
|
```
|
||||||
helper.init(Application) -> Sdk.init(appId, appKey); Sdk.start()
|
helper.init(Application) -> Sdk.init(appId, appKey); Sdk.start() // plus plugin download
|
||||||
helper.showSplash(Activity, ViewGroup, onClose: () -> Unit)
|
helper.showSplash(Activity, ViewGroup, onClose: () -> Unit)
|
||||||
helper.showInterstitial(Activity)
|
helper.showInterstitial(Activity)
|
||||||
helper.showReward(Activity, onStart: () -> Unit, onFinish: (Boolean) -> Unit)
|
helper.showReward(Activity, onStart: () -> Unit, onFinish: (Boolean) -> Unit)
|
||||||
helper.preload*(Activity)
|
helper.preload*(Activity)
|
||||||
helper.canShow*(Activity): Boolean
|
helper.canShow*(Activity): Boolean // == isReady / isLoaded / isAdReady
|
||||||
```
|
```
|
||||||
|
|
||||||
Read each method body and classify:
|
**Preferred play: readiness always false + init is a no-op.**
|
||||||
|
|
||||||
| Method | Contains | Safe patch |
|
1. Make the readiness predicate constant `false` (`isReady` / `canShow*` / `isLoaded` / `isAdReady`).
|
||||||
|
2. Make `init` an empty method (`return-void`) so the SDK never builds a network stack and never requests a plugin.
|
||||||
|
3. Only then touch show/load methods, and only for slots that bypass the readiness predicate.
|
||||||
|
|
||||||
|
Why this order works:
|
||||||
|
- Callers of a readiness predicate **already own a "no ad available" branch**, and the app ships and tests it. Returning `false` routes every slot through that existing path: no crash, no hang, no new code path.
|
||||||
|
- Killing init removes the SDK's network, cache, and download activity entirely. That is the difference between "the ad is hidden" and "the ad subsystem never started", and only the second claim is provable (Step 4).
|
||||||
|
- Per-slot patching (splash this week, interstitial next) leaves the SDK alive and downloading. Avoid it unless a slot ignores the predicate.
|
||||||
|
|
||||||
|
Method classification, once the helper is located:
|
||||||
|
|
||||||
|
| Method | Contains | Patch |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Init | `Sdk.init(...)` + `Sdk.start()` | Replace body with `return-void` |
|
| Init | `Sdk.init(...)` + `Sdk.start()`, plugin download | body -> `return-void` |
|
||||||
| Show splash/interstitial | constructs an ad object, `loadAd()` / `show()` | Replace body, **but must still invoke the completion callback** |
|
| Readiness / gate | `isAdReady`, `canShow`, `isLoaded` | return `false` / `0` |
|
||||||
| Show reward | same, plus success/failure lambdas | Replace body; decide deliberately whether to grant the reward |
|
| Show splash / interstitial | constructs an ad object, `loadAd()` / `show()` | `return-void`, **but preserve the completion callback** |
|
||||||
| Preload/warm-up | `RewardVideoAutoAd.init(...)` | `return-void` |
|
| Show reward | same, plus success/failure lambdas | `return-void`; decide deliberately whether to grant the reward |
|
||||||
| Gate check | `isAdReady`, `canShow` | return `false` / `0` |
|
| Preload / warm-up | `RewardVideoAutoAd.init(...)`, cache priming | `return-void` |
|
||||||
|
|
||||||
### The callback trap (this one breaks startup)
|
### The callback trap (this one breaks startup)
|
||||||
|
|
||||||
A splash/loading ad usually takes a **completion lambda** as a parameter. The app's startup state machine waits for that lambda before it dismisses the splash screen.
|
A splash/loading ad usually takes a **completion lambda** as a parameter, and the app's startup state machine waits for that lambda before dismissing the splash screen.
|
||||||
|
|
||||||
**If you replace the method with a bare `return-void`, the lambda never fires, and the app hangs forever on the splash screen.**
|
**If you replace the method with a bare `return-void`, the lambda never fires and the app hangs on the splash screen forever.** A patch that "removed the ad" and produced a frozen splash is this bug, not a repackaging bug.
|
||||||
|
|
||||||
Correct shape — skip the ad, still complete:
|
Correct shape, skip the ad and still complete:
|
||||||
|
|
||||||
```
|
```
|
||||||
.method public final showSplash(Landroid/app/Activity;Landroid/view/ViewGroup;Lkotlin/jvm/functions/Function0;)V
|
.method public final showSplash(Landroid/app/Activity;Landroid/view/ViewGroup;Lkotlin/jvm/functions/Function0;)V
|
||||||
@@ -57,34 +79,59 @@ Correct shape — skip the ad, still complete:
|
|||||||
.end method
|
.end method
|
||||||
```
|
```
|
||||||
|
|
||||||
**Before patch, prove the contract.** Read the listener class the method constructs and find which callback triggers the completion lambda (`onAdDismiss`? `onAdError`? `onAdLoadTimeout`?). Only that tells you whether to call it on success, failure, or unconditionally.
|
**Before patching, prove the contract.** Read the listener class the method constructs and find which callback triggers the completion lambda (`onAdDismiss`? `onAdError`? `onAdLoadTimeout`?). That tells you whether to call it on success, on failure, or unconditionally.
|
||||||
|
|
||||||
For a reward ad whose lambdas are `(onStart, (Boolean) -> Unit)`, granting the reward client-side is a choice: `onFinish(true)` will make the app treat the reward as earned. Check whether the reward is validated server-side before promising it.
|
For a reward ad whose lambdas are `(onStart, (Boolean) -> Unit)`, granting the reward client-side is a choice: `onFinish(true)` makes the app treat the reward as earned. Check whether the reward is validated server-side before promising it.
|
||||||
|
|
||||||
|
### Global timestamp gates (one write, many slots)
|
||||||
|
|
||||||
|
Some "free reading + ads" apps do not gate ads per slot. They store one local timestamp and every slot, cooldown, and interval check reads it: splash, chapter-break interstitial, unlock prompt, "next ad in N minutes". Patching call sites one by one always misses some.
|
||||||
|
|
||||||
|
- **Find it:** dump the app's preference keys and filter for ad-ish names (`ad`, `free`, `cooldown`, `interval`, `end_time`, `expire`, `next_`). Then hook `SharedPreferences.getLong/getInt/getString` and watch which key is read immediately before a slot appears. The key read across several screens is the gate.
|
||||||
|
- **Patch it:** in `Application.onCreate()`, write a far-future value as early as possible (for example `4102444800000L`). Avoid `Long.MAX_VALUE`: implementations often compute `end - now` and can misjudge an extreme value. One write disables every reader at once, which is far more stable than editing each consumer.
|
||||||
|
- **If the app overwrites it later:** observe what it writes back under different start sequences. A single startup write is not automatically permanent for keys the app re-asserts; then patch the writer or the reader instead.
|
||||||
|
- **Same technique, two key semantics:** "ad-free until `<date>`" keys and "cooldown/interval" keys both respond to this, but the value differs. For a cooldown key, a far-future value means "next ad is never due", which is what you want, while a progress/check-in flow reading the same key may behave oddly.
|
||||||
|
- **Side effects are mandatory in the delivery notes.** The same key is often reused as a business field (the anchor for a new-user ad-free window, a remaining-quota counter). Forcing it to a far-future value changes those business judgements. Usually harmless for a de-ad goal, but state it explicitly and confirm the related screens do not show a broken state.
|
||||||
|
- With DataStore instead of SharedPreferences, a write inside `Application.onCreate` needs a coroutine or a blocking write; see `runtime-data.md`, and never block the main thread on persistence.
|
||||||
|
|
||||||
|
### "Ads" vs ad-gating UI
|
||||||
|
|
||||||
|
Some blocking dialogs contain no ad slot at all ("watch an ad to unlock", skip/continue prompts, cooldown notices). If the goal is "the app is usable", they count as ads and must be handled too. Handle them by letting the gate pass, not by deleting the dialog view.
|
||||||
|
|
||||||
|
- Where the gate reads the timestamp key above, the timestamp patch already covers it.
|
||||||
|
- Where it reads a boolean or a `canUnlock()` method, patch that predicate.
|
||||||
|
- **Business-coupled entrances need a check first.** "Watch a video to earn benefits" may be a real feature. Decide with evidence: find the callers (`scripts/find_refs.py`) and see whether the entrance leads to a **server request** or a **local entitlement write**. A server-issued entitlement cannot be created client-side, so releasing the gate only fools the UI while the server still refuses; an entrance that changes purely local UI state is safe to release.
|
||||||
|
|
||||||
## Step 3: choose the patch layer
|
## Step 3: choose the patch layer
|
||||||
|
|
||||||
From safest to riskiest (see `references/dex-patching.md` for the full table):
|
From safest to riskiest (full technique table in `dex-patching.md`):
|
||||||
|
|
||||||
1. **Kill SDK init** — the SDK never starts; its networks are never contacted. Highest-value single patch. Verify nothing else depends on the SDK being initialized.
|
1. **Kill SDK init** (Step 2) — the SDK never starts; its networks, caches, and plugin downloads never happen. Highest-value single patch. Verify nothing else silently depended on init.
|
||||||
2. **No-op the show/preload methods** — preserving callbacks.
|
2. **Neutralize the readiness gate** — `isReady`/`canShow` return false; routes existing "no ad" branches.
|
||||||
3. **Neutralize the gate** — `isAdReady`/`canShow` return false.
|
3. **No-op the show/preload methods** — only when 1 and 2 are insufficient, always preserving callbacks.
|
||||||
4. **Filter server-issued ad data at the consumption layer** — drop entries for the target position before they reach UI state.
|
4. **Global gate state** — one startup write covers every reader (timestamp / preference keys above).
|
||||||
5. *(avoid)* **Renderer-level suppression** — only if the component is genuinely ad-exclusive. Check `references/pitfalls.md` P6 first.
|
5. **Filter server-issued ad data at the consumption layer** — drop entries for the target position before they reach UI state.
|
||||||
6. *(avoid)* **Transport-level blocking** — see `references/pitfalls.md` P5.
|
6. *(avoid)* **Renderer-level suppression** — only if the component is genuinely ad-exclusive. Check `pitfalls.md` P6 first.
|
||||||
|
7. *(avoid)* **Transport-level blocking** — see `pitfalls.md` P5.
|
||||||
|
|
||||||
## Step 4: verify — and verify it properly
|
## Step 4: verify — three independent kinds of evidence
|
||||||
|
|
||||||
Logcat silence alone is weak evidence. Use several independent signals:
|
Logcat silence alone is weak evidence. A credible "ads are gone" claim needs all three, plus the neighbours:
|
||||||
|
|
||||||
1. **Process stays alive** through startup and across a few screens; no `FATAL EXCEPTION`, no `VerifyError`, no `uncaughtException`.
|
1. **UI, screen by screen** — splash, home banner, detail page, reader/player, reward button, unlock prompt. Capture before/after screenshots (`verification.md` §screenshot discipline).
|
||||||
2. **SDK log keywords absent** — `anythink`, `ATSDK`, `Pangle`, `TTAd`, `GDT`, etc. should not appear at all.
|
2. **Logs** — the SDK's characteristic tags are **entirely absent**, not merely quiet: `anythink`, `ATSDK`, `Pangle`, `TTAd`, `GDT`, `ksad`, and so on.
|
||||||
3. **DNS / socket level (strongest, and cheap)** — hook or observe name resolution. If the ad SDK's network stack is initialized it will resolve its own domains. Many ad SDKs **bypass the system proxy**, so proxy logs may miss them; a runtime `InetAddress` hook does not.
|
3. **Filesystem, the strongest signal** — the app's private directory contains **no ad SDK working directory at all**:
|
||||||
- Before: a burst of ad/tracker domains.
|
```bash
|
||||||
- After: **none at all** — meaning the init path never ran, not merely "shown but not rendered".
|
adb shell "su -c 'ls -la /data/data/<app.package>/files/'"
|
||||||
4. **Exercise the screens that had ads** — splash, home banner, detail page, player, reward button.
|
```
|
||||||
5. **Confirm nothing unrelated regressed** — images load, video plays, lists scroll, login works.
|
For a dynamic-plugin SDK, an absent `<sdk>_p` tree proves the init gate never ran. It is not an artifact you can fake by hiding a view. If the tree exists, something still initializes the SDK: the init patch did not take effect, or a second init path exists.
|
||||||
|
|
||||||
A useful framing for the report: distinguish *"the ad was hidden"* from *"the ad subsystem never started"*. The second is a much stronger claim, and DNS/socket evidence can prove it.
|
Additional cheap evidence:
|
||||||
|
4. **DNS / socket level** — a runtime `InetAddress` hook. Before: a burst of ad/tracker domains. After: none. Many ad SDKs bypass the system proxy, so a proxy log can miss them while this does not.
|
||||||
|
5. **Exercise the screens that had ads** and confirm nothing unrelated regressed: images, playback, lists, pagination, downloads, login.
|
||||||
|
6. **Clean install** — uninstall first, then install. Data left from a previous run can hide a failed init patch (`pitfalls.md` P11).
|
||||||
|
|
||||||
|
Frame the result at the right strength: *"the ad was hidden"* vs *"the ad subsystem never started"*. Only evidence 3 and 4 support the second (`verification.md` §the claim ladder).
|
||||||
|
|
||||||
## Step 5: what is usually NOT removable
|
## Step 5: what is usually NOT removable
|
||||||
|
|
||||||
@@ -92,6 +139,7 @@ Be honest about residual ads rather than breaking the app to chase them:
|
|||||||
|
|
||||||
- **Server-driven content promos** deep in a feature's own data payload, where the screen's load depends on the same request. Suppressing them requires a data-consumption patch, not transport blocking.
|
- **Server-driven content promos** deep in a feature's own data payload, where the screen's load depends on the same request. Suppressing them requires a data-consumption patch, not transport blocking.
|
||||||
- **Ads delivered as content** (a sponsored "article" or a native card with no SDK marker) — indistinguishable from real content without runtime tracing.
|
- **Ads delivered as content** (a sponsored "article" or a native card with no SDK marker) — indistinguishable from real content without runtime tracing.
|
||||||
- **Ads whose SDK init also enables other features.** Removing init can break functionality that silently depended on it.
|
- **Ads whose SDK init also enables other features.** Removing init can break functionality that silently depended on it. Verify before shipping; a working app with one residual ad beats a broken app.
|
||||||
|
- **Server-issued entitlements gated behind watching an ad.** The gate can be opened client-side, the entitlement cannot be created client-side.
|
||||||
|
|
||||||
If a residual ad cannot be removed without breaking something, say so, and say exactly which coupling caused it. That is a better deliverable than a broken APK.
|
If a residual ad cannot be removed without breaking something, say so, and say exactly which coupling caused it. That is a better deliverable than a broken APK.
|
||||||
|
|||||||
+252
-21
@@ -9,31 +9,68 @@ Use Frida when you need to answer:
|
|||||||
- Which domains does it resolve, and when?
|
- Which domains does it resolve, and when?
|
||||||
- Is the app detecting my instrumentation?
|
- Is the app detecting my instrumentation?
|
||||||
|
|
||||||
## Setup
|
## Setup — version alignment is a hard gate
|
||||||
|
|
||||||
1. **Matching versions.** The `frida` Python package on the host and `frida-server` on the device must be the same version. Mismatch produces confusing handshake failures.
|
**Do this first.** Nearly every "Frida is broken on this device" report is a version mismatch, and the error text rarely says so. The host `frida` package and the device-side `frida-server` must be the **same version**, and that version must actually support the device's Android release. Align, then debug everything else.
|
||||||
2. **Run the server as root**, and give it a non-obvious name — many hardened apps look for processes/files named `frida-server`.
|
|
||||||
```bash
|
| Symptom | Cause | Fix |
|
||||||
adb push frida-server-<ver>-android-<abi> /data/local/tmp/
|
|---|---|---|
|
||||||
adb shell "su -c 'mkdir -p /data/local/tmp/.svc'"
|
| `unable to locate Android dynamic linker` | host/server too new for this Android release | drop to an older frida line (16.x is a safe baseline for older ROMs) |
|
||||||
adb shell "su -c 'cp -f /data/local/tmp/frida-server-<ver>-android-<abi> /data/local/tmp/.svc/kwork'"
|
| `Java is not defined` / `Java.perform is not a function` | that build ships no bundled Java bridge | align to a build that has it, or inline the bridge (see *17+ gotchas*) |
|
||||||
adb shell "su -c 'chmod 755 /data/local/tmp/.svc/kwork'"
|
| `Failed to connect to remote frida-server` / `unexpected message` / `invalid message` | host package and device server differ | download both from the same release tag |
|
||||||
adb shell "su -c '/data/local/tmp/.svc/kwork' &"
|
| `Java.choose` / `Java.use` throws immediately | bridge present but the VM is not ready | wrap everything in `Java.perform` |
|
||||||
```
|
| Attached, but hooks fire in the wrong process | USB auto-selection grabbed the emulator | use an explicit remote device (below) |
|
||||||
3. **Keep the server alive.** Launching it through a shell that exits can get it reaped. Hold the connection open from the host (spawn it from a script that stays alive), or use a supervisor on device.
|
| `HOOK-OK` prints and nothing ever fires | not a version problem | go to *Hook never fires* |
|
||||||
4. **Pick the right ABI** — `getprop ro.product.cpu.abi`.
|
|
||||||
5. **Spawn, don't attach, when timing matters.** Attaching usually misses startup (init, first network calls, splash logic).
|
### Prefer a remote device over USB
|
||||||
```python
|
|
||||||
device = frida.get_usb_device()
|
With a physical device *and* an emulator attached, `frida.get_usb_device()` can silently pick the emulator — you then attach to the wrong process and chase phantom failures for a long time. Address the device explicitly:
|
||||||
pid = device.spawn([pkg])
|
|
||||||
session = device.attach(pid)
|
```bash
|
||||||
# ... load script, wait until hooks are installed ...
|
adb devices -l # get <serial>
|
||||||
device.resume(pid) # only resume AFTER hooks are ready
|
adb -s <serial> forward tcp:27042 tcp:27042
|
||||||
```
|
frida -H 127.0.0.1:27042 -f <app.package> -l probe.js --no-pause
|
||||||
**Common bug:** resuming before the script has finished loading loses the first ~100–300 ms, which is exactly where init happens. Have the script `send()` a ready signal and resume only after receiving it.
|
```
|
||||||
|
|
||||||
|
```python
|
||||||
|
device = frida.get_device_manager().add_remote_device('127.0.0.1:27042')
|
||||||
|
```
|
||||||
|
|
||||||
|
Keep `-s <serial>` on every other adb call too, or you will drift between targets mid-session.
|
||||||
|
|
||||||
|
### Start the device server so it survives
|
||||||
|
|
||||||
|
Run it as root, give it a non-obvious name out of obvious paths, and **detach it from the shell's session**. A process started with `nohup ... &` from an `adb shell` that then exits gets reaped mid-run — the hooks work for a minute and then stop.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb push frida-server-<ver>-android-<abi> /data/local/tmp/
|
||||||
|
adb shell "su -c 'mkdir -p /data/local/tmp/.svc'"
|
||||||
|
adb shell "su -c 'cp -f /data/local/tmp/frida-server-<ver>-android-<abi> /data/local/tmp/.svc/kwork'"
|
||||||
|
adb shell "su -c 'chmod 755 /data/local/tmp/.svc/kwork'"
|
||||||
|
adb shell "su -c 'setsid /data/local/tmp/.svc/kwork >/dev/null 2>&1 </dev/null &'"
|
||||||
|
adb shell "su -c 'pgrep -f kwork'" # must still be alive after the shell returned
|
||||||
|
```
|
||||||
|
|
||||||
|
If `setsid` is unavailable, hold the adb connection open instead: run `adb shell "su -c '/data/local/tmp/.svc/kwork'"` as a host-side background job and leave it running for the whole session.
|
||||||
|
|
||||||
|
### Spawn, don't attach, when timing matters
|
||||||
|
|
||||||
|
ABI must match the device (`getprop ro.product.cpu.abi`). Attaching usually misses startup — init, the first network calls, and splash logic all happen before you get a session.
|
||||||
|
|
||||||
|
```python
|
||||||
|
device = frida.get_device_manager().add_remote_device('127.0.0.1:27042')
|
||||||
|
pid = device.spawn([pkg])
|
||||||
|
session = device.attach(pid)
|
||||||
|
# ... load the script, wait until it reports that hooks are installed ...
|
||||||
|
device.resume(pid) # resume ONLY after hooks are ready
|
||||||
|
```
|
||||||
|
|
||||||
|
**Common bug:** resuming before the script has finished loading loses the first ~100–300 ms, which is exactly where init happens. Have the script `send()` a ready signal (the probe template below sends `PROBE-READY`) and resume only after receiving it.
|
||||||
|
|
||||||
## Frida 17+ gotchas
|
## Frida 17+ gotchas
|
||||||
|
|
||||||
|
The preferred fix for every symptom in the table above is **version alignment**. Use these only when you genuinely must run a 17+ build.
|
||||||
|
|
||||||
- The **built-in Java bridge was removed**. `Java.perform(...)` is not available unless you inline the bridge yourself:
|
- The **built-in Java bridge was removed**. `Java.perform(...)` is not available unless you inline the bridge yourself:
|
||||||
```python
|
```python
|
||||||
bridge = open(<site-packages>/frida_tools/bridges/java.js, encoding='utf-8').read()
|
bridge = open(<site-packages>/frida_tools/bridges/java.js, encoding='utf-8').read()
|
||||||
@@ -100,6 +137,200 @@ Inet.getAllByName.overload('java.lang.String').implementation = function (h) {
|
|||||||
```
|
```
|
||||||
This is often the **decisive evidence** for an ad-removal claim: if the ad SDK's domains are *never resolved*, the subsystem never started — a much stronger statement than "logcat was quiet". It also works for SDKs that bypass the system HTTP proxy.
|
This is often the **decisive evidence** for an ad-removal claim: if the ad SDK's domains are *never resolved*, the subsystem never started — a much stronger statement than "logcat was quiet". It also works for SDKs that bypass the system HTTP proxy.
|
||||||
|
|
||||||
|
## The four-layer probe template
|
||||||
|
|
||||||
|
When the question is "why did this request fail / where did it go", do not hook one class and hope. Ship **one long-lived script that hooks four layers at once**; whichever layer fires first localizes the problem immediately. This is the single most productive artifact of a runtime investigation — reuse it verbatim.
|
||||||
|
|
||||||
|
The layers, and why each one is there:
|
||||||
|
|
||||||
|
1. **The app's own network wrapper** — enumerate `getDeclaredMethods` and wrap *every* overload, so you do not have to guess the entry point.
|
||||||
|
2. **OkHttp end to end** — `newCall`, `Request$Builder.build`, `RealCall.execute`, `AsyncCall.run`, `RealInterceptorChain.proceed`. The last two are the ones that fire for asynchronous calls.
|
||||||
|
3. **`java.net.URL.openConnection`** — plenty of login/register paths never touch OkHttp; they use `HttpsURLConnection` and a completely different trust configuration.
|
||||||
|
4. **`Throwable.getMessage`** — pulls out the original text of exceptions an upper layer caught and swallowed. Without it, a caught failure looks like "nothing happened".
|
||||||
|
|
||||||
|
Rules that make the difference between a usable probe and a wasted session:
|
||||||
|
|
||||||
|
- **Write to a file as well as to `send()`.** Stdout is lossy and the CLI drops messages; the file is the evidence.
|
||||||
|
- **Let it stay resident.** Hook before the app touches the network, then leave it running while you drive the UI. A one-shot script misses everything that happens after its first second.
|
||||||
|
- **Wrap every hook in its own `try/catch`.** One missing class must not take down the other three layers.
|
||||||
|
- **Never capture an overload in a `var`.** Inside a loop, `var ov = ...` leaves every hook pointing at the *last* overload: they install cleanly and then mis-report forever. Use `let` or `.forEach()`.
|
||||||
|
|
||||||
|
`scripts/frida_probe.js` is the same probe in a fuller form (per-arity dispatch, DNS layer, dedup and hard caps). The script below is the minimal portable one — paste it into any loader and it adapts to the target.
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
// probe.js — four-layer network + swallowed-exception probe.
|
||||||
|
const APP_PKG = '<app.package>';
|
||||||
|
const APP_NET_CLASS = 'com.example.app.net.HttpHelper'; // the app's own wrapper, once you have found it
|
||||||
|
const LOGFILE = '/data/user/0/' + APP_PKG + '/files/probe.log'; // app-private: always writable by the app
|
||||||
|
const MAX_THROW = 400; // Throwable.getMessage is hot — cap it
|
||||||
|
const INTERESTING = /(network|http|ssl|cert|fail|timeout|refused|unable|error|exception)/i; // widen for the app's UI language
|
||||||
|
|
||||||
|
let out = null;
|
||||||
|
try { out = new File(LOGFILE, 'a'); } catch (e) {}
|
||||||
|
|
||||||
|
function log(ev, data) {
|
||||||
|
const line = JSON.stringify(Object.assign({ t: Date.now(), ev: ev }, data));
|
||||||
|
try { send(line); } catch (e) {}
|
||||||
|
try { if (out !== null) { out.write(line + '\n'); out.flush(); } } catch (e) {}
|
||||||
|
}
|
||||||
|
|
||||||
|
function stack(n) {
|
||||||
|
try {
|
||||||
|
const t = Java.use('java.lang.Throwable').$new();
|
||||||
|
return Java.use('android.util.Log').getStackTraceString(t).split('\n').slice(1, (n || 6) + 1).join(' | ');
|
||||||
|
} catch (e) { return '<no stack>'; }
|
||||||
|
}
|
||||||
|
|
||||||
|
function sa(args) { // a hook must never die inside its own logging
|
||||||
|
try {
|
||||||
|
const o = {};
|
||||||
|
for (let i = 0; i < args.length; i++) o['a' + i] = args[i] === null ? 'null' : '' + args[i];
|
||||||
|
return o;
|
||||||
|
} catch (e) { return { err: '' + e }; }
|
||||||
|
}
|
||||||
|
|
||||||
|
// 1) the app's own wrapper: every declared method, every overload, separately wrapped
|
||||||
|
function hookAppNet(cls) {
|
||||||
|
try {
|
||||||
|
const C = Java.use(cls);
|
||||||
|
const ms = C.class.getDeclaredMethods();
|
||||||
|
for (let i = 0; i < ms.length; i++) {
|
||||||
|
try {
|
||||||
|
const name = ms[i].getName();
|
||||||
|
const ps = ms[i].getParameterTypes();
|
||||||
|
const sig = [];
|
||||||
|
for (let j = 0; j < ps.length; j++) sig.push(ps[j].getName());
|
||||||
|
const ov = C[name].overload.apply(C[name], sig);
|
||||||
|
ov.implementation = function () {
|
||||||
|
try { log('APP-NET', { cls: cls, m: name, args: sa(arguments), stack: stack(5) }); }
|
||||||
|
catch (e) { log('PROBE-ERR', { at: 'app-net', msg: '' + e }); }
|
||||||
|
return ov.apply(this, arguments);
|
||||||
|
};
|
||||||
|
} catch (e) { log('HOOK-SKIP', { cls: cls, m: ms[i].getName(), msg: '' + e }); }
|
||||||
|
}
|
||||||
|
log('HOOK-OK', { layer: 'app-net', cls: cls, n: ms.length });
|
||||||
|
} catch (e) { log('HOOK-FAIL', { layer: 'app-net', cls: cls, msg: '' + e }); }
|
||||||
|
}
|
||||||
|
|
||||||
|
// 2) OkHttp end to end — internal class names moved between major versions, so try every location
|
||||||
|
function hookOkHttp() {
|
||||||
|
const targets = [
|
||||||
|
['okhttp3.OkHttpClient', 'newCall'],
|
||||||
|
['okhttp3.Request$Builder', 'build'],
|
||||||
|
['okhttp3.RealCall', 'execute'],
|
||||||
|
['okhttp3.RealCall$AsyncCall', 'run'],
|
||||||
|
['okhttp3.internal.http.RealInterceptorChain', 'proceed'], // okhttp 3.x
|
||||||
|
['okhttp3.internal.connection.RealInterceptorChain', 'proceed'] // okhttp 4.x / 5.x
|
||||||
|
];
|
||||||
|
for (let i = 0; i < targets.length; i++) {
|
||||||
|
const cn = targets[i][0], mn = targets[i][1];
|
||||||
|
try {
|
||||||
|
const ovs = Java.use(cn)[mn].overloads; // every overload of that name; use .overload('<sig>') if this is undefined
|
||||||
|
for (let j = 0; j < ovs.length; j++) {
|
||||||
|
const ov = ovs[j]; // block-scoped: every hook keeps its own overload
|
||||||
|
ov.implementation = function () {
|
||||||
|
try { log('OKHTTP', { cls: cn, m: mn, args: sa(arguments), stack: stack(6) }); }
|
||||||
|
catch (e) { log('PROBE-ERR', { at: cn, msg: '' + e }); }
|
||||||
|
return ov.apply(this, arguments);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
log('HOOK-OK', { layer: 'okhttp', cls: cn, m: mn, overloads: ovs.length });
|
||||||
|
} catch (e) { log('HOOK-SKIP', { layer: 'okhttp', cls: cn, m: mn, msg: '' + e }); }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3) java.net.URL — login/register frequently bypasses OkHttp entirely
|
||||||
|
function hookRawUrl() {
|
||||||
|
try {
|
||||||
|
const U = Java.use('java.net.URL');
|
||||||
|
const o0 = U.openConnection.overload();
|
||||||
|
o0.implementation = function () {
|
||||||
|
try { log('RAW-URL', { url: '' + this.toString(), stack: stack(6) }); } catch (e) {}
|
||||||
|
return o0.apply(this, arguments);
|
||||||
|
};
|
||||||
|
try {
|
||||||
|
const o1 = U.openConnection.overload('java.net.Proxy');
|
||||||
|
o1.implementation = function () {
|
||||||
|
try { log('RAW-URL', { url: '' + this.toString(), proxy: true, stack: stack(6) }); } catch (e) {}
|
||||||
|
return o1.apply(this, arguments);
|
||||||
|
};
|
||||||
|
} catch (e) {}
|
||||||
|
log('HOOK-OK', { layer: 'url' });
|
||||||
|
} catch (e) { log('HOOK-FAIL', { layer: 'url', msg: '' + e }); }
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4) Throwable.getMessage — the original text of an exception an upper layer caught and swallowed
|
||||||
|
function hookThrowable() {
|
||||||
|
try {
|
||||||
|
const T = Java.use('java.lang.Throwable');
|
||||||
|
const seen = {};
|
||||||
|
let n = 0;
|
||||||
|
T.getMessage.implementation = function () {
|
||||||
|
const msg = this.getMessage(); // self-call inside the replacement is fine (Frida guards it)
|
||||||
|
try {
|
||||||
|
if (msg !== null && msg !== undefined && n < MAX_THROW) {
|
||||||
|
const cls = '' + this.getClass().getName();
|
||||||
|
const interesting = cls.indexOf('Exception') >= 0 || cls.indexOf('Error') >= 0 || INTERESTING.test('' + msg);
|
||||||
|
const key = cls + '|' + msg;
|
||||||
|
if (interesting && seen[key] === undefined) { // dedup: this method is on a very hot path
|
||||||
|
seen[key] = 1; n++;
|
||||||
|
log('THROW', { cls: cls, msg: '' + msg, stack: stack(4) });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (e) {}
|
||||||
|
return msg;
|
||||||
|
};
|
||||||
|
log('HOOK-OK', { layer: 'throwable' });
|
||||||
|
} catch (e) { log('HOOK-FAIL', { layer: 'throwable', msg: '' + e }); }
|
||||||
|
}
|
||||||
|
|
||||||
|
Java.perform(function () {
|
||||||
|
hookAppNet(APP_NET_CLASS);
|
||||||
|
hookOkHttp();
|
||||||
|
hookRawUrl();
|
||||||
|
hookThrowable();
|
||||||
|
log('PROBE-READY', { pkg: APP_PKG, logfile: LOGFILE }); // the host waits for this before resuming
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
Read the log after driving the UI. `RAW-URL` (with no `OKHTTP` events) is how you learn the request went over `HttpsURLConnection` — a different trust path entirely (`references/server-api.md` § TLS). `THROW` is how you learn an upper layer swallowed the real error.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# pull the record (app-private path needs root to read)
|
||||||
|
adb -s <serial> shell "su -c 'cat /data/user/0/<app.package>/files/probe.log'" > probe.log
|
||||||
|
```
|
||||||
|
|
||||||
|
## Hook never fires — debug in this order
|
||||||
|
|
||||||
|
`HOOK-SKIP` / `HOOK-FAIL` lines answer this before you start guessing. When every layer reports `HOOK-OK` and still no event arrives:
|
||||||
|
|
||||||
|
1. **Does the class / method / overload actually exist?** A misspelled obfuscated name, a renamed okhttp internal class, or an overload taking `Object` instead of `String` all produce a wrapper that is simply never invoked.
|
||||||
|
2. **Is the ClassLoader the right one?** Multi-dex, plugin-loaded and packed apps can hold several loaders; `Java.use` uses the app loader by default and throws `ClassNotFoundException` for a class that is plainly in the APK. Enumerate and switch:
|
||||||
|
```javascript
|
||||||
|
Java.enumerateClassLoaders({
|
||||||
|
onMatch: function (l) { try { if (l.findClass('<app.net.Class>')) Java.classFactory.loader = l; } catch (e) {} },
|
||||||
|
onComplete: function () {}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
3. **Is that code path reached at all?** Layer 4 settles it: if `THROW` events show the app failing earlier, your hook's call site is never executed and no amount of hooking will help.
|
||||||
|
4. **Did your UI action trigger business logic?** A tap that looks fine but fails a local form check returns before any network call. Verify the input actually reached the field (`references/environment.md` § UI automation), not just that the button animated.
|
||||||
|
|
||||||
|
## `Java.choose` also matches dead instances
|
||||||
|
|
||||||
|
`Java.choose('com.example.app.MainActivity', ...)` returns **every** instance the VM still tracks, including finished and destroyed ones. Walking their view tree then yields an empty list — that is normal, not a bug in your script. Filter first, and prefer a direct reflective call over simulating a tap:
|
||||||
|
|
||||||
|
```javascript
|
||||||
|
Java.choose('com.example.app.MainActivity', {
|
||||||
|
onMatch: function (a) {
|
||||||
|
try {
|
||||||
|
if (a.isFinishing() || a.isDestroyed()) return; // dead instance: skip it
|
||||||
|
send({ tag: 'live', view: '' + a.findViewById(<viewId>) });
|
||||||
|
} catch (e) {}
|
||||||
|
},
|
||||||
|
onComplete: function () {}
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
## Reading obfuscated code at runtime
|
## Reading obfuscated code at runtime
|
||||||
|
|
||||||
When names are meaningless (`a7`, `x6`, `uf0`), do not patch from names. Instead:
|
When names are meaningless (`a7`, `x6`, `uf0`), do not patch from names. Instead:
|
||||||
|
|||||||
@@ -89,6 +89,61 @@ su -c "restorecon -R /data/user/0/<pkg>"
|
|||||||
```
|
```
|
||||||
`<uid>` from `dumpsys package <pkg> | grep userId=`.
|
`<uid>` from `dumpsys package <pkg> | grep userId=`.
|
||||||
|
|
||||||
|
## Driving the UI from adb
|
||||||
|
|
||||||
|
Automated UI interaction is where verification loops usually break, and most of the breakage looks like a bug in your script. Most of it is not.
|
||||||
|
|
||||||
|
### `input tap` may simply not work on a given control
|
||||||
|
|
||||||
|
On some OEM ROMs `input tap <x> <y>` is silently ignored for certain controls while working fine for others at neighboring coordinates. **Do not conclude that your automation is wrong.** Before doubting the script:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb -s <serial> shell "su -c 'uiautomator dump /sdcard/ui.xml'"
|
||||||
|
adb -s <serial> pull /sdcard/ui.xml
|
||||||
|
# bounds="[x1,y1][x2,y2]" -> tap the center: ((x1+x2)/2, (y1+y2)/2)
|
||||||
|
adb -s <serial> shell "su -c 'input tap <cx> <cy>'"
|
||||||
|
```
|
||||||
|
|
||||||
|
If the control still does not react, bypass the UI path entirely — `am start -n <app.package>/<activity>`, or invoke the logic from Frida. A finished task needs the *code path*, not the tap.
|
||||||
|
|
||||||
|
`input` requires `INJECT_EVENTS`, which the plain `shell` user does not have. An unprivileged `input` **fails silently** — no error, no effect. Always wrap it: `su -c 'input tap ...'`.
|
||||||
|
|
||||||
|
`input text` additionally mangles or drops non-ASCII input. For CJK text, either install an ADB-driven helper IME or set the field from a runtime call rather than typing.
|
||||||
|
|
||||||
|
### Screen evidence without pixels
|
||||||
|
|
||||||
|
`screencap` returns a 0-byte file on some ROMs. First try writing on-device and pulling:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb -s <serial> shell "su -c 'screencap -p /sdcard/x.png'" && adb -s <serial> pull /sdcard/x.png
|
||||||
|
```
|
||||||
|
|
||||||
|
If that is also empty, do not fight it: `uiautomator dump`'s XML is **better** evidence — text, diffable, and it records the real control tree and its contents. (If the dump fails with `could not get idle state`, retry once; a paused animation is usually the cause.)
|
||||||
|
|
||||||
|
### Verify form input by reading it back
|
||||||
|
|
||||||
|
Filling a form and tapping submit is **not** a verified interaction. Read the actual field contents and lengths back first:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb -s <serial> shell "su -c 'input text <value>'" # no literal spaces; %s encodes one
|
||||||
|
adb -s <serial> shell "su -c 'uiautomator dump /sdcard/ui.xml'"
|
||||||
|
adb -s <serial> pull /sdcard/ui.xml
|
||||||
|
grep -o 'text="[^"]*"' ui.xml # every populated field
|
||||||
|
```
|
||||||
|
|
||||||
|
A whole class of "the button does nothing" is really a local validation rejecting the input — two password fields of different length, a required field empty, a format check. The app returns **before** issuing any request, so a network probe stays silent and the tap looks broken. Compare the lengths you read back, fix the input, retry.
|
||||||
|
|
||||||
|
### ROM background freezing kills your hooks
|
||||||
|
|
||||||
|
Aggressive ROMs freeze backgrounded apps; the logcat signature is a process state transition `state: R -> F` (running to frozen). Once frozen, hooks stop firing and network calls stop, which looks exactly like a broken probe.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb -s <serial> shell "su -c 'dumpsys deviceidle whitelist +<app.package>'"
|
||||||
|
adb -s <serial> shell "su -c 'cmd appops set <app.package> RUN_IN_BACKGROUND allow'"
|
||||||
|
```
|
||||||
|
|
||||||
|
While debugging, **do not press HOME** — backgrounding the app is what triggers the freeze. Return to it with `am start -n <app.package>/<activity>` and keep it in the foreground for the whole session.
|
||||||
|
|
||||||
## Signal extraction (what to actually read)
|
## Signal extraction (what to actually read)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
@@ -248,3 +248,114 @@ Never build device commands inline in the host shell. Either:
|
|||||||
- put the device-side logic in a script that you push and execute.
|
- put the device-side logic in a script that you push and execute.
|
||||||
|
|
||||||
Same rule for `javac`: always pass `-encoding UTF-8` when sources contain non-ASCII, or the compiler reads them as the platform default and fails.
|
Same rule for `javac`: always pass `-encoding UTF-8` when sources contain non-ASCII, or the compiler reads them as the platform default and fails.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P13. Trusting `apksigner verify` output at face value
|
||||||
|
|
||||||
|
**Symptom**
|
||||||
|
You signed with v1+v2+v3 explicitly enabled, then verification reports:
|
||||||
|
```
|
||||||
|
Verified using v1 scheme (JAR signing): false
|
||||||
|
Verified using v2 scheme (APK Signature Scheme v2): false
|
||||||
|
Verified using v3 scheme (APK Signature Scheme v3): true
|
||||||
|
```
|
||||||
|
Looks like v1/v2 silently did not happen, so you go re-engineer the signing step.
|
||||||
|
|
||||||
|
**Root cause**
|
||||||
|
`apksigner verify` decides **which schemes it is meaningful to check from the APK's own `minSdkVersion`**. When `minSdkVersion >= 24`, v1 is not required for install, and the tool reports it as `false` rather than "not applicable". The signatures are present and valid.
|
||||||
|
|
||||||
|
**Why it is hard to see**
|
||||||
|
Nothing in the output says "skipped because of minSdk". It reads exactly like a failure.
|
||||||
|
|
||||||
|
**Do instead**
|
||||||
|
Always verify with an explicit range so every scheme is evaluated:
|
||||||
|
```
|
||||||
|
apksigner verify --print-certs --verbose --min-sdk-version 21 --max-sdk-version 34 <apk>
|
||||||
|
```
|
||||||
|
Cross-check the fact independently: a real v1 signature means `META-INF/*.SF` and `META-INF/*.RSA` exist in the zip.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P14. Treating same-size dex dumps as duplicates
|
||||||
|
|
||||||
|
**Symptom**
|
||||||
|
You deduplicate a memory dump by file size, keep one of each, and later find the kept dex is unusable (or silently wrong).
|
||||||
|
|
||||||
|
**Root cause**
|
||||||
|
Two dumps from the same process can have **identical byte length but different content** — different classes, even different dex version headers. One may additionally be structurally broken (header fields inconsistent with body, parser walks off the end of a table).
|
||||||
|
|
||||||
|
**Why it is hard to see**
|
||||||
|
Size equality is a tempting shortcut and is usually right for *file* duplicates. Here it is coincidence: two distinct dex objects were allocated to equal-length blocks.
|
||||||
|
|
||||||
|
**Do instead**
|
||||||
|
- Deduplicate by hash, never by size.
|
||||||
|
- Validate every candidate before trusting it: check the `dex\n0xx` magic, then sanity-check the reported `file_size` / `header_size` / map offsets against the actual byte length.
|
||||||
|
- Cross-check against the packing format when possible: with length-preserving encryption, **the encrypted payload's byte length equals the plaintext dex's byte length** — that mapping is the strongest signal for which dump is the original.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P15. Frida version skew produces errors that look like a broken target
|
||||||
|
|
||||||
|
**Symptom**
|
||||||
|
Attach fails or the script dies immediately with errors such as:
|
||||||
|
```
|
||||||
|
unable to locate Android dynamic linker
|
||||||
|
Java is not defined
|
||||||
|
```
|
||||||
|
on a device where Frida is clearly running.
|
||||||
|
|
||||||
|
**Root cause**
|
||||||
|
A host `frida` package newer than the device's `frida-server` (or the reverse) is unsupported. Additionally, some newer host versions dropped the built-in Java bridge, so `Java.perform` is undefined unless you inline the bridge yourself.
|
||||||
|
|
||||||
|
**Why it is hard to see**
|
||||||
|
The error names the linker or a missing global, not a version mismatch. It reads like an Android compatibility problem or an anti-instrumentation defense.
|
||||||
|
|
||||||
|
**Do instead**
|
||||||
|
- Pin host package and device server to the **identical** version before debugging anything else. Print both versions side by side first.
|
||||||
|
- If the target is an older Android release, prefer the oldest version that still supports your API needs rather than the newest.
|
||||||
|
- On a device with multiple attached targets, do not rely on automatic USB selection — see `references/dynamic-frida.md`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P16. Blaming your own patch for a server-side TLS failure
|
||||||
|
|
||||||
|
**Symptom**
|
||||||
|
After repacking, the app launches and browsing works, but **login / registration** fails with a network error. The obvious suspect is the new signature breaking the API contract, so you start hunting for a signature check in the client.
|
||||||
|
|
||||||
|
**Root cause**
|
||||||
|
The failure is at the TLS layer, not the application layer: the API host's certificate is expired (or the chain does not validate), and that particular request path validates against the **system trust store**. Shipping a different signature is irrelevant.
|
||||||
|
|
||||||
|
Crucially, one app can carry **two independent trust chains**: requests through the app's own HTTP client (which may install a permissive `SSLSocketFactory` and `HostnameVerifier`) succeed, while requests through `java.net.URL.openConnection()` use the system defaults and fail. That is why "some features work" and "login does not".
|
||||||
|
|
||||||
|
**Why it is hard to see**
|
||||||
|
The user-visible message is a generic "network error". The real exception is usually swallowed by the app's own `try/catch`. And a client-side patch is the most recent change, so it gets blamed by default.
|
||||||
|
|
||||||
|
**Do instead**
|
||||||
|
- Capture the whole exception chain before theorizing. The give-away is
|
||||||
|
`CertPathValidatorException: timestamp check failed` → `CertificateException: Chain validation failed` → `SSLHandshakeException: Chain validation failed`.
|
||||||
|
- Confirm independently of the app: strictly validate the host's certificate from your host machine and check `notAfter` against the device clock. See `references/tls-and-cert.md`.
|
||||||
|
- Run the control: does the **unmodified original** fail the same way on the same device and network? If yes, it was never your patch.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## P17. Trusting UI automation to prove whether a patch worked
|
||||||
|
|
||||||
|
**Symptom**
|
||||||
|
Your script taps a button, nothing happens, and you conclude the patch broke the control. Or you tap, see no visible change, and conclude the feature is dead.
|
||||||
|
|
||||||
|
**Root cause**
|
||||||
|
Device input and screenshots are far less reliable than they look:
|
||||||
|
- `input tap` can silently fail on specific widgets even with correct coordinates (ROM-dependent).
|
||||||
|
- `input` needs `INJECT_EVENTS`; under a plain shell it fails quietly.
|
||||||
|
- `screencap` can return a **zero-byte** file on some ROMs.
|
||||||
|
- Form submission can be rejected by local validation before any request is made, so "the button does nothing" is a validation failure, not a broken handler.
|
||||||
|
|
||||||
|
**Why it is hard to see**
|
||||||
|
All of these produce the same observable: nothing happens. A zero-byte screenshot often goes unnoticed and is treated as "no change".
|
||||||
|
|
||||||
|
**Do instead**
|
||||||
|
- Read back the widget tree (`uiautomator dump`) instead of trusting pixels: it gives real `bounds`, control text, and **field contents with lengths**. Verify every field is populated correctly *before* submitting.
|
||||||
|
- Compare field values, not just presence — one case that burned an hour was two password fields of different length, causing local validation to `return` before any network call.
|
||||||
|
- Treat "no visible change" as unproven, not as a negative result: confirm with an independent signal (logcat, a runtime probe, or a server-side request appearing in the capture).
|
||||||
|
- If a tap does not register, fall back to launching the Activity directly or invoking the handler, rather than retrying coordinates.
|
||||||
|
|||||||
+49
-4
@@ -23,13 +23,57 @@ Read `AndroidManifest.xml` → `application android:name`.
|
|||||||
|---|---|
|
|---|---|
|
||||||
| The app's own class (e.g. `com.example.app.App`) | **No packer.** Dex is directly editable. |
|
| The app's own class (e.g. `com.example.app.App`) | **No packer.** Dex is directly editable. |
|
||||||
| A vendor class (`com.stub.StubApp`, `com.secneo...`, `com.tencent.StubShell`, `s.h.e.l.l.*`, `com.nagain.*`, …) | **Packed.** Unpack first. |
|
| A vendor class (`com.stub.StubApp`, `com.secneo...`, `com.tencent.StubShell`, `s.h.e.l.l.*`, `com.nagain.*`, …) | **Packed.** Unpack first. |
|
||||||
| `android:appComponentFactory` overridden to a vendor class | **Packed or wrapped** — often a runtime shell that loads a second APK. |
|
| A short meaningless name that does not match the app's package (e.g. `com.bc41`), plus `android:appComponentFactory` pointing at another short class | **Dex-level whole-APK packer.** The shell registers its own Application/proxy under names unrelated to the app. |
|
||||||
| `assets/` contains a second `.apk`, `.jar`, or an encrypted blob | Wrapped/loader build. |
|
| `classes.dex` is only tens of KB and holds nothing but shell classes | The real code is not on disk. |
|
||||||
|
| A shell library in `lib/<abi>/` (`libDexHelper.so`, `libjiagu*.so`, `libshell*.so`, …) | Native half of the shell: decrypts and loads the payload. |
|
||||||
|
| One or more large high-entropy `assets/` blobs, possibly **disguised as JPEG** (valid magic bytes, no decodable image) | Encrypted payload. |
|
||||||
|
| `assets/` contains a second `.apk` or `.jar` | Wrapped/loader build. |
|
||||||
|
|
||||||
Signs in `lib/`: lone `.so` files with names unlike normal libraries (a custom-named shell lib), plus unusual `assets/` artifacts.
|
Decide from sizes and structure, not from names:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
unzip -l app.apk | sort -k1 -n | tail -20 # biggest entries: assets? libs? dex?
|
||||||
|
```
|
||||||
|
|
||||||
**Loaded-shell check (highest fidelity):** if the app runs and its own classes are reachable, it is unpacked *at runtime* regardless of what the manifest says. If the app's own code is not among the dex classes, it is packed on disk.
|
**Loaded-shell check (highest fidelity):** if the app runs and its own classes are reachable, it is unpacked *at runtime* regardless of what the manifest says. If the app's own code is not among the dex classes, it is packed on disk.
|
||||||
|
|
||||||
|
### Unpacking a dex-level packer
|
||||||
|
|
||||||
|
**1. Map payload → dumped dex by length.** If the shell uses **length-preserving** encryption, each encrypted `assets/` payload has **exactly the same byte count as the plaintext dex** it decrypts to. That equality is the strongest single discriminator for "which dump is a real original dex" — far more reliable than sorting dumps by size. Measure it per file before trusting any dump.
|
||||||
|
|
||||||
|
**2. Check how the real dex is loaded.** A log line like `InMemoryDexFile[cookie=[0, ...]]` means the dex is built in memory from the payload and never written to disk, so there is no dropped file to find. Use a **memory-scanning** dumper (`frida-dexdump`, or your own script walking the class loaders) instead of hunting for artifacts under the app's data directory. See `references/dynamic-frida.md`.
|
||||||
|
|
||||||
|
**3. Two dumper caveats that cost real time.**
|
||||||
|
- `-D <device-id>` does **not** accept a remote device (`Device '<id>' not found`). Register a remote device with `-H <host>:<port>`; keep `-D` for local/USB ids.
|
||||||
|
- The `-o <dir>` output directory **must already exist** — the tool does not create it. Otherwise every dex write fails and the run still ends with `All done`, which reads exactly like a successful dump of nothing. `mkdir -p out` first, and judge the result by the file count, never by the final message.
|
||||||
|
|
||||||
|
**4. Filter the dumps before using any of them.** A memory dump contains three kinds of garbage:
|
||||||
|
- **Duplicate copies of one dex.** Identical byte count does **not** imply identical content — files of the same size have been measured to differ in ~73% of their bytes, carry a different dex version, and fail to parse. Dedupe by **content hash**, never by size.
|
||||||
|
- **SDK plugin dexes downloaded at runtime**, which were never part of the APK.
|
||||||
|
- **Structurally broken dexes** whose header and body disagree.
|
||||||
|
|
||||||
|
Cheap header check that catches the third class, plus the hash used for the first:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import struct, hashlib
|
||||||
|
d = open(p, 'rb').read()
|
||||||
|
assert d[:4] == b'dex\n' # magic
|
||||||
|
assert struct.unpack_from('<I', d, 32)[0] == len(d) # file_size == actual size
|
||||||
|
assert struct.unpack_from('<I', d, 36)[0] == 112 # header_size
|
||||||
|
sha = hashlib.sha256(d).hexdigest()
|
||||||
|
```
|
||||||
|
|
||||||
|
Then confirm survivors actually parse (`baksmali`/dexlib2) or load on device. Header consistency alone is not proof of a usable dex.
|
||||||
|
|
||||||
|
**5. Which dump is the app's own business dex.** Count string-feature hits per dump instead of trusting file order:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python scripts/dex_strings.py <dump_dir> --find 'Lcom/example/app/' --per-file
|
||||||
|
python scripts/dex_strings.py <dump_dir> --find 'Lcom/example/adsdk/' --per-file
|
||||||
|
```
|
||||||
|
|
||||||
|
Heuristic: the business dex mixes the app's package prefix (hundreds of hits) with an ad-SDK prefix (thousands of hits). A pure SDK plugin dex has one vendor prefix at its maximum count and **zero** app-package hits. Zero app-package hits means it is not the app.
|
||||||
|
|
||||||
## 3. Where does the app's own code live?
|
## 3. Where does the app's own code live?
|
||||||
|
|
||||||
| Indicator | Code location |
|
| Indicator | Code location |
|
||||||
@@ -48,7 +92,7 @@ For a **Kotlin/Java app**, note the module layout: app code often lives in a sma
|
|||||||
python scripts/dex_strings.py <dex_dir> --find 'Lcom/example/app/' --per-file
|
python scripts/dex_strings.py <dex_dir> --find 'Lcom/example/app/' --per-file
|
||||||
```
|
```
|
||||||
|
|
||||||
Consequence for editing: you usually only need to replace **one or two dex files**, which keeps the repack minimal.
|
Consequence for editing: you usually only need to replace **one or two dex files**, which keeps the repack minimal. When the app is packed, run this check against the **dumped** dexes, not the ones on disk (§2).
|
||||||
|
|
||||||
## 4. SDK and library inventory
|
## 4. SDK and library inventory
|
||||||
|
|
||||||
@@ -102,6 +146,7 @@ Write a short profile before patching. Minimum:
|
|||||||
```
|
```
|
||||||
package / version / ABI
|
package / version / ABI
|
||||||
packed? (evidence)
|
packed? (evidence)
|
||||||
|
if packed: packer family, unpack method, kept dumps + sha256
|
||||||
code location (dex / flutter / native)
|
code location (dex / flutter / native)
|
||||||
app code in which dex files
|
app code in which dex files
|
||||||
ad SDK(s) and the app's wrapper class
|
ad SDK(s) and the app's wrapper class
|
||||||
|
|||||||
@@ -30,6 +30,17 @@ Recompressing them produces builds that fail to install or misbehave.
|
|||||||
|
|
||||||
**5. Signing creates new `MANIFEST.MF`/`*.SF`/`*.RSA`.** That is expected — the check is that **no signature artifact from the ORIGINAL** survives, and **no non-signature entry was lost**.
|
**5. Signing creates new `MANIFEST.MF`/`*.SF`/`*.RSA`.** That is expected — the check is that **no signature artifact from the ORIGINAL** survives, and **no non-signature entry was lost**.
|
||||||
|
|
||||||
|
## Repacking an unpacked (de-shelled) app
|
||||||
|
|
||||||
|
When the sample was packed, the dex you are about to patch came out of a memory dump while the APK still carries the shell. Build a **de-shelled base APK** first, then treat it as an ordinary APK for the rest of this document.
|
||||||
|
|
||||||
|
1. **Assemble the base.** Start from the original APK: replace `classes*.dex` with the dumped real dexes, and delete the shell library under `lib/<abi>/` plus the encrypted `assets/` payloads. Strip **only** signature artifacts — `META-INF/*.SF|*.RSA|*.DSA|*.EC` and the top-level `MANIFEST.MF`. Never delete the whole `META-INF/` (`references/pitfalls.md` P1).
|
||||||
|
2. **Decode with `apktool`.** `apktool d base.apk` gives the smali tree and a **readable text manifest**.
|
||||||
|
3. **Fix the manifest.** Point `application android:name` at the app's real Application class and **remove** `android:appComponentFactory`. Grep the decoded tree for leftover shell class references before building.
|
||||||
|
4. **Patch, then rebuild.** `apktool b` → `zipalign -p -f 4` → `apksigner` (v1+v2+v3) → install and verify against the checklist at the end of this file.
|
||||||
|
|
||||||
|
Why `apktool` rather than editing binary AXML in place: changing `android:name` in binary AXML means hand-editing the string pool and the attribute/chunk sizes around it, and a mis-sized chunk produces an APK that installs but throws far from the edit, typically in component lookup at startup. The text round-trip moves the risk to "did the rebuild preserve everything else", which item 5 of the checklist verifies directly.
|
||||||
|
|
||||||
## Pipeline
|
## Pipeline
|
||||||
|
|
||||||
`scripts/repack.py` implements all of the above. Conceptually:
|
`scripts/repack.py` implements all of the above. Conceptually:
|
||||||
@@ -76,7 +87,8 @@ zipalign -p -f 4 unsigned.apk aligned.apk
|
|||||||
apksigner sign --ks ks.jks --ks-pass pass:<pass> --key-pass pass:<pass> \
|
apksigner sign --ks ks.jks --ks-pass pass:<pass> --key-pass pass:<pass> \
|
||||||
--v1-signing-enabled true --v2-signing-enabled true --v3-signing-enabled true \
|
--v1-signing-enabled true --v2-signing-enabled true --v3-signing-enabled true \
|
||||||
--out signed.apk aligned.apk
|
--out signed.apk aligned.apk
|
||||||
apksigner verify --print-certs -v signed.apk
|
apksigner verify --print-certs --verbose \
|
||||||
|
--min-sdk-version 21 --max-sdk-version 34 signed.apk
|
||||||
```
|
```
|
||||||
|
|
||||||
Enable **v1 + v2 + v3**. v1 is needed for older Android; v2/v3 for modern verification.
|
Enable **v1 + v2 + v3**. v1 is needed for older Android; v2/v3 for modern verification.
|
||||||
@@ -89,6 +101,12 @@ keytool -genkeypair -v -keystore ks.jks -alias <alias> \
|
|||||||
-dname "CN=<name>, OU=dev, O=dev, L=NA, ST=NA, C=NA"
|
-dname "CN=<name>, OU=dev, O=dev, L=NA, ST=NA, C=NA"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## `apksigner verify` picks schemes from the APK's own minSdk
|
||||||
|
|
||||||
|
`apksigner verify` decides which signature schemes to check from the APK's own `minSdkVersion`. With `minSdk >= 24` it prints v1/v2 as `false` **by design**, even though `META-INF/*.SF` is present and the signature is valid. That output is indistinguishable from "signing did not apply" and sends you into a re-signing loop over a file that was never wrong.
|
||||||
|
|
||||||
|
Always pass the explicit range (Route B above) and read the scheme list from that run only. Never judge a build from a default-argument `apksigner verify`.
|
||||||
|
|
||||||
## Installing over an existing app
|
## Installing over an existing app
|
||||||
|
|
||||||
| Situation | Command |
|
| Situation | Command |
|
||||||
@@ -115,11 +133,11 @@ Useful detail: keeping the same keystore across builds lets you iterate with `-r
|
|||||||
|
|
||||||
Signature verification proves the file is well-formed. It does **not** prove the app works. Always:
|
Signature verification proves the file is well-formed. It does **not** prove the app works. Always:
|
||||||
|
|
||||||
1. `apksigner verify` (or equivalent) passes.
|
1. `apksigner verify` passes with the explicit SDK range above — a default-args run can report v1/v2 `false` on a perfectly valid signature.
|
||||||
2. Install succeeds.
|
2. Install succeeds.
|
||||||
3. Launch succeeds; process stays alive.
|
3. Launch succeeds; process stays alive.
|
||||||
4. `logcat` shows no `FATAL EXCEPTION` / `VerifyError` / `IncompatibleClassChangeError` / `uncaughtException`.
|
4. `logcat` shows no `FATAL EXCEPTION` / `VerifyError` / `IncompatibleClassChangeError` / `uncaughtException`.
|
||||||
5. `META-INF/services/*` count matches the original.
|
5. `META-INF/services/*` count matches the original.
|
||||||
6. Unchanged dex files are byte-identical to the original (compare hashes).
|
6. Unchanged dex files are byte-identical to the original (compare hashes); in a de-shelled build, compare against the de-shelled base instead.
|
||||||
|
|
||||||
Build a **control** at least once: same pipeline, zero patches. If the control fails, your pipeline or environment is at fault, not your patch (`references/pitfalls.md` P9).
|
Build a **control** at least once: same pipeline, zero patches. If the control fails, your pipeline or environment is at fault, not your patch (`references/pitfalls.md` P9).
|
||||||
|
|||||||
@@ -0,0 +1,139 @@
|
|||||||
|
# Third-party builds — auditing a "cracked" or "modded" APK before trusting it
|
||||||
|
|
||||||
|
## Why this exists
|
||||||
|
|
||||||
|
A modded build of the exact app you are targeting is the most tempting shortcut available:
|
||||||
|
it appears to prove the goal is achievable, and it may already contain the patch you were
|
||||||
|
about to write. It is also the single most common source of wasted hours and of real risk,
|
||||||
|
for two reasons.
|
||||||
|
|
||||||
|
1. **It is usually not editable.** Authors who redistribute mods protect them, often with
|
||||||
|
more layers than the original app ever had. You cannot treat someone else's build as a
|
||||||
|
patching workbench.
|
||||||
|
2. **Its target-audience behaviour is unverified.** The binary may carry injected components,
|
||||||
|
extra endpoints, or embedded credentials. "It runs and the ads are gone" tells you nothing
|
||||||
|
about what else it does.
|
||||||
|
|
||||||
|
Treat a third-party build as an **untrusted sample to be audited**, never as a baseline.
|
||||||
|
The useful output of an audit is a decision plus evidence — not a binary you start from.
|
||||||
|
|
||||||
|
## Step 1: Can you even edit it?
|
||||||
|
|
||||||
|
Find the app's own package prefix (the one that appears in its own class definitions, e.g.
|
||||||
|
`com/<vendor>/<app>/`) and count how many times it appears as a **defined class** in each
|
||||||
|
plaintext dex.
|
||||||
|
|
||||||
|
```
|
||||||
|
business-prefix hits per dex:
|
||||||
|
classes.dex 0 <- shell
|
||||||
|
classes2.dex 0
|
||||||
|
classes3.dex 0
|
||||||
|
classes4.dex 0
|
||||||
|
```
|
||||||
|
|
||||||
|
Zero everywhere means the business code has been **moved out of the plaintext dexes** — the
|
||||||
|
build is protected at a level you cannot patch directly, even though it launches and works.
|
||||||
|
Stop here and treat it as a black box.
|
||||||
|
|
||||||
|
Also check the reported dex count against the file count: a manifest-declared app with only
|
||||||
|
library classes present is the same signal.
|
||||||
|
|
||||||
|
## Step 2: How many protection layers does it carry?
|
||||||
|
|
||||||
|
Mods frequently stack protection from multiple vendors, sometimes because the author
|
||||||
|
re-protected an already-protected app. Look for independent, coexisting families:
|
||||||
|
|
||||||
|
| Layer kind | Typical static evidence |
|
||||||
|
|---|---|
|
||||||
|
| Commercial packer | shell `Application`/`appComponentFactory` in the manifest, a `lib*protect*.so`, encrypted assets, `assets/*.jar` with near-maximal entropy |
|
||||||
|
| DEX-to-native (dex2c) | a "stub" `lib*.so` containing the tool's name, `EntryPoint`/`protect`-style symbols, and **business class names as strings** alongside a near-empty dex layer |
|
||||||
|
| Anti-debugging / RASP | strings such as `TracerPid`, `ptrace`, `NoNewPrivs`, `CheckPtraceSelf`, `AntiFrida`, plus a self-protection log path |
|
||||||
|
| Generic hardening marker files | small text files under `assets/` naming a protection vendor and a protection timestamp |
|
||||||
|
|
||||||
|
The practical consequence of an anti-debugging layer is that Frida-based dynamic work will hit
|
||||||
|
a self-check before it hits your hook. Note it, and prefer static conclusions on such a sample.
|
||||||
|
|
||||||
|
## Step 3: What did the author ADD?
|
||||||
|
|
||||||
|
This is where the audit earns its keep. Diff against the **original build of the same version**
|
||||||
|
(referenced as `<original>` below).
|
||||||
|
|
||||||
|
### New components
|
||||||
|
|
||||||
|
Build the set difference of `activity` / `service` / `receiver` / `provider` declarations,
|
||||||
|
then check whether each new class name actually exists in the plaintext dexes.
|
||||||
|
|
||||||
|
- Class exists and is readable → you can judge its purpose.
|
||||||
|
- Class name appears **nowhere** in plaintext but shows up inside a native library's strings →
|
||||||
|
the logic is in the native/dex2c layer, purpose unknown. Say so.
|
||||||
|
- Nothing registered with an implicit-launch intent-filter (`BOOT_COMPLETED`, `MAIN`) → it cannot
|
||||||
|
be started implicitly, which limits (but does not eliminate) the risk.
|
||||||
|
- A provider with an `authorities` value containing a package-name rewrite (a suffix/prefix that
|
||||||
|
is not the current package) is a strong hint the build was produced by **repackaging or app
|
||||||
|
cloning**, not by editing the original.
|
||||||
|
|
||||||
|
### New outbound endpoints
|
||||||
|
|
||||||
|
Extract every URL/domain/IP from the modded build's `lib/*.so`, `assets/*`, and dex strings,
|
||||||
|
and subtract the original's set. Any domain that exists **only** in the modded build is the most
|
||||||
|
important finding in the audit.
|
||||||
|
|
||||||
|
Rank what you find:
|
||||||
|
- A domain that also serves install attribution or update checks is explainable.
|
||||||
|
- A domain paired with an authorization-looking path (e.g. a `GET...`-style endpoint name
|
||||||
|
nearby in the same string neighbourhood) is *not* explainable by de-ad work.
|
||||||
|
|
||||||
|
### Embedded secrets
|
||||||
|
|
||||||
|
Search native libraries and assets for private-key markers:
|
||||||
|
```
|
||||||
|
-----BEGIN ... PRIVATE KEY----- , PRIVATE KEY , PKCS#8 , RSA PRIVATE
|
||||||
|
```
|
||||||
|
A **private key shipped inside a client artifact** means its "authorization" or "integrity"
|
||||||
|
path is not a security boundary — anyone can extract the key. Report it as a red flag on the
|
||||||
|
build's trust claims, regardless of intent.
|
||||||
|
|
||||||
|
## Step 4: Capability picture — read both sides
|
||||||
|
|
||||||
|
A native library's import table is only half the story.
|
||||||
|
|
||||||
|
- **What it does not have** is meaningful: no `socket`/`connect`, no `execve`/`fork`/`system`,
|
||||||
|
no `ptrace`, no installer-session strings → no direct evidence of command execution or
|
||||||
|
silent install.
|
||||||
|
- **What it references** can override that impression: string references to Java-layer
|
||||||
|
`java/net/Socket`, `InetSocketAddress`, reflection entry points, or executor services mean the
|
||||||
|
capability exists **through the Java layer**, off the native import table.
|
||||||
|
|
||||||
|
State findings as "capability present / no evidence of use" rather than "safe" — an import table
|
||||||
|
is evidence of shape, not of intent.
|
||||||
|
|
||||||
|
## Step 5: The verdict
|
||||||
|
|
||||||
|
Do not hand back a binary. Hand back a decision with evidence and explicit unknowns.
|
||||||
|
|
||||||
|
```
|
||||||
|
VERDICT: use / use with caution / do not use
|
||||||
|
EVIDENCE:
|
||||||
|
- business code plaintext? yes/no (+ counts)
|
||||||
|
- protection layers list
|
||||||
|
- injected components list, each with purpose known/unknown
|
||||||
|
- endpoints present only in this build list
|
||||||
|
- embedded credentials yes/no
|
||||||
|
UNCONFIRMED:
|
||||||
|
- <each item you could not establish, stated plainly>
|
||||||
|
```
|
||||||
|
|
||||||
|
"Use with caution" is the honest answer whenever a build is un-analysable **and** carries new
|
||||||
|
endpoints or unknown injected components. Do not upgrade it to "safe" because it launches
|
||||||
|
cleanly, and do not call it malicious without evidence — both are guesses.
|
||||||
|
|
||||||
|
## Using the audit result
|
||||||
|
|
||||||
|
Two legitimate outcomes:
|
||||||
|
|
||||||
|
1. The audit shows the mod is an honest de-ad of the same version → you may use it to **learn the
|
||||||
|
approach** (diff its patch points against the original), but still rebuild from the original
|
||||||
|
yourself so the artifact you ship has a known provenance.
|
||||||
|
2. The audit shows it is re-protected, injected, or un-analysable → discard it and do the work on
|
||||||
|
the original. That is usually faster than reverse-engineering someone else's protection *and*
|
||||||
|
your target's at the same time.
|
||||||
@@ -0,0 +1,178 @@
|
|||||||
|
# TLS and certificate failures
|
||||||
|
|
||||||
|
**Symptom class:** after a repack, login or registration fails with a TLS error while the rest of the app behaves normally. This is the most common way a *correct* patch gets blamed for a *pre-existing, server-side* condition. Work the five steps below before changing a single byte of the app.
|
||||||
|
|
||||||
|
## Fast path
|
||||||
|
|
||||||
|
| # | Action | Evidence |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | Read the full exception chain, prove it is a date rule | `THROW` events from the probe, or logcat |
|
||||||
|
| 2 | Validate the certificate from a host you trust | `python scripts/tls_check.py <host>` |
|
||||||
|
| 3 | Compare another host from the same app, and the device clock | same script, `adb shell date` |
|
||||||
|
| 4 | Decide which trust chain owns the failing request | `RAW-URL` vs `OKHTTP` probe events |
|
||||||
|
| 5 | Only then patch the system-trust path | fix template below |
|
||||||
|
|
||||||
|
## Symptoms
|
||||||
|
|
||||||
|
- Login or registration reports a generic network failure: `Network request failed: Chain validation failed`.
|
||||||
|
- Everything else works: home, feed, images, playback, guest browsing.
|
||||||
|
- The full chain, visible only if something captures swallowed exceptions:
|
||||||
|
|
||||||
|
```
|
||||||
|
javax.net.ssl.SSLHandshakeException: Chain validation failed
|
||||||
|
caused by: java.security.cert.CertificateException: Chain validation failed
|
||||||
|
caused by: java.security.cert.CertPathValidatorException: timestamp check failed
|
||||||
|
```
|
||||||
|
|
||||||
|
- `timestamp check failed` is the decisive token. The validator rejected the chain on a **date rule**, not on an untrusted issuer or a hostname mismatch. Exactly two conditions produce it: the certificate is expired or not yet valid, or the **clock of whoever performs the check is wrong**.
|
||||||
|
|
||||||
|
## Step 1: get the real host out of the runtime
|
||||||
|
|
||||||
|
Never trust a host recovered from strings alone; a build config can hold dead or staging hosts. Get the host the app actually connects to:
|
||||||
|
|
||||||
|
- Run the four-layer probe: `scripts/frida_probe.js`, launched and streamed to a log by `scripts/run_probe.py` (setup, version alignment, and hooking strategy: `dynamic-frida.md`). Three event labels carry what you need.
|
||||||
|
- `RAW-URL`: the request went through `java.net.URL.openConnection` / `openStream`, the system trust path.
|
||||||
|
- `OKHTTP`: the request went through the app's own OkHttp client, a different trust path.
|
||||||
|
- `THROW`: the original text of an exception an upper layer caught and swallowed. This is where the real TLS message appears when the UI only shows a generic network error.
|
||||||
|
|
||||||
|
Output you need: the exact URL, its host, and which label logged it.
|
||||||
|
|
||||||
|
## Step 2: validate the chain out of band (no app involved)
|
||||||
|
|
||||||
|
This step is independent of the app, the patch, and the device. It runs on a host whose clock you trust:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python scripts/tls_check.py <host> # strict verification + certificate detail
|
||||||
|
python scripts/tls_check.py <host> <host2> # compare hosts in one run
|
||||||
|
python scripts/tls_check.py <host> --json # machine-readable; exit 2 if any host fails
|
||||||
|
```
|
||||||
|
|
||||||
|
The script validates strictly (system CAs, expiry, hostname) and, when verification fails, still decodes the peer certificate locally, so a failure arrives with a reason instead of a stack trace:
|
||||||
|
|
||||||
|
```
|
||||||
|
EXPIRED certificate has expired
|
||||||
|
NOT-YET-VALID certificate is not valid yet
|
||||||
|
HOSTNAME-MISMATCH chain is fine, the name does not match
|
||||||
|
UNTRUSTED-CA self-signed, or issuer not in the trust store
|
||||||
|
CHAIN-BROKEN bad signature / invalid CA / incomplete chain
|
||||||
|
```
|
||||||
|
|
||||||
|
Interpretation:
|
||||||
|
|
||||||
|
- `EXPIRED` reported from a trustworthy clock is proof that the chain is rejected by the certificate itself, not by the device, the network, or your patch.
|
||||||
|
- A clean pass means the certificate is fine and the failure is a client-side trust decision or a clock problem. Go to Step 3.
|
||||||
|
- Non-443 endpoint, or an address that needs a different SNI name (an IP connect, a CDN edge): `--port <port>` and `--sni <host>`. Getting this wrong produces a `HOSTNAME-MISMATCH` that has nothing to do with the real problem.
|
||||||
|
|
||||||
|
Minimal equivalent, if the script is not at hand:
|
||||||
|
|
||||||
|
```python
|
||||||
|
import socket, ssl
|
||||||
|
ctx = ssl.create_default_context() # strict: system CAs, expiry, hostname
|
||||||
|
with socket.create_connection(("<host>", 443), timeout=10) as raw:
|
||||||
|
with ctx.wrap_socket(raw, server_hostname="<host>") as tls:
|
||||||
|
print("OK", tls.version())
|
||||||
|
# on failure: ssl.SSLCertVerificationError: ... certificate has expired
|
||||||
|
```
|
||||||
|
|
||||||
|
Dates, subject, and issuer in one line, if you prefer no script at all:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
openssl s_client -connect <host>:443 -servername <host> </dev/null 2>/dev/null \
|
||||||
|
| openssl x509 -noout -subject -issuer -dates
|
||||||
|
```
|
||||||
|
|
||||||
|
## Step 3: compare another host, and check the device clock
|
||||||
|
|
||||||
|
Run the same check against every host you extracted from the app, then compare:
|
||||||
|
|
||||||
|
- A very common shape: the business API host is expired while the config, image, or CDN host is valid. Same app, same network, same machine, different certificates. That pattern exonerates the network path and points at one certificate.
|
||||||
|
- Check the device clock; it costs one command:
|
||||||
|
```bash
|
||||||
|
adb shell date # compare against a known-correct time source
|
||||||
|
```
|
||||||
|
If Step 2 passes but the device fails, the cause is the device clock, not the certificate. Correcting the clock and relaxing the trust policy are different decisions; do not do both silently.
|
||||||
|
- Check the **unpatched original** in the same environment. See `verification.md` §the control build rule and `pitfalls.md` P9. If the original fails identically, the patch is not the cause, and no amount of dex work will fix it.
|
||||||
|
|
||||||
|
## Step 4: which trust chain owns the failing request
|
||||||
|
|
||||||
|
One app can carry two independent trust chains, which is exactly why "login is broken but everything else works".
|
||||||
|
|
||||||
|
| Path | Client | Trust configuration | Failure signature |
|
||||||
|
|---|---|---|---|
|
||||||
|
| OkHttp | OkHttp / Retrofit, via `newCall` | app-supplied `SSLSocketFactory` + `HostnameVerifier`, often pinning | pin-related messages (`CertificatePinner`, pin verification failed), or nothing at all if the app relaxed it |
|
||||||
|
| `java.net.URL.openConnection()` | `HttpsURLConnection`, also used by many image loaders, downloaders, and legacy HTTP wrappers | **platform default** trust store and default verifier (`HttpsURLConnection.setDefault*`) | `Chain validation failed` / `timestamp check failed`, the signature above |
|
||||||
|
|
||||||
|
Discriminating, in increasing order of certainty:
|
||||||
|
|
||||||
|
1. **Probe events.** The failing request appears under exactly one label from Step 1: `RAW-URL` (system path) or `OKHTTP` (app client). This is a fast screen, not proof.
|
||||||
|
2. **Trust-manager attribution (definitive).** Hook every `X509TrustManager.checkServerTrusted` implementation and print the instance class name plus a short `new Exception().getStackTrace()`. If the caller frames sit in `okhttp3.*`, or in a custom class the app installs on its own client, the OkHttp path owns the failure. If they sit in a platform trust manager reached through `HttpsURLConnection` / `SSLSocket` internals, the system path owns it. Wording alone is version-dependent, so trust the caller frames, not the message.
|
||||||
|
3. **Cross-check with the fix below.** If OkHttp owns the failing request, the template does nothing for it: that path never reads `HttpsURLConnection` defaults and needs a client-level change (`SSLSocketFactory` / `HostnameVerifier` / pinner; see `dex-patching.md` §where to patch).
|
||||||
|
|
||||||
|
Why the asymmetry exists: the app configured its own trust behavior for OkHttp (a relaxed verifier or a pin), so that path never consults the platform date rule, while the legacy `HttpsURLConnection` path still validates against the system trust store and rejects the expired chain.
|
||||||
|
|
||||||
|
## Step 5: fix template (system path only)
|
||||||
|
|
||||||
|
Install permissive TLS defaults for the `HttpsURLConnection` path at the earliest point of `Application.onCreate()`:
|
||||||
|
|
||||||
|
```java
|
||||||
|
// Call from Application.onCreate() before any other initialization.
|
||||||
|
private static void installPermissiveTlsDefaults() {
|
||||||
|
try {
|
||||||
|
javax.net.ssl.SSLContext ctx = javax.net.ssl.SSLContext.getInstance("TLS");
|
||||||
|
ctx.init(null, new javax.net.ssl.TrustManager[]{
|
||||||
|
new javax.net.ssl.X509TrustManager() {
|
||||||
|
public void checkClientTrusted(java.security.cert.X509Certificate[] c, String a) {}
|
||||||
|
public void checkServerTrusted(java.security.cert.X509Certificate[] c, String a) {}
|
||||||
|
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
|
||||||
|
return new java.security.cert.X509Certificate[0];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, new java.security.SecureRandom());
|
||||||
|
javax.net.ssl.HttpsURLConnection.setDefaultSSLSocketFactory(ctx.getSocketFactory());
|
||||||
|
javax.net.ssl.HttpsURLConnection.setDefaultHostnameVerifier(
|
||||||
|
new javax.net.ssl.HostnameVerifier() {
|
||||||
|
public boolean verify(String hostname, javax.net.ssl.SSLSession session) { return true; }
|
||||||
|
});
|
||||||
|
} catch (Exception ignored) {
|
||||||
|
// Leave platform defaults in place: pre-patch behavior, no new failure mode.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Land it as a small utility class plus its inner classes in smali, or as a dexlib2 rewrite that inserts the call at the head of `Application.onCreate` (`dex-patching.md`). Keep it inside the app's own package; do not edit third-party classes.
|
||||||
|
|
||||||
|
Why this shape is correct:
|
||||||
|
|
||||||
|
- It changes **one** trust decision: the default `HttpsURLConnection` factory and verifier. OkHttp derives its trust manager through `TrustManagerFactory` and its own socket factory, so app-side pinning and verifier behavior is untouched. You are not silently weakening a control the app deliberately added.
|
||||||
|
- Call it **before any other initialization**. Globals set after the first `HttpsURLConnection` use still apply, but a library that cached a factory reference earlier keeps the old one, which produces a confusing partial fix.
|
||||||
|
- The scope is still wider than "login". **Every request that goes through `URL.openConnection()`** inherits it, including image loaders, download managers, and older push or analytics SDKs. State that in the report instead of implying you touched one endpoint.
|
||||||
|
- The `catch` that leaves platform defaults alone is deliberate: the worst case is the behavior you started with.
|
||||||
|
- This is a **client-side workaround for an external condition**. The correct fix is renewing the server certificate. Say so, and do not present the patch as the fix.
|
||||||
|
|
||||||
|
Do not: change the device clock, install your own CA into the device trust store (device-side, harder to roll back, and not part of a shippable artifact), rewrite the request host to a different domain, or reach for `System.setProperty` tricks. Each of them either misses the actual cause or widens the change far beyond the failing flow.
|
||||||
|
|
||||||
|
## If OkHttp is the failing path
|
||||||
|
|
||||||
|
Then the failure is about the app's own client configuration, not the platform defaults. Typical causes: a pin that no longer matches, a hostname verifier rejecting a changed host, or a custom `SSLSocketFactory` built from a bundled keystore. Locate it the same way (Step 4, item 2) and patch at the client construction site instead of applying the template above. `dynamic-frida.md` covers finding the real construction site; `dex-patching.md` covers the patch layer.
|
||||||
|
|
||||||
|
## Method notes
|
||||||
|
|
||||||
|
- **Order of investigation is the lesson.** Every piece of evidence above comes from the server, the network, or the platform before anything is attributed to your patch: full exception chain, real host, out-of-band validation, second-host comparison, control build. Only when all of them point at a client trust decision is a patch justified. `pitfalls.md` P16 is this mistake written up as a failure.
|
||||||
|
- **This is a control-build case, not a patch case.** `verification.md` §the control build rule applies directly: same pipeline, zero patches. If the control fails the same way, stop debugging the patch.
|
||||||
|
- **Report the residual honestly.** "The server certificate is expired; the client patch relaxes validation on the `HttpsURLConnection` path only, OkHttp is unchanged, the proper fix is on the server" is the useful statement. "I fixed login" is not.
|
||||||
|
- **Claim strength.** This evidence chain reaches rung 6 on the `verification.md` claim ladder: the mechanism is proven, not merely observed. Say which rung you reached.
|
||||||
|
|
||||||
|
## Triage checklist
|
||||||
|
|
||||||
|
```
|
||||||
|
[ ] Full exception chain captured, contains "timestamp check failed"
|
||||||
|
[ ] Exact host and URL obtained from a runtime probe (not from strings)
|
||||||
|
[ ] Standalone strict validation run: EXPIRED / clean pass
|
||||||
|
[ ] A second host from the same app compared (isolates the certificate)
|
||||||
|
[ ] Device clock checked against a known-correct source
|
||||||
|
[ ] Original unpatched build reproduces the same failure (control build)
|
||||||
|
[ ] Failing request attributed to a trust manager implementation, not to a message
|
||||||
|
[ ] If system path: permissive defaults installed at the top of Application.onCreate
|
||||||
|
[ ] If OkHttp path: client-level config is the patch target instead
|
||||||
|
[ ] Delivery notes state the client-side scope and the server-side root cause
|
||||||
|
```
|
||||||
@@ -14,9 +14,10 @@ Each rung is stronger evidence. Climb as high as the task requires, and **state
|
|||||||
| 3 | "It installs and launches" | Process alive, `logcat` free of fatal signatures |
|
| 3 | "It installs and launches" | Process alive, `logcat` free of fatal signatures |
|
||||||
| 4 | "The target behavior changed" | The specific feature observed changed (screen exercised, endpoint observed, UI confirmed) |
|
| 4 | "The target behavior changed" | The specific feature observed changed (screen exercised, endpoint observed, UI confirmed) |
|
||||||
| 5 | "Nothing else regressed" | Adjacent features exercised: images, playback, lists, login, settings |
|
| 5 | "Nothing else regressed" | Adjacent features exercised: images, playback, lists, login, settings |
|
||||||
| 6 | "The mechanism is proven" | Independent evidence of *why* — e.g. the SDK's domains are never resolved because init never ran |
|
| 6 | "The mechanism is proven" | Independent evidence of *why* — e.g. the SDK's domains are never resolved because init never ran, or the SDK never created its working directory under the app's private storage |
|
||||||
|
| 7 | "The *distributed* artifact is the verified one" | Re-download the published file and compare its hash against the local build you tested |
|
||||||
|
|
||||||
**Rung 3 is the minimum for any deliverable. Rung 4 for a claim that the patch solved the problem. Rung 5 before handing it to a user. Rung 6 when the claim is "the subsystem is dead", not merely "the ad is hidden".**
|
**Rung 3 is the minimum for any deliverable. Rung 4 for a claim that the patch solved the problem. Rung 5 before handing it to a user. Rung 6 when the claim is "the subsystem is dead", not merely "the ad is hidden". Rung 7 whenever the deliverable leaves your machine — an upload can truncate, a build can be re-signed by a pipeline step, and the person downloading it has no way to know.**
|
||||||
|
|
||||||
## The control build rule
|
## The control build rule
|
||||||
|
|
||||||
@@ -71,7 +72,7 @@ Failure signature → cause table lives in `references/environment.md`.
|
|||||||
|
|
||||||
Launching proves nothing about your change. Exercise it.
|
Launching proves nothing about your change. Exercise it.
|
||||||
|
|
||||||
- **Ads removed** → open the screens that had ads (splash, home, detail, player, reward button). Confirm absence *and* confirm the screens still function.
|
- **Ads removed** → open the screens that had ads (splash, home, detail, player, reward button). Confirm absence *and* confirm the screens still function. Use **three independent signals**, not one: the screens look right, the SDK's log tags disappeared, and the SDK never created its working directory under the app's private storage. An SDK that silently failed to fill an ad slot looks exactly like an SDK that was never initialised — only the last two signals tell them apart.
|
||||||
- **A gate patched** → walk the gated flow end to end.
|
- **A gate patched** → walk the gated flow end to end.
|
||||||
- **A timeout/limit patched** → trigger it.
|
- **A timeout/limit patched** → trigger it.
|
||||||
- **Data patched** → stop the app, clear/re-write the value, cold start, re-check (proves it is not an in-memory artifact).
|
- **Data patched** → stop the app, clear/re-write the value, cold start, re-check (proves it is not an in-memory artifact).
|
||||||
@@ -98,17 +99,20 @@ Build: <path> <sha256>
|
|||||||
Base: <original apk name/version>
|
Base: <original apk name/version>
|
||||||
Changes: <dex → what changed>, one line each
|
Changes: <dex → what changed>, one line each
|
||||||
Structural: dex_classdiff result (per dex)
|
Structural: dex_classdiff result (per dex)
|
||||||
Signature: verified (v1/v2/v3)
|
Signature: verified with --min-sdk-version 21 (v1/v2/v3); report the raw command, not just "ok"
|
||||||
Device: <model / android / abi / rooted?>
|
Device: <model / android / abi / rooted?>
|
||||||
Runtime: pid stable over N s; no fatal signatures
|
Runtime: pid stable over N s; no fatal signatures
|
||||||
Behavior: <screens exercised>, <before/after>
|
Behavior: <screens exercised>, <before/after>
|
||||||
Regressions: <adjacent features checked>
|
Regressions: <adjacent features checked>
|
||||||
Rung reached: 0..6
|
Distribution: <where it was published> re-downloaded, sha256 matches the tested build
|
||||||
|
Rung reached: 0..7
|
||||||
Residual: <what is NOT fixed, and the exact reason>
|
Residual: <what is NOT fixed, and the exact reason>
|
||||||
```
|
```
|
||||||
|
|
||||||
**Be explicit about residuals.** "Ad X remains because it is delivered inside the screen's main data payload, and suppressing it at the transport layer takes the whole screen down" is a useful, honest result. Silently omitting it is not.
|
**Be explicit about residuals.** "Ad X remains because it is delivered inside the screen's main data payload, and suppressing it at the transport layer takes the whole screen down" is a useful, honest result. Silently omitting it is not.
|
||||||
|
|
||||||
|
**Separate the two audiences.** The artifact goes to an end user; the analysis report goes to whoever asked for the work. User-facing notes stay neutral and practical — package name and version, install prerequisites, the signature-conflict warning (a different signer means the old install must be removed first), the checksum, and a scope/disclaimer line. Keep the mechanism, the patch points, and the reasoning in your own report, not in the file the end user opens.
|
||||||
|
|
||||||
## Anti-patterns
|
## Anti-patterns
|
||||||
|
|
||||||
- Declaring success from a successful assembly.
|
- Declaring success from a successful assembly.
|
||||||
@@ -117,3 +121,6 @@ Residual: <what is NOT fixed, and the exact reason>
|
|||||||
- Quoting emulator results as device results.
|
- Quoting emulator results as device results.
|
||||||
- Reporting a fixed feature without checking adjacent features.
|
- Reporting a fixed feature without checking adjacent features.
|
||||||
- Hiding a known residual failure.
|
- Hiding a known residual failure.
|
||||||
|
- Blaming your own patch for a failure the **unmodified original** also has — run the control build first.
|
||||||
|
- Accepting a UI automation result as proof when the interaction may never have fired: verify field contents and look for the actual request, not just the screen.
|
||||||
|
- Treating a feature-scoped network error as a patching problem before checking the server's certificate (`references/tls-and-cert.md`).
|
||||||
|
|||||||
+314
-33
@@ -1,20 +1,82 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
"""向 app 的 DataStore(video_preferences) 注入 ad_free_expires_at 大值，验证它能否抑制广告 UI。
|
"""Encode or inject AndroidX DataStore (Preferences) protobuf entries.
|
||||||
|
|
||||||
Preferences 的 protobuf 结构：
|
WHY THIS EXISTS
|
||||||
PreferenceMap { map<string, Value> preferences = 1; }
|
---------------
|
||||||
entry: 0x0A <len(key)> <key> 0x12 <len(value)> <value>
|
Some feature gates are not decided in code but in a preferences file that the app
|
||||||
Value : boolean=1(float wire), float=2, integer=3(varint), long=4(varint), string=5, ...
|
reads at runtime, e.g. a "free until <timestamp>" value. Changing the code alone can
|
||||||
|
then be pointless, while writing the state directly proves the hypothesis in one
|
||||||
|
shot: if the behavior changes, the key really is the gate; if it does not, the gate
|
||||||
|
is elsewhere (usually server-side).
|
||||||
|
|
||||||
所以要写一个 longKey："ad_free_expires_at" -> Value{ long = 大值 }
|
WHAT THE FILE IS
|
||||||
Value 载荷 = 0x20 <varint(long)> (field 4, wire type 0 -> tag 0x20)
|
----------------
|
||||||
|
DataStore/Preferences serializes to a protobuf on disk. On a device it lives under
|
||||||
|
the app's private directory, typically:
|
||||||
|
<app_files_dir>/datastore/<name>.preferences_pb
|
||||||
|
and the container is:
|
||||||
|
message PreferenceMap { map<string, Value> preferences = 1; }
|
||||||
|
message Value { oneof { bool boolean = 1; float float = 2; int32 integer = 3;
|
||||||
|
int64 long = 4; string string = 5; double double = 6;
|
||||||
|
bytes bytes = 8; } }
|
||||||
|
|
||||||
|
The map field is the part people get wrong: every entry needs BOTH the outer
|
||||||
|
0x0A <len> (the map field itself) AND the inner 0x0A <len key> <key> +
|
||||||
|
0x12 <len value> <value>. Writing only the inner form produces a file that
|
||||||
|
deserializes into nothing, and some apps then die on the next read with an
|
||||||
|
exception that is swallowed by their crash reporter -- no stack, no clue.
|
||||||
|
|
||||||
|
Bytes writes here are the exact encoding, so this script doubles as a reference.
|
||||||
|
|
||||||
|
USAGE
|
||||||
|
-----
|
||||||
|
# write a fresh file with one big "never expires" timestamp
|
||||||
|
python datastore_inject.py --file example.preferences_pb \
|
||||||
|
--key example_expiry_epoch_ms --type long --value 4102444800000
|
||||||
|
|
||||||
|
# keep every other key from an existing file and replace/add just this one
|
||||||
|
python datastore_inject.py --in pulled.preferences_pb --file patched.preferences_pb \
|
||||||
|
--key example_expiry_epoch_ms --type long --value 4102444800000
|
||||||
|
|
||||||
|
# add more than one entry
|
||||||
|
python datastore_inject.py --file out.pb --key example_flag --type bool --value true \
|
||||||
|
--extra example_name=hello:string
|
||||||
|
|
||||||
|
# inspect a pulled file (what keys exist, and what they decode to)
|
||||||
|
python datastore_inject.py --in pulled.preferences_pb --list
|
||||||
|
|
||||||
|
Deploying the result: the app must not be running while you overwrite the file
|
||||||
|
(it will rewrite the file on exit), so push, fix ownership/selinux context if
|
||||||
|
needed, then start the app. Verify by reading the key back with --list.
|
||||||
|
|
||||||
|
Pure standard library, Python 3.9+.
|
||||||
"""
|
"""
|
||||||
|
import argparse
|
||||||
import struct
|
import struct
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
|
DEFAULT_FILE = 'example.preferences_pb'
|
||||||
|
DEFAULT_KEY = 'example_expiry_epoch_ms'
|
||||||
|
DEFAULT_VALUE = '4102444800000' # ~2100-01-01 in ms: "far future" in either unit
|
||||||
|
TYPES = ('long', 'int', 'bool', 'float', 'double', 'string', 'bytes')
|
||||||
|
|
||||||
|
# tag = (field_number << 3) | wire_type, per the Value oneof above.
|
||||||
|
VALUE_TAG = {
|
||||||
|
'bool': (1, 0),
|
||||||
|
'float': (2, 5),
|
||||||
|
'integer': (3, 0),
|
||||||
|
'long': (4, 0),
|
||||||
|
'string': (5, 2),
|
||||||
|
'double': (6, 1),
|
||||||
|
'bytes': (8, 2),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def varint(n):
|
def varint(n):
|
||||||
|
"""Unsigned LEB128. Negative int32/int64 are encoded as 64-bit twos complement."""
|
||||||
|
if n < 0:
|
||||||
|
n += 1 << 64
|
||||||
out = bytearray()
|
out = bytearray()
|
||||||
while True:
|
while True:
|
||||||
b = n & 0x7F
|
b = n & 0x7F
|
||||||
@@ -26,41 +88,260 @@ def varint(n):
|
|||||||
return bytes(out)
|
return bytes(out)
|
||||||
|
|
||||||
|
|
||||||
def map_entry(key, value_payload):
|
def read_varint(buf, pos):
|
||||||
"""编码 PreferenceMap 里的一个 map entry。
|
result = 0
|
||||||
|
shift = 0
|
||||||
|
while True:
|
||||||
|
if pos >= len(buf):
|
||||||
|
raise ValueError('truncated varint at offset %d' % pos)
|
||||||
|
b = buf[pos]
|
||||||
|
pos += 1
|
||||||
|
result |= (b & 0x7F) << shift
|
||||||
|
if not (b & 0x80):
|
||||||
|
return result, pos
|
||||||
|
shift += 7
|
||||||
|
if shift > 70:
|
||||||
|
raise ValueError('varint too long at offset %d' % pos)
|
||||||
|
|
||||||
★关键（踩过坑）：map<string,Value> preferences = 1 是一个 map 字段，
|
|
||||||
编码时必须**两层** 0x0A：
|
def tag_bytes(field, wire):
|
||||||
外层: 0x0A <len(entry)> <- 字段1的 tag + 长度
|
return varint((field << 3) | wire)
|
||||||
内层: 0x0A <len(key)> <key> 0x12 <len(value)> <value>
|
|
||||||
只写内层（丢掉外层 tag+长度）会让 DataStore 反序列化失败，
|
|
||||||
app 直接 uncaughtException 崩溃，而且没有堆栈（被 UCrash 吞掉），极难定位。
|
def encode_value(kind, raw):
|
||||||
|
"""Encode the Value payload (the bytes after the inner 0x12 <len>)."""
|
||||||
|
if kind == 'bool':
|
||||||
|
field, wire = VALUE_TAG['bool']
|
||||||
|
truthy = str(raw).strip().lower() in ('1', 'true', 'yes', 'on')
|
||||||
|
return tag_bytes(field, wire) + varint(1 if truthy else 0)
|
||||||
|
if kind in ('long', 'int'):
|
||||||
|
field, wire = VALUE_TAG['long' if kind == 'long' else 'integer']
|
||||||
|
return tag_bytes(field, wire) + varint(int(str(raw), 0))
|
||||||
|
if kind == 'float':
|
||||||
|
field, wire = VALUE_TAG['float']
|
||||||
|
return tag_bytes(field, wire) + struct.pack('<f', float(raw))
|
||||||
|
if kind == 'double':
|
||||||
|
field, wire = VALUE_TAG['double']
|
||||||
|
return tag_bytes(field, wire) + struct.pack('<d', float(raw))
|
||||||
|
if kind == 'string':
|
||||||
|
field, wire = VALUE_TAG['string']
|
||||||
|
data = str(raw).encode('utf-8')
|
||||||
|
return tag_bytes(field, wire) + varint(len(data)) + data
|
||||||
|
if kind == 'bytes':
|
||||||
|
field, wire = VALUE_TAG['bytes']
|
||||||
|
data = bytes.fromhex(str(raw))
|
||||||
|
return tag_bytes(field, wire) + varint(len(data)) + data
|
||||||
|
raise ValueError('unknown type: %s' % kind)
|
||||||
|
|
||||||
|
|
||||||
|
def encode_entry(key, value_payload):
|
||||||
|
"""Encode one PreferenceMap entry.
|
||||||
|
|
||||||
|
Two levels of 0x0A on purpose:
|
||||||
|
outer: 0x0A <len(inner)> <- the map field (field 1)
|
||||||
|
inner: 0x0A <len(key)> <key> 0x12 <len(value)> <value>
|
||||||
|
Dropping the outer tag+length is the classic mistake: DataStore then
|
||||||
|
deserializes an empty map, and the failure surfaces far away from here.
|
||||||
"""
|
"""
|
||||||
kb = key.encode('utf-8')
|
kb = key.encode('utf-8')
|
||||||
inner = (b'\x0a' + varint(len(kb)) + kb
|
inner = (tag_bytes(1, 2) + varint(len(kb)) + kb
|
||||||
+ b'\x12' + varint(len(value_payload)) + value_payload)
|
+ tag_bytes(2, 2) + varint(len(value_payload)) + value_payload)
|
||||||
return b'\x0a' + varint(len(inner)) + inner
|
return tag_bytes(1, 2) + varint(len(inner)) + inner
|
||||||
|
|
||||||
|
|
||||||
def entry(key, value_payload):
|
def split_entries(data):
|
||||||
"""兼容旧调用：返回 map_entry。"""
|
"""Split a PreferenceMap into raw top-level field slices."""
|
||||||
return map_entry(key, value_payload)
|
pos = 0
|
||||||
|
entries = []
|
||||||
|
while pos < len(data):
|
||||||
|
start = pos
|
||||||
|
tag, pos = read_varint(data, pos)
|
||||||
|
wire = tag & 0x07
|
||||||
|
if wire == 2:
|
||||||
|
length, pos = read_varint(data, pos)
|
||||||
|
pos += length
|
||||||
|
elif wire == 0:
|
||||||
|
_, pos = read_varint(data, pos)
|
||||||
|
elif wire == 5:
|
||||||
|
pos += 4
|
||||||
|
elif wire == 1:
|
||||||
|
pos += 8
|
||||||
|
else:
|
||||||
|
raise ValueError('unsupported wire type %d at offset %d' % (wire, start))
|
||||||
|
if pos > len(data):
|
||||||
|
raise ValueError('truncated field at offset %d' % start)
|
||||||
|
entries.append((tag, data[start:pos]))
|
||||||
|
return entries
|
||||||
|
|
||||||
|
|
||||||
|
def decode_entry(raw):
|
||||||
|
"""Return (key, value_payload) of one entry, or (None, None) if malformed."""
|
||||||
|
try:
|
||||||
|
pos = 0
|
||||||
|
tag, pos = read_varint(raw, pos)
|
||||||
|
if (tag >> 3) != 1 or (tag & 7) != 2:
|
||||||
|
return None, None
|
||||||
|
length, pos = read_varint(raw, pos)
|
||||||
|
inner = raw[pos:pos + length]
|
||||||
|
|
||||||
|
p = 0
|
||||||
|
t1, p = read_varint(inner, p)
|
||||||
|
if (t1 >> 3) != 1:
|
||||||
|
return None, None
|
||||||
|
klen, p = read_varint(inner, p)
|
||||||
|
key = inner[p:p + klen].decode('utf-8', 'replace')
|
||||||
|
p += klen
|
||||||
|
|
||||||
|
t2, p = read_varint(inner, p)
|
||||||
|
if (t2 >> 3) != 2:
|
||||||
|
return None, None
|
||||||
|
vlen, p = read_varint(inner, p)
|
||||||
|
return key, inner[p:p + vlen]
|
||||||
|
except Exception:
|
||||||
|
return None, None
|
||||||
|
|
||||||
|
|
||||||
|
def describe_value(payload):
|
||||||
|
"""Best-effort decode of a Value payload for the --list output."""
|
||||||
|
try:
|
||||||
|
pos = 0
|
||||||
|
tag, pos = read_varint(payload, pos)
|
||||||
|
field, wire = tag >> 3, tag & 7
|
||||||
|
if field == 1 and wire == 0:
|
||||||
|
val, _ = read_varint(payload, pos)
|
||||||
|
return 'bool=%s' % bool(val)
|
||||||
|
if field == 3 and wire == 0:
|
||||||
|
val, _ = read_varint(payload, pos)
|
||||||
|
return 'integer=%d' % val
|
||||||
|
if field == 4 and wire == 0:
|
||||||
|
val, _ = read_varint(payload, pos)
|
||||||
|
return 'long=%d' % val
|
||||||
|
if field == 2 and wire == 5:
|
||||||
|
return 'float=%r' % struct.unpack('<f', payload[pos:pos + 4])[0]
|
||||||
|
if field == 6 and wire == 1:
|
||||||
|
return 'double=%r' % struct.unpack('<d', payload[pos:pos + 8])[0]
|
||||||
|
if field == 5 and wire == 2:
|
||||||
|
length, pos = read_varint(payload, pos)
|
||||||
|
return 'string=%r' % payload[pos:pos + length].decode('utf-8', 'replace')
|
||||||
|
if field == 8 and wire == 2:
|
||||||
|
length, pos = read_varint(payload, pos)
|
||||||
|
return 'bytes=%s' % payload[pos:pos + length].hex()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return 'raw=%s' % payload.hex()
|
||||||
|
|
||||||
|
|
||||||
|
def parse_extra(items):
|
||||||
|
"""--extra key=value[:type] -> [(key, type, value)]"""
|
||||||
|
out = []
|
||||||
|
for item in items:
|
||||||
|
if '=' not in item:
|
||||||
|
raise SystemExit('--extra expects key=value[:type], got %r' % item)
|
||||||
|
key, rest = item.split('=', 1)
|
||||||
|
kind = 'string'
|
||||||
|
if ':' in rest:
|
||||||
|
rest, kind = rest.rsplit(':', 1)
|
||||||
|
if kind not in TYPES:
|
||||||
|
raise SystemExit('--extra %s: unknown type %r (choose from %s)'
|
||||||
|
% (key, kind, ', '.join(TYPES)))
|
||||||
|
out.append((key, kind, rest))
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
# 保留已有键：player_page_open_count = 1 (int32, field 3 -> tag 0x18)
|
ap = argparse.ArgumentParser(
|
||||||
existing = entry('player_page_open_count', b'\x18\x01')
|
description='Encode or inject AndroidX DataStore (Preferences) protobuf '
|
||||||
# 注入一个远超任何现实时间的时间戳（毫秒/秒都覆盖）
|
'entries, with the correct two-level map encoding.')
|
||||||
BIG = 9999999999999
|
ap.add_argument('--file', default=DEFAULT_FILE,
|
||||||
injected = entry('ad_free_expires_at', b'\x20' + varint(BIG))
|
help='output .pb path (default: %s)' % DEFAULT_FILE)
|
||||||
data = existing + injected
|
ap.add_argument('--key', default=DEFAULT_KEY,
|
||||||
out = sys.argv[1] if len(sys.argv) > 1 else 'video_preferences.preferences_pb'
|
help='preference key to write (default: %s)' % DEFAULT_KEY)
|
||||||
with open(out, 'wb') as f:
|
ap.add_argument('--value', default=DEFAULT_VALUE,
|
||||||
f.write(data)
|
help='value for --key (default: %s)' % DEFAULT_VALUE)
|
||||||
print('[ok] wrote %s (%d bytes)' % (out, len(data)))
|
ap.add_argument('--type', choices=TYPES, default='long',
|
||||||
|
help='value type (default: long)')
|
||||||
|
ap.add_argument('--in', dest='existing', default=None,
|
||||||
|
help='existing .pb to read: preserves all other keys')
|
||||||
|
ap.add_argument('--extra', action='append', default=[],
|
||||||
|
help='key=value[:type], repeatable, added alongside --key')
|
||||||
|
ap.add_argument('--list', action='store_true',
|
||||||
|
help='list the entries of --in and exit (no writing)')
|
||||||
|
ap.add_argument('--dry-run', action='store_true',
|
||||||
|
help='print what would be written, write nothing')
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
if args.list:
|
||||||
|
if not args.existing:
|
||||||
|
raise SystemExit('--list needs --in <file.preferences_pb>')
|
||||||
|
with open(args.existing, 'rb') as fh:
|
||||||
|
data = fh.read()
|
||||||
|
print('[in] %s (%d bytes)' % (args.existing, len(data)))
|
||||||
|
shown = 0
|
||||||
|
for tag, raw in split_entries(data):
|
||||||
|
key, payload = decode_entry(raw)
|
||||||
|
if key is None:
|
||||||
|
print(' <unparsed field tag=0x%02x, %d bytes>' % (tag, len(raw)))
|
||||||
|
continue
|
||||||
|
print(' %-32s %s (value=%d bytes)' % (key, describe_value(payload), len(payload)))
|
||||||
|
shown += 1
|
||||||
|
print('[ok] %d entry(ies)' % shown)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
# Build the entry list: preserved entries first, then the injected ones.
|
||||||
|
chunks = []
|
||||||
|
if args.existing:
|
||||||
|
with open(args.existing, 'rb') as fh:
|
||||||
|
original = fh.read()
|
||||||
|
replaced = {args.key}
|
||||||
|
replaced.update(k for k, _, _ in parse_extra(args.extra))
|
||||||
|
kept = 0
|
||||||
|
for _tag, raw in split_entries(original):
|
||||||
|
key, _payload = decode_entry(raw)
|
||||||
|
if key is None:
|
||||||
|
chunks.append(raw) # unknown field: keep it verbatim
|
||||||
|
continue
|
||||||
|
if key in replaced:
|
||||||
|
print('[keep-skip] %s is being replaced' % key)
|
||||||
|
continue
|
||||||
|
chunks.append(raw)
|
||||||
|
kept += 1
|
||||||
|
print('[in] preserved %d existing entry(ies) from %s' % (kept, args.existing))
|
||||||
|
|
||||||
|
target = encode_entry(args.key, encode_value(args.type, args.value))
|
||||||
|
chunks.append(target)
|
||||||
|
print('[inject] %s (%s) = %s -> %d bytes' % (args.key, args.type, args.value, len(target)))
|
||||||
|
|
||||||
|
for key, kind, value in parse_extra(args.extra):
|
||||||
|
entry = encode_entry(key, encode_value(kind, value))
|
||||||
|
chunks.append(entry)
|
||||||
|
print('[inject] %s (%s) = %s -> %d bytes' % (key, kind, value, len(entry)))
|
||||||
|
|
||||||
|
data = b''.join(chunks)
|
||||||
|
|
||||||
|
# Self-check: re-parse what we just built. A file that does not parse back is
|
||||||
|
# exactly the failure mode that kills the app with no usable stack.
|
||||||
|
found = {}
|
||||||
|
for _tag, raw in split_entries(data):
|
||||||
|
key, payload = decode_entry(raw)
|
||||||
|
if key is not None:
|
||||||
|
found[key] = describe_value(payload)
|
||||||
|
for key in [args.key] + [k for k, _t, _v in parse_extra(args.extra)]:
|
||||||
|
if key not in found:
|
||||||
|
raise SystemExit('internal error: %r does not parse back; refusing to write' % key)
|
||||||
|
print('[check] re-parsed %d entry(ies); %s = %s' % (len(found), args.key, found[args.key]))
|
||||||
print('[hex] %s' % data.hex(' '))
|
print('[hex] %s' % data.hex(' '))
|
||||||
print('[note] ad_free_expires_at = %d, varint=%s' % (BIG, varint(BIG).hex(' ')))
|
|
||||||
|
if args.dry_run:
|
||||||
|
print('[dry-run] nothing written')
|
||||||
|
return 0
|
||||||
|
|
||||||
|
with open(args.file, 'wb') as fh:
|
||||||
|
fh.write(data)
|
||||||
|
print('[ok] wrote %s (%d bytes)' % (args.file, len(data)))
|
||||||
|
print('[note] push it to <app_files_dir>/datastore/<name>.preferences_pb while the app '
|
||||||
|
'is NOT running, then start the app and read the key back with --list')
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
main()
|
sys.exit(main())
|
||||||
|
|||||||
@@ -22,33 +22,111 @@ import java.util.Collections;
|
|||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* classes7.dex 定点改动（单次读写，绝不要连续重写两次）。
|
* dexlib2 surgical rewrites -- worked example: force one lambda's emit() to hand a
|
||||||
|
* constant downstream instead of the value it read from its source.
|
||||||
*
|
*
|
||||||
* 只做一件事：把免广告到期时间的读取 lambda 改成恒返回"未过期"。
|
* WHAT IT PATCHES
|
||||||
|
* <target>$$inlined$map$1$2.emit(Object, Continuation)Object
|
||||||
|
* -> send a fixed wide (long) constant to the downstream collector, unconditionally
|
||||||
*
|
*
|
||||||
* VideoDataStore$currentAdFreeExpiresAt$$inlined$map$1$2.emit(Object; Continuation)Object
|
* WHY THIS SHAPE
|
||||||
* -> 无条件向下游 collector 发一个超大 Long
|
* Kotlin's `map { ... }` on a Flow compiles to a synthetic class whose emit()
|
||||||
|
* reads the source value from a field and forwards it to the collector. Replacing
|
||||||
|
* that single forwarding body turns "value read from the data source" into
|
||||||
|
* "constant", without touching the data source itself. Doing it in dex keeps the
|
||||||
|
* change inside the APK, so a fresh install behaves the same way -- writing the
|
||||||
|
* value into the app's runtime state instead would not survive a reinstall.
|
||||||
*
|
*
|
||||||
* 效果等价于 DataStore 里 ad_free_expires_at 永远没有过期，
|
* ONE READ, ONE WRITE. Never rewrite the same dex twice in the same run: dexlib2
|
||||||
* 「看广告领特权」弹窗因此不再出现，而且这个改动写死在代码里，
|
* round-trips degrade an R8-optimized dex, and the second pass is where it shows.
|
||||||
* 全新安装同样生效（DataStore 那份是运行时数据，随 APK 分发不了）。
|
|
||||||
*
|
*
|
||||||
* ⚠️ 不要顺手 patch Lx6;->d：那是通用的图片卡片 Composable，
|
* ADAPT TO YOUR TARGET
|
||||||
* 内容和广告共用，改成 return-void 会导致正常图片/播放链路一起失效。
|
* java PatchMethodExample <in.dex> <out.dex> [targetClass] [fieldOwner:fieldName:fieldType]
|
||||||
|
* [collectorIface] [continuationType]
|
||||||
|
* Every optional argument defaults to an obviously-fake example value below, and
|
||||||
|
* the effective values are printed at startup, so a run is never ambiguous.
|
||||||
|
*
|
||||||
|
* HOW TO FIND THE REAL VALUES
|
||||||
|
* * target class: search the dex strings for the source's field/data key, then read
|
||||||
|
* the call site with baksmali -- see scripts/dex_strings.py and scripts/find_refs.py.
|
||||||
|
* * collector iface / continuation type: read them off the emit() signature in smali.
|
||||||
|
* After R8 they are usually single letters; do not guess, copy them.
|
||||||
|
*
|
||||||
|
* DO NOT PATCH SHARED CODE BY ACCIDENT
|
||||||
|
* A generic card/image composable is commonly shared between ordinary content and
|
||||||
|
* ad slots. Turning such a method into `return-void` breaks normal rendering too.
|
||||||
|
* Before patching anything, count its callers (scripts/find_refs.py).
|
||||||
*/
|
*/
|
||||||
public class PatchClasses7v2 {
|
public class PatchMethodExample {
|
||||||
|
|
||||||
private static final String AD_CLASS =
|
/** Synthetic class that owns the lambda: <Owner>$<member>$$inlined$map$1$2. */
|
||||||
"Lcyc/data/datastore/VideoDataStore$currentAdFreeExpiresAt$$inlined$map$1$2;";
|
private static String TARGET_CLASS =
|
||||||
private static final long BIG = 4102444800000L; // ~2100-01-01 (ms)
|
"Lcom/example/app/data/ExampleStore$currentValue$$inlined$map$1$2;";
|
||||||
|
|
||||||
|
/** The field on the lambda that holds the downstream collector. */
|
||||||
|
private static String FIELD_OWNER = "Lcom/example/app/data/ExampleStore$currentValue$$inlined$map$1$2;";
|
||||||
|
private static String FIELD_NAME = "b";
|
||||||
|
private static String FIELD_TYPE = "Lcom/example/app/Collector;";
|
||||||
|
|
||||||
|
/** The collector type and the Continuation type used by emit(). */
|
||||||
|
private static String COLLECTOR_IFACE = "Lcom/example/app/Collector;";
|
||||||
|
private static String CONTINUATION_TYPE = "Lkotlin/coroutines/Continuation;";
|
||||||
|
|
||||||
|
private static final long CONSTANT = 4102444800000L; // ~2100-01-01 (ms)
|
||||||
|
|
||||||
|
private static void usage() {
|
||||||
|
System.out.println("usage: PatchMethodExample <in.dex> <out.dex> "
|
||||||
|
+ "[targetClass] [fieldOwner:fieldName:fieldType] "
|
||||||
|
+ "[collectorIface] [continuationType]");
|
||||||
|
System.out.println(" defaults (example values, replace for your target):");
|
||||||
|
System.out.println(" targetClass " + TARGET_CLASS);
|
||||||
|
System.out.println(" fieldRef " + FIELD_OWNER + ":" + FIELD_NAME + ":" + FIELD_TYPE);
|
||||||
|
System.out.println(" collectorIface " + COLLECTOR_IFACE);
|
||||||
|
System.out.println(" continuationType " + CONTINUATION_TYPE);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void applyArgs(String[] args) {
|
||||||
|
if (args.length > 2 && !args[2].isEmpty()) {
|
||||||
|
TARGET_CLASS = args[2];
|
||||||
|
}
|
||||||
|
if (args.length > 3 && !args[3].isEmpty()) {
|
||||||
|
String[] parts = args[3].split(":");
|
||||||
|
if (parts.length != 3) {
|
||||||
|
throw new IllegalArgumentException(
|
||||||
|
"fieldRef must be owner:name:type, got " + args[3]);
|
||||||
|
}
|
||||||
|
FIELD_OWNER = parts[0];
|
||||||
|
FIELD_NAME = parts[1];
|
||||||
|
FIELD_TYPE = parts[2];
|
||||||
|
}
|
||||||
|
if (args.length > 4 && !args[4].isEmpty()) {
|
||||||
|
COLLECTOR_IFACE = args[4];
|
||||||
|
}
|
||||||
|
if (args.length > 5 && !args[5].isEmpty()) {
|
||||||
|
CONTINUATION_TYPE = args[5];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
public static void main(String[] args) throws Exception {
|
public static void main(String[] args) throws Exception {
|
||||||
|
if (args.length < 2) {
|
||||||
|
usage();
|
||||||
|
System.exit(2);
|
||||||
|
}
|
||||||
|
applyArgs(args);
|
||||||
|
|
||||||
String in = args[0], out = args[1];
|
String in = args[0], out = args[1];
|
||||||
|
System.out.println("[in ] " + in);
|
||||||
|
System.out.println("[cfg] targetClass=" + TARGET_CLASS);
|
||||||
|
System.out.println("[cfg] fieldRef=" + FIELD_OWNER + ":" + FIELD_NAME + ":" + FIELD_TYPE);
|
||||||
|
System.out.println("[cfg] collectorIface=" + COLLECTOR_IFACE);
|
||||||
|
System.out.println("[cfg] continuationType=" + CONTINUATION_TYPE);
|
||||||
|
System.out.println("[cfg] constant=" + CONSTANT);
|
||||||
|
|
||||||
org.jf.dexlib2.iface.DexFile dex =
|
org.jf.dexlib2.iface.DexFile dex =
|
||||||
DexFileFactory.loadDexFile(new File(in), Opcodes.forApi(34));
|
DexFileFactory.loadDexFile(new File(in), Opcodes.forApi(34));
|
||||||
|
|
||||||
List<ClassDef> outClasses = new ArrayList<ClassDef>();
|
List<ClassDef> outClasses = new ArrayList<ClassDef>();
|
||||||
int p = 0;
|
int patched = 0;
|
||||||
|
|
||||||
for (ClassDef cd : dex.getClasses()) {
|
for (ClassDef cd : dex.getClasses()) {
|
||||||
List<Method> direct = new ArrayList<Method>();
|
List<Method> direct = new ArrayList<Method>();
|
||||||
@@ -57,7 +135,7 @@ public class PatchClasses7v2 {
|
|||||||
for (Method m : cd.getVirtualMethods()) virtual.add(m);
|
for (Method m : cd.getVirtualMethods()) virtual.add(m);
|
||||||
boolean touched = false;
|
boolean touched = false;
|
||||||
|
|
||||||
if (cd.getType().equals(AD_CLASS)) {
|
if (cd.getType().equals(TARGET_CLASS)) {
|
||||||
for (int pass = 0; pass < 2; pass++) {
|
for (int pass = 0; pass < 2; pass++) {
|
||||||
List<Method> list = (pass == 0) ? direct : virtual;
|
List<Method> list = (pass == 0) ? direct : virtual;
|
||||||
for (int i = 0; i < list.size(); i++) {
|
for (int i = 0; i < list.size(); i++) {
|
||||||
@@ -67,18 +145,23 @@ public class PatchClasses7v2 {
|
|||||||
|| m.getImplementation() == null) {
|
|| m.getImplementation() == null) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
// Register layout for emit(Object, Continuation):
|
||||||
|
// v0 scratch (return value)
|
||||||
|
// v1:v2 the wide constant we build
|
||||||
|
// v3 = p0 this (the lambda)
|
||||||
|
// v4 = p1 the value being forwarded (ignored)
|
||||||
|
// v5 = p2 the downstream Continuation
|
||||||
List<Instruction> body = new ArrayList<Instruction>();
|
List<Instruction> body = new ArrayList<Instruction>();
|
||||||
// v0 = collector, v1:v2 = long, p0=3 p1=4 p2=5
|
|
||||||
body.add(new ImmutableInstruction22c(Opcode.IGET_OBJECT, 0, 3,
|
body.add(new ImmutableInstruction22c(Opcode.IGET_OBJECT, 0, 3,
|
||||||
new ImmutableFieldReference(AD_CLASS, "b", "Lkw2;")));
|
new ImmutableFieldReference(FIELD_OWNER, FIELD_NAME, FIELD_TYPE)));
|
||||||
body.add(new ImmutableInstruction51l(Opcode.CONST_WIDE, 1, BIG));
|
body.add(new ImmutableInstruction51l(Opcode.CONST_WIDE, 1, CONSTANT));
|
||||||
body.add(new ImmutableInstruction35c(Opcode.INVOKE_STATIC, 2, 1, 2, 0, 0, 0,
|
body.add(new ImmutableInstruction35c(Opcode.INVOKE_STATIC, 2, 1, 2, 0, 0, 0,
|
||||||
new ImmutableMethodReference("Ljava/lang/Long;", "valueOf",
|
new ImmutableMethodReference("Ljava/lang/Long;", "valueOf",
|
||||||
Collections.singletonList("J"), "Ljava/lang/Long;")));
|
Collections.singletonList("J"), "Ljava/lang/Long;")));
|
||||||
body.add(new ImmutableInstruction11x(Opcode.MOVE_RESULT_OBJECT, 1));
|
body.add(new ImmutableInstruction11x(Opcode.MOVE_RESULT_OBJECT, 1));
|
||||||
body.add(new ImmutableInstruction35c(Opcode.INVOKE_INTERFACE, 3, 0, 1, 5, 0, 0,
|
body.add(new ImmutableInstruction35c(Opcode.INVOKE_INTERFACE, 3, 0, 1, 5, 0, 0,
|
||||||
new ImmutableMethodReference("Lkw2;", "emit",
|
new ImmutableMethodReference(COLLECTOR_IFACE, "emit",
|
||||||
Arrays.asList("Ljava/lang/Object;", "Lwj1;"),
|
Arrays.asList("Ljava/lang/Object;", CONTINUATION_TYPE),
|
||||||
"Ljava/lang/Object;")));
|
"Ljava/lang/Object;")));
|
||||||
body.add(new ImmutableInstruction11x(Opcode.MOVE_RESULT_OBJECT, 0));
|
body.add(new ImmutableInstruction11x(Opcode.MOVE_RESULT_OBJECT, 0));
|
||||||
body.add(new ImmutableInstruction11x(Opcode.RETURN_OBJECT, 0));
|
body.add(new ImmutableInstruction11x(Opcode.RETURN_OBJECT, 0));
|
||||||
@@ -87,22 +170,33 @@ public class PatchClasses7v2 {
|
|||||||
m.getParameters(), m.getReturnType(), m.getAccessFlags(),
|
m.getParameters(), m.getReturnType(), m.getAccessFlags(),
|
||||||
m.getAnnotations(), m.getHiddenApiRestrictions(),
|
m.getAnnotations(), m.getHiddenApiRestrictions(),
|
||||||
new ImmutableMethodImplementation(6, body, null, null)));
|
new ImmutableMethodImplementation(6, body, null, null)));
|
||||||
p++;
|
patched++;
|
||||||
touched = true;
|
touched = true;
|
||||||
System.out.println("[patch] emit -> BIG=" + BIG + " (list=" + pass + ")");
|
System.out.println("[patch] emit -> constant=" + CONSTANT
|
||||||
|
+ " (list=" + pass + ")");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!touched) outClasses.add(cd);
|
if (!touched) {
|
||||||
else outClasses.add(new ImmutableClassDef(
|
outClasses.add(cd);
|
||||||
cd.getType(), cd.getAccessFlags(), cd.getSuperclass(), cd.getInterfaces(),
|
} else {
|
||||||
cd.getSourceFile(), cd.getAnnotations(), cd.getStaticFields(),
|
outClasses.add(new ImmutableClassDef(
|
||||||
cd.getInstanceFields(), direct, virtual));
|
cd.getType(), cd.getAccessFlags(), cd.getSuperclass(), cd.getInterfaces(),
|
||||||
|
cd.getSourceFile(), cd.getAnnotations(), cd.getStaticFields(),
|
||||||
|
cd.getInstanceFields(), direct, virtual));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
DexFileFactory.writeDexFile(out, new ImmutableDexFile(Opcodes.forApi(34), outClasses));
|
DexFileFactory.writeDexFile(out, new ImmutableDexFile(Opcodes.forApi(34), outClasses));
|
||||||
System.out.println("[done] patched=" + p);
|
System.out.println("[out ] " + out + " patched=" + patched);
|
||||||
if (p == 0) System.exit(1);
|
if (patched == 0) {
|
||||||
|
// The usual causes: wrong target class (R8 renamed it), the method is not
|
||||||
|
// emit(Object,Continuation), or the class is not in THIS dex.
|
||||||
|
System.err.println("[fail] target not found in this dex. Check: the exact "
|
||||||
|
+ "TARGET_CLASS descriptor, that emit(Object,Continuation) exists there, "
|
||||||
|
+ "and that you loaded the dex that actually contains this class.");
|
||||||
|
System.exit(1);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,572 @@
|
|||||||
|
/*
|
||||||
|
* frida_probe.js -- four-layer full-chain network probe for a hardened Android app.
|
||||||
|
*
|
||||||
|
* WHY THIS SHAPE
|
||||||
|
* --------------
|
||||||
|
* When you cannot tell why a request never reaches the server (or why it fails
|
||||||
|
* silently), a single-layer hook always leaves you guessing. This script hooks
|
||||||
|
* FOUR independent layers at once, so *whichever* layer the app actually uses
|
||||||
|
* produces evidence:
|
||||||
|
*
|
||||||
|
* [APP] the app's own network wrapper class(es) -> what business method asked
|
||||||
|
* [OKHTTP] OkHttp full chain -> newCall/build/execute/async/proceed
|
||||||
|
* [RAW-URL] java.net.URL.openConnection -> HttpsURLConnection, which uses the
|
||||||
|
* SYSTEM trust store and therefore
|
||||||
|
* fails differently from OkHttp
|
||||||
|
* [THROW] java.lang.Throwable.getMessage -> recovers the exception text that an
|
||||||
|
* upper layer swallowed with catch {}
|
||||||
|
*
|
||||||
|
* The RAW-URL vs OKHTTP distinction is the single most valuable signal here: an app
|
||||||
|
* commonly has two unrelated trust chains, so "login fails but browsing works" is
|
||||||
|
* explained by which of the two fired. See LESSONS G3.
|
||||||
|
*
|
||||||
|
* Every hook is installed inside its own try/catch. One missing class (a different
|
||||||
|
* OkHttp major version, a renamed obfuscated class) never disables the other three
|
||||||
|
* layers. Failures are reported as HOOK-FAIL lines instead of killing the script.
|
||||||
|
*
|
||||||
|
* ADAPT TO YOUR TARGET (only this block matters)
|
||||||
|
* ---------------------------------------------
|
||||||
|
* 1) APP_NET_CLASSES -> replace with the app's own network wrapper class name(s)
|
||||||
|
* 2) APP_PKG_PREFIX -> the app's package prefix, used only to filter noise
|
||||||
|
* 3) ENABLE_* flags -> turn layers off if they are too noisy
|
||||||
|
* Everything else is target-independent.
|
||||||
|
*
|
||||||
|
* HOW TO RUN
|
||||||
|
* ----------
|
||||||
|
* Prefer the bundled injector, which also writes a log file and stays resident:
|
||||||
|
* python run_probe.py frida_probe.js 10 --pkg com.example.app
|
||||||
|
* Or by hand against a remote device:
|
||||||
|
* frida -H 127.0.0.1:27042 -p <pid> -l frida_probe.js
|
||||||
|
*
|
||||||
|
* OUTPUT FORMAT
|
||||||
|
* -------------
|
||||||
|
* Every line is <ISO-ish local timestamp> <TAG> <key=value ...>
|
||||||
|
* TAGS: READY | HOOK-FAIL | FATAL | APP | OKHTTP-NEWCALL | OKHTTP-BUILD |
|
||||||
|
* OKHTTP-EXEC | OKHTTP-ASYNC | OKHTTP-PROCEED | RAW-URL | DNS | THROW
|
||||||
|
* Add STACK to a line only where a call chain is genuinely useful (APP, RAW-URL),
|
||||||
|
* because it is the expensive part.
|
||||||
|
*/
|
||||||
|
|
||||||
|
/* ===================================================================
|
||||||
|
* 1. ADAPT THESE LINES
|
||||||
|
* =================================================================== */
|
||||||
|
// The app's own network wrapper / API helper class(es). Enumerate ALL declared
|
||||||
|
// methods at runtime, so you do not have to know the obfuscated method names.
|
||||||
|
// Example: 'com.example.app.net.HttpHelper', 'com.example.app.api.ApiClient'
|
||||||
|
var APP_NET_CLASSES = ['com.example.app.net.HttpHelper'];
|
||||||
|
|
||||||
|
// Used only to filter THROW noise: messages from classes whose name contains this
|
||||||
|
// prefix are always kept. Leave as-is if you do not care.
|
||||||
|
var APP_PKG_PREFIX = 'com.example.app';
|
||||||
|
|
||||||
|
// Layer switches. Turn a layer off if it produces more noise than signal.
|
||||||
|
var ENABLE_APP = true; // [APP] app's own wrapper classes (needs APP_NET_CLASSES)
|
||||||
|
var ENABLE_OKHTTP = true; // [OKHTTP] OkHttp newCall/build/execute/AsyncCall/proceed
|
||||||
|
var ENABLE_RAW_URL = true; // [RAW-URL] java.net.URL.openConnection (HttpsURLConnection)
|
||||||
|
var ENABLE_THROWABLE = true; // [THROW] Throwable.getMessage, deduplicated
|
||||||
|
var ENABLE_DNS = true; // [DNS] InetAddress.getAllByName, one line per unique host
|
||||||
|
var STACK_ON_APP = true; // attach a stack to [APP] hits
|
||||||
|
var STACK_ON_RAW_URL = true; // attach a stack to [RAW-URL] hits
|
||||||
|
var STACK_LINES = 12; // how many stack frames to keep
|
||||||
|
|
||||||
|
// Safety valves: this probe must never become the thing that kills the app.
|
||||||
|
var MAX_STRING = 3000; // truncate any single value
|
||||||
|
var THROWABLE_MAX_LINES = 400;// hard cap on THROW lines
|
||||||
|
var THROWABLE_STACK = false; // stacks on every Throwable line are usually too much
|
||||||
|
|
||||||
|
/* ===================================================================
|
||||||
|
* 2. Plumbing (target-independent)
|
||||||
|
* =================================================================== */
|
||||||
|
|
||||||
|
function pad(n, w) {
|
||||||
|
var s = '' + n;
|
||||||
|
while (s.length < w) s = '0' + s;
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Local wall-clock with milliseconds. Local, not UTC, because you will be
|
||||||
|
// comparing these lines against logcat and your own shell history.
|
||||||
|
function stamp() {
|
||||||
|
var d = new Date();
|
||||||
|
return d.getFullYear() + '-' + pad(d.getMonth() + 1, 2) + '-' + pad(d.getDate(), 2) + ' ' +
|
||||||
|
pad(d.getHours(), 2) + ':' + pad(d.getMinutes(), 2) + ':' + pad(d.getSeconds(), 2) + '.' +
|
||||||
|
pad(d.getMilliseconds(), 3);
|
||||||
|
}
|
||||||
|
|
||||||
|
function str(v, max) {
|
||||||
|
if (v === null) return 'null';
|
||||||
|
if (v === undefined) return 'undefined';
|
||||||
|
var s;
|
||||||
|
try { s = '' + v; } catch (e) { s = '<unprintable>'; }
|
||||||
|
max = max || MAX_STRING;
|
||||||
|
if (s.length > max) s = s.substring(0, max) + '...<+' + (s.length - max) + ' chars>';
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sendLine(tag, fields) {
|
||||||
|
var parts = [];
|
||||||
|
if (fields) {
|
||||||
|
for (var k in fields) {
|
||||||
|
if (!fields.hasOwnProperty(k)) continue;
|
||||||
|
if (fields[k] === null || fields[k] === undefined || fields[k] === '') continue;
|
||||||
|
parts.push(k + '=' + str(fields[k]));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var line = { t: stamp(), tag: tag, msg: parts.join(' ') };
|
||||||
|
try { send(line); } catch (e) { /* transport gone; nothing sane to do */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
// Call-chain capture. Expensive, so it is opt-in per hook.
|
||||||
|
function stackText(lines) {
|
||||||
|
try {
|
||||||
|
var Log = Java.use('android.util.Log');
|
||||||
|
var Throwable = Java.use('java.lang.Throwable');
|
||||||
|
var full = Log.getStackTraceString(Throwable.$new());
|
||||||
|
var rows = ('' + full).split('\n');
|
||||||
|
var kept = [];
|
||||||
|
// Drop the probe's own frames at the top; they are never the interesting part.
|
||||||
|
for (var i = 0; i < rows.length && kept.length < (lines || STACK_LINES); i++) {
|
||||||
|
if (rows[i].indexOf('frida') >= 0) continue;
|
||||||
|
if (rows[i].indexOf('java.lang.Throwable') >= 0) continue;
|
||||||
|
if (rows[i].indexOf('android.util.Log') >= 0) continue;
|
||||||
|
kept.push(rows[i].replace(/^\s+/, ''));
|
||||||
|
}
|
||||||
|
return kept.join(' <- ');
|
||||||
|
} catch (e) {
|
||||||
|
return '<stack unavailable: ' + e + '>';
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// The whole point: a failure to install one hook must not affect the others.
|
||||||
|
function guard(label, fn) {
|
||||||
|
try {
|
||||||
|
fn();
|
||||||
|
return true;
|
||||||
|
} catch (e) {
|
||||||
|
sendLine('HOOK-FAIL', { where: label, err: e });
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run one whole layer and return how many hooks it really installed.
|
||||||
|
// > 0 that many hooks are live
|
||||||
|
// = 0 the layer installed nothing (its classes were not found); NOT a success
|
||||||
|
// =-1 the layer is switched off in the config header
|
||||||
|
// Returning a count instead of a boolean matters: a layer whose classes are absent
|
||||||
|
// still returns normally (each inner hook catches its own error), so a boolean
|
||||||
|
// "did not throw" would report a layer as installed when it hooked nothing at all.
|
||||||
|
function runLayer(label, fn) {
|
||||||
|
try {
|
||||||
|
return fn();
|
||||||
|
} catch (e) {
|
||||||
|
sendLine('HOOK-FAIL', { where: 'install ' + label, err: e });
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ===================================================================
|
||||||
|
* 3. Layer [APP]: the app's own network wrapper
|
||||||
|
* =================================================================== */
|
||||||
|
// Why getDeclaredMethods: obfuscated wrappers routinely expose the same name with
|
||||||
|
// several overloads, and a decompiler's short names (a, b, c) are not reliable
|
||||||
|
// enough to enumerate by hand. Enumerate at runtime, wrap each overload, and let
|
||||||
|
// the stack tell you which one the business code called.
|
||||||
|
// Build a replacement of a fixed arity that reports the hit and then delegates.
|
||||||
|
//
|
||||||
|
// Two deliberate choices here, both about not breaking the app:
|
||||||
|
// * The original is NOT cached in a variable. Frida does not hand out the pre-hook
|
||||||
|
// implementation through `overload.implementation` (it is undefined until you
|
||||||
|
// replace it), so `var orig = ov.implementation; ... orig.apply(...)` would
|
||||||
|
// throw a TypeError on the first real call and propagate straight into the app's
|
||||||
|
// network path. Calling the same name on `this` from inside the replacement is
|
||||||
|
// the documented way to reach the original, and it re-resolves the overload from
|
||||||
|
// the runtime argument types.
|
||||||
|
// * The delegate call is written out per arity instead of spread/apply, so no
|
||||||
|
// engine-specific behaviour is involved. Methods with more than 8 parameters are
|
||||||
|
// reported and left alone rather than bound incorrectly.
|
||||||
|
function appReplacement(className, methodName, arity) {
|
||||||
|
var label = className + '.' + methodName;
|
||||||
|
var note = function (argc) {
|
||||||
|
sendLine('APP', {
|
||||||
|
call: label,
|
||||||
|
argc: argc,
|
||||||
|
stack: STACK_ON_APP ? stackText() : null
|
||||||
|
});
|
||||||
|
};
|
||||||
|
switch (arity) {
|
||||||
|
case 0: return function () {
|
||||||
|
note(arguments.length); return this[methodName]();
|
||||||
|
};
|
||||||
|
case 1: return function (a0) {
|
||||||
|
note(arguments.length); return this[methodName](a0);
|
||||||
|
};
|
||||||
|
case 2: return function (a0, a1) {
|
||||||
|
note(arguments.length); return this[methodName](a0, a1);
|
||||||
|
};
|
||||||
|
case 3: return function (a0, a1, a2) {
|
||||||
|
note(arguments.length); return this[methodName](a0, a1, a2);
|
||||||
|
};
|
||||||
|
case 4: return function (a0, a1, a2, a3) {
|
||||||
|
note(arguments.length); return this[methodName](a0, a1, a2, a3);
|
||||||
|
};
|
||||||
|
case 5: return function (a0, a1, a2, a3, a4) {
|
||||||
|
note(arguments.length); return this[methodName](a0, a1, a2, a3, a4);
|
||||||
|
};
|
||||||
|
case 6: return function (a0, a1, a2, a3, a4, a5) {
|
||||||
|
note(arguments.length); return this[methodName](a0, a1, a2, a3, a4, a5);
|
||||||
|
};
|
||||||
|
case 7: return function (a0, a1, a2, a3, a4, a5, a6) {
|
||||||
|
note(arguments.length); return this[methodName](a0, a1, a2, a3, a4, a5, a6);
|
||||||
|
};
|
||||||
|
case 8: return function (a0, a1, a2, a3, a4, a5, a6, a7) {
|
||||||
|
note(arguments.length); return this[methodName](a0, a1, a2, a3, a4, a5, a6, a7);
|
||||||
|
};
|
||||||
|
default: return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function installAppLayer() {
|
||||||
|
if (!ENABLE_APP) return -1;
|
||||||
|
if (!APP_NET_CLASSES.length) {
|
||||||
|
sendLine('HOOK-FAIL', { where: 'APP', err: 'APP_NET_CLASSES is empty; edit the header' });
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
var wrappedTotal = 0;
|
||||||
|
APP_NET_CLASSES.forEach(function (className) {
|
||||||
|
guard('APP ' + className, function () {
|
||||||
|
var Cls = Java.use(className);
|
||||||
|
var declared = Cls.class.getDeclaredMethods();
|
||||||
|
var total = declared.length;
|
||||||
|
var wrapped = 0;
|
||||||
|
var skipped = 0;
|
||||||
|
for (var i = 0; i < total; i++) {
|
||||||
|
(function (idx) {
|
||||||
|
var methodName;
|
||||||
|
try { methodName = '' + declared[idx].getName(); } catch (e) { return; }
|
||||||
|
guard('APP ' + className + '.' + methodName, function () {
|
||||||
|
var overloads = Cls[methodName];
|
||||||
|
if (!overloads || !overloads.overloads) return;
|
||||||
|
overloads.overloads.forEach(function (ov) {
|
||||||
|
var argTypes;
|
||||||
|
try { argTypes = ov.argumentTypes; } catch (e) { argTypes = null; }
|
||||||
|
var arity = (argTypes && typeof argTypes.length === 'number') ? argTypes.length : -1;
|
||||||
|
var label = className + '.' + methodName + '(' + (argTypes ? argTypes.join(',') : '?') + ')';
|
||||||
|
guard('APP bind ' + label, function () {
|
||||||
|
var replacement = appReplacement(className, methodName, arity);
|
||||||
|
if (replacement === null) {
|
||||||
|
skipped++;
|
||||||
|
sendLine('APP', {
|
||||||
|
call: label,
|
||||||
|
skipped: 'arity ' + arity + ' exceeds 8; left unwrapped on purpose'
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
ov.implementation = replacement;
|
||||||
|
wrapped++;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
})(i);
|
||||||
|
}
|
||||||
|
wrappedTotal += wrapped;
|
||||||
|
sendLine('APP', {
|
||||||
|
call: className,
|
||||||
|
declared_methods: total,
|
||||||
|
overloads_wrapped: wrapped,
|
||||||
|
overloads_skipped: skipped
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
|
return wrappedTotal;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ===================================================================
|
||||||
|
* 4. Layer [OKHTTP]: the whole OkHttp chain
|
||||||
|
* =================================================================== */
|
||||||
|
// Five independent anchors on purpose. Any single one can be missing depending on
|
||||||
|
// the OkHttp major version and on R8 inlining, and each still answers a different
|
||||||
|
// question:
|
||||||
|
// newCall -> a request object reached OkHttp (earliest reliable URL)
|
||||||
|
// build -> the URL existed even if the call was never executed
|
||||||
|
// execute -> synchronous call actually ran, with status code and duration
|
||||||
|
// AsyncCall.run-> asynchronous call actually ran (enqueue path)
|
||||||
|
// proceed -> every actual network step, including redirects and retries
|
||||||
|
function installOkHttpLayer() {
|
||||||
|
if (!ENABLE_OKHTTP) return -1;
|
||||||
|
|
||||||
|
var installed = 0;
|
||||||
|
var can = function (label, fn) { if (guard(label, fn)) installed++; };
|
||||||
|
|
||||||
|
can('OKHTTP newCall', function () {
|
||||||
|
var Client = Java.use('okhttp3.OkHttpClient');
|
||||||
|
Client.newCall.overload('okhttp3.Request').implementation = function (req) {
|
||||||
|
var url = '<unknown>';
|
||||||
|
try { url = '' + req.url().toString(); } catch (e) { url = '<url error: ' + e + '>'; }
|
||||||
|
sendLine('OKHTTP-NEWCALL', { url: url });
|
||||||
|
return this.newCall(req);
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
can('OKHTTP Request$Builder.build', function () {
|
||||||
|
var Builder = Java.use('okhttp3.Request$Builder');
|
||||||
|
Builder.build.implementation = function () {
|
||||||
|
var req = this.build();
|
||||||
|
var url = '<unknown>';
|
||||||
|
try { url = '' + req.url().toString(); } catch (e) { url = '<url error: ' + e + '>'; }
|
||||||
|
sendLine('OKHTTP-BUILD', { url: url });
|
||||||
|
return req;
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
can('OKHTTP RealCall.execute', function () {
|
||||||
|
var RealCall = Java.use('okhttp3.internal.connection.RealCall');
|
||||||
|
RealCall.execute.implementation = function () {
|
||||||
|
var url = '<unknown>';
|
||||||
|
try { url = '' + this.request().url().toString(); } catch (e) { url = '<url error: ' + e + '>'; }
|
||||||
|
var t0 = Date.now();
|
||||||
|
var resp = this.execute();
|
||||||
|
var code = '?';
|
||||||
|
try { code = '' + resp.code(); } catch (e) { code = '<code error>'; }
|
||||||
|
sendLine('OKHTTP-EXEC', { url: url, ms: Date.now() - t0, code: code });
|
||||||
|
return resp;
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
can('OKHTTP RealCall$AsyncCall.run', function () {
|
||||||
|
var AsyncCall = Java.use('okhttp3.internal.connection.RealCall$AsyncCall');
|
||||||
|
AsyncCall.run.implementation = function () {
|
||||||
|
// The URL is not reliably reachable from here across OkHttp versions; the
|
||||||
|
// newCall/build hooks above supply it. This hook proves *when* the async
|
||||||
|
// path actually started and finished, which is what matters for timing.
|
||||||
|
var t0 = Date.now();
|
||||||
|
this.run();
|
||||||
|
sendLine('OKHTTP-ASYNC', { phase: 'done', ms: Date.now() - t0 });
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
can('OKHTTP RealInterceptorChain.proceed', function () {
|
||||||
|
var Chain = Java.use('okhttp3.internal.http.RealInterceptorChain');
|
||||||
|
Chain.proceed.overload('okhttp3.Request').implementation = function (req) {
|
||||||
|
var url = '<unknown>';
|
||||||
|
try { url = '' + req.url().toString(); } catch (e) { url = '<url error: ' + e + '>'; }
|
||||||
|
var t0 = Date.now();
|
||||||
|
var resp = this.proceed(req);
|
||||||
|
var code = '?';
|
||||||
|
try { code = '' + resp.code(); } catch (e) { code = '<code error>'; }
|
||||||
|
sendLine('OKHTTP-PROCEED', { url: url, ms: Date.now() - t0, code: code });
|
||||||
|
return resp;
|
||||||
|
};
|
||||||
|
});
|
||||||
|
return installed;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ===================================================================
|
||||||
|
* 5. Layer [RAW-URL]: HttpsURLConnection -> the SYSTEM trust store
|
||||||
|
* =================================================================== */
|
||||||
|
// Many apps route login/registration through java.net.URL.openConnection instead
|
||||||
|
// of OkHttp. That path validates against the system trust store, so an expired or
|
||||||
|
// untrusted server certificate breaks it while the OkHttp path keeps working.
|
||||||
|
// If this layer fires but the OkHttp layer stays silent, you are on the other
|
||||||
|
// trust chain -- check the certificate out-of-band with tls_check.py.
|
||||||
|
function installRawUrlLayer() {
|
||||||
|
if (!ENABLE_RAW_URL) return -1;
|
||||||
|
|
||||||
|
var URL;
|
||||||
|
try {
|
||||||
|
URL = Java.use('java.net.URL');
|
||||||
|
} catch (e) {
|
||||||
|
sendLine('HOOK-FAIL', { where: 'RAW-URL java.net.URL', err: e });
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
var installed = 0;
|
||||||
|
var can = function (label, fn) { if (guard(label, fn)) installed++; };
|
||||||
|
|
||||||
|
can('RAW-URL openConnection()', function () {
|
||||||
|
// Report AFTER the original returns, so we can also name the concrete connection
|
||||||
|
// class (HttpURLConnection vs HttpsURLConnection) -- that distinction is what
|
||||||
|
// tells you which trust chain the request is about to use.
|
||||||
|
// As in the APP layer, the original is reached via `this.openConnection(...)`
|
||||||
|
// rather than a cached function reference.
|
||||||
|
URL.openConnection.overload().implementation = function () {
|
||||||
|
var url = '<unknown>';
|
||||||
|
try { url = '' + this.toString(); } catch (e) { url = '<url error: ' + e + '>'; }
|
||||||
|
var conn = this.openConnection();
|
||||||
|
var connCls = '?';
|
||||||
|
try { connCls = '' + conn.getClass().getName(); } catch (e) { connCls = '<class error>'; }
|
||||||
|
sendLine('RAW-URL', {
|
||||||
|
url: url,
|
||||||
|
kind: 'no-arg',
|
||||||
|
conn: connCls,
|
||||||
|
stack: STACK_ON_RAW_URL ? stackText() : null
|
||||||
|
});
|
||||||
|
return conn;
|
||||||
|
};
|
||||||
|
});
|
||||||
|
|
||||||
|
can('RAW-URL openConnection(Proxy)', function () {
|
||||||
|
URL.openConnection.overload('java.net.Proxy').implementation = function (proxy) {
|
||||||
|
var url = '<unknown>';
|
||||||
|
try { url = '' + this.toString(); } catch (e) { url = '<url error: ' + e + '>'; }
|
||||||
|
var conn = this.openConnection(proxy);
|
||||||
|
var connCls = '?';
|
||||||
|
try { connCls = '' + conn.getClass().getName(); } catch (e) { connCls = '<class error>'; }
|
||||||
|
sendLine('RAW-URL', {
|
||||||
|
url: url,
|
||||||
|
kind: 'proxy',
|
||||||
|
conn: connCls,
|
||||||
|
stack: STACK_ON_RAW_URL ? stackText() : null
|
||||||
|
});
|
||||||
|
return conn;
|
||||||
|
};
|
||||||
|
});
|
||||||
|
return installed;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ===================================================================
|
||||||
|
* 6. Layer [THROW]: recover swallowed exception text
|
||||||
|
* =================================================================== */
|
||||||
|
// An upper-layer `catch (Exception e) { }` hides exactly the message you need.
|
||||||
|
// Hooking Throwable.getMessage is the cheapest way to get it back, but it is a
|
||||||
|
// very hot method, so: only concrete Exception/Error subclasses, deduplicated by
|
||||||
|
// class+message, hard-capped, and stacks off by default.
|
||||||
|
function installThrowableLayer() {
|
||||||
|
if (!ENABLE_THROWABLE) return -1;
|
||||||
|
|
||||||
|
// Returns 1 only when the hook is really bound to the runtime.
|
||||||
|
return guard('THROWABLE getMessage', function () {
|
||||||
|
var seen = {};
|
||||||
|
var printed = 0;
|
||||||
|
var suppressed = 0;
|
||||||
|
var Throwable = Java.use('java.lang.Throwable');
|
||||||
|
|
||||||
|
Throwable.getMessage.implementation = function () {
|
||||||
|
var msg;
|
||||||
|
try {
|
||||||
|
msg = this.getMessage();
|
||||||
|
} catch (e) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
var cls = '' + this.getClass().getName();
|
||||||
|
var looksLikeError = (cls.indexOf('Exception') >= 0 || cls.indexOf('Error') >= 0);
|
||||||
|
var isAppClass = (cls.indexOf(APP_PKG_PREFIX) === 0);
|
||||||
|
if (looksLikeError || isAppClass) {
|
||||||
|
var key = cls + '|' + msg;
|
||||||
|
if (seen[key]) {
|
||||||
|
seen[key]++;
|
||||||
|
} else if (printed < THROWABLE_MAX_LINES) {
|
||||||
|
seen[key] = 1;
|
||||||
|
printed++;
|
||||||
|
sendLine('THROW', {
|
||||||
|
cls: cls,
|
||||||
|
msg: msg,
|
||||||
|
stack: THROWABLE_STACK ? stackText() : null
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
suppressed++;
|
||||||
|
if (suppressed === 1 || suppressed % 200 === 0) {
|
||||||
|
sendLine('THROW', { note: 'cap reached, further unique messages suppressed', printed: printed });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (e) {
|
||||||
|
// Never let the reporting path throw: this hook is on a very hot path.
|
||||||
|
}
|
||||||
|
return msg;
|
||||||
|
};
|
||||||
|
}) ? 1 : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ===================================================================
|
||||||
|
* 7. Optional layer [DNS]: which hosts the app really resolves
|
||||||
|
* =================================================================== */
|
||||||
|
// Cheap, high-signal: if an SDK's domains are never resolved, that subsystem never
|
||||||
|
// started -- much stronger evidence than "logcat was quiet". Deduplicated per host
|
||||||
|
// so a retry loop cannot flood the log.
|
||||||
|
function installDnsLayer() {
|
||||||
|
if (!ENABLE_DNS) return -1;
|
||||||
|
|
||||||
|
return guard('DNS getAllByName', function () {
|
||||||
|
var seen = {};
|
||||||
|
var Inet = Java.use('java.net.InetAddress');
|
||||||
|
Inet.getAllByName.overload('java.lang.String').implementation = function (host) {
|
||||||
|
var first = false;
|
||||||
|
try {
|
||||||
|
var key = '' + host;
|
||||||
|
if (!seen[key]) { seen[key] = 1; first = true; }
|
||||||
|
} catch (e) { /* ignore */ }
|
||||||
|
if (first) sendLine('DNS', { host: host });
|
||||||
|
return this.getAllByName(host);
|
||||||
|
};
|
||||||
|
}) ? 1 : 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ===================================================================
|
||||||
|
* 8. Boot
|
||||||
|
* =================================================================== */
|
||||||
|
var JAVA_BRIDGE_HINT =
|
||||||
|
'align the host frida package and the on-device frida-server to the same 16.x version; ' +
|
||||||
|
'17.x can fail to locate the Android dynamic linker and removes the built-in Java bridge';
|
||||||
|
|
||||||
|
function main() {
|
||||||
|
if (typeof Java === 'undefined' || Java === null) {
|
||||||
|
// frida 17+ removed the built-in Java bridge.
|
||||||
|
sendLine('FATAL', {
|
||||||
|
err: 'Java bridge is not available in this frida runtime',
|
||||||
|
fix: JAVA_BRIDGE_HINT
|
||||||
|
});
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
// Java.perform can fail BEFORE the callback ever runs: on a non-Android host, and
|
||||||
|
// on frida 17.x, where it throws "Java API not available". Without this catch the
|
||||||
|
// whole script dies as an opaque `type: error` message with a stack pointing into
|
||||||
|
// frida-java-bridge, which tells you nothing about the version mismatch that
|
||||||
|
// actually caused it.
|
||||||
|
try {
|
||||||
|
Java.perform(function () {
|
||||||
|
var installed = [];
|
||||||
|
var failed = [];
|
||||||
|
var disabled = [];
|
||||||
|
[
|
||||||
|
['APP', installAppLayer],
|
||||||
|
['OKHTTP', installOkHttpLayer],
|
||||||
|
['RAW-URL', installRawUrlLayer],
|
||||||
|
['THROW', installThrowableLayer],
|
||||||
|
['DNS', installDnsLayer]
|
||||||
|
].forEach(function (pair) {
|
||||||
|
var count = runLayer(pair[0], pair[1]);
|
||||||
|
if (count === -1) disabled.push(pair[0]);
|
||||||
|
else if (count > 0) installed.push(pair[0] + ':' + count);
|
||||||
|
else failed.push(pair[0]);
|
||||||
|
});
|
||||||
|
sendLine('READY', {
|
||||||
|
installed: installed.join(',') || 'none',
|
||||||
|
failed: failed.join(',') || 'none',
|
||||||
|
disabled: disabled.join(',') || 'none',
|
||||||
|
note: 'hooks are live; now drive the app UI'
|
||||||
|
});
|
||||||
|
if (failed.length) {
|
||||||
|
sendLine('READY-HINT', {
|
||||||
|
failed: failed.join(','),
|
||||||
|
fix: 'a layer reporting 0 hooks means its classes were not found: the app may '
|
||||||
|
+ 'not use that library, R8 may have renamed the class, or the classloader '
|
||||||
|
+ 'differs. Layers that installed keep working -- debug the failed ones by '
|
||||||
|
+ 'their HOOK-FAIL line, not the working ones.'
|
||||||
|
});
|
||||||
|
}
|
||||||
|
});
|
||||||
|
} catch (e) {
|
||||||
|
sendLine('FATAL', {
|
||||||
|
err: 'Java.perform failed: ' + e,
|
||||||
|
fix: JAVA_BRIDGE_HINT
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Last-resort net: any unexpected top-level error still leaves a readable line in
|
||||||
|
// the log instead of only a raw frida script error.
|
||||||
|
try {
|
||||||
|
main();
|
||||||
|
} catch (e) {
|
||||||
|
sendLine('FATAL', { err: 'probe aborted: ' + e, note: 'no hooks were installed' });
|
||||||
|
}
|
||||||
+298
-99
@@ -1,46 +1,47 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
# -*- coding: utf-8 -*-
|
# -*- coding: utf-8 -*-
|
||||||
"""Repack + resign pipeline for the ciyuan Android app.
|
"""Repack an APK with replaced dex files, drop ONLY the signature entries, and re-sign.
|
||||||
|
|
||||||
Pipeline: replace dex files -> strip META-INF/ -> zip (AndroidManifest.xml and
|
Pipeline: replace dex files -> drop signature artifacts -> zip, keeping
|
||||||
resources.arsc kept STORED) -> zipalign + sign via uber-apk-signer -> verify.
|
AndroidManifest.xml and resources.arsc STORED -> zipalign + sign -> verify.
|
||||||
|
|
||||||
Keystore defaults (generated on first use with keytool):
|
Everything machine-specific is a parameter or resolved from PATH. No JDK path is
|
||||||
file : <work>/out/cyc.keystore
|
baked in, no keystore password is baked in, no target app is assumed.
|
||||||
alias : cyc
|
|
||||||
storepass / keypass : cyc123456
|
Keystore handling:
|
||||||
|
--ks <path> is where the keystore lives. If it does not exist, one is generated
|
||||||
|
with keytool and its password is stored next to it as <ks>.pass.txt, so the next
|
||||||
|
run reuses it. Passwords are never hardcoded in this script. That file is a local
|
||||||
|
artifact -- do not commit it.
|
||||||
|
|
||||||
Usage examples:
|
Usage examples:
|
||||||
# roundtrip: reuse original dexes unchanged, proves the pipeline itself works
|
# roundtrip: reuse the original dexes unchanged, proves the pipeline itself works
|
||||||
python tools/repack.py --apk work/orig.apk --dexdir work/x_orig --out work/out/roundtrip.apk
|
python repack.py --apk work/orig.apk --dexdir work/x_orig --out work/out/roundtrip.apk
|
||||||
|
|
||||||
# real patch: swap a single dex (name=path), other dexes kept from the apk
|
# real patch: swap a single dex (name=path), other dexes kept from the apk
|
||||||
python tools/repack.py --apk work/orig.apk --dex classes8.dex=work/x_orig/patch/classes8.dex \
|
python repack.py --apk work/orig.apk --dex classes8.dex=work/patch/classes8.dex \
|
||||||
--out work/out/patched.apk
|
--out work/out/patched.apk
|
||||||
|
|
||||||
|
# no signing, no java tooling required at all
|
||||||
|
python repack.py --apk work/orig.apk --dexdir work/x_orig --out work/unsigned.apk --no-sign
|
||||||
|
|
||||||
Notes:
|
Notes:
|
||||||
* All java tooling is invoked through python subprocess (never through the
|
* All java tooling is invoked through python subprocess, never through a host
|
||||||
PowerShell argument passing path, which drops arguments).
|
shell (PowerShell drops arguments in ways that look like tool failures).
|
||||||
* uber-apk-signer silently skips already signed apks (reports 0 processed),
|
* uber-apk-signer silently skips already-signed apks (reports 0 processed), which
|
||||||
which is why META-INF/ is always stripped before signing.
|
is why signature entries are always dropped before signing.
|
||||||
|
* uber-apk-signer is used for SIGNING to keep the verified path unchanged;
|
||||||
|
apksigner is used for VERIFYING when present, with an explicit sdk range.
|
||||||
"""
|
"""
|
||||||
import argparse
|
import argparse
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
|
import secrets
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
import zipfile
|
import zipfile
|
||||||
|
|
||||||
JAVA = r'C:\Program Files\Java\jdk-17.0.4.1\bin\java.exe'
|
|
||||||
KEYTOOL = r'C:\Program Files\Java\jdk-17.0.4.1\bin\keytool.exe'
|
|
||||||
JARSIGNER = r'C:\Program Files\Java\jdk-17.0.4.1\bin\jarsigner.exe'
|
|
||||||
SIGNER_JAR = os.environ.get('APK_SIGNER_JAR', 'uber-apk-signer.jar')
|
|
||||||
|
|
||||||
KS_PATH = None # pass --ks on the command line
|
|
||||||
KS_ALIAS = os.environ.get('APK_KS_ALIAS', 'apkreverse')
|
|
||||||
KS_PASS = os.environ.get('APK_KS_PASS', '')
|
|
||||||
|
|
||||||
STORE_ONLY = ('AndroidManifest.xml', 'resources.arsc')
|
STORE_ONLY = ('AndroidManifest.xml', 'resources.arsc')
|
||||||
|
|
||||||
|
|
||||||
@@ -49,27 +50,118 @@ def log(msg):
|
|||||||
|
|
||||||
|
|
||||||
def run(cmd, **kw):
|
def run(cmd, **kw):
|
||||||
r = subprocess.run(cmd, capture_output=True, text=True, errors='replace', **kw)
|
return subprocess.run(cmd, capture_output=True, text=True, errors='replace', **kw)
|
||||||
return r
|
|
||||||
|
|
||||||
|
|
||||||
def ensure_keystore(path=KS_PATH, alias=KS_ALIAS, password=KS_PASS):
|
# ---------------------------------------------------------------------------
|
||||||
if os.path.exists(path):
|
# Tool resolution: PATH first, explicit flag second, never a baked-in path.
|
||||||
log('[ks] reuse %s' % path)
|
# ---------------------------------------------------------------------------
|
||||||
return path
|
def sibling_tool(exe, name):
|
||||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
"""Look for `name` next to `exe`, for JDKs whose bin dir is not on PATH."""
|
||||||
cmd = [KEYTOOL, '-genkeypair', '-keystore', path, '-alias', alias,
|
if not exe:
|
||||||
|
return None
|
||||||
|
folder = os.path.dirname(exe)
|
||||||
|
for cand in (name, name + '.exe', name + '.bat'):
|
||||||
|
path = os.path.join(folder, cand)
|
||||||
|
if os.path.exists(path):
|
||||||
|
return path
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_tool(name, explicit=None, hint_exe=None):
|
||||||
|
if explicit:
|
||||||
|
return explicit
|
||||||
|
found = shutil.which(name)
|
||||||
|
if found:
|
||||||
|
return found
|
||||||
|
return sibling_tool(hint_exe, name)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_tools(args):
|
||||||
|
"""Locate java/keytool/jarsigner/zipalign/apksigner. Missing ones stay None."""
|
||||||
|
java = resolve_tool('java', args.java)
|
||||||
|
tools = {
|
||||||
|
'java': java,
|
||||||
|
'keytool': resolve_tool('keytool', args.keytool, java),
|
||||||
|
'jarsigner': resolve_tool('jarsigner', args.jarsigner, java),
|
||||||
|
'zipalign': resolve_tool('zipalign', args.zipalign),
|
||||||
|
'apksigner': resolve_tool('apksigner', args.apksigner),
|
||||||
|
}
|
||||||
|
return tools
|
||||||
|
|
||||||
|
|
||||||
|
def require(tools, key, why):
|
||||||
|
if tools.get(key):
|
||||||
|
return tools[key]
|
||||||
|
raise SystemExit(
|
||||||
|
"error: '%s' was not found on PATH, and it is needed to %s.\n"
|
||||||
|
" install a JDK (17+ recommended) / Android build-tools and put them on PATH,\n"
|
||||||
|
" or pass the path explicitly (--%s <path>).\n"
|
||||||
|
" Path lookup is deliberate: a machine-specific absolute path must never be\n"
|
||||||
|
" baked into this script." % (key, why, key))
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Keystore
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
def read_pwd_file(path):
|
||||||
|
if not os.path.exists(path):
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
with open(path, encoding='utf-8') as fh:
|
||||||
|
txt = fh.read().strip()
|
||||||
|
return txt or None
|
||||||
|
except OSError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_keystore(ks_path, alias, password, keytool):
|
||||||
|
"""Return the keystore password, generating the keystore on first use."""
|
||||||
|
pwd_file = ks_path + '.pass.txt'
|
||||||
|
|
||||||
|
if os.path.exists(ks_path):
|
||||||
|
pwd = password or read_pwd_file(pwd_file)
|
||||||
|
if not pwd:
|
||||||
|
raise SystemExit(
|
||||||
|
'error: keystore %s exists but its password is unknown.\n'
|
||||||
|
' pass --ks-pass <pw>, or write the password into %s' % (ks_path, pwd_file))
|
||||||
|
log('[ks] reuse %s (alias=%s)' % (ks_path, alias))
|
||||||
|
return pwd
|
||||||
|
|
||||||
|
pwd = password or read_pwd_file(pwd_file)
|
||||||
|
generated = pwd is None
|
||||||
|
if generated:
|
||||||
|
# keytool rejects passwords shorter than 6 characters.
|
||||||
|
pwd = secrets.token_urlsafe(12)
|
||||||
|
|
||||||
|
folder = os.path.dirname(ks_path)
|
||||||
|
if folder:
|
||||||
|
os.makedirs(folder, exist_ok=True)
|
||||||
|
cmd = [keytool, '-genkeypair', '-keystore', ks_path, '-alias', alias,
|
||||||
'-keyalg', 'RSA', '-keysize', '2048', '-validity', '10000',
|
'-keyalg', 'RSA', '-keysize', '2048', '-validity', '10000',
|
||||||
'-storepass', password, '-keypass', password,
|
'-storepass', pwd, '-keypass', pwd,
|
||||||
'-dname', 'CN=apkreverse, OU=dev, O=dev, L=NA, ST=NA, C=NA']
|
'-dname', 'CN=apkreverse, OU=dev, O=dev, L=NA, ST=NA, C=NA']
|
||||||
r = run(cmd)
|
r = run(cmd)
|
||||||
if r.returncode != 0:
|
if r.returncode != 0:
|
||||||
log('[ks] keytool failed rc=%d\n%s\n%s' % (r.returncode, r.stdout, r.stderr))
|
log('[ks] keytool failed rc=%d\n%s\n%s' % (r.returncode, r.stdout, r.stderr))
|
||||||
raise SystemExit(1)
|
raise SystemExit(1)
|
||||||
log('[ks] generated %s (alias=%s pass=%s)' % (path, alias, password))
|
|
||||||
return path
|
try:
|
||||||
|
with open(pwd_file, 'w', encoding='utf-8') as fh:
|
||||||
|
fh.write(pwd + '\n')
|
||||||
|
os.chmod(pwd_file, 0o600)
|
||||||
|
except OSError as exc:
|
||||||
|
log('[ks] could not persist the password: %s' % exc)
|
||||||
|
|
||||||
|
log('[ks] generated %s (alias=%s)' % (ks_path, alias))
|
||||||
|
if generated:
|
||||||
|
log('[ks] password stored in %s -- local artifact, do not commit it' % pwd_file)
|
||||||
|
return pwd
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Packing
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
def collect_replacement_dex(dexdir=None, dex_pairs=None):
|
def collect_replacement_dex(dexdir=None, dex_pairs=None):
|
||||||
repl = {}
|
repl = {}
|
||||||
if dexdir:
|
if dexdir:
|
||||||
@@ -79,16 +171,17 @@ def collect_replacement_dex(dexdir=None, dex_pairs=None):
|
|||||||
for pair in (dex_pairs or []):
|
for pair in (dex_pairs or []):
|
||||||
if '=' not in pair:
|
if '=' not in pair:
|
||||||
raise SystemExit('--dex expects name=path, got %r' % pair)
|
raise SystemExit('--dex expects name=path, got %r' % pair)
|
||||||
name, p = pair.split('=', 1)
|
name, path = pair.split('=', 1)
|
||||||
repl[name] = p
|
repl[name] = path
|
||||||
return repl
|
return repl
|
||||||
|
|
||||||
|
|
||||||
def is_signature_entry(name):
|
def is_signature_entry(name):
|
||||||
"""True only for JAR/APK signature artifacts directly under META-INF/.
|
"""True ONLY for JAR/APK signature artifacts directly under META-INF/.
|
||||||
|
|
||||||
META-INF/services/**, META-INF/androidx/**, META-INF/native-image/** and all
|
META-INF/services/**, META-INF/androidx/**, META-INF/native-image/** and every
|
||||||
other subdirectories are RUNTIME RESOURCES and must survive repacking.
|
other subdirectory are RUNTIME RESOURCES and must survive repacking. Anything
|
||||||
|
that is not directly under META-INF/ is kept by construction.
|
||||||
"""
|
"""
|
||||||
if not name.upper().startswith('META-INF/'):
|
if not name.upper().startswith('META-INF/'):
|
||||||
return False
|
return False
|
||||||
@@ -101,20 +194,20 @@ def is_signature_entry(name):
|
|||||||
return up.endswith(('.SF', '.RSA', '.DSA', '.EC'))
|
return up.endswith(('.SF', '.RSA', '.DSA', '.EC'))
|
||||||
|
|
||||||
|
|
||||||
def build_unsigned(apk, repl, out_apk, drop_meta=True):
|
def build_unsigned(apk, repl, out_apk, drop_signatures=True):
|
||||||
"""Write a new apk with dex replaced and only signature entries dropped.
|
"""Write a new apk with dex replaced and only signature entries dropped.
|
||||||
|
|
||||||
IMPORTANT (root cause of a crash, do not regress):
|
IMPORTANT (root cause of a startup crash, do not regress):
|
||||||
Stripping the WHOLE META-INF/ breaks ServiceLoader-based runtime wiring.
|
Stripping the WHOLE META-INF/ breaks ServiceLoader-based runtime wiring. Android
|
||||||
This APK ships ServiceLoader registrations that Android reads at runtime:
|
reads these registries at runtime, and they live in the APK:
|
||||||
META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory -> kc
|
META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory
|
||||||
META-INF/services/io.ktor.client.HttpClientEngineContainer -> OkHttpEngineContainer
|
META-INF/services/io.ktor.client.HttpClientEngineContainer
|
||||||
META-INF/services/io.ktor.serialization.kotlinx.KotlinxSerializationExtensionProvider
|
META-INF/services/<lib>.core.* (third-party SDK registries)
|
||||||
META-INF/services/kotlinx.coroutines.CoroutineExceptionHandler -> wc
|
META-INF/services/<obfuscated-class-name> (R8-renamed providers)
|
||||||
META-INF/services/com.arialyy.aria.* , META-INF/services/wt2 , META-INF/services/xx1
|
Deleting them makes the app die at startup with something like:
|
||||||
Deleting them makes the app die at startup with:
|
|
||||||
IllegalStateException: Module with the Main dispatcher is missing ...
|
IllegalStateException: Module with the Main dispatcher is missing ...
|
||||||
So only signature artifacts are removed here; META-INF subdirectories are kept.
|
and the message never points at META-INF, so the cause is very hard to find.
|
||||||
|
Therefore: drop signature artifacts only, keep every META-INF subdirectory.
|
||||||
"""
|
"""
|
||||||
seen = set()
|
seen = set()
|
||||||
replaced_names = set(repl)
|
replaced_names = set(repl)
|
||||||
@@ -124,7 +217,8 @@ def build_unsigned(apk, repl, out_apk, drop_meta=True):
|
|||||||
base = os.path.basename(name)
|
base = os.path.basename(name)
|
||||||
if item.is_dir():
|
if item.is_dir():
|
||||||
continue
|
continue
|
||||||
if drop_meta and is_signature_entry(name):
|
if drop_signatures and is_signature_entry(name):
|
||||||
|
log('[zip] - %s (signature artifact)' % name)
|
||||||
continue
|
continue
|
||||||
if name in replaced_names:
|
if name in replaced_names:
|
||||||
continue
|
continue
|
||||||
@@ -138,8 +232,8 @@ def build_unsigned(apk, repl, out_apk, drop_meta=True):
|
|||||||
zout.writestr(zi, data)
|
zout.writestr(zi, data)
|
||||||
seen.add(name)
|
seen.add(name)
|
||||||
for name, path in sorted(repl.items()):
|
for name, path in sorted(repl.items()):
|
||||||
with open(path, 'rb') as f:
|
with open(path, 'rb') as fh:
|
||||||
data = f.read()
|
data = fh.read()
|
||||||
zi = zipfile.ZipInfo(name, date_time=(2024, 1, 1, 0, 0, 0))
|
zi = zipfile.ZipInfo(name, date_time=(2024, 1, 1, 0, 0, 0))
|
||||||
zi.compress_type = zipfile.ZIP_DEFLATED
|
zi.compress_type = zipfile.ZIP_DEFLATED
|
||||||
zout.writestr(zi, data)
|
zout.writestr(zi, data)
|
||||||
@@ -160,12 +254,23 @@ def zip_report(apk):
|
|||||||
return rows
|
return rows
|
||||||
|
|
||||||
|
|
||||||
def sign_apk(unsigned_apk, workdir, ks, alias=KS_ALIAS, password=KS_PASS):
|
# ---------------------------------------------------------------------------
|
||||||
|
# Sign / verify
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
def sign_apk(unsigned_apk, workdir, ks, alias, password, tools, signer_jar):
|
||||||
|
java = require(tools, 'java', 'run the signer jar')
|
||||||
|
if not os.path.exists(signer_jar):
|
||||||
|
raise SystemExit(
|
||||||
|
'error: signer jar not found: %s\n'
|
||||||
|
' pass --signer-jar <path>, or set APK_SIGNER_JAR.\n'
|
||||||
|
' Any zipalign+apksigner based signer works here.' % signer_jar)
|
||||||
|
|
||||||
outdir = os.path.join(workdir, 'signed')
|
outdir = os.path.join(workdir, 'signed')
|
||||||
if os.path.isdir(outdir):
|
if os.path.isdir(outdir):
|
||||||
shutil.rmtree(outdir, ignore_errors=True)
|
shutil.rmtree(outdir, ignore_errors=True)
|
||||||
os.makedirs(outdir, exist_ok=True)
|
os.makedirs(outdir, exist_ok=True)
|
||||||
cmd = [JAVA, '-jar', SIGNER_JAR,
|
|
||||||
|
cmd = [java, '-jar', signer_jar,
|
||||||
'--apks', unsigned_apk,
|
'--apks', unsigned_apk,
|
||||||
'--ks', ks, '--ksAlias', alias,
|
'--ks', ks, '--ksAlias', alias,
|
||||||
'--ksPass', password, '--ksKeyPass', password,
|
'--ksPass', password, '--ksKeyPass', password,
|
||||||
@@ -175,67 +280,149 @@ def sign_apk(unsigned_apk, workdir, ks, alias=KS_ALIAS, password=KS_PASS):
|
|||||||
log(r.stdout.strip())
|
log(r.stdout.strip())
|
||||||
if r.stderr.strip():
|
if r.stderr.strip():
|
||||||
log('[sign][stderr] ' + r.stderr.strip())
|
log('[sign][stderr] ' + r.stderr.strip())
|
||||||
|
|
||||||
cand = os.path.join(outdir, os.path.basename(unsigned_apk))
|
cand = os.path.join(outdir, os.path.basename(unsigned_apk))
|
||||||
if not os.path.exists(cand):
|
if not os.path.exists(cand):
|
||||||
outs = [os.path.join(outdir, f) for f in os.listdir(outdir)] if os.path.isdir(outdir) else []
|
outs = [os.path.join(outdir, f) for f in os.listdir(outdir)] \
|
||||||
|
if os.path.isdir(outdir) else []
|
||||||
if not outs:
|
if not outs:
|
||||||
raise SystemExit('signing produced no output')
|
raise SystemExit('signing produced no output')
|
||||||
cand = outs[0]
|
cand = outs[0]
|
||||||
return cand, r.stdout
|
return cand
|
||||||
|
|
||||||
|
|
||||||
def verify_apk(apk):
|
def verify_apk(apk, tools, signer_jar=None, expect_signed=True):
|
||||||
log('== verify: %s' % apk)
|
log('== verify: %s' % apk)
|
||||||
ok = True
|
ok = True
|
||||||
# 1) uber-apk-signer verify (also re-checks zipalign)
|
sig_checks = 0 # real signature-verification tools that actually ran
|
||||||
r = run([JAVA, '-jar', SIGNER_JAR, '-a', apk, '-y'])
|
|
||||||
log('[verify:signer] rc=%d\n%s' % (r.returncode, r.stdout.strip()))
|
# 1) apksigner -- the authoritative check, but ONLY with an explicit sdk range.
|
||||||
if r.returncode != 0 or 'Verified' not in r.stdout:
|
#
|
||||||
ok = ok and ('DOES NOT VERIFY' not in r.stdout)
|
# READ THIS BEFORE CONCLUDING THE SIGNATURE IS BROKEN:
|
||||||
# 2) jarsigner (v1 / jAR signature)
|
# `apksigner verify` with no range checks the signature schemes implied by the
|
||||||
r2 = run([JARSIGNER, '-verify', '-certs', apk])
|
# APK's own minSdkVersion. With minSdk >= 24 it prints v1/v2 as false while the
|
||||||
txt = (r2.stdout or '') + (r2.stderr or '')
|
# files in META-INF are perfectly fine, which looks exactly like a failed signing
|
||||||
log('[verify:jarsigner] rc=%d\n%s' % (r2.returncode, txt.strip()[:2000]))
|
# step. Always pass --min-sdk-version / --max-sdk-version so the v1/v2/v3 results
|
||||||
# 3) keytool cert dump
|
# are meaningful.
|
||||||
r3 = run([KEYTOOL, '-printcert', '-jarfile', apk])
|
if tools.get('apksigner'):
|
||||||
log('[verify:keytool] rc=%d\n%s' % (r3.returncode, r3.stdout.strip()[:1500]))
|
sig_checks += 1
|
||||||
# 4) v2/v3 APK Signing Block presence
|
r = run([tools['apksigner'], 'verify', '--print-certs', '--verbose',
|
||||||
with open(apk, 'rb') as f:
|
'--min-sdk-version', '21', '--max-sdk-version', '34', apk])
|
||||||
blob = f.read()
|
log('[verify:apksigner] rc=%d\n%s' % (r.returncode, ((r.stdout or '') +
|
||||||
has_v2 = b'APK Sig Block 42' in blob
|
(r.stderr or '')).strip()[:3000]))
|
||||||
log('[verify:v2block] %s' % ('present' if has_v2 else 'MISSING'))
|
if r.returncode != 0:
|
||||||
return ok and (r.returncode == 0)
|
ok = False
|
||||||
|
else:
|
||||||
|
log('[verify:apksigner] skipped: apksigner not on PATH.\n'
|
||||||
|
' Do NOT judge the signature from a bare `apksigner verify`: with\n'
|
||||||
|
' minSdk >= 24 its default range reports v1/v2 as false even when the\n'
|
||||||
|
' signature is valid. The correct invocation is:\n'
|
||||||
|
' apksigner verify --print-certs --verbose --min-sdk-version 21 '
|
||||||
|
'--max-sdk-version 34 <apk>')
|
||||||
|
|
||||||
|
# 2) signer jar's own verify (also re-checks alignment)
|
||||||
|
if signer_jar and os.path.exists(signer_jar) and tools.get('java'):
|
||||||
|
sig_checks += 1
|
||||||
|
r = run([tools['java'], '-jar', signer_jar, '-a', apk, '-y'])
|
||||||
|
log('[verify:signer] rc=%d\n%s' % (r.returncode, (r.stdout or '').strip()))
|
||||||
|
if r.returncode != 0 and 'DOES NOT VERIFY' in (r.stdout or ''):
|
||||||
|
ok = False
|
||||||
|
|
||||||
|
# 3) jarsigner (v1 / JAR signature)
|
||||||
|
if tools.get('jarsigner'):
|
||||||
|
sig_checks += 1
|
||||||
|
r = run([tools['jarsigner'], '-verify', '-certs', apk])
|
||||||
|
txt = ((r.stdout or '') + (r.stderr or '')).strip()
|
||||||
|
log('[verify:jarsigner] rc=%d\n%s' % (r.returncode, txt[:2000]))
|
||||||
|
if 'jar verified' not in txt and 'verified' not in txt.lower():
|
||||||
|
log('[verify:jarsigner] note: not reported as verified (see output above)')
|
||||||
|
|
||||||
|
# 4) keytool cert dump
|
||||||
|
if tools.get('keytool'):
|
||||||
|
r = run([tools['keytool'], '-printcert', '-jarfile', apk])
|
||||||
|
log('[verify:keytool] rc=%d\n%s' % (r.returncode, (r.stdout or '').strip()[:1500]))
|
||||||
|
|
||||||
|
# 5) zipalign check
|
||||||
|
if tools.get('zipalign'):
|
||||||
|
r = run([tools['zipalign'], '-c', '-v', '4', apk])
|
||||||
|
log('[verify:zipalign] rc=%d' % r.returncode)
|
||||||
|
if r.returncode != 0:
|
||||||
|
log((r.stdout or '').strip()[:1500])
|
||||||
|
else:
|
||||||
|
log('[verify:zipalign] skipped: zipalign not on PATH')
|
||||||
|
|
||||||
|
# 6) v2/v3 APK Signing Block presence, read straight from the file
|
||||||
|
with open(apk, 'rb') as fh:
|
||||||
|
blob = fh.read()
|
||||||
|
log('[verify:v2block] %s' % ('present' if b'APK Sig Block 42' in blob else 'MISSING'))
|
||||||
|
|
||||||
|
# A build nobody could verify is not a passing build. Saying OK here would
|
||||||
|
# violate the first rule of this skill ("an APK is not done until it is
|
||||||
|
# verified"): report UNVERIFIED and fail instead.
|
||||||
|
if expect_signed and sig_checks == 0:
|
||||||
|
log('[result] UNVERIFIED: no signature verification tool was available '
|
||||||
|
'(apksigner / jarsigner / signer jar). This is NOT an OK result.')
|
||||||
|
return False
|
||||||
|
return ok
|
||||||
|
|
||||||
|
|
||||||
def fingerprint(apk):
|
def fingerprint(apk, tools):
|
||||||
r = run([KEYTOOL, '-printcert', '-jarfile', apk])
|
if not tools.get('keytool'):
|
||||||
sha = re.findall(r'(SHA1|SHA256):\s*([0-9A-F:]+)', r.stdout)
|
return []
|
||||||
return sha
|
r = run([tools['keytool'], '-printcert', '-jarfile', apk])
|
||||||
|
return re.findall(r'(SHA1|SHA256):\s*([0-9A-F:]+)', r.stdout or '')
|
||||||
|
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
ap = argparse.ArgumentParser()
|
ap = argparse.ArgumentParser(
|
||||||
ap.add_argument('--apk', required=True)
|
description='Repack an APK with replaced dex files, drop only the signature '
|
||||||
ap.add_argument('--out', required=True)
|
'entries, re-sign and verify.')
|
||||||
ap.add_argument('--dexdir')
|
ap.add_argument('--apk', required=True, help='source apk to use as the template')
|
||||||
ap.add_argument('--dex', action='append', default=[], help='name=path, repeatable')
|
ap.add_argument('--out', required=True, help='output apk path')
|
||||||
ap.add_argument('--ks', default=KS_PATH)
|
ap.add_argument('--dexdir', help='directory of classes*.dex to swap in')
|
||||||
ap.add_argument('--no-sign', action='store_true')
|
ap.add_argument('--dex', action='append', default=[],
|
||||||
ap.add_argument('--workdir', default=None)
|
help='name=path, repeatable, e.g. classes8.dex=work/patch/classes8.dex')
|
||||||
|
ap.add_argument('--workdir', default=None,
|
||||||
|
help='scratch directory (default: next to --out)')
|
||||||
|
ap.add_argument('--ks', default=None, help='keystore path (created on first use)')
|
||||||
|
ap.add_argument('--ks-alias', default='apkreverse', help='keystore alias')
|
||||||
|
ap.add_argument('--ks-pass', default=None,
|
||||||
|
help='keystore password; default: <ks>.pass.txt, else generated')
|
||||||
|
ap.add_argument('--signer-jar', default=None,
|
||||||
|
help='jar used for zipalign+sign (default: $APK_SIGNER_JAR or '
|
||||||
|
'uber-apk-signer.jar)')
|
||||||
|
ap.add_argument('--no-sign', action='store_true', help='stop after writing the apk')
|
||||||
|
ap.add_argument('--java', default=None, help='path to java (default: from PATH)')
|
||||||
|
ap.add_argument('--keytool', default=None, help='path to keytool (default: from PATH)')
|
||||||
|
ap.add_argument('--jarsigner', default=None, help='path to jarsigner (default: from PATH)')
|
||||||
|
ap.add_argument('--zipalign', default=None, help='path to zipalign (default: from PATH)')
|
||||||
|
ap.add_argument('--apksigner', default=None, help='path to apksigner (default: from PATH)')
|
||||||
args = ap.parse_args()
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
tools = resolve_tools(args)
|
||||||
|
signer_jar = args.signer_jar or os.environ.get('APK_SIGNER_JAR', 'uber-apk-signer.jar')
|
||||||
|
|
||||||
apk = os.path.abspath(args.apk)
|
apk = os.path.abspath(args.apk)
|
||||||
out = os.path.abspath(args.out)
|
out = os.path.abspath(args.out)
|
||||||
if not os.path.exists(apk):
|
if not os.path.exists(apk):
|
||||||
raise SystemExit('missing apk: %s' % apk)
|
raise SystemExit('missing apk: %s' % apk)
|
||||||
os.makedirs(os.path.dirname(out), exist_ok=True)
|
|
||||||
os.makedirs(args.workdir, exist_ok=True)
|
# Default the scratch dir next to --out; never to a machine-specific path.
|
||||||
|
workdir = os.path.abspath(args.workdir) if args.workdir else os.path.dirname(out)
|
||||||
|
if not workdir:
|
||||||
|
workdir = os.getcwd()
|
||||||
|
os.makedirs(workdir, exist_ok=True)
|
||||||
|
out_folder = os.path.dirname(out)
|
||||||
|
if out_folder:
|
||||||
|
os.makedirs(out_folder, exist_ok=True)
|
||||||
|
|
||||||
repl = collect_replacement_dex(args.dexdir, args.dex)
|
repl = collect_replacement_dex(args.dexdir, args.dex)
|
||||||
log('[in ] %s (%d bytes)' % (apk, os.path.getsize(apk)))
|
log('[in ] %s (%d bytes)' % (apk, os.path.getsize(apk)))
|
||||||
log('[repl] %s' % ', '.join('%s<-%s' % (k, v) for k, v in sorted(repl.items())))
|
log('[tools] java=%s keytool=%s apksigner=%s zipalign=%s'
|
||||||
|
% (tools['java'], tools['keytool'], tools['apksigner'], tools['zipalign']))
|
||||||
|
log('[repl] %s' % (', '.join('%s<-%s' % (k, v) for k, v in sorted(repl.items())) or '<none>'))
|
||||||
|
|
||||||
unsigned = os.path.join(args.workdir, 'unsigned.apk')
|
unsigned = os.path.join(workdir, 'unsigned.apk')
|
||||||
if os.path.exists(unsigned):
|
if os.path.exists(unsigned):
|
||||||
os.remove(unsigned)
|
os.remove(unsigned)
|
||||||
build_unsigned(apk, repl, unsigned)
|
build_unsigned(apk, repl, unsigned)
|
||||||
@@ -246,19 +433,31 @@ def main():
|
|||||||
if args.no_sign:
|
if args.no_sign:
|
||||||
shutil.copy2(unsigned, out)
|
shutil.copy2(unsigned, out)
|
||||||
else:
|
else:
|
||||||
ks = ensure_keystore(args.ks)
|
if not args.ks:
|
||||||
signed, _ = sign_apk(unsigned, args.workdir, ks)
|
raise SystemExit(
|
||||||
|
"error: --ks is required when signing (there is no default keystore).\n"
|
||||||
|
" e.g. --ks work/out/release.keystore")
|
||||||
|
keytool = require(tools, 'keytool', 'create the keystore on first use')
|
||||||
|
password = ensure_keystore(os.path.abspath(args.ks), args.ks_alias,
|
||||||
|
args.ks_pass, keytool)
|
||||||
|
signed = sign_apk(unsigned, workdir, os.path.abspath(args.ks), args.ks_alias,
|
||||||
|
password, tools, signer_jar)
|
||||||
shutil.copy2(signed, out)
|
shutil.copy2(signed, out)
|
||||||
|
|
||||||
log('[out] %s (%d bytes)' % (out, os.path.getsize(out)))
|
log('[out] %s (%d bytes)' % (out, os.path.getsize(out)))
|
||||||
log('== zip layout of final apk')
|
log('== zip layout of final apk')
|
||||||
for row in zip_report(out):
|
for row in zip_report(out):
|
||||||
log(' ' + row)
|
log(' ' + row)
|
||||||
ok = verify_apk(out)
|
|
||||||
sha = fingerprint(out)
|
ok = verify_apk(out, tools, signer_jar, expect_signed=not args.no_sign)
|
||||||
for algo, val in sha:
|
if not args.no_sign:
|
||||||
log('[cert] %s %s' % (algo, val))
|
for algo, val in fingerprint(out, tools):
|
||||||
log('[result] %s' % ('OK' if ok else 'CHECK-FAILED'))
|
log('[cert] %s %s' % (algo, val))
|
||||||
|
log('[result] %s' % ('OK' if ok else 'CHECK-FAILED'))
|
||||||
|
else:
|
||||||
|
log('[result] UNSIGNED (--no-sign): good for inspection, not installable as-is')
|
||||||
|
log('[reminder] repacking is not done until the app launches and the changed '
|
||||||
|
'behavior is exercised on a device: see scripts/install_test.py')
|
||||||
return 0 if ok else 2
|
return 0 if ok else 2
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,360 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""Inject a Frida probe script into a running Android app and stay resident.
|
||||||
|
|
||||||
|
WHY THIS EXISTS
|
||||||
|
---------------
|
||||||
|
`frida -U -f <pkg> -l script.js` has two failure modes that cost real debugging time:
|
||||||
|
1. With a real device AND an emulator both attached, the USB auto-selection picks
|
||||||
|
the emulator, so you attach to the wrong process and see nothing. This tool
|
||||||
|
prefers an explicit remote device over `adb forward`, which cannot be hijacked.
|
||||||
|
2. Probe output that only goes to a terminal is lost the moment the terminal
|
||||||
|
scrolls, the session detaches, or the app crashes. Here every message is
|
||||||
|
written to a log file *and* to stdout, with both the device timestamp and the
|
||||||
|
host arrival timestamp -- that pair is itself evidence when you suspect a
|
||||||
|
clock skew problem.
|
||||||
|
|
||||||
|
It also answers the three questions that otherwise produce a bare traceback:
|
||||||
|
* is the `frida` python module importable, and which version?
|
||||||
|
* is `adb` reachable, and is the target process actually running?
|
||||||
|
* is an on-device frida-server answering on the forwarded port?
|
||||||
|
|
||||||
|
USAGE
|
||||||
|
-----
|
||||||
|
python run_probe.py frida_probe.js 10 --pkg com.example.app
|
||||||
|
python run_probe.py frida_probe.js 5 --pkg com.example.app --device <serial>
|
||||||
|
python run_probe.py frida_probe.js 5 --pkg com.example.app --via usb
|
||||||
|
python run_probe.py frida_probe.js 2 --pkg com.example.app --spawn
|
||||||
|
python run_probe.py frida_probe.js 10 --pkg com.example.app --log my.log --port 27043
|
||||||
|
|
||||||
|
<script.js> the probe to inject (e.g. frida_probe.js, see its header to adapt)
|
||||||
|
<minutes> how long to stay attached (default 10)
|
||||||
|
|
||||||
|
NOTES
|
||||||
|
-----
|
||||||
|
* The host `frida` package and the on-device `frida-server` must be the SAME
|
||||||
|
version, and 16.x is the version to align on: 17.x can fail to locate the
|
||||||
|
Android dynamic linker and removes the built-in Java bridge.
|
||||||
|
* `--spawn` is the right choice when the problem happens at startup: attach-only
|
||||||
|
misses init and the first network calls. Hooks are installed before resume.
|
||||||
|
* Log file: a timestamped name is generated unless --log is given.
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
DEFAULT_PORT = 27042
|
||||||
|
DEFAULT_MINUTES = 10
|
||||||
|
DEFAULT_SERVER_PORT = 27042 # the port frida-server listens on, on the device
|
||||||
|
|
||||||
|
FRIDA_INSTALL_HINT = (
|
||||||
|
'install the matching host package and the matching device server, e.g.\n'
|
||||||
|
' pip install frida==16.7.19\n'
|
||||||
|
' # then push frida-server-16.7.19-android-<abi> to the device and run it as root'
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def host_ts():
|
||||||
|
return datetime.now().strftime('%Y-%m-%d %H:%M:%S.%f')[:-3]
|
||||||
|
|
||||||
|
|
||||||
|
class Logger(object):
|
||||||
|
"""Write every line to stdout AND to a file, flushing both immediately."""
|
||||||
|
|
||||||
|
def __init__(self, path):
|
||||||
|
self.path = path
|
||||||
|
self.fh = open(path, 'a', encoding='utf-8', errors='replace')
|
||||||
|
|
||||||
|
def line(self, text):
|
||||||
|
print(text, flush=True)
|
||||||
|
self.fh.write(text + '\n')
|
||||||
|
self.fh.flush()
|
||||||
|
|
||||||
|
def close(self):
|
||||||
|
try:
|
||||||
|
self.fh.close()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def adb_run(adb, serial, args, timeout=30):
|
||||||
|
cmd = [adb]
|
||||||
|
if serial:
|
||||||
|
cmd += ['-s', serial]
|
||||||
|
cmd += args
|
||||||
|
return subprocess.run(cmd, capture_output=True, text=True,
|
||||||
|
errors='replace', timeout=timeout)
|
||||||
|
|
||||||
|
|
||||||
|
def find_adb(explicit=None):
|
||||||
|
if explicit:
|
||||||
|
return explicit
|
||||||
|
return shutil.which('adb') or shutil.which('adb.exe')
|
||||||
|
|
||||||
|
|
||||||
|
def get_pid(adb, serial, pkg):
|
||||||
|
"""Return (pid, raw_output). pid is None when the process is not running."""
|
||||||
|
try:
|
||||||
|
r = adb_run(adb, serial, ['shell', 'pidof', pkg])
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
return None, 'adb timed out'
|
||||||
|
raw = ((r.stdout or '') + (r.stderr or '')).strip()
|
||||||
|
if r.returncode != 0 or not raw:
|
||||||
|
return None, raw
|
||||||
|
first = raw.split()[0]
|
||||||
|
if not first.isdigit():
|
||||||
|
return None, raw
|
||||||
|
return int(first), raw
|
||||||
|
|
||||||
|
|
||||||
|
def attach_remote(frida, port, logger):
|
||||||
|
mgr = frida.get_device_manager()
|
||||||
|
logger.line('[host %s] registering remote device 127.0.0.1:%d' % (host_ts(), port))
|
||||||
|
return mgr.add_remote_device('127.0.0.1:%d' % port)
|
||||||
|
|
||||||
|
|
||||||
|
def attach_usb(frida, device_id, logger):
|
||||||
|
if device_id:
|
||||||
|
logger.line('[host %s] selecting usb device %s' % (host_ts(), device_id))
|
||||||
|
return frida.get_device(device_id, timeout=5)
|
||||||
|
logger.line('[host %s] selecting usb device (auto; an emulator can win this race)'
|
||||||
|
% host_ts())
|
||||||
|
return frida.get_usb_device(timeout=5)
|
||||||
|
|
||||||
|
|
||||||
|
def format_message(message):
|
||||||
|
"""Turn a frida message into one log line. Never raises."""
|
||||||
|
kind = message.get('type')
|
||||||
|
if kind == 'send':
|
||||||
|
payload = message.get('payload')
|
||||||
|
if isinstance(payload, dict):
|
||||||
|
dev = payload.get('t') or '-'
|
||||||
|
tag = payload.get('tag') or 'SEND'
|
||||||
|
msg = payload.get('msg')
|
||||||
|
if msg is None:
|
||||||
|
msg = payload
|
||||||
|
return dev, '%s %s' % (tag, msg)
|
||||||
|
return '-', 'SEND %r' % (payload,)
|
||||||
|
if kind == 'error':
|
||||||
|
desc = message.get('description') or ''
|
||||||
|
where = '%s:%s' % (message.get('fileName') or '?', message.get('lineNumber') or '?')
|
||||||
|
stack = message.get('stack') or ''
|
||||||
|
out = 'FRIDA-ERROR %s (%s)' % (desc, where)
|
||||||
|
if stack:
|
||||||
|
out += '\n ' + stack.replace('\n', '\n ')
|
||||||
|
return '-', out
|
||||||
|
return '-', '%s %r' % (kind, message)
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(
|
||||||
|
description='Inject a Frida probe into a running Android app, log everything '
|
||||||
|
'to stdout and a file, and stay attached for N minutes.')
|
||||||
|
ap.add_argument('script', help='path to the frida JS probe, e.g. frida_probe.js')
|
||||||
|
ap.add_argument('minutes', nargs='?', type=float, default=DEFAULT_MINUTES,
|
||||||
|
help='how long to stay attached (default %d)' % DEFAULT_MINUTES)
|
||||||
|
ap.add_argument('--pkg', default=None, help='target package name, e.g. com.example.app')
|
||||||
|
ap.add_argument('--device', default=None,
|
||||||
|
help='adb serial (remote mode) or frida device id (usb mode)')
|
||||||
|
ap.add_argument('--via', choices=['remote', 'usb'], default='remote',
|
||||||
|
help='remote = adb forward + explicit remote device (default, '
|
||||||
|
'cannot be hijacked by an emulator); usb = frida.get_usb_device()')
|
||||||
|
ap.add_argument('--port', type=int, default=DEFAULT_PORT,
|
||||||
|
help='host-side forwarded port (default %d)' % DEFAULT_PORT)
|
||||||
|
ap.add_argument('--server-port', type=int, default=DEFAULT_SERVER_PORT,
|
||||||
|
help='port frida-server listens on, on the device (default %d)'
|
||||||
|
% DEFAULT_SERVER_PORT)
|
||||||
|
ap.add_argument('--spawn', action='store_true',
|
||||||
|
help='spawn the app instead of attaching, so hooks are live before '
|
||||||
|
'the first network call')
|
||||||
|
ap.add_argument('--log', default=None, help='log file path (default: auto-named)')
|
||||||
|
ap.add_argument('--adb', default=None, help='path to adb (default: from PATH)')
|
||||||
|
ap.add_argument('--script-timeout', type=float, default=30.0,
|
||||||
|
help='seconds to wait for the script to load')
|
||||||
|
ap.add_argument('--keep-forward', action='store_true',
|
||||||
|
help='leave the adb forward in place after exit')
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
if not os.path.isfile(args.script):
|
||||||
|
print('probe script not found: %s' % args.script)
|
||||||
|
return 2
|
||||||
|
if args.spawn and not args.pkg:
|
||||||
|
print('--spawn needs --pkg (there is no pid to attach to)')
|
||||||
|
return 2
|
||||||
|
|
||||||
|
log_name = args.log or ('probe-%s-%s.log'
|
||||||
|
% (args.pkg or 'target', datetime.now().strftime('%Y%m%d-%H%M%S')))
|
||||||
|
logger = Logger(log_name)
|
||||||
|
logger.line('# probe log %s' % os.path.abspath(log_name))
|
||||||
|
logger.line('# script=%s minutes=%s via=%s pkg=%s'
|
||||||
|
% (os.path.abspath(args.script), args.minutes, args.via, args.pkg))
|
||||||
|
|
||||||
|
# 1) frida python package
|
||||||
|
try:
|
||||||
|
import frida
|
||||||
|
except ImportError:
|
||||||
|
logger.line(host_ts() + ' FATAL cannot import the frida python module.\n' +
|
||||||
|
FRIDA_INSTALL_HINT)
|
||||||
|
logger.close()
|
||||||
|
return 3
|
||||||
|
logger.line('[host %s] frida python %s' % (host_ts(), getattr(frida, '__version__', '?')))
|
||||||
|
|
||||||
|
# 2) adb
|
||||||
|
adb = find_adb(args.adb)
|
||||||
|
if not adb:
|
||||||
|
logger.line(host_ts() + ' FATAL adb not found.\n'
|
||||||
|
' put platform-tools on PATH, or pass --adb <path>')
|
||||||
|
logger.close()
|
||||||
|
return 3
|
||||||
|
logger.line('[host %s] adb %s' % (host_ts(), adb))
|
||||||
|
|
||||||
|
pid = None
|
||||||
|
forward_added = False
|
||||||
|
device = None
|
||||||
|
session = None
|
||||||
|
script = None
|
||||||
|
counts = {}
|
||||||
|
deadline = time.monotonic() + max(args.minutes, 0) * 60.0
|
||||||
|
|
||||||
|
try:
|
||||||
|
# 3) remote device over adb forward (default), or plain USB selection
|
||||||
|
if args.via == 'remote':
|
||||||
|
try:
|
||||||
|
r = adb_run(adb, args.device,
|
||||||
|
['forward', 'tcp:%d' % args.port, 'tcp:%d' % args.server_port])
|
||||||
|
except subprocess.TimeoutExpired:
|
||||||
|
logger.line(host_ts() + ' FATAL adb forward timed out; is the device responsive?')
|
||||||
|
return 3
|
||||||
|
if r.returncode != 0:
|
||||||
|
msg = ((r.stdout or '') + (r.stderr or '')).strip()
|
||||||
|
logger.line(host_ts() + ' FATAL adb forward failed.\n'
|
||||||
|
' adb said: %s\n'
|
||||||
|
' if it mentions more than one device, pass --device <serial>;\n'
|
||||||
|
' check the device is visible with: adb devices' % msg)
|
||||||
|
return 3
|
||||||
|
forward_added = True
|
||||||
|
logger.line('[host %s] forwarded tcp:%d -> device tcp:%d'
|
||||||
|
% (host_ts(), args.port, args.server_port))
|
||||||
|
device = attach_remote(frida, args.port, logger)
|
||||||
|
else:
|
||||||
|
device = attach_usb(frida, args.device, logger)
|
||||||
|
|
||||||
|
# 4) pid: spawn when asked, otherwise attach to the running process
|
||||||
|
if args.spawn:
|
||||||
|
try:
|
||||||
|
pid = device.spawn([args.pkg])
|
||||||
|
except Exception as exc:
|
||||||
|
logger.line(host_ts() + ' FATAL spawn failed for %s: %s\n'
|
||||||
|
' check the package is installed: adb shell pm list packages | grep %s'
|
||||||
|
% (args.pkg, exc, args.pkg))
|
||||||
|
return 3
|
||||||
|
logger.line('[device] spawned %s pid=%s' % (args.pkg, pid))
|
||||||
|
else:
|
||||||
|
if not args.pkg:
|
||||||
|
logger.line(host_ts() + ' FATAL --pkg is required when attaching '
|
||||||
|
'(or use --spawn with --pkg)')
|
||||||
|
return 3
|
||||||
|
pid, raw = get_pid(adb, args.device, args.pkg)
|
||||||
|
if pid is None:
|
||||||
|
logger.line(host_ts() + ' FATAL process not found: %s\n'
|
||||||
|
' adb output: %s\n'
|
||||||
|
' start the app first, or use --spawn to launch it under the probe;\n'
|
||||||
|
' if pidof is unavailable on this ROM, try: adb shell ps -A'
|
||||||
|
% (args.pkg, raw or '<empty>'))
|
||||||
|
return 3
|
||||||
|
logger.line('[device] pid of %s = %d' % (args.pkg, pid))
|
||||||
|
|
||||||
|
# 5) attach + load
|
||||||
|
try:
|
||||||
|
session = device.attach(pid)
|
||||||
|
except Exception as exc:
|
||||||
|
logger.line(host_ts() + ' FATAL attach failed: %s\n'
|
||||||
|
' usual causes: frida-server not running on the device, a\n'
|
||||||
|
' host/server version mismatch, or an anti-instrumentation check.\n'
|
||||||
|
' %s' % (exc, FRIDA_INSTALL_HINT))
|
||||||
|
return 3
|
||||||
|
|
||||||
|
with open(args.script, encoding='utf-8') as fh:
|
||||||
|
source = fh.read()
|
||||||
|
|
||||||
|
def on_message(message, data):
|
||||||
|
dev, text = format_message(message)
|
||||||
|
tag = 'OTHER'
|
||||||
|
if isinstance(message.get('payload'), dict):
|
||||||
|
tag = message['payload'].get('tag') or 'SEND'
|
||||||
|
elif message.get('type') == 'error':
|
||||||
|
tag = 'FRIDA-ERROR'
|
||||||
|
counts[tag] = counts.get(tag, 0) + 1
|
||||||
|
logger.line('[host %s] [dev %s] %s' % (host_ts(), dev, text))
|
||||||
|
if message.get('type') == 'error':
|
||||||
|
desc = message.get('description') or ''
|
||||||
|
if 'Java API not available' in desc or 'Java is not defined' in desc:
|
||||||
|
logger.line('[hint] the script loaded but this runtime has no Java bridge. '
|
||||||
|
'That is a version problem, not a hook problem: align the host '
|
||||||
|
'frida package and the on-device frida-server to the same 16.x '
|
||||||
|
'version, then retry. Do not debug the hooks until READY appears.')
|
||||||
|
|
||||||
|
script = session.create_script(source)
|
||||||
|
script.on('message', on_message)
|
||||||
|
try:
|
||||||
|
script.load()
|
||||||
|
except Exception as exc:
|
||||||
|
logger.line(host_ts() + ' FATAL script load failed: %s\n'
|
||||||
|
' if this says "Java is not defined", the runtime has no Java\n'
|
||||||
|
' bridge: align host frida and on-device frida-server to 16.x.'
|
||||||
|
% exc)
|
||||||
|
return 3
|
||||||
|
|
||||||
|
if args.spawn:
|
||||||
|
device.resume(pid)
|
||||||
|
logger.line('[device] resumed pid=%d (hooks were installed before resume)' % pid)
|
||||||
|
|
||||||
|
logger.line('[host %s] attached; staying resident for %s minute(s). '
|
||||||
|
'Now drive the app UI.' % (host_ts(), args.minutes))
|
||||||
|
|
||||||
|
while time.monotonic() < deadline:
|
||||||
|
time.sleep(0.5)
|
||||||
|
|
||||||
|
except KeyboardInterrupt:
|
||||||
|
logger.line('[host %s] interrupted; detaching' % host_ts())
|
||||||
|
finally:
|
||||||
|
if script is not None:
|
||||||
|
try:
|
||||||
|
script.unload()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if session is not None:
|
||||||
|
try:
|
||||||
|
session.detach()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
if forward_added and not args.keep_forward:
|
||||||
|
try:
|
||||||
|
adb_run(adb, args.device, ['forward', '--remove', 'tcp:%d' % args.port])
|
||||||
|
logger.line('[host %s] removed adb forward tcp:%d' % (host_ts(), args.port))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
|
||||||
|
logger.line('')
|
||||||
|
logger.line('== summary: %d message(s)' % sum(counts.values()))
|
||||||
|
for tag in sorted(counts):
|
||||||
|
logger.line(' %-16s %d' % (tag, counts[tag]))
|
||||||
|
if not counts:
|
||||||
|
logger.line(' no probe output at all. Ordered checklist:')
|
||||||
|
logger.line(' 1. did READY appear? if not, the script never loaded')
|
||||||
|
logger.line(' 2. is the class/overload name you configured really present?')
|
||||||
|
logger.line(' 3. ClassLoader: the hooks bind against the app classloader')
|
||||||
|
logger.line(' 4. did the UI action actually reach the business code?')
|
||||||
|
logger.line(' 5. check THROW lines: a local validation may have returned early')
|
||||||
|
logger.line('# log written to %s' % os.path.abspath(log_name))
|
||||||
|
logger.close()
|
||||||
|
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
sys.exit(main())
|
||||||
@@ -0,0 +1,323 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
"""TLS certificate health check for one or more hosts.
|
||||||
|
|
||||||
|
WHY THIS EXISTS
|
||||||
|
---------------
|
||||||
|
When a repacked app's login/registration fails with
|
||||||
|
`Chain validation failed` / `SSLHandshakeException`, the cause is often the
|
||||||
|
*server* certificate, not your patch. This script answers that question
|
||||||
|
independently of the app, so you can rule the server side in or out **before**
|
||||||
|
touching the APK. It is step 2 of the three-step TLS triage:
|
||||||
|
1. take the real request URL / host out of the runtime probe (see frida_probe.js)
|
||||||
|
2. verify that host's certificate out-of-band <-- this script
|
||||||
|
3. compare with another host of the same app (usually one of them is fine),
|
||||||
|
which proves the failure is host-specific and not your clock/network
|
||||||
|
|
||||||
|
WHY IT STILL PRINTS THE CERTIFICATE WHEN VERIFICATION FAILS
|
||||||
|
-----------------------------------------------------------
|
||||||
|
`ssl.SSLSocket.getpeercert()` returns an empty dict whenever the handshake was
|
||||||
|
not validated, and `verification_mode=CERT_NONE` never validates. So on failure
|
||||||
|
we re-read the peer certificate in DER form and decode it locally. That is what
|
||||||
|
turns "it failed" into "it failed because notAfter is 39 days in the past".
|
||||||
|
|
||||||
|
WHAT IT REPORTS
|
||||||
|
---------------
|
||||||
|
Per host: strict verification result, protocol/cipher, subject, issuer,
|
||||||
|
notBefore, notAfter, days remaining, and a classification of the failure:
|
||||||
|
EXPIRED certificate has expired
|
||||||
|
NOT-YET-VALID certificate is not valid yet
|
||||||
|
HOSTNAME-MISMATCH chain is fine, the name does not match
|
||||||
|
UNTRUSTED-CA self-signed, or issuer not in the trust store
|
||||||
|
CHAIN-BROKEN bad signature / invalid CA / incomplete chain
|
||||||
|
REVOKED revoked by the issuer
|
||||||
|
OTHER anything else, with the raw OpenSSL message
|
||||||
|
|
||||||
|
Exit code: 0 if every host verified strictly, 2 if any host failed.
|
||||||
|
Pure standard library, Python 3.9+.
|
||||||
|
|
||||||
|
USAGE
|
||||||
|
-----
|
||||||
|
python tls_check.py api.example.com
|
||||||
|
python tls_check.py api.example.com cdn.example.com
|
||||||
|
python tls_check.py api.example.com --port 8443
|
||||||
|
python tls_check.py 203.0.113.10 --sni api.example.com # IP + real SNI name
|
||||||
|
python tls_check.py api.example.com --json # machine-readable
|
||||||
|
"""
|
||||||
|
import argparse
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import socket
|
||||||
|
import ssl
|
||||||
|
import sys
|
||||||
|
import tempfile
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
DEFAULT_PORT = 443
|
||||||
|
DEFAULT_TIMEOUT = 10
|
||||||
|
|
||||||
|
# OpenSSL X509_V_ERR_* -> (classification, human readable)
|
||||||
|
VERIFY_CODE_CLASS = {
|
||||||
|
7: ('CHAIN-BROKEN', 'certificate signature failure'),
|
||||||
|
9: ('NOT-YET-VALID', 'certificate is not yet valid'),
|
||||||
|
10: ('EXPIRED', 'certificate has expired'),
|
||||||
|
18: ('UNTRUSTED-CA', 'self-signed certificate'),
|
||||||
|
19: ('UNTRUSTED-CA', 'self-signed certificate in certificate chain'),
|
||||||
|
20: ('UNTRUSTED-CA', 'unable to get local issuer certificate'),
|
||||||
|
21: ('UNTRUSTED-CA', 'unable to verify the first certificate'),
|
||||||
|
24: ('CHAIN-BROKEN', 'invalid CA certificate'),
|
||||||
|
27: ('REVOKED', 'certificate revoked'),
|
||||||
|
62: ('HOSTNAME-MISMATCH', 'hostname mismatch'),
|
||||||
|
}
|
||||||
|
|
||||||
|
# What a failure means for the reverse-engineering workflow.
|
||||||
|
CLASS_ACTION = {
|
||||||
|
'EXPIRED': 'server-side problem: the certificate chain is not trustworthy '
|
||||||
|
'any more. Re-signing the APK cannot fix this, and a client patch '
|
||||||
|
'is the only way to keep the app usable.',
|
||||||
|
'NOT-YET-VALID': 'either the server is misconfigured or the device clock is '
|
||||||
|
'wrong. Check the device date before blaming the app.',
|
||||||
|
'HOSTNAME-MISMATCH': 'the chain is valid but the connection is using the '
|
||||||
|
'wrong name. Check for a proxy, a hosts entry, or a '
|
||||||
|
'hardcoded IP + wrong SNI.',
|
||||||
|
'UNTRUSTED-CA': 'the presented chain is not rooted in the system trust store. '
|
||||||
|
'Expected for a MITM/proxy setup; suspicious for a public API.',
|
||||||
|
'CHAIN-BROKEN': 'the server sent an incomplete or internally inconsistent '
|
||||||
|
'chain. Often a missing intermediate certificate.',
|
||||||
|
'REVOKED': 'the certificate was revoked by its issuer.',
|
||||||
|
'OTHER': 'read the raw OpenSSL message below; this is not one of the '
|
||||||
|
'well-known failure classes.',
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def fmt_name(parts):
|
||||||
|
"""Flatten ((('commonName','x'),),) into 'commonName=x'."""
|
||||||
|
if not parts:
|
||||||
|
return None
|
||||||
|
flat = []
|
||||||
|
for rdn in parts:
|
||||||
|
for key, val in rdn:
|
||||||
|
flat.append('%s=%s' % (key, val))
|
||||||
|
return ', '.join(flat)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_asn1_time(text):
|
||||||
|
"""'Nov 29 23:59:59 2026 GMT' -> aware UTC datetime (or None)."""
|
||||||
|
if not text:
|
||||||
|
return None
|
||||||
|
cleaned = text.strip()
|
||||||
|
if cleaned.endswith(' GMT') or cleaned.endswith(' UTC'):
|
||||||
|
cleaned = cleaned[:-4].rstrip()
|
||||||
|
try:
|
||||||
|
return datetime.strptime(cleaned, '%b %d %H:%M:%S %Y').replace(tzinfo=timezone.utc)
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def decode_der(der):
|
||||||
|
"""Decode a DER certificate without ever verifying it.
|
||||||
|
|
||||||
|
Uses the CPython private helper that backs `getpeercert()`; if it is not
|
||||||
|
available we still return what we can (length + fingerprint), because a
|
||||||
|
partially decoded certificate beats no certificate at all.
|
||||||
|
"""
|
||||||
|
info = {'der_bytes': len(der),
|
||||||
|
'sha256': hashlib.sha256(der).hexdigest(),
|
||||||
|
'sha1': hashlib.sha1(der).hexdigest()}
|
||||||
|
helper = getattr(getattr(ssl, '_ssl', None), '_test_decode_cert', None)
|
||||||
|
if helper is None:
|
||||||
|
return info
|
||||||
|
path = None
|
||||||
|
try:
|
||||||
|
pem = ssl.DER_cert_to_PEM_cert(der)
|
||||||
|
with tempfile.NamedTemporaryFile('w', suffix='.pem', delete=False) as fh:
|
||||||
|
path = fh.name
|
||||||
|
fh.write(pem)
|
||||||
|
info.update(helper(path))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
if path and os.path.exists(path):
|
||||||
|
try:
|
||||||
|
os.remove(path)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
return info
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_peer_cert(host, port, sni, timeout):
|
||||||
|
"""Read the peer certificate even when the chain cannot be validated."""
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
with socket.create_connection((host, port), timeout=timeout) as raw:
|
||||||
|
with ctx.wrap_socket(raw, server_hostname=sni) as tls:
|
||||||
|
der = tls.getpeercert(binary_form=True)
|
||||||
|
return decode_der(der) if der else {}
|
||||||
|
|
||||||
|
|
||||||
|
def strict_connect(host, port, sni, timeout):
|
||||||
|
"""Full strict verification: system trust store + hostname match."""
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
with socket.create_connection((host, port), timeout=timeout) as raw:
|
||||||
|
with ctx.wrap_socket(raw, server_hostname=sni) as tls:
|
||||||
|
return {
|
||||||
|
'cert': tls.getpeercert() or {},
|
||||||
|
'protocol': tls.version(),
|
||||||
|
'cipher': (tls.cipher() or ('', '', ''))[0],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def check_host(host, port, sni, timeout):
|
||||||
|
"""Return one result dict; never raises for network/TLS problems."""
|
||||||
|
res = {
|
||||||
|
'host': host,
|
||||||
|
'port': port,
|
||||||
|
'sni': sni,
|
||||||
|
'ok': False,
|
||||||
|
'classification': None,
|
||||||
|
'reason': None,
|
||||||
|
'verify_code': None,
|
||||||
|
'protocol': None,
|
||||||
|
'cipher': None,
|
||||||
|
'subject': None,
|
||||||
|
'issuer': None,
|
||||||
|
'not_before': None,
|
||||||
|
'not_after': None,
|
||||||
|
'days_remaining': None,
|
||||||
|
'fingerprint_sha256': None,
|
||||||
|
'action': None,
|
||||||
|
'error': None,
|
||||||
|
}
|
||||||
|
|
||||||
|
try:
|
||||||
|
hit = strict_connect(host, port, sni, timeout)
|
||||||
|
except ssl.SSLCertVerificationError as exc:
|
||||||
|
code = getattr(exc, 'verify_code', None)
|
||||||
|
cls, human = VERIFY_CODE_CLASS.get(code, ('OTHER', getattr(exc, 'verify_message', str(exc))))
|
||||||
|
res['classification'] = cls
|
||||||
|
res['reason'] = human
|
||||||
|
res['verify_code'] = code
|
||||||
|
res['action'] = CLASS_ACTION.get(cls)
|
||||||
|
res['error'] = '%s: %s' % (type(exc).__name__, exc)
|
||||||
|
except (ssl.SSLError, socket.error, OSError) as exc:
|
||||||
|
res['classification'] = 'OTHER'
|
||||||
|
res['reason'] = 'connection or handshake failed before certificate validation'
|
||||||
|
res['action'] = CLASS_ACTION['OTHER']
|
||||||
|
res['error'] = '%s: %s' % (type(exc).__name__, exc)
|
||||||
|
return res
|
||||||
|
else:
|
||||||
|
cert = hit['cert']
|
||||||
|
res.update({'ok': True, 'protocol': hit['protocol'], 'cipher': hit['cipher'],
|
||||||
|
'subject': fmt_name(cert.get('subject')),
|
||||||
|
'issuer': fmt_name(cert.get('issuer')),
|
||||||
|
'not_before': cert.get('notBefore'),
|
||||||
|
'not_after': cert.get('notAfter')})
|
||||||
|
res['days_remaining'] = days_left(res['not_after'])
|
||||||
|
return res
|
||||||
|
|
||||||
|
# Verification failed: still try to show the real certificate.
|
||||||
|
try:
|
||||||
|
cert = fetch_peer_cert(host, port, sni, timeout)
|
||||||
|
except Exception as exc:
|
||||||
|
res['error'] = (res['error'] or '') + ' | peek failed: %s' % exc
|
||||||
|
return res
|
||||||
|
|
||||||
|
res.update({'subject': fmt_name(cert.get('subject')),
|
||||||
|
'issuer': fmt_name(cert.get('issuer')),
|
||||||
|
'not_before': cert.get('notBefore'),
|
||||||
|
'not_after': cert.get('notAfter'),
|
||||||
|
'fingerprint_sha256': cert.get('sha256')})
|
||||||
|
res['days_remaining'] = days_left(res['not_after'])
|
||||||
|
return res
|
||||||
|
|
||||||
|
|
||||||
|
def days_left(not_after):
|
||||||
|
dt = parse_asn1_time(not_after)
|
||||||
|
if dt is None:
|
||||||
|
return None
|
||||||
|
return (dt - datetime.now(timezone.utc)).days
|
||||||
|
|
||||||
|
|
||||||
|
def report(res, out=sys.stdout):
|
||||||
|
w = out.write
|
||||||
|
w('== %s:%d%s\n' % (res['host'], res['port'],
|
||||||
|
'' if res['sni'] == res['host'] else ' (SNI: %s)' % res['sni']))
|
||||||
|
if res['ok']:
|
||||||
|
w(' status : OK (strict verification passed)\n')
|
||||||
|
else:
|
||||||
|
w(' status : FAIL - %s\n' % res['classification'])
|
||||||
|
w(' reason : %s\n' % res['reason'])
|
||||||
|
if res['verify_code'] is not None:
|
||||||
|
w(' verify_code : %d\n' % res['verify_code'])
|
||||||
|
if res['protocol']:
|
||||||
|
w(' protocol : %s cipher: %s\n' % (res['protocol'], res['cipher']))
|
||||||
|
w(' subject : %s\n' % (res['subject'] or '<undecoded>'))
|
||||||
|
w(' issuer : %s\n' % (res['issuer'] or '<undecoded>'))
|
||||||
|
w(' notBefore : %s\n' % (res['not_before'] or '<unknown>'))
|
||||||
|
w(' notAfter : %s\n' % (res['not_after'] or '<unknown>'))
|
||||||
|
days = res['days_remaining']
|
||||||
|
if days is not None:
|
||||||
|
if days < 0:
|
||||||
|
w(' validity : EXPIRED %d days ago\n' % (-days))
|
||||||
|
else:
|
||||||
|
w(' days remaining : %d\n' % days)
|
||||||
|
if res['fingerprint_sha256']:
|
||||||
|
w(' sha256 : %s\n' % res['fingerprint_sha256'])
|
||||||
|
if res['action'] and not res['ok']:
|
||||||
|
w(' action : %s\n' % res['action'])
|
||||||
|
if res['error']:
|
||||||
|
w(' raw error : %s\n' % res['error'])
|
||||||
|
w('\n')
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
ap = argparse.ArgumentParser(
|
||||||
|
description='Verify TLS certificates for one or more hosts, strictly, '
|
||||||
|
'and classify any failure (expired / hostname mismatch / untrusted CA).')
|
||||||
|
ap.add_argument('hosts', nargs='+', metavar='HOST',
|
||||||
|
help='hostname or IP to check, e.g. api.example.com')
|
||||||
|
ap.add_argument('--port', type=int, default=DEFAULT_PORT,
|
||||||
|
help='TCP port (default %d)' % DEFAULT_PORT)
|
||||||
|
ap.add_argument('--sni', default=None,
|
||||||
|
help='override the SNI/hostname to validate against; useful when '
|
||||||
|
'HOST is a bare IP, e.g. --sni api.example.com')
|
||||||
|
ap.add_argument('--timeout', type=float, default=DEFAULT_TIMEOUT,
|
||||||
|
help='connect timeout in seconds (default %d)' % DEFAULT_TIMEOUT)
|
||||||
|
ap.add_argument('--json', action='store_true', help='emit JSON instead of a report')
|
||||||
|
args = ap.parse_args()
|
||||||
|
|
||||||
|
results = []
|
||||||
|
for host in args.hosts:
|
||||||
|
sni = args.sni or host
|
||||||
|
results.append(check_host(host, args.port, sni, args.timeout))
|
||||||
|
|
||||||
|
if args.json:
|
||||||
|
json.dump(results, sys.stdout, indent=2, sort_keys=True)
|
||||||
|
sys.stdout.write('\n')
|
||||||
|
else:
|
||||||
|
for res in results:
|
||||||
|
report(res)
|
||||||
|
|
||||||
|
bad = [r for r in results if not r['ok']]
|
||||||
|
good = [r for r in results if r['ok']]
|
||||||
|
if len(results) > 1:
|
||||||
|
if bad and good:
|
||||||
|
print('SUMMARY: %d/%d hosts failed. At least one host on this app '
|
||||||
|
'verifies fine, so the failure is host-specific (server side), '
|
||||||
|
'not a clock/network problem on your side.'
|
||||||
|
% (len(bad), len(results)))
|
||||||
|
elif bad:
|
||||||
|
print('SUMMARY: every host failed. Suspect the network path, a proxy, '
|
||||||
|
'or the device/host clock before blaming the app.')
|
||||||
|
else:
|
||||||
|
print('SUMMARY: all %d hosts verified strictly.' % len(results))
|
||||||
|
for r in bad:
|
||||||
|
print('FAIL %s -> %s' % (r['host'], r['classification']))
|
||||||
|
|
||||||
|
return 0 if all(r['ok'] for r in results) else 2
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
sys.exit(main())
|
||||||
Reference in New Issue
Block a user