Add deliverable-form discipline, execution identity, and four reusable scripts

Generalized from a long hardened-target engagement. Additions target the failure
modes that cost the most time and are not yet covered; nothing here is sample-specific.

New discipline
- SKILL.md: a third override rule -- deliver in the environment the request requires.
  A root-only or live-instrumentation result can look like success while failing the
  actual requirement (unrooted, installable, self-contained). Also adds two classify
  questions: what form the deliverable must take, and which architecture/library is
  actually executing.
- long-task-discipline.md: "the most expensive drift" -- a constraint table (privilege,
  modification form, ABI, network, persistence, distribution), the checkpoint question,
  and how to report a privileged workaround as a fallback rather than a deliverable.
  Plus "bound every wait": timeouts on every command, budgets calibrated from
  measurement so slow and hung are distinguishable, and sampling an observable you can
  inspect instead of sleeping blind.
- environment.md: look at the screen instead of driving blind; which of the two screen
  evidence kinds is trustworthy (an empty accessibility tree is common for
  runtime-rendered UI, where the image is primary rather than a fallback); emulator
  console control and adb recovery; preflight before every experiment block.
- native-and-so.md: what the device claims vs what the package manager chose vs what is
  actually mapped; translation layers and their consequences for hooks and native
  checks; fat-APK ABI selection.
- runtime-data.md: when the app rewrites your edit (server refresh vs local
  regeneration), why chmod/chown cannot work because the app deletes and recreates the
  file, and the immutable-attribute fix; decoding a value that only looks encrypted.
- framework-runtimes.md: string encoding traps (a UTF-8 scan of a UTF-16 snapshot
  returns nothing and reads as "the strings were stripped"), and non-ASCII obfuscated
  identifiers -- resolve by shape, not by name.
- pitfalls.md: P23 the fix lives in a file the app rewrites; P24 the artifact changed
  but the wrong one is executing; P25 "search found nothing" treated as "not there".

New scripts (parameterized, path-agnostic, all with explicit timeouts)
- preflight.py  environment check: device, root, ABI/translation, clock skew, leftover
                proxy/forwards, dead device server, chosen ABI, free space
- lib_map.py    what is actually mapped into a live process: path, base, architecture
                from the ELF header, and whether it came from the APK or was
                materialized at runtime
- blob_decode.py search (don't guess) the framing of a stored value, then re-encode the
                edited payload with the same parameters
- snap.py       bounded burst screenshots + control-tree capture, with a stall detector
                and an explicit verdict on whether the tree is usable

All four were exercised against a live device while writing: preflight correctly
flags multi-device ambiguity and detected the ARM translation layer; lib_map read real
ELF headers and identified the translation shim; blob_decode round-tripped a
rotation+deflate+base64 blob at realistic size in ~0.2s; snap detected a stalled screen
across three hashes.
This commit is contained in:
inliver233
2026-09-19 22:08:01 +08:00
parent 55432780fc
commit 0ea42265b7
12 changed files with 1537 additions and 34 deletions
+276
View File
@@ -0,0 +1,276 @@
#!/usr/bin/env python3
"""Decode (and re-encode) an opaque single-value blob cached by an app.
Many apps cache server-issued configuration as ONE opaque string inside a
preferences file or a key-value store. The encoding is usually some stack of:
outer: base64, base64url, or hex
transform: a cyclic rotation of the byte stream (so the compressed stream
does not begin at its header), sometimes preceded by a short
header to skip
inner: raw deflate, zlib, or gzip
You do not have to guess the parameters. This script searches them and reports
every combination that decodes to something structured, then can re-encode your
edited payload with the same parameters so it can be written back.
Why rotation matters: a plain base64->inflate attempt fails on these blobs, and
the failure looks like "this value is encrypted". Exhaustively trying cyclic
shifts is cheap and frequently succeeds, which turns an opaque blob into an
editable JSON/XML document.
Usage:
# decode the value of a key inside a prefs XML
python blob_decode.py --prefs-xml prefs.xml --name flutter.remoteConfig
# decode a blob stored in its own file (or piped on stdin)
python blob_decode.py --file value.txt
cat value.txt | python blob_decode.py
# keep searching harder (slower) and write the best result out
python blob_decode.py --file value.txt --max-skip 64 --out decoded.bin
# re-encode an edited payload with the parameters that worked
python blob_decode.py --encode --file decoded.bin --outer base64 --inner raw --cut 15435 --out value.txt
Notes:
* Output is written to --out; nothing on device is touched.
* The winning parameters are printed as a copy-pasteable line for --encode.
"""
from __future__ import annotations
import argparse
import base64
import binascii
import json
import re
import sys
import time
import zlib
INNERS = [("raw", -15), ("zlib", 15), ("gzip", 31)]
# --------------------------------------------------------------------------- #
# outer encodings
# --------------------------------------------------------------------------- #
def outer_decode(text: str, which: str) -> bytes | None:
t = text.strip().replace("\n", "").replace("\r", "").replace(" ", "")
try:
if which == "base64":
return base64.b64decode(t + "=" * (-len(t) % 4))
if which == "base64url":
return base64.urlsafe_b64decode(t + "=" * (-len(t) % 4))
if which == "hex":
return binascii.unhexlify(re.sub(r"[^0-9a-fA-F]", "", t))
except Exception:
return None
return None
def outer_encode(data: bytes, which: str) -> str:
if which == "base64":
return base64.b64encode(data).decode("ascii")
if which == "base64url":
return base64.urlsafe_b64encode(data).decode("ascii")
if which == "hex":
return binascii.hexlify(data).decode("ascii")
raise ValueError("unknown outer encoding: %s" % which)
def sniff_outer(text: str) -> list[str]:
"""Cheap guess first so the common case is instant."""
t = text.strip()
guesses = []
if re.fullmatch(r"[0-9a-fA-F\s]+", t) and len(t) >= 16:
guesses.append("hex")
if re.fullmatch(r"[A-Za-z0-9+/\s=]+", t):
guesses.append("base64")
if re.fullmatch(r"[A-Za-z0-9\-_\s=]+", t):
guesses.append("base64url")
# always keep base64 last-resort, then everything else
for w in ("base64", "base64url", "hex"):
if w not in guesses:
guesses.append(w)
return list(dict.fromkeys(guesses))
# --------------------------------------------------------------------------- #
# inner transforms
# --------------------------------------------------------------------------- #
def try_inflate(body: bytes, max_out: int | None = None) -> tuple[str, bytes] | None:
best = None
for name, wbits in INNERS:
try:
if max_out is None:
out = zlib.decompress(body, wbits)
else:
d = zlib.decompressobj(wbits)
out = d.decompress(body, max_out)
if out:
if best is None or len(out) > len(best[1]):
best = (name, out)
except Exception:
continue
return best
def inflate_raw(data: bytes) -> bytes:
co = zlib.compressobj(9, zlib.DEFLATED, -15)
return co.compress(data) + co.flush()
def score(out: bytes) -> float:
"""How much does this look like a real document we can edit?"""
if not out:
return -1.0
printable = sum(1 for b in out[:4096] if 32 <= b < 127 or b in (9, 10, 13))
ratio = printable / min(len(out), 4096)
s = ratio * 10.0
head = out[:64].lstrip()
if head[:1] in (b"{", b"[", b"<", b"#"):
s += 6.0
if b'"' in out[:512] or b"=" in out[:512]:
s += 2.0
if len(out) < 32:
s -= 3.0
return s
# --------------------------------------------------------------------------- #
def search(blob: bytes, max_skip: int, log) -> list[dict]:
hits: list[dict] = []
n = len(blob)
t0 = time.time()
for skip in range(0, min(max_skip, n) + 1):
base = blob[skip:]
m = len(base)
if m < 8:
break
for k in range(m):
cand = base[k:] + base[:k]
got = try_inflate(cand, max_out=8_000_000)
if got:
name, out = got
hits.append({
"skip": skip, "cut": k, "inner": name, "size": len(out),
"score": round(score(out), 2), "data": out,
})
if hits and log and skip == 0:
log(" first hit found with skip=0 after %.1fs" % (time.time() - t0))
if log and skip and skip % 16 == 0 and time.time() - t0 > 30:
log(" ... skip=%d, %.0fs elapsed" % (skip, time.time() - t0))
log(" search finished in %.1fs (%d candidate(s))" % (time.time() - t0, len(hits)))
return hits
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
src = ap.add_mutually_exclusive_group()
src.add_argument("--file", help="file containing the encoded value")
src.add_argument("--prefs-xml", help="a preferences XML file to pull the value out of")
ap.add_argument("--name", help="key name when using --prefs-xml")
ap.add_argument("--outer", choices=["base64", "base64url", "hex", "auto"], default="auto")
ap.add_argument("--max-skip", type=int, default=0,
help="also try dropping 0..N leading bytes before the rotation search (slower)")
ap.add_argument("--min-size", type=int, default=200, help="ignore results smaller than this")
ap.add_argument("--top", type=int, default=5, help="how many candidates to print")
ap.add_argument("--out", help="write the best decoded result here")
ap.add_argument("--encode", action="store_true", help="re-encode instead of decoding")
ap.add_argument("--inner", choices=[n for n, _ in INNERS], default="raw")
ap.add_argument("--cut", type=int, default=0, help="rotation used for write-back")
a = ap.parse_args()
def log(msg: str) -> None:
print(msg, file=sys.stderr)
# ------------------------------------------------------------------ encode
if a.encode:
if not a.file:
log("--encode requires --file (the payload to encode)")
return 2
payload = open(a.file, "rb").read()
body = inflate_raw(payload) if a.inner == "raw" else zlib.compress(payload, 9)
if a.inner == "zlib":
body = zlib.compress(payload, 9)
cut = a.cut % max(len(body), 1)
rotated = body[-cut:] + body[:-cut] if cut else body
enc = outer_encode(rotated, "base64" if a.outer == "auto" else a.outer)
if a.out:
open(a.out, "w", encoding="utf-8", newline="\n").write(enc)
log("wrote %d chars to %s" % (len(enc), a.out))
else:
print(enc)
log("write-back parameters: outer=%s inner=%s cut=%d"
% ("base64" if a.outer == "auto" else a.outer, a.inner, cut))
return 0
# ------------------------------------------------------------------ input
if a.prefs_xml:
if not a.name:
log("--prefs-xml requires --name <key>")
return 2
xml = open(a.prefs_xml, encoding="utf-8", errors="replace").read()
m = re.search(r'name="%s"[^>]*>([^<]*)<' % re.escape(a.name), xml)
if not m:
log("key not found in XML: %s" % a.name)
log("keys present: %s" % ", ".join(re.findall(r'name="([^"]+)"', xml)[:40]))
return 2
text = m.group(1)
log("value length: %d" % len(text))
elif a.file:
text = open(a.file, encoding="utf-8", errors="replace").read()
else:
text = sys.stdin.read()
if not text.strip():
log("empty input")
return 2
# ------------------------------------------------------------------ decode
candidates = [a.outer] if a.outer != "auto" else sniff_outer(text)
for which in candidates:
blob = outer_decode(text, which)
if blob is None or len(blob) < 8:
log("outer=%s -> not decodable, skipping" % which)
continue
log("outer=%s -> %d bytes; searching rotation (and skip<=%d)" % (which, len(blob), a.max_skip))
hits = [h for h in search(blob, a.max_skip, log) if h["size"] >= a.min_size]
if not hits:
continue
hits.sort(key=lambda h: (-h["score"], -h["size"]))
print("=" * 66)
print("outer=%s candidates=%d" % (which, len(hits)))
for h in hits[:a.top]:
print(" score=%-6s skip=%-4d cut=%-7d inner=%-5s size=%d"
% (h["score"], h["skip"], h["cut"], h["inner"], h["size"]))
best = hits[0]
print("-" * 66)
print("write-back line:")
print(" python blob_decode.py --encode --file <edited_payload> --outer %s --inner %s --cut %d"
% (which, best["inner"], best["cut"]))
print("-" * 66)
preview = best["data"][:1200]
try:
print(preview.decode("utf-8"))
except UnicodeDecodeError:
print(preview.decode("latin1"))
print("(non-UTF-8; first 64 bytes: %s)" % best["data"][:64].hex())
if a.out:
open(a.out, "wb").write(best["data"])
log("wrote %d bytes to %s" % (len(best["data"]), a.out))
return 0
print("no decodable candidate found.", file=sys.stderr)
print("Next steps:", file=sys.stderr)
print(" * raise --max-skip, or pass --outer explicitly", file=sys.stderr)
print(" * the blob may be genuinely encrypted (a keyed cipher) rather than merely", file=sys.stderr)
print(" encoded -- confirm before spending time on it: a keyed blob should look", file=sys.stderr)
print(" random, whereas an encoded one still has structure once the framing is", file=sys.stderr)
print(" removed. See references/runtime-data.md.", file=sys.stderr)
return 1
if __name__ == "__main__":
sys.exit(main())
+218
View File
@@ -0,0 +1,218 @@
#!/usr/bin/env python3
"""Report which native libraries are ACTUALLY mapped into a live process.
`getprop ro.product.cpu.abi` tells you what the device claims to be. It does not
tell you what is executing. Those differ more often than people expect:
* an x86_64 emulator running an app whose only native libraries are arm64,
because an ARM translation layer is present;
* a hardened app whose real libraries are materialized into its data
directory at runtime, so the paths you see are not the paths inside the APK;
* a fat APK where the package manager picked a different ABI than you assumed,
so you patch a `.so` that never loads.
This script reads the live mapping table and reports, per library: the path, the
base address, and the architecture of the actual ELF on disk. It classifies each
path so the interesting ones stand out (system / from-APK / runtime-materialized).
Requires root to read another process's maps.
Usage:
python lib_map.py --pkg com.example.app
python lib_map.py --pkg com.example.app --app-only
python lib_map.py --pid 1234 --all
python lib_map.py --pkg com.example.app --json
python lib_map.py --pkg com.example.app --grep ssl
Reading it:
* A library whose path sits under the app's data dir (not under the APK's
lib/<abi> dir) was written at runtime. That is a packer/dropper fingerprint
and it will not be present in a repacked APK -- patch or replace with care.
* If app libraries are 64-bit ARM but the device primary ABI is x86_64,
a translator is in play. Timing and some native checks will differ.
* If the library you patched does not appear here at all, your patch cannot
matter. That is a plan problem, not a patch problem.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import subprocess
import sys
from collections import defaultdict
E_MACHINE = {
0x03: "x86", 0x3E: "x86_64", 0x28: "arm", 0xB7: "aarch64",
0x08: "mips", 0xF3: "riscv", 0x16: "s390", 0x15: "ppc",
}
TRANSLATOR_MARKERS = ("houdini", "libndk", "native_bridge", "libntv", "armtrans",
"libhoudini", "ndk_translation", "libarm", "exagear", "box64")
def run(cmd: list[str], timeout: int = 60) -> str:
try:
p = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=timeout)
return (p.stdout or "") + (p.stderr or "")
except Exception as e:
return "ERROR: %s" % e
def classify(path: str, pkg: str | None) -> str:
if path.startswith(("/system/", "/apex/", "/vendor/", "/product/", "/odm/", "/system_ext/")):
return "system"
if re.search(r"/data/app/", path):
return "apk-lib"
if pkg and (("/data/data/%s" % pkg) in path or ("/data/user/0/%s" % pkg) in path):
return "materialized"
if "/data/" in path:
return "data-other"
return "other"
def read_elf_arch(sh_fn, path: str) -> str:
"""Read the ELF header on the device and name the machine. '?' means the read failed."""
hdr = sh_fn("dd if='%s' bs=20 count=1 2>/dev/null | od -An -tx1 2>/dev/null" % path)
hexs = re.sub(r"[^0-9a-fA-F]", "", hdr)
if len(hexs) < 40:
return "?"
b = bytes.fromhex(hexs[:40])
if b[:4] != b"\x7fELF":
return "not-elf"
cls = {1: 32, 2: 64}.get(b[4], 0)
little = b[5] == 1
mach = int.from_bytes(b[18:20], "little" if little else "big")
if mach in E_MACHINE:
return E_MACHINE[mach] # the machine name already implies the width
return "0x%02x/%db" % (mach, cls or 0)
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
tgt = ap.add_mutually_exclusive_group(required=True)
tgt.add_argument("--pkg", help="target package name")
tgt.add_argument("--pid", type=int, help="target pid")
ap.add_argument("--adb", default=os.environ.get("ADB", "adb"))
ap.add_argument("--serial", default=None)
ap.add_argument("--all", action="store_true", help="include system libraries")
ap.add_argument("--app-only", action="store_true", help="only APK libs and runtime-materialized libs")
ap.add_argument("--grep", default=None, help="only paths matching this regex")
ap.add_argument("--no-arch", action="store_true", help="skip reading ELF headers (faster)")
ap.add_argument("--max-libs", type=int, default=120, help="cap how many libraries get an arch probe")
ap.add_argument("--json", action="store_true", dest="as_json")
a = ap.parse_args()
def sh(cmd: str, timeout: int = 60) -> str:
base = [a.adb] + (["-s", a.serial] if a.serial else []) + ["shell", cmd]
return run(base, timeout)
def su(cmd: str, timeout: int = 90) -> str:
base = [a.adb] + (["-s", a.serial] if a.serial else []) + ["shell", 'su -c "%s"' % cmd]
return run(base, timeout)
pid = a.pid
if a.pkg and not pid:
out = sh("pidof %s" % a.pkg).strip()
if not out:
print("no running process for %s -- launch it and let it settle first" % a.pkg, file=sys.stderr)
return 2
pid = int(out.split()[0])
maps = su("cat /proc/%d/maps" % pid)
if not maps.strip() or "No such file" in maps:
print("cannot read /proc/%d/maps (root required, or the process is gone)" % pid, file=sys.stderr)
return 2
entries: dict[str, dict] = {}
for line in maps.splitlines():
m = re.match(r"^([0-9a-f]+)-([0-9a-f]+)\s+(\S+)\s+([0-9a-f]+)\s+\S+\s+\S+\s*(.*)$", line)
if not m:
continue
start, end, perms, off, path = m.groups()
path = path.strip()
if not path or path.startswith("["):
continue
if not path.endswith(".so") and ".so." not in path:
continue
e = entries.setdefault(path, {"path": path, "perms": set(), "base": start, "count": 0})
e["perms"].add(perms)
e["count"] += 1
if perms.startswith("r-x") or perms.startswith("r--"):
e["base"] = min(e["base"], start)
dev_abi = sh("getprop ro.product.cpu.abi").strip()
abilist = sh("getprop ro.product.cpu.abilist").strip()
pkg = a.pkg
if not pkg:
cmdline = su("cat /proc/%d/cmdline" % pid).split("\x00")[0].strip()
pkg = cmdline or None
rows = []
probed = 0
for path, e in sorted(entries.items()):
kind = classify(path, pkg)
if a.app_only and kind not in ("apk-lib", "materialized"):
continue
# --grep is an explicit narrow request, so honour it across all kinds
if not a.all and not a.app_only and not a.grep and kind == "system":
continue
if a.grep and not re.search(a.grep, path):
continue
rows.append({"path": path, "base": e["base"], "kind": kind,
"perms": "/".join(sorted(e["perms"])), "maps": e["count"], "arch": "?"})
if not a.no_arch:
for r in rows[:a.max_libs]:
r["arch"] = read_elf_arch(su, r["path"])
probed += 1
# translation markers
markers = sorted({p for p in entries if any(t in p.lower() for t in TRANSLATOR_MARKERS)})
app_archs = sorted({r["arch"] for r in rows if r["kind"] in ("apk-lib", "materialized") and r["arch"] not in ("?", "not-elf")})
host_is_x86 = "x86" in dev_abi
translated = bool(markers) or (host_is_x86 and any(x.startswith(("arm", "aarch64")) for x in app_archs))
if a.as_json:
print(json.dumps({"pid": pid, "pkg": pkg, "device_abi": dev_abi, "abilist": abilist,
"translation_suspected": translated, "translator_markers": markers,
"libs": rows}, indent=2, ensure_ascii=False))
return 0
print("pid=%s pkg=%s" % (pid, pkg or "?"))
print("device primary abi=%s abilist=[%s]" % (dev_abi, abilist))
print("%-13s %-9s %-6s %-6s %s" % ("arch", "kind", "perms", "maps", "path"))
print("-" * 100)
for r in rows:
print("%-13s %-9s %-6s %-6d %s" % (r["arch"], r["kind"], r["perms"], r["maps"], r["path"]))
print("-" * 100)
print("mapped libraries shown: %d" % len(rows))
if not a.no_arch and probed < len(rows):
print("(architecture probed for the first %d; raise --max-libs to cover the rest -- '?' means not probed)"
% probed)
if markers:
print("\nTRANSLATOR MARKERS PRESENT:")
for mk in markers:
print(" " + mk)
print(" -> native ARM code is being translated. Timing differs from a real ARM device,")
print(" and native integrity checks may behave differently. Verify the final artifact")
print(" on the ABI the user actually runs.")
if app_archs:
print("\napp library architectures: %s" % ", ".join(app_archs))
if translated:
print("translation suspected: YES")
mat = [r["path"] for r in rows if r["kind"] == "materialized"]
if mat:
print("\nRUNTIME-MATERIALIZED libraries (not inside the APK):")
for p in mat[:20]:
print(" " + p)
print(" -> written at runtime by the app or its shell. A repacked APK will not contain")
print(" them in this location; account for that before assuming your patch is loaded.")
if not rows:
print("\nnothing matched. Widen with --all, or check that the process is the one that draws the UI.")
return 0
if __name__ == "__main__":
sys.exit(main())
+250
View File
@@ -0,0 +1,250 @@
#!/usr/bin/env python3
"""Preflight: prove the environment is sane BEFORE you blame your patch.
Run this at the start of every experiment block, and again whenever something
fails in a way you did not expect. A large share of "my patch broke the app"
turns out to be device state, a dead device server, a leftover proxy setting, or
a clock skew -- all of which look exactly like a broken artifact.
It is deliberately read-only: it changes nothing except optional cleanup of a
device-wide proxy setting and port forwards it created itself.
Usage:
python preflight.py
python preflight.py --serial <serial> --pkg com.example.app
python preflight.py --pkg com.example.app --expect-root
python preflight.py --cleanup # remove proxy + frida port forwards it finds
python preflight.py --json # machine-readable summary
Exit code is 0 when no BLOCKER was found, 1 otherwise. WARN never fails the run.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import shutil
import subprocess
import sys
BLOCKER, WARN, OK, INFO = "BLOCKER", "WARN", "OK", "INFO"
class Report:
def __init__(self) -> None:
self.rows: list[tuple[str, str, str, str]] = []
def add(self, level: str, area: str, detail: str, fix: str = "") -> None:
self.rows.append((level, area, detail, fix))
def worst(self) -> str:
for lv in (BLOCKER, WARN, OK, INFO):
if any(r[0] == lv for r in self.rows):
return lv
return OK
def run(cmd: list[str], timeout: int = 60) -> tuple[int, str]:
try:
p = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=timeout)
return p.returncode, (p.stdout or "") + (p.stderr or "")
except FileNotFoundError:
return 127, "not found: %s" % cmd[0]
except subprocess.TimeoutExpired:
return 124, "TIMEOUT after %ss: %s" % (timeout, " ".join(cmd[:4]))
except Exception as e: # pragma: no cover - defensive
return 1, "error: %s" % e
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--adb", default=os.environ.get("ADB", "adb"), help="adb executable (default: $ADB or 'adb')")
ap.add_argument("--serial", default=None, help="device serial; required when more than one is attached")
ap.add_argument("--pkg", default=None, help="target package to check (optional)")
ap.add_argument("--expect-root", action="store_true", help="treat missing root as a BLOCKER")
ap.add_argument("--cleanup", action="store_true", help="remove leftover proxy setting and frida forwards")
ap.add_argument("--json", action="store_true", dest="as_json")
a = ap.parse_args()
r = Report()
ADB = a.adb
# ---- host tools -------------------------------------------------------
for tool, why in (
("java", "dexlib2 patcher, apksigner"),
("python", "all scripts"),
):
found = shutil.which(tool)
r.add(OK if found else WARN, "host:" + tool, found or "not on PATH", "" if found else "install it if you need " + why)
for tool in ("apksigner", "zipalign", "aapt", "aapt2"):
found = shutil.which(tool)
r.add(OK if found else INFO, "host:" + tool, found or "not on PATH",
"" if found else "only needed for signing/manifest work; build-tools on PATH helps")
# ---- adb + device selection ------------------------------------------
if shutil.which(ADB) is None and not os.path.exists(ADB):
r.add(BLOCKER, "adb", "executable not found: %s" % ADB, "pass --adb <path> or set $ADB")
return finish(r, a.as_json)
rc, out = run([ADB, "devices", "-l"])
if rc != 0:
r.add(BLOCKER, "adb", "adb devices failed: %s" % out.strip()[:160], "start the adb server; check USB/emulator")
return finish(r, a.as_json)
devices = []
for line in out.splitlines()[1:]:
line = line.strip()
if not line:
continue
parts = line.split()
if len(parts) >= 2:
devices.append((parts[0], parts[1], line))
online = [d for d in devices if d[1] == "device"]
if not devices:
r.add(BLOCKER, "device", "no device attached", "boot the emulator/connect the device, then re-run")
return finish(r, a.as_json)
if len(online) > 1 and not a.serial:
r.add(BLOCKER, "device", "%d devices online but no --serial given" % len(online),
"pass --serial explicitly, or adb will pick one at random and you will debug the wrong target")
if not online:
bad = ", ".join("%s(%s)" % (d[0], d[1]) for d in devices)
r.add(BLOCKER, "device", "attached but not usable: %s" % bad,
"unauthorized -> accept the USB debugging prompt; offline -> reconnect/reboot the emulator")
return finish(r, a.as_json)
serial = a.serial or online[0][0]
r.add(OK, "device", "using %s (of %d online)" % (serial, len(online)), "")
def sh(cmd: str, timeout: int = 60) -> str:
return run([ADB, "-s", serial, "shell", cmd], timeout=timeout)[1]
def su(cmd: str, timeout: int = 90) -> str:
return run([ADB, "-s", serial, "shell", 'su -c "%s"' % cmd], timeout=timeout)[1]
# ---- device facts -----------------------------------------------------
abi = sh("getprop ro.product.cpu.abi").strip()
abilist = sh("getprop ro.product.cpu.abilist").strip()
rel = sh("getprop ro.build.version.release").strip()
sdk = sh("getprop ro.build.version.sdk").strip()
model = sh("getprop ro.product.model").strip()
sec = sh("getenforce").strip()
r.add(OK, "device:os", "Android %s (sdk %s) %s" % (rel, sdk, model), "")
r.add(OK, "device:abi", "%s abilist=[%s]" % (abi, abilist), "")
r.add(OK if sec.lower() in ("permissive", "") else INFO, "device:selinux", sec or "?", "")
# translation layer / emulator hints -- these change how native code behaves
trans = []
for probe in ("ro.dalvik.vm.native.bridge", "ro.enable.native.bridge.exec", "ro.boot.native_bridge"):
v = sh("getprop " + probe).strip()
if v and v != "0":
trans.append("%s=%s" % (probe, v))
r.add(WARN if trans else OK, "device:translation",
"; ".join(trans) if trans else "no ARM translation declared",
"native arm libraries run through a translator here: timing differs and some native checks misbehave" if trans else "")
if re.search(r"^1$", sh("getprop ro.kernel.qemu").strip()):
r.add(INFO, "device:type", "emulator (ro.kernel.qemu=1)",
"final verification belongs on the real ABI the user will run")
# ---- root -------------------------------------------------------------
idout = su("id")
has_root = "uid=0" in idout
lvl = OK if has_root else (BLOCKER if a.expect_root else WARN)
r.add(lvl, "root", "available" if has_root else "NOT available (%s)" % idout.strip()[:60],
"" if has_root else "anything beyond static analysis needs root: data dirs, Frida, file locks")
# ---- clock (silently breaks TLS work) --------------------------------
dev_epoch = sh("date +%s").strip()
if dev_epoch.isdigit():
drift = abs(int(dev_epoch) - int(__import__("time").time()))
r.add(OK if drift < 120 else WARN, "clock",
"device/host drift %ds" % drift,
"" if drift < 120 else "an expired/wrong certificate can be nothing but this clock; fix before TLS triage")
# ---- leftover state that fakes a failure -----------------------------
proxy = sh("settings get global http_proxy").strip()
if proxy and proxy not in ("null", ":0"):
r.add(WARN, "state:proxy", "device-wide http_proxy=%s" % proxy,
"a leftover proxy makes every request fail; clear with: settings put global http_proxy :0")
if a.cleanup:
sh("settings put global http_proxy :0")
r.add(INFO, "state:proxy", "cleared (--cleanup)", "")
else:
r.add(OK, "state:proxy", "not set", "")
rc_f, fwd = run([ADB, "-s", serial, "forward", "--list"])
if fwd.strip():
r.add(INFO, "state:forwards", fwd.strip().replace("\n", " | ")[:200],
"stale forwards point frida at the wrong place; 'adb forward --remove-all' resets them")
if a.cleanup:
run([ADB, "-s", serial, "forward", "--remove-all"])
r.add(INFO, "state:forwards", "removed (--cleanup)", "")
# ---- frida server reachability ---------------------------------------
srv = su("pgrep -f frida-server || pgrep -f kwork")
if srv.strip().isdigit() or srv.strip().split("\n")[0].strip().isdigit():
r.add(OK, "frida:server", "device server process alive (pid %s)" % srv.strip().split("\n")[0].strip(), "")
else:
r.add(INFO, "frida:server", "no device server process found",
"only needed for dynamic work; see references/dynamic-frida.md (name it out of obvious paths)")
# ---- target package ---------------------------------------------------
if a.pkg:
pm = sh("pm list packages | grep -F %s" % a.pkg)
installed = ("package:" + a.pkg) in pm or a.pkg in pm
r.add(OK if installed else INFO, "target", "installed" if installed else "not installed", "")
if installed:
path = sh("pm path %s" % a.pkg).strip()
uid = su("dumpsys package %s | grep -m1 userId=" % a.pkg).strip()
r.add(INFO, "target:apk", path[:160] or "?", "")
r.add(INFO, "target:uid", uid or "?", "re-check after every reinstall: uid increments")
# the native library dir the package manager will use for this device
nat = sh("dumpsys package %s | grep -m3 primaryCpuAbi" % a.pkg).strip()
if nat:
r.add(INFO, "target:abi", nat.replace("\n", " | ")[:160],
"this is the ABI the package manager chose for THIS device")
maps = su("cat /proc/$(pidof %s | awk '{print $1}')/maps 2>/dev/null | head -1" % a.pkg)
if maps.strip():
r.add(INFO, "target:running", "process is running", "prefer a clean cold start before measuring anything")
else:
r.add(OK, "target:running", "not running", "")
# ---- free space -------------------------------------------------------
# Report by device node, not by the mount-point column: a block device can be
# mounted at more than one path (an ARM translation shim does exactly this on
# some emulators), and `df` will then label /data with an unrelated path.
space = ""
for line in sh("df -h /data 2>/dev/null").splitlines()[1:]:
cols = line.split()
if len(cols) >= 5:
space = "%s: %s available of %s (%s used)" % (cols[0], cols[3], cols[1], cols[4])
break
if space:
r.add(INFO, "device:space", space, "")
return finish(r, a.as_json)
def finish(r: Report, as_json: bool) -> int:
if as_json:
print(json.dumps({"worst": r.worst(),
"rows": [dict(level=l, area=ar, detail=d, fix=f) for l, ar, d, f in r.rows]},
indent=2, ensure_ascii=False))
return 1 if r.worst() == BLOCKER else 0
width = max(len(x[1]) for x in r.rows) if r.rows else 10
for level, area, detail, fix in r.rows:
print("%-8s %-*s %s" % ("[" + level + "]", width, area, detail))
if fix:
print(" " * 9 + "%-*s -> %s" % (width, "", fix))
print("\nworst: %s" % r.worst())
if r.worst() == BLOCKER:
print("Fix every BLOCKER before running another experiment. Do not attribute a failure to your patch yet.")
return 1
return 0
if __name__ == "__main__":
sys.exit(main())
+203
View File
@@ -0,0 +1,203 @@
#!/usr/bin/env python3
"""Capture what is actually on screen -- and say whether the control tree is usable.
Why this exists
---------------
Driving a UI blind (tap a coordinate, wait, tap again) is the single most
expensive habit in device work. A look at the screen resolves in one step what
coordinate guessing cannot resolve in five: the layout moved, a different dialog
is up, a countdown is frozen, a button is disabled, text says why.
Two independent kinds of evidence, and they are not interchangeable:
* the **image** -- always available, always truthful about what is rendered.
It is the only evidence for UI drawn by a runtime or a web view, where no
real controls exist.
* the **control tree** (`uiautomator dump`) -- precise, diffable, gives exact
bounds and text. Frequently EMPTY for canvas/webview/cross-platform UI.
This script grabs both, reports the size and whether the tree had content, so
you know which one is actually informative before you rely on it. It never
touches app state.
Usage
-----
# one look
python snap.py --out shots --tag before
# watch a wait: 6 samples, 3 s apart (bounded: hard cap of 20 samples)
python snap.py --out shots --tag waiting --count 6 --interval 3
# tree only, no images
python snap.py --out shots --tag tree --tree-only
# when nothing is on screen, capture and it will say so rather than guess
python snap.py --out shots --tag check --serial <serial>
Reading the output
------------------
* `bytes=0` after both capture paths -> the device refused; see the note in the
output rather than concluding the screen is blank.
* `tree=EMPTY` -> do not plan around accessibility bounds. Use the image and
reason from pixels, or drive the app through a non-UI path.
* Identical hashes across samples -> nothing is changing. Stop waiting and go
find out why (this is the stall detector).
"""
from __future__ import annotations
import argparse
import hashlib
import os
import re
import subprocess
import sys
import time
MAX_SAMPLES = 20 # hard cap so a typo cannot produce an unbounded loop
def run(cmd: list[str], timeout: int) -> tuple[int, str, str]:
try:
p = subprocess.run(cmd, capture_output=True, text=True, errors="replace", timeout=timeout)
return p.returncode, p.stdout or "", p.stderr or ""
except subprocess.TimeoutExpired:
return 124, "", "TIMEOUT after %ss: %s" % (timeout, " ".join(cmd[:4]))
except FileNotFoundError:
return 127, "", "not found: %s" % cmd[0]
class Dev:
def __init__(self, adb: str, serial: str | None):
self.adb = adb
self.base = [adb] + (["-s", serial] if serial else [])
self.serial = serial
def sh(self, cmd: str, timeout: int = 60) -> str:
return run(self.base + ["shell", cmd], timeout)[1]
def su(self, cmd: str, timeout: int = 90) -> str:
return run(self.base + ["shell", 'su -c "%s"' % cmd], timeout)[1]
def pull(self, remote: str, local: str, timeout: int = 120) -> bool:
rc, _, err = run(self.base + ["pull", remote, local], timeout)
return rc == 0 and os.path.exists(local) and os.path.getsize(local) > 0
def tree_stats(dev: Dev, workdir: str) -> tuple[str, int, int]:
"""Return (status, node_count, text_count). status in {OK, EMPTY, FAILED}."""
xml = "/data/local/tmp/_snap_ui.xml"
dev.sh("rm -f " + xml)
out = dev.su("uiautomator dump %s" % xml, timeout=90)
if "dumped to" not in out and "UI hierchary" not in out and os.sep not in out:
# some builds print nothing useful; fall through to the file check anyway
pass
local = os.path.join(workdir, "_snap_ui.xml")
if not dev.pull(xml, local, timeout=120):
return "FAILED", 0, 0
try:
data = open(local, encoding="utf-8", errors="replace").read()
except Exception:
return "FAILED", 0, 0
nodes = len(re.findall(r"<node", data))
texts = len([t for t in re.findall(r'text="([^"]*)"', data) if t.strip()])
if nodes == 0:
return "EMPTY", 0, 0
# A tree with nodes but almost no text is usually a shell with no real widgets.
return ("OK" if texts else "SPARSE"), nodes, texts
def capture_image(dev: Dev, workdir: str, name: str) -> tuple[int, str]:
remote = "/data/local/tmp/%s.png" % name
dev.su("rm -f " + remote)
# on-device write then pull: the streaming form returns 0 bytes on some ROMs
dev.su("screencap -p %s" % remote, timeout=90)
local = os.path.join(workdir, name + ".png")
if dev.pull(remote, local, timeout=120):
n = os.path.getsize(local)
if n > 0:
return n, hashlib.sha256(open(local, "rb").read()).hexdigest()[:12]
# fallback: stream to host
rc, out, err = run(dev.base + ["exec-out", "screencap -p"], timeout=120)
if rc == 0:
data = out.encode("latin1", "ignore") if isinstance(out, str) else b""
if data:
open(local, "wb").write(data)
return len(data), hashlib.sha256(data).hexdigest()[:12]
return 0, ""
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("--adb", default=os.environ.get("ADB", "adb"))
ap.add_argument("--serial", default=None, help="device serial; required when several are attached")
ap.add_argument("--out", default="snapshots", help="output directory (created if missing)")
ap.add_argument("--tag", default="snap", help="label used in filenames")
ap.add_argument("--count", type=int, default=1, help="samples to take (capped at %d)" % MAX_SAMPLES)
ap.add_argument("--interval", type=float, default=2.0, help="seconds between samples")
ap.add_argument("--tree-only", action="store_true", help="skip images; only read the control tree")
ap.add_argument("--image-only", action="store_true", help="skip the control tree")
a = ap.parse_args()
if a.count > MAX_SAMPLES:
print("count capped at %d (asked for %d)" % (MAX_SAMPLES, a.count), file=sys.stderr)
a.count = MAX_SAMPLES
workdir = os.path.abspath(a.out)
os.makedirs(workdir, exist_ok=True)
dev = Dev(a.adb, a.serial)
# cheap reachability gate with a timeout, so we fail fast and informatively
rc, out, err = run(dev.base + ["shell", "echo __ok__"], timeout=30)
if "__ok__" not in out:
print("device not answering: %s%s" % (out.strip()[:120], (" / " + err.strip()[:120]) if err else ""),
file=sys.stderr)
print("check `adb devices`; if the emulator process is alive but nothing is listening on the adb port,",
file=sys.stderr)
print("restart the instance through the vendor console (see references/environment.md).", file=sys.stderr)
return 2
tree_status = "skipped"
nodes = texts = 0
if not a.image_only:
tree_status, nodes, texts = tree_stats(dev, workdir)
print("control tree: %s (nodes=%d, non-empty text=%d)" % (tree_status, nodes, texts))
if tree_status in ("EMPTY", "FAILED"):
print(" -> the accessibility tree is not usable here. Do NOT plan around bounds from it.")
print(" The image is the primary evidence for this UI. Cross-platform runtimes, web views")
print(" and canvas-drawn UI frequently expose no real controls at all.")
elif tree_status == "SPARSE":
print(" -> tree exists but carries almost no text; treat it as unreliable and read the image.")
prev_hash = None
same_run = 0
for i in range(a.count):
tag = a.tag if a.count == 1 else "%s_%02d" % (a.tag, i)
if a.tree_only:
print("%-14s tree-only sample" % tag)
else:
size, digest = capture_image(dev, workdir, tag)
if size == 0:
print("%-14s bytes=0 -> device refused the capture. Not evidence that the screen is blank." % tag)
else:
note = ""
if digest == prev_hash:
same_run += 1
note = " (identical to previous -> nothing changed)"
else:
same_run = 0
prev_hash = digest
print("%-14s bytes=%-8d sha=%s%s" % (tag, size, digest, note))
print("%-14s %s" % ("", os.path.join(workdir, tag + ".png")))
if same_run >= 2:
print(" -> %d identical samples in a row. This is a stall, not a slow operation." % (same_run + 1))
print(" Stop waiting: inspect the image, or check the app is still alive and in the foreground.")
if i < a.count - 1:
time.sleep(max(0.0, a.interval))
print("\nNow LOOK at the image(s) before deciding the next action.")
print("Driving blind and waiting is how rounds get wasted; the screen usually states the reason.")
return 0
if __name__ == "__main__":
sys.exit(main())