Developer tests frontier LLM secret recovery against own obfuscated app build

A developer has published a first-hand experiment on dev.to in which they asked a frontier large language model to recover the secrets embedded in their own application. The build had been hardened so that configuration values no longer appeared as readable strings, and the question was whether a capable model, working only from the decompiled artifact, could work backward to the values the application can decode for itself.

The hardening under test

According to the dev.to writeup, the scheme rests on an explicit boundary: the master key ships inside the binary. Because the application must decode its own configuration, everything needed for decoding is present in the artifact. The author frames this as obfuscation, not cryptography — the goal is to raise the cost of reverse engineering, not to make recovery impossible.

Concretely, configuration values are XOR-encoded, each with a working key derived at runtime from a single 32-byte master key, and those working keys are never stored at rest. In the binary, the encoded values appear as numeric byte-array literals that resemble hashes, and no string reads as a key.

The author also decorrelated the outputs: the derivation strips shared prefixes so that similar secrets do not produce similar-looking arrays. The point, as the post explains, is to block an investigator's cheaper route — clustering values that look alike, inferring they share a purpose, and then locating the code that consumes them.

Three implementations, byte for byte

The derivation exists in three independent places: the application runtime, a build-time tool, and a native build script, written in three different languages. All three must agree byte for byte, because differences in how languages handle bytes and numeric operations can break decoding when one implementation's output becomes another's input. The author treats this as one of two distinct questions the hardening had to answer: exact agreement between implementations for correctness, and difficulty of blind interpretation for opacity.

An adversarial court of agents

To investigate the build, the author organized LLM agents into an adversarial court. Three neutral researchers gathered facts from different angles — one examined code, permissions and obfuscation patterns; another focused on network and data; the third investigated libraries and behavior. A prosecutor then argued for findings with a deliberate bias toward identifying problems, an advocate supplied mitigating or benign explanations, and an impartial judge weighed the record and decided which conclusions had enough support.

The separation addresses a known failure mode of single-prompt analysis: once one explanation looks plausible, subsequent observations get fitted around it. The opposed roles make objections part of the process rather than an afterthought. The author is candid about the limits, though — the agents are not independent sources of truth and can share blind spots or make the same mistake.

What counted as evidence

Conclusions required two or three independent technical indicators; a single suspicious pattern remained only a lead. Independence carries most of the weight: several agents repeating one observation do not create several pieces of evidence, and describing one byte array in three different ways is not corroboration. A strong case had to connect an encoded-looking array to reachable decoding logic, and then connect the decoded result to its actual use in the application.

Reachability was an explicit check — code existing in a decompiled build does not establish that the application executes it, and the author argues an interpretation resting on a theoretical path deserves less confidence than one supported by observed behavior. The advocate also had a concrete task: construct a legitimate explanation for the evidence and identify what further evidence would distinguish it from the accusatory one.

Why it matters

For anyone shipping logic or configuration to devices, this is a direct probe of a boundary that has quietly shifted. Obfuscation has always been a cost-raising measure calibrated against human patience and tooling; frontier LLMs change both, and testing one's own artifact against a capable model is now a cheap, sensible step before release.

The methodology may matter as much as the target. The dev.to piece treats a model's conclusions as claims to be challenged rather than answers, with role separation, corroboration thresholds and reachability checks doing the work. That skepticism extends to the null result: the author writes that the hardest question was how much confidence to place in a failed recovery attempt — a fitting caution, given the post's own warning that the agents can share blind spots. A secret that survives one LLM's attempt has not thereby been proven safe.