KPEG

Every photo on your phone is a 5MB brick. Your whole life's library depends on servers you don't control, one account ban, one broken drive, and a decade of memories evaporates. JPEG was designed in 1992 when a 1 megapixel image was science fiction. It's time for a new format.

KPEG stores the DNA of a photo in ~2,000 bytes, small enough to live in a tweet, a QR code, or a blockchain transaction. Not pixel data: a 16-color k-means palette, an 8×8 composition grid, Sobel-extracted keypoints, a 32x32 edge map of object boundaries, and a Claude-Vision-authored semantic scene description referencing a personal library of faces, places, and objects the user has registered. All compressed with Brotli Q11 into a tiny binary container.

To reconstruct, FLUX Pro v1.1 + PuLID FLUX (face identity embedded at generation time) + FLUX Kontext Pro (multi-pass scene refinement) uses the bitmap as a color/composition anchor, the scene JSON as a prompt, and your library selfies/place-photos/object-refs to restore identity and background with fidelity. Result: a visually-similar reconstruction from a seed that fits in a text message.

Every encoded photo is anchored to Hedera, HFS stores the .kpeg, HCS logs an immutable "who, what, when" consensus entry, HTS mints an NFT of ownership, at under $0.001 per photo.

The real magic: KPEG gets BETTER over time. As AI models improve, as your library grows, as session inference fills gaps between stored photos, every past memory sharpens. Your 2026 KPEGs decoded in 2040 will look better than they do today. Compression ratio: ~2,500×. Format: open-spec, privacy-first, chain-anchored. Face identification is 100% on-device (ML Kit + local SQLite embeddings), no biometrics ever leave the phone.

This isn't compression. It's photo DNA. Your memories, grown on demand.

ARCHITECTURE Two-person team split: one built the Flutter Android app + Python/Flask API layer + Hedera blockchain integration; the other built the AI encode/decode engine. Everything runs on localhost over WiFi — app on the phone, Flask backend on a laptop, hackathon-mode. Flutter 3.41 + ML Kit on-device, Python 3.12 + Flask + SQLite on the backend, Anthropic Claude + fal.ai + Hedera (Python SDK, no Solidity) for services.

THE BINARY PROMPT, EVOLVED OVER THREE DAYS We started with a naive color map: just a k-means palette + a downsampledimage. Reconstructions were soupy, composition was lost, identities were hallucinated. We iterated:

• v1 — 16-color k-means palette + 8×8 color grid. Reconstructions had the right mood but wrong structure. • v2 — added Sobel-extracted keypoints (up to 500, R/G/B + position packed into 3 bytes each). Now fine detail anchored correctly. • v3 — added 32×32 binary edge map (128 bytes) for object boundaries. FLUX finally understood WHERE things were, not just what colors. • v4 — library-matched references: Claude Sonnet 4.5 Vision analyzes each photo against a DB catalog of registered people/places/objects and emits a structured scene JSON with bounding boxes and ref IDs. This killed hallucination — FLUX now reconstructs YOUR face from YOUR selfie, not a random stranger. • Final — Brotli Q11 compresses the JSON to ~500B. Total: 12B header + ~1.5KB bitmap (palette+grid+keypoints+edges) + ~500B JSON + 2B CRC ≈ 2.5KB.

AI PIPELINE (DECODE, MULTI-PASS) Naive single-pass generation was unreliable — faces duplicated across group photos, places drifted, bitmap was ignored. We split it:

• Stage 1 (identity): PuLID FLUX embeds the primary face identity DURING generation (not as post-process face-swap). We learned this the hard way after fal-ai/face-swap silently started returning 404s — the errors were swallowed by a bare except that made it look like face swap was working when it was being skipped entirely. Debug pipeline caught it. • Stage 2 (scene): FLUX Kontext Pro refines with angle-matched place photos, distinctive object references from the library, AND the bitmap itself as a reference image. Priority order: faces > bitmap > places > objects, capped at 8 references per request. • Stage 3 (high quality): Clarity Upscaler 2× → 2048px output.

HEDERA (three services, best-effort) Per photo: HFS uploads the .kpeg on-chain → HCS logs a consensus message ("user X encoded image Y at time Z, file_id=F") → HTS mints an NFT referencing the file. Everything is best-effort — encode succeeds locally even if any Hedera step fails. Uses hiero-sdk-python natively (zero Solidity, zero Node.js, entire stack is Python).

PRIVACY (first-class) Face detection AND identification happen 100% on-device. ML Kit detects bboxes, we crop to 64×64 grayscale, extract 4096-byte embeddings, store in local SQLite, match via cosine similarity. No biometric data ever touches the network. Only face crop selfies travel to the server — and only because the decoder needs reference photos for reconstruction.

NOTABLE HACKS • Place photo angle matching was silently broken: rows with NULL compass_heading had distance=0 and always won first place. Fix: coalesce NULL to a large penalty value in the SQL distance calc. • Cross-machine path portability: DB stored absolute paths from the dev's machine ("/home/user/library/..."). Decoder on any other machine returned empty reference lists, causing FLUX to hallucinate everything. Fix: rewrite paths at read-time against local LIBRARY_DIR. • PuLID face leakage: in group photos, the single reference face was being applied to ALL people. Fix: explicit prompt constraint ("The reference face belongs ONLY to Person 1. The other N people must have COMPLETELY DIFFERENT faces.") plus spatial bounding-box anchors in the prompt. • Silent error swallowing: a bare except hid face-swap deprecation for hours. Lesson: log everything, fail loud during debug. • Debug pipeline: built an offline diagnostic tool that simulates the full decode — dumps every prompt, every reference image as base64, every library path lookup — so we could inspect what FLUX was actually receiving without burning API credits.

WHY HEDERA (partner benefit) Ethereum L1 per-photo cost would be ~$20. Hedera: <$0.001. Plus carbon-negative, 10K TPS, and HCS is purpose-built for ordered consensus logs — exactly what a photo provenance trail needs. The Python SDK let us keep the entire stack in one language, no Solidity context-switch.

Encoder pipeline (Flutter app + Python/Flask backend):

1. Phone captures photo + sensor metadata (GPS, compass, tilt, lens)
2. ML Kit detects faces on-device; 64×64 grayscale embeddings match against local SQLite face library via cosine similarity
3. Backend extracts 16-color custom palette (k-means), builds 8×8 composition grid, extracts keypoints for fine detail
4. Claude Sonnet 4.5 Vision analyzes the photo against the library catalog and produces a structured scene JSON referencing known people/places/objects
5. Everything compressed with Brotli Q11 → packed into binary container (12B header + bitmap + JSON + CRC = ~2KB total)
6. Hedera integration: HFS upload → HCS consensus log → HTS NFT mint

Decoder pipeline:

1. Unpack binary, decompress JSON, render bitmap as color guide
2. Resolve library refs (face selfies, place photos, objects) to data URLs with priority ordering (faces > bitmap > places > objects)
3. Stage 1: FLUX Pro v1.1 generates base photo from prompt
4. Stage 2: FLUX Pro Kontext refines with reference images — the bitmap serves as a color/composition anchor even for t2i tiers
5. Stage 3 (high quality): Clarity Upscaler 2× for 2048px output

Stack: Flutter 3.41 (Android), Python 3.12 + Flask + SQLite, Anthropic Claude Sonnet 4.5, fal.ai (FLUX Pro v1.1 + Kontext Pro + Clarity), Hedera (hiero-sdk-python) — HCS + HFS + HTS.

What's Next • Video KPEG (10-sec clips in ~10KB via keyframe + motion DNA) • Session Inference (AI generates missing photos between stored KPEGs) • Public Place Libraries ("Pokémon Go for venues" — shared place DBs) • Post-quantum signatures for 400-year durability • iOS app + open-source reference decoder • Format standardization proposal (IETF / W3C)