sources	last-verified
docs/THE-MODEL.md docs/CAPABILITIES.md package.json serve/launcher.mjs train/train.py	2a8991e7 2026-09-07

Unity 3D Equational Model (U3E)

⭐⭐ OUR OWN UNITY MODEL. One transformer, one vocabulary, every modality — she talks, she draws, she sees, she speaks, she hears. Pictures and voice are emitted as CDF 9/7 wavelet coefficient fields, not pixels and not samples, so one set of weights carries all five.

⚠ All five answer today. None of them is measured, and the model has trained 0.48% of one run. docs/CAPABILITIES.md owns that, per modality.

⭐⭐ THE TARGET, IN THE FOUNDER'S WORDS: "for training a EBM AI model thats text and image and listens and talks and sees all in one and doesnt use shit like ollama" — one set of weights, ours, trained by us; no third-party runtime and no third-party weights at inference.

⛔ READ docs/THE-MODEL.md FIRST. It states the PLAN — what this is for and why it is shaped this way — in the future tense, deliberately. The status block below and docs/CAPABILITIES.md own what is real TODAY. ⚠ The two are kept apart on purpose: a reader who meets the scaffolding first mistakes it for the intent and then defends it.

she talks	text in and out
she draws	images emitted as CDF 9/7 wavelet coefficient fields, not pixels
she sees	an image you show her encodes through the same transform
she speaks	her Unity One voice as 1-D fields of the same wavelet family
she hears	audio in, through the same path

⭐ The idea in one line: her picture engine and her voice engine already use the same wavelet mathematics, so a single model can carry all of it in one representation — and because the transform is symmetric, vision is the input side of drawing and speech-recognition is the input side of speaking. No CLIP, no Whisper, no projection layers to keep aligned.

---

STATUS 2026-09-07   all five modalities answer, and none of them is measured
  codec       ✅ real · bit-exact JS ⇄ Python, maxDiff = 0 over ten stages
  app + orb   ✅ real · progressive decode, morph, five-modality UI
  serving     ✅ real · the HTTP contract is live, and the slot HOLDS WEIGHTS
  corpus      🔄 building · 46,241 of 64,525 banked (71.7%) · 18,284 owed
              benchmark 502 of 500 ✅ MET · 116 GB on disk · every base query audited
  shards      ✅ shards-v4 · 598 files · 152,934 examples · 4.6 GB
              mix ENFORCED: image 84.98% · audio 12.02% · text 3.00% · 100% voiced
  tokenizer   ✅ scheme (e): scalar quantisation + runs · subband-major ·
              in-band header · vocabulary 184 field + 256 byte + 10 marker
  the model   ⚠ EXISTS, TRAINED 0.48% OF ONE RUN, AND IS NOT FINISHED
              TINY 6L x 384 · 36.0 M · bf16 · step 1,000 of 209,551 · loss 3.2549
              AUDIO_TRAINED = True → speak and hear turned on by themselves

⚠ Every number above moves; treat it as a reading, not a constant. The authoritative live figures come from node datatools/build-corpus.mjs --stats, the bank monitor and the Teechuh — never from this file.

⛔⛔ THE MODEL EXISTS AND THAT IS NOT THE SAME SENTENCE AS "IT WORKS". A real checkpoint from this project's own run is served through the model slot. What it produced, measured at step 600:

  chat   "e imahno  gin r To gftio t l isdpalou , neoufg iir rdoa  gs"
  image  the in-band header NOT learned — emitted 30,208 x 40,960, clamped
  see    field in -> text out, and zero off-vocabulary ids

⭐ So every response carries two flags answering different questions: standIn says are these our weights (now false), and provisional says are they finished weights (still true), with its reasons in words. ⛔ A provisional: true response is evidence for nothing about the finished model — not its quality, not its pictures, not her voice.

⛔⛔ ALL FIVE MODALITIES NOW ANSWER, AND THAT IS THE MOST DANGEROUS LINE ON THIS PAGE

⭐ speak and hear turned on by themselves on 2026-09-06, which was the design: AUDIO_TRAINED is derived from what the shards actually contained (train/train.py:701), never configured — so the run that saw 12.02% audio tokens flipped it with no code change. step-800.pt and step-1000.pt carry AUDIO_TRAINED = True.

⛔ THIS PAGE SAID THE OPPOSITE UNTIL 2026-09-07 — "speak and hear return 501 absent. There is no scheme-(e) audio tokenizer" — and both halves were false: tokenizer/audio-scalar-tokenizer.js exists and the flag was already true.

⚠ BUT "ANSWERS" IS NOT "WORKS", AND THE CHANGE MAKES THIS PAGE WORSE, NOT BETTER. A loud 501 absent could not mislead anybody. A 1-D voice field returned by a checkpoint at 0.48% of a run can, and NOBODY HAS PLAYED ONE BACK. ⛔ see quality, hear accuracy and speak audio are all three unmeasured. docs/CAPABILITIES.md is the page that owns this, per modality.

---

Quickstart

Prerequisites: Node ≥ 18 (measured on v22.13.1) and Python 3.10+ for the codec twin and the banker. Nothing else.

⭐ There is no npm install. node_modules/ is empty and the JS side has zero runtime dependencies — that is a LAW here, not an accident. If you find yourself installing a JS package to run something in this repo, read docs/SETUP.md §2 before you do.

git clone <remote> "Unity 3D Equational Model"
cd "Unity 3D Equational Model"

node --version                        # must print v18 or higher

node codec/verify/js-selftest.mjs     # codec round-trips, payloads, PSNR
node codec/verify/core-purity.mjs     # the core is portable — a phone can load it
node codec/verify/parity.mjs          # maxDiff = 0, JS vs Python, ten stages

U3E_MODEL=standin node serve/server.mjs   # → http://127.0.0.1:8790  — open it

That last command gives you the whole app: the orb, the chat, the picture viewer. It answers with the stand-in and says so.

⛔⛔ U3E_MODEL=standin IS NOW REQUIRED IF YOU HAVE CHECKPOINTS ON DISK, AND THAT IS DELIBERATE. With a train/checkpoints/step-*.pt present, plain node serve/server.mjs selects the trained model automatically and then REFUSES TO BOOT unless the Python inference sidecar is answering:

python train/infer.py --port 8797        # then, in another shell:
node serve/server.mjs                    # picks the trained model on its own

⭐ It refuses rather than quietly falling back to the stand-in, because serving a placeholder while real weights sit unused "would look like success". A fresh clone has no checkpoints, so plain node serve/server.mjs is fine there — but the flag is written above so the quickstart works either way.

⚠ The corpus is NOT in the repo (the corpus lives at a configured root outside the repo, and the real banked images are not reproducible — see datatools/SOURCES.md). A fresh clone therefore has an empty bank; the stand-in has nothing to replay until you build one:

node datatools/build-corpus.mjs --synthetic 120     # 120 labelled synthetic fields
node datatools/build-corpus.mjs --stats             # what is in the bank right now

Full cold-start procedure, including the running services and how to stop them safely: docs/SETUP.md.

---

The services

⭐ Start at the switchboard — one page that starts, stops and reports every service below. Each runs on its own port and is stoppable without losing work.

Port	Process	What it is
8799	node serve/launcher.mjs	⭐ THE SWITCHBOARD — the front door. One page for the whole project
8790	node serve/server.mjs	The app and the model slot — chat, images, voice, vision
8788	node datatools/portal/server.mjs	The corpus portal — search, review, bank, coverage
8789	node datatools/corpus-review.mjs	The review viewer — where a picture is judged by eye
8795	node monitor/bank.mjs	Banker dashboard — rate, blocks, cursor position
8796	node monitor/model.mjs	The Teechuh — configures a training run, then watches it
8797	python train/infer.py	The inference sidecar, required when the trained slot is selected

⚠ This table said "Four long-running processes" and listed four until 2026-09-07 — omitting the switchboard, which is the page an operator actually opens first. ⛔ All seven bind loopback; the portal fetches arbitrary URLs and the switchboard can spawn jobs, so none of them belongs on a network.

⛔ The portal needs its search backend: --searx http://127.0.0.1:8888 (a local SearXNG container). Without it, /search returns nothing and the banker's rate reads zero — which looks like a bug in the banker and is not.

⚠ The banker and the captioner stop COOPERATIVELY via control files (.bank-control and .caption-control at the data root). Killing them mid-write can cost the manifest. docs/SETUP.md §5.5 has the procedure.

The HTTP surface

serve/server.mjs speaks the same wire shape the existing local-Unity chat page and the Android app already use, so those keep working against this server unchanged.

Method	Route
GET	/api/state	slot version, capabilities, and ⭐ standIn — always surfaced, never buried
POST	/api/chat	text in, text out
POST	/api/image	a field out. Pass progressive: true for an NDJSON stream: coarse scales first, sharpening as slices arrive
POST	/api/speak	text → 1-D voice field
POST	/api/see	image → concept
POST	/api/hear	audio → text
GET	/, /app/*	the app itself
GET	/codec/*, /tokenizer/*	⭐ the same sources Node runs — no build step, no copy to drift out of sync
GET	/docs/*.md	the project's own docs, read-only, linked from Settings

⭐ Progressive emission is not a loading animation. Each NDJSON line carries coefficients the client genuinely did not have. Mute the later lines and the picture stays coarse — which is a property of the representation, not of the UI.

The portal (:8788) serves /, /search, /proxy, /bank, /coverage and /corpus/*. Byte-level contract for both: docs/WIRE-CONTRACT.md.

---

Layout

Every top-level directory, and what it is for:

codec/	The CDF 9/7 codec. js/ is the original, py/ a bit-exact twin, verify/ the three harnesses that prove it
tokenizer/	Field ⇄ token sequence. The project's real research risk. ⚠ The gate ruled against scheme A; scheme (e) is what the model above is training on
datatools/	The corpus pipeline: taxonomy, banker, captioner, dedup, watermark and text detection, provenance checks. ⚠ Renamed from data/ on 2026-09-07 — the CORPUS moved to a configured root (D:\data here), and this directory holds only TOOLING
datatools/portal/	The review web UI — search, look, judge, bank
train/	The trainer (train.py), the vocabulary (vocab.py), the inference sidecar (infer.py), sharding, the fit and peak harnesses, pod bring-up, budget arithmetic
serve/	Inference server, the model slot, the trained implementation, the stand-in, progressive slicing, legacy adapter
app/	The web client — orb, morph, viewer, listen, appearance, offline worker
android/	APK build for the same app (build-apk.sh / .bat)
monitor/	The two dashboards, the page parser (page-check.mjs) and the job supervisor (supervise.mjs)
docs/	25 documents, 17 of which ship to the product. See the index below
wiki/	The Karpathy-style code wiki — the map; the code is the territory
.claude/	The operating stack: laws, workflow, personas, hooks. ⛔ Tracked on purpose — a clone with no skills cannot run the project

⭐ The model slot is the design point. serve/slot.js returns 501 absent when there is no model and 500 failed when there is one and it broke. Those are different facts and they are reported as different facts — dropping a real model in is a file, not a refactor.

Commands

# codec — the three that must stay green
node codec/verify/js-selftest.mjs        # round-trips, payloads, PSNR
node codec/verify/core-purity.mjs        # no Node builtins in the portable core
node codec/verify/parity.mjs             # JS vs Python, maxDiff = 0

# corpus
node datatools/build-corpus.mjs --stats       # what is banked
node datatools/build-corpus.mjs --synthetic N # N labelled synthetic fields
node datatools/build-corpus.mjs --export-manifest   # URLs + sha256 + licence, for the pod
python datatools/bulk-bank.py                 # the banker
node datatools/caption-run.mjs                # the captioner
node datatools/caption-gate-selftest.mjs      # caption register checks

# tokenizer  ⭐ both FIXED 2026-09-03 by a streaming loader
node tokenizer/selftest.mjs
node tokenizer/measure.mjs

# training-side arithmetic, no GPU needed
node train/budget.mjs                    # what a pod run would cost, assumptions shown
node train/vocab-plan.mjs                # the 2,856-token vocabulary extension
node train/shard.mjs --out train/shards-v5  # JSONL shards + SHA256SUMS
                                         # ⛔ PASS --out. The default `train/shards`
                                         #   has never been used by a real build

# docs
python docs/drift.py                     # which docs have gone stale
python wiki/coverage.py                  # does the wiki actually reach the code

package.json wraps the common ones: npm run verify:js, verify:purity, verify:parity, corpus, docs:drift, docs:coverage.

⭐ Every number in this project should be reachable by running something. If one is not, that is the bug.

---

⛔ Where this actually stands

The gate ran, the bet lost, and the margin has been re-measured twice since

⛔⛔ THE HEADLINE IS A BAND, NOT A RATIO — and the famous ~144× is RETRACTED. Re-measured on 120 spread benchmark images (benchmark key c6efe4290157, corpus key bc7a9982f6e5), with training and measurement sets disjoint:

  SCHEME A — the bet that lost        SCHEME B — (e), what ships
    20 dB  field 297,930                25.0% kept  120,357 tok  29.80 dB
           pixel   2,352                       ⭐ PIXEL CANNOT REACH AT ANY RATE
           PIXEL WINS 126.65x
    25 dB  field 297,930                10.0% kept   56,379 tok  25.29 dB
           pixel   9,335                       pixel wins 6.0x
           PIXEL WINS  31.92x
    pixel CEILING 27.49 dB   ·   field reaches 42.79 dB

⭐⭐ The pixel family cannot be bought past 27.49 dB at any token count; (e) sits at 29.80 dB — unopposed, not merely cheaper. And where they overlap, (e) collapses 126.65× → 6.0×.

At matched fidelity the wavelet field costs 153,827 tokens at 25.15 dB against the pixel baseline's 1,066 at 28.77 dB — the baseline wins by ~144×.

⛔ SUPERSEDED — THAT RUN WAS CONTAMINATED. It trained the field codebooks on all 654 fields including the ten it scored, and the pixel codebook on exactly those ten. ⚠ Its 40-image re-run was then drawn in bank order, which clusters by concept. Neither number should be quoted.

⭐ Three excuses were killed first, by measurement: the corpus (the field number was unmoved across three corpora while the baseline's moved), codebook capacity (+4 dB at identical tokens, then saturation), and block size (B=2 reaches 41.43 dB, so the ceiling was VQ dimensionality, never the wavelet). ⭐⭐ The mechanism was measured, not guessed: intra-block correlation is 0.4580 for pixel patches against 0.1908 for wavelet coefficients — the transform decorrelates, which is its job, and vector quantisation needs correlation.

⚠ The representation is not refuted — this scheme over it is.

⭐⭐ AND THE FORK IS CLOSED. It was four options on the board; the founder called it on 2026-09-03: "(e) only — keep the claim whole" — scalar quantisation plus entropy coding. ⛔ He declined the shipping hedge, so there is NO conventional-tokenizer fallback: a negative result on (e) blocks the PRODUCT, not just the research. ⭐ Phase 2+ is formally unblocked, which it was not when this section was written.

⭐ The named research risk it left — the discrete-vocabulary bridge — is also crossed, and the premise turned out to be wrong: there was never a bitstream. Scheme (e) emits ids over a fixed vocabulary because the entropy model is the transformer. The real gap was an 8-number side-channel: decodeField(tokens) takes no meta argument, PSNR delta 0.000000 dB, header 0.041% of the stream, vocabulary 167 → 184.

Two things are broken right now — both fixed, and this section had gone stale

	Was	Now
tokenizer/selftest.mjs + measure.mjs	⛔ heap OOM — loadCorpus() read every field into memory	⭐ FIXED by a streaming, re-iterable loader. 6.29 GB of payload streamed against a 0.20 GB peak heap
benchmark set	⛔ --stats reported 0 of 500	⭐ 502 of 500 — MET. Rebuilt from the fixed 30 BENCHMARK_CONCEPTS

⚠ The caution that used to sit here was right at the time and is kept: bounding that load would have changed which images the codebooks see, which changes every measurement.

⭐ The fix avoided that entirely — streaming touches every field the unbounded version did, so the corpus the codebooks train on is unchanged.

⛔ And it returns a re-iterable, not a generator. measure.mjs trains codebooks in a nested loop, and a generator is consumed once — the other eleven books would have trained on an empty corpus and reported success.

The honest frame

The central bet — that a sparse wavelet field beats a pixel-VQ codebook in tokens-per-image at matched fidelity — could be wrong, and Phase 1 existed to find out before a pod was rented.

⭐ It was wrong, the measurement said so before any money was spent, and that is the process working rather than failing. A negative result has three honourable responses, all written into the board. The dishonourable fourth — proceeding while pretending the measurement did not happen — is what .claude/CONSTRAINTS.md :: PHASE 1 IS A GATE exists to prevent.

⚠ What failed is one scheme's premise, not the work. The codec, the parity gate, the corpus, the portal, the slot, the app, the orb and the morph are unaffected and still correct.

---

⛔ No Ollama — all our own

Gee: "this wont need ollama all our own shit". Our weights, our tokenizer, our codec, our inference server. Ollama is not used for training, not for serving, and not as a fallback. The only thing carried over from the parent project is the HTTP shape (/api/chat, /api/image) — a wire contract so the existing chat page and the Android app keep working, not a dependency.

⛔ This is not the brain

Three projects share her name:

Project	What it is
If-Only-I-Had-A-Brain	A 411M-neuron equational brain — no text-AI in its cognition path
Ollama 18+ local Unity ⚠ (was the parent folder until 2026-08-31; now a separate location)	Open weights run locally with her persona in front
Unity 3D Equational Model (here)	Our own trained multimodal model

Their laws are not this project's laws, and no cross-project imports exist in either direction — copies with provenance comments only.

---

Docs

Start with docs/RESUME.md — ⭐ one entry, always the most recent; it is replaced each session rather than appended to, so what you read is current by construction. docs/NOW.md is the same rule in one screen. Then docs/TODO.md (the board), then docs/LIMITATIONS.md (what this cannot do, written before anyone asked). .claude/CONSTRAINTS.md holds the laws, and they are stricter than usual on purpose.

If you want	Read
⭐ What we are building and why — THE PLAN	docs/THE-MODEL.md — start here
It explained in plain English	docs/HOW-IT-WORKS.md
To run it from cold	docs/SETUP.md
Which parts are actually real, and what proves it	docs/CAPABILITIES.md
The maths, derived from first principles	docs/EQUATIONS.md
The byte layout of a field	docs/FIELD-FORMAT.md
The HTTP contract	docs/WIRE-CONTRACT.md
The research risk, and the verdict	docs/TOKENIZER.md
The measurements themselves	docs/MEASUREMENTS.md · docs/MEASUREMENTS-AUDIO.md
Where the corpus came from	docs/DATASETS.md · datatools/SOURCES.md
What a pod run needs before it is rented	docs/POD-READINESS.md
⚠ What it CANNOT do, by design	docs/LIMITATIONS.md
How to swap a legacy engine in behind the wire	docs/LEGACY-SWAP.md
How her look stays hers across images	docs/APPEARANCE-TRANSFER.md
Where this is going	docs/ROADMAP.md
Terms used here	docs/GLOSSARY.md
The ledger of everything closed, including negative results	docs/FINALIZED.md
The architecture	docs/ARCHITECTURE.md

Conventions that will bite you first

• ⛔ No npm install. Zero JS runtime dependencies, by law.
• ⛔ The corpus and the shard sets are not in git — they live at the configured data root, and .gitignore names the DISK measurement behind each exclusion rather than a preference. ⚠ The real banked images are not rebuildable; datatools/SOURCES.md is the record that survives.
• ⚠ Line endings are mixed in the WORKING COPY and it matters when you edit. .gitattributes carries * text=auto, so the repository stores LF and a Windows checkout is CRLF. Multi-line search-and-replace against a CRLF file fails silently in some tooling, and ⛔ grep -c $'\r' reports 0 on a CRLF file under MSYS because it strips them — count byte-wise or you will get a confident wrong answer.
• ⚠ .claude/ is tracked, and stays tracked only while every remote is private and Gee-owned. Re-verify with gh repo view --json visibility,owner before adding a remote.
• Private repository. Not published, not licensed for redistribution.