B4: wire row decode into catalog + activate FieldFilter end-to-end #7
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "feature/dune-extract-b4-catalog-wiring"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Final Phase B task. Wires B3's DataTable row-decode output through
FieldFilterintocatalog_writer— every row that B1+B2+B3 can reach now flows into Markdown / JSON / CSV catalog outputs.What landed
uasset_parser.Summary.folder_name— the package's/Game/...path; the row scan identifies a parsed.uassetby its own ground truth (bypasses the B1-FU virtual-vs-physical mapping).enrichment.CategoryCatalog.rows—dict[DataTable_filename, {row_name: {field: value}}]+total_rowsproperty.enrichment.enrich_with_row_data()— walks every pak's physical entries, parses DataTable-class.uasset, pairs with the physically-adjacent.uexp, filters each row throughFieldFilter.filter_row, attaches to the matching category.__main__— calls it afterbuild_catalog; reports row stats; warns when no Oodle backend;DUNE_EXTRACT_SKIP_ROWS=1disables.catalog_writer— emits rows in all three formats:categories[cat].datatable_rows+rows_decoded.items-rows.csv.End-to-end results
The Weapons catalog now carries
DT_WeaponStats Damagerow:StatDisplayName=UI/ItemStat_Damage, StatStep=0.1, bIncreasingIsGoodForPlayer=✓.FieldFilter activation
Genuinely live for the first time — every row dict is
.filter_row()-ed before publication.--include-asset-paths/--include-lore/--include-dev-commentarynow affect real output. The 14 currently-reached tables are stat-definition tables (mechanical fields), so the flags don't visibly change THIS run; they matter once richer item tables unlock.No regression
probe_pak_entries.pyprobe_read_uasset.pyprobe_datatable.py Systems.pakimplementedHonest coverage note
The B1 virtual-vs-physical bottleneck that bounded B3 bounds B4: 14 DataTables out of ~1,725 inventoried in the FDI. Richer per-item tables (
DT_BaseItems_Weapons, shown as—) unlock when B1 widens or the scan gains an FDI-driven name→offset mapping. Wiring is complete; surface grows with B1.Test plan
Lands the final Phase B task: every row that B1+B2+B3 can reach now flows through FieldFilter into Markdown / JSON / CSV outputs. Changes: * uasset_parser.Summary gains folder_name (the package's /Game/... path) so enrich_with_row_data identifies a parsed .uasset by its own ground truth, bypassing the B1-FU virtual-vs-physical mapping. * enrichment.CategoryCatalog gains rows: dict (DataTable filename -> {row_name: {field: value}}) + total_rows property. * enrichment.enrich_with_row_data(catalog, pak_map, paks_dir, field_filter) walks every pak's physical entries, parses any DataTable-class .uasset, pairs it with the adjacent .uexp, filters each row through FieldFilter.filter_row, and attaches to the matching category. Returns stats {tables_parsed, rows_recovered, oodle_available, scanned_*}. * __main__.py calls it after build_catalog; reports row stats; warns when no Oodle backend. DUNE_EXTRACT_SKIP_ROWS=1 disables. * catalog_writer emits rows in every format: - Markdown: per-category "DataTable row contents" section, one subsection per reached table with column-per-field tables (bools as ✓/✗, FText as `key`, truncated long strings); the DataTable inventory shows a "Rows decoded" column. - JSON: categories[cat].datatable_rows + rows_decoded. - CSV: long-format items-rows.csv (category, datatable, row_name, field_name, value). End-to-end run: 14 DataTables / 159 rows recovered with the ooz backend (Dungeons.pak NPCRecipes 50, Input.pak IA/IMC tables 69, Systems.pak ArmorStats 4 + WeaponStats 36). Weapons catalog now carries Damage row: StatDisplayName=UI/ItemStat_Damage, StatStep=0.1, bIncreasingIsGoodForPlayer=✓. FieldFilter is genuinely live for the first time — every row dict is .filter_row()-ed before publication. The --include-asset-paths / --include-lore / --include-dev-commentary flags now affect real output (no-visible effect on the current 14 stat-definition tables; they'll matter once richer item tables unlock). No regression — B1/B2/B3 probes reproduce same numbers (B1 574/574 adjacency, B2 4/4, B3 5 tables / 60 rows). Coverage of the broader DataTable set remains bounded by B1's physical-entry recovery.