Red5Sorcery Data Sculptor · Milestone WASM-002

Teachable Scenario: First Materialization

PRE-QA CANDIDATE · DRAFT / NOT FROZEN SPEC BASIS: PRD Revision BL QA STATUS: Pending independent Claude adversarial review
AUTHORITY & STATUS BOUNDARY: This document is a derived instructional walkthrough and acceptance reference, not an authoritative Engineering Standard or a second specification contract[cite: 3]. Numbered requirements in the Product Backlog PRD Revision BL govern unconditionally[cite: 1, 3]. All candidate WASM-002 requirements remain UNDER REVIEW pending Claude QA review and human freeze[cite: 1, 3].
Design Creed — “Data work you can explain.”
Data Sculptor is designed to reduce forensic uncertainty by combining bounded-memory processing, strict source certification, deterministic row identity, governed Work Orders, visible artifacts, and durable Receipts[cite: 1, 3]:
I know what I asked the computer to do. I know what it actually did. I know what happened when it couldn’t do it. And I can show somebody else.[cite: 1]

1. Scenario Scope & Target Goal PRD: W002-MAT-002 / W002-RID-002

An office analyst receives an uninspected, valid-UTF-8 Statistics Canada CSV data drop (crime_stats_2025.csv)[cite: 1, 2, 3]. Operating within a supported browser context, the analyst must validate a local folder workspace (Suitcase), author a declarative Work Order selecting three columns, save the Work Order into an immutable physical artifact, and execute materialization to establish a brand-new Store[cite: 1, 2, 3].

This walkthrough teaches run-local source continuity, exact byte accounting, deterministic row alignment, and visible governed persistence[cite: 1, 3]. It deliberately documents the happy path for First Materialization establishing a new Store; it does not claim automatic cross-session cryptographic identity for source bytes (which belongs to the deferred V1 FINGERPRINT / LCK workflow), nor does it cover refusal or recovery mechanics[cite: 1, 3].

2. Preconditions & Initial Boundary PRD: PB-STOR-001 / W002-UI-017

Scenario Host Context

A supported Chromium-based browser context exposing the File System Access API (showDirectoryPicker)[cite: 2, 3]. Core analytical execution is local-first and does not require a proprietary Red5Sorcery backend[cite: 1, 3]. Packaging details remain separate from core semantics[cite: 3].

Local Directory State

An accessible local folder chosen as the Suitcase, containing initially exactly one file: crime_stats_2025.csv (UTF-8, 100,000 logical records, multiline quoted descriptions, no byte-order anomalies)[cite: 1, 3].

Pre-Validation Gate

The browser front end has embedded the validated Help message catalog (v1) in #ds-help-catalog[cite: 1, 2]. The workspace remains locked against Work Order saving, Run, or analytical execution except Help and the Suitcase-selection control[cite: 2, 3].

3. Step-by-Step Governed Execution Walkthrough PRD: W002-STOR-022 / W002-ENC-001 / W002-MAT-002

Step 1: Suitcase Validation Gate
PHASE 0 · GATE

Analyst Action: The analyst clicks [Choose & validate Suitcase] and selects the folder containing crime_stats_2025.csv via the native folder picker[cite: 2].

Governed Engine Behavior (PRD: W002-STOR-022 through 022.3, W002-UI-017/018)[cite: 1, 3]:

  • Data Sculptor allocates a canonical physical filename matching __DS__CER__YYYYMMDDTHHMMSSmmmZ__<uuid-v4>.html[cite: 1].
  • The engine creates, completely writes, and successfully closes one governed CER HTML certificate in the root of the folder[cite: 1, 3].
  • The retained certificate proves that the canonical governed filename form was accepted and writable at that recorded UTC timestamp[cite: 1, 3]. Validation performs no validation-only deletion or rename[cite: 1, 3].
  • Limitation Boundary: The CER does not certify free disk capacity, future analytical success, or later SCR-to-final rename operations[cite: 1, 3].

Illustrative Observable Behavior: The Suitcase gate indicator shifts to validated, and directory/editor controls unlock[cite: 2, 3]. Exact visible colors and badge strings remain UI presentation evidence[cite: 3].

Step 2: Work Order Drafting (WIP State)
PHASE 1 · DRAFT

Analyst Action: In the Work Order editor, the analyst authors a declarative DS-STEPS specification and clicks [Validate draft][cite: 1, 2]:

# Establishing Crime Analytics Store
SUITCASE: .
WORK ORDER AS "Establish Crime Analytics 2025"
SOURCE "crime_stats_2025.csv"

KEEP
  HEADER "REF_DATE" AS "Reporting Year"
  HEADER "GEO" AS "Jurisdiction"
  COLUMN 7 AS "Violent Crime Rate"

MATERIALIZE

Governed Engine Behavior (PRD: W002-STEPS-001/001.1, W002-WKO-010, W002-UI-001)[cite: 1, 3]:

  • The engine parses the text: verifies SUITCASE: ., singular WORK ORDER AS, non-empty SOURCE, valid KEEP block with at least one selector, and terminal MATERIALIZE[cite: 1, 2].
  • Confirms that column ordinals are 1-based integers (≥ 1) and aliases are non-empty[cite: 1, 2].
  • Contract Invariant: Analytical intent is expressed strictly in Work Order text; point-and-click source or column selection is not authoritative[cite: 1, 3]. An editable draft is a non-executable WIP; the Run control remains disabled[cite: 1, 2, 3].
Step 3: Save Immutable Work Order Transaction
PHASE 2 · PUBLISH WKO

Analyst Action: The analyst clicks [Save NEW Work Order TXT] (or presses Ctrl+Shift+S)[cite: 2].

Governed Engine Behavior (PRD: W002-STOR-018.2/.3/.4, PB-STOR-013/014)[cite: 1, 3]:

  • The engine verifies that the non-empty alias "Establish Crime Analytics 2025" does not collide within the newest valid WKO MAP registry[cite: 1, 2, 3].
  • A single save-transaction UTC timestamp is established[cite: 1, 2, 3]. Data Sculptor persists the draft text to __DS__WKO__YYYYMMDDTHHMMSSmmmZ__<uuid-1>.txt[cite: 1, 2].
  • The complete successor WKO MAP is constructed under a governed scratch identity (SCR), closed, reopened, and fully validated against referenced visible WKO files[cite: 1, 3].
  • The validated candidate is promoted by host rename to __DS__MAP__YYYYMMDDTHHMMSSmmmZ__<uuid-2>.csv[cite: 1, 2, 3]. This rename is the alias-registry commit point[cite: 1, 3]. The WKO and successor WKO MAP share the transaction UTC and have distinct UUIDs[cite: 1, 3].
  • Failure Handling: If WKO MAP publication fails, the WKO is rolled back out of committed state where safely possible, or retained as attributable recovery evidence, and reported through Help[cite: 1, 3].

Illustrative Observable Behavior: The saved WKO becomes immutable and is loaded as the active Run target[cite: 1, 2, 3]. The Run button unlocks[cite: 2]. Editing this text immediately detaches it into a new non-executable WIP draft[cite: 1, 2, 3].

Step 4: Governed Run — Certification, Materialization & Publication
PHASE 3 · EXECUTE & COMMIT

Analyst Action: The analyst clicks [Run loaded saved Work Order] (or presses Ctrl+Enter)[cite: 2].

Governed Engine Behavior (PRD: W002-ENC-001..005, W002-MAT-001..003, W002-RID-002, W002-RCP-001..010)[cite: 1, 3]:

  • Stage 1 (Strict Whole-Source UTF-8 Certification): The engine streams crime_stats_2025.csv through bounded input windows using the ASCII-first strict inline validator (or an equivalent conforming implementation)[cite: 1, 3]. The 4 MiB WASM-001b window size is a proven reference point, not a frozen production constant[cite: 1, 3]. Certification must pass before any materialization begins[cite: 1]. The same run-local selected source instance remains bound across Stage 1 and Stage 2; Stage 2 must not re-resolve or substitute the source[cite: 1, 3].
  • Stage 2 (Single-Pass Structural Materialization): Stage 2 restarts at byte offset zero of that same source instance, consumes any offset-zero UTF-8 BOM as metadata, parses the header once, resolves all requested KEEP selectors, and traverses the logical data records in one single forward pass[cite: 1, 3]. Embedded newlines inside quoted fields are preserved within their logical record and do not advance row identity[cite: 1].
  • Bounded Staging & Candidate Creation: Because this run establishes a brand-new Store, the engine automatically generates the Store's permanent row identity (RID)[cite: 1, 3]. The engine incrementally writes four visible scratch candidate artifacts: one for the RID (1..N) and three for the requested columns (COL)[cite: 1, 3]. Memory and staging buffers remain strictly bounded; the SCR files themselves grow with output size[cite: 1, 3].
  • All-or-Refuse Publication Boundary: After every candidate reaches EOF, all four streams reconcile to exactly N = 100,000 accepted logical rows, and candidate validation passes[cite: 1, 3]:
    1. Data Sculptor allocates one publication-transaction UTC token[cite: 1, 3].
    2. The RID candidate is promoted first by host rename: SCR → __DS__RID__<UTC>__<uuid-R>.csv[cite: 1, 3].
    3. COL candidates are promoted in resolved KEEP order: SCR → __DS__COL__<UTC>__<uuid-C1..C3>.csv[cite: 1, 3]. Physical COL headers retain the exact decoded source headers; assigned AS aliases live in MAP metadata[cite: 1, 3].
    4. The complete 9-column successor Store MAP is written as SCR, closed, reopened, and fully validated[cite: 1, 3].
    5. The final SCR-to-MAP host rename occurs last and is the Store-state commit point[cite: 1, 3]. The newly published RID, COL, and Store MAP artifacts share the exact publication-transaction UTC[cite: 1, 3].
  • Automatic Run Receipt: Once the Store state is committed, Data Sculptor automatically publishes a line-oriented plain-text Receipt (__DS__RCP__<UTC_rcp>__<uuid-rcp>.txt) carrying its own creation timestamp, exact WKO lineage, certified byte count, logical row count, and RESULT: SUCCESS[cite: 1, 3]. (Refused runs also attempt a receipt; receipt-persistence failure is its own governed evidence failure)[cite: 1, 3].

4. Read-Only Execution Output Observation PRD: W002-UI-022 / W002-RCP-009

The read-only Execution Output surface reports run progression, stage facts, and resulting governed artifact identities[cite: 1, 2, 3]. The following stream is an illustrative, non-normative rendering and does not replace the authoritative WKO, Store MAP, Receipt, or canonical diagnostic event[cite: 1, 3]: 

> RUN __DS__WKO__20260922T213000100Z__3fa85f64-8c1b-4f9e-a7d2-1b8f3a9e2c4d.txt
WORK ORDER VALIDATED · OPERATION: MATERIALIZE
Source bound: crime_stats_2025.csv (run-local instance)
Stage 1 UTF-8 Certification: PASS (142,503,112 source bytes verified)
Stage 2 Single-Pass Materialization: 100,000 logical rows accepted
RID Published: __DS__RID__20260922T213005120Z__a1b2c3d4-1111-4000-8000-000000000001.csv (Action: CREATED)
COL Published (1/3): __DS__COL__20260922T213005120Z__e5f6a7b8-2222-4000-8000-000000000002.csv [Reporting Year]
COL Published (2/3): __DS__COL__20260922T213005120Z__c9d0e1f2-3333-4000-8000-000000000003.csv [Jurisdiction]
COL Published (3/3): __DS__COL__20260922T213005120Z__3a4b5c6d-4444-4000-8000-000000000004.csv [Violent Crime Rate]
Store MAP Committed: __DS__MAP__20260922T213005120Z__7e8f9a0b-5555-4000-8000-000000000005.csv
Receipt Published: __DS__RCP__20260922T213005850Z__f1e2d3c4-6666-4000-8000-000000000006.txt
EXECUTION COMPLETE · RESULT: SUCCESS

5. Visible Physical Artifact Ledger PRD: PB-STOR-001 / PB-STOR-006 / PB-STOR-013

Under this deliberately empty-start happy-path fixture—one source initially present, successful CER validation, one saved WKO/WKO MAP transaction, first materialization of three columns, successful Store MAP commit, and successful Receipt publication—the Suitcase ends with exactly ten visible files[cite: 3]. Real Suitcases may contain multiple Stores, historical snapshots, inventories, or recovery evidence[cite: 1, 3].

Data Sculptor intentionally persists project/analytical state only in the visible Suitcase; incidental browser/OS runtime caches outside Data Sculptor control are explicitly outside this contract[cite: 1, 3].

Canonical Filename Class Format Role & Governance Rules
crime_stats_2025.csv Input CSV Analyst-owned input file. Data Sculptor does not modify the source in place or rename it into __DS__[cite: 1, 3].
__DS__CER__...html CER HTML Retained Suitcase Validation Certificate. Proves folder writability at creation UTC; remains visible evidence[cite: 1, 2, 3].
__DS__WKO__...txt WKO TXT Authorizing Work Order artifact. Immutable text document; authoring timestamp remains its own[cite: 1, 3].
__DS__MAP__...csv (1) MAP CSV WKO Alias Registry Profile (2 columns: wko_alias,wko_filename). Promotion is save commit point[cite: 1, 3].
__DS__RID__...csv RID CSV Store Row Identity Spine (1 column, integers 1..100000 with UTF-8 BOM and LF endings). Generated once on Store establishment[cite: 1, 3].
__DS__COL__...csv (1) COL CSV Materialized column: REF_DATE. Header retains source text; Reporting Year alias recorded in MAP[cite: 1, 3].
__DS__COL__...csv (2) COL CSV Materialized column: GEO. Aligned to the 100,000 RID axis in exact logical record order[cite: 1, 3].
__DS__COL__...csv (3) COL CSV Materialized column: Col 7. Header retains source text; Violent Crime Rate alias recorded in MAP[cite: 1, 3].
__DS__MAP__...csv (2) MAP CSV Store MAP Profile (9 columns). Complete snapshot binding Store metadata, RID file, and COL mappings. Host rename from SCR is commit point[cite: 1, 3].
__DS__RCP__...txt RCP TXT Canonical Execution Receipt. Deterministic line-oriented KEY: VALUE UTF-8 text with LF endings; records exact WKO lineage and facts[cite: 1, 3].

6. Governed Architectural Invariants Illustrated Summary of Rev BL Candidate Invariants

1. Absolute Gate Precedence No Work Order saving, Run, or analytical processing occurs before successful CER validation; Help and Suitcase selection remain available[cite: 1, 2, 3].
2. Draft / WKO Separation Analytical intent lives in text. Editing occurs in non-executable WIP state; only saved immutable WKO artifacts can be invoked[cite: 1, 2, 3].
3. Implicit Row Spine The 1..N RID artifact is mandatory engine infrastructure created automatically on first Store materialization and reused on subsequent same-Store runs[cite: 1, 3].
4. Single-Pass Bounded Execution Stage 2 structural materialization processes multiple selected columns and RID in one single forward pass under bounded memory staging and backpressure[cite: 1, 3].
5. All-or-Refuse Publication Outputs stage as SCR candidates; RID promotes first, COLs in resolved KEEP order, and successor Store MAP promotes last as the sole Store-state commit point[cite: 1, 3].
6. Governed Visible Persistence No Data Sculptor-controlled project state is intentionally persisted outside the flat Suitcase. Project history remains fully inspectable by ordinary OS tools[cite: 1, 3].

7. Review Questions and Non-Blocking Follow-ons Categorized for Prioritization

In reconciling this scenario against Revision BL and Spock's audit, the following questions arose. They are categorized to distinguish true specification ambiguities from implementation policies, UX ideas, and documentation planning[cite: 3]:

  1. [Implementation Policy] WKO MAP Rollback Boundary on Interrupted Save (PRD: W002-STOR-018.3)[cite: 1, 3]:
    When saving a WKO, the draft WKO is persisted first, and then the successor WKO MAP candidate is validated and promoted[cite: 1, 3]. The PRD states that if MAP promotion fails, Data Sculptor must roll back the new WKO out of committed state where safely possible, or retain attributable SCR/RCV evidence[cite: 1, 3]. In a browser/WASM context using the File System Access API, should the engine actively attempt an immediate removeEntry() on the newly written WKO file if MAP commit fails, or should it immediately transition the WKO file into a tombstoned/recovery name to preserve forensics without partial state?
  2. [Specification/QA] Reconciling Store Source Filename vs. Explicit MAP Inheritance (PRD: W002-STOR-019.2/.3)[cite: 1, 3]:
    In automatic Store MAP inheritance, Data Sculptor matches the exact SOURCE filename against store_source_filename[cite: 1, 3]. If an analyst renames their source CSV externally, automatic inheritance treats this as a brand-new Store establishing a new RID[cite: 1, 3]. The PRD allows overriding this via MAP "__DS__MAP__..."[cite: 1, 3]. Does Rev BL intend an externally renamed source to be compatible with explicit historical MAP selection, and if so, what evidence establishes that compatibility? Right now the PRD does not justify teaching MAP selection as a rename-recovery technique.
  3. [UX Enhancement] Execution Output vs. Terminal Lineage Standards (PRD: W002-UI-022)[cite: 1, 3]:
    The prototype displays a live progress terminal[cite: 2]. Spock correctly emphasized that Execution Output is an observation surface rather than an authoritative ledger[cite: 3]. To avoid analyst confusion between what is displayed on screen and what is written to the RCP file, should the UI provide a direct one-click action to view the persisted RCP text file directly in the workspace, reinforcing that the file on disk is the authoritative record?
  4. [Documentation Planning] Structure of Subsequent Teachable Scenarios[cite: 3]:
    Spock recommended keeping this document strictly focused on First Materialization and creating a companion scenario for "Explainable Refusal & Help"[cite: 3]. Should Scenario #2 focus on a failed UTF-8 certification encountering Windows-1252 bytes and demonstrating the CVT conversion suggestion, or should it focus on a structural CSV syntax refusal (e.g., ragged columns or multiline quote exhaustion)[cite: 1, 3]?

8. Source References