Topazdocs
Build & Delivery

Toolchain Status

The current Topaz product identity, available delivery products, and decision-relevant limits.

Topaz 5.20.0 is the current product. The current language mode is topaz-5.20.

The source snapshot behind 5.20.0 is published at github.com/studiohaze/topaz as a frozen mirror. Development continues privately, and official releases are published on topaz.ooo and npm.

Confirm an installation with:

BASH
topaz version --verbose

The command reports the compiler, language mode, runtime, and Rust backend identity. Include its complete output when reporting an installation problem.

Choose the compiler

Topaz uses Rust Stage 0 as the default for every compiler-bearing command. The installed Stage 2 self compiler remains available when explicitly selected with --compiler self on its admitted current-mode routes:

BASH
topaz check app/main.tpz
topaz check app/main.tpz --compiler self
topaz test --root app --locked --compiler self
topaz emit app/main.tpz --compiler self
topaz emit app/main.tpz --target python --compiler self --out-dir python-source
topaz build app/main.tpz --compiler self --out-dir app-product
topaz build app/main.tpz --target web --compiler self --out-dir web-product
topaz dev --root web-app --locked --compiler self
topaz fmt app/main.tpz --compiler self
topaz lsp --compiler self
topaz doc --root app --locked --compiler self --out-dir api-docs
topaz compiler observe app/main.tpz --compiler self --out-dir observation
topaz check app/main.tpz --compiler rust
topaz compiler status --json

An omitted selection means that when --compiler is absent, Topaz runs Rust Stage 0 by default. Explicit self specifies --compiler self to invoke the installed Stage 2 self compiler on its admitted current-mode routes. Execution proceeds under the chosen compiler, with no automatic retry or silent fallback. The explicit self selection covers parse, dump-ast, check, current profiles, export reports, run, test, bench, locked packages, compiler observe, checked Rust and Python emit, native and Web-family builds, Web/service dev, formatting, LSP, and package documentation. One LSP process retains one compiler choice; restart the editor session after changing it.

Add --verbose to a compiler-bearing command to print whether the resolved selection was explicit, the current default, or compatibility.

Compiler intent is preserved until the effective language mode is known. For a package command, the CLI reads and validates only the root topaz.toml before choosing the compiler. Dependency resolution, lock verification, source loading, caches, and output work occur later. A package that selects an older language mode uses Rust when the selector is omitted or explicitly rust, while explicit self fails before that later work. This ordering keeps current-mode default selection and declared Rust compatibility as separate, observable decisions.

The installed process validates and decodes the exact immutable self-hosted compiler image once, then shares that prepared program across self compiler and LSP worker threads. Its cache key includes the image, source set, schema, runtime template, and toolchain identities. A mismatch stops self work. Target source, diagnostics, generated source, and managed outputs are calculated fresh for each request and are never reused as prepared compiler state. This preparation remains within documented cold-start, LSP, and memory limits.

compiler status --json is the machine-readable source of truth for the installed producer, program-image and source-set identities, supported routes, declines, each route's omitted choice, default, compatibility and recovery choices, and no-fallback policy. Managed artifacts also record whether the selection was explicit, the current default, or compatibility.

Installed recovery boundary

In the installed product, package checks, tests, builds, and source-free runs use the Rust Stage 0 default. Explicit self remains separately selectable by passing --compiler self on admitted current-mode routes. Rebuilding a managed output with Rust and self replaces compiler provenance atomically.

The same installed default completes the two-module learning application through check, its selected test, native release packaging, source removal, and execution. The field ledger closes only resolved product problems. Broader compiler and integration campaigns remain release-candidate checks.

The installed Stage 1 and Stage 2 paths can each produce a fresh validated observation from new source. An explicit self selection can then build and run a new source-free native product.

The exact npm installation carries both compiler choices in one executable and does not require a repository checkout. Products built with either choice run after their Topaz source is removed. If the installed self-hosted compiler image is missing or damaged, a self request fails without retrying or reusing a Rust result. Rerun the original command with --compiler rust only when you explicitly choose recovery. Rust remains usable independently, and the published Stage 0 recovery kit is a separate offline reconstruction path rather than an automatic fallback.

The installed self compiler manages a versioned, complete current-mode compilation product: ordered modules, tokens and AST, resolution and exports, typed profile diagnostics, lowering and runtime requirements, generated Rust, and exact invocation/result provenance. The host adapter validates and decodes those facts. It does not recreate a missing compiler decision. Shared package and runtime hosts consume the same validated product for admitted command routes.

Self-hosted Rust emit retains the fixed-point IR table and adds only the ordinary application host facade. A target-only shared runtime executes that table. It contains no compiler image and cannot run a Rust parser, resolver, checker, lowerer, or emitter. Native artifact manifests record the selected compiler, producer, compiler and target source sets, compilation product, generated-source digest, and targetCompilerFallback: false.

Available product paths

  • The CLI formats, checks, tests, runs, emits, and builds files or packages.
  • Native build creates a platform-specific executable.
  • Python build creates program.py with topaz_py_rt.py and requires Python 3.11 or newer at runtime.
  • Raw Web and Web Worker create WASM packages for an existing JavaScript host.
  • Web Application creates a complete managed static browser product.
  • The HTTP service creates a managed native HTTP/1.1 process with fixed listener, request, queue, deadline, log, and shutdown settings.
  • Package commands use manifests, locks, path dependencies, and verified local vendored dependencies.
  • topaz lispex embed run evaluates one Lispex request with declared resource limits using the evaluator built into the installed binary. topaz lispex embed info --json reports its component, profile, contract, runtime, and no-fallback identity.
  • The browser Playground provides check and interpreter execution without installing the native CLI.

Run the Lispex evaluator

The product takes four named regular files or paths. The output directory must not exist:

BASH
topaz lispex embed run \
  --source rule.lspx \
  --input value.lpxvalue \
  --limits limits.json \
  --output lispex-result

A complete evaluation writes result.lpxvalue and report.json. Deterministic semantic failure or limit exhaustion writes only report.json. Request refusal, contract failure, or engine failure leaves no output directory. Provisional result and transcript bytes never cross the boundary.

The evaluator, profile, runtime policy, and admission are fixed by the installed product. Evaluator and profile selectors, sidecar downloads, callbacks, imports, and fallback are absent. The report records Topaz execution and admission; the application artifact API produces portable Lispex-format cores. See Lispex Evaluator and LIT for the complete distinction.

Topaz exposes the locked std.lispex complete-current-profile API through the interpreter and all five native release targets. Follow Run Lispex Rules for the manifest, lock, build, artifact, and replay workflow. generated-python, raw-web, worker-web, managed-web, http-service, no-capability, and mcp-empty-component-set routes reject before artifact output or execution. Fallback is absent.

Inspect compiler observations

Capture an installed compiler observation

The installed CLI can capture the current checked compiler pipeline through generated Rust source as one canonical managed bundle:

BASH
topaz compiler observe --root my-app --locked --out-dir compiler-observation
topaz compiler validate compiler-observation

Put the output outside my-app. Otherwise a later observation can see its own managed directory while collecting package-directory facts. The bundle contains exact source bytes plus source-set, raw and layout token, AST, resolution, structured type, call, closure-capture, source-free lowering, runtime-leaf, generated-Rust, diagnostic, request, response, and provenance records. Treat it with the same confidentiality as the source package.

Validate the observation bundle

validate is read-only. It checks canonical encoding, schemas, ordering, cross-references, completeness, sizes, and digests without recompiling or reading the original source tree. Provenance identifies the producer as Rust Stage 0. That fact describes this observation path. It does not identify the implementation language of other compiler stages.

Use the explicit front-end preview

The explicit preview route runs the current Topaz-authored lexer, layout implementation, parser, logical import closure, name resolver, and static checker, hosted by Rust Stage 0:

BASH
topaz compiler preview --root my-app --locked --out-dir typed-preview
topaz compiler validate typed-preview

Its bundle ends at the typed phase and records engine: topaz-front-end-preview, producer/result stage 0/0, and Rust Stage 0 as its bootstrap host. The Topaz front end covers the current grammar, canonical AST, logical closure over package, mounted dependency, and standard modules, exact scopes, declarations, references and exports, static types, calls, captures, and structured diagnostics. Source facts, AST and typed nodes, depth, and shell exchange have documented ceilings. This route ends at the typed phase; ordinary compiler commands own lowering, emission, build, and execution, while the Stage 1 Preview is the separate route below. A preview failure stops without retrying target front-end work through Rust. Ordinary compiler commands follow the installed route-aware default.

Run the Stage 1 Compiler Preview

The installed product also exposes an explicit Stage 1 Compiler Preview:

BASH
topaz compiler preview main.tpz --producer stage1 --terminal rust-source --out-dir stage1-observation
topaz compiler validate stage1-observation

The Stage 1 Preview consumes the versioned exchange and source-free private IR produced by the Topaz-authored lowering and Rust emitter. It compiles an ordinary package without linking or retrying a Rust target parser, resolver, checker, lowerer, emitter, or interpreter. The managed bundle records producer/result stage 1/1, the compiler source-set identity, generated-source identity, runtime template, and targetCompilerFallback: false.

The regenerate_stage1_c1 release tool parses its CLI options once into typed producer modes and owned paths. It defaults to interpreted only when --producer is absent, requires an input manifest for linked-c1, and rejects unused manifests, missing values, duplicates, and unknown options before generation. Rust 1.96 release observations recorded Stage 0→1 as 98,301,332 bytes with null input SHA in 1,343.36 seconds and Stage 1→2 as 98,289,249 bytes with exact input-manifest and program-image SHA and targetCompilerFallback: false in 2,122.96 seconds.

The generate_stage2_r2 release tool now shares the generator CLI option, SHA-256, and atomic-write ownership used by regenerate_stage1_c1. It consumes the three required R2 paths in one typed pass and rejects missing values, duplicates, and unknown options before generation. One Rust 1.96 --release --locked observation produced 98,289,249 bytes in 2,087.51 seconds with exact input C2 manifest and program-image SHA, targetCompilerFallback: false, and generated Rust bytes and SHA matching the existing linked C2 product.

The retired run_stage1_c1_canary workspace binary had no live script, CI, or explicit Cargo-target consumer and duplicated a private 64-round fact loop, formatting-sensitive JSON provenance checks, response parsing, and SHA output. It has been removed; official C1 generation and comparison continue through regenerate_stage1_c1 and check_stage1_comparison, which consume the shared typed product path.

The live stage2_fixed_point_case verification entrypoint now accepts exactly one OS-native input path and rejects missing or extra input before source processing. Its host borrows the canonical source and creates owned source facts only when answering a request; the independent C1/C2 front-end agreement and output record remain unchanged.

The Stage 1 runtime build now reads operand, label, and module-operation arrays through borrowed exact-size iterators and serializes them directly, without three transient vectors used only to recover lengths. Compact-image bytes and C1/C2 identities remain the observed product boundary; this is a build-time ownership change, not a runtime performance claim.

The Stage 1 runtime manifest admission, C1 manifest producer, and installed C2 identity admission now derive the exact Rust toolchain value from their Cargo package metadata instead of maintaining three additional literals. Both packages inherit the workspace rust-version; sealed manifest values and bytes remain unchanged. This is a build-time metadata-ownership change, not a toolchain upgrade, support expansion, or general reproducibility claim.

The Stage 1 runtime build now carries its precomputed immutable C1/C2 program-image SHA-256 identities into the embedded compiler descriptor. Public image-identity helpers and product-manifest descriptors borrow those static values without rehashing complete images or allocating prefixed descriptor strings; independent C2 runtime admission hashing still rejects corruption and expected-identity drift before execution.

Pipeline placeholder planning now searches every executable stage argument-expression form while assigning each nested pipeline RHS to its own stage. Placeholder-free stages use inserted-lead; placeholder-bearing stages keep the written argument list and evaluate one pipe-lead bound to underscore, preserving argument shape, source order, closure capture, and nested-stage isolation across Stage 0, self-hosted lowering, the Stage 1 product runtime, generated Rust, and generated Python. This repairs existing SPEC section 11 behavior and extends only the internal call-evaluation vocabulary without changing a schema ID.

Keyword field roles after dot or safe-dot operators and before explicit record-field colons no longer open let, const, case, for, or concurrent construct state. Stage 0 and the live self-host layout source now preserve following multi-statement blocks and nested record-pattern braces while retaining genuine construct heads and all 21 existing keyword field names.

The Python differential harness now passes the separate Rust 1.96 strict-clippy workspace after removing two temporary String reborrows and simplifying one nested condition and one single-arm match. Product compiler, runtime, self-hosted source, and existing test behavior are unchanged.

Rust Stage 0 and the current live self-host parser source now apply the existing TPZ2012 binding-name rule to constant declarations, so const None: Option<int> = None is rejected at the None name. The sealed embedded self-compiler image is unchanged and remains historical until candidate regeneration.

Stage 0 Rust and the live self-host Topaz parser now detect duplicate selected imports by tracking seen exported-source and bound-local names across a single source-order scan rather than comparing all item pairs. This emits exactly one TPZ2011 diagnostic for each later offending spec in source-token order, preserves exported-source precedence on dual collisions, and retains existing diagnostic codes, messages, and token spans without regenerating the sealed embedded C2 image.

The topaz_parser crate now keeps parse_layout_tokens private, removing a consumerless public helper together with a self-comparison test that compared debug representations against themselves. parse_staged and its four stage records remain the public product boundary consumed by resolver and kernel observation paths. Product parsing, AST structures, diagnostics, self-host source, and language semantics are unchanged.

Stage 0 Rust and the live self-host resolver and checker now apply the existing initializer immediate/delayed boundary to conditional expression children. The left operand of &&, ||, and ?? and an optional-call receiver remain scanned for top-level forward references, while the short-circuit right operand and optional-call arguments are delayed because runtime evaluation can skip them. Reaching one before a referenced binding exists still produces the existing dynamic unbound fault. SPEC §17 and ADR-086 now name both families; this is a static over-rejection fix, not a new language feature or a runtime-safety guarantee.

The topaz_resolve crate and FileProvider interface now represent source loading through a typed SourceRead boundary (Present, Missing, Unreadable, InvalidUtf8), replacing the previous Option<String> folding that misreported unreadable files and invalid UTF-8 as TPZ3001 missing modules. Resolution and kernel host-fact replay now emit the existing TPZ3003 source-loader admission diagnostic for unreadable and non-UTF-8 inputs while centralizing physical read classification in the resolver. Diagnostic codes, schemas, AST structures, self-host source, and public release state are unchanged.

The parser corpus tests and repository-only CLI check-corpus command now consume one corpus-extract authority for the frozen 5.1 cardinalities and six-area 5.2 table. The redundant 5.2 area-list assertion is gone, and phase comments describe current parse, resolve, check, and exec ownership. Fixture bytes, product language behavior, and public release state are unchanged.

Understand the comparison boundary

The comparison boundary keeps semantic observations, diagnostics, generated source, generated-product behavior, and provenance separate. Rust Stage 0 and the Stage 1 Preview agree on the declared multi-module and rejected diagnostic cases. For the declared comparison product, the linked and interpreted Stage 1 paths produce identical generated Rust, while Stage 0 and Stage 1 generated Rust differ. The compiled Stage 1 product prints exactly 42. A wrong producer identity or damaged managed Stage 1 product is rejected instead of retrying the target through Stage 0.

Compare without a repository clone

The installed package can perform the exact comparison without a repository clone:

BASH
topaz compiler observe main.tpz --terminal rust-source --out-dir rust-observation
topaz compiler preview main.tpz --producer stage1 --terminal rust-source --out-dir stage1-observation
topaz compiler validate rust-observation
topaz compiler validate stage1-observation
topaz compiler compare --layer semantic rust-observation stage1-observation

After the two managed observations are written, validation and comparison no longer read the original target source. If Stage 1 declines or faults, it fails without writing a replacement observation and prints the separate compiler observe recovery command. It never runs that command implicitly. Use --compiler rust on an ordinary check or build command when you explicitly want Rust Stage 0.

Keep generated Rust provenance separate

Compiler-generated Rust keeps producer and build provenance outside its canonical source bytes. That separation lets later compiler stages compare generated source byte for byte without erasing who produced it. The installed Stage 1 command remains an explicit producer Preview and is separate from the ordinary route default.

Verify the generated compiler source

Using the public Rust Stage 0 implementation, the verifier compiles the 14 Topaz compiler source files into a fresh Rust source artifact. That artifact matches the SHA-256 digest and byte length recorded in the checked-in manifest and is byte-for-byte identical to the artifact produced by executing the checked-in program image with the public runtime on the same source set and compilation request. The verifier also re-extracts the program image from the fresh Stage 0 output and verifies it byte-for-byte against the checked-in image.

The checked-in program image is itself a derived artifact, so these checks establish a content-addressed generated-source identity and cross-route round-trip consistency; they do not establish an independently derived C2 or a Stage-2 fixed point.

Stage-2 claim correction record

Choose a comparison boundary

Compare two complete observations by selecting the boundary that answers your question:

BASH
topaz compiler compare --layer semantic observation-a observation-b
topaz compiler compare --layer generated-source observation-a observation-b
topaz compiler compare --layer provenance observation-a observation-b
topaz compiler compare --layer native-binary program-a program-b

The command prints one canonical JSON record within the documented size limit and returns a failing exit status when that layer differs. Semantic comparison stops at the first different compiler phase. Generated source and provenance remain independent, so a producer identity change does not by itself become a language mismatch.

Recover the Rust Stage 0 seed

A qualifying self-hosting release carries a separate recovery manifest and deterministic source archive alongside the five normal platform binaries. The archive fixes the Rust compiler source, lockfile, vendored dependencies and licenses, runtime templates, protocol schemas, compiler workload, and reconstruction tools. It also records the embedded Topaz-authored compiler files as an independently verified source set. Rebuilding it twice produces identical archive bytes.

The recovery set does not bundle the Rust toolchain. Reconstruction requires the pinned Rust toolchain to be installed already and then builds offline from the archived vendor directory. This is an independent recovery and provenance route, not the normal installer path, and it does not make the Rust recovery chain self-hosted.

Check bootstrap-safe compiler source

The Bootstrap Profile is a machine-readable restriction of the current language for compiler-kernel source. It adds no syntax or dialect. It accepts a locked deterministic package only when its resolved operations avoid ambient host capabilities, extern modules, floating point, concurrency, resources, testing APIs, and other host-dependent leaves:

BASH
topaz check --profile bootstrap --locked --root compiler-kernel

Human diagnostics include a stable bootstrap/* rule. With --format json, stderr contains machine-readable profile diagnostics and stdout contains one summary. Resolved identity classifies a user function named print as a local value. Renamed or aliased forbidden host operations retain their host identity.

Decision-relevant limits

All targets share parsing, module resolution, and static checking, but host-dependent operations are not universally available. A target that cannot preserve an operation must reject it before writing a product. In particular, generated Python does not support the current fixed-Huffman DEFLATE, fixed zlib, or RS(255,223) helpers. Their generation fails explicitly. Consult the backend pages before choosing a target for binary-media work.

The HTTP product implements managed HTTP/1.1 with fixed listener, request, queue, deadline, log, and shutdown controls. TLS, HTTP/2, WebSocket, outgoing network access, ambient environment authority, shared mutable Topaz state, and a general Web framework are outside this product.

The Web Application host uses declared browser capabilities. Raw Web and Worker products require an existing JavaScript host. Playground runs source in the browser; validate deployment with the generated product outside the source workspace.

Concurrent task ordering is unspecified. Join tasks before ordered output.

The installed evaluator handles topaz lispex embed, while Topaz target selection is separate and --target lispex rejects before artifact output. LIT is the Lispex test and integration surface from one source lineage and remains separate from the installed evaluator and output backends.

Start with fmt --check, check, and test. Use run for a command application or dev for a Web Application or HTTP service. Build only the target you intend to deliver and copy its complete managed directory.