Topaz 6.0
Topaz is a general-purpose programming language with typesetting built in. Code that does the same job looks the same no matter who writes it.
591rules15chapters340APIs in 19 modules109diagnostics1,332examples
Tour
Know another language? Read these in order.
import std.ioimport std.bigint { BigInt }let labels = ["12", "cover", "14"]let pages = labels.map((text) => match (BigInt.parse(text, 10)) { case Some(number) => number.toInt() case None() => None()})io.writeLine("{pages}")[Some(12), None, Some(14)]
map goes after the list. BigInt.parse reads each string, and toInt converts a parsed number. The entry for “cover” becomes None.
Who, for what
Teams building a book typesetting engine
Build an engine that takes a manuscript and decides where lines and pages fall. Paragraph breaking and the Korean, Japanese and Chinese line-break rules come from the standard library. The same line-breaking code runs in the browser preview and in print composition.
Teams whose rules differ by publisher
Publisher decisions such as where a chapter starts live in Lispex rule files. When each client wants its own style, the line-breaking app code stays the same.
Teams building web tools
Write a function that handles input in Topaz, build it as a browser package and call it from JavaScript. The browser tools on jeni.kr work this way.
std.typeset
std.typeset in Topaz 6.0 is the standard module that breaks paragraphs into lines. The program below sets one Korean sentence at a width of ten Hangul syllables. Looking at the whole paragraph keeps a lone word off the last line.
import std.ioimport std.typeset { BreakCluster, BreakPolicy, BreakMode, KinsokuRules, BreakOpportunity, Paragraph, Item, Box, Penalty, BreakRule, DistanceGrid, CostModel, LineGeometry, JustificationPolicy, JustificationMode, KnuthPlassPolicy, ParagraphPlan, classifyBreaks, breakParagraph,}let sentence = "문단 전체를 보고 줄을 나누면 마지막 줄이 짧게 남지 않고 줄마다 길이가 고르게 됩니다."let clusters = sentence.scalars().map(cluster)let result = match (classifyBreaks(clusters, koreanRules(clusters))) { case Ok(breaks) => breakParagraph(paragraph(clusters, breaks)) case Err(error) => Err(error)}match (result) { case Ok(plans) => { io.writeLine("Whole paragraph\n{linesOf(clusters, plans.optimized)}\n") io.writeLine("Line by line\n{linesOf(clusters, plans.greedy)}") } case Err(error) => io.writeLine("{error}")}Whole paragraph 문단 전체를 보고 줄을 나누면 마지막 줄이 짧게 남지 않고 줄마다 길이가 고르게 됩니다. Line by line 문단 전체를 보고 줄을 나누면 마지막 줄이 짧게 남지 않고 줄마다 길이가 고르게 됩니다.
The main part of the program. The full source adds the helpers that set character widths and build the paragraph.
import std.ioimport std.typeset { BreakCluster, BreakPolicy, BreakMode, KinsokuRules, BreakOpportunity, Paragraph, Item, Box, Penalty, BreakRule, DistanceGrid, CostModel, LineGeometry, JustificationPolicy, JustificationMode, KnuthPlassPolicy, ParagraphPlan, classifyBreaks, breakParagraph,}const lineWidth = 100pt// A measured Hangul syllable is 10pt wide and a space is 2.5pt.function cluster(text: string) -> BreakCluster { BreakCluster { text: text, advance: if (text == " ") { 2.5pt } else { 10pt }, trackingAfter: 0pt }}// Unicode line breaking (UAX #14) plus the Korean line-breaking rules (KLReq). A word that fits on a line stays whole.function koreanRules(clusters: Array<BreakCluster>) -> BreakPolicy { BreakPolicy { mode: BreakMode.Word(), rules: KinsokuRules.Unicode16Ko(), forbidBefore: [], forbidAfter: [], boundaryAllowed: clusters.map((c: BreakCluster) => true), generatedHyphenEnds: [], protectFittingLatin: false, preserveShortWords: false, narrowestWordTarget: lineWidth, bodyWordTarget: lineWidth }}// One box per cluster and a zero penalty at each break opportunity. The last opportunity always ends the line.function paragraph(clusters: Array<BreakCluster>, breaks: Array<BreakOpportunity>) -> Paragraph { let boxes: Array<Item> = [for index in (0..<clusters.length) => { let c = clusters[index] Item.BoxItem(Box { unitIndex: index, text: c.text, width: c.advance, trackingAfter: 0pt }) }] let penalties: Array<Item> = [for opportunity in (breaks) => { let last = opportunity.unitEnd == clusters.length Item.PenaltyItem(Penalty { unitEnd: opportunity.unitEnd, width: 0pt, kind: opportunity.kind, flagged: false, rule: if (last) { BreakRule.Forced() } else { BreakRule.Allowed(0) } }) }] Paragraph { grid: DistanceGrid.MilliPoint(), items: [...boxes, ...penalties], costModel: CostModel.FixedPoint(), geometry: LineGeometry { bodyTarget: lineWidth, firstTarget: lineWidth, dropCapSpan: 0, dropCapInset: 0pt }, justification: JustificationPolicy { mode: JustificationMode.Unjustified(), minGapCount: 0, minLineWidthRatioM: 0, minFinalLineWidthRatioM: None(), maxAdjustment: 0pt, maxGapAdjustment: 0pt, maxGapRatioR: 0, latinSpacing: None(), cjkStretchRatio: 0.0 }, knuthPlass: KnuthPlassPolicy { toleranceR: 0, linePenalty: 0, fitnessDemerit: 0, flaggedDemerit: 0, finalFlaggedDemerit: 0 } }}// Join the clusters of each line and drop the space at the end of the line.function linesOf(clusters: Array<BreakCluster>, plan: ParagraphPlan) -> string { let mut text = "" for lineIndex in (0..<plan.lines.length) { if (lineIndex > 0) { text = text + "\n" } let line = plan.lines[lineIndex] for unit in (line.start..<line.trimmedEndExclusive) { text = text + clusters[unit].text } } text}let sentence = "문단 전체를 보고 줄을 나누면 마지막 줄이 짧게 남지 않고 줄마다 길이가 고르게 됩니다."let clusters = sentence.scalars().map(cluster)let result = match (classifyBreaks(clusters, koreanRules(clusters))) { case Ok(breaks) => breakParagraph(paragraph(clusters, breaks)) case Err(error) => Err(error)}match (result) { case Ok(plans) => { io.writeLine("Whole paragraph\n{linesOf(clusters, plans.optimized)}\n") io.writeLine("Line by line\n{linesOf(clusters, plans.greedy)}") } case Err(error) => io.writeLine("{error}")}Whole paragraph 문단 전체를 보고 줄을 나누면 마지막 줄이 짧게 남지 않고 줄마다 길이가 고르게 됩니다. Line by line 문단 전체를 보고 줄을 나누면 마지막 줄이 짧게 남지 않고 줄마다 길이가 고르게 됩니다.
Get the breaks chosen for the whole paragraph together with a greedy, line-by-line result. Choose the Knuth-Plass cost model or a fixed-point cost model.
Find break opportunities with the Unicode 16.0 line breaking algorithm (UAX #14).
Add the Korean (KLReq), Japanese (JLReq), or Chinese (CLReq) rules, and an opening bracket never ends a line while a closing bracket or period never starts one. The Japanese rules follow the JLReq character classes.
The Unicode East Asian Width property decides breaks around quotation marks and brackets next to CJK text.
Set how far justification may stretch or shrink the space between Latin words, and how far it may stretch the space between CJK characters.
Write lengths such as 12pt, 3mm, 1cm, 1in, and 1em directly in code and convert between units.
ATLAS
ATLAS turns a manuscript into print-ready book pages. Here is a page it set, with two of its line-breaking predicates rewritten for Topaz 6.0. ATLAS names these functions with Korean typesetting terms, shown here in English.
function nextIsNotLatinSpace(next: Option<string>) -> bool { match (next) { case Some(text) => !isLatinSpace(text) case None() => true }}function nextHasNoLineStartBan(next: Option<string>) -> bool { match (next) { case Some(text) => !hasLineStartBan(text) case None() => true }}false true false true
Two ATLAS line-breaking predicates rewritten for Topaz 6.0, with names translated into English. The full sample adds small policy helpers and inputs.
import std.io// Small sample policies for the two predicates below.function isLatinSpace(text: string) -> bool { text == " "}function hasLineStartBan(text: string) -> bool { text == "."}function nextIsNotLatinSpace(next: Option<string>) -> bool { match (next) { case Some(text) => !isLatinSpace(text) case None() => true }}function nextHasNoLineStartBan(next: Option<string>) -> bool { match (next) { case Some(text) => !hasLineStartBan(text) case None() => true }}io.writeLine("{nextIsNotLatinSpace(Some(" "))} {nextIsNotLatinSpace(None())}")io.writeLine("{nextHasNoLineStartBan(Some("."))} {nextHasNoLineStartBan(None())}")false true false true
Why Topaz handles typesetting well
Common tasks have one way to write them. Whether a person or an agent writes the code, it takes the same shape, so a review only has to follow what the code does.
Converting a number to text and collection operations are methods on the value. Use map on an array and toString on a number. Format the source with topaz fmt.
Collection operationsA value that may be missing is an Option, an operation that may fail returns a Result, and cleanup is written with defer. At the end of a paragraph there is no next character, so a line-break rule receives None rather than an error.
Null, Option, Result & FaultsPrinting to the screen and reading files both use a standard module you import. A reviewer finds every point of input and output from the imports.
Standard Library OverviewPut a fixed set of cases, such as line-break classes, in an enum, and the compiler checks that each match handles every one. When you add a case later, the build reports each match that does not handle it yet.
Patterns & Control FlowAn int is a signed 64-bit integer, and overflow stops the program at that point instead of continuing with a wrong value. When you measure line widths and layout costs in integers, an overflow cannot hide in the result.
Runtime BehaviorDirect runs, native executables and browser packages follow the same language rules. So the on-screen preview and the print layout break lines at the same places.
Execution TargetsLispex
Computing such as line breaking happens in Topaz code. Decisions that change with each publisher and book, such as which page a chapter starts on or how many lines may stand alone, go into separate Lispex rules. Topaz 6.0 runs Lispex 2.0 rules through std.lispex with separate limits for preparation, value conversion and evaluation. Change a rule source and prepare it again, and the same manuscript takes another style without any change to the application code.
(define rule (lambda (input) (let ((min-lines 2)) (if (< input min-lines) "next-page" "keep"))))All of lispex-en.scm
import std.ioimport std.bigint { BigInt }import std.lispex { LispexPackage, LispexHostLimits, LispexPreparationLimits, LispexSource, LispexSourceBundle, PreparedLispexRule, LispexDatumView, LispexValue, applicationPackage, defaultLimits, prepare, valueFromDatum, valueView, evaluate,}export function main(args: Array<string>, stdin: string) -> Result<int, string> { let application = packageOf()? let preset = defaultLimits() let rule = ruleFrom(application, stdin, preset.preparation)? for linesLeft in ([1, 2]) { let input = numberOf(application, "{linesLeft}", preset.host)? let evaluation = match (evaluate(rule, input, preset.evaluation)) { case Ok(value) => value case Err(_) => { return Err("The evaluation could not complete.") } } match (evaluation.settlement) { case Complete(result) => io.writeLine("lines left {linesLeft} {textOf(result, preset.host)?}") case SemanticFailure(_) => { return Err("The rule returned a language failure.") } case LimitExhaustion(_) => { return Err("The rule reached its evaluation limit.") } } } return Ok(0)}lines left 1 next-page lines left 2 keep
The main part of lispex-en.tpz. The full source includes the preparation and value helpers.
import std.ioimport std.bigint { BigInt }import std.lispex { LispexPackage, LispexHostLimits, LispexPreparationLimits, LispexSource, LispexSourceBundle, PreparedLispexRule, LispexDatumView, LispexValue, applicationPackage, defaultLimits, prepare, valueFromDatum, valueView, evaluate,}function packageOf() -> Result<LispexPackage, string> { match (applicationPackage()) { case Ok(value) => Ok(value) case Err(_) => Err("The application package was not admitted.") }}function ruleFrom(application: LispexPackage, source: string, limits: LispexPreparationLimits) -> Result<PreparedLispexRule, string> { let bundle = LispexSourceBundle { entryModule: "widow-lines.scm", entryBinding: "rule", sources: [LispexSource { name: "widow-lines.scm", bytes: source.encodeUtf8() }], } match (prepare(application, bundle, limits)) { case Ok(rule) => Ok(rule) case Err(_) => Err("The rule could not be prepared.") }}function numberOf(application: LispexPackage, text: string, limits: LispexHostLimits) -> Result<LispexValue, string> { let numerator = match (BigInt.parse(text, 10)) { case Some(value) => value case None() => { return Err("The input must be an integer.") } } let denominator = match (BigInt.parse("1", 10)) { case Some(value) => value case None() => { return Err("The denominator must be an integer.") } } match (valueFromDatum(application, LispexDatumView.Number(numerator, denominator), limits)) { case Ok(value) => Ok(value) case Err(_) => Err("The input value was not admitted.") }}function textOf(value: LispexValue, limits: LispexHostLimits) -> Result<string, string> { match (valueView(value, limits)) { case Ok(Text(text)) => Ok(text) case _ => Err("The rule must return text.") }}export function main(args: Array<string>, stdin: string) -> Result<int, string> { let application = packageOf()? let preset = defaultLimits() let rule = ruleFrom(application, stdin, preset.preparation)? for linesLeft in ([1, 2]) { let input = numberOf(application, "{linesLeft}", preset.host)? let evaluation = match (evaluate(rule, input, preset.evaluation)) { case Ok(value) => value case Err(_) => { return Err("The evaluation could not complete.") } } match (evaluation.settlement) { case Complete(result) => io.writeLine("lines left {linesLeft} {textOf(result, preset.host)?}") case SemanticFailure(_) => { return Err("The rule returned a language failure.") } case LimitExhaustion(_) => { return Err("The rule reached its evaluation limit.") } } } return Ok(0)}lines left 1 next-page lines left 2 keep
Prepare a rule with prepare and run it with evaluate.
View LispexBuilt with Topaz
The core of the Topaz compiler is written in Topaz. Browser tools are written in the same language.
function assignmentRoot(node: SyntaxNode) -> Option<SyntaxNode> { if (node.kind == "expression/identifier") { return Some(node) } match (child(node, "object")) { case Some(object) => assignmentRoot(object) case None() => None() }}expression/identifier None
The assignment-root walk from compiler/crates/topaz_self_frontend/topaz/checker/expression_core.tpz rewritten for Topaz 6.0
import std.io// These small inputs exercise the traversal independently of compiler internals.record SyntaxNode { kind: string }function child(node: SyntaxNode, name: string) -> Option<SyntaxNode> { if (node.kind == "expression/member" && name == "object") { Some(SyntaxNode { kind: "expression/identifier" }) } else { None() }}function assignmentRoot(node: SyntaxNode) -> Option<SyntaxNode> { if (node.kind == "expression/identifier") { return Some(node) } match (child(node, "object")) { case Some(object) => assignmentRoot(object) case None() => None() }}match (assignmentRoot(SyntaxNode { kind: "expression/member" })) { case Some(root) => io.writeLine(root.kind) case None() => io.writeLine("missing")}io.writeLine("{assignmentRoot(SyntaxNode { kind: "expression/literal" })}")expression/identifier None
Code written in Topaz checks names and types and generates the Rust. Use topaz check to check a program and topaz build to create an executable.
Built with TopazThe JSON formatter on jeni.kr tidies input with a parser written in Topaz, and reports the line and column when a closing bracket is missing.
Call it from JavaScriptExplore Topaz
See what Topaz is designed for, what its code looks like, and where it fits before writing your first program.
Meet TopazStart with one small program and continue step by step to a checked, tested, two-module application.
Start learningBreak a paragraph into lines with std.typeset. Set widths such as 12pt or 3mm, the line breaking rules of each language, and the spacing of justified lines.
Open the typesetting referenceStart from the syntax map, then look up language rules, the standard library, and the tools.
Open the syntax map