Skip to content
Topaz6.0 · Before release
Download

Search documentation

Specification 6.0 · Not released · Tested product 5.21.3

Use arrow keys to choose a result. Press Enter to open.

Type an API name, diagnostic number, rule, or project topic.

    Library

    std.typeset.classifyBreaks

    Classifies atomic measured clusters with deterministic boundary order.

    classifyBreaks(clusters:Array<BreakCluster>, policy:BreakPolicy) -> Result<Array<BreakOpportunity>, TypesetError>
    Receiver
    Not applicable
    Minimum arity
    2
    Maximum arity
    2

    Declaration and parameters

    Link

    Availability

    Included by default: Pending

    Import std.typeset in language mode 5.21.

    The current compiler selects std/typeset.tpz for this language mode. Availability in other released editions and inclusion in each installed distribution are Pending.

    Source evidence

    Receiver

    Parameter constraints

    BreakKind.CjkCluster() constructs the variant for the existing CJK and East Asian cluster classification. BreakKind.KoreanCluster() is absent and rejected.

    Renaming this variant preserves the classification predicates and the effect of the selected Unicode, locale, custom punctuation, and shaping-boundary policies.

    A CjkCluster category does not itself permit a break. classifyBreaks returns the opportunities admitted by its supplied policy.

    For the same ordered input values, policies, calculation grid, and product release, classifyBreaks and breakParagraph produce the same result. This includes boundary order, selected breaks, line widths, costs, and the returned error variant.

    Planning reads measured text and policy values without changing caller arrays. It does not shape text, load fonts, or write output. Ordinary argument evaluation finishes before the operation begins.

    The selected cost model, admission rules, and search bounds define plan quality. The two reported plans expose their own costs for inspection.

    classifyBreaks reads measured clusters in array order and returns Result<Array<BreakOpportunity>, TypesetError>. It does not shape text, normalize strings, split a cluster, or change any supplied record or array.

    Each BreakCluster.text is the scalar string for one indivisible measured unit. Empty text still occupies one cluster index. advance includes trackingAfter. A span width sums its advances and removes the trackingAfter of its final cluster.

    advance and advance minus trackingAfter must be nonnegative point lengths. Zero advance and signed trackingAfter are accepted. Invalid extents return TypesetError.InvalidInput. Length arithmetic follows the common Length contract.

    classifyBreaks sums stored point ticks and compares them with the stored point targets. It has no DistanceGrid argument and does not quantize to a paragraph grid. Exact equality with a target counts as fitting.

    boundaryAllowed has exactly one entry for each cluster. Entry i governs the boundary after cluster i. The last entry must be true for nonempty input. A wrong mask length or a false last entry returns TypesetError.InvalidInput.

    Validation checks mask length and terminal permission, word targets, forbidBefore, forbidAfter, generated hyphen indices, and measured extents in that order. Array checks visit entries in input order. Unicode mandatory crossings are checked before boundary conflicts are visited in increasing cluster order. An empty result follows successful structural and policy validation.

    An empty cluster array with a valid empty boundary mask returns Ok with an empty opportunity array. It does not create a synthetic unit or terminal opportunity.

    Successful nonempty results contain increasing, unique unitEnd values from one through the cluster count. unitEnd is an exclusive cluster index rather than a byte or scalar offset. The final opportunity has unitEnd equal to the cluster count and kind ForcedEnd.

    Repeated calls with equal cluster values and equal policies produce the same ordered opportunities and categories. Validation and classification do not read host locale, perform I/O, invoke caller callbacks, or mutate input values. Argument expressions use the ordinary language evaluation order.

    A BreakOpportunity carries a boundary and its category. It does not carry a penalty width or BreakRule. A caller preserving mandatory boundaries maps every ForcedEnd opportunity to a Forced penalty before calling breakParagraph.

    Result

    Effects

    Failures

    Result and effects

    Link

    The function validates its measured clusters and explicit break policy, then returns ordered opportunities. Empty input has an empty result. Nonempty successful results end with ForcedEnd. Invalid input and policy values are reported through TypesetError without mutating the supplied values.

    Why

    Link

    Measured clusters retain shaping boundaries while explicit Unicode and locale policies choose line-break opportunities. Their separation lets the same paragraph model serve Chinese, Japanese, Korean, and mixed text without an implicit language-dependent category name.

    Related entries

    Link

    std.typeset

    Name the CJK cluster break category

    Link
    Ruletopaz.stdlib.contract.cjk-cluster.rule

    Name the CJK cluster break category

    BreakKind.CjkCluster() constructs the variant for the existing CJK and East Asian cluster classification. BreakKind.KoreanCluster() is absent and rejected.

    Renaming this variant preserves the classification predicates and the effect of the selected Unicode, locale, custom punctuation, and shaping-boundary policies.

    A CjkCluster category does not itself permit a break. classifyBreaks returns the opportunities admitted by its supplied policy.

    Why

    A break category names the text it classifies. CjkCluster covers the existing Chinese, Japanese, and Korean classification path and matches the module's CjkInterCharacter vocabulary. Locale policy continues to determine where a break is allowed.

    Source evidence

    Use the CJK cluster variant

    Link
    main.tpzUse the CJK cluster variantOpen in Playground
    1. import std.typeset as ts
    2. let kind: ts.BreakKind = ts.BreakKind.CjkCluster()
    Expected outputexit 0
    No stdout.
    $ topaz check main.tpz --format json · target product 5.21.3
    Expected result

    Expected result

    Checking accepts the program. Exit code 0.

    Source evidence

    Reject the former KoreanCluster name

    Link
    main.tpzReject the former KoreanCluster nameOpen in Playground
    1. import std.typeset as ts
    2. let kind = ts.BreakKind.KoreanCluster()
    Expected outputNonzero exit
    No stdout.
    $ topaz check main.tpz --format json · target product 5.21.3
    Expected result

    Expected result

    Rejection during check. Nonzero exit.

    • TPZ5006 stdlib-removed-korean-cluster at main.tpz:2:12
    Source evidence

    Diagnostics

    Link

    Unknown member or field

    Repeat a paragraph plan deterministically

    Link
    Ruletopaz.stdlib.contract.typeset-determinism.rule

    Repeat a paragraph plan deterministically

    For the same ordered input values, policies, calculation grid, and product release, classifyBreaks and breakParagraph produce the same result. This includes boundary order, selected breaks, line widths, costs, and the returned error variant.

    Planning reads measured text and policy values without changing caller arrays. It does not shape text, load fonts, or write output. Ordinary argument evaluation finishes before the operation begins.

    The selected cost model, admission rules, and search bounds define plan quality. The two reported plans expose their own costs for inspection.

    Why

    Repeatable paragraph layout lets applications cache plans and reproduce published pages. An explicit search contract separates reproducibility from the quality of a chosen plan.

    Source evidence

    Retain a deterministic choice among equal cost candidates

    Link
    main.tpzRetain a deterministic choice among equal cost candidatesOpen in Playground
    1. import std.typeset as ts
    2. import std.io
    3. import std.test
    4. let items: Array<ts.Item> = [
    5. ts.Item.BoxItem(ts.Box { unitIndex: 0, text: "word", width: 10pt, trackingAfter: 0pt }),
    6. ts.Item.PenaltyItem(ts.Penalty { unitEnd: 1, width: 0pt, rule: ts.BreakRule.Allowed(0), kind: ts.BreakKind.LatinSpace(), flagged: false }),
    7. ts.Item.PenaltyItem(ts.Penalty { unitEnd: 1, width: 0pt, rule: ts.BreakRule.Allowed(0), kind: ts.BreakKind.ClosingPunctuation(), flagged: false })
    8. ]
    9. let input = ts.Paragraph {
    10. grid: ts.DistanceGrid.MilliPoint(),
    11. items: items,
    12. geometry: ts.LineGeometry { bodyTarget: 10pt, firstTarget: 10pt, dropCapSpan: 0, dropCapInset: 0pt },
    13. justification: ts.JustificationPolicy {
    14. mode: ts.JustificationMode.Unjustified(), minGapCount: 0,
    15. minLineWidthRatioM: 0, minFinalLineWidthRatioM: None(),
    16. maxAdjustment: 0pt, maxGapAdjustment: 0pt, maxGapRatioR: 0,
    17. latinSpacing: None(), cjkStretchRatio: 0.0
    18. },
    19. costModel: ts.CostModel.FixedPoint(),
    20. knuthPlass: ts.KnuthPlassPolicy { toleranceR: 100000000, linePenalty: 10, fitnessDemerit: 0, flaggedDemerit: 0, finalFlaggedDemerit: 0 }
    21. }
    22. function category(kind: ts.BreakKind) -> string {
    23. match (kind) {
    24. case LatinSpace() => "space"
    25. case ClosingPunctuation() => "punctuation"
    26. case _ => "unexpected kind"
    27. }
    28. }
    29. for pass in ([0, 1]) {
    30. let result = test.assertOk(ts.breakParagraph(input))
    31. let optimized = result.optimized.lines[0]
    32. let greedy = result.greedy.lines[0]
    33. io.writeLine("{pass} {optimized.start} {optimized.endExclusive} {category(optimized.kind)} {category(greedy.kind)}")
    34. }
    Expected outputexit 0
    0 0 1 space punctuation
    1 0 1 space punctuation
    
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    Expected stdout
    1. 0 0 1 space punctuation
    2. 1 0 1 space punctuation
    Source evidence

    Report the same first failing stage for repeated invalid input

    Link
    main.tpzReport the same first failing stage for repeated invalid inputOpen in Playground
    1. import std.typeset as ts
    2. import std.io
    3. let items: Array<ts.Item> = [
    4. ts.Item.BoxItem(ts.Box { unitIndex: 2, text: "a", width: 1pt, trackingAfter: 0pt })
    5. ]
    6. let input = ts.Paragraph {
    7. grid: ts.DistanceGrid.MilliPoint(),
    8. items: items,
    9. geometry: ts.LineGeometry { bodyTarget: 0pt, firstTarget: 0pt, dropCapSpan: 0, dropCapInset: 0pt },
    10. justification: ts.JustificationPolicy {
    11. mode: ts.JustificationMode.Unjustified(), minGapCount: 0,
    12. minLineWidthRatioM: 0, minFinalLineWidthRatioM: None(),
    13. maxAdjustment: 0pt, maxGapAdjustment: 0pt, maxGapRatioR: 0,
    14. latinSpacing: None(), cjkStretchRatio: 0.0
    15. },
    16. costModel: ts.CostModel.FixedPoint(),
    17. knuthPlass: ts.KnuthPlassPolicy { toleranceR: 100000000, linePenalty: 10, fitnessDemerit: 0, flaggedDemerit: 0, finalFlaggedDemerit: 0 }
    18. }
    19. for pass in ([0, 1]) {
    20. match (ts.breakParagraph(input)) {
    21. case Err(InvalidPolicy(_)) => io.writeLine("{pass} InvalidPolicy")
    22. case _ => io.writeLine("unexpected result")
    23. }
    24. }
    Expected outputexit 0
    0 InvalidPolicy
    1 InvalidPolicy
    
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    Expected stdout
    1. 0 InvalidPolicy
    2. 1 InvalidPolicy
    Source evidence

    Classify measured clusters into an ordered boundary sequence

    Link
    Ruletopaz.stdlib.contract.typeset-break-inputs.rule

    Classify measured clusters into an ordered boundary sequence

    classifyBreaks reads measured clusters in array order and returns Result<Array<BreakOpportunity>, TypesetError>. It does not shape text, normalize strings, split a cluster, or change any supplied record or array.

    Each BreakCluster.text is the scalar string for one indivisible measured unit. Empty text still occupies one cluster index. advance includes trackingAfter. A span width sums its advances and removes the trackingAfter of its final cluster.

    advance and advance minus trackingAfter must be nonnegative point lengths. Zero advance and signed trackingAfter are accepted. Invalid extents return TypesetError.InvalidInput. Length arithmetic follows the common Length contract.

    classifyBreaks sums stored point ticks and compares them with the stored point targets. It has no DistanceGrid argument and does not quantize to a paragraph grid. Exact equality with a target counts as fitting.

    boundaryAllowed has exactly one entry for each cluster. Entry i governs the boundary after cluster i. The last entry must be true for nonempty input. A wrong mask length or a false last entry returns TypesetError.InvalidInput.

    Validation checks mask length and terminal permission, word targets, forbidBefore, forbidAfter, generated hyphen indices, and measured extents in that order. Array checks visit entries in input order. Unicode mandatory crossings are checked before boundary conflicts are visited in increasing cluster order. An empty result follows successful structural and policy validation.

    An empty cluster array with a valid empty boundary mask returns Ok with an empty opportunity array. It does not create a synthetic unit or terminal opportunity.

    Successful nonempty results contain increasing, unique unitEnd values from one through the cluster count. unitEnd is an exclusive cluster index rather than a byte or scalar offset. The final opportunity has unitEnd equal to the cluster count and kind ForcedEnd.

    Repeated calls with equal cluster values and equal policies produce the same ordered opportunities and categories. Validation and classification do not read host locale, perform I/O, invoke caller callbacks, or mutate input values. Argument expressions use the ordinary language evaluation order.

    A BreakOpportunity carries a boundary and its category. It does not carry a penalty width or BreakRule. A caller preserving mandatory boundaries maps every ForcedEnd opportunity to a Forced penalty before calling breakParagraph.

    Why

    A measured cluster is the unit that shaping can safely keep together. Exclusive indices connect classification to paragraph items without converting through text offsets. A complete terminal boundary and deterministic order make the output suitable for reproducible layout.

    Source evidence

    Classify atomic clusters in repeatable boundary order

    Link
    main.tpzClassify atomic clusters in repeatable boundary orderOpen in Playground
    1. import std.typeset as ts
    2. import std.io
    3. import std.test
    4. let clusters = [ts.BreakCluster { text: "한", advance: 1pt, trackingAfter: 0pt }, ts.BreakCluster { text: "글", advance: 1pt, trackingAfter: 0pt }]
    5. let selected = ts.BreakPolicy {
    6. mode: ts.BreakMode.Character(),
    7. rules: ts.KinsokuRules.AtlasDefault(),
    8. forbidBefore: [],
    9. forbidAfter: [],
    10. boundaryAllowed: [true, true],
    11. generatedHyphenEnds: [],
    12. protectFittingLatin: false,
    13. preserveShortWords: false,
    14. narrowestWordTarget: 0pt,
    15. bodyWordTarget: 0pt
    16. }
    17. for pass in ([0, 1]) {
    18. let breaks = test.assertOk(ts.classifyBreaks(clusters, selected))
    19. for boundary in (breaks) {
    20. let category = match (boundary.kind) {
    21. case CjkCluster() => "CJK"
    22. case ForcedEnd() => "end"
    23. case _ => "unexpected kind"
    24. }
    25. io.writeLine("{pass} {boundary.unitEnd} {category}")
    26. }
    27. }
    28. io.writeLine(clusters[0].text)
    Expected outputexit 0
    0 1 CJK
    0 2 end
    1 1 CJK
    1 2 end
    한
    
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    Expected stdout
    1. 0 1 CJK
    2. 0 2 end
    3. 1 1 CJK
    4. 1 2 end
    5. 한
    Source evidence

    Classify an empty cluster sequence

    Link
    main.tpzClassify an empty cluster sequenceOpen in Playground
    1. import std.typeset as ts
    2. import std.io
    3. import std.test
    4. let clusters: Array<ts.BreakCluster> = []
    5. let selected = ts.BreakPolicy {
    6. mode: ts.BreakMode.Character(),
    7. rules: ts.KinsokuRules.AtlasDefault(),
    8. forbidBefore: [],
    9. forbidAfter: [],
    10. boundaryAllowed: [],
    11. generatedHyphenEnds: [],
    12. protectFittingLatin: false,
    13. preserveShortWords: false,
    14. narrowestWordTarget: 0pt,
    15. bodyWordTarget: 0pt
    16. }
    17. let breaks = test.assertOk(ts.classifyBreaks(clusters, selected))
    18. io.writeLine("{breaks.length}")
    Expected outputexit 0
    0
    
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    Expected stdout
    1. 0
    Source evidence

    Reject a mask with the wrong number of entries

    Link
    main.tpzReject a mask with the wrong number of entriesOpen in Playground
    1. import std.typeset as ts
    2. import std.io
    3. import std.test
    4. function outcome(value: Result<Array<ts.BreakOpportunity>, ts.TypesetError>) -> string {
    5. match (value) {
    6. case Ok(_) => "Ok"
    7. case Err(InvalidInput(_)) => "InvalidInput"
    8. case Err(InvalidPolicy(_)) => "InvalidPolicy"
    9. case Err(_) => "unexpected error"
    10. }
    11. }
    12. let clusters = [ts.BreakCluster { text: "a", advance: 1pt, trackingAfter: 0pt }]
    13. let selected = ts.BreakPolicy {
    14. mode: ts.BreakMode.Character(),
    15. rules: ts.KinsokuRules.AtlasDefault(),
    16. forbidBefore: [],
    17. forbidAfter: [],
    18. boundaryAllowed: [],
    19. generatedHyphenEnds: [],
    20. protectFittingLatin: false,
    21. preserveShortWords: false,
    22. narrowestWordTarget: 0pt,
    23. bodyWordTarget: 0pt
    24. }
    25. io.writeLine(outcome(ts.classifyBreaks(clusters, selected)))
    Expected outputexit 0
    InvalidInput
    
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    Expected stdout
    1. InvalidInput
    Source evidence

    Reject negative measured and untracked cluster extents

    Link
    main.tpzReject negative measured and untracked cluster extentsOpen in Playground
    1. import std.typeset as ts
    2. import std.io
    3. import std.test
    4. function outcome(value: Result<Array<ts.BreakOpportunity>, ts.TypesetError>) -> string {
    5. match (value) {
    6. case Ok(_) => "Ok"
    7. case Err(InvalidInput(_)) => "InvalidInput"
    8. case Err(InvalidPolicy(_)) => "InvalidPolicy"
    9. case Err(_) => "unexpected error"
    10. }
    11. }
    12. let selected = ts.BreakPolicy {
    13. mode: ts.BreakMode.Character(),
    14. rules: ts.KinsokuRules.AtlasDefault(),
    15. forbidBefore: [],
    16. forbidAfter: [],
    17. boundaryAllowed: [true],
    18. generatedHyphenEnds: [],
    19. protectFittingLatin: false,
    20. preserveShortWords: false,
    21. narrowestWordTarget: 0pt,
    22. bodyWordTarget: 0pt
    23. }
    24. io.writeLine(outcome(ts.classifyBreaks([ts.BreakCluster { text: "a", advance: -1pt, trackingAfter: 0pt }], selected)))
    25. io.writeLine(outcome(ts.classifyBreaks([ts.BreakCluster { text: "b", advance: 1pt, trackingAfter: 2pt }], selected)))
    Expected outputexit 0
    InvalidInput
    InvalidInput
    
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    Expected stdout
    1. InvalidInput
    2. InvalidInput
    Source evidence

    Reject a masked paragraph end in every ruleset

    Link
    main.tpzReject a masked paragraph end in every rulesetOpen in Playground
    1. import std.typeset as ts
    2. import std.io
    3. import std.test
    4. function outcome(value: Result<Array<ts.BreakOpportunity>, ts.TypesetError>) -> string {
    5. match (value) {
    6. case Ok(_) => "Ok"
    7. case Err(InvalidInput(_)) => "InvalidInput"
    8. case Err(InvalidPolicy(_)) => "InvalidPolicy"
    9. case Err(_) => "unexpected error"
    10. }
    11. }
    12. let clusters = [ts.BreakCluster { text: "a", advance: 1pt, trackingAfter: 0pt }]
    13. let atlas = ts.BreakPolicy {
    14. mode: ts.BreakMode.Character(),
    15. rules: ts.KinsokuRules.AtlasDefault(),
    16. forbidBefore: [],
    17. forbidAfter: [],
    18. boundaryAllowed: [false],
    19. generatedHyphenEnds: [],
    20. protectFittingLatin: false,
    21. preserveShortWords: false,
    22. narrowestWordTarget: 0pt,
    23. bodyWordTarget: 0pt
    24. }
    25. let unicode = ts.BreakPolicy {
    26. mode: ts.BreakMode.Character(),
    27. rules: ts.KinsokuRules.Unicode16Ja(),
    28. forbidBefore: [],
    29. forbidAfter: [],
    30. boundaryAllowed: [false],
    31. generatedHyphenEnds: [],
    32. protectFittingLatin: false,
    33. preserveShortWords: false,
    34. narrowestWordTarget: 0pt,
    35. bodyWordTarget: 0pt
    36. }
    37. io.writeLine(outcome(ts.classifyBreaks(clusters, atlas)))
    38. io.writeLine(outcome(ts.classifyBreaks(clusters, unicode)))
    Expected outputexit 0
    InvalidInput
    InvalidInput
    
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    Expected stdout
    1. InvalidInput
    2. InvalidInput
    Source evidence

    Keep exact point measurements at the classifier boundary

    Link
    main.tpzKeep exact point measurements at the classifier boundaryOpen in Playground
    1. import std.typeset as ts
    2. import std.test
    3. import std.io
    4. let clusters = [
    5. ts.BreakCluster { text: "a", advance: 0.5002pt, trackingAfter: 0pt },
    6. ts.BreakCluster { text: "b", advance: 0.5002pt, trackingAfter: 0pt }
    7. ]
    8. let policy = ts.BreakPolicy {
    9. mode: ts.BreakMode.Word(), rules: ts.KinsokuRules.AtlasDefault(),
    10. forbidBefore: [], forbidAfter: [], boundaryAllowed: [true, true],
    11. generatedHyphenEnds: [], protectFittingLatin: false, preserveShortWords: false,
    12. narrowestWordTarget: 0pt, bodyWordTarget: 1pt
    13. }
    14. let result = test.assertOk(ts.classifyBreaks(clusters, policy))
    15. for boundary in (result) {
    16. io.writeLine("{boundary.unitEnd}")
    17. }
    Expected outputexit 0
    1
    2
    
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    Expected stdout
    1. 1
    2. 2
    Source evidence

    Menu

    Download