Skip to content
Topaz6.0 · Before release
Download

Search documentation

Specification 6.0 · Not released · Tested product 5.21.3

Use arrow keys to choose a result. Press Enter to open.

Type an API name, diagnostic number, rule, or project topic.

    Library

    std.parser

    Classify ASCII values and scan spans using Unicode scalar positions.

    APIs in this module

    12 APIs

    NameModule
    std.parser.isAsciiAlnumstd.parser
    std.parser.isAsciiAlnumCodestd.parser
    std.parser.isAsciiAlphastd.parser
    std.parser.isAsciiAlphaCodestd.parser
    std.parser.isAsciiDigitstd.parser
    std.parser.isAsciiDigitCodestd.parser
    std.parser.isAsciiWhitespacestd.parser
    std.parser.isAsciiWhitespaceCodestd.parser
    std.parser.takeWhileAsciiAlnumstd.parser
    std.parser.takeWhileAsciiAlphastd.parser
    std.parser.takeWhileAsciiDigitstd.parser
    std.parser.takeWhileAsciiWhitespacestd.parser

    Module

    Link

    Code predicates classify an integer. Indexed predicates classify one Unicode scalar in a string. Span scans return the first position after consecutive matches. Every position uses the same scalar unit as string indexing.

    Why

    Link

    Explicit ASCII classes give parsers stable boundaries without locale rules. Scalar positions compose with existing string indexing and slicing.

    Availability

    Link

    Import std.parser in language mode 5.21. The current compiler selects std/parser.tpz for this language mode. Availability in other released editions and inclusion in each installed distribution are Pending.

    Public API

    Link

    std.parser.isAsciiDigitCode

    std.parser.isAsciiAlphaCode

    std.parser.isAsciiAlnumCode

    std.parser.isAsciiWhitespaceCode

    std.parser.isAsciiDigit

    std.parser.isAsciiAlpha

    std.parser.isAsciiAlnum

    std.parser.isAsciiWhitespace

    std.parser.takeWhileAsciiDigit

    std.parser.takeWhileAsciiAlpha

    std.parser.takeWhileAsciiAlnum

    std.parser.takeWhileAsciiWhitespace

    Pending contracts

    Link

    Classify integer ASCII codes

    Link
    Ruletopaz.stdlib.contract.parser-ascii-codes.rule

    Classify integer ASCII codes

    isAsciiDigitCode returns true for integers from 48 through 57, inclusive. These are the ASCII digits from 0 through 9.

    isAsciiAlphaCode returns true for integers from 65 through 90 or from 97 through 122, inclusive. These are ASCII uppercase and lowercase letters.

    isAsciiAlnumCode returns true when the digit predicate or the letter predicate returns true.

    isAsciiWhitespaceCode returns true for 9, 10, 11, 12, 13, and 32. These represent tab, line feed, vertical tab, form feed, carriage return, and space.

    Every other int returns false. Negative values, surrogate code values, non ASCII characters, and integers above the Unicode range do not cause a diagnostic.

    Each call evaluates its input once. Classification has no side effects and does not consult a locale, normalize text, or fold case.

    Why

    Exact integer sets make ASCII parsing independent of host text rules. A false result handles a value outside the class without introducing an error channel.

    Examples Include each ASCII endpoint and exclude its neighborsReturn false for invalid code points and non ASCII values

    Source evidence

    Include each ASCII endpoint and exclude its neighbors

    Link

    run · running · Verification: not-run

    main.tpzInclude each ASCII endpoint and exclude its neighborsOpen in Playground
    1. import std.parser
    2. import std.io
    4. for cp in ([47, 48, 57, 58, 64, 65, 90, 91, 96, 97, 122, 123, 8, 9, 10, 11, 12, 13, 14, 31, 32, 33]) {
    5. let digit = parser.isAsciiDigitCode(cp)
    6. let alpha = parser.isAsciiAlphaCode(cp)
    7. let alnum = parser.isAsciiAlnumCode(cp)
    8. let space = parser.isAsciiWhitespaceCode(cp)
    9. io.writeLine("{digit}/{alpha}/{alnum}/{space}")
    10. }
    Expected outputexit 0
    false/false/false/false
true/false/true/false
true/false/true/false
false/false/false/false
false/false/false/false
false/true/true/false
false/true/true/false
false/false/false/false
false/false/false/false
false/true/true/false
false/true/true/false
false/false/false/false
false/false/false/false
false/false/false/true
false/false/false/true
false/false/false/true
false/false/false/true
false/false/false/true
false/false/false/false
false/false/false/false
false/false/false/true
false/false/false/false
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    1. false/false/false/false
    2. true/false/true/false
    3. true/false/true/false
    4. false/false/false/false
    5. false/false/false/false
    6. false/true/true/false
    7. false/true/true/false
    8. false/false/false/false
    9. false/false/false/false
    10. false/true/true/false
    11. false/true/true/false
    12. false/false/false/false
    13. false/false/false/false
    14. false/false/false/true
    15. false/false/false/true
    16. false/false/false/true
    17. false/false/false/true
    18. false/false/false/true
    19. false/false/false/false
    20. false/false/false/false
    21. false/false/false/true
    22. false/false/false/false

    This exact example has not been executed.

    Source evidence

    Return false for invalid code points and non ASCII values

    Link

    run · running · Verification: not-run

    main.tpzReturn false for invalid code points and non ASCII valuesOpen in Playground
    1. import std.parser
    2. import std.io
    4. for cp in ([-1, 128, 233, 55296, 1114112, 9223372036854775807]) {
    5. let digit = parser.isAsciiDigitCode(cp)
    6. let alpha = parser.isAsciiAlphaCode(cp)
    7. let alnum = parser.isAsciiAlnumCode(cp)
    8. let space = parser.isAsciiWhitespaceCode(cp)
    9. io.writeLine("{digit}/{alpha}/{alnum}/{space}")
    10. }
    Expected outputexit 0
    false/false/false/false
false/false/false/false
false/false/false/false
false/false/false/false
false/false/false/false
false/false/false/false
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    1. false/false/false/false
    2. false/false/false/false
    3. false/false/false/false
    4. false/false/false/false
    5. false/false/false/false
    6. false/false/false/false

    This exact example has not been executed.

    Source evidence

    Classify a scalar position

    Link
    Ruletopaz.stdlib.contract.parser-ascii-positions.rule

    Classify a scalar position

    The index counts Unicode scalars from zero. A multibyte scalar occupies one position. A combining scalar occupies its own position.

    A present scalar is classified by the corresponding code predicate. Non ASCII scalars return false, including a nonbreaking space and letters outside the ASCII ranges.

    A negative index or an index at or beyond the scalar length returns false. Index zero in an empty string also returns false. These cases do not produce a diagnostic.

    The result agrees with applying the corresponding code predicate to text.codePointAt(index) when it returns Some(code). A None() result means false.

    The text expression is evaluated before the index expression, once each. Classification leaves the string unchanged and performs no I/O.

    Why

    A shared scalar coordinate system lets parser positions pass directly to string operations. An absent position is a failed predicate, so end of input needs no exceptional control flow.

    Examples Keep scalar positions through multibyte and combining textTreat negative positions and end of input as false

    Source evidence

    Keep scalar positions through multibyte and combining text

    Link

    run · running · Verification: not-run

    main.tpzKeep scalar positions through multibyte and combining textOpen in Playground
    1. import std.parser
    2. import std.io
    4. let text = "é5éZ  "
    5. for index in ([0, 1, 2, 3, 4, 5, 6]) {
    6. let digit = parser.isAsciiDigit(text, index)
    7. let alpha = parser.isAsciiAlpha(text, index)
    8. let alnum = parser.isAsciiAlnum(text, index)
    9. let space = parser.isAsciiWhitespace(text, index)
    10. io.writeLine("{digit}/{alpha}/{alnum}/{space}")
    11. }
    Expected outputexit 0
    false/false/false/false
true/false/true/false
false/true/true/false
false/false/false/false
false/true/true/false
false/false/false/false
false/false/false/true
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    1. false/false/false/false
    2. true/false/true/false
    3. false/true/true/false
    4. false/false/false/false
    5. false/true/true/false
    6. false/false/false/false
    7. false/false/false/true

    This exact example has not been executed.

    Source evidence

    Treat negative positions and end of input as false

    Link

    run · running · Verification: not-run

    main.tpzTreat negative positions and end of input as falseOpen in Playground
    1. import std.parser
    2. import std.io
    4. let text = "7"
    5. for index in ([-1, 1, 2, 9223372036854775807]) {
    6. let digit = parser.isAsciiDigit(text, index)
    7. let alpha = parser.isAsciiAlpha(text, index)
    8. let alnum = parser.isAsciiAlnum(text, index)
    9. let space = parser.isAsciiWhitespace(text, index)
    10. io.writeLine("{digit}/{alpha}/{alnum}/{space}")
    11. }
    12. let empty = ""
    13. let digit = parser.isAsciiDigit(empty, 0)
    14. let alpha = parser.isAsciiAlpha(empty, 0)
    15. let alnum = parser.isAsciiAlnum(empty, 0)
    16. let space = parser.isAsciiWhitespace(empty, 0)
    17. io.writeLine("{digit}/{alpha}/{alnum}/{space}")
    Expected outputexit 0
    false/false/false/false
false/false/false/false
false/false/false/false
false/false/false/false
false/false/false/false
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    1. false/false/false/false
    2. false/false/false/false
    3. false/false/false/false
    4. false/false/false/false
    5. false/false/false/false

    This exact example has not been executed.

    Source evidence

    Scan a maximal ASCII span

    Link
    Ruletopaz.stdlib.contract.parser-ascii-spans.rule

    Scan a maximal ASCII span

    Each scan uses the class named by its corresponding ASCII predicate. The starting position and result follow the zero based scalar positions.

    Clamp a negative start to zero. Clamp a start greater than the scalar length to that length. The first position examined is the clamped start.

    Inspect consecutive positions from left to right. Stop at the first nonmatching scalar or at the scalar length. Return that exclusive end position.

    If the first scalar does not match, return the clamped start. Starting at end of input returns that end. Every scan of an empty string returns zero.

    A non ASCII scalar ends the span. Scanning does not skip it, normalize it, or advance across it to find a later match.

    The text expression is evaluated before the start expression, once each. Scans do not modify the text or perform I/O. An out of range start produces a bounded result without a diagnostic.

    For a clamped start begin and returned end, text.slice(begin, end) is the matched text. The scan reports a position rather than a character count.

    Why

    Maximal spans let callers resume at the first unconsumed scalar. Clamping uses the same bounds as string slicing and gives empty input a useful result.

    Examples Find the exclusive end for every ASCII classStop at non ASCII scalars without changing position unitsClamp starts and keep the first failing position

    Source evidence

    Find the exclusive end for every ASCII class

    Link

    run · running · Verification: not-run

    main.tpzFind the exclusive end for every ASCII classOpen in Playground
    1. import std.parser
    2. import std.io
    4. let digit = parser.takeWhileAsciiDigit("123x", 0)
    5. let alpha = parser.takeWhileAsciiAlpha("abZ9", 0)
    6. let alnum = parser.takeWhileAsciiAlnum("a9Z_", 0)
    7. let space = parser.takeWhileAsciiWhitespace("\t\n\r x", 0)
    8. io.writeLine("{digit}/{alpha}/{alnum}/{space}")
    Expected outputexit 0
    3/3/3/4
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    1. 3/3/3/4

    This exact example has not been executed.

    Source evidence

    Stop at non ASCII scalars without changing position units

    Link

    run · running · Verification: not-run

    main.tpzStop at non ASCII scalars without changing position unitsOpen in Playground
    1. import std.parser
    2. import std.io
    4. let digit = parser.takeWhileAsciiDigit("é12x", 1)
    5. let combining = parser.takeWhileAsciiAlpha("éA", 0)
    6. let suffix = parser.takeWhileAsciiAlpha("éA", 2)
    7. let nonbreaking = parser.takeWhileAsciiWhitespace("  ", 0)
    8. let alnum = parser.takeWhileAsciiAlnum("Aé", 0)
    9. io.writeLine("{digit}/{combining}/{suffix}/{nonbreaking}/{alnum}")
    Expected outputexit 0
    3/1/3/0/1
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    1. 3/1/3/0/1

    This exact example has not been executed.

    Source evidence

    Clamp starts and keep the first failing position

    Link

    run · running · Verification: not-run

    main.tpzClamp starts and keep the first failing positionOpen in Playground
    1. import std.parser
    2. import std.io
    4. let text = "12"
    5. for start in ([-3, 0, 1, 2, 99]) {
    6. let digit = parser.takeWhileAsciiDigit(text, start)
    7. let alpha = parser.takeWhileAsciiAlpha(text, start)
    8. let alnum = parser.takeWhileAsciiAlnum(text, start)
    9. let space = parser.takeWhileAsciiWhitespace(text, start)
    10. io.writeLine("{digit}/{alpha}/{alnum}/{space}")
    11. }
    12. let empty = ""
    13. let digit = parser.takeWhileAsciiDigit(empty, 99)
    14. let alpha = parser.takeWhileAsciiAlpha(empty, -1)
    15. let alnum = parser.takeWhileAsciiAlnum(empty, 0)
    16. let space = parser.takeWhileAsciiWhitespace(empty, 1)
    17. io.writeLine("{digit}/{alpha}/{alnum}/{space}")
    Expected outputexit 0
    2/0/2/0
2/0/2/0
2/1/2/1
2/2/2/2
2/2/2/2
0/0/0/0
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    1. 2/0/2/0
    2. 2/0/2/0
    3. 2/1/2/1
    4. 2/2/2/2
    5. 2/2/2/2
    6. 0/0/0/0

    This exact example has not been executed.

    Source evidence

    Menu

    Download