Skip to content
Topaz6.0 · Before release
Download

Search documentation

Specification 6.0 · Not released · Tested product 5.21.3

Use arrow keys to choose a result. Press Enter to open.

Type an API name, diagnostic number, rule, or project topic.

    Library · std.bytes

    Bytes.decodeUtf8

    Decodes this byte sequence as UTF-8 text.

    decodeUtf8() -> Result<string, string>
    Module
    std.bytes
    Receiver
    Bytes
    Minimum arity
    0
    Maximum arity
    0

    Declaration and parameters

    Link

    Availability

    Included by default: Pending

    Import std.bytes in language mode 5.21.

    The current compiler selects std/bytes.tpz for this language mode. Availability in other released editions and inclusion in each installed distribution are Pending.

    Source evidence

    Receiver

    Bytes

    Parameter constraints

    Additional contracts are Pending.

    Bytes.decodeUtf8 returns Ok with the exact Unicode scalar sequence encoded by valid UTF-8 bytes.

    Empty Bytes returns Ok with an empty string. A leading UTF-8 encoding of U+FEFF remains in the returned string.

    Overlong encodings, isolated continuation bytes, incomplete sequences, surrogate encodings, and values above U+10FFFF return Err with a string.

    Decoding does not insert replacement characters, normalize scalars, alter newline sequences, or discard a byte order mark.

    For a string, encodeUtf8 followed by decodeUtf8 returns the original string. Re-encoding any successfully decoded sequence recovers its original bytes.

    The receiver evaluates once and remains unchanged. No text is exposed before the entire sequence has passed validation.

    Related rule: Decode UTF-8 without replacing data

    Result

    Additional contracts are Pending.

    The declared result type is Result<string, string>.

    Effects

    Pending

    Failures

    Pending

    Result and effects

    Link

    Return the exact decoded scalar sequence in Ok. Malformed UTF-8 returns Err without replacement text or a decoded prefix.

    Resolved contracts

    Link

    Related entries

    Link

    std.bytes

    Decode UTF-8 without replacing data

    Link
    Ruletopaz.stdlib.contract.bytes-utf8.rule

    Decode UTF-8 without replacing data

    Bytes.decodeUtf8 returns Ok with the exact Unicode scalar sequence encoded by valid UTF-8 bytes.

    Empty Bytes returns Ok with an empty string. A leading UTF-8 encoding of U+FEFF remains in the returned string.

    Overlong encodings, isolated continuation bytes, incomplete sequences, surrogate encodings, and values above U+10FFFF return Err with a string.

    Decoding does not insert replacement characters, normalize scalars, alter newline sequences, or discard a byte order mark.

    For a string, encodeUtf8 followed by decodeUtf8 returns the original string. Re-encoding any successfully decoded sequence recovers its original bytes.

    The receiver evaluates once and remains unchanged. No text is exposed before the entire sequence has passed validation.

    Why

    Lossless decoding keeps malformed input visible and preserves distinctions needed by file formats, checksums, and text layout.

    Examples Retain a byte order mark and combining sequenceReject malformed UTF-8 categories

    Source evidence

    Retain a byte order mark and combining sequence

    Link

    run · running · Verification: not-run

    main.tpzRetain a byte order mark and combining sequenceOpen in Playground
    1. import std.bytes { Bytes }
    2. import std.io
    3. import std.test
    5. for hex in (["", "efbbbf", "65cc81", "0d0a", "f09f9880"]) {
    6. let decoded = test.assertOk(test.assertOk(Bytes.fromHex(hex)).decodeUtf8())
    7. io.writeLine(decoded.encodeUtf8().toHex())
    8. }
    Expected outputexit 0
    
efbbbf
65cc81
0d0a
f09f9880
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    2. efbbbf
    3. 65cc81
    4. 0d0a
    5. f09f9880

    This exact example has not been executed.

    Source evidence

    Reject malformed UTF-8 categories

    Link

    run · running · Verification: not-run

    main.tpzReject malformed UTF-8 categoriesOpen in Playground
    1. import std.bytes { Bytes }
    2. import std.io
    3. import std.test
    5. for hex in (["80", "c080", "e282", "eda080", "f4908080"]) {
    6. match (test.assertOk(Bytes.fromHex(hex)).decodeUtf8()) {
    7. case Err(_) => io.writeLine("invalid UTF-8")
    8. case Ok(_) => io.writeLine("accepted")
    9. }
    10. }
    Expected outputexit 0
    invalid UTF-8
invalid UTF-8
invalid UTF-8
invalid UTF-8
invalid UTF-8
    $ topaz run main.tpz · target product 5.21.3
    Expected result

    Expected result

    Execution completes. Exit code 0.

    No stderr.

    Expected stdout
    1. invalid UTF-8
    2. invalid UTF-8
    3. invalid UTF-8
    4. invalid UTF-8
    5. invalid UTF-8

    This exact example has not been executed.

    Source evidence

    Menu

    Download