Scope and ownership
СсылкаLexical structure identifies source spellings and statement boundaries. Literal values, escapes and template forms belong to Literals and templates. Name lookup and binding restrictions belong to Bindings and scope. Operator grouping belongs to Expressions and operators.
Contextual forms retain the owners listed below.
Identifiers, underscore and labels
СсылкаIdentifierSpelling = IdentifierStart IdentifierContinue* ;IdentifierStart = ( "_" | UnicodeLetter | NonAsciiEmoji ) - DefaultIgnorable ;IdentifierContinue = ( IdentifierStart | UnicodeNumber | UnicodeMark ) - DefaultIgnorable ;Identifier = IdentifierSpelling - ReservedWord - "_" ;Underscore = "_" ;Label = "'" IdentifierSpelling ;
In this EBNF excerpt, * denotes repetition and - excludes a scalar or spelling. Unicode comprises Mn, Mc, and Me. Default is Unicode 16.0.0 Default_Ignorable_Code_Point. The complete Identifier must already be NFC. The NFC condition applies to the whole sequence, not to each scalar separately. Reserved is defined in the next section.
Identifier characters
An identifier starts with _, a Unicode letter, or a non-ASCII scalar with the Unicode Emoji property. Later scalars may also be Unicode numbers and combining marks.
The character classes use Unicode 16.0.0. Letters use Lu, Ll, Lt, Lm, Lo, numbers use Nd, Nl, No, and combining marks use Mn, Mc, Me. A combining mark cannot start a name.
Exclude every scalar with Default_ from both positions, even when it is a letter or combining mark. This includes variation selectors, join controls, combining grapheme joiner, Hangul fillers, and emoji tag characters. No script or emoji sequence receives an exception. Each scalar in a multi-scalar emoji name must qualify independently. String data follows the string rules.
Причина
Programs can name values in the languages of the documents they process. Combining marks continue a name so those spellings do not depend on precomposed letters. A scalar that is normally invisible must not distinguish two names. Excluding Default_Ignorable_Code_Point from both positions gives names one rule without script, font, or emoji-presentation exceptions.
Complete identifier spellings
When no string delimiter immediately follows, the lexer classifies the complete identifier spelling. A spelling in Reserved becomes a keyword token. The complete spelling _ becomes an underscore token. Other spellings become identifier tokens.
Причина
Classifying a complete word prevents a keyword prefix from splitting a longer name. A distinct wildcard token keeps discarded values separate from named bindings.
Label spellings
' followed by Identifier forms a label token. Its body is not reclassified as a keyword.
Причина
The label prefix distinguishes a control target from a value name. Its separate token class makes the target visible before name lookup.
See Pattern matching and Expressions and operators for uses of _. Blocks and control flow describes label binding and jump targets.
Canonical source spelling
The complete name must already be in Unicode 16.0.0 NFC form in the source. This applies to declarations, references, members, fields, label bodies, and template tag names. Check the whole sequence before keyword or tag classification.
Bindings and lookup compare the admitted scalar sequence exactly. They do not normalize a spelling silently or create a second alias. NFC uses canonical equivalence. It does not apply compatibility normalization or case folding.
A non-NFC name produces TPZ0007 during lex. Mark the complete name span, excluding a label's apostrophe and a tag's string delimiter, and suggest its NFC spelling. Review the replacement for collisions before changing the source.
Причина
Canonically equivalent name spellings denote the same written name to a reader. Requiring one NFC source spelling prevents hidden aliases during binding and lookup. An explicit rejection and suggested replacement let the programmer review any resulting collision before changing the source. String contents remain data because normalizing them would change bytes the program may need to preserve.
Reserved words
СсылкаReservedWord ="let" | "mut" | "const" | "type" | "function" | "return" | "if"| "else" | "match" | "case" | "for" | "in" | "by" | "while"| "break" | "continue" | "defer" | "concurrent" | "true" | "false" | "null" ;
The literal values of true, false and null are specified in Literals and templates.
Identifier name positions
A name position that requires an identifier rejects keyword tokens.
Причина
Keeping keywords out of binding positions prevents a declaration from replacing the meaning of a language form.
Field name positions
A field-name position accepts an identifier token or a keyword token.
Причина
A field position already identifies a data member. Accepting keyword spellings there preserves names from structured data without an escape syntax.
Bindings and scope owns binding restrictions, including the treatment of None. Types and Functions and generics own compiler-defined type names and type-parameter restrictions.
Contextual spellings and their owners
СсылкаContextual identifier spellings
For identifier tokens with the spellings below, the parser assigns a contextual role when the surrounding form matches.
| Spelling | Role | Canonical owner |
|---|---|---|
enum, record, newtype | Nominal declaration heads | Records, enums and newtypes |
derives | Derivation clause after a nominal declaration head | Records, enums and newtypes; Protocols and implementations |
protocol, impl, using | Protocol, implementation and using forms | Protocols and implementations |
import, export | Program-level module forms | Modules and visibility |
as | Import alias clause | Modules and visibility |
use | Recognized and rejected module-item spelling | Modules and visibility |
set, map | Collection forms before a brace | Collections and comprehensions |
loop | Loop head, optionally followed by a label | Blocks and control flow |
self | First-parameter recognition | Functions and generics; receiver contexts in Protocols and implementations |
timeout | Named timeout option in a concurrent form | Concurrency |
Причина
Recognizing these words at their construct heads lets them remain ordinary names elsewhere. The surrounding grammar supplies the role.
The named chapters own these forms' grammar and meaning. Literals and templates owns tags and interpolation.
Punctuation tokens
СсылкаPunctuation ="..." | "..<" | "??=" | "**" | ".." | "?." | "??" | "<="| ">=" | ">>" | "==" | "!=" | "+=" | "-=" | "*=" | "/="| "%=" | "&&" | "||" | "=>" | "->" | "(" | ")"| "[" | "]" | "{" | "}" | "+" | "-" | "*" | "/"| "%" | "." | "?" | "<" | ">" | "=" | "!" | "~"| "|" | "," | ":" | ";" ;
See Expressions and operators, Types, and Pattern matching for the syntactic roles of these tokens. The adjacent >> token supplies two type-closing delimiters and has no expression-operator meaning. The retired |> spelling is not a punctuation token or an expression form.
Punctuation token boundaries
The punctuation scanner chooses the longest matching punctuation spelling. Comments, string delimiters, and label prefixes have separate lexical forms. The adjacent >> token supplies two type-closing delimiters. It has no expression-operator meaning.
Причина
Longest matching keeps a compound operator together. Its meaning does not depend on trying shorter operator spellings first.
Whitespace and comments
СсылкаHorizontalSpace = " " | HT ;Newline = LF | CR LF | CR ;LineComment = "//" NonNewlineScalar* ;
Non denotes a source scalar other than LF or CR. A comment starts with / and ends before LF, CRLF, CR, or the end of the source. Each physical break remains available to the normal layout rules.
Horizontal space and newlines
Space (U+0020) and horizontal tab (U+0009) separate tokens. LF (U+000A), CRLF, and bare CR (U+000D) each supply one line break to layout processing. Each spelling preserves the same statement boundary. Indentation does not delimit blocks. Braces and the surrounding syntax determine block and list boundaries.
Причина
Equivalent line endings preserve a program when a file moves between editors or operating systems. Braces keep scope independent of indentation.
Line comments
A line comment begins with / and ends before the next LF, CRLF, or bare CR, or at the end of the source. Its terminating line break remains available to layout processing.
Причина
One comment opener covers both explanatory lines and trailing comments. Preserving its terminating line break keeps comments from joining surrounding statements.
Block comment spellings are rejected
Write every comment line with /. The / spelling is rejected with TPZ2001 during parsing. Its opener is tokenized as ordinary / and * punctuation. Comment markers inside a string remain text.
Причина
Line comments give ignored source text one spelling. Prefixing each line handles a paragraph of commentary without a closing delimiter or a nesting exception. Line comments also preserve the physical break that separates surrounding statements. A block comment and successive line comments express the same intent to set explanatory text aside. A short mid-line comment can move to a preceding line without changing that intent.
Line breaks and statement boundaries
СсылкаStatementSeparator = StatementNewline ;
StatementNewline denotes a significant LF, CRLF, or CR line ending. A semicolon is rejected. Blocks and control flow defines final values and explicit discard.
Statement boundaries
LF, CRLF, and bare CR separate completed items at program level and in statement blocks when no continuation condition applies, including the break that ends a line comment. A semicolon is rejected and never discards a value. Blank lines do not introduce an extra item. No implicit separator is inserted immediately before a closing delimiter or the end of the source. The final expression remains the block value across trailing line breaks and comments. To discard that value explicitly, write let _.
Причина
A semicolon separator and a line boundary both separate completed statements. Newlines express that separation. Discarding a result is a different intent and is written explicitly with let _. Trailing whitespace and comments therefore leave the final expression value unchanged.
Line breaks inside delimiter forms
A line break continues the surrounding form while its active context is parentheses, square brackets, interpolation, record fields, or import selections. An explicit ; in those contexts produces TPZ1001. Use , between list elements.
Причина
The enclosing delimiter identifies an unfinished expression or list. Commas keep element boundaries explicit when a long form spans several lines.
Continuation around tokens
A newline continues the form at these token positions.
| Position of the line break | Token spellings |
|---|---|
| Before | >, >>, else |
| After | +, -, !, ~, ,, =>, (, [ |
| Either side | **, *, /, %, ., ., <, <=, >=, ==, !=, in, &&, \|\|, ??, =, +=, -, *=, /, %=, ??=, ., ?., by |
Причина
Continuation keeps an unfinished operator expression together. The lexer records line breaks and the layout pass uses adjacent tokens to decide its boundaries.
Continuation around pattern alternatives
A line break on either side of | continues a binding, case, or for pattern while that pattern is being read.
Причина
The alternatives belong to one pattern. Keeping them together lets the same binding requirements apply across a multiline alternative list.
Collections and comprehensions, Modules and visibility, and Functions and generics own list grammars. Expressions and operators owns precedence. Pattern matching owns pattern grammar.
Explicit call type arguments have a recorded implementation limitation involving the gap between > and (. Its intended contract and workaround remain in Functions and generics. That spacing-sensitive behavior does not add a lexical prohibition.
Source locations
СсылкаUnknown source characters
An unrecognized source character produces TPZ0001 at that character's location.
Причина
Rejecting an unknown character at its location prevents silent removal from changing a program.
Diagnostic source positions
CLI diagnostic labels use 1-based line, col, end, and end. Columns count Unicode scalar values from the start of the line. The lo and hi fields use 0-based UTF-8 byte offsets. Both end positions are exclusive.
Причина
Scalar columns locate characters without depending on UTF-8 byte width. Byte offsets address exact source slices, and exclusive ends compose adjacent spans.
Examples and non-examples
СсылкаIdentifier spellings and contextual names
Ссылкаlet answer_2: int = 42let 값: int = answer_2let 🟦: int = 값let ifx: int = 🟦let record: int = ifxlet loop: int = record
Нет стандартного вывода.Ожидаемый результат
Ожидаемый результат
Проверка принимает программу. Код завершения 0.
Наблюдаемые утверждения · Пройдено · 5.21.3
Код завершения 0.
Нет стандартного вывода.
Нет вывода ошибок.
- Пройдено · expected 0; observed 0
- Пройдено · expected 0; observed 0
- Пройдено · expected 0; observed 0
- Пройдено · expected 0; observed 0
Comments and continued expressions
Ссылка// A comment occupies its prefixed line.let first: int = 1 // A line comment.let second: int =first +2let third: int = (second* 2)
Нет стандартного вывода.Ожидаемый результат
Ожидаемый результат
Проверка принимает программу. Код завершения 0.
The retired block opener is rejected without a closer
Ссылкаlet value: int = 1/* unfinished
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
diagnostic-tpz2001 at
example.tpz:2:1
A line comment needs no closing delimiter
Ссылкаlet value: int = 1// A line comment also ends at the end of the source.
Нет стандартного вывода.Ожидаемый результат
Ожидаемый результат
Проверка принимает программу. Код завершения 0.
A semicolon in an array list
Ссылкаlet values: Array<int> = [1; 2]
Нет стандартного вывода.error[TPZ1001]: `;` does not separate items inside a delimiter list; use `,`
--> example.tpz:1:28
|
1 | let values: Array<int> = [1; 2]
| ^
error[TPZ2001]: expected `]`
--> example.tpz:1:30
|
1 | let values: Array<int> = [1; 2]
| ^
error[TPZ2001]: expected an expression
--> example.tpz:1:31
|
1 | let values: Array<int> = [1; 2]
| ^
error[TPZ2001]: expected a statement
--> example.tpz:1:31
|
1 | let values: Array<int> = [1; 2]
| ^
example.tpz: 4 diagnostics
Ожидаемый результат
Ожидаемый результат
Этап отклонения layout. Ненулевой код завершения.
semicolon-in-delimiter-list at
example.tpz:1:28
Наблюдаемые утверждения · Пройдено · 5.21.3
Код завершения 1.
Нет стандартного вывода.
error[TPZ1001]: `;` does not separate items inside a delimiter list; use `,`--> example.tpz:1:28|1 | let values: Array<int> = [1; 2]| ^error[TPZ2001]: expected `]`--> example.tpz:1:30|1 | let values: Array<int> = [1; 2]| ^error[TPZ2001]: expected an expression--> example.tpz:1:31|1 | let values: Array<int> = [1; 2]| ^error[TPZ2001]: expected a statement--> example.tpz:1:31|1 | let values: Array<int> = [1; 2]| ^example.tpz: 4 diagnostics
- Пройдено · checking TPZ1001; cause semicolon-in-delimiter-list; primary location asserted; additional diagnostics allow
- Пройдено · expected nonzero; observed 1
Separate array elements with a comma
Ссылкаlet values: Array<int> = [1, 2]
Нет стандартного вывода.Ожидаемый результат
Ожидаемый результат
Проверка принимает программу. Код завершения 0.
Наблюдаемые утверждения · Пройдено · 5.21.3
Код завершения 0.
Нет стандартного вывода.
Нет вывода ошибок.
- Пройдено · expected 0; observed 0
A keyword used as a function name
Ссылкаfunction if() { }
Нет стандартного вывода.error[TPZ2001]: expected a function name
--> example.tpz:1:10
|
1 | function if() { }
| ^^
example.tpz: 1 diagnostic
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
expected-function-name at
example.tpz:1:10
Наблюдаемые утверждения · Пройдено · 5.21.3
Код завершения 1.
Нет стандартного вывода.
error[TPZ2001]: expected a function name--> example.tpz:1:10|1 | function if() { }| ^^example.tpz: 1 diagnostic
- Пройдено · checking TPZ2001; cause expected-function-name; primary location asserted; additional diagnostics allow
- Пройдено · checking TPZ2001; cause expected-function-name; primary location asserted; additional diagnostics allow
- Пройдено · expected nonzero; observed 1
Give the function an identifier
Ссылкаfunction choose() { }
Нет стандартного вывода.Ожидаемый результат
Ожидаемый результат
Проверка принимает программу. Код завершения 0.
Наблюдаемые утверждения · Пройдено · 5.21.3
Код завершения 0.
Нет стандартного вывода.
Нет вывода ошибок.
- Пройдено · expected 0; observed 0
- Пройдено · expected 0; observed 0
An unknown character after a Unicode name
Ссылкаlet 이름 = @
Нет стандартного вывода.error[TPZ0001]: unknown character `@`
--> example.tpz:1:10
|
1 | let 이름 = @
| ^
error[TPZ2001]: expected an expression
--> example.tpz:2:1
|
2 |
| ^
example.tpz: 2 diagnostics
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
unknown-source-character at
example.tpz:1:10
Наблюдаемые утверждения · Пройдено · 5.21.3
Код завершения 1.
Нет стандартного вывода.
error[TPZ0001]: unknown character `@`--> example.tpz:1:10|1 | let 이름 = @| ^error[TPZ2001]: expected an expression--> example.tpz:2:1|2 || ^example.tpz: 2 diagnostics
- Пройдено · checking TPZ0001; cause unknown-source-character; primary location asserted; additional diagnostics allow
- Пройдено · checking TPZ0001; cause unknown-source-character; primary location asserted; additional diagnostics allow
- Пройдено · expected nonzero; observed 1
This example exercises the primary line and scalar column. Byte offsets and end positions remain pending product verification.
LF separates completed statements
Ссылкаimport std.iolet value = 1-2io.writeLine("{value}")
1
Ожидаемый результат
Ожидаемый результат
Выполнение завершается. Код завершения 0.
1
A block comment with LF is rejected
Ссылкаimport std.iolet value = 1 /* beforeafter */ +2io.writeLine("{value}")
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
diagnostic-tpz2001 at
example.tpz:2:16
LF ends a line comment
Ссылкаimport std.io// commentio.writeLine("after")io.writeLine("tail")
after
tail
Ожидаемый результат
Ожидаемый результат
Выполнение завершается. Код завершения 0.
aftertail
CRLF separates completed statements
Ссылкаimport std.iolet value = 1-2io.writeLine("{value}")
1
Ожидаемый результат
Ожидаемый результат
Выполнение завершается. Код завершения 0.
1
A block comment with CRLF is rejected
Ссылкаimport std.iolet value = 1 /* beforeafter */ +2io.writeLine("{value}")
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
diagnostic-tpz2001 at
example.tpz:2:16
CRLF ends a line comment
Ссылкаimport std.io// commentio.writeLine("after")io.writeLine("tail")
after
tail
Ожидаемый результат
Ожидаемый результат
Выполнение завершается. Код завершения 0.
aftertail
Bare CR separates completed statements
Ссылкаimport std.iolet value = 1 -2io.writeLine("{value}")
1
Ожидаемый результат
Ожидаемый результат
Выполнение завершается. Код завершения 0.
1
A block comment with CR is rejected
Ссылкаimport std.iolet value = 1 /* before after */ +2io.writeLine("{value}")
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
diagnostic-tpz2001 at
example.tpz:2:16
Bare CR ends a line comment
Ссылкаimport std.io// comment io.writeLine("after")io.writeLine("tail")
after
tail
Ожидаемый результат
Ожидаемый результат
Выполнение завершается. Код завершения 0.
aftertail
A block comment on one line is rejected
Ссылкаimport std.iolet value = 1 /* no line break */ +2io.writeLine("{value}")
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
diagnostic-tpz2001 at
example.tpz:2:16
Parentheses do not admit block comments
Ссылкаimport std.iolet value = (1 /* before after */ +2)io.writeLine("{value}")
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
diagnostic-tpz2001 at
example.tpz:2:17
Adjacent declarations need a statement boundary
Ссылкаlet first = 1 let second = 2
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
expected-statement-separator at
example.tpz:1:15
A newline separates commented items
Ссылкаlet first = 1let second = 2 // The newline separates the items.
Нет стандартного вывода.Ожидаемый результат
Ожидаемый результат
Проверка принимает программу. Код завершения 0.
A semicolon cannot separate program items
Ссылкаlet first = 1; let second = 2
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
expected-statement-separator at
example.tpz:1:14
A semicolon cannot discard a block tail
Ссылкаfunction value() { 7; }
Ожидаемый результат
Ожидаемый результат
Этап отклонения parse. Ненулевой код завершения.
expected-statement-separator at
example.tpz:1:21
Combining marks continue NFC names
Ссылкаlet q́: int = 1let का: int = q́let x⃝: int = काloop 'q́ { break 'q́ }
Нет стандартного вывода.Ожидаемый результат
Ожидаемый результат
Проверка принимает программу. Код завершения 0.
q + U+0301 is Mn, U+093E in का is Mc, and U+20DD in x⃝ is Me. Each complete name is already NFC.
Use the NFC spelling throughout the source
Ссылкаlet café: int = 1let 가: int = 2let K: int = 3let value: int = café + 가 + K
Нет стандартного вывода.Ожидаемый результат
Ожидаемый результат
Проверка принимает программу. Код завершения 0.
A combining mark cannot start a name
Ссылкаlet ́name = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
unknown-source-character at
example.tpz:1:5
The leading scalar is U+0301 COMBINING ACUTE ACCENT.
Reject an invisible Hangul filler at the start
Ссылкаlet ᅟname = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
unknown-source-character at
example.tpz:1:5
U+115F HANGUL CHOSEONG FILLER is a Letter and Default_Ignorable_Code_Point.
Reject an invisible Hangul filler in a name
Ссылкаlet aㅤ = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
unknown-source-character at
example.tpz:1:6
U+3164 HANGUL FILLER is excluded even though its General_Category is Lo.
Reject a variation selector in a name
Ссылкаlet ✈️ = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
unknown-source-character at
example.tpz:1:6
The rejected scalar is U+FE0F VARIATION SELECTOR-16. Use the admitted name ✈ without the selector.
Reject a joiner in an emoji name
Ссылкаlet 👩💻 = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
unknown-source-character at
example.tpz:1:6
The rejected scalar is U+200D ZERO WIDTH JOINER. A displayed emoji cluster does not override scalar admission.
Reject a combining grapheme joiner
Ссылкаlet a͏ = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
unknown-source-character at
example.tpz:1:6
U+034F COMBINING GRAPHEME JOINER is Mn and Default_Ignorable_Code_Point. The exclusion takes precedence over combining-mark continuation.
Reject decomposed source names
Ссылкаlet café = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
identifier-not-nfc at
example.tpz:1:5
NFC spells this complete name café.
Reject composable Hangul jamo in a name
Ссылкаlet 가 = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
identifier-not-nfc at
example.tpz:1:5
The name contains U+1100 and U+1161. NFC replaces the sequence with U+AC00 가.
Reject a scalar with a different NFC spelling
Ссылкаlet K = 1
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
identifier-not-nfc at
example.tpz:1:5
U+212A KELVIN SIGN has the canonical NFC spelling U+004B K. This is a canonical normalization rule, not NFKC or case folding.
Apply NFC to label bodies
Ссылкаloop '가 { break }
Ожидаемый результат
Ожидаемый результат
Этап отклонения lex. Ненулевой код завершения.
identifier-not-nfc at
example.tpz:1:7
The primary name span excludes the opening apostrophe. Write the label body 가.