yamleam/token

Token type emitted by the lexer.

yamleam v0.1 uses a line-oriented lexer: each logical (non-blank, non-comment) line is converted into a list of tokens together with the line’s indent column. The parser consumes these line-token groups.

Types

Line

</>

A single logical line of YAML, normalized: blank lines and pure comments are dropped before this form is produced. indent is the column of the first non-whitespace character.

pub type Line {
  Line(indent: Int, line_number: Int, tokens: List(Token))
}

Constructors

Line(indent: Int, line_number: Int, tokens: List(Token))

Token

</>

pub type Token {
  Token(kind: TokenKind, position: position.Position)
}

Constructors

Token(kind: TokenKind, position: position.Position)

TokenKind

</>

pub type TokenKind {
  PlainScalar(text: String)
  QuotedScalar(text: String)
  Colon
  Dash
  DocumentStart
  DocumentEnd
  FlowSeqStart
  FlowSeqEnd
  FlowMapStart
  FlowMapEnd
  Comma
  AnchorMarker(name: String)
  AliasMarker(name: String)
}

Constructors

```
PlainScalar(text: String)
```
A plain (unquoted) scalar — the text as written, not yet type-resolved.
```
QuotedScalar(text: String)
```
A quoted scalar — string contents with escapes already decoded. Always decoded to a YamlString regardless of content.
```
Colon
```
The : separator of a mapping entry.
```
Dash
```
The - marker opening a block-sequence item.
```
DocumentStart
```
The --- directive end / document start marker. Appears as the only content of a line at column 1.
```
DocumentEnd
```
The ... document end marker. Appears as the only content of a line at column 1.
```
FlowSeqStart
```
The [ opening a flow-style sequence.
```
FlowSeqEnd
```
The ] closing a flow-style sequence.
```
FlowMapStart
```
The { opening a flow-style mapping.
```
FlowMapEnd
```
The } closing a flow-style mapping.
```
Comma
```
The , separator between items in a flow collection.
```
AnchorMarker(name: String)
```
An anchor declaration &name attached to the next value.
```
AliasMarker(name: String)
```
An alias reference *name substituting a previously-anchored value.