AST Design

Design AST nodes that preserve all information for round-trip formatting.

Design Principles

1. Use Spanned<T> for all children - Enables error locations and source mapping
2. Include punctuation tokens - Needed for exact round-trip
3. Track trivia - Comments and newlines for formatting

Document Structure

/// The root of a TOML document.
/// Contains a sequence of items (key-value pairs or tables).
#[derive(Debug, Clone)]
pub struct Document {
    pub items: Vec<DocumentItem>,
}

/// A single item in the document: either a top-level key-value or a table section.
#[derive(Debug, Clone)]
pub enum DocumentItem {
    /// A blank line or comment
    Trivia(Trivia),
    /// A key = value pair at the top level
    KeyValue(Spanned<KeyValue>),
    /// A [table] section
    Table(Spanned<Table>),
}

• Document is the root containing all items
• DocumentItem distinguishes top-level elements
• Trivia captures non-semantic content

Keys

/// A TOML key, which can be bare, quoted, or dotted.
#[derive(Debug, Clone)]
pub enum Key {
    /// Bare key: `foo`
    Bare(tokens::BareKeyToken),
    /// Quoted key: `"foo.bar"`
    Quoted(tokens::BasicStringToken),
    /// Dotted key: `foo.bar.baz`
    Dotted(DottedKey),
}

/// A dotted key like `server.host.name`
#[derive(Debug, Clone)]
pub struct DottedKey {
    pub first: Spanned<SimpleKey>,
    pub rest: Vec<(Spanned<tokens::DotToken>, Spanned<SimpleKey>)>,
}

/// A simple (non-dotted) key
#[derive(Debug, Clone)]
pub enum SimpleKey {
    Bare(tokens::BareKeyToken),
    Quoted(tokens::BasicStringToken),
}

• Key enum handles all key forms
• DottedKey preserves dot tokens for round-trip
• SimpleKey is the base case (bare or quoted)

Values

/// A TOML value.
#[derive(Debug, Clone)]
pub enum Value {
    /// String value
    String(tokens::BasicStringToken),
    /// Integer value
    Integer(tokens::IntegerToken),
    /// Boolean true
    True(tokens::TrueToken),
    /// Boolean false
    False(tokens::FalseToken),
    /// Array value
    Array(Array),
    /// Inline table value
    InlineTable(InlineTable),
}

Each variant stores its token type directly, preserving the original representation.

Key-Value Pairs

/// A key-value pair: `key = value`
#[derive(Debug, Clone)]
pub struct KeyValue {
    pub key: Spanned<Key>,
    pub eq: Spanned<tokens::EqToken>,
    pub value: Spanned<Value>,
}

Note how eq stores the equals token—this enables formatting choices like key=value vs key = value.

Tables

/// A table section: `[section]` or `[section.subsection]`
#[derive(Debug, Clone)]
pub struct Table {
    pub lbracket: Spanned<tokens::LBracketToken>,
    pub name: Spanned<Key>,
    pub rbracket: Spanned<tokens::RBracketToken>,
    pub items: Vec<TableItem>,
}

/// An item within a table section.
#[derive(Debug, Clone)]
pub enum TableItem {
    Trivia(Trivia),
    KeyValue(Box<Spanned<KeyValue>>),
}

• Brackets stored explicitly for round-trip
• Items include trivia for blank lines/comments within table

Arrays

/// An array: `[1, 2, 3]`
#[derive(Debug, Clone)]
pub struct Array {
    pub lbracket: Spanned<tokens::LBracketToken>,
    pub items: Vec<ArrayItem>,
    pub rbracket: Spanned<tokens::RBracketToken>,
}

/// An item in an array, including trailing trivia.
#[derive(Debug, Clone)]
pub struct ArrayItem {
    pub value: Spanned<Value>,
    pub comma: Option<Spanned<tokens::CommaToken>>,
}

ArrayItem includes optional trailing comma—essential for preserving:

[1, 2, 3]     # No trailing comma
[1, 2, 3,]    # With trailing comma

Inline Tables

/// An inline table: `{ key = value, ... }`
#[derive(Debug, Clone)]
pub struct InlineTable {
    pub lbrace: Spanned<tokens::LBraceToken>,
    pub items: Vec<InlineTableItem>,
    pub rbrace: Spanned<tokens::RBraceToken>,
}

/// An item in an inline table.
#[derive(Debug, Clone)]
pub struct InlineTableItem {
    pub kv: Spanned<KeyValue>,
    pub comma: Option<Spanned<tokens::CommaToken>>,
}

Similar structure to arrays, with key-value pairs instead of values.

Why This Design?

Span Preservation

Every Spanned<T> carries source location:

let kv: Spanned<KeyValue> = stream.parse()?;
let key_span = &kv.value.key.span;  // Location of key
let eq_span = &kv.value.eq.span;    // Location of '='
let val_span = &kv.value.value.span; // Location of value

Round-trip Fidelity

Storing tokens enables exact reconstruction:

// Original: key = "value"
// After parse → print:
//   key = "value"  (identical)

Trivia Handling

Without trivia tracking:

# Comment lost!
key = value

With trivia in AST:

# Comment preserved
key = value