grammar

This page gives the full grammar of ghūl, derived from the compiler's parser.

The grammar is written in W3C EBNF, the notation used by the XML and XPath specifications:

Notation	Meaning
A ::= ...	defines the symbol A
A B	A followed by B
A | B	A or B
A?	zero or one A
A*	zero or more A
A+	one or more A
( ... )	grouping
"is"	a literal terminal
[a-z]	a character in the given set
[^"]	any character not in the set
A - B	an A that is not also a B

CamelCase symbols are grammar productions; Identifier, IntegerLiteral and the other symbols defined under lexical grammar are tokens produced by the tokenizer.

A few constructs are resolved by the parser using context that a context-free grammar cannot express (operator precedence, and a small number of genuinely context-sensitive forms). These are called out in prose where they arise, and the operator precedence table is given at the end.

lexical grammar

The tokenizer turns source text into a stream of tokens. Whitespace (spaces, tabs, carriage returns and newlines) separates tokens, and how much of it there is almost never matters. Two things about it do: whether a token is the first on its line, which is what lets a statement terminator be left off, and how far one construct is indented, which matters in a single case. Both are covered under statement terminators below. Comments are discarded before parsing.

comments

LineComment  ::= "//" [^#xA]*
BlockComment ::= "/*" ( [^*] | "*" [^/] )* "*/"

Block comments do not nest: the first */ ends the comment.

identifiers

Identifier        ::= PlainIdentifier | EscapedIdentifier
PlainIdentifier   ::= Letter ( Letter | Digit )*
EscapedIdentifier ::= "`" ( Letter | Digit | "_" )+
                    | "`" OperatorChar+
Letter            ::= [a-zA-Z_]
Digit             ::= [0-9]

QualifiedIdentifier ::= Identifier ( "." Identifier )*

A PlainIdentifier may not be one of the reserved words. To use a reserved word (or an operator symbol) as an ordinary identifier, prefix it with a backtick: `field, `+.

reserved words

The following words are keywords and cannot be used as plain identifiers:

abstract  assert    await     break     case      cast      catch
class     continue  default   do        elif      else      enum
esac      false     fi        field     finally   for       if
in        innate    is        isa       lav       let       mut
namespace new       null      od        operator  private   protected
ptr       public    rec       ref       return    self      si
static    struct    super     then      throw     trait     true
try       typeof    union     use       val       when      while
yield     yrt

A few words are contextual: they look like identifiers to the tokenizer but the parser recognises them in specific positions: optional as a type-parameter kind constraint, out as a type-parameter variance modifier, open as a class extensibility modifier.

numeric literals

IntegerLiteral ::= DecimalInteger | HexInteger
DecimalInteger ::= Digit ( Digit | "_" )* IntegerSuffix?
HexInteger     ::= ( "0x" | "0X" ) HexDigit ( HexDigit | "_" )* IntegerSuffix?
HexDigit       ::= [0-9a-fA-F]
IntegerSuffix  ::= ( "s" | "S" | "u" | "U" )? [bBcCsSiIlLwW]?

FloatLiteral   ::= Digit ( Digit | "_" )* "." ( Digit | "_" )* Exponent? FloatSuffix?
Exponent       ::= ( "e" | "E" ) "-"? ( Digit | "_" )+
FloatSuffix    ::= "s" | "S" | "d" | "D" | "m" | "M"

Underscores within a number are for readability and are ignored. A float literal must contain a .; the type suffix selects single (s/S), double (d/D) or decimal (m/M), and a float literal without one is a double. An integer suffix selects the integer type and signedness.

character and string literals

CharLiteral   ::= "'" ( EscapeSequence | [^'] ) "'"
StringLiteral ::= '"' StringElement* '"'
StringElement ::= EscapeSequence | [^"#xA\]

EscapeSequence ::= "\" ( "t" | "n" | "r" | "\" | OctalDigit+ | [^#xA] )
OctalDigit     ::= [0-7]

A string literal may not span a newline. Two string literals separated only by whitespace or comments are concatenated into a single literal; the whitespace can include line breaks, which is how a long string is split across lines. A ; between them ends the chain, so a statement that ends on a string literal needs a ; where the next statement begins with one.

Inside a string literal, { begins an interpolation and } ends it; a literal brace is written {{ or }}.

interpolated strings

A string literal containing { ... } is tokenized as a sequence of fragments rather than a single StringLiteral. The parser assembles these as an interpolated string expression:

InterpolatedString ::= EnterString
                       Interpolation
                       ( ContinueString Interpolation )*
                       ExitString
Interpolation      ::= Expression ( "," Expression )? ( ":" FormatString )?

EnterString, ContinueString, ExitString and FormatString are the fragments of literal text surrounding and following each interpolated expression. The optional , introduces an alignment and the optional : a format specifier.

operators

Operator     ::= OperatorChar+
OperatorChar ::= [-!$%^&*+=|:@~#\<>.?/] | UnicodeSymbol

UnicodeSymbol is any character above U+007E that .NET classifies as a symbol (this admits operators such as ×, ÷, ∩, ∪, ∧, ∨, ≈, ≡).

Operators are tokenized greedily: the longest run of operator characters forms one operator, with one exception: a . immediately after a leading ! or ? ends the operator, so that x!.foo and x?.foo parse as a member access on an unwrap/has-value, not as the operators !. or ?..

A handful of operator spellings are recognised as dedicated tokens rather than general operators: =, :, ., ->, =>, ? and @.

statement terminators

Every ";" written in the productions below can be left off where the next token opens a new source line: the line break stands in for it. End of file ends a line too, so the last construct in a file needs no terminator. A ";" is only required between two constructs written on one line.

Terminator ::= ";" | Boundary

Boundary is not a token. It is the position before a token that is the first on its source line, and before end of input.

The parser accepts a Terminator only where the grammar could accept a ";", so the inference asks one question at one kind of position: is the current token the first on its line? That leaves the rest to the productions themselves. A line break ends a construct that is complete; one that is not runs on to the next line, so a trailing operator, an unclosed bracket, and an argument list still waiting for its ) need no rule at all.

line-start tokens

Four tokens continue a construct that is already complete, which is how member chains and pipes wrap:

ContinuationLead ::= "." | "?" | "|>" | "ref"

Five could have continued one - as a call, an index, an explicit generic application, a function literal's rec marker and an infix operand - and deliberately do not:

BoundaryLead ::= "(" | "[" | "`[" | "rec" | Operator

So a wrapped operator expression puts the operator at the end of the line rather than the start of the next, and a line-start rec is a recursive self-call rather than a marker for the expression above. Postfix modifiers follow the same rule without needing to be listed: a modifier is read only on its declaration's own line, so a line-start public, static or pure belongs to the next member.

constructs that end at a line break

Three productions consult the boundary directly rather than through a Terminator.

Return takes the next line's expression as its value where that line opens with a token that can begin an expression, and is a void return otherwise. The two readings never compete: a statement written after a return in the same block would be unreachable, so a closing keyword is the only thing that legitimately follows one.

A parenthesised group is a tuple or a block expression, and a boundary commits the block reading exactly as a written ";" does. A top-level "," commits the tuple reading, and has always arrived first when it is going to, so the two never contend. A line-start operator is excluded from the block commit, which keeps (a ... + b) from being misread as two statements.

Assert is the one construct whose reading depends on how far a line is indented. An else opening the line after a bare assert is the assert's own message clause where its column is at least the assert's, and the else of the enclosing if or case arm where it is dedented past it. This is the only place indentation is significant; everywhere else ghūl ignores it.

compilation unit

A source file is a sequence of definitions:

CompilationUnit ::= Definition*

Definition ::= Namespace
             | Use
             | Class
             | Trait
             | Struct
             | Union
             | Enum
             | Partial
             | Impl
             | Member
             | PragmaDefinition

A Member (function, property or indexer) appearing directly in a compilation unit or namespace is a global function, global variable or global indexer.

definitions

namespace and use

Namespace ::= "namespace" QualifiedIdentifier "is" Definition* "si"

Use ::= "use" QualifiedIdentifier ";"
      | "use" Identifier "=" QualifiedIdentifier ";"

The second form of Use introduces an alias for a namespace or symbol.

class, trait and struct

Class  ::= "class"  Identifier TypeParameters? PrimaryParameters? Ancestors? Modifiers
           ClassyBody

Trait  ::= "trait"  Identifier TypeParameters? Ancestors? Modifiers
           "is" Definition* "si"

Struct ::= "struct" Identifier TypeParameters? PrimaryParameters? Ancestors? Modifiers
           ClassyBody

ClassyBody ::= "is" ClassBodyDefinition* "si"
             | ";"           /* requires PrimaryParameters; equivalent to an empty `is` ... `si` body */

TypeParameters ::= "[" TypeParameter ( "," TypeParameter )* "]"
TypeParameter  ::= Identifier ( ":" TypeParameterConstraints )? Variance?
TypeParameterConstraints
               ::= TypeExpression KindConstraint? "new"?    /* type bound */
                 | KindConstraint "new"?                    /* kind only */
                 | "new"                                    /* ctor only */
KindConstraint ::= "class" | "struct" | "optional"
Variance       ::= "out" | "in"

Ancestors ::= ":" TypeList

PrimaryParameters     ::= "(" PrimaryParameter ( "," PrimaryParameter )* ")"
PrimaryParameter      ::= Identifier ":" TypeExpression PrimaryParamModifier?
PrimaryParamModifier  ::= "public" | "field" | "init"

ClassBodyDefinition ::= Definition | SuperCallDeclaration
SuperCallDeclaration ::= "super" "(" ExpressionList? ")" ";"

Ancestors lists the base class and/or implemented traits.

A type parameter has zero or more constraints: a type bound (which the actual type argument must derive from), a kind constraint (class / struct / optional), a constructor constraint (new), and on a trait a variance modifier (out for covariant, in for contravariant) - in that order. Only a single type bound per parameter is currently supported. Variance is only legal on a trait's type parameters.

PrimaryParameters declare a class or struct's primary constructor inline. Each parameter becomes a parameter of the synthesised init and an auto-generated field or property of the same name and declared type. A trailing modifier on a parameter overrides the default visibility - public for a public read-write property, field for a plain field, init to suppress field generation. A parameter named _x produces a private field; a body field or property declaration matching the parameter (under the same _x/x rule) overrides auto-generation.

A SuperCallDeclaration is a class-body shorthand for calling the superclass init with the given expressions. Each expression resolves with the primary parameters in scope. Primary parameters consumed by super(...) are excluded from auto-generation. A secondary init(.., extras) overload uses .. to splice the primary parameters into its argument list; an implicit chain to the primary init runs before the secondary's body.

union

Union   ::= "union" Identifier TypeParameters? Modifiers "is" Variant+ "si"
Variant ::= Identifier ( "(" VariableList ")" )? "default"? ";"

Each Variant optionally has fields, written as a parenthesised list of name: Type variables. A trailing default marks one variant as the union's default: the one the ? test and ! unwrap operators target on a union value.

partial and impl

Partial ::= "partial" TypeExpression "is" ClassBodyDefinition* "si"
Impl    ::= "impl" TypeExpression "for" TypeExpression "is" ClassBodyDefinition* "si"

A Partial block adds members to a class, struct, or union already declared in the same assembly. An Impl block additionally makes its target satisfy a trait; the trait's type arguments are written on the target after for (impl Printer for List[T]), and inside the body self has the target's type. Either target can be a qualified name, including a single union variant. See partial and impl blocks.

enum

Enum       ::= "enum" Identifier Modifiers "is"
               EnumMember ( "," EnumMember )* "si"
EnumMember ::= Identifier ( "=" Expression )?

members: functions, properties and indexers

A Member is a function, a property or an indexer. They share a leading name and modifiers; the parser distinguishes them by what follows the name.

Member ::= Function | Property | Indexer

function

Function     ::= FunctionName TypeParameters?
                 "(" VariableList? ")" ReturnType? Modifiers ( Body | ";" )
FunctionName ::= Identifier | Operator
ReturnType   ::= "->" TypeExpression

Body ::= "is" StatementList "si"
       | "=>" Expression
       | "innate" QualifiedIdentifier

A function may be named by an Operator, which defines that operator. A function with no body (just ;) is abstract. A => or innate body is terminated by ;; a block body (is … si) is not.

property

Property          ::= Identifier ( ":" TypeExpression )? Modifiers
                      PropertyAccessors? ";"?
PropertyAccessors ::= PropertyGetter ( "," PropertySetter )?
                    | PropertySetter ( "," PropertyGetter )?
PropertyGetter    ::= Body
PropertySetter    ::= "=" Identifier Body

A property with no accessors and the field modifier declares a field. A PropertySetter names the value parameter after =. As with functions, a => or innate accessor body is terminated by ; and a block body is not.

indexer

Indexer ::= Identifier? "[" Variable "]" ( ":" TypeExpression )? Modifiers
            PropertyAccessors? ";"?

modifiers

Modifiers      ::= AccessModifier? StorageClass? TypeModifier* "pure"?
AccessModifier ::= "public" | "protected" | "private"
StorageClass   ::= "static" | "field"
TypeModifier   ::= "abstract" | "open"

abstract and open are postfix modifiers on a class (abstract bars direct construction, open allows cross-assembly subclassing). pure is a postfix modifier asserting store-freedom - only reads, never writes to the heap - accepted on a function or method, on a function type, and on a class, struct or trait header (requiring every instance member to be pure). stable is a postfix modifier on a property asserting that two adjacent reads agree on presence and runtime type.

pragmas

PragmaDefinition ::= Pragma Definition
Pragma           ::= "@" QualifiedIdentifier ( "(" ExpressionList? ")" )?

A Pragma annotates the definition (or statement) that follows it.

type expressions

TypeExpression ::= PrimaryType TypeSuffix*

PrimaryType ::= QualifiedIdentifier
              | QualifiedIdentifier "[" TypeList "]"   /* generic type */
              | QualifiedIdentifier "[" "]"            /* array type */
              | Identifier ":" TypeExpression          /* named tuple element */
              | "(" TypeList ")"                       /* tuple, or grouping */
              | "(" TypeList? ")" "->" TypeExpression  /* function type */

TypeSuffix ::= "[]"                  /* array */
             | "ref"                 /* by-reference */
             | "ptr"                 /* pointer */
             | "?"                   /* nullable */
             | "->" TypeExpression   /* function type */
             | "." Identifier        /* member type */

TypeList ::= TypeExpression ( "," TypeExpression )*

( T ) is just T in parentheses; parentheses group, e.g. to disambiguate (a -> b) -> c from a -> b -> c. A parenthesised list of two or more types is a tuple type. Empty parentheses are meaningful only as ( ) -> T, a function type taking no arguments. A name: Type element gives a tuple element a name.

variables

Variable     ::= VariableLeft ( ":" TypeExpression )? "mut"? ( "=" Expression )?
VariableLeft ::= Identifier
               | "(" VariableLeft ( "," VariableLeft )* ")"
VariableList ::= Variable ( "," Variable )*

The parenthesised form of VariableLeft destructures a tuple. A bare let local variable is immutable unless followed by mut.

statements

A statement list is a sequence of statements, separated by terminators. The terminator has no meaning of its own: a function body's tail value is judged by its type, so whether the last statement is terminated never changes what the body returns.

StatementList ::= ( Statement Terminator? )*

Statement ::= Let
            | Return
            | Throw
            | Assert
            | Yield
            | If
            | Case
            | Try
            | Loop
            | For
            | Break
            | Continue
            | PragmaStatement
            | Labelled
            | Assignment
            | ExpressionStatement

local variable definitions, return, throw, assert, yield

Let    ::= "let" "use"? VariableList ( "in" Expression )?
Return ::= "return" Expression?
Throw  ::= "throw" Expression?
Assert ::= "assert" Expression ( "else" Expression )? ( "in" Expression )?
Yield  ::= "yield" Expression

let use defines a local variable holding a disposable, whose dispose is called when the variable goes out of scope.

The let … in … form is a let-in expression used as a statement. The assert … in … tail behaves the same way: a passing assert yields the trailing expression, a failing one throws.

yield is permitted only inside a generator function, one whose return type is Ghul.Pipes.Pipe[T].

if

If          ::= "if"   IfCondition "then" StatementList
                ( "elif" IfCondition "then" StatementList )*
                ( "else"             StatementList )?
                "fi"
IfCondition ::= Expression
              | "let" Variable        /* if-let local variable */

The if let form defines a local variable whose initializer must be present; a type ascription on it (if let c: T = e) tests that the value is a T.

case

Case ::= "case" Expression
         ( "when" ( ExpressionList | Variable ) "then" StatementList )*
         ( "else" StatementList )?
         "esac"

Each when takes either a comma-separated list of value-equality expressions or a pattern, matching the same type-test, destructure, and literal-leaf forms as if let. case is also an expression: each arm's last expression is the arm's value.

try

Try ::= "try" StatementList
        ( "catch" Variable StatementList )*
        ( "finally" StatementList )?
        "yrt"

loops

Loop ::= ( "while" Expression )? "do" StatementList "od"
For  ::= "for" Variable "in" Expression "do" StatementList "od"

A do … od with no while is an unconditional loop.

break, continue and labels

Break    ::= "break" Identifier?
Continue ::= "continue" Identifier?
Labelled ::= Identifier ":" Statement

A Labelled statement may be targeted by break or continue with the matching label.

assignment and expression statements

Assignment          ::= Expression "=" Expression
ExpressionStatement ::= Expression

PragmaStatement     ::= Pragma Statement

expressions

An expression is a sequence of operands joined by binary operators. The parser resolves operator nesting by precedence; the grammar below gives the flat structure.

Expression ::= UnaryExpression ( Operator UnaryExpression )*

|| is the yield infix used to produce a value from a generator step; it has the lowest precedence and does not chain.

unary expressions

UnaryExpression ::= Operator UnaryExpression      /* prefix operator */
                  | "await"  UnaryExpression      /* await expression */
                  | PostfixExpression

An await E expression is permitted only inside an asynchronous function, one whose return type is Tasks.TASK[T] (or Tasks.TASK), and evaluates to the result of the awaited task once it completes. Used as the right-hand side of let, as a bare statement (await E;), or in any operand position.

postfix expressions

PostfixExpression ::= PrimaryExpression PostfixSuffix*

PostfixSuffix ::= "(" ExpressionList? ")"     /* call */
                | "[" ExpressionList "]"      /* index expression, or generic application */
                | "`[" TypeList "]"           /* explicit generic application */
                | "." Identifier              /* member access */
                | "?"                         /* has-value test */
                | "!"                         /* unwrap */
                | "ref"                       /* by-reference */
                | "|"                         /* pipe */
                | "|>" PostfixExpression      /* thread-first call */

A [ ... ] suffix is either an index expression (an access through an indexer) or a generic type application, depending on whether its contents resolve as expressions or as types; `[ ... ] forces the generic-application reading.

The |> thread-first suffix routes its left side into the call on its right as that call's first argument, so x |> f(a) is f(x, a). Its right side must be call-shaped, and chaining is left-associative, so x |> f(a) |> g(b) is g(f(x, a), b). See thread-first calls.

function literals

A primary expression (or a parenthesised argument list) followed by ->, =>, is or rec is a function literal:

FunctionLiteral   ::= FunctionArguments ( "->" TypeExpression )? "rec"? Body
FunctionArguments ::= "(" VariableList? ")"
                    | Identifier

rec marks the literal as recursive, so it may refer to itself.

primary expressions

PrimaryExpression ::= Identifier
                    | Literal
                    | "(" ExpressionList? ")"                /* tuple or grouping */
                    | "[" ExpressionList "]" ( ":" TypeExpression )?  /* list literal */
                    | "cast" TypeExpression "(" Expression ")"
                    | "isa" TypeExpression "(" Expression ")"
                    | "typeof" TypeExpression
                    | "default" ( "[" TypeExpression "]" )?
                    | "self"
                    | "super"
                    | "rec"
                    | If                                     /* if-expression */
                    | Case                                   /* case-expression */
                    | "(" StatementList ")"                  /* block expression */
                    | "val" StatementList "lav"              /* block expression, historical spelling */
                    | "let" "use"? VariableList "in" Expression   /* let-in */
                    | "assert" Expression ( "else" Expression )? "in" Expression  /* assert-in */

Literal ::= IntegerLiteral
          | FloatLiteral
          | StringLiteral
          | CharLiteral
          | InterpolatedString
          | "true" | "false"
          | "null"

ExpressionList ::= Expression ( "," Expression )*

A list literal [ a, b, ... ] builds a List; it requires at least one element (use LIST[T]() for an empty list).

Within an ExpressionList that forms call arguments or a tuple, an element of the form Identifier ":" TypeExpression? ( "=" Expression )? is an inline local variable definition rather than a plain identifier; this is the only place that form is accepted.

operator precedence

ghūl has no fixed list of binary operators: any operator token may be used infix. Precedence is assigned by a table of built-in operators plus a first-character heuristic for everything else, so the grammar's flat Expression ::= UnaryExpression ( Operator UnaryExpression )* is disambiguated by the following levels, tightest first:

Precedence	Operators
(prefix unary, member access, call, index - tightest)
user‑8	(user-defined)
multiplication	* × ✕ / % ÷
user‑7	(user-defined)
addition	+ -
user‑6	(user-defined)
bitwise	& | ¦ ^ ∩ ∪
user‑5	(user-defined, default)
shift	<< >> >>>
user‑4	(user-defined)
range	.. ::
user‑3	(user-defined)
relational	== != =~ !~ < > >= <= ≈ ≡
user‑2	(user-defined)
boolean	/\ \/ ∧ ∨
user‑1	(user-defined)
yield infix	||

All binary operators are left-associative. Prefix unary operators, member access, calls and indexing bind more tightly than any binary operator.

A user-defined operator (any operator not in the table above) is assigned a precedence from its first character, modelled on OCaml and F#: operators starting with * / % bind as multiplication, + - as addition, and so on; an operator with no recognised first character defaults to user‑5. The @precedence("op", "level") pragma overrides the precedence of a named operator. Both arguments must be string literals (a numeric level is not accepted), and level names a precedence level: user-1 … user-8, or one of the built-in level names boolean, relational, range, shift, bitwise, addition and multiplication.