Autosplitting Language Grammar

[just-ero]

Hello,

I've been trying to figure it out by myself for a while now, but keep running into some problems with the grammar I want to create. The grammar should parse LiveSplit's Autosplitting Language, a scripting language containing one custom block and several other blocks containing plain C# code.

The script must begin with 1 or more state blocks, defining a process name and, optionally, a version. The state block must open and close with braces and can contain 0 or more variable declarations:

state("ProcessName") {
  int Foo : 0x123456, 0x78;
  bool Bar : 0x123456, 0x78, 0x90
}

state("ProcessName", "version") {
  string32 Baz : "Module.dll", 0x123456;
}

init { /* C# code */ }

• The strings in the state arguments should be allowed to take any valid C# string.
• Allowed types are the following:

  Type : 'bool' | 'sbyte' | 'byte' | 'short' | 'ushort' | 'int' | 'uint' | 'long' | 'ulong' | 'float' | 'double' | 'string' [0-9]+ | 'byte' [0-9]+;

• The identifier must be any valid C# identifier.
• An optional module name can follow, separated with a comma.
• At least one integer (signed, any valid C# kind; decimal, hex, binary), optionally followed by any amount of additional integers.
• Must conclude with trailing semicolon, except for the final declaration (backwards compatibility with our previous parser).

---

Grammar Files

grammar BaseGrammar;

fragment CSharpCharacter
    : '\\u' HexDigit HexDigit HexDigit HexDigit
    | '\\x' HexDigit HexDigit HexDigit HexDigit
    | '\\x' HexDigit HexDigit
    | '\\' [0abfnrtv'"\\]
    | ~["\\]
    ;

fragment StringLiteral
    : '"' CSharpCharacter*? '"'
    ;

fragment Digit
    : [0-9]
    ;

fragment HexDigit
    : [0-9a-fA-F]
    ;

fragment DecimalNumber
    : '-'? [0-9][0-9_]*
    ;

fragment HexNumber
    : '-'? '0x' [0-9a-fA-F_]+
    ;

fragment BinaryNumber
    : '-'? '0b' [01_]+
    ;

fragment Number
    : DecimalNumber
    | HexNumber
    | BinaryNumber
    ;

WS
    : [ \t\r\n]+
    -> skip
    ;

grammar StateDeclarationGrammar;

import BaseGrammar;

Type
    : 'bool'
    | 'sbyte'
    | 'byte'
    | 'short'
    | 'ushort'
    | 'int'
    | 'uint'
    | 'long'
    | 'ulong'
    | 'float'
    | 'double'
    | 'string' Digit+
    | 'byte' Digit+
    ;

Id : [_a-zA-Z][_a-zA-Z0-9]* ;

Module : StringLiteral ;
BaseOffset : Number ;
Offset : Number ;

declaration : Type Id ':' (Module ',')? BaseOffset (',' Offset)* ';' ;

grammar StateGrammar;

import StateDeclarationGrammar;

ProcessName : StringLiteral ;
Version : StringLiteral ;

state : 'state' '(' ProcessName (',' Version)? ')' '{' declaration* '}' ;

grammar AslGrammar;

import StateGrammar;

fragment Comment
    : '/*' .*? '*/'
    | '//' ~('\r' | '\n')*
    ;

Skip
    : (StringLiteral | Comment)
    -> skip
    ;

Name
    : 'startup'
    | 'onStart'
    | 'onSplit'
    | 'onReset'
    | 'init'
    | 'update'
    | 'start'
    | 'split'
    | 'reset'
    | 'gameTime'
    | 'isLoading'
    | 'exit'
    | 'shutdown'
    ;

fragment BlockBody : .*? ;
Block : '{' BlockBody '}' ;

action : Name Block ;

script : state+ action* ;

---

Unfortunately, I already run into an issue with the process name: line 1:8 missing ProcessName at ')'
as well as another strange problem: line 1:10 mismatched input '{ }' expecting '{'

This is my input:

state("") { }

Since my StringLiteral rule is a fragment, I'm not sure how I'm supposed to include it in my parser rule state.

How can I work around this problem?
Also, is there a good way to extract the contents of the actions' body?
Anything I can clean up?

[jimidle]

I think you need to step back and read through some existing grammars and tutorials. The grammar you have is not close to being valid.

…

On Wed, Aug 21, 2024 at 16:10 Ero ***@***.***> wrote: Hello, I've been trying to figure it out by myself for a while now, but keep running into some problems with the grammar I want to create. The grammar should parse LiveSplit's Autosplitting Language, a scripting language containing one custom block and several other blocks containing plain C# code. The script must begin with 1 or more state blocks, defining a process name and, optionally, a version. The state block must open and close with braces and can contain 0 or more variable declarations: state("ProcessName") { int Foo : 0x123456, 0x78; bool Bar : 0x123456, 0x78, 0x90} state("ProcessName", "version") { string32 Baz : "Module.dll", 0x123456;} init { /* C# code */ } - The strings in the state arguments should be allowed to take any valid C# string. - Allowed types are the following: Type : 'bool' | 'sbyte' | 'byte' | 'short' | 'ushort' | 'int' | 'uint' | 'long' | 'ulong' | 'float' | 'double' | 'string' [0-9]+ | 'byte' [0-9]+; - The identifier must be any valid C# identifier. - An optional module name can follow, separated with a comma. - At least one integer (signed, any valid C# kind; decimal, hex, binary), optionally followed by any amount of additional integers. - Must conclude with trailing semicolon, *except* for the final declaration (backwards compatibility with our previous parser). ------------------------------ Grammar Files grammar BaseGrammar; fragment CSharpCharacter : '\\u' HexDigit HexDigit HexDigit HexDigit | '\\x' HexDigit HexDigit HexDigit HexDigit | '\\x' HexDigit HexDigit | '\\' [0abfnrtv'"\\] | ~["\\] ;fragment StringLiteral : '"' CSharpCharacter*? '"' ;fragment Digit : [0-9] ;fragment HexDigit : [0-9a-fA-F] ;fragment DecimalNumber : '-'? [0-9][0-9_]* ;fragment HexNumber : '-'? '0x' [0-9a-fA-F_]+ ;fragment BinaryNumber : '-'? '0b' [01_]+ ;fragment Number : DecimalNumber | HexNumber | BinaryNumber ;WS : [ \t\r\n]+ -> skip ; grammar StateDeclarationGrammar; import BaseGrammar; Type : 'bool' | 'sbyte' | 'byte' | 'short' | 'ushort' | 'int' | 'uint' | 'long' | 'ulong' | 'float' | 'double' | 'string' Digit+ | 'byte' Digit+ ; Id : [_a-zA-Z][_a-zA-Z0-9]* ; Module : StringLiteral ;BaseOffset : Number ;Offset : Number ; declaration : Type Id ':' (Module ',')? BaseOffset (',' Offset)* ';' ; grammar StateGrammar; import StateDeclarationGrammar; ProcessName : StringLiteral ;Version : StringLiteral ; state : 'state' '(' ProcessName (',' Version)? ')' '{' declaration* '}' ; grammar AslGrammar; import StateGrammar; fragment Comment : '/*' .*? '*/' | '//' ~('\r' | '\n')* ; Skip : (StringLiteral | Comment) -> skip ; Name : 'startup' | 'onStart' | 'onSplit' | 'onReset' | 'init' | 'update' | 'start' | 'split' | 'reset' | 'gameTime' | 'isLoading' | 'exit' | 'shutdown' ; fragment BlockBody : .*? ;Block : '{' BlockBody '}' ; action : Name Block ; script : state+ action* ; ------------------------------ Unfortunately, I already run into an issue with the process name: line 1:8 missing ProcessName at ')' as well as another strange problem: line 1:10 mismatched input '{ }' expecting '{' This is my input: state("") { } Since my StringLiteral rule is a fragment, I'm not sure how I'm supposed to include it in my parser rule state. How can I work around this problem? Also, is there a good way to extract the contents of the actions' body? Anything I can clean up? — Reply to this email directly, view it on GitHub <#4678>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJ7TMCOJ54J2XTMJKJJ3CTZSUF47AVCNFSM6AAAAABM4426YSVHI2DSMVQWIX3LMV43ERDJONRXK43TNFXW4OZXGA4DANRWGA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[just-ero]

After some research, I've adjusted my files to this:

AslLexer.g4

lexer grammar AslLexer;

WS
    : [ \t\r\n]+
    -> skip
    ;

// Literals
Minus : '-' ;
Quote : '"' ;
Comma : ',' ;
Colon : ':' ;
LeftParen : '(' ;
RightParen : ')' ;
LeftBrace : '{' ;
RightBrace : '}' ;
Semicolon : ';' ;
StateWord : 'state' ;

// C# Identifiers
fragment IdentifierStart
    : [a-zA-Z_]
    ;

fragment IdentifierPart
    : [a-zA-Z0-9_]
    ;

Identifier : IdentifierStart IdentifierPart* ;

// Strings
fragment CSharpCharacter
    : '\\u' HexDigit HexDigit HexDigit HexDigit
    | '\\x' HexDigit HexDigit HexDigit HexDigit
    | '\\x' HexDigit HexDigit
    | '\\' [0abfnrtv'"\\]
    | ~["\\]
    ;

StringLiteral
    : Quote CSharpCharacter*? Quote
    ;

// Digits
fragment Digit
    : [0-9]
    ;

fragment HexDigit
    : [0-9a-fA-F]
    ;

Digits
    : Digit+
    ;

// C# number literals
fragment DecimalNumber
    : Minus? [0-9] [0-9_]*
    ;

fragment HexNumber
    : Minus? '0' [xX] [0-9a-fA-F_]+
    ;

fragment BinaryNumber
    : Minus? '0' [bB] [01_]+
    ;

Number
    : DecimalNumber
    | HexNumber
    | BinaryNumber
    ;

// Comments
fragment DelimitedCommentStart : '/*' ;
fragment DelimitedCommentEnd : '*/' ;
fragment DelimitedComment
    : DelimitedCommentStart .*? DelimitedCommentEnd
    ;

fragment LineCommentStart : '//' ;
fragment LineComment
    : LineCommentStart ~[\r\n]*
    ;

Comment
    : DelimitedComment
    | LineComment
    ;

// ActionName
fragment Startup : 'startup' ;
fragment OnStart : 'onStart' ;
fragment OnSplit : 'onSplit' ;
fragment OnReset : 'onReset' ;
fragment Init : 'init' ;
fragment Update : 'update' ;
fragment Start : 'start' ;
fragment Split : 'split' ;
fragment Reset : 'reset' ;
fragment GameTime : 'gameTime' ;
fragment IsLoading : 'isLoading' ;
fragment Exit : 'exit' ;
fragment Shutdown : 'shutdown' ;

ActionName
    : Startup
    | OnStart
    | OnSplit
    | OnReset
    | Init
    | Update
    | Start
    | Split
    | Reset
    | GameTime
    | IsLoading
    | Exit
    | Shutdown
    ;

// Types
fragment Bool : 'bool' ;
fragment SByte : 'sbyte' ;
fragment Byte : 'byte' ;
fragment Short : 'short' ;
fragment UShort : 'ushort' ;
fragment Int : 'int' ;
fragment UInt : 'uint' ;
fragment Long : 'long' ;
fragment ULong : 'ulong' ;
fragment Float : 'float' ;
fragment Double : 'double' ;
fragment String : 'string' ;

Type
    : Bool
    | SByte
    | Byte
    | Short
    | UShort
    | Int
    | UInt
    | Long
    | ULong
    | Float
    | Double
    | String Digits
    | Byte Digits
    ;

// Codeblocks
fragment BlockBody : .*? ;

CodeBlock
    : LeftBrace BlockBody RightBrace
    ;

AslParser.g4

parser grammar AslParser;

options {
    tokenVocab = AslLexer;
}

declaration
    : Type Id=Identifier Colon (ModuleName=StringLiteral Comma)? BaseOffset=Number (Comma Offsets+=Number)* Semicolon
    ;

state
    : StateWord LeftParen ProcessName=StringLiteral (Comma Version=StringLiteral)? RightParen LeftBrace declaration* RightBrace
    ;

action : Name=ActionName CodeBlock ;

script : state+ action* ;

But I'm still running into some problems:

• input: state("foo"){}
  • error: line 1:12 mismatched input '{}' expecting '{'

[just-ero]

I found out that it's due to these rules:

CodeBlock
    : BraceL BlockBody BraceR
    ;

fragment BlockBody : .*? ;

This captures {} as a CodeBlock, instead of the required part in the state rule.

What do I need to do to capture this correctly? Specifically, I need whitespace to stay intact inside each block's braces.

[just-ero]

I was a bit more thorough at https://stackoverflow.com/q/78901061/26432683.