Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: AST #14

Merged
merged 66 commits into from
Nov 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
6b4983c
feat: AST types
CodyJasonBennett Mar 21, 2023
6f6793a
Merge branch 'main' into ast
CodyJasonBennett Jun 20, 2023
efaffe1
refactor: narrow ReturnStatement
CodyJasonBennett Jun 20, 2023
e8590f3
feat: parser, type branding
CodyJasonBennett Jun 21, 2023
6583245
feat(parser): PoC reflection pass
CodyJasonBennett Nov 8, 2023
aa29f5d
chore: lint
CodyJasonBennett Nov 8, 2023
4b5e049
refactor: Node => AST for TS
CodyJasonBennett Nov 8, 2023
790523c
chore: inline bitmap sweep
CodyJasonBennett Nov 8, 2023
80a15b7
refactor: use classes for AST
CodyJasonBennett Nov 8, 2023
ba9bb97
fix: harden qualifiers
CodyJasonBennett Nov 8, 2023
68d0908
refactor: more basic parsing of methods, block statements
CodyJasonBennett Nov 8, 2023
a990ca4
chore: update demo
CodyJasonBennett Nov 8, 2023
b6e197c
Merge branch 'main' into ast
CodyJasonBennett Nov 8, 2023
c41e02a
chore: cleanup
CodyJasonBennett Nov 8, 2023
080e520
refactor: parse nested blocks
CodyJasonBennett Nov 8, 2023
a15c438
chore: abstract statement parse
CodyJasonBennett Nov 8, 2023
9b28dd1
refactor: parse while, do-while, switch-case
CodyJasonBennett Nov 8, 2023
7b87e70
chore: cleanup
CodyJasonBennett Nov 8, 2023
cfe7379
chore: cleanup
CodyJasonBennett Nov 8, 2023
03b4c3a
chore: parseBody => parseStatements
CodyJasonBennett Nov 8, 2023
b6f16ab
refactor: AST union extends Node
CodyJasonBennett Nov 8, 2023
8c60a90
refactor: harden qualifier gather
CodyJasonBennett Nov 9, 2023
16d5849
refactor: cleanup var scan
CodyJasonBennett Nov 9, 2023
4f34a8d
refactor: handle default switch case
CodyJasonBennett Nov 9, 2023
7745c22
refactor: parse block scopes
CodyJasonBennett Nov 9, 2023
1118ca2
refactor: cleanup for parse
CodyJasonBennett Nov 9, 2023
3707f38
refactor: cleanup for parse
CodyJasonBennett Nov 9, 2023
9d8dc7d
refactor: skip assignment expressions
CodyJasonBennett Nov 9, 2023
9652414
fix: unmangle
CodyJasonBennett Nov 9, 2023
1dffe01
refactor: back-track for declarations
CodyJasonBennett Nov 9, 2023
8acba65
chore: cleanup
CodyJasonBennett Nov 10, 2023
5380511
refactor: parse struct
CodyJasonBennett Nov 10, 2023
1f8a515
refactor: functions have optional body
CodyJasonBennett Nov 10, 2023
dd06092
refactor: parse struct members
CodyJasonBennett Nov 10, 2023
95f40e4
refactor: split gather into read/consume
CodyJasonBennett Nov 10, 2023
7773c80
refactor: parse expressions
CodyJasonBennett Nov 11, 2023
e369a4b
refactor: parse args
CodyJasonBennett Nov 11, 2023
12e44e0
refactor: Type node for complex types
CodyJasonBennett Nov 12, 2023
6aa441b
refactor: Type accepts optional parameters
CodyJasonBennett Nov 12, 2023
7e66faa
chore(tests): add GLSL expression coverage
CodyJasonBennett Nov 12, 2023
5f49871
refactor: callexpr from accessor
CodyJasonBennett Nov 12, 2023
ca16ea5
chore: split up agnostic tests
CodyJasonBennett Nov 12, 2023
2a216fa
chore(tests): add GLSL statements coverage
CodyJasonBennett Nov 13, 2023
a95ed6f
refactor: parse for init as VariableDeclaration
CodyJasonBennett Nov 14, 2023
11b81ef
refactor: parse switch case scope
CodyJasonBennett Nov 14, 2023
25729a5
chore: cleanup
CodyJasonBennett Nov 14, 2023
2444cbb
refactor: track unary handedness
CodyJasonBennett Nov 16, 2023
d0e3ac9
feat: generator (GLSL)
CodyJasonBennett Nov 16, 2023
3ac993b
refactor: fix unary handedness
CodyJasonBennett Nov 17, 2023
23451f7
chore: cleanup empty block scope
CodyJasonBennett Nov 17, 2023
2238f4e
refactor: precision statements
CodyJasonBennett Nov 17, 2023
5a619a3
refactor: layout qualifiers
CodyJasonBennett Nov 17, 2023
6eac369
refactor: method type qualifiers
CodyJasonBennett Nov 17, 2023
caf366c
refactor: var declarators, kind
CodyJasonBennett Nov 18, 2023
dbef312
refactor: comma-separated lists
CodyJasonBennett Nov 18, 2023
c661968
chore: cleanup
CodyJasonBennett Nov 18, 2023
86e2dbf
refactor: parse var array type
CodyJasonBennett Nov 19, 2023
734686c
refactor: array expressions
CodyJasonBennett Nov 19, 2023
3a3c581
refactor: preprocessor statements
CodyJasonBennett Nov 19, 2023
c1054ad
refactor: callexpr + memberexpr
CodyJasonBennett Nov 19, 2023
95a220d
refactor: parse user types
CodyJasonBennett Nov 19, 2023
1fbea18
refactor: harden qualifier match
CodyJasonBennett Nov 21, 2023
5b0e90e
refactor: fix comma-separated list case
CodyJasonBennett Nov 21, 2023
c1c7215
chore: fix layout case
CodyJasonBennett Nov 21, 2023
241bae7
chore: cleanup
CodyJasonBennett Nov 21, 2023
aec2489
chore: add compiler docs
CodyJasonBennett Nov 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
355 changes: 352 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,41 @@

Tools and IntelliSense for GLSL and WGSL.

## Table of Contents

- [Installation](#installation)
- [Tokenize](#tokenize)
- [Minify](#minify)
- [Parse](#parse)
- [Generate](#generate)
- [AST](#ast)
- [Literal](#literal)
- [Identifier](#identifier)
- [Type](#type)
- [VariableDeclaration](#variabledeclaration)
- [VariableDeclarator](#variabledeclarator)
- [StructDeclaration](#structdeclaration)
- [FunctionDeclaration](#functiondeclaration)
- [UnaryExpression](#unaryexpression)
- [BinaryExpression](#binaryexpression)
- [TernaryExpression](#ternaryexpression)
- [CallExpression](#callexpression)
- [MemberExpression](#memberexpression)
- [ArrayExpression](#arrayexpression)
- [BlockStatement](#blockstatement)
- [IfStatement](#ifstatement)
- [ForStatement](#forstatement)
- [WhileStatement](#whilestatement)
- [DoWhileStatement](#dowhilestatement)
- [SwitchStatement](#switchstatement)
- [SwitchCase](#switchcase)
- [ReturnStatement](#returnstatement)
- [PreprocessorStatement](#preprocessorstatement)
- [PrecisionStatement](#precisionstatement)
- [ContinueStatement](#continuestatement)
- [BreakStatement](#breakstatement)
- [DiscardStatement](#discardstatement)

## Installation

To install, use your preferred package manager:
Expand All @@ -28,6 +63,13 @@ Or, use a CDN:

Tokenizes a string of GLSL or WGSL code, returning an array of `Token` objects, where each `Token` object represents a single syntax feature in the input code.

```ts
interface Token {
type: 'whitespace' | 'comment' | 'symbol' | 'bool' | 'float' | 'int' | 'identifier' | 'keyword'
value: string
}
```

<details>
<summary>GLSL Example</summary>

Expand Down Expand Up @@ -175,11 +217,318 @@ const minified: string = minify(code: string, {
To shared mangled interfaces when using `mangleExternal`, declare and re-use a `mangleMap` between shaders:

```ts
const programOpts = { mangle: true, mangleExternals: true, mangleMap: new Map() }
const options = { mangle: true, mangleExternals: true, mangleMap: new Map() }

// #version 300 es\nin vec2 a;out vec2 b;void main(){b=a;}
minify(`#version 300 es\nin vec2 sstt;out vec2 c;void main(){c=sstt;}`, programOpts)
minify(`#version 300 es\nin vec2 sstt;out vec2 c;void main(){c=sstt;}`, options)

// #version 300 es\nin vec2 b;out vec4 a[gl_MaxDrawBuffers];void main(){a[0]=b.sstt;}
minify(`#version 300 es\nin vec2 c;out vec4 data[gl_MaxDrawBuffers];void main(){data[0]=c.sstt;}`, programOpts)
minify(`#version 300 es\nin vec2 c;out vec4 data[gl_MaxDrawBuffers];void main(){data[0]=c.sstt;}`, options)
```

## Parse

Parses a string of GLSL (WGSL is WIP) code into an [AST](#ast).

```ts
const ast: AST[] = parse(code: string)
```

## Generate

Generates a string of GLSL (WGSL is WIP) code from an [AST](#ast).

```ts
const code: string = generate(ast: AST[], {
target: 'GLSL' // | 'WGSL'
})
```

## AST

An [Abstract Syntax Tree](https://en.wikipedia.org/wiki/Abstract_syntax_tree) loosely based on [ESTree](https://github.com/estree/estree) for GLSL and WGSL grammars.

### Literal

A shader literal representing a `bool`, `float`, `int`, or `uint` type.

```ts
class Literal {
value: string
}
```

### Identifier

A variable identifier.

```ts
class Identifier {
value: string
}
```

### Type

Represents a type specifier and its parameters (WGSL specific).

```ts
class Type {
name: string
parameters: (Type | Literal | Identifier)[] | null
}
```

### VariableDeclaration

A variable declaration with an optional binding layout, type qualifiers, kind (WGSL only), and declarators (e.g. a comma-separated list).

```ts
class VariableDeclaration {
layout: Record<string, string | boolean> | null
qualifiers: string[]
kind: 'var' | 'let' | 'const' | null
type: Type | Identifier
declarations: VariableDeclarator[]
}
```

#### VariableDeclarator

A single named declarator as part of a `VariableDeclaration`.

```ts
class VariableDeclarator {
name: string
value: AST | null
}
```

### StructDeclaration

A struct declaration. Can be used as a type or constructor.

```ts
class StructDeclaration {
name: string
members: VariableDeclaration[]
}
```

### FunctionDeclaration

A function declaration with an optional type qualifier and arguments.

```ts
class FunctionDeclaration {
name: string
type: Type | Identifier
qualifiers: string[]
args: VariableDeclaration[]
body: BlockStatement | null
}
```

### UnaryExpression

A unary expression with a left or right handed operator.

```ts
class UnaryExpression {
operator: string
left: AST | null
right: AST | null
}
```

### BinaryExpression

A binary expression with a left and right operand.

```ts
class BinaryExpression {
operator: string
left: AST
right: AST
}
```

### TernaryExpression

A ternary or conditional expression.

```ts
class TernaryExpression {
test: AST
consequent: AST
alternate: AST
}
```

### CallExpression

A call expression.

```ts
class CallExpression {
callee: AST
args: AST[]
}
```

### MemberExpression

A member expression.

```ts
class MemberExpression {
object: AST
property: AST
}
```

### ArrayExpression

An array expression. `members` can be empty if uninitialized.

```ts
class ArrayExpression {
type: Type
members: AST[]
}
```

### BlockStatement

A block statement.

```ts
class BlockStatement {
body: AST[]
}
```

### IfStatement

An if statement.

```ts
class IfStatement {
test: AST
consequent: AST
alternate: AST | null
}
```

### ForStatement

A for statement.

```ts
class ForStatement {
init: AST | null
test: AST | null
update: AST | null
body: AST
}
```

### WhileStatement

A while statement.

```ts
class WhileStatement {
test: AST
body: AST
}
```

### DoWhileStatement

A do-while statement.

```ts
class DoWhileStatement {
test: AST
body: AST
}
```

### SwitchStatement

A switch statement.

```ts
class SwitchStatement {
discriminant: AST
cases: SwitchCase[]
}
```

#### SwitchCase

A switch-case statement. `test` is null for a `default` case.

```ts
class SwitchCase {
test: AST | null
consequent: AST[]
}
```

### ReturnStatement

A return statement with an optional argument.

```ts
class ReturnStatement {
argument: Literal | Identifier | UnaryExpression | null
}
```

### PreprocessorStatement

A GLSL preprocessor statement with an optional value.

```ts
class PreprocessorStatement {
name: string
value: AST[] | null
}
```

### PrecisionStatement

A GLSL precision statement.

```ts
class PrecisionStatement {
precision: 'lowp' | 'mediump' | 'highp'
type: Type
}
```

### ContinueStatement

A continue statement.

```ts
class ContinueStatement {}
```

### BreakStatement

A break statement.

```ts
class BreakStatement {}
```

### DiscardStatement

A discard statement.

```ts
class DiscardStatement {}
```
7 changes: 5 additions & 2 deletions demo/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@
}
</style>
<script type="module">
{
import 'shaderkit'
</script>
<script type="module">
if (false) {
const canvas = document.body.appendChild(document.createElement('canvas'))
const gl = canvas.getContext('webgl2')
const vertex = gl.createShader(gl.VERTEX_SHADER)
Expand Down Expand Up @@ -49,7 +52,7 @@
}
</script>
<script type="module">
{
if (false) {
const canvas = document.body.appendChild(document.createElement('canvas'))
const context = canvas.getContext('webgpu')
const format = navigator.gpu.getPreferredCanvasFormat()
Expand Down
Loading
Loading