Skip to content

Commit

Permalink
Add demo and improve text.
Browse files Browse the repository at this point in the history
Gohla committed Nov 27, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
1 parent 3722573 commit 8d971fe
Showing 2 changed files with 26 additions and 6 deletions.
2 changes: 1 addition & 1 deletion src/4_example/c_4_grammar.pest
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
num = { ASCII_DIGIT+ }
num = @{ ASCII_DIGIT+ }

main = { SOI ~ num ~ EOI }

30 changes: 25 additions & 5 deletions src/4_example/index.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,25 @@
# Example: Interactive Parser Development

To demonstrate what can be done with the programmatic incremental build system we just created, we will create a simple "interactive parser development" example.
In this example, we can interactively develop a grammar for a new (programming) language, and test that grammar against several example files written in the new language.
To demonstrate what can be done with the programmatic incremental build system we just created, we will create a simple "parser development" example.
In this example, we can develop a grammar for a new (programming) language, and test that grammar against several example files written in the new language.

It will have both a batch mode and an interactive mode.
In the batch mode, the grammar is checked and compiled, the example program files are parsed with the grammar, and the results are printed to the terminal.
The interactive mode will start up an interactive editor in which we can develop and test the grammar interactively.
We will develop tasks to perform grammar compilation and parsing, and incrementally execute them with PIE.
Both batch and interactive mode will use the same tasks!

We will use [pest](https://pest.rs/) as the parser framework, because it is written in Rust and can be easily embedded into an application.
Pest uses Parsing Expression Grammars (PEGs) which are easy to understand, which is also good for this example.

For the GUI, we will use [Ratatui](https://ratatui.rs/), which is a cross-platform terminal GUI framework, along with [tui-textarea](https://github.com/rhysd/tui-textarea) for a text editor widget.
We could use a more featured GUI framework like [egui](https://github.com/emilk/egui), but for this example we'll keep it simple and runnable in a terminal.

As a little teaser, this is what the interactive mode looks like:

<script src="https://asciinema.org/a/VfP8uiZ0MSs5QgzY0BhIxKp6L.js" id="asciicast-VfP8uiZ0MSs5QgzY0BhIxKp6L" async data-autoplay="true" data-loop="true" data-speed="1.25" data-idleTimeLimit="1" data-theme="solarized-dark"></script>
[//]: # (![Interactive mode demo]&#40;demo.gif&#41;)

We will continue as follows:

1) Implement compilation of pest grammars and parsing of text with the compiled grammar.
@@ -195,13 +206,22 @@ To test this out, we need a grammar and some test files. Create `grammar.pest`:
```

```admonish info title="Pest Grammars"
It's not important for this example to understand pest grammars, but I will explain the basics of this grammar.
You don't need to fully understand pest grammars to finish this example.
However, I will explain the basics of this grammar here.
Feel free to learn and experiment more if you are interested.
This grammar parses numbers with the `num` rule.
Grammars are [lists of rules](https://pest.rs/book/grammars/syntax.html#syntax-of-pest-grammars), such as `num` and `main`.
This grammar parses numbers with the `num` rule, matching 1 or more `ASCII_DIGIT` with [repetition](https://pest.rs/book/grammars/syntax.html#repetition).
The `main` rule ensures that there is no additional text before and after a `num` rule, using [`SOI` (start of input) `EOI` (end of input)](https://pest.rs/book/grammars/syntax.html#start-and-end-of-input), and using the [`~` operator to sequence](https://pest.rs/book/grammars/syntax.html#sequence) these rules.
We set the [`WHITESPACE` builtin rule](https://pest.rs/book/grammars/syntax.html#implicit-whitespace) to `{ " " | "\t" | "\n" | "\r" }` so that spaces, tabs, newlines, and carriage return characters are implicitly allowed between returns.
We set the [`WHITESPACE` builtin rule](https://pest.rs/book/grammars/syntax.html#implicit-whitespace) to `{ " " | "\t" | "\n" | "\r" }` so that spaces, tabs, newlines, and carriage return characters are implicitly allowed between sequenced rules.
The `@` operator before `{` indicates that it is an [atomic rule](https://pest.rs/book/grammars/syntax.html#atomic), disallowing implicit whitespace.
We want this on the `num` rule so that we can't add spaces in between digits of a number (try removing it and see!)
The `_` operator before `{` indicates that it is a [silent rule](https://pest.rs/book/grammars/syntax.html#silent) that does not contribute to the parse result.
This is important when processing the parse result into an [Abstract Syntax Tree (AST)](https://pest.rs/book/examples/json.html#ast-generation).
In this example we just print the parse result, so silent rules are not really needed, but I included it for completeness.
```

Create `test_1.txt` with:

0 comments on commit 8d971fe

Please sign in to comment.