Skip to content

Commit

Permalink
Support Double Escaping
Browse files Browse the repository at this point in the history 
Add support for double-escaping (e.g. `\\**`).
Lacking proper references on which characters are eligible for this
type of escaping, I'm supporting it for the same characters-set used
in our single-escaping implementation (which is also guesswork).
  • Loading branch information
@tajmone
tajmone committed Feb 3, 2024
1 parent c58aa38 commit 9a1d968
Show file tree
Hide file tree
Showing 3 changed files with 96 additions and 38 deletions.
26 changes: 17 additions & 9 deletions Syntaxes/Asciidoctor.sublime-syntax
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1733,17 +1733,25 @@ contexts:
#########
# ESCAPES
#########
# In AsciiDoc, certain characters need to be escaped with a backslash:
# # & - ` * _ + . ! ( ) { } [ ] \ > :
# @TODO: This might not be a complete list of characters need to be escaped.
# @TODO: Add support for double \\ escaping ('\\__', '\\**', etc.).
# In AsciiDoc, certain characters need to be escaped with a backslash,
# either a single '/' or double '//' depending on whether the special
# char occurs once or twice, respectively.
# @TODO: These might not be complete lists of characters which can be escaped.
# Originally:
# - match: \\(?=[-`*_#+.!(){}\[\]\\>:])
# @TODO: The double-escape characters are just guesses, need to check them.

escapes:
- match: \\[\#&\-`*_+\.!(){}\[\]\\>:]
scope: constant.character.escape.asciidoc

# originally:
# - match: \\(?=[-`*_#+.!(){}\[\]\\>:])
- include: escapes_double # (1)
- include: escapes_single # (2)
escapes_double:
# chars: # & - ` * _ + . ! ( ) { } [ ] \ > :
- match: '\\\\[\#&\-`*_+\.!(){}\[\]\\>:]{2}'
scope: constant.character.escape.double.asciidoc
escapes_single:
# chars: # & - ` * _ + . ! ( ) { } [ ] \ > :
- match: '\\[\#&\-`*_+\.!(){}\[\]\\>:]'
scope: constant.character.escape.single.asciidoc


##############
Expand Down
35 changes: 27 additions & 8 deletions Tests/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# AsciiDoc Syntax Test Suite

This directory contains the test suite for the AsciiDoc ST4 syntax.

This directory contains the testsuite for the AsciiDoc ST4 syntax.
Most tests are self-documenting and can be converted to HTML via Rake by typing `rake` in the Shell/CMD.

-----

Expand All @@ -11,7 +11,8 @@ This directory contains the test suite for the AsciiDoc ST4 syntax.

- [Directory Contents](#directory-contents)
- [Naming Conventions](#naming-conventions)
- [Introduction](#introduction)
- [HTML-Convertible Test Files](#html-convertible-test-files)
- [Plain Test Files](#plain-test-files)
- [Running the Tests](#running-the-tests)
- [Adding Test Files](#adding-test-files)
- [Disabled Tests](#disabled-tests)
Expand Down Expand Up @@ -40,27 +41,43 @@ The following naming conventions apply to this directory tree:
- `___*.*` — Files and folders ignored by Git.


# Introduction
# HTML-Convertible Test Files

Every `*.asciidoc` test file is also a valid AsciiDoc source file, which can be converted to HTML via Rake for the following purposes:

1. Verify with Asciidoctor that the test file is a valid AsciiDoc source, and capture any formatting errors and edge cases (i.e. formatting not working as expected).
2. Simplify tracking the syntax tests by reading them as documents that explain the nature of the tests, providing examples and references links.

For this purpose, syntax constructs are usually presented twice in a row: first in their source form (within a listing block), then in their HTML rendered form (within an example block, where comments are used for scope testing).

The size-bloat added by this approach to the test files shouldn't be a concern, for tests are only carried out by developers and maintainers of the syntax.
On the other hand, this approach simplifies understanding how the test suite works, making it easier to join the project for anyone wishing to contributing to the syntax.

Besides, AsciiDoc being a rather complex syntax, the need for well-documented tests is a felt need, because there are many edge cases to keep in mind during the tests, and because any changes to the syntax definition could easily introduce unexpected breaking behaviors.

If a test file has the `*.adoc` extension it's means that it's a malformed AsciiDoc document.
This usually happens because of the need to use an alternative comment delimiter in the first line, in order to allow correct testing of some rare edge-case contexts, which can make the document unconvertable due interference with its header.
To avoid cluttering the final document with excessive syntax examples, syntax tests which should be excluded from the final HTML document are placed inside a conditional preprocessor directive evaluating to false:

```asciidoc
ifeval::[0 == 1]
...
endif::[]
```

This also allows carrying out tests outside example blocks, as it's usually done for documented tests.

These `*.adoc` files won't be converted to HTML, but you can still read them in ST4, for they are only partially malformed, and can still be used for manual inspection.

# Plain Test Files

If a test file has the `*.adoc` extension it means that it's a malformed AsciiDoc document, and it will not be converted to HTML by Rake.

These files either contain tests for malformed constructs, which would fail conversion, or because they require using an alternative comment delimiter in the first line, in order to allow correct testing of some rare edge-cases contexts, which makes the document nonconvertible due malformed header.

Although these `*.adoc` files won't be converted to HTML, you can still view them in ST4 for manual inspection, to ensure that malformed contents are handled properly by the syntax, which should fallback gracefully in order to prevent breaking syntax highlighting for the rest of the document.


# Running the Tests

To run the tests:
To run the entire testsuite:

1. Open and select the `Syntaxes/Asciidoctor.sublime-syntax` file in ST4,
2. Hold <kbd>Ctrl</kbd>+<kbd>Shift</kbd>+<kbd>B</kbd> to bring up the __Build With__ options,
Expand All @@ -70,6 +87,8 @@ After having run the tests once, you can henceforth use just <kbd>Ctrl</kbd>+<kb

Alternatively, you can use the menus __Tools__ » __Build With__ and __Tools__ » __Build__.

To test a single file, apply the same as above but from within the target test file.

For more info on how syntax testing works, see:

- [Sublime Text Documentation » Syntax Definitions » Testing]
Expand Down
73 changes: 52 additions & 21 deletions Tests/syntax_test_Escapes.asciidoc
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
// SYNTAX TEST "Packages/ST4-Asciidoctor/Syntaxes/Asciidoctor.sublime-syntax"
= Escape Sequences

Asciidoctor rules for backslash escaping are rather entangled, because a `\` is treated as an escape only when it precedes a punctuation character that would otherwise act as an actual special symbol (formatting, etc.).
If the symbol following the `\` is just a literal character in the (unescaped) context, than the `\` is treated as a literal backslash too.
Asciidoctor rules for backslash escaping are rather entangled, because a `\` (or `\\` for double char escaping) is treated as an escape only when it precedes a punctuation character that would otherwise act as an actual special symbol (formatting, etc.).
If the symbol following the `\` is just a literal character in the (unescaped) context, than the `\` is treated as a literal backslash too and preserved in the final document.

* https://docs.asciidoctor.org/asciidoc/latest/subs/prevent/#escape-with-backslashes[AsciiDoc Documentation » Escape with backslashes^]

Expand All @@ -24,32 +24,32 @@ IMPORTANT: Check Asciidoctor source code for the list of escapeable charaters.

ifeval::[0 == 1]
Test: \# \& \- \` \* \_ \+ \. \!
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single

Test: \( \) \{ \} \[ \] \\ \> \:
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
// ^^ constant.character.escape.single
endif::[]


== Double Backlash Escaping

Double backlash escaping (`\\`) is currently unsupported:
Double backlash escaping (`\\`) is also supported:

____
To prevent two adjacent characters (e.g., `__`, `##`), from being interpreted as AsciiDoc syntax you need to precede it with two backslashes (`\\`).
Expand All @@ -67,5 +67,36 @@ Text \\__func__ will be enclosed in double underscores, not italicized.

=======================================================================
Text \\__func__ will be enclosed in double underscores, not italicized.
// ^^^^ constant.character.escape.double
=======================================================================


WARNING: Currently, double escaping supports the same characters as single escaping (listed above), although the actual list of candidate characters might differ in reality---I'm simply guessing them.

IMPORTANT: Check Asciidoctor source code for the list of double escapeable charaters.

ifeval::[0 == 1]
Test: \\## \\&& \\-- \\`` \\** \\__
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double

Test: \\++ \\.. \\!! \\(( \\)) \\{{
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double

Test: \\}} \\[[ \\]] \\\\ \\>> \\::
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
// ^^^^ constant.character.escape.double
endif::[]

0 comments on commit 9a1d968

Please sign in to comment.