Allow whitespace at complex-message start (#854)

* Allow whitespace at complex-message start

* Apply suggestions from code review

Co-authored-by: Addison Phillips <addison@unicode.org>

* Remove whitespace from simple-start-char

---------

Co-authored-by: Addison Phillips <addison@unicode.org>

spec/message.abnf

@@ -1,11 +1,11 @@
 message           = simple-message / complex-message
 
-simple-message    = [simple-start pattern]
+simple-message    = [s] [simple-start pattern]
 simple-start      = simple-start-char / escaped-char / placeholder
 pattern           = *(text-char / escaped-char / placeholder)
 placeholder       = expression / markup
 
-complex-message   = *(declaration [s]) complex-body [s]
+complex-message   = [s] *(declaration [s]) complex-body [s]
 declaration       = input-declaration / local-declaration / reserved-statement
 complex-body      = quoted-pattern / matcher
 
@@ -86,7 +86,7 @@ name-char  = name-start / DIGIT / "-" / "."
            / %xB7 / %x300-36F / %x203F-2040
 
 ; Restrictions on characters in various contexts
-simple-start-char = content-char / s / "@" / "|"
+simple-start-char = content-char / "@" / "|"
 text-char         = content-char / s / "." / "@" / "|"
 quoted-char       = content-char / s / "." / "@" / "{" / "}"
 reserved-char     = content-char / "."

spec/syntax.md

@@ -153,11 +153,14 @@ message = simple-message / complex-message
 ```
 
 A **_<dfn>simple message</dfn>_** contains a single _pattern_,
-with restrictions on its first character.
+with restrictions on its first non-whitespace character.
 An empty string is a valid _simple message_.
 
+Whitespace at the start or end of a _simple message_ is significant,
+and a part of the _text_ of the _message_.
+
 ```abnf
-simple-message = [simple-start pattern]
+simple-message = [s] [simple-start pattern]
 simple-start   = simple-start-char / escaped-char / placeholder
 ```
 
@@ -169,8 +172,11 @@ and consists of:
 1. an optional list of _declarations_, followed by
 2. a _complex body_
 
+Whitespace at the start or end of a _complex message_ is not significant,
+and does not affect the processing of the _message_.
+
 ```abnf
-complex-message = *(declaration [s]) complex-body [s]
+complex-message = [s] *(declaration [s]) complex-body [s]
 ```
 
 ### Declarations
@@ -300,16 +306,16 @@ U+007B LEFT CURLY BRACKET `{`, and U+007D RIGHT CURLY BRACKET `}`
 MUST be escaped as `\\`, `\{`, and `\}` respectively.
 
 In the ABNF, _text_ is represented by non-empty sequences of
-`simple-start-char`, `text-char`, and `escaped-char`.
-The first of these is used at the start of a _simple message_,
+`simple-start-char`, `text-char`, `escaped-char`, and `s`.
+The production `simple-start-char` represents the first non-whitespace in a _simple message_
 and matches `text-char` except for not allowing U+002E FULL STOP `.`.
 The ABNF uses `content-char` as a shared base for _text_ and _quoted literal_ characters.
 
 Whitespace in _text_, including tabs, spaces, and newlines is significant and MUST
 be preserved during formatting.
 
 ```abnf
-simple-start-char = content-char / s / "@" / "|"
+simple-start-char = content-char / "@" / "|"
 text-char         = content-char / s / "." / "@" / "|"
 quoted-char       = content-char / s / "." / "@" / "{" / "}"
 reserved-char     = content-char / "."

test/tests/syntax.json

@@ -36,6 +36,11 @@
       "src": "hello {|world|}",
       "exp": "hello world"
     },
+    {
+      "description": "message -> simple-message -> s simple-start pattern -> s simple-start-char pattern -> ...",
+      "src": "\n hello\t",
+      "exp": "\n hello\t"
+    },
     {
       "src": "hello {$place}",
       "params": [
@@ -134,6 +139,11 @@
       "src": ".input{$x}{{}}",
       "exp": ""
     },
+    {
+      "description": "message -> complex-message -> s *(declaration [s]) complex-body s -> s declaration complex-body s -> s input-declaration complex-body s -> s input variable-expression complex-body s",
+      "src": "\t.input{$x}{{}}\n",
+      "exp": ""
+    },
     {
       "description": "message -> complex-message -> *(declaration [s]) complex-body -> declaration declaration complex-body -> input-declaration input-declaration complex-body -> input variable-expression input variable-expression complex-body",
       "src": ".input{$x}.input{$y}{{}}",
@@ -145,8 +155,8 @@
       "exp": ""
     },
     {
-      "description": "message -> complex-message -> *(declaration [s]) complex-body s -> complex-body s",
-      "src": "{{}} ",
+      "description": "message -> complex-message -> s *(declaration [s]) complex-body s -> s complex-body s",
+      "src": "  {{}}  ",
       "exp": ""
     },
     {
@@ -398,8 +408,7 @@
           "source": "|42|",
           "value": "42"
         }
-      ],
-      "exp": "42"
+      ]
     },
     {
       "description": "... literal -> quoted-literal -> \"|\" \"|\" ...",