diff --git a/doc/changes/changes_4.2.0.md b/doc/changes/changes_4.2.0.md index df2f523a..ba15761f 100644 --- a/doc/changes/changes_4.2.0.md +++ b/doc/changes/changes_4.2.0.md @@ -1,4 +1,4 @@ -# OpenFastTrace 4.2.0, released ??? +# OpenFastTrace 4.2.0, released 2025-05-19 Code name: Markdown code blocks @@ -8,6 +8,8 @@ In this release we changed the behavior of the Markdown importer, so that if we We also added a whole section about understanding and fixing broken links between specification items to the user guide. +The new token `oft:on|off` allows switching off OFT parsing for certain text passages in Markdown and RST documents. + ## Features * #437: Upgrade build and test dependencies on top of 4.1.0 @@ -17,5 +19,6 @@ We also added a whole section about understanding and fixing broken links betwee * #427: Removed old `CHANGELOG.md` file and merged missing parts into release history. * #431: Documented "unwanted coverage" in user guide. +* #449: Fix parsing past end of "needs" paragraph. * #440: Added Tag importer support for TOML files. * #442: Added support for javascript file extensions `.cjs`, `.mjs` and `.ejs` diff --git a/doc/spec/design.md b/doc/spec/design.md index 47a084b6..e8c34b87 100644 --- a/doc/spec/design.md +++ b/doc/spec/design.md @@ -220,6 +220,24 @@ Covers: Needs: impl, utest, itest +### Line Parser for Lightweight Markup Import + +RST and Markdown share a common underlying parser that operates on a line-by-line basis. + +##### Disabling OFT Parsing for Parts of a Markup File +`dsn~disabling-oft-parsing-for-parts-of-a-markup-file~1` + +When it encounters the token `oft:off`, the line parser stops extracting specification items until it + +* either encounters the token `oft:on` +* or reaches the end of the current document. + +Covers: + +* `req~disabling-oft-parsing-for-parts-of-a-markup-file~1` + +Needs: impl, utest + ## Tracing ### Tracing Needed Coverage @@ -756,8 +774,10 @@ The Markdown Importer supports forwarding required coverage from one artifact ty The following example shows an architectural specification item that forwards the needed coverage directly to the detailed design and an integration test: + arch --> dsn, itest : req~skip-this-requirement~1 - + + Covers: * `req~artifact-type-forwarding-in-markdown~1` diff --git a/doc/spec/system_requirements.md b/doc/spec/system_requirements.md index d739ab9f..7ef1f3b6 100644 --- a/doc/spec/system_requirements.md +++ b/doc/spec/system_requirements.md @@ -101,6 +101,8 @@ The same benefits as for [Markdown](#markdown-import) apply: * is portable across platforms * easy to process with text manipulation tools +Needs: req + ### ReqM2 Import `feat~reqm2-import~1` @@ -251,9 +253,43 @@ Needs: dsn ### Supported Formats +#### Common Requirements for Lightweight Markup Import + +Typical OFT specification are written in a lightweight markup language like [Markdown](#markdown-import) or [ReStructured Text](#restructured-text-rst-import). Before we go into the specifics, this section discusses the common requirements. + +##### Disabling OFT Parsing for Parts of a Markup File +`req~disabling-oft-parsing-for-parts-of-a-markup-file~1` + +OFT-enhanced markup allows excluding text blocks from OFT parsing with the syntax `oft:on|off`. + +Example for Markdown: + + + This part of the document will not be parsed for OFT specification items. + + Until the end marker or the end of the current document is reached + + +Example for RST: + + .. oft:off + Not imported. + .. oft:on + +Rationale: + +This allows creating OFT examples that do not contribute to the code and avoid accidental recognition of specification items in text that is not supposed to contain them. + +Covers: + +* [feat~markdown-import~1](#markdown-import) +* [feat~rst-import~1](#restructured-text-rst-import) + +Needs: dsn + #### Markdown -Markdown is a simple ASCII-based markup format that is designed to be human readable in the source. While it can be rendered into HTML, it is perfectly eye-friendly even before rendering. +Markdown is a simple ASCII-based markup format that is designed to be human-readable in the source. While it can be rendered into HTML, it is perfectly eye-friendly even before rendering. Markdown focuses on content over formatting by giving the document structure like headlines, paragraphs and lists. The combination of being lightweight, human-readable and structure-oriented makes it a good fit for writing specifications as code. diff --git a/doc/user_guide.md b/doc/user_guide.md index 94b39dea..42f3e894 100644 --- a/doc/user_guide.md +++ b/doc/user_guide.md @@ -261,8 +261,6 @@ Requirements should be accompanied by a rationale in all cases where the reason the details are up to the detailed design. Needs: dsn - - `Needs`, `Rationale` and `Comment` are OpenFastTrace keywords that tell OpenFastTrace how to process the following content. There are other keywords in the context of specification items written in Markdown described in the following sections. @@ -298,6 +296,22 @@ Given the Feature `feat~rubber-ducky~1` exists and needs a `req`. A requirement Covers: - feat~rubber-ducky~1 +##### `Needs` + +The `Needs` keyword states which artifact types are needed to cover the current specification item. It is followed by a list of artifact types that are needed, each one written on a new line starting with a bullet character (`+`, `*`, or `-`) followed by the artifact type abbreviation. `Needs` comes in two flavors: as one-liner or as list. + +**Variant a) one-line `needs`** + + Needs: impl, utest, itest + +**Variant b) as List** + + Needs: + - dsn + - uman + +Please note that you cannot mix the two styles in one specification item. + ##### `Depends` The `Depends` keyword defines dependencies between specification items. It is followed by a list of items the current specification item depends on, each one written on a new line starting witch a bullet character (`+`, `*`, or `-`) followed by the referenced specification item id. At the moment this has no effect on the HTML or plaintext output, but only if the `-o aspec` option is used. This has no effect on the coverage of specification items. @@ -332,6 +346,26 @@ is functionally equivalent to Tags are described in detail later in this document, see section [Distributing the Detailing Work](#distributing-the-detailing-work). +### Excluding Parts of a Specification Document for OFT Parsing + +Sometimes you want specific sections or a whole document to be excluded from OFT parsing. One reason could be that it is a document that contains an OFT example, that should not contribute to the trace. Or, you could have data in a document and don't want to risk that something accidentally looks like an OFT artifact. + +To switch of scanning use the token `oft:on|off` in your document at the appropriate location. + +Markdown example: + + + This part is ignored by OFT. + + Here OFT scans again. + +ReStructured text example: + + .. oft:off + This part is ignored by OFT. + .. oft:on + Here OFT scans again. + ### Delegating Requirement Coverage Consider a situation where you are responsible for the high-level software architecture of your project. You define the component breakdown, the interfaces and the interworking of the components. You get your requirements from a system requirement specification, but it turns out many of those incoming requirements are at a detail level that does not require design decisions on inter-component-level but rather affects the internals of a single component. diff --git a/importer/lightweightmarkup/src/main/java/org/itsallcode/openfasttrace/importer/lightweightmarkup/statemachine/LineParserState.java b/importer/lightweightmarkup/src/main/java/org/itsallcode/openfasttrace/importer/lightweightmarkup/statemachine/LineParserState.java index 649eba11..1e477701 100644 --- a/importer/lightweightmarkup/src/main/java/org/itsallcode/openfasttrace/importer/lightweightmarkup/statemachine/LineParserState.java +++ b/importer/lightweightmarkup/src/main/java/org/itsallcode/openfasttrace/importer/lightweightmarkup/statemachine/LineParserState.java @@ -7,7 +7,7 @@ public enum LineParserState { /** - * Parser started (at beginning of the file) or outside of a specification + * Parser started (at beginning of the file) or outside a specification * item */ START, @@ -23,8 +23,10 @@ public enum LineParserState RATIONALE, /** Inside a comment section */ COMMENT, - /** Inside a section defining the required coverage */ - NEEDS, + /** Inside a section defining the required coverage (inline form) */ + NEEDS_LINE, + /** Required coverage (list form) */ + NEEDS_LIST, /** Found a title */ TITLE, /** Found tags */ diff --git a/importer/lightweightmarkup/src/main/java/org/itsallcode/openfasttrace/importer/lightweightmarkup/statemachine/LineParserStateMachine.java b/importer/lightweightmarkup/src/main/java/org/itsallcode/openfasttrace/importer/lightweightmarkup/statemachine/LineParserStateMachine.java index 85343798..7a8158da 100644 --- a/importer/lightweightmarkup/src/main/java/org/itsallcode/openfasttrace/importer/lightweightmarkup/statemachine/LineParserStateMachine.java +++ b/importer/lightweightmarkup/src/main/java/org/itsallcode/openfasttrace/importer/lightweightmarkup/statemachine/LineParserStateMachine.java @@ -2,6 +2,9 @@ import java.util.*; import java.util.logging.Logger; +import java.util.regex.Pattern; + +import static java.util.regex.Pattern.UNICODE_CHARACTER_CLASS; /** * This machine implements the core of a state based parser. @@ -19,10 +22,15 @@ public class LineParserStateMachine { private static final Logger LOG = Logger.getLogger(LineParserStateMachine.class.getName()); + private static final Pattern PARSER_OFF_PATTERN = Pattern.compile("(?:^|\\W)oft:off(?:\\W|$)", + UNICODE_CHARACTER_CLASS); + private static final Pattern PARSER_ON_PATTERN = Pattern.compile("(?:^|\\W)oft:on(?:\\W|$)", + UNICODE_CHARACTER_CLASS); private LineParserState state = LineParserState.START; private String lastToken = ""; private final Transition[] transitions; + private boolean enabled = true; /** * Create a new instance of the {@link LineParserStateMachine} @@ -48,7 +56,29 @@ public LineParserStateMachine(final Transition[] transitions) * patterns that span multiple lines like underlined titles in * Markdown or RST. */ + // [impl -> dsn~disabling-oft-parsing-for-parts-of-a-markup-file~1] public void step(final String line, final String nextLine) + { + if (enabled) + { + if (PARSER_OFF_PATTERN.matcher(line).find()) + { + enabled = false; + } + else + { + stepEnabled(line, nextLine); + } + } + else + { + if (PARSER_ON_PATTERN.matcher(line).find()) { + enabled = true; + } + } + } + + private void stepEnabled(final String line, final String nextLine) { boolean matched = false; for (final Transition entry : this.transitions) diff --git a/importer/markdown/src/main/java/org/itsallcode/openfasttrace/importer/markdown/MarkdownImporter.java b/importer/markdown/src/main/java/org/itsallcode/openfasttrace/importer/markdown/MarkdownImporter.java index 586eb113..85374d10 100644 --- a/importer/markdown/src/main/java/org/itsallcode/openfasttrace/importer/markdown/MarkdownImporter.java +++ b/importer/markdown/src/main/java/org/itsallcode/openfasttrace/importer/markdown/MarkdownImporter.java @@ -42,7 +42,7 @@ protected Transition[] configureTransitions() transition(START , SPEC_ITEM , MdPattern.ID , this::beginItem ), transition(START , TITLE , SECTION_TITLE , this::rememberTitle ), transition(START , START , MdPattern.FORWARD , this::forward ), - transition(START , CODE_BLOCK , MdPattern.CODE_BEGIN , () -> {} ), + transition(START , CODE_BLOCK , MdPattern.CODE_BEGIN , () -> {} ), transition(START , START , MdPattern.EVERYTHING , () -> {} ), transition(TITLE , SPEC_ITEM , MdPattern.ID , this::beginItem ), @@ -59,11 +59,12 @@ protected Transition[] configureTransitions() transition(SPEC_ITEM , COMMENT , MdPattern.COMMENT , this::beginComment ), transition(SPEC_ITEM , COVERS , MdPattern.COVERS , () -> {} ), transition(SPEC_ITEM , DEPENDS , MdPattern.DEPENDS , () -> {} ), - transition(SPEC_ITEM , NEEDS , MdPattern.NEEDS_INT , this::addNeeds ), - transition(SPEC_ITEM , NEEDS , MdPattern.NEEDS , () -> {} ), + transition(SPEC_ITEM , SPEC_ITEM , MdPattern.NEEDS_INT , this::addNeeds ), + transition(SPEC_ITEM , NEEDS_LIST , MdPattern.NEEDS , () -> {} ), transition(SPEC_ITEM , TAGS , MdPattern.TAGS_INT , this::addTag ), transition(SPEC_ITEM , TAGS , MdPattern.TAGS , () -> {} ), transition(SPEC_ITEM , DESCRIPTION, MdPattern.DESCRIPTION, this::beginDescription ), + transition(SPEC_ITEM , START , MdPattern.FORWARD , () -> {endItem(); forward();} ), transition(SPEC_ITEM , DESCRIPTION, MdPattern.NOT_EMPTY , this::beginDescription ), transition(DESCRIPTION, SPEC_ITEM , MdPattern.ID , this::beginItem ), @@ -72,10 +73,11 @@ protected Transition[] configureTransitions() transition(DESCRIPTION, COMMENT , MdPattern.COMMENT , this::beginComment ), transition(DESCRIPTION, COVERS , MdPattern.COVERS , () -> {} ), transition(DESCRIPTION, DEPENDS , MdPattern.DEPENDS , () -> {} ), - transition(DESCRIPTION, NEEDS , MdPattern.NEEDS_INT , this::addNeeds ), - transition(DESCRIPTION, NEEDS , MdPattern.NEEDS , () -> {} ), + transition(DESCRIPTION, SPEC_ITEM , MdPattern.NEEDS_INT , this::addNeeds ), + transition(DESCRIPTION, NEEDS_LIST , MdPattern.NEEDS , () -> {} ), transition(DESCRIPTION, TAGS , MdPattern.TAGS_INT , this::addTag ), transition(DESCRIPTION, TAGS , MdPattern.TAGS , () -> {} ), + transition(DESCRIPTION, START , MdPattern.FORWARD , () -> {endItem(); forward();} ), transition(DESCRIPTION, DESCRIPTION, MdPattern.EVERYTHING , this::appendDescription ), transition(RATIONALE , SPEC_ITEM , MdPattern.ID , this::beginItem ), @@ -83,8 +85,8 @@ protected Transition[] configureTransitions() transition(RATIONALE , COMMENT , MdPattern.COMMENT , this::beginComment ), transition(RATIONALE , COVERS , MdPattern.COVERS , () -> {} ), transition(RATIONALE , DEPENDS , MdPattern.DEPENDS , () -> {} ), - transition(RATIONALE , NEEDS , MdPattern.NEEDS_INT , this::addNeeds ), - transition(RATIONALE , NEEDS , MdPattern.NEEDS , () -> {} ), + transition(RATIONALE , SPEC_ITEM , MdPattern.NEEDS_INT , this::addNeeds ), + transition(RATIONALE , NEEDS_LIST , MdPattern.NEEDS , () -> {} ), transition(RATIONALE , TAGS , MdPattern.TAGS_INT , this::addTag ), transition(RATIONALE , TAGS , MdPattern.TAGS , () -> {} ), transition(RATIONALE , RATIONALE , MdPattern.EVERYTHING , this::appendRationale ), @@ -93,8 +95,8 @@ protected Transition[] configureTransitions() transition(COMMENT , TITLE , SECTION_TITLE , () -> {endItem(); rememberTitle();}), transition(COMMENT , COVERS , MdPattern.COVERS , () -> {} ), transition(COMMENT , DEPENDS , MdPattern.DEPENDS , () -> {} ), - transition(COMMENT , NEEDS , MdPattern.NEEDS_INT , this::addNeeds ), - transition(COMMENT , NEEDS , MdPattern.NEEDS , () -> {} ), + transition(COMMENT , SPEC_ITEM , MdPattern.NEEDS_INT , this::addNeeds ), + transition(COMMENT , NEEDS_LIST , MdPattern.NEEDS , () -> {} ), transition(COMMENT , RATIONALE , MdPattern.RATIONALE , this::beginRationale ), transition(COMMENT , TAGS , MdPattern.TAGS_INT , this::addTag ), transition(COMMENT , TAGS , MdPattern.TAGS , () -> {} ), @@ -107,8 +109,8 @@ protected Transition[] configureTransitions() transition(COVERS , RATIONALE , MdPattern.RATIONALE , this::beginRationale ), transition(COVERS , COMMENT , MdPattern.COMMENT , this::beginComment ), transition(COVERS , DEPENDS , MdPattern.DEPENDS , () -> {} ), - transition(COVERS , NEEDS , MdPattern.NEEDS_INT , this::addNeeds ), - transition(COVERS , NEEDS , MdPattern.NEEDS , () -> {} ), + transition(COVERS , SPEC_ITEM , MdPattern.NEEDS_INT , this::addNeeds ), + transition(COVERS , NEEDS_LIST , MdPattern.NEEDS , () -> {} ), transition(COVERS , COVERS , MdPattern.EMPTY , () -> {} ), transition(COVERS , TAGS , MdPattern.TAGS_INT , this::addTag ), transition(COVERS , TAGS , MdPattern.TAGS , () -> {} ), @@ -121,8 +123,8 @@ protected Transition[] configureTransitions() transition(DEPENDS , RATIONALE , MdPattern.RATIONALE , this::beginRationale ), transition(DEPENDS , COMMENT , MdPattern.COMMENT , this::beginComment ), transition(DEPENDS , DEPENDS , MdPattern.DEPENDS , () -> {} ), - transition(DEPENDS , NEEDS , MdPattern.NEEDS_INT , this::addNeeds ), - transition(DEPENDS , NEEDS , MdPattern.NEEDS , () -> {} ), + transition(DEPENDS , SPEC_ITEM , MdPattern.NEEDS_INT , this::addNeeds ), + transition(DEPENDS , NEEDS_LIST , MdPattern.NEEDS , () -> {} ), transition(DEPENDS , DEPENDS , MdPattern.EMPTY , () -> {} ), transition(DEPENDS , COVERS , MdPattern.COVERS , () -> {} ), transition(DEPENDS , TAGS , MdPattern.TAGS_INT , this::addTag ), @@ -131,18 +133,17 @@ protected Transition[] configureTransitions() // [impl->dsn~md.needs-coverage-list-single-line~2] // [impl->dsn~md.needs-coverage-list~1] - transition(NEEDS , SPEC_ITEM , MdPattern.ID , this::beginItem ), - transition(NEEDS , TITLE , SECTION_TITLE , () -> {endItem(); rememberTitle();}), - transition(NEEDS , RATIONALE , MdPattern.RATIONALE , this::beginRationale ), - transition(NEEDS , COMMENT , MdPattern.COMMENT , this::beginComment ), - transition(NEEDS , DEPENDS , MdPattern.DEPENDS , () -> {} ), - transition(NEEDS , NEEDS , MdPattern.NEEDS_INT , this::addNeeds ), - transition(NEEDS , NEEDS , MdPattern.NEEDS_REF , this::addNeeds ), - transition(NEEDS , NEEDS , MdPattern.EMPTY , () -> {} ), - transition(NEEDS , COVERS , MdPattern.COVERS , () -> {} ), - transition(NEEDS , TAGS , MdPattern.TAGS_INT , this::addTag ), - transition(NEEDS , TAGS , MdPattern.TAGS , () -> {} ), - transition(NEEDS , START , MdPattern.FORWARD , () -> {endItem(); forward();} ), + transition(NEEDS_LIST , SPEC_ITEM , MdPattern.ID , this::beginItem ), + transition(NEEDS_LIST , TITLE , SECTION_TITLE , () -> {endItem(); rememberTitle();}), + transition(NEEDS_LIST , RATIONALE , MdPattern.RATIONALE , this::beginRationale ), + transition(NEEDS_LIST , COMMENT , MdPattern.COMMENT , this::beginComment ), + transition(NEEDS_LIST , DEPENDS , MdPattern.DEPENDS , () -> {} ), + transition(NEEDS_LIST , NEEDS_LIST , MdPattern.NEEDS_REF , this::addNeeds ), + transition(NEEDS_LIST , DESCRIPTION, MdPattern.EMPTY , () -> {} ), + transition(NEEDS_LIST , COVERS , MdPattern.COVERS , () -> {} ), + transition(NEEDS_LIST , TAGS , MdPattern.TAGS_INT , this::addTag ), + transition(NEEDS_LIST , TAGS , MdPattern.TAGS , () -> {} ), + transition(NEEDS_LIST , START , MdPattern.FORWARD , () -> {endItem(); forward();} ), transition(TAGS , TAGS , MdPattern.TAG_ENTRY , this::addTag ), transition(TAGS , SPEC_ITEM , MdPattern.ID , this::beginItem ), @@ -150,9 +151,9 @@ protected Transition[] configureTransitions() transition(TAGS , RATIONALE , MdPattern.RATIONALE , this::beginRationale ), transition(TAGS , COMMENT , MdPattern.COMMENT , this::beginComment ), transition(TAGS , DEPENDS , MdPattern.DEPENDS , () -> {} ), - transition(TAGS , NEEDS , MdPattern.NEEDS_INT , this::addNeeds ), - transition(TAGS , NEEDS , MdPattern.NEEDS , () -> {} ), - transition(TAGS , NEEDS , MdPattern.EMPTY , () -> {} ), + transition(TAGS , SPEC_ITEM , MdPattern.NEEDS_INT , this::addNeeds ), + transition(TAGS , NEEDS_LIST , MdPattern.NEEDS , () -> {} ), + transition(TAGS , SPEC_ITEM , MdPattern.EMPTY , () -> {} ), transition(TAGS , COVERS , MdPattern.COVERS , () -> {} ), transition(TAGS , TAGS , MdPattern.TAGS , () -> {} ), transition(TAGS , TAGS , MdPattern.TAGS_INT , this::addTag ), @@ -163,6 +164,19 @@ protected Transition[] configureTransitions() // @formatter:on } + /** + * Define a transition in the parser statemachine. + * + * @param from + * state to be matched against the parsers current state + * @param to + * state the parser will be in if the transition happened + * @param pattern + * line pattern to be matched for this transition to happen + * @param action + * action to take as during the transition + * @return transition definition + */ private static Transition transition(final LineParserState from, final LineParserState to, final MdPattern pattern, final TransitionAction action) { diff --git a/importer/markdown/src/test/java/org/itsallcode/openfasttrace/importer/markdown/TestMarkdownMarkupImporter.java b/importer/markdown/src/test/java/org/itsallcode/openfasttrace/importer/markdown/TestMarkdownMarkupImporter.java index c967840d..544ab519 100644 --- a/importer/markdown/src/test/java/org/itsallcode/openfasttrace/importer/markdown/TestMarkdownMarkupImporter.java +++ b/importer/markdown/src/test/java/org/itsallcode/openfasttrace/importer/markdown/TestMarkdownMarkupImporter.java @@ -2,6 +2,7 @@ import static org.hamcrest.Matchers.emptyIterable; import static org.itsallcode.matcher.auto.AutoMatcher.contains; +import static org.itsallcode.openfasttrace.api.core.SpecificationItemId.createId; import static org.itsallcode.openfasttrace.testutil.core.ItemBuilderFactory.item; import org.itsallcode.openfasttrace.api.core.SpecificationItemId; @@ -215,4 +216,31 @@ void testWhenCodeBlockIsInsideCommentSectionThenItIsImportedAsPartOfComment() .location("file_with_code_block_in_comment.md", 1) .build())); } -} + + + // [utest -> dsn~disabling-oft-parsing-for-parts-of-a-markup-file~1] + @Test + void testDisablingMarkdownParsingForATextBlock() { + assertImport("disable_parsing.md", """ + `req~stop-parsing~1` + + The next part must not be parsed: + + + `req~do-not-parse-me~2` + + Invisible. + + Needs: utest + + + Needs: impl + """, + contains(item() + .id(createId("req", "stop-parsing", 1)) + .description("The next part must not be parsed:") + .addNeedsArtifactType("impl") + .location("disable_parsing.md", 1) + .build())); + } +} \ No newline at end of file diff --git a/importer/restructuredtext/src/main/java/org/itsallcode/openfasttrace/importer/restructuredtext/RestructuredTextImporter.java b/importer/restructuredtext/src/main/java/org/itsallcode/openfasttrace/importer/restructuredtext/RestructuredTextImporter.java index e6381f76..979ca6a3 100644 --- a/importer/restructuredtext/src/main/java/org/itsallcode/openfasttrace/importer/restructuredtext/RestructuredTextImporter.java +++ b/importer/restructuredtext/src/main/java/org/itsallcode/openfasttrace/importer/restructuredtext/RestructuredTextImporter.java @@ -60,11 +60,12 @@ protected Transition[] configureTransitions() transition(SPEC_ITEM , TITLE , SECTION_TITLE , () -> {endItem(); rememberTitle();}), transition(SPEC_ITEM , COVERS , RstPattern.COVERS , () -> {} ), transition(SPEC_ITEM , DEPENDS , RstPattern.DEPENDS , () -> {} ), - transition(SPEC_ITEM , NEEDS , RstPattern.NEEDS_INT , this::addNeeds ), - transition(SPEC_ITEM , NEEDS , RstPattern.NEEDS , () -> {} ), + transition(SPEC_ITEM , SPEC_ITEM , RstPattern.NEEDS_INT , this::addNeeds ), + transition(SPEC_ITEM , NEEDS_LINE , RstPattern.NEEDS , () -> {} ), transition(SPEC_ITEM , TAGS , RstPattern.TAGS_INT , this::addTag ), transition(SPEC_ITEM , TAGS , RstPattern.TAGS , () -> {} ), transition(SPEC_ITEM , DESCRIPTION, RstPattern.DESCRIPTION, this::beginDescription ), + transition(SPEC_ITEM , START , RstPattern.FORWARD , () -> {endItem(); forward();} ), transition(SPEC_ITEM , DESCRIPTION, RstPattern.NOT_EMPTY , this::beginDescription ), transition(DESCRIPTION, SPEC_ITEM , RstPattern.ID , this::beginItem ), @@ -73,8 +74,8 @@ protected Transition[] configureTransitions() transition(DESCRIPTION, COMMENT , RstPattern.COMMENT , this::beginComment ), transition(DESCRIPTION, COVERS , RstPattern.COVERS , () -> {} ), transition(DESCRIPTION, DEPENDS , RstPattern.DEPENDS , () -> {} ), - transition(DESCRIPTION, NEEDS , RstPattern.NEEDS_INT , this::addNeeds ), - transition(DESCRIPTION, NEEDS , RstPattern.NEEDS , () -> {} ), + transition(DESCRIPTION, SPEC_ITEM , RstPattern.NEEDS_INT , this::addNeeds ), + transition(DESCRIPTION, NEEDS_LINE , RstPattern.NEEDS , () -> {} ), transition(DESCRIPTION, TAGS , RstPattern.TAGS_INT , this::addTag ), transition(DESCRIPTION, TAGS , RstPattern.TAGS , () -> {} ), transition(DESCRIPTION, START , RstPattern.FORWARD , () -> {endItem(); forward();} ), @@ -86,8 +87,8 @@ protected Transition[] configureTransitions() transition(RATIONALE , COMMENT , RstPattern.COMMENT , this::beginComment ), transition(RATIONALE , COVERS , RstPattern.COVERS , () -> {} ), transition(RATIONALE , DEPENDS , RstPattern.DEPENDS , () -> {} ), - transition(RATIONALE , NEEDS , RstPattern.NEEDS_INT , this::addNeeds ), - transition(RATIONALE , NEEDS , RstPattern.NEEDS , () -> {} ), + transition(RATIONALE , SPEC_ITEM , RstPattern.NEEDS_INT , this::addNeeds ), + transition(RATIONALE , NEEDS_LINE , RstPattern.NEEDS , () -> {} ), transition(RATIONALE , TAGS , RstPattern.TAGS_INT , this::addTag ), transition(RATIONALE , TAGS , RstPattern.TAGS , () -> {} ), transition(RATIONALE , RATIONALE , RstPattern.EVERYTHING , this::appendRationale ), @@ -96,8 +97,8 @@ protected Transition[] configureTransitions() transition(COMMENT , TITLE , SECTION_TITLE , () -> {endItem(); rememberTitle();}), transition(COMMENT , COVERS , RstPattern.COVERS , () -> {} ), transition(COMMENT , DEPENDS , RstPattern.DEPENDS , () -> {} ), - transition(COMMENT , NEEDS , RstPattern.NEEDS_INT , this::addNeeds ), - transition(COMMENT , NEEDS , RstPattern.NEEDS , () -> {} ), + transition(COMMENT , SPEC_ITEM , RstPattern.NEEDS_INT , this::addNeeds ), + transition(COMMENT , NEEDS_LINE , RstPattern.NEEDS , () -> {} ), transition(COMMENT , RATIONALE , RstPattern.RATIONALE , this::beginRationale ), transition(COMMENT , TAGS , RstPattern.TAGS_INT , this::addTag ), transition(COMMENT , TAGS , RstPattern.TAGS , () -> {} ), @@ -110,8 +111,8 @@ protected Transition[] configureTransitions() transition(COVERS , RATIONALE , RstPattern.RATIONALE , this::beginRationale ), transition(COVERS , COMMENT , RstPattern.COMMENT , this::beginComment ), transition(COVERS , DEPENDS , RstPattern.DEPENDS , () -> {} ), - transition(COVERS , NEEDS , RstPattern.NEEDS_INT , this::addNeeds ), - transition(COVERS , NEEDS , RstPattern.NEEDS , () -> {} ), + transition(COVERS , SPEC_ITEM , RstPattern.NEEDS_INT , this::addNeeds ), + transition(COVERS , NEEDS_LINE , RstPattern.NEEDS , () -> {} ), transition(COVERS , COVERS , RstPattern.EMPTY , () -> {} ), transition(COVERS , TAGS , RstPattern.TAGS_INT , this::addTag ), transition(COVERS , TAGS , RstPattern.TAGS , () -> {} ), @@ -124,8 +125,8 @@ protected Transition[] configureTransitions() transition(DEPENDS , RATIONALE , RstPattern.RATIONALE , this::beginRationale ), transition(DEPENDS , COMMENT , RstPattern.COMMENT , this::beginComment ), transition(DEPENDS , DEPENDS , RstPattern.DEPENDS , () -> {} ), - transition(DEPENDS , NEEDS , RstPattern.NEEDS_INT , this::addNeeds ), - transition(DEPENDS , NEEDS , RstPattern.NEEDS , () -> {} ), + transition(DEPENDS , SPEC_ITEM , RstPattern.NEEDS_INT , this::addNeeds ), + transition(DEPENDS , NEEDS_LINE , RstPattern.NEEDS , () -> {} ), transition(DEPENDS , DEPENDS , RstPattern.EMPTY , () -> {} ), transition(DEPENDS , COVERS , RstPattern.COVERS , () -> {} ), transition(DEPENDS , TAGS , RstPattern.TAGS_INT , this::addTag ), @@ -134,18 +135,17 @@ protected Transition[] configureTransitions() // [impl->dsn~md.needs-coverage-list-single-line~2] // [impl->dsn~md.needs-coverage-list~1] - transition(NEEDS , SPEC_ITEM , RstPattern.ID , this::beginItem ), - transition(NEEDS , TITLE , SECTION_TITLE , () -> {endItem(); rememberTitle();}), - transition(NEEDS , RATIONALE , RstPattern.RATIONALE , this::beginRationale ), - transition(NEEDS , COMMENT , RstPattern.COMMENT , this::beginComment ), - transition(NEEDS , DEPENDS , RstPattern.DEPENDS , () -> {} ), - transition(NEEDS , NEEDS , RstPattern.NEEDS_INT , this::addNeeds ), - transition(NEEDS , NEEDS , RstPattern.NEEDS_REF , this::addNeeds ), - transition(NEEDS , NEEDS , RstPattern.EMPTY , () -> {} ), - transition(NEEDS , COVERS , RstPattern.COVERS , () -> {} ), - transition(NEEDS , TAGS , RstPattern.TAGS_INT , this::addTag ), - transition(NEEDS , TAGS , RstPattern.TAGS , () -> {} ), - transition(NEEDS , START , RstPattern.FORWARD , () -> {endItem(); forward();} ), + transition(NEEDS_LINE , SPEC_ITEM , RstPattern.ID , this::beginItem ), + transition(NEEDS_LINE , TITLE , SECTION_TITLE , () -> {endItem(); rememberTitle();}), + transition(NEEDS_LINE , RATIONALE , RstPattern.RATIONALE , this::beginRationale ), + transition(NEEDS_LINE , COMMENT , RstPattern.COMMENT , this::beginComment ), + transition(NEEDS_LINE , DEPENDS , RstPattern.DEPENDS , () -> {} ), + transition(NEEDS_LINE , NEEDS_LINE , RstPattern.NEEDS_REF , this::addNeeds ), + transition(NEEDS_LINE , NEEDS_LINE , RstPattern.EMPTY , () -> {} ), + transition(NEEDS_LINE , COVERS , RstPattern.COVERS , () -> {} ), + transition(NEEDS_LINE , TAGS , RstPattern.TAGS_INT , this::addTag ), + transition(NEEDS_LINE , TAGS , RstPattern.TAGS , () -> {} ), + transition(NEEDS_LINE , START , RstPattern.FORWARD , () -> {endItem(); forward();} ), transition(TAGS , TAGS , RstPattern.TAG_ENTRY , this::addTag ), transition(TAGS , TITLE , SECTION_TITLE , () -> {endItem(); rememberTitle();}), @@ -153,9 +153,9 @@ protected Transition[] configureTransitions() transition(TAGS , RATIONALE , RstPattern.RATIONALE , this::beginRationale ), transition(TAGS , COMMENT , RstPattern.COMMENT , this::beginComment ), transition(TAGS , DEPENDS , RstPattern.DEPENDS , () -> {} ), - transition(TAGS , NEEDS , RstPattern.NEEDS_INT , this::addNeeds ), - transition(TAGS , NEEDS , RstPattern.NEEDS , () -> {} ), - transition(TAGS , NEEDS , RstPattern.EMPTY , () -> {} ), + transition(TAGS , SPEC_ITEM , RstPattern.NEEDS_INT , this::addNeeds ), + transition(TAGS , NEEDS_LINE , RstPattern.NEEDS , () -> {} ), + transition(TAGS , SPEC_ITEM , RstPattern.EMPTY , () -> {} ), transition(TAGS , COVERS , RstPattern.COVERS , () -> {} ), transition(TAGS , TAGS , RstPattern.TAGS , () -> {} ), transition(TAGS , TAGS , RstPattern.TAGS_INT , this::addTag ), diff --git a/importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/TestRestructuredTextImporter.java b/importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/TestRestructuredTextImporter.java index 5a733a76..40b3cdf2 100644 --- a/importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/TestRestructuredTextImporter.java +++ b/importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/TestRestructuredTextImporter.java @@ -1,6 +1,7 @@ package org.itsallcode.openfasttrace.importer.restructuredtext; import static org.itsallcode.matcher.auto.AutoMatcher.contains; +import static org.itsallcode.openfasttrace.api.core.SpecificationItemId.createId; import static org.itsallcode.openfasttrace.testutil.core.ItemBuilderFactory.item; import org.itsallcode.openfasttrace.api.core.SpecificationItemId; @@ -143,4 +144,30 @@ void testLessThenThreeUnderliningCharactersAreNotDetectedAsTitleUnderlines() .location("z", 3) .build())); } + + // [utest -> dsn~disabling-oft-parsing-for-parts-of-a-markup-file~1] + @Test + void testDisablingRstParsingForATextBlock() { + assertImport("disable_parsing.rst", """ + `req~stop-parsing~1` + + The next part must not be parsed: + + .. oft:off + `req~do-not-parse-me~2` + + Invisible. + + Needs: utest + .. oft:on + + Needs: impl + """, + contains(item() + .id(createId("req", "stop-parsing", 1)) + .description("The next part must not be parsed:") + .addNeedsArtifactType("impl") + .location("disable_parsing.rst", 1) + .build())); + } } diff --git a/importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/RstSectionTitlePatternTest.java b/importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/TestRstSectionTitlePattern.java similarity index 86% rename from importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/RstSectionTitlePatternTest.java rename to importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/TestRstSectionTitlePattern.java index 82e04e7f..fc9ecf04 100644 --- a/importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/RstSectionTitlePatternTest.java +++ b/importer/restructuredtext/src/test/java/org/itsallcode/openfasttrace/importer/restructuredtext/TestRstSectionTitlePattern.java @@ -1,8 +1,10 @@ package org.itsallcode.openfasttrace.importer.restructuredtext; import static org.hamcrest.MatcherAssert.assertThat; +import static org.hamcrest.Matchers.hasSize; import static org.hamcrest.Matchers.is; import static org.junit.jupiter.api.Assertions.assertAll; +import static org.junit.jupiter.api.Assertions.fail; import java.util.List; import java.util.Optional; @@ -12,7 +14,7 @@ import org.junit.jupiter.params.provider.Arguments; import org.junit.jupiter.params.provider.MethodSource; -class RstSectionTitlePatternTest +class TestRstSectionTitlePattern { static Stream testCases() @@ -85,9 +87,15 @@ void test(final String line, final String nextLine, final String expected) } else { - assertAll(() -> assertThat( - "Lines '" + line + "' + '" + nextLine + "' should be recognized as a section title", - result.isPresent(), is(true)), () -> assertThat(result.get().get(0), is(expected))); + if(result.isPresent()) { + final List matches = result.get(); + assertAll( + () -> assertThat(matches, hasSize(1)), + () -> assertThat(matches.get(0), is(expected)) + ); + } else { + fail("No match found for '" + line + "' + '" + nextLine + "'"); + } } } } diff --git a/testutil/src/main/java/org/itsallcode/openfasttrace/testutil/importer/lightweightmarkup/AbstractLightWeightMarkupImporterTest.java b/testutil/src/main/java/org/itsallcode/openfasttrace/testutil/importer/lightweightmarkup/AbstractLightWeightMarkupImporterTest.java index 53765a5a..c0fea3ac 100644 --- a/testutil/src/main/java/org/itsallcode/openfasttrace/testutil/importer/lightweightmarkup/AbstractLightWeightMarkupImporterTest.java +++ b/testutil/src/main/java/org/itsallcode/openfasttrace/testutil/importer/lightweightmarkup/AbstractLightWeightMarkupImporterTest.java @@ -3,6 +3,7 @@ import static org.hamcrest.MatcherAssert.assertThat; import static org.hamcrest.Matchers.equalTo; import static org.itsallcode.matcher.auto.AutoMatcher.contains; +import static org.itsallcode.openfasttrace.api.core.SpecificationItemId.createId; import static org.itsallcode.openfasttrace.testutil.core.ItemBuilderFactory.item; import static org.itsallcode.openfasttrace.testutil.importer.ImportAssertions.assertImportWithFactory; import static org.itsallcode.openfasttrace.testutil.importer.ImportAssertions.runImporterOnText; @@ -523,7 +524,7 @@ void testItemIdSupportsUTF8Characaters() Needs: arch """, contains(item() - .id(SpecificationItemId.createId("req", "zellzustandsänderung", 1)) + .id(createId("req", "zellzustandsänderung", 1)) .title("Die Implementierung muss den Zustand einzelner Zellen ändern") .description("Ermöglicht die Aktualisierung des Zustands von lebenden und toten Zellen" + " in jeder Generation.") @@ -544,15 +545,61 @@ void testHeaderBelongsToNextItem() `req~item2~1 Item 2 description """, - contains(item().id(SpecificationItemId.createId("req", "item1", 1)) + contains(item().id(createId("req", "item1", 1)) .title("Item 1") .description("Item 1 description") .location("file", 2 + titleLocationOffset) .build(), - item().id(SpecificationItemId.createId("req", "item2", 1)) + item().id(createId("req", "item2", 1)) .title("Item 2") .description("Item 2 description") .location("file", 6 + (2 * titleLocationOffset)) .build())); } + + // This is a regression test for https://github.com/itsallcode/openfasttrace/issues/449 + @Test + void testParsingNeedsIgnoresExtraListItems() { + assertImport("needs_with_extra_list_items.md", """ + `feat~the-feature~1` + + Needs: arch + + * this must not be in needs section + """, + contains(item() + .id(createId("feat", "the-feature", 1)) + .addNeedsArtifactType("arch") + .description("* this must not be in needs section") + .location("needs_with_extra_list_items.md", 1) + .build())); + } + + @Test + void testNeedsAfterCovers() { + assertImport("needs_after_covers.md", """ + `dsn~needs~3` + + Description with a bulleted list + + * this + * that + + Covers: + + * `req~needs~2` + + Needs: itest + """, + contains(item() + .id(createId("dsn", "needs", 3)) + .description("Description with a bulleted list" + + System.lineSeparator() + + System.lineSeparator() + "* this" + + System.lineSeparator() + "* that") + .addCoveredId(createId("req", "needs", 2)) + .addNeedsArtifactType("itest") + .location("needs_after_covers.md", 1) + .build())); + } }