fix: removes multiline from default regex modifiers [breaking]

[M4tteoP]

This PR evaluates the needed changes to align Coraza to Modsec v2 behavior rather than Modsec v3 in terms of default modifiers when compiling regexes. This necessity has been raised by coreruleset/coreruleset#3277.

We are currently running with two default modifiers:

• s (dotall)
• m (multiline mode)

While Modsec v2 (the reference implementation for the CRS) has:

• PCRE2_DOTALL
• PCRE2_DOLLAR_ENDONLY

Running the CRS rules with multiline may lead to false positives and lower performance.

Opening the PR to further discuss this, and whether we accept a breaking change for v3.* or not.

[M4tteoP]

I was not able to find the DOLLAR_ENDONLY modifier for RE2, but it seems that re2 by default behaves like DOLLAR_ENDONLY enabled.
See also https://regex101.com/r/mUcy69/1 (Mind the regex option D (dollar end only) enabled).
Am I missing something?

[codecov]

Codecov Report

Attention: Patch coverage is 53.84615% with 6 lines in your changes missing coverage. Please review.

Project coverage is 16.40%. Comparing base (4ff1f76) to head (242f972).
Report is 122 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/operators/rx.go	53.84%	4 Missing and 2 partials ⚠️

❗ There is a different number of reports uploaded between BASE (4ff1f76) and HEAD (242f972). Click for more details.

HEAD has 5 uploads less than BASE

Flag	BASE (4ff1f76)	HEAD (242f972)
default	1	0
examples	1	0
ftw	1	0
ftw-multiphase	1	0
tinygo	1	0

Additional details and impacted files

@@             Coverage Diff             @@
##             main     #876       +/-   ##
===========================================
- Coverage   82.72%   16.40%   -66.32%     
===========================================
  Files         162      148       -14     
  Lines        9080     8835      -245     
===========================================
- Hits         7511     1449     -6062     
- Misses       1319     7204     +5885     
+ Partials      250      182       -68     

Flag	Coverage Δ
default	?
default+	16.40% <53.84%> (?)
examples	?
ftw	?
ftw-multiphase	?
tinygo	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[anuraaga]

Good to see ftw passing with this change. I wouldn't merge it until seeing the PR to ModSec v3 doing the same, otherwise someone will just ask in the future why we aren't aligned with ModSec v3.

[jcchavezs]

Maybe it is a good idea to coordinate this in @M4tteoP @airween

[M4tteoP]

Totally 💯

[M4tteoP]

Apart from aligning the behavior with ModSec, are we going to accept this breaking change and release it into a 3.x version or we aim to have it into 4.x? If the latter is the case, I would guard the feature with a build flag and merge this PR with the fix disabled by default

[jptosso]

We agreed to implement this as a build tag for v3 and mandatory for v4
Once merged, another PR has to be created to remove the flag and force it by default

[M4tteoP]

We agreed to implement this as a build tag for v3 and mandatory for v4

Done, added coraza.rule.no_regex_multiline build tag. It should be ready to be reviewed.

[jptosso]

can you add this to the testing matrix? within the github actions. As mentioned here #1182, we should start covering all permutations of flags

[jptosso]

Please add this build tag to magefile.go

[M4tteoP]

Done, let's if tests are happy ^_^

[M4tteoP]

Looks so 🚀
Note: codecov is failing because the new logic is tested on a new test file that has the conditional build, so we have coverage, but codecov does not get it.

[jptosso]

LGTM