Rule 941310: False positive

[Rolandwalraven]

Type of Issue

Incorrect blocking (false positive)

Description

This innocent german text triggered rule 941310
DE_Matten & Sitzbezüge > Fußmatten_MT

Audit Logs / Triggered Rule Numbers

Matched Data: \xbcge > found within ARGS:*********: de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt

Your Environment

CRS version: 3.1.0
ModSec version: 2.9.2-1
Apache/2.4.29

Confirmation

[x] I have removed any personal data (email addresses, IP addresses,
passwords, domain names) from any logs posted.

[dune73]

Sorry for the inconvenience. However, I can't reproduce your finding, so I'm closing this for now. However, I suggest you reproduce this yourself with curl and reopen this issue together with the exact curl call.

Thanks for working against 3.2.0. Using the latest version helps us with our work.

[Rolandwalraven]

@dune73: I'm sorry we use CRS version 3.1.0. I was distracted by [ver "OWASP_CRS/3.2.0"] in the error log. Anyway i can trigger the rule with this curl request:

curl -X POST -d "test=DE_Matten%20%26%20Sitzbez%C3%BCge%20%3E%20Fu%C3%9Fmatten_MT" "https://DOMAIN.TLD/"

[dune73]

Negative.

Please provide your alert message.

[Rolandwalraven]

[Wed Dec 04 14:57:22.277027 2019] [:error] [pid 8562:tid 140488478672640] [client 11.22.33.44:55097] [client 11.22.33.44] ModSecurity: Warning. Pattern match "\\xbc[^\\\\xbe>][\\xbe>]|<[^\\\\xbe]\\xbe" at ARGS:test. [file "/etc/modsecurity/modsecurity.d/owasp-modsecurity-crs/rules/REQUEST-941-APPLICATION-ATTACK-XSS.conf"] [line "646"] [id "941310"] [msg "US-ASCII Malformed Encoding XSS Filter - Attack Detected."] [data "Matched Data: \xbcge > found within ARGS:test: de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt"] [severity "CRITICAL"] [ver "OWASP_CRS/3.2.0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-tomcat"] [tag "attack-xss"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "OWASP_CRS/WEB_ATTACK/XSS"] [tag "WASCTC/WASC-8"] [tag "WASCTC/WASC-22"] [tag "OWASP_TOP_10/A3"] [tag "OWASP_AppSensor/IE1"] [tag "CAPEC-242"] [hostname "xxxxxx"] [uri "/"] [unique_id "Xee7QnjhHkR1Cr9VH@B4igAAAAE"]

[fgsch]

This is unrelated but which paranoia level are you using?
As this attack is pretty old I wonder if we should move this rule.

[Rolandwalraven]

Level 1 (default)

[dune73]

This is very odd. I can't reproduce.

@fgsch: Can you get this rule to trigger on said payload?

Otherwise, it may be worth to upgrade to Apache and ModSec to 2.4.41 / 2.9.3.

[fgsch]

I haven't tried but I recognise the rule and based on the matched data I see the issue.
Unfortunately this is a problem with some rules in combination with languages other than english.

[dune73]

Totally so. I would not be surprised to see an FP here. But I can't seem to reproduce despite the welcome curl call.

[theseion]

The problem is present on 3.3/dev. There was a change to that particular rule and that change introduced a couple of problems. We're looking into it.

[theseion]

This is the direct conversion of the old regex to the single byte version: (?:\xbc|\xbe).*(?:\xbc|\xbe|>)|(?:\xbc|\xbe|<).*(?:\xbc|\xbe). The character classes in the new regex don't work. I think we should start with the direct conversion and maybe use the optimizer to figure out what to do (@franbuehler thanks for the help).

[theseion]

I figured out how the evasion targeted by rule 941310 works. Look at the following UTF-8 string: ¼script¾alert(¢XSS¢)¼/script¾
If a web server transmits this payload with an encoding of US-ASCII the string will be interpreted as
B<script>Balert(B"XSS"B)B</script>B
This is because US-ASCII uses only 7 bits to encode a character.

The bit sequence for the UTF-8 character ¼ (hexadecimal: C2 BE) is
11000010 10111100
and when you strip the most significant bit from both bytes you get
1000010 0111100
which in US-ASCII are the two characters B and <. Let's do the same for the other two characters:

UTF-8: ¢ (C2 A2) -> 11000010 10100010
US-ASCII: 1000010 0100010 -> B "

UTF-8: ¾ (C2 BC) -> 11000010 10111110
US-ASCII: 1000010 01111101 -> B >

What I'm not sure about is what happens to the B character. As I see it, the resulting string would not be parseable as JS but I don't have a Tomcat server to actually look at the output (see https://www.owasp.org/index.php/XSS_Filter_Evasion_Cheat_Sheet). Anyway, assuming that the B is stripped, all we need for the evasion to work is that the second byte of a two-byte UTF-8 matches the wanted character in US-ASCII. It's possible that the same technique would work with multi-byte characters as well, so, in general, we simply need a byte sequence where the last byte is the one we want.

[fgsch]

Sorry to be late at the party. Is this a follow up from the original report or some other string matching?
If so, can you share what it is matching?

Also, can you elaborate on:

The character classes in the new regex don't work. I think we should start with the direct conversion and maybe use the optimizer to figure out what to do

[theseion]

Yes this is a follow up to the original. He failed to mention that he was using rules from 3.3/dev. The regular expression of rule 941310 was modified in aa2794a.

The string with the match is de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt, in original encoding de_matten & sitzbezüge > fußmatten_mt. The character ü is C3 BC in hexadecimal.

This is the regular expression in 3.3/dev: \xbc[^\xbe>]*[\xbe>]|<[^\xbe]*\xbe.
"The character classes don't work" was clearly an overstatement :) What I meant to say was that \xbc[^\xbe>]*[\xbe>] matches \xbcge > because the first byte of the sequence is no longer being considered. After having understood the problem, however, I see that it is actually correct to only consider the last byte of the sequence. The question is how to prevent false positives now. Maybe by requiring that \xbc be followed by script?

Thanks for looking at this.

[dune73]

Results from the CRS project chat on March 2, 2020: We appreciate @theseion working on this. Thanks in advance!

#1683 (comment)

[NullIsNot0]

I have spent few days trying to figure out why t:urlDecodeUni did not convert latvian or russian characters to their "simplified" version. For example ļ to get converted to l. My main setup is Docker container which uses Ubuntu 18.04 as base image, but have managed to set up also Alpine Linux and Debian (from https://github.com/CRS-support/modsecurity-docker) with the same dissatisfying results. Then I found out that params need to be converted to unicode (with t:utf8toUnicode) before t:urlDecodeUni can be used. Don't know how do you set up your systems that t:urlDecodeUni works, but I have to use t:utf8toUnicode.

ModSecurity config:
SecUnicodeMapFile unicode.mapping 20127 with default version of unicode.mapping file.

CRS version: 3.3/dev (latest available)

I also get lots (I mean really lot) of sqli-attack false-positives on forms where people fill their information in russian or latvian languages. It's because CRS does not do proper unicode decoding.
My suggestion is to add t:utf8toUnicode everywhere t:urlDecodeUni is used and fix unicode.mapping for it to contain as much codepages as necessary for CRS to fully "understand" users inputs. Unless there is an easier solution for this. What do you think?

My test results:

Simplified version of rule 941310:

SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "@rx \xbc[^\xbe>]*[\xbe>]|<[^\xbe]*\xbe" \
  "id:941310,\
  phase:2,\
  block,\
  capture,\
  t:none,t:urlDecodeUni,t:lowercase,t:urlDecode,t:htmlEntityDecode,t:jsDecode,\
  msg:'US-ASCII Malformed Encoding XSS Filter - Attack Detected',\
  logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}'"

String: DE_Matten & Sitzbezüge > Fußmatten_MT (triggers this rule - false positive) (curl http://simple-http-post-test.fakedomain.com?msgbox=DE_Matten+%26+Sitzbez%C3%BCge+%3E+Fu%C3%9Fmatten_MT)

[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] T (0) urlDecodeUni: "DE_Matten & Sitzbez\xc3\xbcge > Fu\xc3\x9fmatten_MT"
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] T (0) lowercase: "de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt"
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] T (0) urlDecode: "de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt"
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] T (0) htmlEntityDecode: "de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt"
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] T (0) jsDecode: "de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt"
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][4] Transformation completed in 1775 usec.
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][4] Executing operator "rx" with param "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" against ARGS:msgbox.
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] Target value: "de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt"
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] Added regex subexpression to TX.0: \xbcge >
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][4] Operator completed in 716 usec.
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] Resolved macro %{TX.0} to: \xbcge >
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] Resolved macro %{MATCHED_VAR_NAME} to: ARGS:msgbox
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][9] Resolved macro %{MATCHED_VAR} to: de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt
[12/Apr/2020:14:14:34 +0000] [simple-http-post-test.fakedomain.com/sid#564f22fb07d0][rid#7fb5240032c0][/][2] Warning. Pattern match "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" at ARGS:msgbox. [file "/opt/apache2/conf.d/127.0.0.1_simple-http-post-test.mydomain.com.conf"] [line "87"] [id "941310"] [msg "US-ASCII Malformed Encoding XSS Filter - Attack Detected"] [data "Matched Data: \xbcge > found within ARGS:msgbox: de_matten & sitzbez\xc3\xbcge > fu\xc3\x9fmatten_mt"]

String: ¼script¾alert(¢XSS¢)¼/script¾ (triggers this rule - ok) (curl http://simple-http-post-test.fakedomain.com?msgbox=%C2%BCscript%C2%BEalert%28%C2%A2XSS%C2%A2%29%C2%BC%2Fscript%C2%BE)

[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] T (0) urlDecodeUni: "\xc2\xbcscript\xc2\xbealert(\xc2\xa2XSS\xc2\xa2)\xc2\xbc/script\xc2\xbe"
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] T (0) lowercase: "\xc2\xbcscript\xc2\xbealert(\xc2\xa2xss\xc2\xa2)\xc2\xbc/script\xc2\xbe"
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] T (0) urlDecode: "\xc2\xbcscript\xc2\xbealert(\xc2\xa2xss\xc2\xa2)\xc2\xbc/script\xc2\xbe"
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] T (0) htmlEntityDecode: "\xc2\xbcscript\xc2\xbealert(\xc2\xa2xss\xc2\xa2)\xc2\xbc/script\xc2\xbe"
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] T (0) jsDecode: "\xc2\xbcscript\xc2\xbealert(\xc2\xa2xss\xc2\xa2)\xc2\xbc/script\xc2\xbe"
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][4] Transformation completed in 2401 usec.
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][4] Executing operator "rx" with param "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" against ARGS:msgbox.
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] Target value: "\xc2\xbcscript\xc2\xbealert(\xc2\xa2xss\xc2\xa2)\xc2\xbc/script\xc2\xbe"
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] Added regex subexpression to TX.0: \xbcscript\xc2\xbe
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][4] Operator completed in 364 usec.
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] Resolved macro %{TX.0} to: \xbcscript\xc2\xbe
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] Resolved macro %{MATCHED_VAR_NAME} to: ARGS:msgbox
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][9] Resolved macro %{MATCHED_VAR} to: \xc2\xbcscript\xc2\xbealert(\xc2\xa2xss\xc2\xa2)\xc2\xbc/script\xc2\xbe
[12/Apr/2020:14:23:34 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d580032c0][/][2] Warning. Pattern match "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" at ARGS:msgbox. [file "/opt/apache2/conf.d/127.0.0.1_simple-http-post-test.mydomain.com.conf"] [line "87"] [id "941310"] [msg "US-ASCII Malformed Encoding XSS Filter - Attack Detected"] [data "Matched Data: \xbcscript\xc2\xbe found within ARGS:msgbox: \xc2\xbcscript\xc2\xbealert(\xc2\xa2xss\xc2\xa2)\xc2\xbc/script\xc2\xbe"]

String: ļoti žēl (triggers this rule - false positive) (curl http://simple-http-post-test.fakedomain.com?msgbox=%C4%BCoti+%C5%BE%C4%93l)

[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] T (0) urlDecodeUni: "\xc4\xbcoti \xc5\xbe\xc4\x93l"
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] T (0) lowercase: "\xc4\xbcoti \xc5\xbe\xc4\x93l"
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] T (0) urlDecode: "\xc4\xbcoti \xc5\xbe\xc4\x93l"
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] T (0) htmlEntityDecode: "\xc4\xbcoti \xc5\xbe\xc4\x93l"
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] T (0) jsDecode: "\xc4\xbcoti \xc5\xbe\xc4\x93l"
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][4] Transformation completed in 5959 usec.
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][4] Executing operator "rx" with param "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" against ARGS:msgbox.
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] Target value: "\xc4\xbcoti \xc5\xbe\xc4\x93l"
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] Added regex subexpression to TX.0: \xbcoti \xc5\xbe
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][4] Operator completed in 468 usec.
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] Resolved macro %{TX.0} to: \xbcoti \xc5\xbe
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] Resolved macro %{MATCHED_VAR_NAME} to: ARGS:msgbox
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][9] Resolved macro %{MATCHED_VAR} to: \xc4\xbcoti \xc5\xbe\xc4\x93l
[12/Apr/2020:14:25:35 +0000] [simple-http-post-test.fakedomain.com/sid#56246bf517d0][rid#7f5d640032c0][/][2] Warning. Pattern match "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" at ARGS:msgbox. [file "/opt/apache2/conf.d/127.0.0.1_simple-http-post-test.mydomain.com.conf"] [line "87"] [id "941310"] [msg "US-ASCII Malformed Encoding XSS Filter - Attack Detected"] [data "Matched Data: \xbcoti \xc5\xbe found within ARGS:msgbox: \xc4\xbcoti \xc5\xbe\xc4\x93l"]

Now add t:utf8toUnicode to rule so the rule becomes this:

SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "@rx \xbc[^\xbe>]*[\xbe>]|<[^\xbe]*\xbe" \
  "id:941310,\
  phase:2,\
  block,\
  capture,\
  t:none,t:utf8toUnicode,t:urlDecodeUni,t:lowercase,t:urlDecode,t:htmlEntityDecode,t:jsDecode,\
  msg:'US-ASCII Malformed Encoding XSS Filter - Attack Detected',\
  logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}'"

String: DE_Matten & Sitzbezüge > Fußmatten_MT (does not trigger this rule - ok) (curl http://simple-http-post-test.fakedomain.com?msgbox=DE_Matten+%26+Sitzbez%C3%BCge+%3E+Fu%C3%9Fmatten_MT)

[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][9] T (0) Utf8toUnicode: "DE_Matten & Sitzbez%u00fcge > Fu%u00dfmatten_MT"
[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][9] T (0) urlDecodeUni: "DE_Matten & Sitzbezuge > Fu\xdfmatten_MT"
[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][9] T (0) lowercase: "de_matten & sitzbezuge > fu\xdfmatten_mt"
[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][9] T (0) urlDecode: "de_matten & sitzbezuge > fu\xdfmatten_mt"
[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][9] T (0) htmlEntityDecode: "de_matten & sitzbezuge > fu\xdfmatten_mt"
[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][9] T (0) jsDecode: "de_matten & sitzbezuge > fu\xdfmatten_mt"
[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][4] Transformation completed in 10397 usec.
[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][4] Executing operator "rx" with param "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" against ARGS:msgbox.
[12/Apr/2020:14:29:52 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e7c0032c0][/][9] Target value: "de_matten & sitzbezuge > fu\xdfmatten_mt"

String: ¼script¾alert(¢XSS¢)¼/script¾ (triggers this rule - ok) (curl http://simple-http-post-test.fakedomain.com?msgbox=%C2%BCscript%C2%BEalert%28%C2%A2XSS%C2%A2%29%C2%BC%2Fscript%C2%BE)

[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] T (0) Utf8toUnicode: "%u00bcscript%u00bealert(%u00a2XSS%u00a2)%u00bc/script%u00be"
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] T (0) urlDecodeUni: "\xbcscript\xbealert(cXSSc)\xbc/script\xbe"
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] T (0) lowercase: "\xbcscript\xbealert(cxssc)\xbc/script\xbe"
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] T (0) urlDecode: "\xbcscript\xbealert(cxssc)\xbc/script\xbe"
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] T (0) htmlEntityDecode: "\xbcscript\xbealert(cxssc)\xbc/script\xbe"
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] T (0) jsDecode: "\xbcscript\xbealert(cxssc)\xbc/script\xbe"
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][4] Transformation completed in 3961 usec.
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][4] Executing operator "rx" with param "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" against ARGS:msgbox.
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] Target value: "\xbcscript\xbealert(cxssc)\xbc/script\xbe"
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] Added regex subexpression to TX.0: \xbcscript\xbe
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][4] Operator completed in 1220 usec.
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] Resolved macro %{TX.0} to: \xbcscript\xbe
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] Resolved macro %{MATCHED_VAR_NAME} to: ARGS:msgbox
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][9] Resolved macro %{MATCHED_VAR} to: \xbcscript\xbealert(cxssc)\xbc/script\xbe
[12/Apr/2020:14:27:53 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e740032c0][/][2] Warning. Pattern match "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" at ARGS:msgbox. [file "/opt/apache2/conf.d/127.0.0.1_simple-http-post-test.mydomain.com.conf"] [line "87"] [id "941310"] [msg "US-ASCII Malformed Encoding XSS Filter - Attack Detected"] [data "Matched Data: \xbcscript\xbe found within ARGS:msgbox: \xbcscript\xbealert(cxssc)\xbc/script\xbe"]

String: ļoti žēl (does not trigger this rule - ok) (curl http://simple-http-post-test.fakedomain.com?msgbox=%C4%BCoti+%C5%BE%C4%93l)

[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][9] T (0) Utf8toUnicode: "%u013coti %u017e%u0113l"
[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][9] T (0) urlDecodeUni: "loti zel"
[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][9] T (0) lowercase: "loti zel"
[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][9] T (0) urlDecode: "loti zel"
[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][9] T (0) htmlEntityDecode: "loti zel"
[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][9] T (0) jsDecode: "loti zel"
[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][4] Transformation completed in 3653 usec.
[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][4] Executing operator "rx" with param "\\xbc[^\\xbe>]*[\\xbe>]|<[^\\xbe]*\\xbe" against ARGS:msgbox.
[12/Apr/2020:14:31:29 +0000] [simple-http-post-test.fakedomain.com/sid#55c4a99307d0][rid#7f8e700032c0][/][9] Target value: "loti zel"

[theseion]

Thanks for that detailed report. My gut tells me that this is probably a separate issue. As I've described above, rule 941310 is there to guard against exploiting an encoding mismatch. I think (without having really looked at it) that what your change does is it modifies the bytes in the request (into Unicode) so that the rule no longer matches. That, however, only goes for the rule. The attack would still be successful because the content is only transformed for the evaluation of the rule, the actual content of the request remains the same. Therefore, when the response is being delivered, nothing will have changed.

So, while your issue may be real, it's likely not related to this issue. Or, if it is, then changing the encoding of the request body is not the way to go.

[NullIsNot0]

My intension is not to skip this rule with real attempt to exploit site with ¼script¾alert(¢XSS¢)¼/script¾, but to reduce false positives on it. Othervise I have to completely disable this rule for most sites with input forms. Because sequence of ļ then ž are quite common in latvian language. For example ļoti žēl - whiche means "so sorry". I can also find similar phrases in russian which trigger this rule, but are not xss attempts.
I don't see other option for this rule not to false trigger, other than to convert unicode characters to their "simple form" in ASCII encoding and only then check if they contain sequence of both ¼, ¾ and \xbc,\xbe.
Encoding issue is causing problems also in sql injection rules and need to be solved there too. I just wanted to know from community, will adding t:utf8toUnicode to rules will be ever considered as potential solution to unicode problem or I have to update these rules just for myself?

[theseion]

Firstly, yes, Unicode characters are a known problem for the CRS in general. Secondly, we need to fix rule 941310 in such a way that it only triggers when it matters, e.g. when coupled with script.

I think you should open a separate issue for your suggestion to use t:utf8toUnicode.