Implementation multi pattern regular expressions by biathlon3 · Pull Request #2161 · tempesta-tech/tempesta

biathlon3 · 2024-07-05T14:07:59Z

To compile regexp required installation of hscollider. For simplicity it built as debian package.

Add repository

sudo install -m 0755 -d /etc/apt/keyrings
sudo wget -O /etc/apt/keyrings/tempesta.asc https://tempesta-tech.com:8081/repository/public-keys/keys/public.key
sudo chmod a+r /etc/apt/keyrings/tempesta.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/tempesta.asc] \
https://tempesta-tech.com:8081/repository/tempesta/ focal main" | sudo tee /etc/apt/sources.list.d/tempesta.list
sudo apt update && sudo apt upgrade

Install the package

apt install linux-regex

biathlon3 · 2024-07-26T13:29:54Z

I squashed it into one commit so it can be reviewed.

Enabling avx512 causing compilation time wanings relateed to string fortification. Compiler can't check the size of the data, because the length of the data calculates at runtime. Do unsafe copy to suppress the warnings.

The config can be included using !include directive in http_chain section. Example: ``` http_chain { #some rules ... !include /etc/tempesta/ua_block_rules.conf } ```

const-t · 2026-02-10T14:26:22Z

I catch kasan BUG in
tests.http_rules.test_http_tables.TestHttpTablesEmptyHdrPattern.test_empty_name. Please run all tests with kasan and kmemleak and check results.

Fixed

Do strict matching to not match not suitable strings like "!includeasd" and " #include"

What has been done: - Added a user-space helper `regex_setup.sh` that passes expressions to hscollider and copies the compiled database to the rex module database. Path to the script can be specified using environment variable REGEX_SETUP_SCRIPT_PATH and path to the regex databases can be specified using REGEX_DIR_PATH variable. Implemented as kernel module parameter. - Moved regular expression code to its own file. - Rex directory added to `rules` file.

In this patch we allocate only the amount of memory needed for the regex ID, instead of allocating memory for the full string pattern

Remove outdated comment and change minimal regex length. Currently we allow even 1 symbol regex, but not empty regex.

`cookie "cookie_name" ~ "cookie_value"` is not supported at this moment, seems it doesn't make sense to have such option, matching by header could be useed instead to find all occurrences in single regex call. Example: `hdr cookie ~ "/cookie_name=cookie_value/"`

EvgeniiMekhanik · 2026-02-19T09:34:29Z

scripts/tempesta.sh

@@ -114,7 +118,7 @@ templater()
 		line=$(echo "$raw_line" | sed -e '/request /s/\\r\\n/\x0d\x0a/g')
 		if [[ ${line:0:1} = \# ]]; then
 			:
-		elif [[ $line =~ '!include' ]]; then
+		elif [[ $line =~ ^[[:space:]]*!include[[:space:]]+.+$ ]]; then


Is it ok that we can have any count of space before include?

EvgeniiMekhanik · 2026-02-19T09:44:36Z

fw/t/unit/regex.c

@@ -0,0 +1 @@
+../../regex.c


No empty line at the end of the file?

EvgeniiMekhanik

Please also add three stress tests with reconfiguration with regexp:

With success reconfiguration regexp config - regexp config
Fail reconfig
Success was regexp now not regexp

biathlon3 linked an issue Jul 5, 2024 that may be closed by this pull request

Multi-pattern regular expressions #496

Open

biathlon3 marked this pull request as draft July 5, 2024 14:08

This comment was marked as outdated.

Sign in to view

biathlon3 force-pushed the ag_Multi-pattern-regular-expressions branch 2 times, most recently from 5c02ff5 to 8c64bfa Compare July 8, 2024 17:09

biathlon3 mentioned this pull request Jul 12, 2024

Tests for Multi-pattern regular expressions tempesta-tech/tempesta-test#648

Open

biathlon3 force-pushed the ag_Multi-pattern-regular-expressions branch 2 times, most recently from 4f4608e to d932ab5 Compare July 25, 2024 11:13

biathlon3 marked this pull request as ready for review July 26, 2024 13:24

biathlon3 force-pushed the ag_Multi-pattern-regular-expressions branch 2 times, most recently from 5f2b864 to 09f70b3 Compare July 29, 2024 18:23

krizhanovsky requested review from EvgeniiMekhanik, const-t and krizhanovsky and removed request for EvgeniiMekhanik and krizhanovsky July 30, 2024 19:40

biathlon3 mentioned this pull request Aug 13, 2024

feat(1808): migrate to linux 6.8.9 #2124

Closed

const-t marked this pull request as draft April 11, 2025 13:33

const-t force-pushed the ag_Multi-pattern-regular-expressions branch 2 times, most recently from 7f2013e to 88c3570 Compare October 27, 2025 16:07

const-t force-pushed the ag_Multi-pattern-regular-expressions branch from 88c3570 to ce3286e Compare November 7, 2025 12:35

const-t force-pushed the ag_Multi-pattern-regular-expressions branch from ce3286e to 5d5654c Compare November 14, 2025 13:20

const-t force-pushed the ag_Multi-pattern-regular-expressions branch from 5d5654c to ddea565 Compare December 19, 2025 16:20

const-t marked this pull request as ready for review December 19, 2025 16:21

const-t force-pushed the ag_Multi-pattern-regular-expressions branch from ddea565 to 37e1965 Compare December 22, 2025 09:14

const-t force-pushed the ag_Multi-pattern-regular-expressions branch from 4ca8b16 to ebc646e Compare January 7, 2026 13:13

const-t added 11 commits February 10, 2026 13:41

feat: remove spaces from header pattern

24fd3a0

tests: add couple of tests and fix already existing

7306b5b

refactor: remove unused fields

a6f7768

refactor: use kernel's min/max

f69cb80

fix: enable avx512

3aca714

Enabling avx512 causing compilation time wanings relateed to string fortification. Compiler can't check the size of the data, because the length of the data calculates at runtime. Do unsafe copy to suppress the warnings.

feat: compile regex with the same standard as kernel

6f994c9

fix: remove *~ token, instead use /i modifier in the pattern

2dfb88c

fix: remove unused argument and add validation

92a5a0d

chore: remove unused file

798ecaf

fix: add error handling when writing regexp file

2c23d7b

feat: add config rules to block clients with suspicious user-agents

d28fc46

The config can be included using !include directive in http_chain section. Example: ``` http_chain { #some rules ... !include /etc/tempesta/ua_block_rules.conf } ```

const-t force-pushed the ag_Multi-pattern-regular-expressions branch 2 times, most recently from f3147f6 to 3a1196e Compare February 10, 2026 13:46

const-t added 11 commits February 11, 2026 10:42

fix: match strictly only "!include path"

fb4218d

Do strict matching to not match not suitable strings like "!includeasd" and " #include"

fix: Allocate only required amount of memory for the regex ID

546c4de

In this patch we allocate only the amount of memory needed for the regex ID, instead of allocating memory for the full string pattern

fix: alignment

2261560

fix: forbid using wildcard with regexp

0325040

fix: regex length validation

dc05f13

Remove outdated comment and change minimal regex length. Currently we allow even 1 symbol regex, but not empty regex.

refactor: add comment and change arguments name

4567afe

feat: compile avx/avx2/avx512 only if it is supported

bf32454

feat: enable avx512vbmi

a5d7ff4

fix: memory leak during config_item release

869b9c6

const-t force-pushed the ag_Multi-pattern-regular-expressions branch from 3a1196e to e3a81a4 Compare February 11, 2026 08:46

const-t requested a review from EvgeniiMekhanik February 13, 2026 16:13

EvgeniiMekhanik reviewed Feb 19, 2026

View reviewed changes

EvgeniiMekhanik approved these changes Feb 19, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implementation multi pattern regular expressions#2161

Implementation multi pattern regular expressions#2161
biathlon3 wants to merge 31 commits intomasterfrom
ag_Multi-pattern-regular-expressions

biathlon3 commented Jul 5, 2024 •

edited by const-t

Loading

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

biathlon3 commented Jul 26, 2024

Uh oh!

const-t commented Feb 10, 2026

Uh oh!

EvgeniiMekhanik Feb 19, 2026

Uh oh!

EvgeniiMekhanik Feb 19, 2026

Uh oh!

EvgeniiMekhanik left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -0,0 +1 @@
		../../regex.c No newline at end of file

Conversation

biathlon3 commented Jul 5, 2024 • edited by const-t Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

biathlon3 commented Jul 26, 2024

Uh oh!

const-t commented Feb 10, 2026

Uh oh!

EvgeniiMekhanik Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

EvgeniiMekhanik Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

EvgeniiMekhanik left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

biathlon3 commented Jul 5, 2024 •

edited by const-t

Loading