Skip to content

Commit

Permalink
vcc_acl_merge parameter: remove subnets and merge adjacent entries
Browse files Browse the repository at this point in the history 
Function:

With the merge acl flag enabled (default: disabled, keeping the
existing behavior), ACLs are optimized in that subnets contained in
other entries are skipped (e.g. if 1.2.3.0/24 is part of the ACL, an
entry for 1.2.3.128/25 will not be added) and adjacent entries get
merged (e.g. if both 1.2.3.0/25 and 1.2.3.128/25 are added, they will
be merged to 1.2.3.0/24).

Skip and merge operations on VCL entries are output as warnings during
VCL compilation as entries from the VCL are processed in order.

Logging under the VCL_acl tag can change with this parameter enabled:
Matches on skipped subnet entries are now logged as matches on the
respective supernet entry. Matches on merged entries are logged with a
shorter netmask which might not be contained in the original ACL as
defined in VCL. Such log entries are marked by "fixed: merged".

Negated ACL entries are excluded from merges.

Implementation:

The sort functions are changed such that the previous semantics are
preserved: negative return values signify "a < b", positive return
values signify "a > b". But additionally the values -2/2 and -3/3 are
introduced (and given enums) to signify "contained in supernet" and
"directly adjacent to". This allows for mostly unchanged code with
vcc_acl_merge disabled.

For the "subnet contained in supernet" case, all contained subnets are
removed. By sort order, caontained subnets are always to be found left
of supernets.

For the "merge adjacent" case, the netmask of the entry with the
smaller network number is decreased by one and the other entry
removed. Because changing the netmask might affect sort order, we
reinsert the changed entry.
  • Loading branch information
@nigoroll
nigoroll committed Jul 6, 2023
1 parent 5bdcc0b commit b28a84d
Show file tree
Hide file tree
Showing 4 changed files with 265 additions and 26 deletions.
126 changes: 126 additions & 0 deletions bin/varnishtest/tests/c00005.vtc
View file
Original file line number Diff line number Diff line change
Expand Up @@ -162,3 +162,129 @@ varnish v1 -errvcl {Non-zero bits in masked part} {
if (client.ip ~ acl1) {}
}
}

# this is both an OK test for pedantic and merge
varnish v1 -vcl {
import std;

backend dummy None;

acl acl1 +log +pedantic +merge {
# bad notation (confusing)
"1.2.3.0"/24;
"1.2.3.64"/26;

# all contained in 1.3.0.0/21 and 1.4.4.0/22
"1.4.4.0"/22;
"1.3.4.0"/23;
"1.3.5.0"/26;
"1.3.6.0"/25;
"1.3.6.128"/25;
"1.3.0.0"/21;
"1.4.7";
"1.4.6.0"/24;

# right,left adjacent
"2.3.2.0"/23;
"2.3.0.0"/23;
# left,right adjacent
"2.3.4.0"/23;
"2.3.6.0"/23;

# phks test case
"10.0.0.0"/23;
"10.0.2.0"/23;

"10.1.0.0"/24;
"10.1.1.0"/24;

"10.2.0.0"/25;
"10.2.0.128"/25;
}

sub vcl_recv {
return (synth(200));
}
sub t {
if (std.ip(req.http.ip) ~ acl1) { }
}
sub vcl_synth {
# variables would be nice, but not in core (yet?)
set req.http.ip = "1.2.3.0"; call t;
set req.http.ip = "1.2.3.63"; call t;
set req.http.ip = "1.2.3.64"; call t;

set req.http.ip = "1.3.4.255"; call t;
set req.http.ip = "1.3.5.0"; call t;
set req.http.ip = "1.3.5.255"; call t;
set req.http.ip = "1.3.6.0"; call t;
set req.http.ip = "1.3.6.140"; call t;
set req.http.ip = "1.3.7.255"; call t;

set req.http.ip = "1.4.5.255"; call t;
set req.http.ip = "1.4.6.64"; call t;
set req.http.ip = "1.4.7.64"; call t;

set req.http.ip = "2.3.0.0"; call t;
set req.http.ip = "2.3.5.255"; call t;

set req.http.ip = "2.2.255.255";call t;
set req.http.ip = "2.3.8.0"; call t;

set req.http.ip = "10.0.3.255"; call t;
set req.http.ip = "10.1.1.255"; call t;
set req.http.ip = "10.2.0.255"; call t;
}
}

logexpect l1 -v v1 -g raw {
expect * 1009 ReqHeader {^\Qip: 1.2.3.0\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.2.3.0"/24\E$}
expect 1 = ReqHeader {^\Qip: 1.2.3.63\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.2.3.0"/24\E$}
expect 1 = ReqHeader {^\Qip: 1.2.3.64\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.2.3.0"/24\E$}

expect 1 = ReqHeader {^\Qip: 1.3.4.255\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.3.0.0"/21\E$}
expect 1 = ReqHeader {^\Qip: 1.3.5.0\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.3.0.0"/21\E$}
expect 1 = ReqHeader {^\Qip: 1.3.5.255\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.3.0.0"/21\E$}
expect 1 = ReqHeader {^\Qip: 1.3.6.0\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.3.0.0"/21\E$}
expect 1 = ReqHeader {^\Qip: 1.3.6.140\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.3.0.0"/21\E$}
expect 1 = ReqHeader {^\Qip: 1.3.7.255\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.3.0.0"/21\E$}

expect 1 = ReqHeader {^\Qip: 1.4.5.255\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.4.4.0"/22\E$}
expect 1 = ReqHeader {^\Qip: 1.4.6.64\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.4.4.0"/22\E$}
expect 1 = ReqHeader {^\Qip: 1.4.7.64\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "1.4.4.0"/22\E$}

expect 1 = ReqHeader {^\Qip: 2.3.0.0\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "2.3.0.0"/21 fixed: merged\E}
expect 1 = ReqHeader {^\Qip: 2.3.5.255\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "2.3.0.0"/21 fixed: merged\E}
expect 1 = ReqHeader {^\Qip: 2.2.255.255\E$$}
expect 0 = VCL_acl {^\QNO_MATCH acl1\E$}
expect 1 = ReqHeader {^\Qip: 2.3.8.0\E$}
expect 0 = VCL_acl {^\QNO_MATCH acl1\E$}

expect 1 = ReqHeader {^\Qip: 10.0.3.255\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "10.0.0.0"/22 fixed: merged\E}
expect 1 = ReqHeader {^\Qip: 10.1.1.255\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "10.1.0.0"/23 fixed: merged\E}
expect 1 = ReqHeader {^\Qip: 10.2.0.255\E$}
expect 0 = VCL_acl {^\QMATCH acl1 "10.2.0.0"/24 fixed: merged\E}
} -start

client c1 {
txreq
rxresp
} -run

logexpect l1 -wait
21 changes: 21 additions & 0 deletions doc/sphinx/reference/vcl.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,27 @@ individually:
However, if the name resolves to both IPv4 and IPv6 you will still
get an error.

* `+merge` - Merge ACL supernets and adjacent networks.

With this parameter set to on, ACLs are optimized in that subnets
contained in other entries are skipped (e.g. if 1.2.3.0/24 is part
of the ACL, an entry for 1.2.3.128/25 will not be added) and
adjacent entries get merged (e.g. if both 1.2.3.0/25 and
1.2.3.128/25 are added, they will be merged to 1.2.3.0/24).

Skip and merge operations on VCL entries are output as warnings
during VCL compilation as entries from the VCL are processed in
order.

Logging under the ``VCL_acl`` tag can change with this parameter
enabled: Matches on skipped subnet entries are now logged as matches
on the respective supernet entry. Matches on merged entries are
logged with a shorter netmask which might not be contained in the
original ACL as defined in VCL. Such log entries are marked by
``fixed: merged``.

Negated ACL entries are never merged.

VCL objects
-----------

Expand Down
22 changes: 13 additions & 9 deletions include/tbl/vsl_tags.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -264,15 +264,19 @@ SLTM(Fetch_Body, 0, "Body fetched from backend",
SLTM(VCL_acl, 0, "VCL ACL check results",
"ACLs with the `+log` flag emits this record with the result.\n\n"
"The format is::\n\n"
"\t%s %s [%s [fixed: %s]]\n"
"\t| | | |\n"
"\t| | | +- Fixed entry (see acl +pedantic flag)\n"
"\t| | +------------ Matching entry (only for MATCH)\n"
"\t| +---------------- Name of the ACL\n"
"\t+-------------------- MATCH or NO_MATCH\n"
"\n"
"MATCH denotes an ACL match\n"
"NO_MATCH denotes that a checked ACL has not matched\n"
"\t%s [%s [%s [fixed: %s]]]\n"
"\t| | | |\n"
"\t| | | +- Fix info (see below)\n"
"\t| | +------------ Matching entry (only for MATCH)\n"
"\t| +---------------- Name of the ACL for MATCH or NO_MATCH\n"
"\t+-------------------- MATCH, NO_MATCH or NO_FAM\n"
"\n"
"* Fix info: either contains network/mask for non-canonical entries "
"(see acl +pedantic flag) or ``merged`` for entries "
"which were the result of a merge operation (see acl +merge flag).\n"
"* ``MATCH`` denotes an ACL match\n"
"* ``NO_MATCH`` denotes that a checked ACL has not matched\n"
"* ``NO_FAM`` denotes a missing address family and should not occur.\n"
"\n"
)

Expand Down
Loading

0 comments on commit b28a84d

Please sign in to comment.