Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

T5307: QoS - traffic-class-map services #3492

Merged
merged 1 commit into from
May 31, 2024
Merged

Conversation

HollyGurza
Copy link
Contributor

added new syntax to work with class match filters in QoS policy

Change Summary

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes)
  • Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
  • Other (please describe):

Related Task(s)

Related PR(s)

Component(s) name

QoS

Proposed changes

Implemented the ability to create named match group and use this in QoS policy classes
Old match syntax is kept and tc filters from the match node are created first then tc filters from the match-group node
new syntax to work with group:

set qos policy ... match-group <match_group_name> # add group to the class by group name
# create\change match group
set qos traffic-match-group <group_name> description <value>
set qos traffic-match-group <group_name> match <match_name> [description | ip | ipv6| mark | vif ] ...
set qos traffic-match-group <group_name> match-group <match_group_name> # inherit matches from another group

Every policy class or traffic match group can contain several groups
Every group can contain several matches
Match in the group configured the same as match in the class excluded nodes ether interface

How to test

Create policy using old syntax:

set qos interface eth0 egress VyOS-HTB
set qos policy shaper VyOS-HTB bandwidth 100mbit
set qos policy shaper VyOS-HTB class 10 bandwidth 40%
set qos policy shaper VyOS-HTB class 10 match AF11 ip dscp AF11
set qos policy shaper VyOS-HTB class 10 match AF41 ip dscp AF41
set qos policy shaper VyOS-HTB class 10 match AF43 ip dscp AF43
set qos policy shaper VyOS-HTB class 10 match CS4 ip dscp CS4
set qos policy shaper VyOS-HTB class 10 priority 1
set qos policy shaper VyOS-HTB class 10 queue-type fair-queue
set qos policy shaper VyOS-HTB class 20 bandwidth 30%
set qos policy shaper VyOS-HTB class 20 match EF ip dscp EF
set qos policy shaper VyOS-HTB class 20 match CS5 ip dscp CS5
set qos policy shaper VyOS-HTB class 20 priority 2
set qos policy shaper VyOS-HTB class 20 queue-type fair-queue
set qos policy shaper VyOS-HTB default bandwidth 20%
set qos policy shaper VyOS-HTB default queue-type fair-queue
commit
tc filter show dev eth0

delete existing policy

del qos policy shaper VyOS-HTB
del qos interface eth0 egress VyOS-HTB
commit
tc filter show dev eth0

create same policy using new syntax

set qos interface eth0 egress VyOS-HTB
# prepare new match group
set qos traffic-match-group VOICE description 'voice shaper'
set qos traffic-match-group VOICE match EF ip dscp EF
set qos traffic-match-group VOICE match CS5 ip dscp CS5
set qos traffic-match-group REAL_TIME_COMMON description 'real time common filters'
set qos traffic-match-group REAL_TIME_COMMON match AF43 ip dscp AF43
set qos traffic-match-group REAL_TIME_COMMON match CS4 ip dscp CS4
set qos traffic-match-group REAL_TIME description 'real time shaper'
set qos traffic-match-group REAL_TIME match AF41 ip dscp AF41
set qos traffic-match-group REAL_TIME match-group REAL_TIME_COMMON
# create same policy using new match groups
set qos policy shaper VyOS-HTB bandwidth 100mbit
set qos policy shaper VyOS-HTB class 10 bandwidth 40%
# old match and new match group can work together
set qos policy shaper VyOS-HTB class 10 match AF11 ip dscp AF11
set qos policy shaper VyOS-HTB class 10 match-group REAL_TIME

set qos policy shaper VyOS-HTB class 10 priority 1
set qos policy shaper VyOS-HTB class 10 queue-type fair-queue
set qos policy shaper VyOS-HTB class 20 bandwidth 30%
# also we can use only match group
set qos policy shaper VyOS-HTB class 20 match-group VOICE

set qos policy shaper VyOS-HTB class 20 priority 2
set qos policy shaper VyOS-HTB class 20 queue-type fair-queue
set qos policy shaper VyOS-HTB default bandwidth 20%
set qos policy shaper VyOS-HTB default queue-type fair-queue
commit
tc filter show dev eth0

compare tc filters after old and new syntax
new:

vyos@vyos# tc filter show dev eth0
filter parent 1: protocol all pref 1 u32 chain 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800: ht divisor 1
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00280000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::801 order 2049 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00880000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::802 order 2050 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00980000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::803 order 2051 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00800000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::804 order 2052 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00800000/00ff0000 at 0
        action order 1:  police 0x1 rate 4Gbit burst 14500b mtu 2Kb action reclassify overhead 0b
        ref 1 bind 1

filter parent 1: protocol all pref 2 u32 chain 0
filter parent 1: protocol all pref 2 u32 chain 0 fh 801: ht divisor 1
filter parent 1: protocol all pref 2 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1:14 not_in_hw
  match 00a00000/00ff0000 at 0
filter parent 1: protocol all pref 2 u32 chain 0 fh 801::801 order 2049 key ht 801 bkt 0 flowid 1:14 not_in_hw
  match 00b80000/00ff0000 at 0
filter parent 1: protocol all pref 2 u32 chain 0 fh 801::802 order 2050 key ht 801 bkt 0 flowid 1:14 not_in_hw
  match 00b80000/00ff0000 at 0
        action order 1:  police 0x2 rate 3Gbit burst 15000b mtu 2Kb action reclassify overhead 0b
        ref 1 bind 1

old:

vyos@vyos# tc filter show dev eth0
filter parent 1: protocol all pref 1 u32 chain 0 
filter parent 1: protocol all pref 1 u32 chain 0 fh 800: ht divisor 1 
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:a not_in_hw 
  match 00280000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::801 order 2049 key ht 800 bkt 0 flowid 1:a not_in_hw 
  match 00880000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::802 order 2050 key ht 800 bkt 0 flowid 1:a not_in_hw 
  match 00980000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::803 order 2051 key ht 800 bkt 0 flowid 1:a not_in_hw 
  match 00800000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::804 order 2052 key ht 800 bkt 0 flowid 1:a not_in_hw 
  match 00800000/00ff0000 at 0
        action order 1:  police 0x1 rate 4Gbit burst 14500b mtu 2Kb action reclassify overhead 0b 
        ref 1 bind 1 

filter parent 1: protocol all pref 2 u32 chain 0 
filter parent 1: protocol all pref 2 u32 chain 0 fh 801: ht divisor 1 
filter parent 1: protocol all pref 2 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1:14 not_in_hw 
  match 00a00000/00ff0000 at 0
filter parent 1: protocol all pref 2 u32 chain 0 fh 801::801 order 2049 key ht 801 bkt 0 flowid 1:14 not_in_hw 
  match 00b80000/00ff0000 at 0
filter parent 1: protocol all pref 2 u32 chain 0 fh 801::802 order 2050 key ht 801 bkt 0 flowid 1:14 not_in_hw 
  match 00b80000/00ff0000 at 0
        action order 1:  police 0x2 rate 3Gbit burst 15000b mtu 2Kb action reclassify overhead 0b 
        ref 1 bind 1

Smoketest result

vyos@vyos:~$ python3 /usr/libexec/vyos/tests/smoke/cli/test_qos.py 
test_01_cake (__main__.TestQoS.test_01_cake) ... ok
test_02_drop_tail (__main__.TestQoS.test_02_drop_tail) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_03_fair_queue (__main__.TestQoS.test_03_fair_queue) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_04_fq_codel (__main__.TestQoS.test_04_fq_codel) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_05_limiter (__main__.TestQoS.test_05_limiter) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_06_network_emulator (__main__.TestQoS.test_06_network_emulator) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_07_priority_queue (__main__.TestQoS.test_07_priority_queue) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_08_random_detect (__main__.TestQoS.test_08_random_detect) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_09_rate_control (__main__.TestQoS.test_09_rate_control) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_10_round_robin (__main__.TestQoS.test_10_round_robin) ... 
Selected QoS policy "qos-policy-eth0" does not exist!

ok
test_11_shaper (__main__.TestQoS.test_11_shaper) ... ok
test_12_shaper_with_red_queue (__main__.TestQoS.test_12_shaper_with_red_queue) ... ok
test_13_shaper_delete_only_rule (__main__.TestQoS.test_13_shaper_delete_only_rule) ... ok
test_14_traffic_match_group (__main__.TestQoS.test_14_traffic_match_group) ... ok
test_14_wrong_traffic_match_group (__main__.TestQoS.test_14_wrong_traffic_match_group) ... 
Can not use both IPv6 and IPv4 in one match (one)!


WARNING: Match group "unexpected" does not exist!

ok

----------------------------------------------------------------------
Ran 15 tests in 56.881s

OK

Checklist:

  • I have read the CONTRIBUTING document
  • I have linked this PR to one or more Phabricator Task(s)
  • I have run the components SMOKETESTS if applicable
  • My commit headlines contain a valid Task id
  • My change requires a change to the documentation
  • I have updated the documentation accordingly

@sever-sever
Copy link
Member

What happens if you use both (old/new) syntaxes that overlap?

@HollyGurza
Copy link
Contributor Author

HollyGurza commented May 21, 2024

What happens if you use both (old/new) syntaxes that overlap?

All matches from match-group will added to match, duplicate matches will be ignored if are exactly equal

No matter whether use both or only one syntax, validating and applying matches will work the same as if they were created through the old syntax

@c-po
Copy link
Member

c-po commented May 21, 2024

Conflicting files
smoketest/scripts/cli/test_qos.py

Copy link

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Copy link

Conflicts have been resolved. A maintainer will review the pull request shortly.

Copy link
Contributor

@fett0 fett0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tested it in my lab ,it seems to be correct . we can add multiple `match-group' and combine them with the old syntax , here there is an example adding a new match-group :

set qos policy shaper VyOS-HTB class 10 bandwidth '30%'
set qos policy shaper VyOS-HTB class 10 description 'dscp_EF_ipprec_5_GETS'
set qos policy shaper VyOS-HTB class 10 match CS4 ip dscp 'CS4'
set qos policy shaper VyOS-HTB class 10 match-group 'Streaming-Video'
set qos policy shaper VyOS-HTB class 10 priority '1'
set qos policy shaper VyOS-HTB class 10 queue-type 'fair-queue'
set qos policy shaper VyOS-HTB class 20 bandwidth '30%'
set qos policy shaper VyOS-HTB class 20 description 'dscp_AF4x_ipprec_4'
set qos policy shaper VyOS-HTB class 20 match ef ip dscp 'EF'
set qos policy shaper VyOS-HTB class 20 priority '2'
set qos policy shaper VyOS-HTB class 20 queue-type 'fair-queue'
set qos policy shaper VyOS-HTB class 30 bandwidth '10%'
set qos policy shaper VyOS-HTB class 30 description 'MC'
set qos policy shaper VyOS-HTB class 30 match-group 'Mission-Critical'
set qos policy shaper VyOS-HTB class 30 priority '3'
set qos policy shaper VyOS-HTB class 30 queue-type 'fair-queue'
set qos policy shaper VyOS-HTB default bandwidth '20%'
set qos policy shaper VyOS-HTB default queue-type 'fq-codel'
set qos traffic-match-group Mission-Critical match AF31 ip dscp 'AF31'
set qos traffic-match-group Mission-Critical match AF32 ip dscp 'AF42'
set qos traffic-match-group Mission-Critical match CS3 ip dscp 'CS3'
set qos traffic-match-group Streaming-Video match AF11 ip dscp 'AF11'
set qos traffic-match-group Streaming-Video match AF41 ip dscp 'AF41'
set qos traffic-match-group Streaming-Video match AF43 ip dscp 'AF43'

they are applied on tc-filter when we check it :

vyos@vyos# sudo tc filter show dev eth0
filter parent 1: protocol all pref 1 u32 chain 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800: ht divisor 1
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::800 order 2048 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00800000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::801 order 2049 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00280000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::802 order 2050 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00880000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::803 order 2051 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00980000/00ff0000 at 0
filter parent 1: protocol all pref 1 u32 chain 0 fh 800::804 order 2052 key ht 800 bkt 0 flowid 1:a not_in_hw
  match 00980000/00ff0000 at 0
	action order 1:  police 0x1 rate 300Mbit burst 15300b mtu 2Kb action reclassify overhead 0b
	ref 1 bind 1

filter parent 1: protocol all pref 2 u32 chain 0
filter parent 1: protocol all pref 2 u32 chain 0 fh 801: ht divisor 1
filter parent 1: protocol all pref 2 u32 chain 0 fh 801::800 order 2048 key ht 801 bkt 0 flowid 1:14 not_in_hw
  match 00b80000/00ff0000 at 0
filter parent 1: protocol all pref 2 u32 chain 0 fh 801::801 order 2049 key ht 801 bkt 0 flowid 1:14 not_in_hw
  match 00b80000/00ff0000 at 0
	action order 1:  police 0x2 rate 300Mbit burst 15300b mtu 2Kb action reclassify overhead 0b
	ref 1 bind 1

filter parent 1: protocol all pref 3 u32 chain 0
filter parent 1: protocol all pref 3 u32 chain 0 fh 802: ht divisor 1
filter parent 1: protocol all pref 3 u32 chain 0 fh 802::800 order 2048 key ht 802 bkt 0 flowid 1:1e not_in_hw
  match 00680000/00ff0000 at 0
filter parent 1: protocol all pref 3 u32 chain 0 fh 802::801 order 2049 key ht 802 bkt 0 flowid 1:1e not_in_hw
  match 00900000/00ff0000 at 0
filter parent 1: protocol all pref 3 u32 chain 0 fh 802::802 order 2050 key ht 802 bkt 0 flowid 1:1e not_in_hw
  match 00600000/00ff0000 at 0
filter parent 1: protocol all pref 3 u32 chain 0 fh 802::803 order 2051 key ht 802 bkt 0 flowid 1:1e not_in_hw
  match 00600000/00ff0000 at 0
	action order 1:  police 0x3 rate 100Mbit burst 15337b mtu 2Kb action reclassify overhead 0b
	ref 1 bind 1

I would like to suggest to merge it only 1.5, so, users can feel comfortable with this new structure and wait for proper documentation.

<constraint>
<regex>[^-].*</regex>
</constraint>
<constraintErrorMessage>Match group name cannot start with hyphen (-)</constraintErrorMessage>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need (-) at the end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

<constraint>
<regex>[^-].*</regex>
</constraint>
<constraintErrorMessage>Match queue name cannot start with hyphen (-)</constraintErrorMessage>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need (-) at the end?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got this message from original match node and just keep it structure

<constraintErrorMessage>Match queue name cannot start with hyphen (-)</constraintErrorMessage>

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

added new syntax to work with class match filters in QoS policy
@sever-sever sever-sever merged commit 5563bcd into vyos:current May 31, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

4 participants