T7282: op-mode: show firewall group filtering and tab completion update

[markh0338]

Change summary

When showing firewall group , any dynamic groups are also shown along with the requested group name. Dynamic-groups, mac-groups and domain-groups are also not shown in tab completion.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Code style update (formatting, renaming)
• Refactoring (no functional changes)
• Migration from an old Vyatta component to vyos-1x, please link to related PR inside obsoleted component
• Other (please describe):

Related Task(s)

T7282

Related PR(s)

How to test / Smoketest result

Checklist:

• I have read the CONTRIBUTING document
• I have linked this PR to one or more Phabricator Task(s)
• I have run the components SMOKETESTS if applicable
• My commit headlines contain a valid Task id
• My change requires a change to the documentation
• I have updated the documentation accordingly

[github-actions]

👍
No issues in PR Title / Commit Title

[markh0338]

Linting failed in the firewall.py file, but not in the section I modified. I can reformat the file if you’d like, but I wanted to keep the focus on the purpose of the PR instead of linting.

[c-po]

Thank you for the submission. Looks like you are mangling two changes into one commit which is - from a software development and life cycle point - not a good idea.

Can you please split this change into two individual commits?

• Update TAB completion helpers
• Fix the for loop by an early exit in the script showing the firewall rules?

[markh0338]

Thank you for the submission. Looks like you are mangling two changes into one commit which is - from a software development and life cycle point - not a good idea.

Can you please split this change into two individual commits?

• Update TAB completion helpers

• Fix the for loop by an early exit in the script showing the firewall rules?

Just to clarify, would you want these completely split with different tasks and PRs? Or just different commits inside this PR?

[c-po]

Please use individual commits inside a single PR referencing the same task id.

[c-po]

Implementation looks good to me

[markh0338]

Rebased on current to sync up other firewall.py changes for remote-groups

[sever-sever]

Completion help is useful for firewall groups

[markh0338]

before this gets merged - I think there is an issue with how the path tab completion works. Admittedly, I don't know how all of that works either currently but there is an issue if a group type isn't found it doesn't continue down the list. so in this commit, if a dynamic group isn't found, the rest won't process.

[markh0338]

Completely overhauled the path completion logic. Previously, when using multiple entries in the completion help, a series of ls commands were chained together to list the directories. However, since these were executed in order, if the first directory had no groups defined, the remaining ls commands would not run—resulting in an incomplete group listing.

With the updated approach, each group directory is checked individually and only listed if it exists, ensuring more accurate and complete output.

[dmbaturin]

This approach will break down when we drop the UnionFS-based config trees in favor of in-memory configs. I cannot see why the issue on hand cannot be fixed by adding <path>firewall group dynamic-group address-group </path> and <path>firewall group dynamic-group ipv6-address-group address-group</path> to the list.

Am I missing anything here?

[markh0338]

This was my exact assumption as well, and it was also the basis for my initial commit for this change. However, for reasons I don’t fully understand, when the list of ls commands is executed, as soon as a missing directory is encountered due to an unconfigured group type, any groups that follow it in the sequence are not listed. Below is an example from the latest rolling release without this patch applied.

Here is the default sequence of ls commands:

ls /opt/vyatta/config/active/firewall/group/address-group 2>/dev/null && 
ls /opt/vyatta/config/active/firewall/group/network-group 2>/dev/null &&
ls /opt/vyatta/config/active/firewall/group/port-group 2>/dev/null && 
ls /opt/vyatta/config/active/firewall/group/interface-group 2>/dev/null &&
ls /opt/vyatta/config/active/firewall/group/ipv6-address-group 2>/dev/null &&
ls /opt/vyatta/config/active/firewall/group/ipv6-network-group 2>/dev/null

If I configure an address-group and an interface-group—skipping network-group and port-group—the interface-group does not appear in tab completion.

Configured groups:

vyos@vyos# show firewall group
 address-group addr-group {
     address 10.1.1.1
     address 10.1.1.2
     address 10.1.1.3
 }
 interface-group int-group {
     interface eth0
     interface eth1
 }

In op-mode, when attempting tab completion:

vyos@vyos:~$ show firewall group
Possible completions:
  <Enter>               Execute the current command
  addr-group            Show firewall group
  detail                Show list view of firewall group

No interface-groups are shown. And if I try to ls the network-group, I get this error:

vyos@vyos:~$ ls /opt/vyatta/config/active/firewall/group/
address-group/   interface-group/
vyos@vyos:~$ ls /opt/vyatta/config/active/firewall/group/network-group
ls: cannot access '/opt/vyatta/config/active/firewall/group/network-group': No such file or directory

My assumption is that since the return code for this command is non-zero, the rest of the commands in the sequence are not executed. As a result, any groups that follow the unconfigured one are not returned. Hope this helps.

[dmbaturin]

I see... I'm going to look into that completion helper.

[markh0338]

Just to expand further—when using && to concatenate ls commands, each command must return a zero exit code for the next command to run. If any command returns a non-zero code, all subsequent commands are skipped.

vyos@vyos:~$ true && echo 'ls command'
ls command
vyos@vyos:~$ false && echo 'ls command'
vyos@vyos:~$ 

I haven’t yet found where these commands are built from the XML, but if semicolons (;) were used instead of double ampersands (&&), then all commands would run regardless of the return code.

vyos@vyos:~$ true ; echo 'ls command'
ls command
vyos@vyos:~$ false ; echo 'ls command'
ls command
vyos@vyos:~$

Here’s the same test using the actual op-mode commands:

vyos@vyos:~$ ls /opt/vyatta/config/active/firewall/group/address-group 2>/dev/null && 
ls /opt/vyatta/config/active/firewall/group/network-group 2>/dev/null && 
ls /opt/vyatta/config/active/firewall/group/port-group 2>/dev/null && 
ls /opt/vyatta/config/active/firewall/group/interface-group 2>/dev/null && 
ls /opt/vyatta/config/active/firewall/group/ipv6-address-group 2>/dev/null && 
ls /opt/vyatta/config/active/firewall/group/ipv6-network-group 2>/dev/null
addr-group
vyos@vyos:~$

vyos@vyos:~$ ls /opt/vyatta/config/active/firewall/group/address-group 2>/dev/null ; 
ls /opt/vyatta/config/active/firewall/group/network-group 2>/dev/null ; 
ls /opt/vyatta/config/active/firewall/group/port-group 2>/dev/null ; 
ls /opt/vyatta/config/active/firewall/group/interface-group 2>/dev/null ; 
ls /opt/vyatta/config/active/firewall/group/ipv6-address-group 2>/dev/null ; 
ls /opt/vyatta/config/active/firewall/group/ipv6-network-group 2>/dev/null
addr-group
int-group
vyos@vyos:~$

I’m sure there may be other consequences to this change that I’m not currently aware of, but I just wanted to offer it up as an option.

[markh0338]

I’m attempting this change to observe its effects. So far, it appears to work fine for this particular issue. However, I haven’t been able to extensively test it yet to ensure there are no negative effects elsewhere.

diff --git a/scripts/build-command-op-templates b/scripts/build-command-op-templates
index d203fdcef..0bb62113e 100755
--- a/scripts/build-command-op-templates
+++ b/scripts/build-command-op-templates
@@ -116,7 +116,7 @@ def get_properties(p):
             if comptype is not None:
                 props["comp_type"] = "imagefiles"
                 comp_exprs.append("echo -n \"<imagefiles>\"")
-            comp_help = " && ".join(comp_exprs)
+            comp_help = " ; ".join(comp_exprs)
             props["comp_help"] = comp_help

[dmbaturin]

From a somewhat cynical point of view, broken completion isn't going to get anyone into a serious trouble — in the worst case they can copy the name or type it by hand. So I'd say we can give it a go and see what happens.

[markh0338]

For sure, it is a minor quality of life improvement. It was a small change that ended up taking me down a rabbit hole of problems. I'll push this update in a bit to see how it goes. From my research I don't think switching to a semicolon will have any negative impact. Thanks!

[github-actions]

CI integration 👍 passed!

Details

CI logs

• CLI Smoketests (no interfaces) 👍 passed
• CLI Smoketests (interfaces only) 👍 passed
• Config tests 👍 passed
• RAID1 tests 👍 passed
• TPM tests 👍 passed

[dmbaturin]

From my quick test it worked, and I think it's safe enough since it will create nothing more than an annoyance even if it breaks some commands completion.

[c-po]

Do not error out on first completion helper not returning 0 (&&) and rather continue with the next completion helper (;)