Library doesn't detect syntax issues in conf files.

[yaroslav-nakonechnikov]

Hello,

with help of this library i've created simple script to validate conf files:

import addonfactory_splunk_conf_parser_lib as conf_parser
import os
import argparse

def check_syntax(filename):
    parser = conf_parser.TABConfigParser()
    try:
        with open(filename, 'r') as f:
            parser.read_file(f)
        print(f"Syntax of the file '{filename}' is correct.")
    except conf_parser.Error as e:  # Catching all configparser errors with a generic class
        print(f"Syntax error in file '{filename}': {e}")
    except FileNotFoundError:
        print(f"File '{filename}' not found.")
    except Exception as e:
        print(f"An unexpected error occurred in file '{filename}': {e}")

def scan_directory_for_conf_files(directory):
    for root, dirs, files in os.walk(directory):
        for file in files:
            if file.endswith('.conf'):
                file_path = os.path.join(root, file)
                check_syntax(file_path)

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description='Syntax checker for .conf files.')
    parser.add_argument('dir_path', type=str, help='Path to direcory to scan')

    args = parser.parse_args()

    # Determine the operation to perform
    if args.dir_path:
        scan_directory_for_conf_files(args.dir_path)
    else:
        directory_to_scan = input("Enter the directory path to scan for .conf files: ")
        scan_directory_for_conf_files(directory_to_scan)

and in that script when file has syntax error - it thinks that all good.

example:

as you see, empty space is there, but it should be wiped.

and it passes wrong multine search:

[artemrys]

@yaroslav-nakonechnikov I think it's an expected behavior for this particular library. I believe it does not check for incorrect SPL syntax in values.

You might try using btool to check for syntax errors in .conf files.

[yaroslav-nakonechnikov]

btool is too heavy to run. and it also skips a lot of syntax issues.

also noticed: when it parses, it also trims lines. Which is fine, if there will be need to store file.

[artemrys]

btool is too heavy to run. and it also skips a lot of syntax issues.

do you mind sharing an example of such behavior?

I can ask around but I never saw a library/tool to parse and validate .conf files.

[yaroslav-nakonechnikov]

sorry for late response.

example is quite easy: conf file is being created locally, without splunk installed.
and installing btool - means to install splunk.

as well for ci pipelines, using splunk is heavy and consumes resources (time, cpu, storage, network)

and moreover - splunk containers are not consistent. we were trying to use as recent versions as possible, but from version to version - it breaks. Never was related to btool tool, but still not comfortable to see how new version breaks working setup.

[yaroslav-nakonechnikov]

so, i managed to run btool without splunk itself... binary looks working, but still - it has too many dependencies like SPLUNK_HOME just to start.
then i had to copy libraries from $SPLUNK_HOME/lib, or do symlinks... but something, which took additional effort. I believe in some OSes it would be possible to install needed ones from official repos. In ubuntu 24.04 it wasn't possible for some.

and then i put file in $SPLUNK_HOME/etc/system/local/savedsearches.conf
where i have issues with syntax - and it failes to detect:

~/splunkforwarder/bin$ ./btool --debug check
No spec file for: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/app.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/default-mode.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/health.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/limits.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/props.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/server.conf
Checking: /home/user/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/web.conf
No spec file for: /home/user/splunkforwarder/etc/apps/introspection_generator_addon/default/app.conf
Checking: /home/user/splunkforwarder/etc/apps/introspection_generator_addon/default/inputs.conf
Checking: /home/user/splunkforwarder/etc/apps/introspection_generator_addon/default/server.conf
Checking: /home/user/splunkforwarder/etc/apps/journald_input/default/authorize.conf
Checking: /home/user/splunkforwarder/etc/apps/journald_input/default/inputs.conf
No spec file for: /home/user/splunkforwarder/etc/apps/search/default/app.conf
Checking: /home/user/splunkforwarder/etc/apps/search/default/props.conf
Checking: /home/user/splunkforwarder/etc/apps/search/default/restmap.conf
Checking: /home/user/splunkforwarder/etc/apps/search/default/transforms.conf
Checking: /home/user/splunkforwarder/etc/apps/splunk_httpinput/default/inputs.conf
No spec file for: /home/user/splunkforwarder/etc/apps/splunk_internal_metrics/default/app.conf
Checking: /home/user/splunkforwarder/etc/apps/splunk_internal_metrics/default/props.conf
Checking: /home/user/splunkforwarder/etc/apps/splunk_internal_metrics/default/transforms.conf
No spec file for: /home/user/splunkforwarder/etc/manager-apps/_cluster/default/indexes.conf
Checking: /home/user/splunkforwarder/etc/system/default/alert_actions.conf
No spec file for: /home/user/splunkforwarder/etc/system/default/app.conf
Checking: /home/user/splunkforwarder/etc/system/default/audit.conf
Checking: /home/user/splunkforwarder/etc/system/default/authentication.conf
Checking: /home/user/splunkforwarder/etc/system/default/authorize.conf
No spec file for: /home/user/splunkforwarder/etc/system/default/conf.conf
Checking: /home/user/splunkforwarder/etc/system/default/default-mode.conf
Checking: /home/user/splunkforwarder/etc/system/default/federated.conf
Checking: /home/user/splunkforwarder/etc/system/default/field_filters.conf
Checking: /home/user/splunkforwarder/etc/system/default/global-banner.conf
Checking: /home/user/splunkforwarder/etc/system/default/health.conf
Checking: /home/user/splunkforwarder/etc/system/default/inputs.conf
Checking: /home/user/splunkforwarder/etc/system/default/limits.conf
Checking: /home/user/splunkforwarder/etc/system/default/livetail.conf
Checking: /home/user/splunkforwarder/etc/system/default/messages.conf
Checking: /home/user/splunkforwarder/etc/system/default/metric_alerts.conf
Checking: /home/user/splunkforwarder/etc/system/default/metric_rollups.conf
Checking: /home/user/splunkforwarder/etc/system/default/outputs.conf
Checking: /home/user/splunkforwarder/etc/system/default/procmon-filters.conf
Checking: /home/user/splunkforwarder/etc/system/default/props.conf
Checking: /home/user/splunkforwarder/etc/system/default/restmap.conf
Checking: /home/user/splunkforwarder/etc/system/default/server.conf
Checking: /home/user/splunkforwarder/etc/system/default/source-classifier.conf
No spec file for: /home/user/splunkforwarder/etc/system/default/telemetry.conf
Checking: /home/user/splunkforwarder/etc/system/default/transforms.conf
Checking: /home/user/splunkforwarder/etc/system/default/visualizations.conf
Checking: /home/user/splunkforwarder/etc/system/default/web-features.conf
Checking: /home/user/splunkforwarder/etc/system/default/web.conf
No spec file for: /home/user/splunkforwarder/etc/system/local/savedsearches.conf

so, still, library does it better even in current version.

[artemrys]

thanks @yaroslav-nakonechnikov, your points are valid.

I did some homework and I see that VSCode extension for Splunk (https://github.com/splunk/vscode-extension-splunk) offers some syntax highlighting and linting capabilities, let us explore it next week. There are also some other extensions that offer similar functionality.

If that would be easy to incorporate into this library (or maybe another one) - I think we can go with it.