app-parser for Commvault audit trail

[rlai0013]

Hi all,

Looking to ingest some Commvault audit trail logs. Unfortunately they are not RFC compliant so they go to the fallback index. I set up a very simple app-parser to get it to the correct index, but there are some issues.

• The logs are not even close to RFC compliant (there is no header at all...) 😢
• The logs come through in batches some time after the event has happened
• Because of the above, the events do not have the correct timestamp

Would be great to have an app-parser created for these logs.

Here are some redacted samples:

AuditTrail: Opid = {119262} Audittime = {21 Oct 2022 11:35:11} Severitylevel = {Low} Username = {my-user} Operation = {User Logged out} Details = { Login Name: DOMAINNAME\my-user Machine: my-host-name01}

AuditTrail: Opid = {119263} Audittime = {21 Oct 2022 11:35:31} Severitylevel = {Medium} Username = {my-user} Operation = {Login Failed} Details = { Login Name: DOMAINNAME\my-user Machine: my-host-name01}

AuditTrail: Opid = {119264} Audittime = {21 Oct 2022 11:36:40} Severitylevel = {Low} Username = {my-user} Operation = {User Logged in} Details = { Login Name: DOMAINNAME\my-user Machine: my-host-name01 Locale: English-UnitedStates}

AuditTrail: Opid = {119271} Audittime = {21 Oct 2022 11:56:55} Severitylevel = {Medium} Operation = {Login Failed} Details = { Login failure: username:[DOMAINNAME\my-user], host/ device:[127.0.0.1], login attempted from: [AdminConsole]}

AuditTrail: Opid = {119265} Audittime = {21 Oct 2022 11:50:11} Severitylevel = {Low} Username = {Name-name name} Operation = {User Logged out} Details = { Logout : Login name: [DOMAINNAME\my-user], Host: [127.0.0.1]}

AuditTrail: Opid = {119269} Audittime = {21 Oct 2022 11:51:16} Severitylevel = {Low} Username = {Name-name name} Operation = {User Logged in} Details = { Login success: username:[Name-name name], host/ device:[127.0.0.1], logged in from: [AdminConsole]}

AuditTrail: Opid = {119276} Audittime = {21 Oct 2022 11:58:05} Severitylevel = {High} Operation = {User account locked} Details = { Login attempts for user [DOMAINNAME\my-user] exceeded limit. Account is locked for [5minute(s)]}

AuditTrail: Opid = {119241} Audittime = {20 Oct 2022 14:32:46} Severitylevel = {Medium} Username = {Name-name name} Operation = {Restore Contents} Details = { Client : [SERVERNAME01_AGClient] Agent Type : [SQL Server] Instance : [SERVERNAME01/SERVERNAME01]: Set to [true] Schedules Schedule Schedule Type: Set to [RESTORE] Operation Type: Set to [Restore] Job Options Restore Options Sql Server Rst Option Drop Connections To Database: Set to [true] Unconditionally overwrite existing database or files: Set to [true] Point-in-time: Set to [true] Device Info: Set to [6 TestDB 0 U:\SQLData\testDB.mdf TestDB 8388608 TestDB ] Device Info: Set to [6 TestDB 0 V:\SQLLog\testDB_log.ldf TestDB_log 8388608 TestDB ] |Device: Set to [TestDB|#0012!TestDB|#0012!TestDB|#0012!U:\SQLData\testDB.mdf|#0012!U:\SQLData\testDB.mdf] |Device: Set to [TestDB|#0012!TestDB|#0012!TestDB_log|#0012!V:\SQLLog\testDB_log.ldf|#0012!V:\SQLLog\testDB_log.ldf] Database: Set to [testDB] Restore Source: Set to [testDB] Browse Option Comm Cell ID: Set to [2] Destination Destination client Client Name: Set to [servername01] Destination Instance Instance Name: Set to [SERVERNAME01] Common Options Subscription Info: Set to [<Api_Subscription subscriptionId ="1178"/>]}

Thanks!

[rjha-splunk]

Can you please share the pcap to me using external slack channel, need to see how the raw sample looks like and if we can consider this data source for a parser.

[rlai0013]

These are already the "raw" samples as seen when running tcpdump and when enabling SC4S_SOURCE_STORE_RAWMSG.

[bparmar-splunk]

@rlai0013,
Is this issue still exist ?

[rlai0013]

Hi @bparmar-splunk
Yes, this issue still exists.

[bparmar-splunk]

@rlai0013,
Could you please answer following questions:

• what is the source (what TA is associated to this sample)?

• What is the sourcetype of the message?

[rlai0013]

@bparmar-splunk

• what is the source (what TA is associated to this sample)?

By TA I assume you mean Splunk add-on. There is no TA for this data

• What is the sourcetype of the message?

There is no TA, so no fixed sourcetype. I'm currently naming it commvault:syslog

[bparmar-splunk]

@rlai0013,
Since this parser does not have TA associated, nor it is RFC compliant. I am mentioning the parser below that parses above messages.

rewrite set_raw_commvault_audit{
    set-tag("wireformat:raw_commvault_audit");
};
filter f_is_raw_commvault_audit{
    tags("wireformat:raw_commvault_audit");
};

block parser app-raw-commvault_audit() {
    channel {
        rewrite{
            r_set_splunk_dest_default(
                index('netfw')
                sourcetype('commvault:syslog')
                vendor("commvault")
                product("audit")
            );
        };
        parser {
            kv-parser(prefix(".values.") pair-separator(" ") template(t_hdr_msg));
        };
        
        parser {
            regexp-parser(
                prefix(".tmp.")
                patterns('(?<message>(?<message_pre>AuditTrail.*)Audittime\s=\s{(?<timestamp>\d{2}\s\w{3}\s\d{4}\s\d{2}:\d{2}:\d{2})}\sSeveritylevel)')
            );
        };
        parser {
            date-parser(
                format(
                    '%d %b %Y %H:%M:%S'
                )
                template("${.tmp.timestamp}")
            );
        };

        rewrite(set_raw);
        rewrite(set_raw_commvault_audit);
        rewrite(r_set_source_identified);
    };

};
application app-raw-commvault_audit[sc4s-raw-syslog] {
	filter {
        message('AuditTrail'  type(string)  flags(prefix));
    };
    parser { app-raw-commvault_audit(); };
};

Please let us know in case of any further queries.

[rlai0013]

Thanks for your help @bparmar-splunk

[nitronarcosis]

@bparmar-splunk Can this be added as a known-vendor sourcetype? Alternately, how can I use this parser in my environment?

[bparmar-splunk]

@nitronarcosis,
Please refer FAQ page for your query.

[akn-br]

I notice that the latest version doesn't have the app-parser for Commvault audit trail? Is it being planned to be re-released in a future update?

Thanks,
AKN

[mstopa-splunk]

Hi @akn-br since the vendor doesn't comply with RFC we provided a local parser that we'd rather not include in to the standard SC4S configuration.

Please save this: #1878 (comment) to your local SC4S directory, for example /opt/sc4s/local/config/app_parsers, and restart SC4S.