How does one match a pattern unless it is part of a bigger pattern?

[ykuksenko]

I would like to match patterns unless they are part of bigger patterns. I have looked at the rg the man pages, issues, discussions general regex docs and elsewhere and was not able to find an answer for this. It is possible I missed it. The purpose of this is to look for values that should be redacted while ignoring common false positives.

• I would like to use multiple patterns that should be avoided (example sha1sum and sha256sum)
• I would also like to get counts of matches per file like the --count option allows for entire directory trees currently.
• I would like to avoid temporary files if possible.

Example: match a social security number unless it is in a SHA1SUM, so no match should be produced for this example.
SHA1SUM: 1cc3b1a908fbcfd15451c4772759549724f38524
SSN_PATTERN: [0-9]{3}-*[0-9]{2}-*[0-9]{4}

• This pattern alone will match the 477275954 part of the whole checksum.
• I am not using word boundaries because I want to be paranoid for some data.

SHA1SUM_PATTERN: \b([a-f0-9]{40})\b

• This pattern should negate the above match because it matches the whole checksum.

Thanks.

[BurntSushi]

The standard technique here is to use shell pipelines. For example, rg '\w{10}' | rg -v foo. But, this requirement:

I would also like to get counts of matches per file like the --count option allows for entire directory trees currently.

Makes using shell pipelines pretty hard. At least, I can't think of a simple approach other than using --pcre2 with negative look-around. You probably want something like #875 to do this better.

[BurntSushi]

It's worth pointing out here that you could write a very simple Rust program using the regex crate in which you can encode arbitrary logic. Depending on the size of your corpus and the size of your largest files, it'd probably be totally fine to just use one thread, use the walkdir crate and read each file fully into memory.