Failure Similarity: The stack trace or error message resembles patterns of past flaky failures.

[jmafoster1]

Based on the techniques detailed in Section 2 of this paper, there are three possible techniques here:

• Simple text-based matching of exception types and stack traces (NOT exception messages, which can contain irrelevant details such as IP addresses that are unique to each run)
• Failure Log Classifier trained on existing known flaky failures.
• Term Frequency-Inverse Document Frequency of tokenised failure logs
• Distance metric between embedding vectors (OpenAI API)

We could also calculate embeddings of failure logs and measure similarity that way.

[jmafoster1]

May want to do some preprocessing on stack traces ("abstraction techniques" from failure dedupliucation) to remove irrelevant info such as IP addresses, line numbers, etc.