Test Suites idea: "Compare this failed test with the last time it succeeded"

[jonbell-lot23]

@jonbell-lot23 and @jcmorrow were chatting about test suites and Josh pointed out that failed tests could automatically pull up the last successful run for comparison's sake. An intriguing idea to keep in our back pocket.

Additional insights:

• When you run 1000 tests and only 1 failed, you really only care about that 1
• When you're on a team of 100 people, it's pretty common to only care about your tests
• It's extremely uncommon to look at a passed test's Replay, but if you are you're probably...
  • Trying to figure out what the test even does, because it's not yours
  • Needing to compare it to a failing test to see what changed
• Josh noted that diff is hard, but in cases where there's only a single hit, we could have an apples to apples comparison, and in some cases that could get someone pretty far to fixing a bug. Exciting!

In the spirit of Bret Victor and making coding/debugging/problem solving as accessible and awesome as possible, we wanted to flag this idea. We thought it was pretty neat.