Snapshot-based reducer tests

[JackYoustra]

Hey all!
I love my logic tests, but it always bothered me that the tests whose primary purpose was to
display UI didn't actually test the UI! So I came up with a few mechanisms to test it.
On the slack I shared some ideas about snapshot reducer test, and want to

• Share what I made
• Get input from what people think is good or bad
• Figure out what method is the best one

The primitive

First, some assumptions:

• Relevant views come in the form of View.init(store:)
• View code is, for the most part, side-effect free (or we can test the parts we care about without worrying about side-effects

Manual submit-based tests:

These are triggered by calling a snapshotting object with the name of the thing being tested.

extension TestStore {
    func storeSubmitter<T: Reducer>(reducer: () -> T) -> StoreOf<T> where State == T.State, Action == T.Action {
        Store(initialState: self.state, reducer: reducer, withDependencies: { $0 = self.dependencies })
    }
    
    struct Snapshotter<T: View, S: TestStore> {
        var count = 0
        let store: S
        let transform: @MainActor (Store<State, Action>) -> T

        @MainActor
        mutating func snapshot(_ name: String, file: StaticString = #file, testName: String = #function, line: UInt = #line) {
            // we probably need to eject dependencies so that we can
            // expose them to the view, if using dependencies in view code
            Dependencies.withDependencies {
                $0 = store.dependencies
            } operation: {
                defer {
                    count += 1
                }
                let v = transform(store.storeSubmitter {
                    Reduce { _, action in
//                        XCTFail("We don't support action submissions, but got \(action)")
                        return .none
                    }
                })
                // sort by order in xcode
                UIView.setAnimationsEnabled(false)
                assertSnapshot(of: v, as: .imageHEIC(layout: .device(config: .iPhone13ProMax)), named: "\(count).\(name)", file: file, testName: testName, line: line)
                UIView.setAnimationsEnabled(true)
            }
        }
    }
    
    func snapshotter<T: View>(view: @MainActor @escaping (Store<State, Action>) -> T) -> Snapshotter<T, TestStore<State, Action>> {
        Snapshotter(store: self, transform: view)
    }
}

and then, later on

var snapshotter = store.snapshotter(view: AppView.init(store:))
// do stuff on store
snapshotter.snapshot("First paint")
// more stuff, more snapshots

This works for my cases right now and can be easily extended to different phones, orientations, etc.

Automatic mode

The idea from this was "snapshot every action" by adding a step to the reducer.
This has a pretty good exhaustivity advantage!

extension Reducer {
    @ReducerBuilder<State, Action>
    func snapshotting<T: View>(view: @escaping (State, Action) -> T?) -> ReducerBuilder<Self.State, Self.Action>._Sequence<Self, Reduce<Self.State, Self.Action>> {
        self
        Reduce { state, action in
            // Client can return nil to skip snapshot
            if let v = view(state, action) {
                assertSnapshot(of: v, as: .imageHEIC(layout: .device(config: .iPhone13ProMax)))
            }
            return .none
        }
    }
    
    func snapshotting<T: View>(view: @escaping (Store<State, Action>) -> T) -> ReducerBuilder<Self.State, Self.Action>._Sequence<Self, Reduce<Self.State, Self.Action>> {
        self.snapshotting { state, _ in
            let v = view(Store(initialState: state, reducer: {
                EmptyReducer()
            }))
            return v
        }
    }
}

Anyway, I didn't pursue this approach because

1. there were a lot of duplicates! I'm not sure how I'd want to handle those, although it'd be cool to have something like a ui changed assertion in a store.send or store.receive to do something like this. Maybe future work?
2. I couldn't figure out how to automatically synthesize stable identifiers. My best idea was to generate the n-th instance of an action, but (1) I couldn't figure out how to do that and (2) it's still not very stable if you add or remove actions in the middle!

Some things I'd like to improve on but haven't figured out how:

• Testing or not just completely disabling animations?
• Waiting for side-effects to resolve before screenshotting? (example: firebase WebImage loads). A sleep would probably be okay here (you could probably find a better way of doing it than that, but I can't even figure out how to do that!)
• Is it a good idea to have the submit-based tests numbered? It makes them unstable but resorts in a very nice timeline that you can follow in finder, while the names give good descriptions about what's going on.

I'm really liking this approach to testing, but it has substantial possibilities for improvement. If anyone has any ideas on this approach (or thinks ditching it for UI test targets is a better idea), please let me know!