Improve graph-by-graph bench (#1317) #4558

Job	Run time
pytester (ubuntu-22.04, 3.10, latest, core)	12m 0s
pytester (ubuntu-22.04, 3.10, latest, ops)	24m 58s
pytester (ubuntu-22.04, 3.10, nightly, core)	12m 29s
pytester (ubuntu-22.04, 3.10, nightly, ops)	24m 51s
pytester (macOS-14, 3.10, latest, core)	8m 0s
pytester (macOS-14, 3.10, latest, ops)	18m 6s
pytester (macOS-14, 3.10, nightly, core)	8m 35s
pytester (macOS-14, 3.10, nightly, ops)	18m 33s
pytester (windows-2022, 3.10, latest, core)	19m 53s
pytester (windows-2022, 3.10, nightly, core)	21m 26s
pytester (ubuntu-22.04, 3.11, latest)	19m 48s
pytester (ubuntu-22.04, 3.12, latest)	23m 37s
testing-guardian	2s
	3h 32m 18s

Provide feedback