Turn some warnings into errors in test suite

[DanielNoord]

Unblocked by #2446

This should hopefully help catch DeprecationWarnings sooner.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.25%. Comparing base (3fb0400) to head (18f1706).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2449   +/-   ##
=======================================
  Coverage   99.25%   99.25%           
=======================================
  Files          40       40           
  Lines        3089     3089           
  Branches      680      680           
=======================================
  Hits         3066     3066           
  Misses         13       13           
  Partials       10       10           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.