With the current dev HEAD of Sphinx (prior to 8.2.0 release) highlight_language (and its default fallback) is not obeyed

[jfbu]

What version of myst-parser are you using?

4.0.0

What version dependencies are you using?

Sphinx version: 8.2.0+/e65bbb96a
Python version: 3.12.3 (CPython)
Docutils version: 0.21.2 
Pygments version: 2.18.0
Jinja2 version: 3.1.4
MyST-Parser version: 3.0.1 and 4.0.0

What operating system are you using?

Mac

Describe the Bug

This was first raised as Sphinx issue 13207, which is now closed because the change should be applied here... I am opening this only so that status can be tracked, I hope it is ok ;-).

Expected Behavior

Code-blocks without explicitly assigned lexers should use the Sphinx default (i.e. python) if highlight_language is not set in conf.py and they should use highlight_language if set.

To Reproduce

Attaching two small examples. Install in your Python environment a Sphinx version at least as recent as its commit e65bbb96 (which is in-between 8.1.3 and 8.2.0) and issue make html to confirm the result shows no highlighting in the page whose source is in Markdown syntax (but has correct highlighting in the page whose source is in reST).

13207_a.zip
13207_b.zip

Update: the change indicated by Adam at sphinx-doc/sphinx#13207 (comment) fixes this not only with the MRE's provided but also in the original context (which has some extra layers specific to it). I tried out the proposed change not only with Sphinx development HEAD but also with its 8.1.3.