gh-115999: Enable BINARY_SUBSCR_GETITEM for free-threaded build

[corona10]

• Issue: Make the specializing interpreter thread-safe in --disable-gil builds #115999

[corona10]

@mpage, @colesbury

To enable BINARY_SUBSCR_GETITEM, we should handle TypeObject and _spec_cache which are mutable by other threads.
To handle this, I have introduced PyType_GetItemFromCache, _PyType_GetItemFromCacheWithVersion, and _PyType_CacheGetItemForSpecialization to lock the type and mutate them at the same transaction.

But I am not sure this approach is proper approach since I noticed that cached items can be mutated before the interpreter gets cached objects, so I have added the logic to check whether the item is NULL or FunctionObject, but please let me know if my approach is wrong.

Anyway, this is the first working version without error. PTAL.

[mpage]

Thanks for working on this! I left a few comments inline.

[mpage]

I think it's probably safe to reload getitem from the cache after the guard passes. (If the getitem in the cache is replaced after the guard passes, the new getitem cannot be mutated in a way that would change the function version: that would require stopping the world). However, I'd prefer that we pass the getitem that is loaded in _BINARY_SUBSCR_CHECK_FUNC to _BINARY_SUBSCR_INIT_CALL.

[corona10]

@mpage Hmm would you like to take a look again?

[mpage]

Overall approach is looking good! I'd still prefer it if we passed the getitem that is loaded in _BINARY_SUBSCR_CHECK_FUNC to _BINARY_SUBSCR_INIT_CALL. That would also allow us to avoid introducing the DEOPT_IF in _BINARY_SUBSCR_INIT_CALL. Is there a reason you don't want to do that?

[mpage]

I can't see it. You may forget to press the comment button, I guess :)

Oops - fixed!

[corona10]

Done :)

[mpage]

Looks good! I noticed one last thing then I think we're good to go.

We need to update the opcode definition of _BINARY_SUBSCR_INIT_CALL in Python/optimizer_bytecodes.c to handle the extra output (getitem) of _BINARY_SUBSCR_CHECK_FUNC.

[corona10]

Done! Even if I am not sure that this change is the correct for tier 2 because I have never handled this before :)

[Fidget-Spinner]

The tier2 change is correct. I should add an automatic script that checks tier2 and tier1 definitions are synced so that you don't trip over this. Thanks Matt for catching that!

[corona10]

@mpage Okay to merge this PR?

[mpage]

I kicked off a couple of benchmark runs earlier today. They should be done shortly. Let’s wait for the results. Otherwise, looks great!

[mpage]

Looks good! Benchmark results:

• ~1% improvement on free-threaded builds
• ~neutral on default builds

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable 3.x has failed when building commit 48c70b8.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/90/builds/5364) and take a look at the build logs.
4. Check if the failure is related to this commit (48c70b8) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/90/builds/5364

Failed tests:

• test_socket

Failed subtests:

• testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le/build/Lib/test/test_socket.py", line 550, in testStream
    msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Clang 3.x has failed when building commit 48c70b8.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/435/builds/5142) and take a look at the build logs.
4. Check if the failure is related to this commit (48c70b8) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/435/builds/5142

Failed tests:

• test_socket

Failed subtests:

• testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/Lib/test/test_socket.py", line 550, in testStream
    msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Clang Installed 3.x has failed when building commit 48c70b8.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/451/builds/5124) and take a look at the build logs.
4. Check if the failure is related to this commit (48c70b8) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/451/builds/5124

Failed tests:

• test_socket

Failed subtests:

• testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.14/test/test_socket.py", line 550, in testStream
    msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable LTO + PGO 3.x has failed when building commit 48c70b8.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/101/builds/5356) and take a look at the build logs.
4. Check if the failure is related to this commit (48c70b8) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/101/builds/5356

Failed tests:

• test_socket

Failed subtests:

• testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_socket.py", line 550, in testStream
    msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable LTO 3.x has failed when building commit 48c70b8.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/541/builds/4746) and take a look at the build logs.
4. Check if the failure is related to this commit (48c70b8) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/541/builds/4746

Failed tests:

• test_socket

Failed subtests:

• testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/test/test_socket.py", line 550, in testStream
    msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Refleaks 3.x has failed when building commit 48c70b8.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/280/builds/1480) and take a look at the build logs.
4. Check if the failure is related to this commit (48c70b8) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/280/builds/1480

Failed tests:

• test_socket

Failed subtests:

• testStream - test.test_socket.ThreadedVSOCKSocketStreamTest.testStream

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.refleak/build/Lib/test/test_socket.py", line 550, in testStream
    msg = self.conn.recv(1024)
PermissionError: [Errno 13] Permission denied

[markshannon]

Looks good! Benchmark results:

• ~1% improvement on free-threaded builds
• ~neutral on default builds

I don't think a 0.3-0.5% slower on average with a 13% slowdown on the gc_traversal benchmark is "neutral".
It might be just noise, but the large slowdown on the gc_traversal benchmark suggests a real issue.

Could we revert this PR until we have a better understanding of what is going on here?

[colesbury]

The gc_traversal benchmark results sure looks like noise to me. That benchmark is dominated by list_traverse and functions in the GC, none of which have changed.

How about you run the benchmarks in the faster-cpython infrastructure and see if you get the same results?