PyPy 7.3.12 was released with python 2.7, python 3.9, python 3.10

[mattip]

Comment:

We should update 3.9 and maybe backport some of the _ssl fixes that closed memory leaks to 3.8.

On the one hand it would be nice to package python3.10 for PyPy. On the other, is starting a new migration for pypy worth the effort for conda-forge: in terms of CI churn and maintainer attention? I don't have a grasp of how popular PyPy is.

[h-vetinari]

IMO, not migrating for PyPy3.10 would be (long-term) equivalent with dropping PyPy in conda-forge completely. I don't think that's really in anyone's interest - there's been a tremendous amount of work invested that we'd be throwing away.

So for me personally it's a no-brainer to migrate for 3.10¹ (though the exact timeline is less important). PyPy has also made great strides and it looks like we'll be able to get past things like grpcio and arrow this time around. 🥳

The one thing I'd like to do more properly this time is to ensure we are able to pass most of the pandas test suite (see here); we'll also have to figure out the cryptography failures. I'd go as far as making those a blocker for starting the migration, because these two packages show up really early in the topologically sorted graph.

CC @conda-forge/core

Footnotes

1. it's similarly obvious IMO to drop PyPy3.8 at the same time, since 2 versions should be plenty, and it also became unsupported upstream. ↩

[mattip]

Dropping PyPy's python 3.8 before dropping CPython's 3.8 might add some friction. I seem to remember some issues around the 3.7 drop cadence being out of sync.

[h-vetinari]

I brought up this subject in the core call this week, and it was mentioned that PyPy-in-conda-forge would be difficult to maintain if the PyPy devs (essentially that's been you, Matti) weren't available to help debug & fix things.

We also wanted to get some better download numbers. Not sure if someone took that up, but out of idle curiosity I did a non-comprehensive analysis. Looking at the downloads for the most recent builds (about 50 days old; no aarch/ppc builds), we have:

arch 👉 python 👇	linux-64	osx-64	win-64	total
pypy38	4,630	1,983	1,742	8,355
pypy39	7,388	2,424	2,384	12,196
total	12,018	4,407	4,126	20,551

I'm not sure how that compares to downloads from pypy.org, but 20k users isn't nothing. It's more than CPython gets downloads on linux-ppc64le in a similar timespan, for example (if we're comparing "nicheness").

However the above doesn't allow a direct comparison to CPython, because the amount of times the various builds have been online aren't the same (and obviously downloads constantly increase over time). Trying to look at some foundational packages (as a proxy for non-trivial environments), I looked at current downloads of numpy 1.25.0, which has been live for a little over 2 weeks (but unfortunately - for the purpose of this comparison - dropped 3.8 support):

arch 👉 python 👇	linux-64	linux-aarch64	linux-ppc64le	osx-64	osx-arm64	win-64	total	%
py39	57,938	1,862	514	10,383	2,632	15,981	89,310	26.82%
pypy39	808	176	140	346	0	406	1,876	0.56%
py310	95,355	1,783	510	12,277	4,522	18,217	132,664	39.83%
py311	74,158	1,577	667	11,833	3,731	17,236	109,202	32.79%
total	228,259	5,398	1,831	34,839	10,885	51,840	333,052	100.00%
%	68.54%	1.62%	0.55%	10.46%	3.27%	15.57%	100.00%

Aside from generally pretty extreme skew across platforms/versions, we still see that PyPy is on par with ppc (even though that's not exactly an apples-to-apples comparison).

While numpy can serve as a proxy for the "data" style of environment, cryptography (underlying requests) can probably be taken as a proxy for almost everything else, and certainly the "web" style of environment. The most recent version has been live in conda-forge for slightly over a month, and has the following numbers:

arch 👉 python 👇	linux-64	linux-aarch64	linux-ppc64le	osx-64	osx-arm64	win-64	total	%
py38	72,159	740	146	10,450	987	22,988	107,470	20.00%
pypy38	287	174	83	118	0	126	788	0.15%
py39	93,391	3,831	196	9,724	3,714	15,013	125,869	23.42%
pypy39	697	158	134	321	0	361	1,671	0.31%
py310	158,490	6,467	477	13,193	6,830	18,557	204,014	37.97%
py311	68,755	1,259	195	10,145	2,232	15,374	97,960	18.23%
total	393,779	12,629	806	43,951	13,763	72,419	537,347	100.00%
%	73.28%	2.35%	0.15%	8.18%	2.56%	13.48%	100.00%

The picture mostly repeats itself here. Personally, I don't think the comparison to CPython here is the relevant metric for success/relevance here, but I gave it for completeness. The question is how much work it is to support pypy as an additional implementation, who does it, how many users can be served by that, and whether that's all worth it.

Regarding the sentiment "I haven't heard any bug reports" you mentioned elsewhere w.r.t. the (lack of) impact of conda-forge packages, the thought occurred to me that this could also be explained by us doing our job well (i.e. we might not package everything, but the stuff we package works pretty well). ;-)

Though again, I have no idea what kind of monthly download numbers PyPy achieves elsewhere. This handy PyPI-based dashboard doesn't have that data on the consumer side, but I raised an issue, because it should hopefully be easy to add.

[mattip]

Thanks.

compares to downloads from pypy.org

Unfortunately we don't have those statistics. The "official" portable, non-conda pypy tarballs are served from https://downloads.python.org/pypy/, which is managed by the PSF. I will ask the PSF infra maintainers if they have any statistics.

PyPy-in-conda-forge would be difficult to maintain if the PyPy devs (essentially that's been you, Matti) weren't available to help debug & fix things

The bus factor here is a little scary, and I am in fact trying to cut back on the amount of time I personally spend on PyPy.

[h-vetinari]

Of course it doesn't have to be you, and things are pretty OK if the migrations run through fine, but any non-trivial debugging is outside of the expertise of almost all maintainers in conda-forge, and aside from the PyPy devs (could be someone else than you though...), there's pretty much no-one else to ask.

[hmaarrfk]

I personally think that our collective time is better spent moving package migrations forward rather than starting a new migration from the ground up regardless of what the user numbers say. One big aspect of conda(-forge) is the ability to integrate packages together. h5py, scikit-learn are both notable packages that are required for my usecase to even start to consider moving to pypy.

Just my thoughts.

I do think that given the work that has been put into documenting the issues, and fixing them upstream, the future migrations should get "easier" but you know.... there are always challenges.

[mattip]

h5py, scikit-learn are both notable packages that are required for my usecase

Much of the data science stack heavily uses c-extensions and refcount sementics, which are fast on CPython and problematic for PyPy. There is some kind of problem with buffer lifetimes in h5py due to refcount semantics, and the scikit-learn CI takes way too long on PyPy to work out what is still failing. Without sponsorship or a champion willing to put in the time to figure out what is going wrong, those feedstocks are unfortunately unlikely to move forward.

[hmaarrfk]

Without sponsorship or a champion willing to put in the time to figure out what is going wrong, those feedstocks are unfortunately unlikely to move forward.

Understood and I am in total agreement with your decisions.

I guess I'm hesitant to use any statistics that could be skewed by our heavy use of CIs.

[h-vetinari]

@mattip, not sure if you've seen any noticeable change now that PyPy points to conda-forge more prominently?

In any case, I think we'll have to come to a decision about the future of PyPy-in-conda-forge. As mentioned above, the sentiment in @conda-forge/core (in the last core call where this was discussed and I was present) was essentially: "we're happy to keep it going, as long as we have support from the PyPy devs".

If you want to spend less time on this (which would be more than fair enough!) and there's no-one else to help from PyPy side to tackle the tricky cases in a migration, then I think we're going to be forced to wind down PyPy support. Note that I'm only speaking for myself when I say this (not cf/core), and I say it despite thinking it's a huge pity. But it is effectively the status quo of the last few months already, and the bitrot over time in such cases is merciless.

[mattip]

I don't think conda-forge should start a pypy3.10 migration yet. Is there a plan for when conda-forge drops python3.8?

I will try to update this feedstock for pypy3.9 and fix #101.

Solving the problems in the comment above seem to be blockers for finishing the pypy3.9 migration. Is there other bitrot you see?

[h-vetinari]

Is there other bitrot you see?

First I want to make clear that this is not a criticism of PyPy, much less you. I think it is an inevitable state of affairs of things for a given (packaged) snapshot, compounded by the fact that if we stop maintaining resp. migrating (non-PyPy) dependencies, this will just fall further and further off the wagon.

For pypy3.9 in particular I think we need to fix cryptography and pandas (IMO much more importantly than h5py/scikit-learn, because the former two are already migrated).

[mattip]

I think we need to fix cryptography and pandas

Could you open an issue for cryptography?

For pandas I see conda-forge/pandas-feedstock#148 has been inactive for quite a while. I will try to take a look.

[h-vetinari]

Could you open an issue for cryptography?

Done: conda-forge/cryptography-feedstock#123

Edit: aaaaaaand it's fixed 😁

[mattip]

PyPy3.8 is still version 7.3.11, PyPy3.9 is 7.3.12. How can I work this into the jinja template in meta.yml? This doesn't work

{% if name_suffix is not defined %}
{% set name_suffix = "3.8" %}
{% endif %}
{% set name = "pypy" ~ name_suffix %}
{% set version = "7.3.11" %}  # [name_suffix == "3.8"]
{% set version = "7.3.12" %}  # [name_suffix == "3.9"]

package:
  name: {{ name }}
  version: {{ version }}

Is there a way to tie 3.8 to 7.3.11 and 3.9 to 7.3.12 in the conda_build_config.yaml?

[h-vetinari]

Is there a way to tie 3.8 to 7.3.11 and 3.9 to 7.3.12 in the conda_build_config.yaml?

I think there's a way to make this work, but the question arises why we should keep 3.8 if upstream pypy does not support it anymore?

[mattip]

I was going to backport the fixes need for pandas to 3.8, but if we can drop 3.8 from the pandas recipe then that might work.

[h-vetinari]

I think pandas already dropped support for 3.8 following NEP29, so probably not worth it, I'd say. OTOH, if it's not a lot of work for you, we do still have pypy 3.8 being built in conda-forge, so fixes for that are certainly not a bad thing.

[mattip]

Closing. PyPy 3.8 was dropped after the last update to this feedstock, which brought it up to PyPY 7.3.12.