Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dropping python 2 and 3.5 and earlier? #102

Open
1 task
jquast opened this issue Nov 21, 2023 · 6 comments · May be fixed by #117
Open
1 task

Dropping python 2 and 3.5 and earlier? #102

jquast opened this issue Nov 21, 2023 · 6 comments · May be fixed by #117

Comments

@jquast
Copy link
Owner

jquast commented Nov 21, 2023

Thoughts,

image

I am a strong believer in backporting and so on, having worked in many restrictive environments, I can sympathize with the small number of the tens of thousands of wcwidth downloads performed every day by python 2.7. And, though they should be version-pinning when working with such legacy software versions, there are cases where they are not.

The reason I held onto python 2 support so long was because it wasn't too difficult, we had found simple 2/3 switches that work in the codebase and the tests have provided nearly 100% coverage.

However I think a majority of the bugs of wcswidth calculations have been resolved, and python 2.7 users can now benefit from this. And that I cannot expect any python 2.7 users will want to make use of any new API's that are discussed in open issues. And for that reason, I am a proponent of dropping support of older python releases.

@vfazio
Copy link

vfazio commented Nov 21, 2023

For informational purposes: https://devguide.python.org/versions/

@GalaxySnail
Copy link
Collaborator

The latest version of setuptools which supports python 2.7 is setuptools 44.1.1 [1], the latest version of pip which supports python 2.7 and 3.5 is pip 20.3.4 [2].

[1] https://setuptools.pypa.io/en/latest/history.html#v45-0-0
[2] https://pip.pypa.io/en/stable/news/#v21-0

If we decide to drop python 2, we can maintain a legacy branch 0.2.x for python 2, and backport bug fixes only (including unicode table updates). The question is: what can we get from dropping python 2 and 3.5 (or even 3.6, 3.7)? Typing on python 2 is difficult, but I don't think #71 is an unsolvable problem, maybe it just need some more investigation.

If we drop python 2 only (requires python>=3.5):

  • we can get rid of all of str unicode confusion
  • lru_cache and typing are available in stdlib
  • syntax for function annotations (PEP 3107): def wcwidth(wc: str, unicode_version: str = 'auto') -> int: ...
  • use python -Im instead of python -m in CI
  • the latest version of mypy which supports --python-version 3.5 is mypy 1.5 [3][4]

[3] https://mypy-lang.blogspot.com/2023/10/mypy-16-released.html
[4] python/mypy@14743a1

If we drop python<=3.5 (requires python>=3.6):

  • PEP 526: syntax for variable annotations

If we drop python<=3.6 (requires python>=3.7):

  • from __future__ import annotaions so that we can use newer syntax such as tuple[int, ...] and str | None
  • unpin virtualenv<20.22.0 and use the latest virtualenv [5]
  • we can use the latest version of mypy

[5] https://tox.wiki/en/4.11.3/faq.html#testing-end-of-life-python-versions

IMO currently it seems not worth to maintain 2 different branches. We can reconsider this if #71 is proven unsolvable unless dropping python 2, or we encounter more problems in the future.

@jquast
Copy link
Owner Author

jquast commented Nov 22, 2023

I agree with everything you say. I don't have much inclination to do the work to remove python 2 support myself, I would only be inclined to remove it when I am unable to reasonably test it or continue to support it.

On 'What do we get?',

there is little to gain, only to drop the maintenance burden for tasks like #71. It is minor burden for the changes I expect to make, even in the ~30 hours of work I recently dedicated, only ~1 hours work was required to ensure python 2 compatibility.

We have previously discussed the idea of Cython or C-compiled assistance for performance (I will open an issue shortly), and, continuing to support python 2 and 3.5, and, the operating systems and build tools that would accompany it, might be too burdensome to maintain at that time, but I would only worry about it then.

And for typing on the whole, #71, I am in no hurry to drop support for ~1 million downloads/month of "users", for the ~10 developer who might benefit from types, and even then I believe there is little to gain there as this library API is basic, obvious and the types are in the docstrings, it's hardly worth the attention!

On 'it seems not worth to maintain 2 different branches'

I propose to drop python 2/mark a release as "final python 2 release". With #91 and #97 now released for python 2.7, and, 'ucs-detect' tool and results mostly complete, and over time with open number of issues low, I can say with high confidence that wcwidth seems to be >99% accurate for all versions of unicode up to 15.0

And, as for Unicode updates, version 15 is able to accurately write a great majority of the world's languages. I expect only very minority and ancient languages, symbols/emojis, and rare punctuation additions in future releases.

And, that in the case of supporting legacy software that is stuck on python 2.7, I see little need to continue to support python 2.7 for Unicode versions 16+. They would not likely also be using the latest 2023 releases of terminal emulators, and also be in need of viewing characters exclusive to future releases like Unicode 16.

@vfazio
Copy link

vfazio commented Nov 22, 2023

The pypi stats don't give great insights on what python versions are pulling which versions AFAICT, so I ran a quick query on the public pypi dataset for the past 30 days

SELECT REGEXP_EXTRACT(details.python, r"[0-9]+\.[0-9]+") AS python_version, file.version, COUNT(*) AS num_downloads FROM `bigquery-public-data.pypi.file_downloads` WHERE file.project = 'wcwidth' AND 
  DATE(timestamp) BETWEEN DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY) AND CURRENT_DATE()
GROUP BY `python_version`, file.version
ORDER BY `python_version`, file.version
python_version version num_downloads
  0.0.1 24
  0.1.0 24
  0.1.1 22
  0.1.2 23
  0.1.3 21
  0.1.4 71
  0.1.5 80
  0.1.6 78
  0.1.7 34878
  0.1.8 18099
  0.1.9 12046
  0.2.0 153
  0.2.1 23
  0.2.10 563454
  0.2.11 75314
  0.2.12 80469
  0.2.2 99
  0.2.3 176
  0.2.4 2942
  0.2.5 608248
  0.2.6 2400764
  0.2.7 16711
  0.2.8 1433167
  0.2.9 1664501
2.7 0.1.4 6
2.7 0.1.5 212
2.7 0.1.6 202
2.7 0.1.7 46574
2.7 0.1.8 6904
2.7 0.1.9 19847
2.7 0.2.10 170671
2.7 0.2.11 29474
2.7 0.2.12 40155
2.7 0.2.2 53
2.7 0.2.3 71
2.7 0.2.4 426
2.7 0.2.5 77507
2.7 0.2.6 42646
2.7 0.2.7 133
2.7 0.2.8 199542
2.7 0.2.9 327602
3.1 0.1.1 2
3.1 0.1.3 36
3.1 0.1.4 1
3.1 0.1.5 4
3.1 0.1.6 2314
3.1 0.1.7 25206
3.1 0.1.8 10131
3.1 0.1.9 48818
3.1 0.2.0 138
3.1 0.2.10 2047626
3.1 0.2.11 252603
3.1 0.2.12 334320
3.1 0.2.2 132
3.1 0.2.3 203
3.1 0.2.4 823
3.1 0.2.5 330516
3.1 0.2.6 1083648
3.1 0.2.7 5479
3.1 0.2.8 2525359
3.1 0.2.9 4161309
3.11 0.1.4 5
3.11 0.1.5 1
3.11 0.1.6 3
3.11 0.1.7 20459
3.11 0.1.8 1376
3.11 0.1.9 25514
3.11 0.2.0 96
3.11 0.2.10 1112439
3.11 0.2.11 151078
3.11 0.2.12 178585
3.11 0.2.2 110
3.11 0.2.3 524
3.11 0.2.4 216
3.11 0.2.5 112013
3.11 0.2.6 418648
3.11 0.2.7 6204
3.11 0.2.8 1394119
3.11 0.2.9 2199217
3.12 0.0.1 1
3.12 0.1.0 1
3.12 0.1.1 1
3.12 0.1.2 1
3.12 0.1.3 1
3.12 0.1.4 6
3.12 0.1.5 4
3.12 0.1.6 3
3.12 0.1.7 291
3.12 0.1.8 106
3.12 0.1.9 1472
3.12 0.2.0 222
3.12 0.2.10 221023
3.12 0.2.11 32029
3.12 0.2.12 36664
3.12 0.2.2 3
3.12 0.2.3 4
3.12 0.2.4 9
3.12 0.2.5 6580
3.12 0.2.6 5668
3.12 0.2.7 46
3.12 0.2.8 204408
3.12 0.2.9 401904
3.13 0.1.8 2
3.13 0.1.9 7
3.13 0.2.10 199
3.13 0.2.11 18
3.13 0.2.12 44
3.13 0.2.5 24
3.13 0.2.6 18
3.13 0.2.8 385
3.13 0.2.9 728
3.3 0.1.7 2
3.3 0.2.10 7
3.3 0.2.11 4
3.3 0.2.12 4
3.3 0.2.5 2
3.3 0.2.6 2
3.3 0.2.8 3
3.3 0.2.9 9
3.4 0.1.6 1
3.4 0.1.7 281
3.4 0.2.10 388
3.4 0.2.11 48
3.4 0.2.12 60
3.4 0.2.5 3
3.4 0.2.6 3
3.4 0.2.8 481
3.4 0.2.9 862
3.5 0.1.5 12
3.5 0.1.6 1
3.5 0.1.7 3258
3.5 0.1.8 40
3.5 0.1.9 317
3.5 0.2.10 5887
3.5 0.2.11 828
3.5 0.2.12 1051
3.5 0.2.3 16
3.5 0.2.4 2698
3.5 0.2.5 5024
3.5 0.2.6 194
3.5 0.2.7 6
3.5 0.2.8 5840
3.5 0.2.9 9973
3.6 0.1.7 51450
3.6 0.1.8 6732
3.6 0.1.9 8665
3.6 0.2.0 84
3.6 0.2.10 684398
3.6 0.2.11 83264
3.6 0.2.12 99211
3.6 0.2.2 31
3.6 0.2.3 9
3.6 0.2.4 546
3.6 0.2.5 54253
3.6 0.2.6 37351
3.6 0.2.7 192
3.6 0.2.8 766426
3.6 0.2.9 1289951
3.7 0.1.4 1
3.7 0.1.5 54
3.7 0.1.6 116
3.7 0.1.7 181080
3.7 0.1.8 31894
3.7 0.1.9 108883
3.7 0.2.0 34
3.7 0.2.10 1587073
3.7 0.2.11 176225
3.7 0.2.12 222093
3.7 0.2.2 697
3.7 0.2.3 1601
3.7 0.2.4 14324
3.7 0.2.5 765784
3.7 0.2.6 201724
3.7 0.2.7 830
3.7 0.2.8 1877494
3.7 0.2.9 3064506
3.8 0.0.1 1
3.8 0.1.0 1
3.8 0.1.3 5
3.8 0.1.4 2
3.8 0.1.6 480
3.8 0.1.7 56617
3.8 0.1.8 120587
3.8 0.1.9 164427
3.8 0.2.0 277
3.8 0.2.10 2216282
3.8 0.2.11 259522
3.8 0.2.12 327542
3.8 0.2.2 289
3.8 0.2.3 547
3.8 0.2.4 4191
3.8 0.2.5 602905
3.8 0.2.6 429148
3.8 0.2.7 2981
3.8 0.2.8 2516766
3.8 0.2.9 4498352
3.9 0.0.1 2
3.9 0.1.4 3
3.9 0.1.5 2
3.9 0.1.6 6
3.9 0.1.7 36367
3.9 0.1.8 8682
3.9 0.1.9 49313
3.9 0.2.0 231
3.9 0.2.10 1369850
3.9 0.2.11 194189
3.9 0.2.12 226996
3.9 0.2.2 249
3.9 0.2.3 3838
3.9 0.2.4 3016
3.9 0.2.5 469177
3.9 0.2.6 1070897
3.9 0.2.7 5369
3.9 0.2.8 1658737
3.9 0.2.9 2633441

@hugovk hugovk linked a pull request Jan 29, 2024 that will close this issue
@hugovk
Copy link
Contributor

hugovk commented Jan 29, 2024

Please see #117 to drop 2.7 and 3.5.

And for typing on the whole, #71, I am in no hurry to drop support for ~1 million downloads/month of "users", for the ~10 developer who might benefit from types, and even then I believe there is little to gain there as this library API is basic, obvious and the types are in the docstrings, it's hardly worth the attention!

You can add me as number 11 who would like types :) I also maintain PrettyTable which accounts for about 10 million downloads/month of wcwidth, around 20%.

@trim21
Copy link
Contributor

trim21 commented Oct 28, 2024

And for typing on the whole, #71, I am in no hurry to drop support for ~1 million downloads/month of "users", for the ~10 developer who might benefit from types, and even then I believe there is little to gain there as this library API is basic, obvious and the types are in the docstrings, it's hardly worth the attention!

I agree with this. But the problem is, adding types doesn't need py2 support to be dropped. If api is basic and stable, we don't even need to change pyi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants