Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3.0.0 #80

Draft
wants to merge 89 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
f368abe
[skip ci] add more key features in docs and README
mansuf Jun 28, 2023
7b2109f
Merge branch 'main' into v3.0.0
mansuf Jul 17, 2023
3442c2e
Refactor base codes using ruff linter and black formatter
mansuf Jul 17, 2023
74a96d7
Fix setup is not working
mansuf Jul 17, 2023
f3fa6f4
Make missing dependencies error is closing the application
mansuf Jul 17, 2023
ee2e9b9
Refactor "format/epub.py"
mansuf Jul 27, 2023
22b84d7
Refactor "format/epub.py" (pt 2)
mansuf Jul 27, 2023
a569e8c
remove warnings in linter for unused imports
mansuf Jul 27, 2023
d56b027
fix error when upload a manga without user
issamansur Oct 18, 2023
35960b5
delete line
issamansur Oct 18, 2023
144c8b5
path placeholders: part 1
mansuf Dec 11, 2023
a799eb8
Fix typing error leading to failed to run
mansuf Dec 11, 2023
46d8f08
path placeholders: part 2
mansuf Dec 11, 2023
3e257fb
Some fixes for placeholder for single and volume formats
mansuf Dec 18, 2023
dfe0c7b
[skip ci] the path placeholders doc is in final state
mansuf Dec 18, 2023
d4e53cf
[skip ci] fix readthedocs build failed
mansuf Dec 18, 2023
22e40cf
[skip ci] fix readthedocs build failed pt. 2
mansuf Dec 18, 2023
fcad487
[skip ci] fix readthedocs build failed pt. 3
mansuf Dec 18, 2023
3698cf1
Add support for manga_title in chapter placeholder
mansuf Dec 18, 2023
b490037
cleanup
mansuf Dec 19, 2023
26fa397
Merge branch 'pr/issamansur/93' into v3.0.0
mansuf Dec 19, 2023
053d3d3
[On progress] Fix date modified is changed when there is no chapter t…
mansuf Dec 19, 2023
587670b
[On progress] Fix date modified is changed when there is no chapter t…
mansuf Dec 19, 2023
e5692fc
Fix download for any converted volume formats is not working
mansuf Dec 19, 2023
d607fa5
Fix date modified is changed when there is no chapter to download (FI…
mansuf Dec 19, 2023
8f81328
Fix epub-volume doesn't create volume cover
mansuf Dec 19, 2023
876a064
Use logger from progress bar manager instead of direct logger
mansuf Dec 19, 2023
5623ff7
change --path default value
mansuf Dec 20, 2023
3f2de15
Add option to ignore missing chapters
mansuf Dec 21, 2023
a8549b2
Add support for oauth2 private client
mansuf Jan 31, 2024
e6e9688
fix docs dependecies
mansuf Jan 31, 2024
e7dd3e4
Implement new rate limit mechanism
mansuf Feb 23, 2024
7e3339e
Fix QueueWorkerReadMarker is flooding the screen when error happened
mansuf Feb 25, 2024
c08f95c
Add download tracker database migrations and more
mansuf Mar 25, 2024
c1358aa
Fix compatibility with python 3.8 and CI failed
mansuf Mar 25, 2024
c4afba7
Fix compatibility with python 3.9 and CI failed
mansuf Mar 25, 2024
19edb78
Fix --replace is not working
mansuf Mar 25, 2024
1e404ec
Breaking change: --create-no-volume
mansuf Mar 26, 2024
bdf7afe
(Bug fix no 1) Breaking change: --create-no-volume
mansuf Mar 26, 2024
b8cc5c4
(Bug fix no 2) Breaking change: --create-no-volume
mansuf Mar 27, 2024
747f5da
Fix config is changed when writing values and no_read is set True
mansuf Mar 30, 2024
22d6162
Add ComicInfo.xml to cbz-volume and cbz-single
mansuf Apr 2, 2024
a3f0ee8
Fix duplicate download
mansuf Apr 2, 2024
80278ff
Remove duplicate code when converting chapters
mansuf Apr 2, 2024
2c9f62d
Fix headers leading to error when requesting to uploads.mangadex.org
mansuf Apr 2, 2024
0d0e724
Fix cbz format error
mansuf Apr 2, 2024
7d4bf03
Fix typo in utils.py
anonymous5555522222 Apr 28, 2024
5af3e31
Merge pull request #102 from anonymous5555522222/patch-1
mansuf May 12, 2024
4b5acb2
Add feature to download manga information only and fix download_chapt…
mansuf Jun 4, 2024
55c0403
Fix the app won't run because of messed up typing
mansuf Jun 4, 2024
83c6b07
Fix normal manga download is not working ffs
mansuf Jun 4, 2024
4004178
Fix normal cover manga download is not working ffs
mansuf Jun 4, 2024
a76e05d
Add <Web> metadata in ComicInfo.xml file
mansuf Jun 4, 2024
2f72a08
Fix mistype URL for cbz format
mansuf Jun 4, 2024
ae3ab36
Fix mistype URL for cbz format (part 2)
mansuf Jun 4, 2024
275d41b
Fix download manga require config to enabled
mansuf Jun 5, 2024
8dc9be3
Commit immediately after executing SQL scripts
mansuf Jun 5, 2024
d64c970
Merge remote-tracking branch 'origin/main' into v3.0.0
mansuf Jun 5, 2024
a0dda9b
[skip ci] it's debug time yeah
mansuf Jun 5, 2024
a0fe2c5
Fix MangaInfo csv mode cannot write some characters
mansuf Jun 5, 2024
4605d90
bump version 3.0.0a
mansuf Jun 6, 2024
d52ae25
Merge branch 'main' into v3.0.0
mansuf Jun 9, 2024
b737e4a
Fix volume covers behaviour
mansuf Jun 10, 2024
ffd2f24
Rename SQL migrations files start with 1
mansuf Jun 13, 2024
b369f22
Added ability to disable metadata creation in any cbz formats
mansuf Jun 14, 2024
dfa593d
Show X-Request-ID (from last successful request) to users if they enc…
mansuf Jun 14, 2024
63577e9
Added ability to set relative to end of chapter or pages with legacy …
mansuf Jun 14, 2024
82a4add
Fix convert_start_end_from_negative()
mansuf Jun 15, 2024
5ca09bf
Fix validator function for stacked progress bar order
mansuf Jun 15, 2024
6ee476c
Merge branch 'main' into v3.0.0
mansuf Jun 23, 2024
6cfc7df
Merge branch 'main' into v3.0.0
mansuf Aug 21, 2024
5a58b2b
Merge branch 'main' into v3.0.0
mansuf Aug 21, 2024
df59a97
[skip ci] Separate advanced usages
mansuf Aug 21, 2024
e7e5865
Add an option to set page size for command and search mode
mansuf Aug 22, 2024
725dd7f
Added an option to run forever
mansuf Aug 24, 2024
32203fa
Added option to change order chapter (newest or oldest)
mansuf Aug 24, 2024
61ec2d7
Fix sql 00002 migration is running indefinitely
mansuf Sep 9, 2024
bd25b3c
Added --group-nomatch-behaviour option
mansuf Sep 9, 2024
03d2593
Bump Pillow to v10.4.0
mansuf Sep 9, 2024
04e927c
Bump requests-doh to v1.0.0
mansuf Sep 9, 2024
c999712
Bum py7zr to v0.22.0
mansuf Sep 9, 2024
823d321
Bump orjson to v3.10.7
mansuf Sep 9, 2024
2c1e172
Bump lxml to v5.3.0
mansuf Sep 9, 2024
4a8bceb
[skip ci] Phase out alpha version, now entering beta version
mansuf Sep 9, 2024
bca2570
Increase limit attempt and delay for delete_file function
mansuf Sep 9, 2024
1b09fda
Fix app is throwing error when showing manga doesn't have chapters
mansuf Jan 8, 2025
6107f42
prepare to support python version 3.11 to 3.13
mansuf Jan 22, 2025
7008998
draft 1
mansuf Jan 22, 2025
fbf5afe
Feature --manga-info-only is finished
mansuf Jan 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ jobs:
runs-on: windows-latest
strategy:
matrix:
python-version: [ '3.8', '3.9', '3.10' ]
python-version: [ '3.10', '3.11', '3.12', '3.13' ]

steps:
# Make sure cancels CI when tags is released
Expand Down Expand Up @@ -206,7 +206,7 @@ jobs:
Write-Output "python_version=${PythonVersion}" | Out-File -FilePath $env:GITHUB_ENV -Append

# Build mangadex-downloader with PyInstaller
# only allow python 3.10
# only allow python 3.13

- name: Setup python (x86)
uses: actions/setup-python@v4
Expand All @@ -215,19 +215,19 @@ jobs:
architecture: x86

- name: Compile script
if: ${{ contains(env.python_version, '3.10') }}
if: ${{ contains(env.python_version, '3.13') }}
run: |
py -${{ matrix.python-version }}-64 -m PyInstaller "mangadex-dl_x64.spec" --distpath "./dist_x64"
py -${{ matrix.python-version }}-32 -m PyInstaller "mangadex-dl_x86.spec" --distpath "./dist_x86"

- name: Run compiled script
if: ${{ contains(env.python_version, '3.10') }}
if: ${{ contains(env.python_version, '3.13') }}
run: |
& ".\dist_x64\mangadex-dl_x64\mangadex-dl_x64.exe" --version
& ".\dist_x86\mangadex-dl_x86\mangadex-dl_x86.exe" --version

- name: Cleanup build
if: contains(env.python_version, '3.10')
if: contains(env.python_version, '3.13')
run: |
# x86 executable
copy "LICENSE" "dist_x86\mangadex-dl_x86"
Expand All @@ -252,21 +252,21 @@ jobs:
Compress-Archive -Path "dist_x64\mangadex-dl" -DestinationPath "mangadex-dl_x64_${{ github.ref_name }}.zip"

- name: Upload artifact (x64)
if: contains(env.python_version, '3.10')
if: contains(env.python_version, '3.13')
uses: actions/upload-artifact@v3
with:
name: mangadex-dl_x64
path: dist_x64/mangadex-dl/

- name: Upload artifact (x86)
if: contains(env.python_version, '3.10')
if: contains(env.python_version, '3.13')
uses: actions/upload-artifact@v3
with:
name: mangadex-dl_x86
path: dist_x86/mangadex-dl/

- name: Get File hash
if: contains(env.python_version, '3.10')
if: contains(env.python_version, '3.13')
run: |
$File_x86 = "mangadex-dl_x86_${{ github.ref_name }}.zip"
$File_x64 = "mangadex-dl_x64_${{ github.ref_name }}.zip"
Expand Down
18 changes: 9 additions & 9 deletions .github/workflows/build_pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ jobs:
runs-on: windows-latest
strategy:
matrix:
python-version: [ '3.8', '3.9', '3.10' ]
python-version: [ '3.10', '3.11', '3.12', '3.13' ]

steps:
- name: Clone repo
Expand Down Expand Up @@ -96,22 +96,22 @@ jobs:
Write-Output "python_version=${PythonVersion}" | Out-File -FilePath $env:GITHUB_ENV -Append

# Build mangadex-downloader with PyInstaller
# only allow python 3.10
# only allow python 3.13

- name: Compile script
if: ${{ contains(env.python_version, '3.10') }}
if: ${{ contains(env.python_version, '3.13') }}
run: |
py -${{ matrix.python-version }}-64 -m PyInstaller "mangadex-dl_x64.spec" --distpath "./dist_x64"
py -${{ matrix.python-version }}-32 -m PyInstaller "mangadex-dl_x86.spec" --distpath "./dist_x86"

- name: Run compiled script
if: ${{ contains(env.python_version, '3.10') }}
if: ${{ contains(env.python_version, '3.13') }}
run: |
& ".\dist_x64\mangadex-dl_x64\mangadex-dl_x64.exe" --version
& ".\dist_x86\mangadex-dl_x86\mangadex-dl_x86.exe" --version

- name: Cleanup build
if: contains(env.python_version, '3.10')
if: contains(env.python_version, '3.13')
run: |
# x86 executable
copy "LICENSE" "dist_x86\mangadex-dl_x86"
Expand All @@ -134,23 +134,23 @@ jobs:
Compress-Archive -Path "dist_x64\mangadex-dl" -DestinationPath "mangadex-dl_x64.zip"

- name: Upload artifact (x64)
if: contains(env.python_version, '3.10')
if: contains(env.python_version, '3.13')
uses: actions/upload-artifact@v3
with:
name: mangadex-dl_x64
path: dist_x64/mangadex-dl/

- name: Upload artifact (x86)
if: contains(env.python_version, '3.10')
if: contains(env.python_version, '3.13')
uses: actions/upload-artifact@v3
with:
name: mangadex-dl_x86
path: dist_x86/mangadex-dl/

# Only build docs in Python 3.10
# Only build docs in Python 3.13

- name: Build docs
if: contains(env.python_version, '3.10')
if: contains(env.python_version, '3.13')
run: |
cd docs
sphinx-build -M "html" "." "_build"
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,5 @@ docs/_build
*.code-workspace
bugs.txt
ideas.txt
*.db
*.db
env
7 changes: 6 additions & 1 deletion .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,14 @@ sphinx:
formats:
- pdf

# Set the OS, Python version and other tools you might need
build:
os: ubuntu-24.04
tools:
python: "3.12"

# Optionally set the version of Python and requirements required to build your docs
python:
version: "3.8"
install:
- method: pip
path: .
Expand Down
15 changes: 15 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python Debugger: Current File",
"type": "debugpy",
"request": "launch",
"program": "test5.py",
"console": "integratedTerminal"
}
]
}
13 changes: 13 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"ruff.lint.args": [
"--config=${workspaceFolder}/ruff.toml"
],
"[python]": {
"editor.formatOnSave": true,
"editor.defaultFormatter": "ms-python.black-formatter",
"editor.codeActionsOnSave": {
"source.fixAll": "explicit"
}
},
"python.formatting.provider": "none"
}
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ Make sure you provide detailed informations like:

Want to add new features ? fix a bug ? made changes to help the app better and faster ? Fork the repository and [send the Pull Request](https://github.com/mansuf/mangadex-downloader/pulls).

**NOTE:** Before sending a pull request, you have to make sure the code that you're writing are compatible with Python 3.8.
Because minimum Python version for developing this app are 3.8
**NOTE:** Before sending a pull request, you have to make sure the code that you're writing are compatible with Python 3.10.
Because minimum Python version for developing this app are 3.10
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.11-alpine
FROM python:3.13-alpine

COPY . /app
WORKDIR /app
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile.optional
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.11
FROM python:3.13

COPY . /app
WORKDIR /app
Expand All @@ -8,7 +8,7 @@ RUN apt update && apt install wget
ENV RUSTUP_HOME=/usr/local/rustup \
CARGO_HOME=/usr/local/cargo \
PATH=/usr/local/cargo/bin:$PATH \
RUST_VERSION=1.78.0
RUST_VERSION=1.84.0
RUN wget https://sh.rustup.rs -O rustup.sh
RUN chmod +x ./rustup.sh
RUN ./rustup.sh -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION
Expand Down
18 changes: 9 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@
[![pypi-release-ver](https://img.shields.io/pypi/v/mangadex-downloader?style=for-the-badge)](https://pypi.org/project/mangadex-downloader)
[![](https://dcbadge.limes.pink/api/server/NENvQ5b5Pt)](https://discord.gg/NENvQ5b5Pt)


# mangadex-downloader

[![Indonesian](https://img.shields.io/badge/Language-Indonesian-blue.svg)](https://github.com/mansuf/mangadex-downloader/blob/main/README.id.md)
Expand Down Expand Up @@ -35,16 +34,17 @@ A command-line tool to download manga from [MangaDex](https://mangadex.org/), wr
- Download manga, cover manga, chapter, or list directly from MangaDex
- Download manga or list from user library
- Find and download MangaDex URLs from MangaDex forums ([https://forums.mangadex.org/](https://forums.mangadex.org/))
- Download manga in each chapters, each volumes, or wrap all chapters into single file
- Search (with filters) and download manga
- Filter chapters with scalantion groups or users
- Manga tags, groups, and users blacklist support
- Batch download support
- Legacy MangaDex url support
- Scanlation groups filter support
- Authentication support
- Authentication (with cache) support
- Control how many chapters and pages you want to download
- Compressed images support
- HTTP / SOCKS proxy support
- DNS-over-HTTPS support
- Multi languages support
- Legacy MangaDex url support
- Save as raw images, EPUB, PDF, Comic Book Archive (.cbz or .cb7)
- Respect API rate limit

***And ability to not download oneshot chapter***

Expand Down Expand Up @@ -123,11 +123,11 @@ You can use tag ending with `-optional`
```sh
# Dockerhub
docker pull mansuf/mangadex-downloader:latest-optional
docker pull mansuf/mangadex-downloader:v2.10.3-optional
docker pull mansuf/mangadex-downloader:v3.0.0-optional

# AWS ECR (Alternative)
docker pull public.ecr.aws/mansuf/mangadex-downloader:latest-optional
docker pull public.ecr.aws/mansuf/mangadex-downloader:v2.10.3-optional
docker pull public.ecr.aws/mansuf/mangadex-downloader:v3.0.0-optional
```

**NOTE**: If you're wondering why optional tags doesn't have arm/v6 platform support.
Expand Down
33 changes: 33 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,38 @@
# Changelog

## v3.0.0

### Breaking changes

- Change `--path` behaviour to absolute path with placeholders support
- Now, if a manga that doesn't have no volume, it will get separated (chapters format) rather than being merged into single file called `No volume.cbz` (example). However if you prefer old behaviour (merge no volume chapters into single file) you can use `--create-no-volume`.
- Dropped support for Python v3.8 and v3.9

### Improvements

- Refactored code base using ruff linter and black formatter
- The app now showing `X-Request-ID` if the app encountered `Unhandled HTTP error` that can be reported to MangaDex devs
- Missing dependencies error is now closing the application (do not ignore it)
- `ComicInfo.xml` file are now generated for volumes as well
- Options `--start-page` and `--end-page` are now support negative values (relative to end of chapter)

### New features

- Added `--ignore-missing-chapters` option to ignore missing chapters
- Added support for oauth private client
- Added `--create-manga-info` option to download manga information only
- Added `--order` option to change chapters released order (newest or oldest)
- Added `--group-nomatch-behaviour` to manage `--group` filter behaviour
- Added `--no-metadata` option to disable metadata creation on any cbz formats
- Added `--page-size` option to manage maximum items fit per page in any commands
- Added `--run-forever` to run the app indefinitely until crashed or stopped manually by user

### Fix bugs

- Fix `download.db` is modified if there is not new chapters available.
- Fix `epub-volume` format doesn't adding cover art
- Fix default volume covers behaviour. (See [#105](https://github.com/mansuf/mangadex-downloader/issues/105) for more info)

## v2.10.3

### Fix bugs
Expand Down
9 changes: 9 additions & 0 deletions docs/cli_ref/cli_options.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,15 @@ Volume cover will be placed in first page in each volume files.
Download sorting method, by default it's selected to `volume`
```

````{option} --ignore-missing-chapters -imc
```{warning}
This option cannot be used with `--no-track` option
```

Ignore missing chapters.
This will prevent the application to re-download the missing chapters.
````

## Chapter page related

```{option} --start-page -sp PAGE
Expand Down
Loading
Loading