Merge pull request #29 from Ljzd-PRO/devel

Bump to v0.3.2

README.md

@@ -52,6 +52,7 @@
 - Customize the **structure** of downloaded post/creator **directories**
 - Search for creators/artists and posts, and **export the results**
 - **Cross-platform** support & **iOS shortcuts** available
+- For Coomer.su / Coomer.party support, check document [Coomer](https://ktoolbox.readthedocs.io/latest/coomer/) for more.
 
 ## Dev Plan
 

README_zh-CN.md

@@ -53,6 +53,7 @@
 - 可自定义下载的作品/画师 **目录结构**
 - 可搜索画师和作品，并 **导出结果**
 - 支持全平台，并提供 **iOS 快捷指令**
+- 对于 Coomer.su / Coomer.party 的支持，请查看文档 [Coomer](https://ktoolbox.readthedocs.io/latest/zh/coomer/)。
 
 ## 开发计划
 

docs/en/commands/guide.md

@@ -0,0 +1,79 @@
+# Guide
+
+Check [Reference](reference.md) for all commands and their flags / parameters.
+
+## Get general help
+
+- `--help`, `-h`
+
+```bash
+ktoolbox -h
+```
+
+## Get help of a command
+
+- `--help`, `-h`
+
+```bash
+ktoolbox download-post -h
+```
+
+## Download a specific post
+
+`download-post`
+
+```bash
+ktoolbox download-post https://kemono.su/fanbox/user/49494721/post/6608808
+```
+??? info "If some files failed to download"
+    If some files failed to download, you can try to execute the command line again, 
+    the downloaded files will be **skipped**.
+
+## Download all posts from a creator
+
+`sync-creator`
+
+```bash
+ktoolbox sync-creator https://kemono.su/fanbox/user/9016
+```
+??? info "Output"
+    By default, you will get a `creator-indices.ktoolbox` file in the creator directory, 
+    you can use it to update the directory anytime.
+
+
+## Update a downloaded creator directory
+
+`sync-creator`
+
+- `--update-with`
+
+```bash
+ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --update-with=./xxx/creator-indices.ktoolbox
+```
+??? info "About `creator-indices.ktoolbox` file"
+    The `creator-indices.ktoolbox` file contains the information and filepath of posts inside the directory.
+
+## Download posts that published within the specified time range
+
+`sync-creator`
+
+- `--start-time`
+- `--end-time`
+
+```bash
+# From 2023-8-5 to 2023-12-6
+ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --start-time=2023-8-5 --end-time=2023-12-6
+
+# From 2023-8-5 to now
+ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --start-time=2023-8-5
+
+# Before 2023-8-5
+ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --end-time=2023-8-5
+```
+
+### Time Format
+
+The time value should match `%Y-%m-%d`, for example:
+- `2023-12-7`
+- `2023-12-07`
+- `2023-12-31`

docs/en/commands.md → docs/en/commands/reference.md

@@ -1,7 +1,4 @@
-# Commands
-
-!!! info "Usage"
-    Visit [Command](index.md#command)
+# Reference
 
 ::: ktoolbox.cli.KToolBoxCli
     options:

docs/en/coomer.md

@@ -0,0 +1,23 @@
+# Coomer
+
+KToolBox support downloading from Coomer.su / Coomer.party, but currently need to **configure** before it can use.
+
+You need to set the configuration by `prod.env` dotenv file or system environment variables:
+```dotenv
+# For Coomer API
+KTOOLBOX_API__NETLOC=coomer.su
+
+# For downloading files from Coomer server
+KTOOLBOX_API__FILES_NETLOC=coomer.su
+```
+
+## About Coomer
+
+Description from [https://coomer.su](https://coomer.su) :
+
+> Coomer is a public archiver for:
+> 
+> - OnlyFans
+> - Fansly
+> 
+> Contributors here upload content and share it here for easy searching and organization. To get started viewing content, either search for creators on the creators page, or search for content on the posts page. If you want to contribute content, head over to the import page.

docs/en/index.md

@@ -68,7 +68,7 @@
 
 ### Command
 
-For more information, use the help command or goto [Commands](commands.md) page.
+For more information, use the help command or goto [Commands](commands/guide.md) page.
 
 #### ❓ Get general help
 ```bash

docs/zh/commands/guide.md

@@ -0,0 +1,77 @@
+# 向导
+
+前往 [参考](reference.md) 所有命令以及它们的标志/参数.
+
+## 获取帮助总览
+
+- `--help`, `-h`
+
+```bash
+ktoolbox -h
+```
+
+## 获取某个命令的帮助信息
+
+- `--help`, `-h`
+
+```bash
+ktoolbox download-post -h
+```
+
+## 下载指定的作品
+
+`download-post`
+
+```bash
+ktoolbox download-post https://kemono.su/fanbox/user/49494721/post/6608808
+```
+??? info "如果部分文件下载失败"
+    如果部分文件下载失败，你可以尝试重新运行命令，已下载完成的文件会被 **跳过**。
+
+## 下载作者的所有作品
+
+`sync-creator`
+
+```bash
+ktoolbox sync-creator https://kemono.su/fanbox/user/9016
+```
+??? info "输出"
+    默认情况下你会在作者目录下得到一个 `creator-indices.ktoolbox` 文件，你可以用它来更新目录。
+
+
+## 更新一个作者目录
+
+`sync-creator`
+
+- `--update-with`
+
+```bash
+ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --update-with=./xxx/creator-indices.ktoolbox
+```
+??? info "关于 `creator-indices.ktoolbox` 文件"
+    `creator-indices.ktoolbox` 包含目录下的所有作品的信息和路径。
+
+## 下载在指定时间范围内发布的作品
+
+`sync-creator`
+
+- `--start-time`
+- `--end-time`
+
+```bash
+# 从 2023-8-5 到 2023-12-6
+ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --start-time=2023-8-5 --end-time=2023-12-6
+
+# 从 2023-8-5 到 现在
+ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --start-time=2023-8-5
+
+# 2023-8-5 之前
+ktoolbox sync-creator https://kemono.su/fanbox/user/641955 --end-time=2023-8-5
+```
+
+### 时间格式
+
+时间值应当符合 `%Y-%m-%d` 格式，例如：
+- `2023-12-7`
+- `2023-12-07`
+- `2023-12-31`

docs/zh/commands/reference.md

@@ -0,0 +1,5 @@
+# 参考
+
+::: ktoolbox.cli.KToolBoxCli
+    options:
+        show_signature: false

docs/zh/coomer.md

@@ -0,0 +1,23 @@
+# Coomer
+
+KToolBox 支持从 Coomer.su / Coomer.party 下载，但目前在它能够使用之前需要进行 **配置**。
+
+你需要通过 dotenv文件 `prod.env` 或系统环境变量来设置配置：
+```dotenv
+# Coomer API
+KTOOLBOX_API__NETLOC=coomer.su
+
+# 用于从 Coomer 服务器下载文件
+KTOOLBOX_API__FILES_NETLOC=coomer.su
+```
+
+## 关于 Coomer
+
+官网 [https://coomer.su](https://coomer.su) 的介绍：
+
+> Coomer is a public archiver for:
+> 
+> - OnlyFans
+> - Fansly
+> 
+> Contributors here upload content and share it here for easy searching and organization. To get started viewing content, either search for creators on the creators page, or search for content on the posts page. If you want to contribute content, head over to the import page.

docs/zh/index.md

@@ -68,7 +68,7 @@
 
 ### 命令
 
-使用帮助命令或前往 [命令](./commands.md) 页面查看更多帮助。
+使用帮助命令或前往 [命令](commands/guide.md) 页面查看更多帮助。
 
 #### ❓ 获取帮助总览
 ```bash

ktoolbox/__init__.py

@@ -1,4 +1,4 @@
 __title__ = "KToolBox"
 # noinspection SpellCheckingInspection
 __description__ = "A useful CLI tool for downloading posts in Kemono.party / .su"
-__version__ = "0.3.1"
+__version__ = "0.3.2"

ktoolbox/action/job.py

@@ -1,19 +1,20 @@
+from datetime import datetime
 from pathlib import Path
-from typing import List, Union, Tuple
+from typing import List, Union, Optional
 
 import aiofiles
 from loguru import logger
 from pathvalidate import sanitize_filename
 
 from ktoolbox.action import ActionRet, fetch_all_creator_posts, FetchInterruptError
-from ktoolbox.action.utils import generate_post_path_name
+from ktoolbox.action.utils import generate_post_path_name, filter_posts_by_time, filter_posts_by_indices
 from ktoolbox.api.model import Post
 from ktoolbox.api.posts import get_creator_post
 from ktoolbox.configuration import config, PostStructureConfiguration
 from ktoolbox.enum import PostFileTypeEnum, DataStorageNameEnum
 from ktoolbox.job import Job, CreatorIndices
 
-__all__ = ["create_job_from_post", "filter_posts_with_indices", "create_job_from_creator"]
+__all__ = ["create_job_from_post", "create_job_from_creator"]
 
 
 async def create_job_from_post(
@@ -82,27 +83,6 @@ async def create_job_from_post(
     return jobs
 
 
-def filter_posts_with_indices(posts: List[Post], indices: CreatorIndices) -> Tuple[List[Post], CreatorIndices]:
-    """
-    Compare and filter posts by ``CreatorIndices`` data
-
-    Only keep posts that was edited after last download.
-
-    :param posts: Posts to filter
-    :param indices: ``CreatorIndices`` data to use
-    :return: A updated ``List[Post]`` and updated **new** ``CreatorIndices`` instance
-    """
-    new_list = list(
-        filter(
-            lambda x: x.id not in indices.posts or x.edited > indices.posts[x.id].edited, posts
-        )
-    )
-    new_indices = indices.model_copy(deep=True)
-    for post in new_list:
-        new_indices.posts[post.id] = post
-    return new_list, new_indices
-
-
 async def create_job_from_creator(
         service: str,
         creator_id: str,
@@ -112,7 +92,9 @@ async def create_job_from_creator(
         all_pages: bool = False,
         o: int = None,
         save_creator_indices: bool = True,
-        mix_posts: bool = None
+        mix_posts: bool = None,
+        start_time: Optional[datetime],
+        end_time: Optional[datetime]
 ) -> ActionRet[List[Job]]:
     """
     Create a list of download job from a creator
@@ -127,6 +109,8 @@ async def create_job_from_creator(
     :param save_creator_indices: Record ``CreatorIndices`` data for update posts from current creator directory
     :param mix_posts: Save all files from different posts at same path, \
      ``update_from``, ``save_creator_indices`` will be ignored if enabled
+    :param start_time: Start time of the time range
+    :param end_time: End time of the time range
     """
     mix_posts = config.job.mix_posts if mix_posts is None else mix_posts
 
@@ -145,13 +129,17 @@ async def create_job_from_creator(
             post_list = ret.data
         else:
             return ActionRet(**ret.model_dump(mode="python"))
+
+    # Filter posts by publish time
+    if start_time or end_time:
+        post_list = list(filter_posts_by_time(post_list, start_time, end_time))
     logger.info(f"Get {len(post_list)} posts, start creating jobs")
 
     # Filter posts and generate ``CreatorIndices``
     if not mix_posts:
         indices = None
         if update_from:
-            post_list, indices = filter_posts_with_indices(post_list, update_from)
+            post_list, indices = filter_posts_by_indices(post_list, update_from)
             logger.info(f"{len(post_list)} posts will be downloaded")
         elif save_creator_indices:  # It's unnecessary to create indices again when ``update_from`` was provided
             indices = CreatorIndices(