Skip to content

Feature/improve monitor #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kenchou merged 3 commits into from
Jun 21, 2025
Merged

Feature/improve monitor #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
942c856
improve monitor
Jun 20, 2025
9774d46
fix path
Jun 21, 2025
e667b7a
update README.md
Jun 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 85 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# file-clean-rust

Clean up (rename/delete) folders and files according to configured rules.

## Motivation

Resources downloaded through P2P networks usually contain a lot of junk files or padding files.
Some clients (such as xunlei) have automatic cleaning features, but `aria2` lacks this functionality.
Therefore, I wrote a tool to clean up directories and files.
Expand Down Expand Up @@ -36,7 +38,86 @@ example:
`file-clean-rust ~/Downloads` dry-run and see result
`file-clean-rust ~/Downloads --prune` prune the target path and see result

## Configuration
## Directory Monitoring

The `monitor-dir.sh` script provides real-time monitoring of directories for newly moved folders.
When a directory is moved into the monitored path, it automatically runs `file-clean-rust` to clean it up.

### Features

- **Cross-platform support**: Works on Linux (using `inotifywait`) and macOS (using `fswatch`)
- **Smart waiting mechanism**: Waits for directory to stabilize before processing to ensure all files are moved
- **Safe path handling**: Correctly handles filenames with spaces and special characters
- **Timeout protection**: Maximum wait time to prevent infinite waiting

### Prerequisites

**Linux/Unix systems:**

```bash
# Ubuntu/Debian
sudo apt-get install inotify-tools

# RHEL/CentOS/Fedora
sudo yum install inotify-tools
# or
sudo dnf install inotify-tools
```

**macOS:**

```bash
brew install fswatch
```

### Script Usage

```bash
# Monitor a single directory
./monitor-dir.sh /data/Downloads/TV/

# Monitor multiple directories
./monitor-dir.sh /data/Downloads/TV/ /data/Downloads/Movies/
```

### Script Configuration

The script uses the following default settings:

- **Maximum wait time**: 60 seconds
- **Stability check time**: 3 seconds (waits for 3 seconds of no file activity)

You can modify these values in the `wait_for_directory_stable()` function:

```bash
local max_wait=60 # Maximum wait time (seconds)
local stable_time=3 # Stability time (seconds)
```

### How it works

1. Monitors specified directories for `moved_to` events
2. When a directory is moved in, starts monitoring that directory for file changes
3. Waits until no file activity is detected for the stability period
4. Runs `file-clean-rust --prune` on the stabilized directory
5. Continues monitoring for new directory movements

### Example Output

```text
使用文件监控工具: fswatch
检测到目录移动事件: /data/Downloads/TV/MyShow.S01.2025/Episode.01.1080p.WEB-DL
事件类型: MOVED_TO,ISDIR
等待目录稳定: /data/Downloads/TV/MyShow.S01.2025/Episode.01.1080p.WEB-DL
检测到文件变化，继续等待...
检测到文件变化，继续等待...
目录已稳定 3 秒，开始处理
开始处理目录: /data/Downloads/TV/MyShow.S01.2025/Episode.01.1080p.WEB-DL
正在扫描文件...
[... file-clean-rust output ...]
```

## File Cleanup Configuration

The default configuration file `.cleanup-patterns.yml` is searched for starting from the specified target path,
moving upwards step by step until the root directory is reached.
Expand All @@ -60,9 +141,9 @@ remove_hash:
# To improve efficiency, this method first matches the file names # 为了提升效率，此方法先匹配文件名，
# and then calculates the hash values. # 再计算哈希值。
# Only if both match will the file be deleted. # 二者都匹配才会删除。
# The file name rules are the same as the `remove` rules # 文件名规则同 `remove` 规则，
# The file name rules are the same as the `remove` rules # 文件名规则同 `remove` 规则，
# and support wildcards and regular expressions. # 支持通配符和正则表达式。
# Note: It is not recommended to use wildcards like *.jpg, # 注：不建议使用 *.jpg 这样的通配符，
# Note: It is not recommended to use wildcards like *.jpg, # 注：不建议使用 *.jpg 这样的通配符，
# as this may result in too many files needing hash calculation. # 可能导致需要计算 hash 的文件过多。
"filename_or_wildcard":
- md5hash1
Expand All @@ -79,5 +160,6 @@ cleanup: |-
```

## Related projects

- [aria2](https://github.com/aria2/aria2)
- [aria2rpc-oversee](https://github.com/kenchou/aria2rpc-oversee)
138 changes: 133 additions & 5 deletions monitor-dir.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,136 @@ if [[ -z $1 ]]; then
fi

BIN_PATH=$(dirname $0)
inotifywait --exclude '(.tmp)' -r -me moved_to "$@" | while read dir action file; do
echo "The file '$file' appeared in directory '$dir' via '$action'"
# do something with the file
${BIN_PATH}/file-clean-rust --prune "${dir}/${file}"
done

# 检测操作系统并设置相应的文件监控工具
detect_file_watcher() {
if command -v inotifywait >/dev/null 2>&1; then
echo "inotifywait"
elif command -v fswatch >/dev/null 2>&1; then
echo "fswatch"
else
echo "error"
fi
}

# 等待目录稳定的函数
wait_for_directory_stable() {
local target_dir="$1"
local max_wait=60 # 最大等待时间（秒）
local stable_time=3 # 稳定时间（秒）
local last_change=0
local start_time=$(date +%s)

echo "等待目录稳定: $target_dir"

local watcher=$(detect_file_watcher)

if [ "$watcher" = "error" ]; then
echo "警告: 未找到文件监控工具，使用固定延迟"
sleep 5
return
fi

# 使用相应的工具监控目录变化
while true; do
current_time=$(date +%s)
elapsed=$((current_time - start_time))

# 如果超过最大等待时间，直接返回
if [ $elapsed -gt $max_wait ]; then
echo "等待超时，继续处理目录"
break
fi

local has_change=false

if [ "$watcher" = "inotifywait" ]; then
# Linux/Unix 系统使用 inotifywait
if timeout $stable_time inotifywait -qq -r -e create,moved_to,modify "$target_dir" 2>/dev/null; then
has_change=true
fi
elif [ "$watcher" = "fswatch" ]; then
# macOS 系统使用 fswatch
if timeout $stable_time fswatch -1 -r "$target_dir" >/dev/null 2>&1; then
has_change=true
fi
fi

if [ "$has_change" = true ]; then
# 有新的文件活动，重置计时器
last_change=$(date +%s)
echo "检测到文件变化，继续等待..."
else
# 超时了，说明在 stable_time 秒内没有文件变化
if [ $last_change -gt 0 ]; then
stable_duration=$((current_time - last_change))
if [ $stable_duration -ge $stable_time ]; then
echo "目录已稳定 ${stable_time} 秒，开始处理"
break
fi
else
# 首次检查就没有变化，说明目录已经稳定
echo "目录已稳定，开始处理"
break
fi
fi

sleep 0.5
done
}

# 检测并启动相应的文件监控
watcher=$(detect_file_watcher)

if [ "$watcher" = "error" ]; then
echo "错误: 需要安装 inotifywait (Linux) 或 fswatch (macOS)"
echo "macOS 安装命令: brew install fswatch"
echo "Linux 安装命令: apt-get install inotify-tools (Ubuntu/Debian) 或 yum install inotify-tools (RHEL/CentOS)"
exit 1
fi

echo "使用文件监控工具: $watcher"

if [ "$watcher" = "inotifywait" ]; then
# Linux/Unix 系统使用 inotifywait
inotifywait --exclude '(.tmp)' -r -m --format '%w%f %e' -e moved_to "$@" | while IFS= read -r line; do
echo "原始事件: $line"
# 检查是否包含 ISDIR
if [[ "$line" == *"ISDIR"* ]]; then
# 使用更安全的方式解析路径和事件
# 找到最后一个空格的位置，分离路径和事件
event_part="${line##* }"
path_part="${line% *}"

echo "检测到目录移动事件: $path_part"
echo "事件类型: $event_part"

# 等待目录稳定后再处理
wait_for_directory_stable "$path_part"

# 处理目录
echo "开始处理目录: $path_part"
"${BIN_PATH}/file-clean-rust" --prune "$path_part"
fi
done
elif [ "$watcher" = "fswatch" ]; then
# macOS 系统使用 fswatch
fswatch -r "$@" | while IFS= read -r changed_path; do
echo "检测到变化: $changed_path"

# 检查是否是新创建的目录
if [ -d "$changed_path" ]; then
# 检查目录是否是新移动过来的（简单的启发式检查）
if [ -n "$(find "$changed_path" -mindepth 1 -maxdepth 1 2>/dev/null)" ]; then
echo "检测到新目录: $changed_path"

# 等待目录稳定后再处理
wait_for_directory_stable "$changed_path"

# 处理目录
echo "开始处理目录: $changed_path"
"${BIN_PATH}/file-clean-rust" --prune "$changed_path"
fi
fi
done
fi