some program redesign
This commit is contained in:
masterdraco
parent
6c164193b3
commit
c924f096e7
359
README.md
359
README.md
@ -1,136 +1,243 @@
|
|||||||
<h1>Torrent Mover v8.0</h1>
|
# Torrent Mover v8.0
|
||||||
|
|
||||||
<div class="section">
|
|
||||||
<h2>Description</h2>
|
|
||||||
<p>
|
|
||||||
<strong>Torrent Mover</strong> is a Bash script designed to automate the processing of completed torrents in Transmission.
|
|
||||||
It moves or copies downloaded files from a Transmission‑reported download location to designated destination directories on your system.
|
|
||||||
This enhanced version includes robust locking, advanced error handling, parallel processing, configurable path mapping, improved archive extraction,
|
|
||||||
and optional file integrity verification.
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="section">
|
|
||||||
<h2>Features</h2>
|
|
||||||
<ul>
|
|
||||||
<li><strong>Automatic Torrent Processing:</strong> Monitors Transmission for completed torrents and processes them based on configurable seeding criteria.</li>
|
|
||||||
<li><strong>Configurable Path Mapping:</strong> Uses Transmission’s reported download path (e.g. <code>/downloads</code>) and maps it to your local file system (e.g. <code>/mnt/dsnas2</code>) via configurable settings.</li>
|
|
||||||
<li><strong>Robust Locking:</strong> Employs <code>flock</code> to ensure that only one instance of the script runs at a time.</li>
|
|
||||||
<li><strong>Advanced Error Handling & Logging:</strong> Global error handler and detailed logging (with DEBUG mode support). Optionally, logs to syslog.</li>
|
|
||||||
<li><strong>Parallel File Operations:</strong> Utilizes GNU Parallel for moving, copying, and generating checksums, enabling efficient multi-threaded processing.</li>
|
|
||||||
<li><strong>Archive Extraction:</strong> Extracts archives (RAR, ZIP, 7z) into subdirectories at the destination—preserving internal structure—while retaining the archive in the source until seeding criteria are met.</li>
|
|
||||||
<li><strong>Directory Deduplication:</strong> Prevents re‑processing the same source directory if multiple torrents reference it.</li>
|
|
||||||
<li><strong>Optional Integrity Verification:</strong> Verifies file integrity by comparing MD5 checksums after transfer.</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="section">
|
|
||||||
<h2>Requirements</h2>
|
|
||||||
<ul>
|
|
||||||
<li>Bash</li>
|
|
||||||
<li>transmission-remote</li>
|
|
||||||
<li>GNU Parallel</li>
|
|
||||||
<li>unrar, unzip, 7z</li>
|
|
||||||
<li>bc</li>
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="section">
|
|
||||||
<h2>Installation</h2>
|
|
||||||
<ol>
|
|
||||||
<li><strong>Download the Script:</strong> Save the script (e.g., <code>torrent-mover.sh</code>) to your desired location (e.g., <code>/usr/local/bin/</code>).</li>
|
|
||||||
<li><strong>Make It Executable:</strong>
|
|
||||||
<pre>chmod +x /usr/local/bin/torrent-mover.sh</pre>
|
|
||||||
</li>
|
|
||||||
<li><strong>Create/Edit the Configuration File:</strong> The script expects a configuration file at <code>/etc/torrent/mover.conf</code>. See the configuration section below.</li>
|
|
||||||
</ol>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="section">
|
|
||||||
<h2>Configuration</h2>
|
|
||||||
<p>Edit or create <code>/etc/torrent/mover.conf</code> with the following content:</p>
|
|
||||||
<pre>
|
|
||||||
# Transmission settings
|
|
||||||
TRANSMISSION_IP="192.168.1.100" # Replace with your Transmission server's IP
|
|
||||||
TRANSMISSION_PORT="9091" # Replace with your Transmission server's port
|
|
||||||
TRANSMISSION_USER="your_username" # Transmission username (if set)
|
|
||||||
TRANSMISSION_PASSWORD="your_password" # Transmission password (if set)
|
|
||||||
|
|
||||||
# Path mapping settings
|
## Description
|
||||||
TRANSMISSION_PATH_PREFIX="/downloads"
|
|
||||||
LOCAL_PATH_PREFIX="/mnt/dsnas2"
|
|
||||||
|
|
||||||
# Destination directories
|
**Torrent Mover** is a Bash script designed to automate the processing of completed torrents in Transmission.
|
||||||
DIR_GAMES_DST="/mnt/dsnas1/Games"
|
It moves or copies downloaded files from a Transmission‑reported download location to designated destination directories on your system.
|
||||||
DIR_APPS_DST="/mnt/dsnas1/Apps"
|
This enhanced version includes a modular architecture, dedicated security user, robust locking, advanced error handling with retry capabilities,
|
||||||
DIR_MOVIES_DST="/mnt/dsnas1/Movies"
|
parallel processing, configurable path mapping, improved archive extraction, and optional file integrity verification.
|
||||||
DIR_BOOKS_DST="/mnt/dsnas1/Books"
|
|
||||||
DEFAULT_DST="/mnt/dsnas1/Other"
|
|
||||||
|
|
||||||
# Additional storage directories (comma-separated list)
|
The system seamlessly organizes content into appropriate directories using smart pattern matching and customizable category detection, helping you maintain a well-structured media library with minimal manual intervention.
|
||||||
STORAGE_DIRS="/mnt/dsnas/Movies"
|
|
||||||
|
|
||||||
# Performance settings
|
## Features
|
||||||
PARALLEL_THREADS="32"
|
|
||||||
PARALLEL_PROCESSING=1
|
|
||||||
|
|
||||||
# Operation mode: "move" or "copy"
|
### Core Features
|
||||||
COPY_MODE="copy"
|
- **Automatic Torrent Processing:** Monitors Transmission for completed torrents and processes them based on configurable seeding criteria.
|
||||||
|
- **Configurable Path Mapping:** Uses Transmission's reported download path and maps it to your local file system via configurable settings.
|
||||||
|
- **Archive Extraction:** Extracts archives (RAR, ZIP, 7z) into subdirectories at the destination—preserving internal structure—while retaining the archive in the source until seeding criteria are met.
|
||||||
|
- **Directory Deduplication:** Prevents re‑processing the same source directory if multiple torrents reference it.
|
||||||
|
|
||||||
|
### Advanced Content Organization
|
||||||
|
- **Smart Content Categorization:** Uses both pattern matching and directory name detection to properly categorize content.
|
||||||
|
- **Regex Pattern Matching:** Define custom regex patterns to precisely organize content into subcategories (documentaries, anime, etc.).
|
||||||
|
- **Multi-Library Support:** Manage content across multiple storage locations with different organization schemes.
|
||||||
|
|
||||||
|
### Enhanced Security & Reliability
|
||||||
|
- **Dedicated Non-Root User:** Uses a dedicated service user with minimal permissions for enhanced security.
|
||||||
|
- **Error Recovery:** Includes retry mechanisms with configurable attempts and delay for network operations.
|
||||||
|
- **Data Integrity Protection:** Optionally verifies file integrity by comparing MD5 checksums after transfer.
|
||||||
|
- **Robust Locking:** Employs `flock` to ensure that only one instance of the script runs at a time.
|
||||||
|
|
||||||
|
### Performance & Engineering
|
||||||
|
- **Modular Architecture:** Code is organized into separate modules for better maintainability and extensibility.
|
||||||
|
- **Parallel File Operations:** Utilizes GNU Parallel for moving, copying, and generating checksums, enabling efficient multi-threaded processing.
|
||||||
|
- **Advanced Error Handling & Logging:** Global error handler and detailed logging (with DEBUG mode support). Optionally, logs to syslog.
|
||||||
|
|
||||||
|
## Requirements
|
||||||
|
|
||||||
|
- Bash
|
||||||
|
- transmission-remote
|
||||||
|
- GNU Parallel
|
||||||
|
- unrar, unzip, 7z
|
||||||
|
- bc
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
1. Run the installation script as root:
|
||||||
|
```
|
||||||
|
sudo ./install.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
2. The script will:
|
||||||
|
- Install all necessary dependencies
|
||||||
|
- Create a dedicated non-root user for security
|
||||||
|
- Set up the configuration file in `/etc/torrent/mover.conf`
|
||||||
|
- Install systemd service and timer
|
||||||
|
- Configure file permissions and log rotation
|
||||||
|
|
||||||
|
3. Enable the service to run every 15 minutes:
|
||||||
|
```
|
||||||
|
sudo systemctl enable --now torrent-mover.timer
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
Edit the configuration file at `/etc/torrent/mover.conf` to customize the behavior of Torrent Mover:
|
||||||
|
|
||||||
|
### Connection Configuration
|
||||||
|
```bash
|
||||||
|
# Transmission connection settings
|
||||||
|
TRANSMISSION_IP="192.168.1.100" # IP address of your Transmission server
|
||||||
|
TRANSMISSION_PORT="9091" # RPC port for Transmission
|
||||||
|
TRANSMISSION_USER="your_username" # Username for authentication (if enabled)
|
||||||
|
TRANSMISSION_PASSWORD="your_password" # Password for authentication (if enabled)
|
||||||
|
|
||||||
|
# Path mapping configuration
|
||||||
|
TRANSMISSION_PATH_PREFIX="/downloads" # Path prefix reported by Transmission
|
||||||
|
LOCAL_PATH_PREFIX="/mnt/dsnas2" # Corresponding local path prefix
|
||||||
|
```
|
||||||
|
|
||||||
|
### Content Organization
|
||||||
|
```bash
|
||||||
|
# Primary content destination directories
|
||||||
|
DIR_GAMES_DST="/mnt/dsnas1/Games" # Games destination
|
||||||
|
DIR_APPS_DST="/mnt/dsnas1/Apps" # Applications destination
|
||||||
|
DIR_MOVIES_DST="/mnt/dsnas1/Movies" # Movies destination
|
||||||
|
DIR_BOOKS_DST="/mnt/dsnas1/Books" # Books/eBooks destination
|
||||||
|
DIR_TV_DST="/mnt/dsnas1/TV" # TV series destination
|
||||||
|
DIR_MUSIC_DST="/mnt/dsnas1/Music" # Music destination
|
||||||
|
DEFAULT_DST="/mnt/dsnas1/Other" # Default for unrecognized content
|
||||||
|
|
||||||
|
# Additional storage libraries (comma-separated)
|
||||||
|
STORAGE_DIRS="/mnt/dsnas/Movies,/mnt/external/Movies" # Additional movie libraries
|
||||||
|
STORAGE_TV_DIRS="/mnt/dsnas/TV,/mnt/external/TV" # Additional TV libraries
|
||||||
|
|
||||||
|
# Custom pattern matching for advanced categorization
|
||||||
|
# Format: "regex_pattern=destination_path;another_pattern=another_path"
|
||||||
|
CUSTOM_PATTERNS=".*documentary.*=${DIR_MOVIES_DST}/Documentary;
|
||||||
|
.*anime.*=${DIR_TV_DST}/Anime;
|
||||||
|
.*linux.*=${DIR_APPS_DST}/Linux;
|
||||||
|
.*tutorial.*=${DIR_BOOKS_DST}/Tutorials"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Security & Performance
|
||||||
|
```bash
|
||||||
|
# Security settings - dedicated non-root user
|
||||||
|
TORRENT_USER="torrent-mover" # Dedicated service user
|
||||||
|
TORRENT_GROUP="torrent-mover" # User's primary group
|
||||||
|
|
||||||
|
# Error recovery configuration
|
||||||
|
MAX_RETRY_ATTEMPTS="3" # Maximum retry attempts for failed operations
|
||||||
|
RETRY_WAIT_TIME="15" # Seconds to wait between retry attempts
|
||||||
|
|
||||||
|
# Performance tuning
|
||||||
|
PARALLEL_THREADS="32" # Number of parallel threads (match CPU cores)
|
||||||
|
PARALLEL_PROCESSING=1 # Enable (1) or disable (0) parallel processing
|
||||||
|
|
||||||
|
# Operation mode
|
||||||
|
COPY_MODE="copy" # "copy" to preserve or "move" to relocate files
|
||||||
|
```
|
||||||
|
|
||||||
|
### Logging & Integrity
|
||||||
|
```bash
|
||||||
# File tracking & integrity
|
# File tracking & integrity
|
||||||
PROCESSED_LOG="/var/log/torrent_processed.log"
|
PROCESSED_LOG="/var/log/torrent_processed.log" # Tracks processed torrents
|
||||||
CHECKSUM_DB="/var/lib/torrent/checksums.db"
|
CHECKSUM_DB="/var/lib/torrent/checksums.db" # Stores file checksums
|
||||||
|
|
||||||
# Logging settings
|
# Logging configuration
|
||||||
LOG_FILE="/var/log/torrent_mover.log"
|
LOG_FILE="/var/log/torrent_mover.log" # Main log file location
|
||||||
LOG_LEVEL="INFO" # Set to "DEBUG" for more verbose logging
|
LOG_LEVEL="INFO" # Logging level: "INFO" or "DEBUG"
|
||||||
USE_SYSLOG="false" # Set to "true" to log messages to syslog
|
USE_SYSLOG="false" # Also log to system syslog: "true" or "false"
|
||||||
|
|
||||||
# Optional integrity verification after transfer ("true" to enable)
|
# Data integrity protection
|
||||||
CHECK_TRANSFER_INTEGRITY="true"
|
CHECK_TRANSFER_INTEGRITY="true" # Verify file integrity after transfers
|
||||||
</pre>
|
```
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="section">
|
|
||||||
<h2>Usage</h2>
|
|
||||||
<p>Run the script using the following options:</p>
|
|
||||||
<ul>
|
|
||||||
<li><strong>Dry-run mode (simulate operations):</strong>
|
|
||||||
<pre>/usr/local/bin/torrent-mover.sh --dry-run</pre>
|
|
||||||
</li>
|
|
||||||
<li><strong>Interactive mode (prompt for confirmation):</strong>
|
|
||||||
<pre>/usr/local/bin/torrent-mover.sh --interactive</pre>
|
|
||||||
</li>
|
|
||||||
<li><strong>Cache warmup mode (pre-calculate checksums):</strong>
|
|
||||||
<pre>/usr/local/bin/torrent-mover.sh --cache-warmup</pre>
|
|
||||||
</li>
|
|
||||||
<li><strong>Debug mode (verbose logging):</strong>
|
|
||||||
<pre>/usr/local/bin/torrent-mover.sh --debug</pre>
|
|
||||||
</li>
|
|
||||||
</ul>
|
|
||||||
<p>You can combine options as needed. For example:</p>
|
|
||||||
<pre>/usr/local/bin/torrent-mover.sh --dry-run --debug</pre>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="section">
|
|
||||||
<h2>How It Works</h2>
|
|
||||||
<ol>
|
|
||||||
<li><strong>Locking:</strong> Uses <code>flock</code> to ensure only one instance runs at a time.</li>
|
|
||||||
<li><strong>Path Translation:</strong> The script translates the Transmission-reported path (e.g., <code>/downloads</code>) to the local file system path (e.g., <code>/mnt/dsnas2</code>) using the configured mapping.</li>
|
|
||||||
<li><strong>Torrent Processing:</strong> Retrieves torrent info via <code>transmission-remote</code> and processes torrents that are 100% complete. It skips torrents already processed or those with duplicate source directories.</li>
|
|
||||||
<li><strong>File Verification & Deduplication:</strong> Compares file checksums between source and destination, and avoids re‑processing if a match is found.</li>
|
|
||||||
<li><strong>Archive Extraction:</strong> Extracts archives (RAR, ZIP, 7z) into subdirectories within the destination while preserving directory structure. The original archive is retained in the source until seeding criteria are met.</li>
|
|
||||||
<li><strong>Seeding Criteria:</strong> Checks seeding ratio and time. When thresholds are met, the torrent is removed from Transmission.</li>
|
|
||||||
<li><strong>Integrity Check (Optional):</strong> Optionally verifies file integrity by comparing MD5 checksums post-transfer.</li>
|
|
||||||
</ol>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
|
|
||||||
|
## Usage
|
||||||
<div class="section">
|
|
||||||
<h2>License</h2>
|
### Main Torrent Mover Script
|
||||||
<p>
|
|
||||||
This script is provided as-is without any warranty. Use it at your own risk. Contributions and improvements are welcome.
|
Run the main script using the following options:
|
||||||
</p>
|
|
||||||
</div>
|
- **Dry-run mode (simulate operations):**
|
||||||
</body>
|
```
|
||||||
|
/usr/local/bin/torrent-mover --dry-run
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Interactive mode (prompt for confirmation):**
|
||||||
|
```
|
||||||
|
/usr/local/bin/torrent-mover --interactive
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Cache warmup mode (pre-calculate checksums):**
|
||||||
|
```
|
||||||
|
/usr/local/bin/torrent-mover --cache-warmup
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Debug mode (verbose logging):**
|
||||||
|
```
|
||||||
|
/usr/local/bin/torrent-mover --debug
|
||||||
|
```
|
||||||
|
|
||||||
|
You can combine options as needed. For example:
|
||||||
|
```
|
||||||
|
/usr/local/bin/torrent-mover --dry-run --debug
|
||||||
|
```
|
||||||
|
|
||||||
|
### Configuration Management Tool
|
||||||
|
|
||||||
|
The system includes a dedicated configuration management tool that helps you safely update and manage your torrent-mover settings:
|
||||||
|
|
||||||
|
```
|
||||||
|
sudo torrent-config [OPTION]
|
||||||
|
```
|
||||||
|
|
||||||
|
Available options:
|
||||||
|
|
||||||
|
- **show** - Display the current configuration with color-coding
|
||||||
|
- **edit** - Edit the configuration in your preferred text editor (automatically creates a backup)
|
||||||
|
- **backup** - Create a timestamped backup of the current configuration
|
||||||
|
- **restore** - List and restore from available backups
|
||||||
|
- **validate** - Check the configuration for errors
|
||||||
|
- **set KEY VALUE** - Update a specific configuration value
|
||||||
|
- **get KEY** - Retrieve the current value of a configuration setting
|
||||||
|
- **default** - Show the default configuration values as a reference
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
```bash
|
||||||
|
# View current configuration
|
||||||
|
sudo torrent-config show
|
||||||
|
|
||||||
|
# Change the copy mode to 'move'
|
||||||
|
sudo torrent-config set COPY_MODE move
|
||||||
|
|
||||||
|
# Add a new pattern for documentaries
|
||||||
|
sudo torrent-config set CUSTOM_PATTERNS ".*documentary.*=${DIR_MOVIES_DST}/Documentary"
|
||||||
|
|
||||||
|
# Edit the configuration file in your preferred editor
|
||||||
|
sudo torrent-config edit
|
||||||
|
|
||||||
|
# View the value of a specific setting
|
||||||
|
sudo torrent-config get TRANSMISSION_IP
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture & Module Organization
|
||||||
|
|
||||||
|
The system uses a modular architecture for improved maintainability:
|
||||||
|
|
||||||
|
- **Main Script (`/usr/local/bin/torrent-mover`)**: Orchestrates the overall process and loads modules
|
||||||
|
- **Common Module**: Contains shared utilities, logging functions and error handling
|
||||||
|
- **File Operations Module**: Handles file transfers, checksums, and integrity verification
|
||||||
|
- **Archive Handler Module**: Specializes in extracting and managing various archive formats
|
||||||
|
- **Transmission Handler Module**: Manages all communication with the Transmission client
|
||||||
|
|
||||||
|
## How It Works
|
||||||
|
|
||||||
|
### Initialization & Configuration
|
||||||
|
1. **Module Loading:** The main script dynamically loads all modules from the `/usr/local/lib/torrent-mover` directory
|
||||||
|
2. **Configuration Processing:** Loads and validates the configuration from `/etc/torrent/mover.conf`
|
||||||
|
3. **Locking:** Uses `flock` to prevent multiple instances from running simultaneously
|
||||||
|
|
||||||
|
### Torrent Processing Workflow
|
||||||
|
1. **Torrent Discovery:** Retrieves the list of torrents from Transmission using retry-enabled API calls
|
||||||
|
2. **Smart Path Translation:** Converts Transmission-reported paths to local filesystem paths using configurable mappings
|
||||||
|
3. **Content Categorization:**
|
||||||
|
- First applies custom regex patterns from the configuration
|
||||||
|
- Falls back to keyword-based directory name detection if no patterns match
|
||||||
|
- Determines the appropriate destination directory for each content type
|
||||||
|
4. **Deduplication & Verification:**
|
||||||
|
- Tracks processed source directories to avoid redundant operations
|
||||||
|
- Generates and compares checksums between source and potential destinations
|
||||||
|
- Skips transfers if identical content is already present in any destination library
|
||||||
|
5. **File Processing:**
|
||||||
|
- Extracts archives with preservation of directory structure
|
||||||
|
- Transfers files using parallel operations when enabled
|
||||||
|
- Verifies integrity after transfer if configured
|
||||||
|
6. **Cleanup & Monitoring:**
|
||||||
|
- Checks seeding ratio and time against configured thresholds
|
||||||
|
- Removes torrents from Transmission when criteria are met
|
||||||
|
- Monitors disk usage across all configured storage directories
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
This script is provided as-is without any warranty. Use it at your own risk. Contributions and improvements are welcome.
|
||||||
@ -13,15 +13,32 @@ DIR_GAMES_DST="/mnt/dsnas1/Games"
|
|||||||
DIR_APPS_DST="/mnt/dsnas1/Apps"
|
DIR_APPS_DST="/mnt/dsnas1/Apps"
|
||||||
DIR_MOVIES_DST="/mnt/dsnas1/Movies"
|
DIR_MOVIES_DST="/mnt/dsnas1/Movies"
|
||||||
DIR_BOOKS_DST="/mnt/dsnas1/Books"
|
DIR_BOOKS_DST="/mnt/dsnas1/Books"
|
||||||
|
DIR_TV_DST="/mnt/dsnas1/TV"
|
||||||
|
DIR_MUSIC_DST="/mnt/dsnas1/Music"
|
||||||
DEFAULT_DST="/mnt/dsnas1/Other"
|
DEFAULT_DST="/mnt/dsnas1/Other"
|
||||||
|
|
||||||
# Storage directories (comma-separated)
|
# Storage directories (comma-separated)
|
||||||
STORAGE_DIRS="/mnt/dsnas/Movies"
|
STORAGE_DIRS="/mnt/dsnas/Movies"
|
||||||
|
STORAGE_TV_DIRS="/mnt/dsnas/TV"
|
||||||
|
|
||||||
# Path mapping
|
# Path mapping
|
||||||
TRANSMISSION_PATH_PREFIX="/downloads"
|
TRANSMISSION_PATH_PREFIX="/downloads"
|
||||||
LOCAL_PATH_PREFIX="/mnt/dsnas2"
|
LOCAL_PATH_PREFIX="/mnt/dsnas2"
|
||||||
|
|
||||||
|
# Security settings
|
||||||
|
# Default user/group for torrent operations (usually debian-transmission)
|
||||||
|
TORRENT_USER="debian-transmission"
|
||||||
|
TORRENT_GROUP="debian-transmission"
|
||||||
|
|
||||||
|
# Custom pattern matching for content categorization
|
||||||
|
# Format: "pattern1=destination1;pattern2=destination2"
|
||||||
|
# Example: ".*\.linux.*=${DIR_LINUX_DST};.*documentary.*=${DIR_DOCUMENTARY_DST}"
|
||||||
|
CUSTOM_PATTERNS=".*documentary.*=${DIR_MOVIES_DST}/Documentary;.*anime.*=${DIR_TV_DST}/Anime"
|
||||||
|
|
||||||
|
# Error recovery settings
|
||||||
|
MAX_RETRY_ATTEMPTS="3"
|
||||||
|
RETRY_WAIT_TIME="15"
|
||||||
|
|
||||||
# Performance settings
|
# Performance settings
|
||||||
PARALLEL_THREADS="32" # Match CPU core count
|
PARALLEL_THREADS="32" # Match CPU core count
|
||||||
PARALLEL_PROCESSING=1
|
PARALLEL_PROCESSING=1
|
||||||
@ -46,4 +63,5 @@ USE_SYSLOG="false"
|
|||||||
# Auto-create directories
|
# Auto-create directories
|
||||||
mkdir -p "${DIR_GAMES_DST}" "${DIR_APPS_DST}" \
|
mkdir -p "${DIR_GAMES_DST}" "${DIR_APPS_DST}" \
|
||||||
"${DIR_MOVIES_DST}" "${DIR_BOOKS_DST}" \
|
"${DIR_MOVIES_DST}" "${DIR_BOOKS_DST}" \
|
||||||
"${DEFAULT_DST}" 2>/dev/null || true
|
"${DIR_TV_DST}" "${DIR_MUSIC_DST}" \
|
||||||
|
"${DEFAULT_DST}" 2>/dev/null || true
|
||||||
124
install.sh
124
install.sh
@ -3,6 +3,7 @@ set -e
|
|||||||
|
|
||||||
# Git repository configuration
|
# Git repository configuration
|
||||||
GIT_REPO="http://192.168.0.236:3000/masterdraco/torrent"
|
GIT_REPO="http://192.168.0.236:3000/masterdraco/torrent"
|
||||||
|
INSTALL_DIR="/tmp/torrent-install"
|
||||||
|
|
||||||
# Check root privileges
|
# Check root privileges
|
||||||
if [ "$EUID" -ne 0 ]; then
|
if [ "$EUID" -ne 0 ]; then
|
||||||
@ -20,6 +21,7 @@ declare -A PKGS=(
|
|||||||
[parallel]="parallel"
|
[parallel]="parallel"
|
||||||
[bc]="bc"
|
[bc]="bc"
|
||||||
[git]="git"
|
[git]="git"
|
||||||
|
[logrotate]="logrotate"
|
||||||
)
|
)
|
||||||
|
|
||||||
for pkg in "${!PKGS[@]}"; do
|
for pkg in "${!PKGS[@]}"; do
|
||||||
@ -31,22 +33,130 @@ for pkg in "${!PKGS[@]}"; do
|
|||||||
done
|
done
|
||||||
|
|
||||||
# Get files from Repo
|
# Get files from Repo
|
||||||
git pull http://192.168.0.236:3000/masterdraco/torrent.git
|
echo "Getting latest files from repository..."
|
||||||
|
if [ -d "$INSTALL_DIR" ]; then
|
||||||
|
cd "$INSTALL_DIR"
|
||||||
|
git fetch
|
||||||
|
git reset --hard origin/main
|
||||||
|
else
|
||||||
|
mkdir -p "$INSTALL_DIR"
|
||||||
|
git clone "$GIT_REPO" "$INSTALL_DIR"
|
||||||
|
cd "$INSTALL_DIR"
|
||||||
|
fi
|
||||||
|
|
||||||
# Create directory structure
|
# Create directory structure
|
||||||
echo "Creating directory structure..."
|
echo "Creating directory structure..."
|
||||||
mkdir -p /etc/torrent
|
mkdir -p /etc/torrent
|
||||||
mkdir -p /usr/local/bin
|
mkdir -p /usr/local/bin
|
||||||
|
mkdir -p /usr/local/lib/torrent-mover
|
||||||
|
mkdir -p /var/lib/torrent
|
||||||
|
mkdir -p /var/log/torrent
|
||||||
|
mkdir -p /etc/systemd/system
|
||||||
|
|
||||||
|
# Create dedicated user for security
|
||||||
|
TORRENT_USER="torrent-mover"
|
||||||
|
TORRENT_GROUP="torrent-mover"
|
||||||
|
|
||||||
|
# Check if user exists and create if not
|
||||||
|
if ! id "$TORRENT_USER" &>/dev/null; then
|
||||||
|
echo "Creating dedicated $TORRENT_USER user for security..."
|
||||||
|
useradd -r -s /bin/false "$TORRENT_USER"
|
||||||
|
fi
|
||||||
|
|
||||||
# Install files
|
# Install files
|
||||||
echo "Installing files..."
|
echo "Installing files..."
|
||||||
cp -v etc/torrent/mover.conf /etc/torrent/
|
install -Dm644 etc/torrent/mover.conf /etc/torrent/mover.conf.new
|
||||||
cp -v usr/local/bin/torrent-mover /usr/local/bin/
|
install -Dm755 usr/local/bin/torrent-mover /usr/local/bin/torrent-mover
|
||||||
chmod +x /usr/local/bin/torrent-mover
|
install -Dm755 usr/local/bin/torrent-config /usr/local/bin/torrent-config
|
||||||
|
|
||||||
|
# Install library modules
|
||||||
|
for module in usr/local/lib/torrent-mover/*.sh; do
|
||||||
|
if [ -f "$module" ]; then
|
||||||
|
install -Dm755 "$module" "/usr/local/lib/torrent-mover/$(basename "$module")"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# Create backup directory for configuration files
|
||||||
|
mkdir -p /etc/torrent/backups
|
||||||
|
chown $TORRENT_USER:$TORRENT_GROUP /etc/torrent/backups
|
||||||
|
|
||||||
|
# If this is a first-time install, copy the default config
|
||||||
|
if [ ! -f "/etc/torrent/mover.conf" ]; then
|
||||||
|
mv /etc/torrent/mover.conf.new /etc/torrent/mover.conf
|
||||||
|
else
|
||||||
|
echo "Existing configuration found at /etc/torrent/mover.conf"
|
||||||
|
echo "New configuration is at /etc/torrent/mover.conf.new"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Create log rotation configuration
|
||||||
|
cat > /etc/logrotate.d/torrent-mover << EOF
|
||||||
|
/var/log/torrent_mover.log /var/log/torrent_processed.log {
|
||||||
|
weekly
|
||||||
|
rotate 4
|
||||||
|
compress
|
||||||
|
delaycompress
|
||||||
|
missingok
|
||||||
|
notifempty
|
||||||
|
create 0640 $TORRENT_USER $TORRENT_GROUP
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Create systemd service
|
||||||
|
cat > /etc/systemd/system/torrent-mover.service << EOF
|
||||||
|
[Unit]
|
||||||
|
Description=Torrent Mover Service
|
||||||
|
After=network.target
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
ExecStart=/usr/local/bin/torrent-mover
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=60
|
||||||
|
User=$TORRENT_USER
|
||||||
|
Group=$TORRENT_GROUP
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# Create systemd timer for periodic execution
|
||||||
|
cat > /etc/systemd/system/torrent-mover.timer << EOF
|
||||||
|
[Unit]
|
||||||
|
Description=Run Torrent Mover every 15 minutes
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnBootSec=5min
|
||||||
|
OnUnitActiveSec=15min
|
||||||
|
AccuracySec=1min
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
|
EOF
|
||||||
|
|
||||||
# Set permissions
|
# Set permissions
|
||||||
echo "Setting permissions..."
|
echo "Setting permissions..."
|
||||||
chmod 600 /etc/torrent/mover.conf
|
chmod 600 /etc/torrent/mover.conf*
|
||||||
chown root:root /etc/torrent/mover.conf
|
chown root:root /etc/torrent/mover.conf*
|
||||||
|
chmod 644 /etc/systemd/system/torrent-mover.service
|
||||||
|
chmod 644 /etc/systemd/system/torrent-mover.timer
|
||||||
|
|
||||||
|
# Set permissions for data directories
|
||||||
|
chown $TORRENT_USER:$TORRENT_GROUP /var/lib/torrent
|
||||||
|
chmod 755 /var/lib/torrent
|
||||||
|
touch /var/log/torrent_mover.log /var/log/torrent_processed.log
|
||||||
|
chown $TORRENT_USER:$TORRENT_GROUP /var/log/torrent_mover.log /var/log/torrent_processed.log
|
||||||
|
chmod 640 /var/log/torrent_mover.log /var/log/torrent_processed.log
|
||||||
|
|
||||||
|
# Ensure torrent-mover user can access required directories
|
||||||
|
echo "Setting up group memberships..."
|
||||||
|
if getent group debian-transmission >/dev/null; then
|
||||||
|
usermod -a -G debian-transmission $TORRENT_USER
|
||||||
|
echo "Added $TORRENT_USER to debian-transmission group"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Reload systemd and enable timer
|
||||||
|
systemctl daemon-reload
|
||||||
|
echo "To enable automatic execution every 15 minutes, run:"
|
||||||
|
echo " systemctl enable --now torrent-mover.timer"
|
||||||
|
echo
|
||||||
|
echo "Installation complete!"
|
||||||
481
usr/local/bin/torrent-config
Executable file
481
usr/local/bin/torrent-config
Executable file
@ -0,0 +1,481 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
#
|
||||||
|
# Torrent Mover Configuration Utility
|
||||||
|
# A helper tool to safely update and manage your torrent-mover configuration
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
CONFIG_PATH="/etc/torrent/mover.conf"
|
||||||
|
BACKUP_DIR="/etc/torrent/backups"
|
||||||
|
DEFAULT_EDITOR="${EDITOR:-nano}"
|
||||||
|
|
||||||
|
# Colors for terminal output
|
||||||
|
RED='\033[0;31m'
|
||||||
|
GREEN='\033[0;32m'
|
||||||
|
YELLOW='\033[0;33m'
|
||||||
|
BLUE='\033[0;34m'
|
||||||
|
NC='\033[0m' # No Color
|
||||||
|
|
||||||
|
print_header() {
|
||||||
|
echo -e "${BLUE}╔════════════════════════════════════════════════════════════╗${NC}"
|
||||||
|
echo -e "${BLUE}║ Torrent Mover Config Utility ║${NC}"
|
||||||
|
echo -e "${BLUE}╚════════════════════════════════════════════════════════════╝${NC}"
|
||||||
|
echo ""
|
||||||
|
}
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
print_header
|
||||||
|
echo -e "Usage: ${GREEN}$(basename "$0")${NC} [OPTION]"
|
||||||
|
echo ""
|
||||||
|
echo "Options:"
|
||||||
|
echo -e " ${YELLOW}edit${NC} Edit the configuration file with your default editor"
|
||||||
|
echo -e " ${YELLOW}backup${NC} Create a backup of the current configuration"
|
||||||
|
echo -e " ${YELLOW}restore${NC} [file] Restore a previous backup (lists available backups if no file specified)"
|
||||||
|
echo -e " ${YELLOW}validate${NC} Check the configuration for errors"
|
||||||
|
echo -e " ${YELLOW}default${NC} Show the default configuration values"
|
||||||
|
echo -e " ${YELLOW}show${NC} Display the current configuration"
|
||||||
|
echo -e " ${YELLOW}set${NC} key value Update a specific configuration value"
|
||||||
|
echo -e " ${YELLOW}get${NC} key Get the value of a specific configuration key"
|
||||||
|
echo -e " ${YELLOW}help${NC} Display this help message"
|
||||||
|
echo ""
|
||||||
|
echo "Examples:"
|
||||||
|
echo " $(basename "$0") edit # Edit the configuration file"
|
||||||
|
echo " $(basename "$0") backup # Create a timestamped backup"
|
||||||
|
echo " $(basename "$0") set COPY_MODE move # Change the copy mode to 'move'"
|
||||||
|
echo " $(basename "$0") get TRANSMISSION_IP # Show the Transmission server IP"
|
||||||
|
echo ""
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if user is root or using sudo
|
||||||
|
check_permissions() {
|
||||||
|
if [ "$EUID" -ne 0 ]; then
|
||||||
|
echo -e "${RED}Error: This command requires root privileges.${NC}"
|
||||||
|
echo "Please run with sudo:"
|
||||||
|
echo -e " ${YELLOW}sudo $(basename "$0") $*${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Create a backup of the current configuration
|
||||||
|
backup_config() {
|
||||||
|
check_permissions "$@"
|
||||||
|
|
||||||
|
if [ ! -f "$CONFIG_PATH" ]; then
|
||||||
|
echo -e "${RED}Error: Configuration file not found at $CONFIG_PATH${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$BACKUP_DIR"
|
||||||
|
local timestamp=$(date +"%Y%m%d_%H%M%S")
|
||||||
|
local backup_file="$BACKUP_DIR/mover.conf.$timestamp"
|
||||||
|
|
||||||
|
cp "$CONFIG_PATH" "$backup_file"
|
||||||
|
echo -e "${GREEN}Configuration backed up to:${NC} $backup_file"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Restore a configuration from backup
|
||||||
|
restore_config() {
|
||||||
|
check_permissions "$@"
|
||||||
|
|
||||||
|
if [ ! -d "$BACKUP_DIR" ]; then
|
||||||
|
echo -e "${RED}Error: Backup directory not found at $BACKUP_DIR${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z "$1" ]; then
|
||||||
|
# List available backups
|
||||||
|
echo -e "${BLUE}Available backups:${NC}"
|
||||||
|
local count=0
|
||||||
|
for file in "$BACKUP_DIR"/mover.conf.*; do
|
||||||
|
if [ -f "$file" ]; then
|
||||||
|
count=$((count+1))
|
||||||
|
local date_part=$(basename "$file" | cut -d. -f3)
|
||||||
|
echo -e "${YELLOW}$count)${NC} $(basename "$file") ($(date -d "${date_part:0:8} ${date_part:9:2}:${date_part:11:2}:${date_part:13:2}" "+%Y-%m-%d %H:%M:%S"))"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ "$count" -eq 0 ]; then
|
||||||
|
echo -e "${YELLOW}No backups found.${NC}"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
read -p "Enter the number of the backup to restore: " selection
|
||||||
|
|
||||||
|
if ! [[ "$selection" =~ ^[0-9]+$ ]] || [ "$selection" -lt 1 ] || [ "$selection" -gt "$count" ]; then
|
||||||
|
echo -e "${RED}Error: Invalid selection.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Get the filename of the selected backup
|
||||||
|
local selected_file=$(ls -1 "$BACKUP_DIR"/mover.conf.* | sed -n "${selection}p")
|
||||||
|
else
|
||||||
|
# Use the specified backup file
|
||||||
|
local selected_file="$BACKUP_DIR/$1"
|
||||||
|
|
||||||
|
if [ ! -f "$selected_file" ]; then
|
||||||
|
echo -e "${RED}Error: Backup file not found at $selected_file${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Create a backup of the current config before restoring
|
||||||
|
backup_config
|
||||||
|
|
||||||
|
# Restore the selected backup
|
||||||
|
cp "$selected_file" "$CONFIG_PATH"
|
||||||
|
echo -e "${GREEN}Configuration restored from:${NC} $selected_file"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Edit the configuration file
|
||||||
|
edit_config() {
|
||||||
|
check_permissions "$@"
|
||||||
|
|
||||||
|
if [ ! -f "$CONFIG_PATH" ]; then
|
||||||
|
echo -e "${RED}Error: Configuration file not found at $CONFIG_PATH${NC}"
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Create a backup before editing
|
||||||
|
backup_config
|
||||||
|
|
||||||
|
# Open in the user's preferred editor
|
||||||
|
$DEFAULT_EDITOR "$CONFIG_PATH"
|
||||||
|
|
||||||
|
# Validate after editing
|
||||||
|
validate_config
|
||||||
|
}
|
||||||
|
|
||||||
|
# Validate the configuration for errors
|
||||||
|
validate_config() {
|
||||||
|
check_permissions "$@"
|
||||||
|
|
||||||
|
if [ ! -f "$CONFIG_PATH" ]; then
|
||||||
|
echo -e "${RED}Error: Configuration file not found at $CONFIG_PATH${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo -e "${BLUE}Validating configuration file...${NC}"
|
||||||
|
|
||||||
|
# Source the config file in a subshell to check for syntax errors
|
||||||
|
if ! (bash -n "$CONFIG_PATH"); then
|
||||||
|
echo -e "${RED}Error: The configuration file contains syntax errors.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Load the configuration
|
||||||
|
source "$CONFIG_PATH"
|
||||||
|
|
||||||
|
# Check mandatory settings
|
||||||
|
local required_vars=(
|
||||||
|
"TRANSMISSION_IP"
|
||||||
|
"TRANSMISSION_PORT"
|
||||||
|
"TRANSMISSION_PATH_PREFIX"
|
||||||
|
"LOCAL_PATH_PREFIX"
|
||||||
|
"DIR_MOVIES_DST"
|
||||||
|
"DIR_APPS_DST"
|
||||||
|
"DIR_GAMES_DST"
|
||||||
|
"DIR_BOOKS_DST"
|
||||||
|
"DEFAULT_DST"
|
||||||
|
"COPY_MODE"
|
||||||
|
)
|
||||||
|
|
||||||
|
local error_count=0
|
||||||
|
for var in "${required_vars[@]}"; do
|
||||||
|
if [ -z "${!var}" ]; then
|
||||||
|
echo -e "${RED}Error: Required setting '$var' is not defined.${NC}"
|
||||||
|
error_count=$((error_count+1))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# Validate COPY_MODE
|
||||||
|
if [ -n "$COPY_MODE" ] && [ "$COPY_MODE" != "copy" ] && [ "$COPY_MODE" != "move" ]; then
|
||||||
|
echo -e "${RED}Error: COPY_MODE must be 'copy' or 'move', not '$COPY_MODE'.${NC}"
|
||||||
|
error_count=$((error_count+1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Validate directory paths
|
||||||
|
local dir_vars=(
|
||||||
|
"DIR_GAMES_DST"
|
||||||
|
"DIR_APPS_DST"
|
||||||
|
"DIR_MOVIES_DST"
|
||||||
|
"DIR_BOOKS_DST"
|
||||||
|
"DIR_TV_DST"
|
||||||
|
"DIR_MUSIC_DST"
|
||||||
|
"DEFAULT_DST"
|
||||||
|
)
|
||||||
|
|
||||||
|
for var in "${dir_vars[@]}"; do
|
||||||
|
if [ -n "${!var}" ]; then
|
||||||
|
if [[ ! "${!var}" == /* ]]; then
|
||||||
|
echo -e "${RED}Error: Directory path for '$var' must be absolute (start with /).${NC}"
|
||||||
|
error_count=$((error_count+1))
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# Check if any pattern in CUSTOM_PATTERNS references undefined variables
|
||||||
|
if [ -n "$CUSTOM_PATTERNS" ]; then
|
||||||
|
IFS=';' read -ra PATTERN_ARRAY <<< "$CUSTOM_PATTERNS"
|
||||||
|
for pattern in "${PATTERN_ARRAY[@]}"; do
|
||||||
|
IFS='=' read -ra PARTS <<< "$pattern"
|
||||||
|
if [ "${#PARTS[@]}" -eq 2 ]; then
|
||||||
|
local dest="${PARTS[1]}"
|
||||||
|
if [[ "$dest" == *'${'*'}'* ]]; then
|
||||||
|
local var_name=$(echo "$dest" | sed -n 's/.*\${//;s/}.*//p')
|
||||||
|
if [ -z "${!var_name}" ]; then
|
||||||
|
echo -e "${RED}Error: Custom pattern uses undefined variable: \${$var_name}${NC}"
|
||||||
|
error_count=$((error_count+1))
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$error_count" -eq 0 ]; then
|
||||||
|
echo -e "${GREEN}Configuration validation passed. No errors found.${NC}"
|
||||||
|
else
|
||||||
|
echo -e "${RED}Configuration validation failed with $error_count error(s).${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Show the current configuration
|
||||||
|
show_config() {
|
||||||
|
if [ ! -f "$CONFIG_PATH" ]; then
|
||||||
|
echo -e "${RED}Error: Configuration file not found at $CONFIG_PATH${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
print_header
|
||||||
|
echo -e "${BLUE}Current Configuration:${NC}"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
# Load config and display it categorized
|
||||||
|
source "$CONFIG_PATH"
|
||||||
|
|
||||||
|
echo -e "${YELLOW}=== Connection Settings ===${NC}"
|
||||||
|
echo -e "TRANSMISSION_IP=${GREEN}${TRANSMISSION_IP:-<not set>}${NC}"
|
||||||
|
echo -e "TRANSMISSION_PORT=${GREEN}${TRANSMISSION_PORT:-<not set>}${NC}"
|
||||||
|
if [ -n "$TRANSMISSION_USER" ]; then
|
||||||
|
echo -e "TRANSMISSION_USER=${GREEN}${TRANSMISSION_USER}${NC}"
|
||||||
|
echo -e "TRANSMISSION_PASSWORD=${GREEN}********${NC}"
|
||||||
|
else
|
||||||
|
echo -e "TRANSMISSION_USER=${YELLOW}<not set>${NC}"
|
||||||
|
echo -e "TRANSMISSION_PASSWORD=${YELLOW}<not set>${NC}"
|
||||||
|
fi
|
||||||
|
echo -e "TRANSMISSION_PATH_PREFIX=${GREEN}${TRANSMISSION_PATH_PREFIX:-<not set>}${NC}"
|
||||||
|
echo -e "LOCAL_PATH_PREFIX=${GREEN}${LOCAL_PATH_PREFIX:-<not set>}${NC}"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo -e "${YELLOW}=== Destination Directories ===${NC}"
|
||||||
|
echo -e "DIR_GAMES_DST=${GREEN}${DIR_GAMES_DST:-<not set>}${NC}"
|
||||||
|
echo -e "DIR_APPS_DST=${GREEN}${DIR_APPS_DST:-<not set>}${NC}"
|
||||||
|
echo -e "DIR_MOVIES_DST=${GREEN}${DIR_MOVIES_DST:-<not set>}${NC}"
|
||||||
|
echo -e "DIR_BOOKS_DST=${GREEN}${DIR_BOOKS_DST:-<not set>}${NC}"
|
||||||
|
echo -e "DIR_TV_DST=${GREEN}${DIR_TV_DST:-<not set>}${NC}"
|
||||||
|
echo -e "DIR_MUSIC_DST=${GREEN}${DIR_MUSIC_DST:-<not set>}${NC}"
|
||||||
|
echo -e "DEFAULT_DST=${GREEN}${DEFAULT_DST:-<not set>}${NC}"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo -e "${YELLOW}=== Additional Storage Libraries ===${NC}"
|
||||||
|
echo -e "STORAGE_DIRS=${GREEN}${STORAGE_DIRS:-<not set>}${NC}"
|
||||||
|
echo -e "STORAGE_TV_DIRS=${GREEN}${STORAGE_TV_DIRS:-<not set>}${NC}"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo -e "${YELLOW}=== Security Settings ===${NC}"
|
||||||
|
echo -e "TORRENT_USER=${GREEN}${TORRENT_USER:-debian-transmission}${NC}"
|
||||||
|
echo -e "TORRENT_GROUP=${GREEN}${TORRENT_GROUP:-debian-transmission}${NC}"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo -e "${YELLOW}=== Performance Settings ===${NC}"
|
||||||
|
echo -e "PARALLEL_THREADS=${GREEN}${PARALLEL_THREADS:-$(nproc)}${NC}"
|
||||||
|
echo -e "PARALLEL_PROCESSING=${GREEN}${PARALLEL_PROCESSING:-1}${NC}"
|
||||||
|
echo -e "COPY_MODE=${GREEN}${COPY_MODE:-<not set>}${NC}"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo -e "${YELLOW}=== Error Recovery ===${NC}"
|
||||||
|
echo -e "MAX_RETRY_ATTEMPTS=${GREEN}${MAX_RETRY_ATTEMPTS:-3}${NC}"
|
||||||
|
echo -e "RETRY_WAIT_TIME=${GREEN}${RETRY_WAIT_TIME:-15}${NC}"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
echo -e "${YELLOW}=== Logging & Integrity ===${NC}"
|
||||||
|
echo -e "LOG_FILE=${GREEN}${LOG_FILE:-/var/log/torrent_mover.log}${NC}"
|
||||||
|
echo -e "LOG_LEVEL=${GREEN}${LOG_LEVEL:-INFO}${NC}"
|
||||||
|
echo -e "USE_SYSLOG=${GREEN}${USE_SYSLOG:-false}${NC}"
|
||||||
|
echo -e "PROCESSED_LOG=${GREEN}${PROCESSED_LOG:-/var/log/torrent_processed.log}${NC}"
|
||||||
|
echo -e "CHECKSUM_DB=${GREEN}${CHECKSUM_DB:-/var/lib/torrent/checksums.db}${NC}"
|
||||||
|
echo -e "CHECK_TRANSFER_INTEGRITY=${GREEN}${CHECK_TRANSFER_INTEGRITY:-true}${NC}"
|
||||||
|
echo ""
|
||||||
|
|
||||||
|
if [ -n "$CUSTOM_PATTERNS" ]; then
|
||||||
|
echo -e "${YELLOW}=== Custom Content Patterns ===${NC}"
|
||||||
|
IFS=';' read -ra PATTERN_ARRAY <<< "$CUSTOM_PATTERNS"
|
||||||
|
for pattern in "${PATTERN_ARRAY[@]}"; do
|
||||||
|
if [ -n "$pattern" ]; then
|
||||||
|
IFS='=' read -ra PARTS <<< "$pattern"
|
||||||
|
if [ "${#PARTS[@]}" -eq 2 ]; then
|
||||||
|
local regex="${PARTS[0]}"
|
||||||
|
local dest="${PARTS[1]}"
|
||||||
|
echo -e "Pattern: ${GREEN}${regex}${NC} → ${BLUE}${dest}${NC}"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Update a specific configuration value
|
||||||
|
set_config_value() {
|
||||||
|
check_permissions "$@"
|
||||||
|
|
||||||
|
if [ -z "$1" ] || [ -z "$2" ]; then
|
||||||
|
echo -e "${RED}Error: Both key and value must be provided.${NC}"
|
||||||
|
echo "Usage: $(basename "$0") set KEY VALUE"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
local key="$1"
|
||||||
|
local value="$2"
|
||||||
|
|
||||||
|
if [ ! -f "$CONFIG_PATH" ]; then
|
||||||
|
echo -e "${RED}Error: Configuration file not found at $CONFIG_PATH${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Create a backup before modifying
|
||||||
|
backup_config
|
||||||
|
|
||||||
|
# Check if the key already exists in the config
|
||||||
|
if grep -q "^$key=" "$CONFIG_PATH"; then
|
||||||
|
# Update the existing key
|
||||||
|
sed -i "s|^$key=.*|$key=\"$value\"|" "$CONFIG_PATH"
|
||||||
|
echo -e "${GREEN}Updated configuration:${NC} $key = \"$value\""
|
||||||
|
else
|
||||||
|
# Add the new key
|
||||||
|
echo "$key=\"$value\"" >> "$CONFIG_PATH"
|
||||||
|
echo -e "${GREEN}Added new configuration:${NC} $key = \"$value\""
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Validate after updating
|
||||||
|
validate_config
|
||||||
|
}
|
||||||
|
|
||||||
|
# Get a specific configuration value
|
||||||
|
get_config_value() {
|
||||||
|
if [ -z "$1" ]; then
|
||||||
|
echo -e "${RED}Error: Key must be provided.${NC}"
|
||||||
|
echo "Usage: $(basename "$0") get KEY"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
local key="$1"
|
||||||
|
|
||||||
|
if [ ! -f "$CONFIG_PATH" ]; then
|
||||||
|
echo -e "${RED}Error: Configuration file not found at $CONFIG_PATH${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Source the config file to get the value
|
||||||
|
source "$CONFIG_PATH"
|
||||||
|
|
||||||
|
if [ -n "${!key+x}" ]; then
|
||||||
|
echo -e "${key}=${GREEN}${!key}${NC}"
|
||||||
|
else
|
||||||
|
echo -e "${RED}Error: Configuration key '$key' is not defined.${NC}"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Show default configuration options
|
||||||
|
show_default_config() {
|
||||||
|
print_header
|
||||||
|
echo -e "${BLUE}Default Configuration Values:${NC}"
|
||||||
|
echo ""
|
||||||
|
cat << EOF
|
||||||
|
# Transmission Settings
|
||||||
|
TRANSMISSION_IP="192.168.1.100"
|
||||||
|
TRANSMISSION_PORT="9091"
|
||||||
|
TRANSMISSION_USER=""
|
||||||
|
TRANSMISSION_PASSWORD=""
|
||||||
|
|
||||||
|
# Path Mapping
|
||||||
|
TRANSMISSION_PATH_PREFIX="/downloads"
|
||||||
|
LOCAL_PATH_PREFIX="/mnt/data"
|
||||||
|
|
||||||
|
# Destination Directories
|
||||||
|
DIR_GAMES_DST="/mnt/media/Games"
|
||||||
|
DIR_APPS_DST="/mnt/media/Apps"
|
||||||
|
DIR_MOVIES_DST="/mnt/media/Movies"
|
||||||
|
DIR_BOOKS_DST="/mnt/media/Books"
|
||||||
|
DIR_TV_DST="/mnt/media/TV"
|
||||||
|
DIR_MUSIC_DST="/mnt/media/Music"
|
||||||
|
DEFAULT_DST="/mnt/media/Other"
|
||||||
|
|
||||||
|
# Additional Storage
|
||||||
|
STORAGE_DIRS=""
|
||||||
|
STORAGE_TV_DIRS=""
|
||||||
|
|
||||||
|
# Security
|
||||||
|
TORRENT_USER="torrent-mover"
|
||||||
|
TORRENT_GROUP="torrent-mover"
|
||||||
|
|
||||||
|
# Error Recovery
|
||||||
|
MAX_RETRY_ATTEMPTS="3"
|
||||||
|
RETRY_WAIT_TIME="15"
|
||||||
|
|
||||||
|
# Performance
|
||||||
|
PARALLEL_THREADS="$(nproc)"
|
||||||
|
PARALLEL_PROCESSING="1"
|
||||||
|
COPY_MODE="copy"
|
||||||
|
|
||||||
|
# Logging & Integrity
|
||||||
|
LOG_FILE="/var/log/torrent_mover.log"
|
||||||
|
LOG_LEVEL="INFO"
|
||||||
|
USE_SYSLOG="false"
|
||||||
|
PROCESSED_LOG="/var/log/torrent_processed.log"
|
||||||
|
CHECKSUM_DB="/var/lib/torrent/checksums.db"
|
||||||
|
CHECK_TRANSFER_INTEGRITY="true"
|
||||||
|
|
||||||
|
# Custom Content Patterns
|
||||||
|
CUSTOM_PATTERNS=".*documentary.*=\${DIR_MOVIES_DST}/Documentary;.*anime.*=\${DIR_TV_DST}/Anime"
|
||||||
|
EOF
|
||||||
|
echo ""
|
||||||
|
}
|
||||||
|
|
||||||
|
# Main command processing
|
||||||
|
case "$1" in
|
||||||
|
edit)
|
||||||
|
edit_config "${@:2}"
|
||||||
|
;;
|
||||||
|
backup)
|
||||||
|
backup_config "${@:2}"
|
||||||
|
;;
|
||||||
|
restore)
|
||||||
|
restore_config "${@:2}"
|
||||||
|
;;
|
||||||
|
validate)
|
||||||
|
validate_config "${@:2}"
|
||||||
|
;;
|
||||||
|
show)
|
||||||
|
show_config
|
||||||
|
;;
|
||||||
|
set)
|
||||||
|
set_config_value "${@:2}"
|
||||||
|
;;
|
||||||
|
get)
|
||||||
|
get_config_value "${@:2}"
|
||||||
|
;;
|
||||||
|
default)
|
||||||
|
show_default_config
|
||||||
|
;;
|
||||||
|
help|--help|-h)
|
||||||
|
usage
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
usage
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
exit 0
|
||||||
@ -1,14 +1,16 @@
|
|||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# Torrent Mover v8.0 - Enhanced & Robust Version with Directory Deduplication,
|
# Torrent Mover v8.0 - Enhanced & Robust Version with modular architecture,
|
||||||
# Improved Archive Handling (keeping archives until ratio limits are reached)
|
# improved error handling, security, and content categorization
|
||||||
#
|
#
|
||||||
# This script processes completed torrents reported by Transmission,
|
# This script processes completed torrents reported by Transmission,
|
||||||
# moving or copying files to designated destination directories.
|
# moving or copying files to designated destination directories.
|
||||||
# It includes robust locking, advanced error handling & notifications,
|
# It includes robust locking, advanced error handling & notifications,
|
||||||
# improved logging, optional post-transfer integrity checks, configurable path mapping,
|
# improved logging, optional post-transfer integrity checks, configurable path mapping,
|
||||||
# and improved archive extraction that preserves directory structure.
|
# and improved archive extraction that preserves directory structure.
|
||||||
#
|
|
||||||
# Future improvements might include using Transmission’s RPC API.
|
# Set script location for importing modules
|
||||||
|
SCRIPT_DIR="$(dirname "$(readlink -f "$0")")"
|
||||||
|
LIB_DIR="/usr/local/lib/torrent-mover"
|
||||||
|
|
||||||
##############################
|
##############################
|
||||||
# Robust Locking with flock #
|
# Robust Locking with flock #
|
||||||
@ -17,52 +19,6 @@ LOCK_FILE="/var/lock/torrent-mover.lock"
|
|||||||
exec 200>"${LOCK_FILE}" || { echo "Cannot open lock file" >&2; exit 1; }
|
exec 200>"${LOCK_FILE}" || { echo "Cannot open lock file" >&2; exit 1; }
|
||||||
flock -n 200 || { echo "Another instance is running." >&2; exit 1; }
|
flock -n 200 || { echo "Another instance is running." >&2; exit 1; }
|
||||||
|
|
||||||
##############################
|
|
||||||
# Global Runtime Variables #
|
|
||||||
##############################
|
|
||||||
DRY_RUN=0
|
|
||||||
INTERACTIVE=0
|
|
||||||
CACHE_WARMUP=0
|
|
||||||
DEBUG=0 # Set to 1 if LOG_LEVEL is DEBUG or --debug is passed
|
|
||||||
|
|
||||||
# To avoid reprocessing the same source directory (across different torrents)
|
|
||||||
declare -A processed_source_dirs
|
|
||||||
|
|
||||||
####################
|
|
||||||
# Logging Functions#
|
|
||||||
####################
|
|
||||||
# All log messages go to stderr.
|
|
||||||
log_debug() {
|
|
||||||
if [[ "${DEBUG}" -eq 1 ]]; then
|
|
||||||
echo -e "[DEBUG] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
|
||||||
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[DEBUG] $*"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
log_info() {
|
|
||||||
echo -e "[INFO] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
|
||||||
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[INFO] $*"
|
|
||||||
}
|
|
||||||
log_warn() {
|
|
||||||
echo -e "[WARN] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
|
||||||
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[WARN] $*"
|
|
||||||
}
|
|
||||||
log_error() {
|
|
||||||
echo -e "[ERROR] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
|
||||||
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[ERROR] $*"
|
|
||||||
}
|
|
||||||
|
|
||||||
#################################
|
|
||||||
# Error Handling & Notifications#
|
|
||||||
#################################
|
|
||||||
error_handler() {
|
|
||||||
local lineno="$1"
|
|
||||||
local msg="$2"
|
|
||||||
log_error "Error on line ${lineno}: ${msg}"
|
|
||||||
# Optionally send a notification (e.g., email)
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
trap 'error_handler ${LINENO} "$BASH_COMMAND"' ERR
|
|
||||||
|
|
||||||
##############################
|
##############################
|
||||||
# Configuration & Validation #
|
# Configuration & Validation #
|
||||||
##############################
|
##############################
|
||||||
@ -79,6 +35,20 @@ if [[ -z "${TRANSMISSION_PATH_PREFIX:-}" || -z "${LOCAL_PATH_PREFIX:-}" ]]; then
|
|||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Load modules
|
||||||
|
for module in "${LIB_DIR}"/*.sh; do
|
||||||
|
if [[ -f "$module" ]]; then
|
||||||
|
source "$module"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# Set defaults for new configuration options
|
||||||
|
TORRENT_USER="${TORRENT_USER:-debian-transmission}"
|
||||||
|
TORRENT_GROUP="${TORRENT_GROUP:-debian-transmission}"
|
||||||
|
MAX_RETRY_ATTEMPTS="${MAX_RETRY_ATTEMPTS:-3}"
|
||||||
|
RETRY_WAIT_TIME="${RETRY_WAIT_TIME:-15}"
|
||||||
|
|
||||||
|
# Enable DEBUG mode if set in config
|
||||||
if [[ "${LOG_LEVEL}" == "DEBUG" ]]; then
|
if [[ "${LOG_LEVEL}" == "DEBUG" ]]; then
|
||||||
DEBUG=1
|
DEBUG=1
|
||||||
fi
|
fi
|
||||||
@ -89,313 +59,10 @@ if [[ -n "${STORAGE_DIRS}" ]]; then
|
|||||||
IFS=',' read -ra STORAGE_DIRS_ARRAY <<< "${STORAGE_DIRS}"
|
IFS=',' read -ra STORAGE_DIRS_ARRAY <<< "${STORAGE_DIRS}"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
##############################
|
#################################
|
||||||
# Helper & Utility Functions #
|
# Error Handling & Notifications#
|
||||||
##############################
|
#################################
|
||||||
|
trap 'error_handler ${LINENO} "$BASH_COMMAND"' ERR
|
||||||
# translate_source: Converts the Transmission‑reported path into the local path.
|
|
||||||
translate_source() {
|
|
||||||
local src="$1"
|
|
||||||
echo "${src/#${TRANSMISSION_PATH_PREFIX}/${LOCAL_PATH_PREFIX}}"
|
|
||||||
}
|
|
||||||
|
|
||||||
# parse_args: Processes command‑line options.
|
|
||||||
parse_args() {
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--dry-run) DRY_RUN=1; shift ;;
|
|
||||||
--interactive) INTERACTIVE=1; shift ;;
|
|
||||||
--cache-warmup) CACHE_WARMUP=1; shift ;;
|
|
||||||
--debug) DEBUG=1; shift ;;
|
|
||||||
--help)
|
|
||||||
echo "Usage: $0 [--dry-run] [--interactive] [--cache-warmup] [--debug]" >&2
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*) echo "Invalid option: $1" >&2; exit 1 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# check_dependencies: Ensures required commands are available.
|
|
||||||
check_dependencies() {
|
|
||||||
local deps=("transmission-remote" "unrar" "unzip" "7z" "parallel" "bc")
|
|
||||||
for dep in "${deps[@]}"; do
|
|
||||||
command -v "${dep}" >/dev/null 2>&1 || { log_error "Missing dependency: ${dep}"; exit 1; }
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# check_disk_usage: Warn if disk usage is over 90%.
|
|
||||||
declare -A CHECKED_MOUNTS=()
|
|
||||||
check_disk_usage() {
|
|
||||||
local dir="$1"
|
|
||||||
[[ -z "${dir}" ]] && return
|
|
||||||
if ! df -P "${dir}" &>/dev/null; then
|
|
||||||
log_warn "Directory not found: ${dir}"
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
local mount_point
|
|
||||||
mount_point=$(df -P "${dir}" | awk 'NR==2 {print $6}')
|
|
||||||
[[ -z "${mount_point}" ]] && return
|
|
||||||
if [[ -z "${CHECKED_MOUNTS["${mount_point}"]+x}" ]]; then
|
|
||||||
local usage
|
|
||||||
usage=$(df -P "${dir}" | awk 'NR==2 {sub(/%/, "", $5); print $5}')
|
|
||||||
if (( usage >= 90 )); then
|
|
||||||
log_warn "Storage warning: ${mount_point} at ${usage}% capacity"
|
|
||||||
fi
|
|
||||||
CHECKED_MOUNTS["${mount_point}"]=1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# init_checksum_db: Initializes the checksum database.
|
|
||||||
init_checksum_db() {
|
|
||||||
mkdir -p "$(dirname "${CHECKSUM_DB}")"
|
|
||||||
touch "${CHECKSUM_DB}" || { log_error "Could not create ${CHECKSUM_DB}"; exit 1; }
|
|
||||||
chmod 600 "${CHECKSUM_DB}"
|
|
||||||
}
|
|
||||||
|
|
||||||
# record_checksums: Generates checksums for files in given directories.
|
|
||||||
record_checksums() {
|
|
||||||
log_info "Generating checksums with ${PARALLEL_THREADS:-$(nproc)} threads"
|
|
||||||
find "$@" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -print0 | \
|
|
||||||
parallel -0 -j ${PARALLEL_THREADS:-$(nproc)} md5sum | sort > "${CHECKSUM_DB}.tmp"
|
|
||||||
mv "${CHECKSUM_DB}.tmp" "${CHECKSUM_DB}"
|
|
||||||
}
|
|
||||||
|
|
||||||
# file_metadata: Returns an md5 hash for file metadata.
|
|
||||||
file_metadata() {
|
|
||||||
find "$1" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort | awk '{print $1}'
|
|
||||||
}
|
|
||||||
|
|
||||||
# files_need_processing: Checks if the source files need processing.
|
|
||||||
files_need_processing() {
|
|
||||||
local src="$1"
|
|
||||||
shift
|
|
||||||
local targets=("$@")
|
|
||||||
|
|
||||||
if [[ ! -d "${src}" ]]; then
|
|
||||||
log_warn "Source directory missing: ${src}"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
log_info "=== FILE VERIFICATION DEBUG START ==="
|
|
||||||
log_info "Source directory: ${src}"
|
|
||||||
log_info "Verification targets: ${targets[*]}"
|
|
||||||
|
|
||||||
local empty_target_found=0
|
|
||||||
for target in "${targets[@]}"; do
|
|
||||||
if [[ ! -d "${target}" ]]; then
|
|
||||||
log_info "Target missing: ${target}"
|
|
||||||
empty_target_found=1
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
|
|
||||||
local file_count
|
|
||||||
file_count=$(find "${target}" -mindepth 1 -maxdepth 1 -print | wc -l)
|
|
||||||
log_debug "File count for target ${target}: ${file_count}"
|
|
||||||
if [[ "${file_count}" -eq 0 ]]; then
|
|
||||||
log_info "Empty target directory: ${target}"
|
|
||||||
empty_target_found=1
|
|
||||||
else
|
|
||||||
log_info "Target contains ${file_count} items: ${target}"
|
|
||||||
log_info "First 5 items:"
|
|
||||||
find "${target}" -mindepth 1 -maxdepth 1 | head -n 5 | while read -r item; do
|
|
||||||
log_info " - ${item##*/}"
|
|
||||||
done
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
if [[ "${empty_target_found}" -eq 1 ]]; then
|
|
||||||
log_info "Empty target detected - processing needed"
|
|
||||||
log_info "=== FILE VERIFICATION DEBUG END ==="
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
log_info "Generating source checksums..."
|
|
||||||
local src_checksums
|
|
||||||
src_checksums=$(find "${src}" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort)
|
|
||||||
log_info "First 5 source checksums:"
|
|
||||||
echo "${src_checksums}" | head -n 5 | while read -r line; do
|
|
||||||
log_info " ${line}"
|
|
||||||
done
|
|
||||||
|
|
||||||
local match_found=0
|
|
||||||
for target in "${targets[@]}"; do
|
|
||||||
log_info "Checking against target: ${target}"
|
|
||||||
log_info "Generating target checksums..."
|
|
||||||
local target_checksums
|
|
||||||
target_checksums=$(find "${target}" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort)
|
|
||||||
log_info "First 5 target checksums:"
|
|
||||||
echo "${target_checksums}" | head -n 5 | while read -r line; do
|
|
||||||
log_info " ${line}"
|
|
||||||
done
|
|
||||||
|
|
||||||
if diff <(echo "${src_checksums}") <(echo "${target_checksums}") >/dev/null; then
|
|
||||||
log_info "Exact checksum match found in: ${target}"
|
|
||||||
match_found=1
|
|
||||||
break
|
|
||||||
else
|
|
||||||
log_info "No match in: ${target}"
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
log_info "=== FILE VERIFICATION DEBUG END ==="
|
|
||||||
[[ "${match_found}" -eq 1 ]] && return 1 || return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
# warm_cache: Pre-calculates checksums for storage directories.
|
|
||||||
warm_cache() {
|
|
||||||
log_info "Starting cache warmup for Movies..."
|
|
||||||
local targets=("${DIR_MOVIES_DST}" "${STORAGE_DIRS_ARRAY[@]}")
|
|
||||||
record_checksums "${targets[@]}"
|
|
||||||
log_info "Cache warmup completed. Checksums stored in ${CHECKSUM_DB}"
|
|
||||||
}
|
|
||||||
|
|
||||||
# is_processed: Checks if the torrent (by hash) has already been processed.
|
|
||||||
is_processed() {
|
|
||||||
grep -q "^${1}$" "${PROCESSED_LOG}" 2>/dev/null
|
|
||||||
}
|
|
||||||
|
|
||||||
# mark_processed: Records a processed torrent.
|
|
||||||
mark_processed() {
|
|
||||||
echo "${1}" >> "${PROCESSED_LOG}"
|
|
||||||
}
|
|
||||||
|
|
||||||
# get_destination: Maps a source directory to a destination directory based on keywords.
|
|
||||||
declare -A PATH_CACHE
|
|
||||||
get_destination() {
|
|
||||||
local source_path="$1"
|
|
||||||
if [[ -n "${PATH_CACHE["${source_path}"]+x}" ]]; then
|
|
||||||
echo "${PATH_CACHE["${source_path}"]}"
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
log_info "Analyzing path: ${source_path}"
|
|
||||||
local destination
|
|
||||||
case "${source_path,,}" in
|
|
||||||
*games*) destination="${DIR_GAMES_DST}";;
|
|
||||||
*apps*) destination="${DIR_APPS_DST}";;
|
|
||||||
*movies*) destination="${DIR_MOVIES_DST}";;
|
|
||||||
*books*) destination="${DIR_BOOKS_DST}";;
|
|
||||||
*) destination="${DEFAULT_DST}";;
|
|
||||||
esac
|
|
||||||
log_info "Mapped to: ${destination}"
|
|
||||||
PATH_CACHE["${source_path}"]="${destination}"
|
|
||||||
echo "${destination}"
|
|
||||||
}
|
|
||||||
|
|
||||||
######################################
|
|
||||||
# Improved Archive Extraction Handler #
|
|
||||||
######################################
|
|
||||||
# For each archive found in the source directory, create a subdirectory in the destination
|
|
||||||
# named after the archive (without its extension) and extract into that subdirectory.
|
|
||||||
# IMPORTANT: The archive is now retained in the source, so it will remain until the ratio
|
|
||||||
# limits are reached and Transmission removes the torrent data.
|
|
||||||
handle_archives() {
|
|
||||||
local src="$1" dst="$2"
|
|
||||||
find "${src}" -type f \( -iname "*.rar" -o -iname "*.zip" -o -iname "*.7z" \) | while read -r arch; do
|
|
||||||
log_info "Extracting archive: ${arch}"
|
|
||||||
local base
|
|
||||||
base=$(basename "${arch}")
|
|
||||||
local subdir="${dst}/${base%.*}"
|
|
||||||
mkdir -p "${subdir}" || { log_error "Failed to create subdirectory ${subdir}"; continue; }
|
|
||||||
case "${arch##*.}" in
|
|
||||||
rar)
|
|
||||||
unrar x -o- "${arch}" "${subdir}" || { log_error "unrar failed for ${arch}"; continue; }
|
|
||||||
;;
|
|
||||||
zip)
|
|
||||||
unzip -o "${arch}" -d "${subdir}" || { log_error "unzip failed for ${arch}"; continue; }
|
|
||||||
;;
|
|
||||||
7z)
|
|
||||||
7z x "${arch}" -o"${subdir}" || { log_error "7z extraction failed for ${arch}"; continue; }
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
log_info "Archive ${arch} retained in source until ratio limits are reached."
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# move_files: Moves files using parallel processing if enabled.
|
|
||||||
move_files() {
|
|
||||||
if (( PARALLEL_PROCESSING )); then
|
|
||||||
parallel -j ${PARALLEL_THREADS:-$(nproc)} mv {} "${1}" ::: "${2}"/*
|
|
||||||
else
|
|
||||||
mv "${2}"/* "${1}"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# copy_files: Copies files using parallel processing if enabled.
|
|
||||||
copy_files() {
|
|
||||||
if (( PARALLEL_PROCESSING )); then
|
|
||||||
parallel -j ${PARALLEL_THREADS:-$(nproc)} cp -r {} "${1}" ::: "${2}"/*
|
|
||||||
else
|
|
||||||
cp -r "${2}"/* "${1}"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# process_copy: Validates directories, then copies/moves files from source to destination.
|
|
||||||
# Optionally verifies integrity after transfer if CHECK_TRANSFER_INTEGRITY is "true".
|
|
||||||
process_copy() {
|
|
||||||
local id="$1" hash="$2" src="$3" dst="$4"
|
|
||||||
if [[ ! -d "${src}" ]]; then
|
|
||||||
log_error "Source directory missing: ${src}"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
if [[ ! -d "${dst}" ]]; then
|
|
||||||
log_info "Creating destination directory: ${dst}"
|
|
||||||
mkdir -p "${dst}" || { log_error "Failed to create directory: ${dst}"; return 1; }
|
|
||||||
chmod 775 "${dst}"
|
|
||||||
chown debian-transmission:debian-transmission "${dst}"
|
|
||||||
fi
|
|
||||||
if [[ ! -w "${dst}" ]]; then
|
|
||||||
log_error "No write permissions for: ${dst}"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
if (( DRY_RUN )); then
|
|
||||||
log_info "[DRY RUN] Would process torrent ${id}:"
|
|
||||||
log_info " - Copy files from ${src} to ${dst}"
|
|
||||||
log_info " - File count: $(find "${src}" -maxdepth 1 -type f | wc -l)"
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
handle_archives "${src}" "${dst}"
|
|
||||||
case "${COPY_MODE}" in
|
|
||||||
move)
|
|
||||||
log_info "Moving files from ${src} to ${dst}"
|
|
||||||
move_files "${dst}" "${src}"
|
|
||||||
;;
|
|
||||||
copy)
|
|
||||||
log_info "Copying files from ${src} to ${dst}"
|
|
||||||
copy_files "${dst}" "${src}"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
if [ $? -eq 0 ]; then
|
|
||||||
if [[ "${CHECK_TRANSFER_INTEGRITY}" == "true" ]]; then
|
|
||||||
log_info "Verifying integrity of transferred files..."
|
|
||||||
local src_checksum target_checksum
|
|
||||||
src_checksum=$(find "${src}" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort)
|
|
||||||
target_checksum=$(find "${dst}" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort)
|
|
||||||
if diff <(echo "${src_checksum}") <(echo "${target_checksum}") >/dev/null; then
|
|
||||||
log_info "Integrity check passed."
|
|
||||||
else
|
|
||||||
log_error "Integrity check FAILED for ${src}"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
log_info "Transfer completed successfully"
|
|
||||||
mark_processed "${hash}"
|
|
||||||
else
|
|
||||||
log_error "Transfer failed for ${src}"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
# process_removal: Removes a torrent via Transmission.
|
|
||||||
process_removal() {
|
|
||||||
local id="$1"
|
|
||||||
if (( DRY_RUN )); then
|
|
||||||
log_info "[DRY RUN] Would remove torrent ${id}"
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
transmission-remote "${TRANSMISSION_IP}:${TRANSMISSION_PORT}" \
|
|
||||||
-n "${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}" \
|
|
||||||
-t "${id}" --remove-and-delete
|
|
||||||
}
|
|
||||||
|
|
||||||
#################
|
#################
|
||||||
# Main Function #
|
# Main Function #
|
||||||
@ -411,16 +78,12 @@ main() {
|
|||||||
"${DIR_BOOKS_DST}"
|
"${DIR_BOOKS_DST}"
|
||||||
"${DEFAULT_DST}"
|
"${DEFAULT_DST}"
|
||||||
)
|
)
|
||||||
for dir in "${REQUIRED_DIRS[@]}"; do
|
|
||||||
if [[ ! -d "${dir}" ]]; then
|
# Add optional directories if defined
|
||||||
log_error "Directory missing: ${dir}"
|
[[ -n "${DIR_TV_DST}" ]] && REQUIRED_DIRS+=("${DIR_TV_DST}")
|
||||||
exit 1
|
[[ -n "${DIR_MUSIC_DST}" ]] && REQUIRED_DIRS+=("${DIR_MUSIC_DST}")
|
||||||
fi
|
|
||||||
if [[ ! -w "${dir}" ]]; then
|
validate_directories "${REQUIRED_DIRS[@]}" || exit 1
|
||||||
log_error "Write permission denied: ${dir}"
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
|
|
||||||
init_checksum_db
|
init_checksum_db
|
||||||
|
|
||||||
@ -429,14 +92,14 @@ main() {
|
|||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
log_info "Starting processing"
|
log_info "Starting processing with user: ${TORRENT_USER}"
|
||||||
declare -A warned_dirs=()
|
declare -A warned_dirs=()
|
||||||
transmission-remote "${TRANSMISSION_IP}:${TRANSMISSION_PORT}" \
|
|
||||||
-n "${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}" -l | awk 'NR>1 && $1 ~ /^[0-9]+$/ {print $1}' | while read -r id; do
|
# Get list of torrents from Transmission
|
||||||
|
get_torrents | while read -r id; do
|
||||||
local info
|
local info
|
||||||
info=$(transmission-remote "${TRANSMISSION_IP}:${TRANSMISSION_PORT}" \
|
info=$(get_torrent_info "${id}")
|
||||||
-n "${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}" -t "${id}" -i)
|
|
||||||
local hash
|
local hash
|
||||||
hash=$(grep "Hash:" <<< "${info}" | awk '{print $2}')
|
hash=$(grep "Hash:" <<< "${info}" | awk '{print $2}')
|
||||||
local ratio
|
local ratio
|
||||||
@ -470,8 +133,15 @@ main() {
|
|||||||
fi
|
fi
|
||||||
local targets=("${dst}")
|
local targets=("${dst}")
|
||||||
case "${dst}" in
|
case "${dst}" in
|
||||||
"${DIR_MOVIES_DST}") targets+=("${STORAGE_DIRS_ARRAY[@]}");;
|
"${DIR_MOVIES_DST}")
|
||||||
|
targets+=("${STORAGE_DIRS_ARRAY[@]}")
|
||||||
|
;;
|
||||||
|
"${DIR_TV_DST}")
|
||||||
|
# If there are TV storage dirs, include them
|
||||||
|
[[ -n "${STORAGE_TV_DIRS}" ]] && IFS=',' read -ra TV_DIRS <<< "${STORAGE_TV_DIRS}" && targets+=("${TV_DIRS[@]}")
|
||||||
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
if ! files_need_processing "${dir}" "${targets[@]}"; then
|
if ! files_need_processing "${dir}" "${targets[@]}"; then
|
||||||
log_info "Skipping copy - files already exist in:"
|
log_info "Skipping copy - files already exist in:"
|
||||||
for target in "${targets[@]}"; do
|
for target in "${targets[@]}"; do
|
||||||
@ -489,11 +159,13 @@ main() {
|
|||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
check_disk_usage "${DIR_GAMES_DST}"
|
# Check disk usage for all directories
|
||||||
check_disk_usage "${DIR_APPS_DST}"
|
for dir in "${REQUIRED_DIRS[@]}"; do
|
||||||
check_disk_usage "${DIR_MOVIES_DST}"
|
check_disk_usage "${dir}"
|
||||||
check_disk_usage "${DIR_BOOKS_DST}"
|
done
|
||||||
check_disk_usage "${DEFAULT_DST}"
|
for dir in "${STORAGE_DIRS_ARRAY[@]}"; do
|
||||||
|
check_disk_usage "${dir}"
|
||||||
|
done
|
||||||
}
|
}
|
||||||
|
|
||||||
######################
|
######################
|
||||||
@ -506,4 +178,4 @@ if (( INTERACTIVE )); then
|
|||||||
[[ "${choice}" =~ ^[Yy]$ ]] || exit 0
|
[[ "${choice}" =~ ^[Yy]$ ]] || exit 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
main
|
main
|
||||||
45
usr/local/lib/torrent-mover/archive_handler.sh
Normal file
45
usr/local/lib/torrent-mover/archive_handler.sh
Normal file
@ -0,0 +1,45 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Archive extraction handler for torrent-mover
|
||||||
|
|
||||||
|
# Improved Archive Extraction Handler
|
||||||
|
# For each archive found in the source directory, create a subdirectory in the destination
|
||||||
|
# named after the archive (without its extension) and extract into that subdirectory.
|
||||||
|
# The archive is retained in the source, so it will remain until the ratio
|
||||||
|
# limits are reached and Transmission removes the torrent data.
|
||||||
|
handle_archives() {
|
||||||
|
local src="$1" dst="$2"
|
||||||
|
find "${src}" -type f \( -iname "*.rar" -o -iname "*.zip" -o -iname "*.7z" \) | while read -r arch; do
|
||||||
|
log_info "Extracting archive: ${arch}"
|
||||||
|
local base
|
||||||
|
base=$(basename "${arch}")
|
||||||
|
local subdir="${dst}/${base%.*}"
|
||||||
|
mkdir -p "${subdir}" || { log_error "Failed to create subdirectory ${subdir}"; continue; }
|
||||||
|
|
||||||
|
# Apply proper permissions to the extraction directory
|
||||||
|
chmod 775 "${subdir}"
|
||||||
|
chown ${TORRENT_USER:-debian-transmission}:${TORRENT_GROUP:-debian-transmission} "${subdir}"
|
||||||
|
|
||||||
|
local extract_success=0
|
||||||
|
case "${arch##*.}" in
|
||||||
|
rar)
|
||||||
|
retry_command "unrar x -o- \"${arch}\" \"${subdir}\"" 3 10
|
||||||
|
extract_success=$?
|
||||||
|
;;
|
||||||
|
zip)
|
||||||
|
retry_command "unzip -o \"${arch}\" -d \"${subdir}\"" 3 10
|
||||||
|
extract_success=$?
|
||||||
|
;;
|
||||||
|
7z)
|
||||||
|
retry_command "7z x \"${arch}\" -o\"${subdir}\"" 3 10
|
||||||
|
extract_success=$?
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
if [ $extract_success -eq 0 ]; then
|
||||||
|
log_info "Archive ${arch} extracted successfully to ${subdir}"
|
||||||
|
log_info "Archive ${arch} retained in source until ratio limits are reached."
|
||||||
|
else
|
||||||
|
log_error "Failed to extract archive ${arch}"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
139
usr/local/lib/torrent-mover/common.sh
Normal file
139
usr/local/lib/torrent-mover/common.sh
Normal file
@ -0,0 +1,139 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Common utility functions and variables for torrent-mover
|
||||||
|
|
||||||
|
# Global Runtime Variables
|
||||||
|
DRY_RUN=0
|
||||||
|
INTERACTIVE=0
|
||||||
|
CACHE_WARMUP=0
|
||||||
|
DEBUG=0
|
||||||
|
|
||||||
|
# To avoid reprocessing the same source directory (across different torrents)
|
||||||
|
declare -A processed_source_dirs
|
||||||
|
|
||||||
|
declare -A CHECKED_MOUNTS=()
|
||||||
|
declare -A PATH_CACHE
|
||||||
|
|
||||||
|
# Logging Functions
|
||||||
|
# All log messages go to stderr.
|
||||||
|
log_debug() {
|
||||||
|
if [[ "${DEBUG}" -eq 1 ]]; then
|
||||||
|
echo -e "[DEBUG] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
||||||
|
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[DEBUG] $*"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
log_info() {
|
||||||
|
echo -e "[INFO] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
||||||
|
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[INFO] $*"
|
||||||
|
}
|
||||||
|
|
||||||
|
log_warn() {
|
||||||
|
echo -e "[WARN] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
||||||
|
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[WARN] $*"
|
||||||
|
}
|
||||||
|
|
||||||
|
log_error() {
|
||||||
|
echo -e "[ERROR] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
||||||
|
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[ERROR] $*"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Error Handling & Notifications
|
||||||
|
error_handler() {
|
||||||
|
local lineno="$1"
|
||||||
|
local msg="$2"
|
||||||
|
log_error "Error on line ${lineno}: ${msg}"
|
||||||
|
# Optionally send a notification (e.g., email)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# translate_source: Converts the Transmission‑reported path into the local path.
|
||||||
|
translate_source() {
|
||||||
|
local src="$1"
|
||||||
|
echo "${src/#${TRANSMISSION_PATH_PREFIX}/${LOCAL_PATH_PREFIX}}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# parse_args: Processes command‑line options.
|
||||||
|
parse_args() {
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--dry-run) DRY_RUN=1; shift ;;
|
||||||
|
--interactive) INTERACTIVE=1; shift ;;
|
||||||
|
--cache-warmup) CACHE_WARMUP=1; shift ;;
|
||||||
|
--debug) DEBUG=1; shift ;;
|
||||||
|
--help)
|
||||||
|
echo "Usage: $0 [--dry-run] [--interactive] [--cache-warmup] [--debug]" >&2
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*) echo "Invalid option: $1" >&2; exit 1 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# check_dependencies: Ensures required commands are available.
|
||||||
|
check_dependencies() {
|
||||||
|
local deps=("transmission-remote" "unrar" "unzip" "7z" "parallel" "bc")
|
||||||
|
for dep in "${deps[@]}"; do
|
||||||
|
command -v "${dep}" >/dev/null 2>&1 || { log_error "Missing dependency: ${dep}"; exit 1; }
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# check_disk_usage: Warn if disk usage is over 90%.
|
||||||
|
check_disk_usage() {
|
||||||
|
local dir="$1"
|
||||||
|
[[ -z "${dir}" ]] && return
|
||||||
|
if ! df -P "${dir}" &>/dev/null; then
|
||||||
|
log_warn "Directory not found: ${dir}"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
local mount_point
|
||||||
|
mount_point=$(df -P "${dir}" | awk 'NR==2 {print $6}')
|
||||||
|
[[ -z "${mount_point}" ]] && return
|
||||||
|
if [[ -z "${CHECKED_MOUNTS["${mount_point}"]+x}" ]]; then
|
||||||
|
local usage
|
||||||
|
usage=$(df -P "${dir}" | awk 'NR==2 {sub(/%/, "", $5); print $5}')
|
||||||
|
if (( usage >= 90 )); then
|
||||||
|
log_warn "Storage warning: ${mount_point} at ${usage}% capacity"
|
||||||
|
fi
|
||||||
|
CHECKED_MOUNTS["${mount_point}"]=1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# retry_command: Execute a command with retries
|
||||||
|
retry_command() {
|
||||||
|
local cmd="$1"
|
||||||
|
local max_attempts="${2:-3}" # Default to 3 attempts
|
||||||
|
local wait_time="${3:-10}" # Default to 10 seconds wait between attempts
|
||||||
|
local attempt=1
|
||||||
|
|
||||||
|
while (( attempt <= max_attempts )); do
|
||||||
|
log_debug "Attempt $attempt of $max_attempts: $cmd"
|
||||||
|
if eval "$cmd"; then
|
||||||
|
return 0
|
||||||
|
else
|
||||||
|
log_warn "Command failed (attempt $attempt): $cmd"
|
||||||
|
if (( attempt == max_attempts )); then
|
||||||
|
log_error "Maximum attempts reached for: $cmd"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
sleep "$wait_time"
|
||||||
|
(( attempt++ ))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# validate_directories: Ensure required directories exist and are writable
|
||||||
|
validate_directories() {
|
||||||
|
local directories=("$@")
|
||||||
|
for dir in "${directories[@]}"; do
|
||||||
|
if [[ ! -d "${dir}" ]]; then
|
||||||
|
log_error "Directory missing: ${dir}"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
if [[ ! -w "${dir}" ]]; then
|
||||||
|
log_error "Write permission denied: ${dir}"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
return 0
|
||||||
|
}
|
||||||
191
usr/local/lib/torrent-mover/file_operations.sh
Normal file
191
usr/local/lib/torrent-mover/file_operations.sh
Normal file
@ -0,0 +1,191 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# File operation functions for torrent-mover
|
||||||
|
|
||||||
|
# init_checksum_db: Initializes the checksum database.
|
||||||
|
init_checksum_db() {
|
||||||
|
mkdir -p "$(dirname "${CHECKSUM_DB}")"
|
||||||
|
touch "${CHECKSUM_DB}" || { log_error "Could not create ${CHECKSUM_DB}"; exit 1; }
|
||||||
|
chmod 600 "${CHECKSUM_DB}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# record_checksums: Generates checksums for files in given directories.
|
||||||
|
record_checksums() {
|
||||||
|
log_info "Generating checksums with ${PARALLEL_THREADS:-$(nproc)} threads"
|
||||||
|
find "$@" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -print0 | \
|
||||||
|
parallel -0 -j ${PARALLEL_THREADS:-$(nproc)} md5sum | sort > "${CHECKSUM_DB}.tmp"
|
||||||
|
mv "${CHECKSUM_DB}.tmp" "${CHECKSUM_DB}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# file_metadata: Returns an md5 hash for file metadata.
|
||||||
|
file_metadata() {
|
||||||
|
find "$1" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort | awk '{print $1}'
|
||||||
|
}
|
||||||
|
|
||||||
|
# files_need_processing: Checks if the source files need processing.
|
||||||
|
files_need_processing() {
|
||||||
|
local src="$1"
|
||||||
|
shift
|
||||||
|
local targets=("$@")
|
||||||
|
|
||||||
|
if [[ ! -d "${src}" ]]; then
|
||||||
|
log_warn "Source directory missing: ${src}"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_info "=== FILE VERIFICATION DEBUG START ==="
|
||||||
|
log_info "Source directory: ${src}"
|
||||||
|
log_info "Verification targets: ${targets[*]}"
|
||||||
|
|
||||||
|
local empty_target_found=0
|
||||||
|
for target in "${targets[@]}"; do
|
||||||
|
if [[ ! -d "${target}" ]]; then
|
||||||
|
log_info "Target missing: ${target}"
|
||||||
|
empty_target_found=1
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
local file_count
|
||||||
|
file_count=$(find "${target}" -mindepth 1 -maxdepth 1 -print | wc -l)
|
||||||
|
log_debug "File count for target ${target}: ${file_count}"
|
||||||
|
if [[ "${file_count}" -eq 0 ]]; then
|
||||||
|
log_info "Empty target directory: ${target}"
|
||||||
|
empty_target_found=1
|
||||||
|
else
|
||||||
|
log_info "Target contains ${file_count} items: ${target}"
|
||||||
|
log_info "First 5 items:"
|
||||||
|
find "${target}" -mindepth 1 -maxdepth 1 | head -n 5 | while read -r item; do
|
||||||
|
log_info " - ${item##*/}"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ "${empty_target_found}" -eq 1 ]]; then
|
||||||
|
log_info "Empty target detected - processing needed"
|
||||||
|
log_info "=== FILE VERIFICATION DEBUG END ==="
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_info "Generating source checksums..."
|
||||||
|
local src_checksums
|
||||||
|
src_checksums=$(find "${src}" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort)
|
||||||
|
log_info "First 5 source checksums:"
|
||||||
|
echo "${src_checksums}" | head -n 5 | while read -r line; do
|
||||||
|
log_info " ${line}"
|
||||||
|
done
|
||||||
|
|
||||||
|
local match_found=0
|
||||||
|
for target in "${targets[@]}"; do
|
||||||
|
log_info "Checking against target: ${target}"
|
||||||
|
log_info "Generating target checksums..."
|
||||||
|
local target_checksums
|
||||||
|
target_checksums=$(find "${target}" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort)
|
||||||
|
log_info "First 5 target checksums:"
|
||||||
|
echo "${target_checksums}" | head -n 5 | while read -r line; do
|
||||||
|
log_info " ${line}"
|
||||||
|
done
|
||||||
|
|
||||||
|
if diff <(echo "${src_checksums}") <(echo "${target_checksums}") >/dev/null; then
|
||||||
|
log_info "Exact checksum match found in: ${target}"
|
||||||
|
match_found=1
|
||||||
|
break
|
||||||
|
else
|
||||||
|
log_info "No match in: ${target}"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
log_info "=== FILE VERIFICATION DEBUG END ==="
|
||||||
|
[[ "${match_found}" -eq 1 ]] && return 1 || return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# warm_cache: Pre-calculates checksums for storage directories.
|
||||||
|
warm_cache() {
|
||||||
|
log_info "Starting cache warmup for Movies..."
|
||||||
|
local targets=("${DIR_MOVIES_DST}" "${STORAGE_DIRS_ARRAY[@]}")
|
||||||
|
record_checksums "${targets[@]}"
|
||||||
|
log_info "Cache warmup completed. Checksums stored in ${CHECKSUM_DB}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# is_processed: Checks if the torrent (by hash) has already been processed.
|
||||||
|
is_processed() {
|
||||||
|
grep -q "^${1}$" "${PROCESSED_LOG}" 2>/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
# mark_processed: Records a processed torrent.
|
||||||
|
mark_processed() {
|
||||||
|
echo "${1}" >> "${PROCESSED_LOG}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# move_files: Moves files using parallel processing if enabled.
|
||||||
|
move_files() {
|
||||||
|
if (( PARALLEL_PROCESSING )); then
|
||||||
|
retry_command "parallel -j ${PARALLEL_THREADS:-$(nproc)} mv {} \"${1}\" ::: \"${2}\"/*" 3 15
|
||||||
|
else
|
||||||
|
retry_command "mv \"${2}\"/* \"${1}\"" 3 15
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# copy_files: Copies files using parallel processing if enabled.
|
||||||
|
copy_files() {
|
||||||
|
if (( PARALLEL_PROCESSING )); then
|
||||||
|
retry_command "parallel -j ${PARALLEL_THREADS:-$(nproc)} cp -r {} \"${1}\" ::: \"${2}\"/*" 3 15
|
||||||
|
else
|
||||||
|
retry_command "cp -r \"${2}\"/* \"${1}\"" 3 15
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# process_copy: Validates directories, then copies/moves files from source to destination.
|
||||||
|
# Optionally verifies integrity after transfer if CHECK_TRANSFER_INTEGRITY is "true".
|
||||||
|
process_copy() {
|
||||||
|
local id="$1" hash="$2" src="$3" dst="$4"
|
||||||
|
if [[ ! -d "${src}" ]]; then
|
||||||
|
log_error "Source directory missing: ${src}"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
if [[ ! -d "${dst}" ]]; then
|
||||||
|
log_info "Creating destination directory: ${dst}"
|
||||||
|
mkdir -p "${dst}" || { log_error "Failed to create directory: ${dst}"; return 1; }
|
||||||
|
chmod 775 "${dst}"
|
||||||
|
chown ${TORRENT_USER:-debian-transmission}:${TORRENT_GROUP:-debian-transmission} "${dst}"
|
||||||
|
fi
|
||||||
|
if [[ ! -w "${dst}" ]]; then
|
||||||
|
log_error "No write permissions for: ${dst}"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
if (( DRY_RUN )); then
|
||||||
|
log_info "[DRY RUN] Would process torrent ${id}:"
|
||||||
|
log_info " - Copy files from ${src} to ${dst}"
|
||||||
|
log_info " - File count: $(find "${src}" -maxdepth 1 -type f | wc -l)"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
handle_archives "${src}" "${dst}"
|
||||||
|
case "${COPY_MODE}" in
|
||||||
|
move)
|
||||||
|
log_info "Moving files from ${src} to ${dst}"
|
||||||
|
move_files "${dst}" "${src}"
|
||||||
|
;;
|
||||||
|
copy)
|
||||||
|
log_info "Copying files from ${src} to ${dst}"
|
||||||
|
copy_files "${dst}" "${src}"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
if [ $? -eq 0 ]; then
|
||||||
|
if [[ "${CHECK_TRANSFER_INTEGRITY}" == "true" ]]; then
|
||||||
|
log_info "Verifying integrity of transferred files..."
|
||||||
|
local src_checksum target_checksum
|
||||||
|
src_checksum=$(find "${src}" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort)
|
||||||
|
target_checksum=$(find "${dst}" -type f ! \( -iname "*.nfo" -o -iname "*.sfv" \) -exec md5sum {} \; | sort)
|
||||||
|
if diff <(echo "${src_checksum}") <(echo "${target_checksum}") >/dev/null; then
|
||||||
|
log_info "Integrity check passed."
|
||||||
|
else
|
||||||
|
log_error "Integrity check FAILED for ${src}"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
log_info "Transfer completed successfully"
|
||||||
|
mark_processed "${hash}"
|
||||||
|
else
|
||||||
|
log_error "Transfer failed for ${src}"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
return 0
|
||||||
|
}
|
||||||
84
usr/local/lib/torrent-mover/transmission_handler.sh
Normal file
84
usr/local/lib/torrent-mover/transmission_handler.sh
Normal file
@ -0,0 +1,84 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# Transmission-related functions for torrent-mover
|
||||||
|
|
||||||
|
# get_destination: Maps a source directory to a destination directory based on keywords and patterns
|
||||||
|
get_destination() {
|
||||||
|
local source_path="$1"
|
||||||
|
if [[ -n "${PATH_CACHE["${source_path}"]+x}" ]]; then
|
||||||
|
echo "${PATH_CACHE["${source_path}"]}"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_info "Analyzing path: ${source_path}"
|
||||||
|
local destination="${DEFAULT_DST}"
|
||||||
|
|
||||||
|
# Match using custom patterns from config file if they exist
|
||||||
|
if [[ -n "${CUSTOM_PATTERNS}" ]]; then
|
||||||
|
log_debug "Using custom patterns from config..."
|
||||||
|
# Parse and apply each pattern
|
||||||
|
IFS=';' read -ra PATTERN_ARRAY <<< "${CUSTOM_PATTERNS}"
|
||||||
|
for pattern in "${PATTERN_ARRAY[@]}"; do
|
||||||
|
IFS='=' read -ra PARTS <<< "${pattern}"
|
||||||
|
if [[ "${#PARTS[@]}" -eq 2 ]]; then
|
||||||
|
local regex="${PARTS[0]}"
|
||||||
|
local dest="${PARTS[1]}"
|
||||||
|
if [[ "${source_path,,}" =~ ${regex,,} ]]; then
|
||||||
|
log_info "Custom pattern match: ${regex} -> ${dest}"
|
||||||
|
destination="${dest}"
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
# If no custom pattern matched, use default category mapping
|
||||||
|
if [[ "${destination}" == "${DEFAULT_DST}" ]]; then
|
||||||
|
case "${source_path,,}" in
|
||||||
|
*games*) destination="${DIR_GAMES_DST}";;
|
||||||
|
*apps*|*applications*|*programs*|*software*) destination="${DIR_APPS_DST}";;
|
||||||
|
*movies*|*film*|*video*) destination="${DIR_MOVIES_DST}";;
|
||||||
|
*books*|*ebook*|*pdf*|*epub*) destination="${DIR_BOOKS_DST}";;
|
||||||
|
*tv*|*series*|*episode*)
|
||||||
|
if [[ -n "${DIR_TV_DST}" ]]; then
|
||||||
|
destination="${DIR_TV_DST}"
|
||||||
|
else
|
||||||
|
destination="${DIR_MOVIES_DST}"
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
*music*|*audio*|*mp3*|*flac*)
|
||||||
|
if [[ -n "${DIR_MUSIC_DST}" ]]; then
|
||||||
|
destination="${DIR_MUSIC_DST}"
|
||||||
|
else
|
||||||
|
destination="${DEFAULT_DST}"
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_info "Mapped to: ${destination}"
|
||||||
|
PATH_CACHE["${source_path}"]="${destination}"
|
||||||
|
echo "${destination}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# process_removal: Removes a torrent via Transmission.
|
||||||
|
process_removal() {
|
||||||
|
local id="$1"
|
||||||
|
if (( DRY_RUN )); then
|
||||||
|
log_info "[DRY RUN] Would remove torrent ${id}"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
retry_command "transmission-remote \"${TRANSMISSION_IP}:${TRANSMISSION_PORT}\" -n \"${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}\" -t \"${id}\" --remove-and-delete" 3 15
|
||||||
|
}
|
||||||
|
|
||||||
|
# get_torrents: Retrieves a list of torrents from Transmission
|
||||||
|
get_torrents() {
|
||||||
|
retry_command "transmission-remote \"${TRANSMISSION_IP}:${TRANSMISSION_PORT}\" -n \"${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}\" -l" 3 20 |
|
||||||
|
awk 'NR>1 && $1 ~ /^[0-9]+$/ {print $1}'
|
||||||
|
}
|
||||||
|
|
||||||
|
# get_torrent_info: Gets detailed info for a specific torrent
|
||||||
|
get_torrent_info() {
|
||||||
|
local id="$1"
|
||||||
|
retry_command "transmission-remote \"${TRANSMISSION_IP}:${TRANSMISSION_PORT}\" -n \"${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}\" -t \"${id}\" -i" 3 15
|
||||||
|
}
|
||||||
Loading…
x
Reference in New Issue
Block a user