torrent/README.md

Torrent Mover v9.1

Description

Torrent Mover is a Bash script designed to automate the processing of completed torrents in Transmission. It moves or copies downloaded files from a Transmission‑reported download location to designated destination directories on your system. This enhanced version includes a modular architecture, dedicated security user, robust locking, advanced error handling with retry capabilities, parallel processing, configurable path mapping, improved archive extraction, shared directory handling, and optional file integrity verification.

The system seamlessly organizes content into appropriate directories using smart pattern matching and customizable category detection, helping you maintain a well-structured media library with minimal manual intervention.

Features

Core Features

• Automatic Torrent Processing: Monitors Transmission for completed torrents and processes them based on configurable seeding criteria.
• Configurable Path Mapping: Uses Transmission's reported download path and maps it to your local file system via configurable settings.
• Archive Extraction: Extracts archives (RAR, ZIP, 7z) into subdirectories at the destination—preserving internal structure—while retaining the archive in the source until seeding criteria are met.
• Directory Deduplication: Prevents re‑processing the same source directory if multiple torrents reference it.
• Shared Directory Handling: Intelligently processes torrents that share the same download directory by matching files to specific torrents.

Advanced Content Organization

• Smart Content Categorization: Uses both pattern matching and directory name detection to properly categorize content.
• Regex Pattern Matching: Define custom regex patterns to precisely organize content into subcategories (documentaries, anime, etc.).
• Multi-Library Support: Manage content across multiple storage locations with different organization schemes.

Enhanced Security & Reliability

• Dedicated Non-Root User: Uses a dedicated service user with minimal permissions for enhanced security.
• Error Recovery: Includes retry mechanisms with configurable attempts and delay for network operations.
• Data Integrity Protection: Optionally verifies file integrity by comparing MD5 checksums after transfer.
• Robust Locking: Employs flock to ensure that only one instance of the script runs at a time.

Performance & Engineering

• Modular Architecture: Code is organized into separate modules for better maintainability and extensibility.
• Parallel File Operations: Utilizes GNU Parallel for moving, copying, and generating checksums, enabling efficient multi-threaded processing.
• Advanced Error Handling & Logging: Global error handler and detailed logging (with DEBUG mode support). Optionally, logs to syslog.

Requirements

• Bash
• transmission-remote
• GNU Parallel
• unrar, unzip, 7z
• bc

Installation

1. Run the installation script as root:

   sudo ./install.sh
   

2. The script will:

   • Install all necessary dependencies
   • Create a dedicated non-root user for security
   • Set up the configuration file in /etc/torrent/mover.conf
   • Install systemd service and timer
   • Configure file permissions and log rotation

3. Enable the service to run every 15 minutes:

   sudo systemctl enable --now torrent-mover.timer
   

Configuration

Edit the configuration file at /etc/torrent/mover.conf to customize the behavior of Torrent Mover:

Connection Configuration

# Transmission connection settings
TRANSMISSION_IP="192.168.1.100"       # IP address of your Transmission server
TRANSMISSION_PORT="9091"              # RPC port for Transmission
TRANSMISSION_USER="your_username"     # Username for authentication (if enabled)
TRANSMISSION_PASSWORD="your_password" # Password for authentication (if enabled)

# Path mapping configuration
TRANSMISSION_PATH_PREFIX="/downloads" # Path prefix reported by Transmission
LOCAL_PATH_PREFIX="/mnt/dsnas2"       # Corresponding local path prefix

Content Organization

# Primary content destination directories
DIR_GAMES_DST="/mnt/dsnas1/Games"     # Games destination
DIR_APPS_DST="/mnt/dsnas1/Apps"       # Applications destination
DIR_MOVIES_DST="/mnt/dsnas1/Movies"   # Movies destination
DIR_BOOKS_DST="/mnt/dsnas1/Books"     # Books/eBooks destination
DIR_TV_DST="/mnt/dsnas1/TV"           # TV series destination
DIR_MUSIC_DST="/mnt/dsnas1/Music"     # Music destination
DEFAULT_DST="/mnt/dsnas1/Other"       # Default for unrecognized content

# Additional storage libraries (comma-separated)
STORAGE_DIRS="/mnt/dsnas/Movies,/mnt/external/Movies" # Additional movie libraries
STORAGE_TV_DIRS="/mnt/dsnas/TV,/mnt/external/TV"      # Additional TV libraries

# Custom pattern matching for advanced categorization
# Format: "regex_pattern=destination_path;another_pattern=another_path"
CUSTOM_PATTERNS=".*documentary.*=${DIR_MOVIES_DST}/Documentary;
                 .*anime.*=${DIR_TV_DST}/Anime;
                 .*linux.*=${DIR_APPS_DST}/Linux;
                 .*tutorial.*=${DIR_BOOKS_DST}/Tutorials"

Security & Performance

# Security settings - dedicated non-root user
TORRENT_USER="torrent-mover"          # Dedicated service user
TORRENT_GROUP="torrent-mover"         # User's primary group

# Error recovery configuration
MAX_RETRY_ATTEMPTS="3"                # Maximum retry attempts for failed operations
RETRY_WAIT_TIME="15"                  # Seconds to wait between retry attempts

# Performance tuning
PARALLEL_THREADS="32"                 # Number of parallel threads (match CPU cores)
PARALLEL_PROCESSING=1                 # Enable (1) or disable (0) parallel processing

# Operation mode
COPY_MODE="copy"                      # "copy" to preserve or "move" to relocate files

Logging & Integrity

# File tracking & integrity
PROCESSED_LOG="/var/log/torrent_processed.log" # Tracks processed torrents
CHECKSUM_DB="/var/lib/torrent/checksums.db"    # Stores file checksums

# Logging configuration
LOG_FILE="/var/log/torrent_mover.log" # Main log file location
LOG_LEVEL="INFO"                      # Logging level: "INFO" or "DEBUG"
USE_SYSLOG="false"                    # Also log to system syslog: "true" or "false"

# Data integrity protection
CHECK_TRANSFER_INTEGRITY="true"       # Verify file integrity after transfers

Usage

Main Torrent Mover Script

Run the main script using the following options:

• Dry-run mode (simulate operations):

  /usr/local/bin/torrent-mover --dry-run
  

• Interactive mode (prompt for confirmation):

  /usr/local/bin/torrent-mover --interactive
  

• Cache warmup mode (pre-calculate checksums):

  /usr/local/bin/torrent-mover --cache-warmup
  

• Debug mode (verbose logging):

  /usr/local/bin/torrent-mover --debug
  

You can combine options as needed. For example:

/usr/local/bin/torrent-mover --dry-run --debug

Helper Scripts

The system includes additional helper scripts for more advanced usage:

• Torrent Processor:

  /usr/local/bin/torrent-processor [OPTIONS]
  

  Available options:

  • --reset - Clear processed log to re-process all torrents
  • --books - Process only book torrents
  • --movies - Process only movie torrents
  • --tv - Process only TV show torrents
  • --apps - Process only application torrents
  • --games - Process only game torrents
  • --id NUMBER - Process a specific torrent ID

  Examples:

  # Process all book torrents (even if previously processed)
  /usr/local/bin/torrent-processor --reset --books
  
  # Process only torrent with ID 123
  /usr/local/bin/torrent-processor --id 123
  

• Smart Processor:

  /usr/local/bin/smart-processor
  

  An alternative processor specifically designed to handle shared directories more intelligently by:

  • Detecting shared download directories
  • Matching files to specific torrents
  • Using content type detection for files
  • Processing multiple torrents efficiently

Configuration Management Tool

The system includes a dedicated configuration management tool that helps you safely update and manage your torrent-mover settings:

sudo torrent-config [OPTION]

Available options:

• show - Display the current configuration with color-coding
• edit - Edit the configuration in your preferred text editor (automatically creates a backup)
• backup - Create a timestamped backup of the current configuration
• restore - List and restore from available backups
• validate - Check the configuration for errors
• set KEY VALUE - Update a specific configuration value
• get KEY - Retrieve the current value of a configuration setting
• default - Show the default configuration values as a reference

Examples:

# View current configuration
sudo torrent-config show

# Change the copy mode to 'move'
sudo torrent-config set COPY_MODE move

# Add a new pattern for documentaries
sudo torrent-config set CUSTOM_PATTERNS ".*documentary.*=${DIR_MOVIES_DST}/Documentary"

# Edit the configuration file in your preferred editor
sudo torrent-config edit

# View the value of a specific setting
sudo torrent-config get TRANSMISSION_IP

Architecture & Module Organization

The system uses a modular architecture for improved maintainability:

• Main Script (/usr/local/bin/torrent-mover): Orchestrates the overall process and loads modules
• Common Module: Contains shared utilities, logging functions and error handling
• File Operations Module: Handles file transfers, checksums, and integrity verification
• Archive Handler Module: Specializes in extracting and managing various archive formats
• Transmission Handler Module: Manages all communication with the Transmission client

How It Works

Initialization & Configuration

1. Module Loading: The main script dynamically loads all modules from the /usr/local/lib/torrent-mover directory
2. Configuration Processing: Loads and validates the configuration from /etc/torrent/mover.conf
3. Locking: Uses flock to prevent multiple instances from running simultaneously

Torrent Processing Workflow

1. Torrent Discovery: Retrieves the list of torrents from Transmission using retry-enabled API calls
2. Smart Path Translation: Converts Transmission-reported paths to local filesystem paths using configurable mappings
3. Content Categorization:
   • First applies custom regex patterns from the configuration
   • Falls back to keyword-based directory name detection if no patterns match
   • Determines the appropriate destination directory for each content type
4. Deduplication & Verification:
   • Tracks processed source directories to avoid redundant operations
   • Generates and compares checksums between source and potential destinations
   • Skips transfers if identical content is already present in any destination library
5. Smart File Matching:
   • Detects when multiple torrents share the same download directory
   • Uses intelligent pattern matching to identify specific files for each torrent
   • Handles shared directories by matching torrent names to specific files
6. File Processing:
   • Extracts archives with preservation of directory structure
   • Transfers files using parallel operations when enabled
   • Verifies integrity after transfer if configured
7. Cleanup & Monitoring:
   • Checks seeding ratio and time against configured thresholds
   • Removes torrents from Transmission when criteria are met
   • Monitors disk usage across all configured storage directories

License

This script is provided as-is without any warranty. Use it at your own risk. Contributions and improvements are welcome.