Compare commits
7 Commits
4c7ebaf5fe
...
91106a244c
Author | SHA1 | Date | |
---|---|---|---|
91106a244c | |||
1119f38fd6 | |||
e64e1115a7 | |||
bf41b9ad71 | |||
4f7cb91bc5 | |||
fb56817e76 | |||
f572a241ef |
51
README.md
51
README.md
@ -1,11 +1,11 @@
|
|||||||
# Torrent Mover v8.0
|
# Torrent Mover v9.1
|
||||||
|
|
||||||
## Description
|
## Description
|
||||||
|
|
||||||
**Torrent Mover** is a Bash script designed to automate the processing of completed torrents in Transmission.
|
**Torrent Mover** is a Bash script designed to automate the processing of completed torrents in Transmission.
|
||||||
It moves or copies downloaded files from a Transmission‑reported download location to designated destination directories on your system.
|
It moves or copies downloaded files from a Transmission‑reported download location to designated destination directories on your system.
|
||||||
This enhanced version includes a modular architecture, dedicated security user, robust locking, advanced error handling with retry capabilities,
|
This enhanced version includes a modular architecture, dedicated security user, robust locking, advanced error handling with retry capabilities,
|
||||||
parallel processing, configurable path mapping, improved archive extraction, and optional file integrity verification.
|
parallel processing, configurable path mapping, improved archive extraction, shared directory handling, and optional file integrity verification.
|
||||||
|
|
||||||
The system seamlessly organizes content into appropriate directories using smart pattern matching and customizable category detection, helping you maintain a well-structured media library with minimal manual intervention.
|
The system seamlessly organizes content into appropriate directories using smart pattern matching and customizable category detection, helping you maintain a well-structured media library with minimal manual intervention.
|
||||||
|
|
||||||
@ -16,6 +16,7 @@ The system seamlessly organizes content into appropriate directories using smart
|
|||||||
- **Configurable Path Mapping:** Uses Transmission's reported download path and maps it to your local file system via configurable settings.
|
- **Configurable Path Mapping:** Uses Transmission's reported download path and maps it to your local file system via configurable settings.
|
||||||
- **Archive Extraction:** Extracts archives (RAR, ZIP, 7z) into subdirectories at the destination—preserving internal structure—while retaining the archive in the source until seeding criteria are met.
|
- **Archive Extraction:** Extracts archives (RAR, ZIP, 7z) into subdirectories at the destination—preserving internal structure—while retaining the archive in the source until seeding criteria are met.
|
||||||
- **Directory Deduplication:** Prevents re‑processing the same source directory if multiple torrents reference it.
|
- **Directory Deduplication:** Prevents re‑processing the same source directory if multiple torrents reference it.
|
||||||
|
- **Shared Directory Handling:** Intelligently processes torrents that share the same download directory by matching files to specific torrents.
|
||||||
|
|
||||||
### Advanced Content Organization
|
### Advanced Content Organization
|
||||||
- **Smart Content Categorization:** Uses both pattern matching and directory name detection to properly categorize content.
|
- **Smart Content Categorization:** Uses both pattern matching and directory name detection to properly categorize content.
|
||||||
@ -164,6 +165,44 @@ You can combine options as needed. For example:
|
|||||||
/usr/local/bin/torrent-mover --dry-run --debug
|
/usr/local/bin/torrent-mover --dry-run --debug
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Helper Scripts
|
||||||
|
|
||||||
|
The system includes additional helper scripts for more advanced usage:
|
||||||
|
|
||||||
|
- **Torrent Processor:**
|
||||||
|
```
|
||||||
|
/usr/local/bin/torrent-processor [OPTIONS]
|
||||||
|
```
|
||||||
|
|
||||||
|
Available options:
|
||||||
|
- `--reset` - Clear processed log to re-process all torrents
|
||||||
|
- `--books` - Process only book torrents
|
||||||
|
- `--movies` - Process only movie torrents
|
||||||
|
- `--tv` - Process only TV show torrents
|
||||||
|
- `--apps` - Process only application torrents
|
||||||
|
- `--games` - Process only game torrents
|
||||||
|
- `--id NUMBER` - Process a specific torrent ID
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
```bash
|
||||||
|
# Process all book torrents (even if previously processed)
|
||||||
|
/usr/local/bin/torrent-processor --reset --books
|
||||||
|
|
||||||
|
# Process only torrent with ID 123
|
||||||
|
/usr/local/bin/torrent-processor --id 123
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Smart Processor:**
|
||||||
|
```
|
||||||
|
/usr/local/bin/smart-processor
|
||||||
|
```
|
||||||
|
|
||||||
|
An alternative processor specifically designed to handle shared directories more intelligently by:
|
||||||
|
- Detecting shared download directories
|
||||||
|
- Matching files to specific torrents
|
||||||
|
- Using content type detection for files
|
||||||
|
- Processing multiple torrents efficiently
|
||||||
|
|
||||||
### Configuration Management Tool
|
### Configuration Management Tool
|
||||||
|
|
||||||
The system includes a dedicated configuration management tool that helps you safely update and manage your torrent-mover settings:
|
The system includes a dedicated configuration management tool that helps you safely update and manage your torrent-mover settings:
|
||||||
@ -229,11 +268,15 @@ The system uses a modular architecture for improved maintainability:
|
|||||||
- Tracks processed source directories to avoid redundant operations
|
- Tracks processed source directories to avoid redundant operations
|
||||||
- Generates and compares checksums between source and potential destinations
|
- Generates and compares checksums between source and potential destinations
|
||||||
- Skips transfers if identical content is already present in any destination library
|
- Skips transfers if identical content is already present in any destination library
|
||||||
5. **File Processing:**
|
5. **Smart File Matching:**
|
||||||
|
- Detects when multiple torrents share the same download directory
|
||||||
|
- Uses intelligent pattern matching to identify specific files for each torrent
|
||||||
|
- Handles shared directories by matching torrent names to specific files
|
||||||
|
6. **File Processing:**
|
||||||
- Extracts archives with preservation of directory structure
|
- Extracts archives with preservation of directory structure
|
||||||
- Transfers files using parallel operations when enabled
|
- Transfers files using parallel operations when enabled
|
||||||
- Verifies integrity after transfer if configured
|
- Verifies integrity after transfer if configured
|
||||||
6. **Cleanup & Monitoring:**
|
7. **Cleanup & Monitoring:**
|
||||||
- Checks seeding ratio and time against configured thresholds
|
- Checks seeding ratio and time against configured thresholds
|
||||||
- Removes torrents from Transmission when criteria are met
|
- Removes torrents from Transmission when criteria are met
|
||||||
- Monitors disk usage across all configured storage directories
|
- Monitors disk usage across all configured storage directories
|
||||||
|
@ -22,6 +22,10 @@ STORAGE_DIRS="/mnt/dsnas/Movies"
|
|||||||
STORAGE_TV_DIRS="/mnt/dsnas/TV"
|
STORAGE_TV_DIRS="/mnt/dsnas/TV"
|
||||||
|
|
||||||
# Path mapping
|
# Path mapping
|
||||||
|
# This maps the transmission-reported download path to the local filesystem path
|
||||||
|
# The script will use this prefix to translate paths between Transmission and local filesystem
|
||||||
|
#
|
||||||
|
# IMPORTANT: Transmission reports paths as /downloads/Books but they are actually in /mnt/dsnas2/Books
|
||||||
TRANSMISSION_PATH_PREFIX="/downloads"
|
TRANSMISSION_PATH_PREFIX="/downloads"
|
||||||
LOCAL_PATH_PREFIX="/mnt/dsnas2"
|
LOCAL_PATH_PREFIX="/mnt/dsnas2"
|
||||||
|
|
||||||
@ -33,7 +37,7 @@ TORRENT_GROUP="debian-transmission"
|
|||||||
# Custom pattern matching for content categorization
|
# Custom pattern matching for content categorization
|
||||||
# Format: "pattern1=destination1;pattern2=destination2"
|
# Format: "pattern1=destination1;pattern2=destination2"
|
||||||
# Example: ".*\.linux.*=${DIR_LINUX_DST};.*documentary.*=${DIR_DOCUMENTARY_DST}"
|
# Example: ".*\.linux.*=${DIR_LINUX_DST};.*documentary.*=${DIR_DOCUMENTARY_DST}"
|
||||||
CUSTOM_PATTERNS=".*documentary.*=${DIR_MOVIES_DST}/Documentary;.*anime.*=${DIR_TV_DST}/Anime"
|
CUSTOM_PATTERNS=".*documentary.*=${DIR_MOVIES_DST}/Documentary;.*anime.*=${DIR_TV_DST}/Anime;.*games.*=${DIR_GAMES_DST};.*apps.*=${DIR_APPS_DST};.*books.*=${DIR_BOOKS_DST};.*tv.*=${DIR_TV_DST};.*series.*=${DIR_TV_DST};.*music.*=${DIR_MUSIC_DST}"
|
||||||
|
|
||||||
# Error recovery settings
|
# Error recovery settings
|
||||||
MAX_RETRY_ATTEMPTS="3"
|
MAX_RETRY_ATTEMPTS="3"
|
||||||
|
14
install.sh
14
install.sh
@ -144,6 +144,20 @@ AccuracySec=1min
|
|||||||
WantedBy=timers.target
|
WantedBy=timers.target
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
|
# Install helper scripts
|
||||||
|
echo "Installing helper scripts..."
|
||||||
|
if [ -f "${SCRIPT_DIR}/usr/local/bin/torrent-processor" ]; then
|
||||||
|
cp "${SCRIPT_DIR}/usr/local/bin/torrent-processor" /usr/local/bin/
|
||||||
|
chmod 755 /usr/local/bin/torrent-processor
|
||||||
|
echo "- Installed torrent-processor"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -f "${SCRIPT_DIR}/usr/local/bin/smart-processor" ]; then
|
||||||
|
cp "${SCRIPT_DIR}/usr/local/bin/smart-processor" /usr/local/bin/
|
||||||
|
chmod 755 /usr/local/bin/smart-processor
|
||||||
|
echo "- Installed smart-processor"
|
||||||
|
fi
|
||||||
|
|
||||||
# Set permissions
|
# Set permissions
|
||||||
echo "Setting permissions..."
|
echo "Setting permissions..."
|
||||||
chmod 600 /etc/torrent/mover.conf*
|
chmod 600 /etc/torrent/mover.conf*
|
||||||
|
183
usr/local/bin/smart-processor
Executable file
183
usr/local/bin/smart-processor
Executable file
@ -0,0 +1,183 @@
|
|||||||
|
#\!/bin/bash
|
||||||
|
|
||||||
|
# Source configuration
|
||||||
|
source /etc/torrent/mover.conf
|
||||||
|
|
||||||
|
# Reset processed log
|
||||||
|
> /var/log/torrent_processed.log
|
||||||
|
|
||||||
|
# Process all torrents - smart version for shared directories
|
||||||
|
echo "Starting smart torrent processor..."
|
||||||
|
echo "This script will identify and copy files for completed torrents"
|
||||||
|
echo "----------------------------------------------------------------"
|
||||||
|
|
||||||
|
# Make sure destination directories exist
|
||||||
|
mkdir -p /mnt/dsnas1/{Books,Movies,TV,Games,Apps,Music,Other}
|
||||||
|
|
||||||
|
# Get list of torrents
|
||||||
|
IDS=$(transmission-remote "${TRANSMISSION_IP}:${TRANSMISSION_PORT}" \
|
||||||
|
--auth "${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}" \
|
||||||
|
--list | tail -n +2 | head -n -1 | awk '{print $1}' | grep -v "Sum:" | grep -v "[a-zA-Z]")
|
||||||
|
|
||||||
|
# Count torrents
|
||||||
|
TOTAL_TORRENTS=$(echo "$IDS" | wc -l)
|
||||||
|
echo "Found $TOTAL_TORRENTS torrents to process"
|
||||||
|
|
||||||
|
# Process each torrent
|
||||||
|
COUNT=0
|
||||||
|
for id in $IDS; do
|
||||||
|
# Progress counter
|
||||||
|
COUNT=$((COUNT+1))
|
||||||
|
|
||||||
|
# Get torrent info
|
||||||
|
INFO=$(transmission-remote "${TRANSMISSION_IP}:${TRANSMISSION_PORT}" \
|
||||||
|
--auth "${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}" \
|
||||||
|
--torrent $id --info)
|
||||||
|
|
||||||
|
# Extract key information
|
||||||
|
NAME=$(echo "$INFO" | grep "Name:" | awk -F": " '{print $2}' | xargs)
|
||||||
|
HASH=$(echo "$INFO" | grep "Hash:" | awk '{print $2}')
|
||||||
|
PERCENT=$(echo "$INFO" | grep "Percent Done:" | awk '{gsub(/%/, ""); print $3 == "None" ? 0 : $3}')
|
||||||
|
LOCATION=$(echo "$INFO" | grep -i "Location:" | awk -F": " '{print $2}' | xargs)
|
||||||
|
|
||||||
|
# Skip if not 100% complete
|
||||||
|
if [ $(bc <<< "$PERCENT < 100") -eq 1 ]; then
|
||||||
|
echo "[$COUNT/$TOTAL_TORRENTS] Skipping incomplete torrent $id: $NAME ($PERCENT%)"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Skip if already processed
|
||||||
|
if grep -q "$HASH" /var/log/torrent_processed.log; then
|
||||||
|
echo "[$COUNT/$TOTAL_TORRENTS] Skipping already processed torrent $id: $NAME"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "[$COUNT/$TOTAL_TORRENTS] Processing torrent $id: $NAME"
|
||||||
|
|
||||||
|
# Apply path mapping
|
||||||
|
SRC="${LOCATION/#$TRANSMISSION_PATH_PREFIX/$LOCAL_PATH_PREFIX}"
|
||||||
|
|
||||||
|
# Set destination based on content type
|
||||||
|
DST="$DEFAULT_DST"
|
||||||
|
|
||||||
|
if [[ "$LOCATION" == */Books* || "$NAME" == *eBook* || "$NAME" == *ePub* ]]; then
|
||||||
|
DST="$DIR_BOOKS_DST"
|
||||||
|
echo " Categorized as: Book"
|
||||||
|
elif [[ "$LOCATION" == */Movies* || "$NAME" == *1080p* || "$NAME" == *720p* ]]; then
|
||||||
|
DST="$DIR_MOVIES_DST"
|
||||||
|
echo " Categorized as: Movie"
|
||||||
|
elif [[ "$LOCATION" == */TV* || "$NAME" == *S0* || "$NAME" == *S1* ]]; then
|
||||||
|
DST="$DIR_TV_DST"
|
||||||
|
echo " Categorized as: TV Show"
|
||||||
|
elif [[ "$LOCATION" == */Games* || "$NAME" == *Game* ]]; then
|
||||||
|
DST="$DIR_GAMES_DST"
|
||||||
|
echo " Categorized as: Game"
|
||||||
|
elif [[ "$LOCATION" == */Apps* || "$NAME" == *App* ]]; then
|
||||||
|
DST="$DIR_APPS_DST"
|
||||||
|
echo " Categorized as: App"
|
||||||
|
elif [[ "$LOCATION" == */Music* || "$NAME" == *MP3* ]]; then
|
||||||
|
DST="$DIR_MUSIC_DST"
|
||||||
|
echo " Categorized as: Music"
|
||||||
|
else
|
||||||
|
echo " Categorized as: Other"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Make sure destination exists
|
||||||
|
mkdir -p "$DST"
|
||||||
|
|
||||||
|
# Now handle the file copying based on directory structure
|
||||||
|
if [ -d "$SRC" ]; then
|
||||||
|
echo " Source path: $SRC"
|
||||||
|
echo " Destination: $DST"
|
||||||
|
|
||||||
|
# Use find to locate specific content files (ignore small files like NFO)
|
||||||
|
FILES_FOUND=0
|
||||||
|
echo " Looking for media files or content..."
|
||||||
|
|
||||||
|
# Try to find files matching this specific torrent name
|
||||||
|
NAME_PATTERN=$(echo "$NAME" | cut -d'-' -f1 | tr '.' ' ' | xargs | tr '[:upper:]' '[:lower:]')
|
||||||
|
NAME_PATTERN=${NAME_PATTERN// /.}
|
||||||
|
|
||||||
|
echo " Searching for files matching pattern: $NAME_PATTERN"
|
||||||
|
|
||||||
|
# Search for matching files or directories
|
||||||
|
MATCHING_FILES=()
|
||||||
|
while IFS= read -r file; do
|
||||||
|
file_basename=$(basename "$file" | tr '[:upper:]' '[:lower:]')
|
||||||
|
|
||||||
|
if [[ "$file_basename" == *"$NAME_PATTERN"* ]]; then
|
||||||
|
size=$(stat -c%s "$file")
|
||||||
|
MATCHING_FILES+=("$file")
|
||||||
|
echo " ✓ Match: $(basename "$file") ($(numfmt --to=iec $size))"
|
||||||
|
fi
|
||||||
|
done < <(find "$SRC" -type f -size +10k | sort -rn -k5 | head -n 20)
|
||||||
|
|
||||||
|
if [ ${#MATCHING_FILES[@]} -gt 0 ]; then
|
||||||
|
echo " Found ${#MATCHING_FILES[@]} matching files for this torrent"
|
||||||
|
|
||||||
|
# Copy up to 3 matched files
|
||||||
|
for ((i=0; i<3 && i<${#MATCHING_FILES[@]}; i++)); do
|
||||||
|
file="${MATCHING_FILES[$i]}"
|
||||||
|
echo " Copying: $(basename "$file") to $DST/"
|
||||||
|
cp -v "$file" "$DST/"
|
||||||
|
FILES_FOUND=$((FILES_FOUND+1))
|
||||||
|
done
|
||||||
|
else
|
||||||
|
echo " No exact matches found - falling back to content type detection"
|
||||||
|
|
||||||
|
# Get a list of content files ordered by size (largest first)
|
||||||
|
while IFS= read -r file; do
|
||||||
|
extension="${file##*.}"
|
||||||
|
extension="${extension,,}" # Convert to lowercase
|
||||||
|
filename=$(basename "$file")
|
||||||
|
|
||||||
|
# Skip small files under 1MB (likely not content)
|
||||||
|
size=$(stat -c%s "$file")
|
||||||
|
|
||||||
|
# Only include files based on type
|
||||||
|
if [[ "$DST" == "$DIR_MOVIES_DST" && "$extension" == @(mkv|mp4|avi) ]]; then
|
||||||
|
echo " Found movie: $filename (Size: $(numfmt --to=iec $size))"
|
||||||
|
echo " Copying to $DST/"
|
||||||
|
cp -v "$file" "$DST/"
|
||||||
|
FILES_FOUND=$((FILES_FOUND+1))
|
||||||
|
elif [[ "$DST" == "$DIR_BOOKS_DST" && "$extension" == @(epub|pdf|mobi) ]]; then
|
||||||
|
echo " Found book: $filename (Size: $(numfmt --to=iec $size))"
|
||||||
|
echo " Copying to $DST/"
|
||||||
|
cp -v "$file" "$DST/"
|
||||||
|
FILES_FOUND=$((FILES_FOUND+1))
|
||||||
|
elif [[ "$DST" == "$DIR_TV_DST" && "$extension" == @(mkv|mp4|avi) ]]; then
|
||||||
|
echo " Found TV episode: $filename (Size: $(numfmt --to=iec $size))"
|
||||||
|
echo " Copying to $DST/"
|
||||||
|
cp -v "$file" "$DST/"
|
||||||
|
FILES_FOUND=$((FILES_FOUND+1))
|
||||||
|
elif [[ "$size" -gt 1000000 ]]; then # 1MB for other content types
|
||||||
|
echo " Found content: $filename (Size: $(numfmt --to=iec $size))"
|
||||||
|
echo " Copying to $DST/"
|
||||||
|
cp -v "$file" "$DST/"
|
||||||
|
FILES_FOUND=$((FILES_FOUND+1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Limit to first 3 content files to avoid excessive copying
|
||||||
|
if [ $FILES_FOUND -ge 3 ]; then
|
||||||
|
echo " Reached limit of 3 content files"
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done < <(find "$SRC" -type f -size +100k | sort -rn -k5 | head -n 10)
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ $FILES_FOUND -gt 0 ]; then
|
||||||
|
echo " ✅ Successfully copied $FILES_FOUND files"
|
||||||
|
# Mark as processed
|
||||||
|
echo "$HASH" >> /var/log/torrent_processed.log
|
||||||
|
else
|
||||||
|
echo " ❌ No suitable content files found"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo " ❌ Source directory not found: $SRC"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "------------------------------------------------------"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Smart torrent processing completed"
|
||||||
|
echo "Processed torrents are recorded in /var/log/torrent_processed.log"
|
@ -115,7 +115,29 @@ main() {
|
|||||||
declare -A warned_dirs=()
|
declare -A warned_dirs=()
|
||||||
|
|
||||||
# Get list of torrents from Transmission
|
# Get list of torrents from Transmission
|
||||||
get_torrents | while read -r id; do
|
log_debug "Getting list of torrents..."
|
||||||
|
local torrent_ids
|
||||||
|
torrent_ids=$(get_torrents)
|
||||||
|
log_info "Found $(echo "$torrent_ids" | wc -l) torrents"
|
||||||
|
|
||||||
|
# Use a regular for loop instead of a pipe to while
|
||||||
|
# to avoid the subshell issue that causes processed_source_dirs to be lost
|
||||||
|
readarray -t torrent_ids_array <<< "$torrent_ids"
|
||||||
|
|
||||||
|
# Print the torrent IDs to debug (always, not just in debug mode)
|
||||||
|
if [[ ${#torrent_ids_array[@]} -eq 0 ]]; then
|
||||||
|
log_info "No torrents found to process"
|
||||||
|
else
|
||||||
|
log_info "Torrent IDs to process: ${torrent_ids_array[*]}"
|
||||||
|
fi
|
||||||
|
for id in "${torrent_ids_array[@]}"; do
|
||||||
|
# Skip empty IDs
|
||||||
|
if [[ -z "$id" ]]; then
|
||||||
|
log_debug "Skipping empty torrent ID"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_debug "Processing torrent ID: $id"
|
||||||
local info
|
local info
|
||||||
info=$(get_torrent_info "${id}")
|
info=$(get_torrent_info "${id}")
|
||||||
|
|
||||||
@ -134,19 +156,73 @@ main() {
|
|||||||
# Extract Transmission-reported directory and translate to local path.
|
# Extract Transmission-reported directory and translate to local path.
|
||||||
local reported_dir
|
local reported_dir
|
||||||
reported_dir=$(grep -i "Location:" <<< "${info}" | awk -F": " '{print $2}' | xargs)
|
reported_dir=$(grep -i "Location:" <<< "${info}" | awk -F": " '{print $2}' | xargs)
|
||||||
|
log_debug "Raw reported directory: '${reported_dir}'"
|
||||||
|
|
||||||
|
# If the reported directory is empty, try to derive it from the name
|
||||||
|
if [[ -z "${reported_dir}" ]]; then
|
||||||
|
local name
|
||||||
|
name=$(grep -i "Name:" <<< "${info}" | awk -F": " '{print $2}' | xargs)
|
||||||
|
log_debug "Torrent name: '${name}'"
|
||||||
|
|
||||||
|
# Check if there are labels we can use
|
||||||
|
local labels
|
||||||
|
labels=$(grep -i "Labels:" <<< "${info}" | awk -F": " '{print $2}' | xargs)
|
||||||
|
log_debug "Torrent labels: '${labels}'"
|
||||||
|
|
||||||
|
if [[ "${labels}" == *"Books"* ]]; then
|
||||||
|
reported_dir="/downloads/Books"
|
||||||
|
elif [[ "${labels}" == *"Movies"* ]]; then
|
||||||
|
reported_dir="/downloads/Movies"
|
||||||
|
elif [[ "${labels}" == *"TV"* ]]; then
|
||||||
|
reported_dir="/downloads/TV"
|
||||||
|
elif [[ "${labels}" == *"Games"* ]]; then
|
||||||
|
reported_dir="/downloads/Games"
|
||||||
|
elif [[ "${labels}" == *"Apps"* ]]; then
|
||||||
|
reported_dir="/downloads/Apps"
|
||||||
|
elif [[ "${labels}" == *"Music"* ]]; then
|
||||||
|
reported_dir="/downloads/Music"
|
||||||
|
else
|
||||||
|
# Default to Other if we can't determine
|
||||||
|
reported_dir="/downloads/Other"
|
||||||
|
fi
|
||||||
|
log_debug "Derived directory from labels: '${reported_dir}'"
|
||||||
|
fi
|
||||||
|
|
||||||
local dir
|
local dir
|
||||||
dir=$(translate_source "${reported_dir}")
|
dir=$(translate_source "${reported_dir}")
|
||||||
log_info "Torrent source directory reported: '${reported_dir}' translated to '${dir}'"
|
log_info "Torrent source directory: '${reported_dir}' translated to '${dir}'"
|
||||||
|
|
||||||
|
# Initialize empty directory mapping if needed
|
||||||
|
if [[ -z "$dir" ]]; then
|
||||||
|
log_warn "Empty directory path detected, using default"
|
||||||
|
dir="${LOCAL_PATH_PREFIX}/Other"
|
||||||
|
fi
|
||||||
|
|
||||||
local dst
|
local dst
|
||||||
dst=$(get_destination "${dir}")
|
dst=$(get_destination "${dir}")
|
||||||
|
|
||||||
|
# Detect same-path mappings (different mounts)
|
||||||
|
if [[ "${dir}" != "${dst}" && "${dir}" =~ ^/mnt/dsnas2/ && "${dst}" =~ ^/mnt/dsnas1/ ]]; then
|
||||||
|
local dir_suffix="${dir#/mnt/dsnas2/}"
|
||||||
|
local dst_suffix="${dst#/mnt/dsnas1/}"
|
||||||
|
if [[ "${dir_suffix}" == "${dst_suffix}" ]]; then
|
||||||
|
log_info "Source and destination are the same logical location with different mounts: ${dir_suffix}"
|
||||||
|
mark_processed "${hash}"
|
||||||
|
continue # Skip to next torrent
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Initialize warned_dirs for this directory if needed
|
||||||
|
if [[ -n "${dir}" ]]; then
|
||||||
[[ -z "${warned_dirs["${dir}"]+x}" ]] && warned_dirs["${dir}"]=0
|
[[ -z "${warned_dirs["${dir}"]+x}" ]] && warned_dirs["${dir}"]=0
|
||||||
|
fi
|
||||||
|
|
||||||
# Avoid processing the same directory more than once.
|
# Avoid processing the same directory more than once.
|
||||||
if [[ -n "${processed_source_dirs["${dir}"]+x}" ]]; then
|
if [[ -n "${processed_source_dirs["${dir}"]+x}" ]]; then
|
||||||
log_info "Directory ${dir} has already been processed; skipping copy for torrent ${id}"
|
log_info "Directory ${dir} has already been processed; skipping copy for torrent ${id}"
|
||||||
elif (( $(bc <<< "${percent_done} >= 100") )) && ! is_processed "${hash}"; then
|
elif (( $(bc <<< "${percent_done} >= 100") )) && ! is_processed "${hash}"; then
|
||||||
log_info "Processing completed torrent ${id} (${percent_done}% done)"
|
log_info "Processing completed torrent ${id} (${percent_done}% done)"
|
||||||
if [[ "${dst}" == "${DEFAULT_DST}" ]] && (( warned_dirs["${dir}"] == 0 )); then
|
if [[ "${dst}" == "${DEFAULT_DST}" ]] && [[ -n "${dir}" ]] && (( warned_dirs["${dir}"] == 0 )); then
|
||||||
log_warn "Using default destination for: ${dir}"
|
log_warn "Using default destination for: ${dir}"
|
||||||
warned_dirs["${dir}"]=1
|
warned_dirs["${dir}"]=1
|
||||||
fi
|
fi
|
||||||
@ -178,6 +254,14 @@ main() {
|
|||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
|
# Print count of processed directories
|
||||||
|
if [[ "${DEBUG}" -eq 1 ]]; then
|
||||||
|
log_debug "Processed source directories count: ${#processed_source_dirs[@]}"
|
||||||
|
for dir in "${!processed_source_dirs[@]}"; do
|
||||||
|
log_debug "Processed directory: $dir"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
# Check disk usage for all directories
|
# Check disk usage for all directories
|
||||||
for dir in "${REQUIRED_DIRS[@]}"; do
|
for dir in "${REQUIRED_DIRS[@]}"; do
|
||||||
check_disk_usage "${dir}"
|
check_disk_usage "${dir}"
|
||||||
|
368
usr/local/bin/torrent-processor
Executable file
368
usr/local/bin/torrent-processor
Executable file
@ -0,0 +1,368 @@
|
|||||||
|
#\!/bin/bash
|
||||||
|
|
||||||
|
# Source configuration
|
||||||
|
source /etc/torrent/mover.conf
|
||||||
|
|
||||||
|
# Create destination directories
|
||||||
|
mkdir -p /mnt/dsnas1/{Books,Movies,TV,Games,Apps,Music,Other}
|
||||||
|
|
||||||
|
# Function to display help
|
||||||
|
show_help() {
|
||||||
|
echo "Torrent Processor - Helper for torrent-mover"
|
||||||
|
echo ""
|
||||||
|
echo "Usage: $0 [OPTIONS]"
|
||||||
|
echo ""
|
||||||
|
echo "Options:"
|
||||||
|
echo " --reset Clear processed log to re-process all torrents"
|
||||||
|
echo " --books Process only book torrents"
|
||||||
|
echo " --movies Process only movie torrents"
|
||||||
|
echo " --tv Process only TV show torrents"
|
||||||
|
echo " --apps Process only application torrents"
|
||||||
|
echo " --games Process only game torrents"
|
||||||
|
echo " --id NUMBER Process a specific torrent ID"
|
||||||
|
echo " --help Show this help message"
|
||||||
|
echo ""
|
||||||
|
echo "Examples:"
|
||||||
|
echo " $0 --reset --books Process all book torrents (even if previously processed)"
|
||||||
|
echo " $0 --id 123 Process only torrent with ID 123"
|
||||||
|
echo ""
|
||||||
|
}
|
||||||
|
|
||||||
|
# Parse command line options
|
||||||
|
RESET=0
|
||||||
|
CATEGORY=""
|
||||||
|
TORRENT_ID=""
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
key="$1"
|
||||||
|
case $key in
|
||||||
|
--reset)
|
||||||
|
RESET=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--books)
|
||||||
|
CATEGORY="books"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--movies)
|
||||||
|
CATEGORY="movies"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--tv)
|
||||||
|
CATEGORY="tv"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--apps)
|
||||||
|
CATEGORY="apps"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--games)
|
||||||
|
CATEGORY="games"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--id)
|
||||||
|
TORRENT_ID="$2"
|
||||||
|
shift
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--help)
|
||||||
|
show_help
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
echo "Unknown option: $key"
|
||||||
|
show_help
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Reset processed log if requested
|
||||||
|
if [ $RESET -eq 1 ]; then
|
||||||
|
echo "Clearing processed log to re-process all torrents"
|
||||||
|
> /var/log/torrent_processed.log
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Remove lock file if it exists
|
||||||
|
rm -f /var/lock/torrent-mover.lock
|
||||||
|
|
||||||
|
# Run torrent-mover based on options
|
||||||
|
if [ -n "$TORRENT_ID" ]; then
|
||||||
|
echo "Processing torrent ID: $TORRENT_ID"
|
||||||
|
|
||||||
|
# Get torrent details
|
||||||
|
info=$(transmission-remote "${TRANSMISSION_IP}:${TRANSMISSION_PORT}" \
|
||||||
|
--auth "${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}" \
|
||||||
|
--torrent $TORRENT_ID --info)
|
||||||
|
|
||||||
|
name=$(echo "$info" | grep "Name:" | awk -F": " '{print $2}' | xargs)
|
||||||
|
echo "Torrent name: $name"
|
||||||
|
|
||||||
|
# Run torrent-mover with specific torrent ID
|
||||||
|
torrent_id="$TORRENT_ID"
|
||||||
|
|
||||||
|
# Check if output directory exists for this torrent
|
||||||
|
output_dir=$(grep "Location:" <<< "$info" | awk -F": " '{print $2}' | xargs)
|
||||||
|
if [[ -n "$output_dir" ]]; then
|
||||||
|
echo "Torrent location: $output_dir"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# We need to modify torrent-mover to handle single IDs
|
||||||
|
# For now, we'll write a small temporary script to process just this ID
|
||||||
|
TMP_SCRIPT=$(mktemp)
|
||||||
|
cat > "$TMP_SCRIPT" << EOF
|
||||||
|
#!/bin/bash
|
||||||
|
source /etc/torrent/mover.conf
|
||||||
|
source /usr/local/lib/torrent-mover/common.sh
|
||||||
|
source /usr/local/lib/torrent-mover/file_operations.sh
|
||||||
|
source /usr/local/lib/torrent-mover/transmission_handler.sh
|
||||||
|
source /usr/local/lib/torrent-mover/archive_handler.sh
|
||||||
|
|
||||||
|
# Set debug mode
|
||||||
|
DEBUG=1
|
||||||
|
|
||||||
|
# Process just this one torrent
|
||||||
|
process_single_torrent() {
|
||||||
|
local id="\$1"
|
||||||
|
log_debug "Processing single torrent ID: \$id"
|
||||||
|
|
||||||
|
# Get torrent info
|
||||||
|
local info cmd
|
||||||
|
cmd="transmission-remote \${TRANSMISSION_IP}:\${TRANSMISSION_PORT} -n \${TRANSMISSION_USER}:\${TRANSMISSION_PASSWORD} -t \${id} -i"
|
||||||
|
info=\$(eval "\$cmd")
|
||||||
|
|
||||||
|
if [[ -z "\$info" ]]; then
|
||||||
|
log_error "Failed to get info for torrent \$id"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Process torrent info just like in the main script
|
||||||
|
local hash
|
||||||
|
hash=\$(grep "Hash:" <<< "\${info}" | awk '{print \$2}')
|
||||||
|
local ratio
|
||||||
|
ratio=\$(grep "Ratio:" <<< "\${info}" | awk '{print \$2 == "None" ? 0 : \$2}' | tr -cd '0-9.')
|
||||||
|
ratio=\${ratio:-0}
|
||||||
|
local time
|
||||||
|
time=\$(grep "Seeding Time:" <<< "\${info}" | awk '{print \$3 == "None" ? 0 : \$3}' | tr -cd '0-9.')
|
||||||
|
time=\${time:-0}
|
||||||
|
local percent_done
|
||||||
|
percent_done=\$(grep "Percent Done:" <<< "\${info}" | awk '{gsub(/%/, ""); print \$3 == "None" ? 0 : \$3}')
|
||||||
|
percent_done=\${percent_done:-0}
|
||||||
|
|
||||||
|
# Extract Transmission-reported directory and translate to local path.
|
||||||
|
local reported_dir
|
||||||
|
reported_dir=\$(grep -i "Location:" <<< "\${info}" | awk -F": " '{print \$2}' | xargs)
|
||||||
|
log_debug "Raw reported directory: '\${reported_dir}'"
|
||||||
|
|
||||||
|
# If the reported directory is empty, try to derive it from the name
|
||||||
|
if [[ -z "\${reported_dir}" ]]; then
|
||||||
|
local name
|
||||||
|
name=\$(grep -i "Name:" <<< "\${info}" | awk -F": " '{print \$2}' | xargs)
|
||||||
|
log_debug "Torrent name: '\${name}'"
|
||||||
|
|
||||||
|
# Check if there are labels we can use
|
||||||
|
local labels
|
||||||
|
labels=\$(grep -i "Labels:" <<< "\${info}" | awk -F": " '{print \$2}' | xargs)
|
||||||
|
log_debug "Torrent labels: '\${labels}'"
|
||||||
|
|
||||||
|
if [[ "\${labels}" == *"Books"* ]]; then
|
||||||
|
reported_dir="/downloads/Books"
|
||||||
|
elif [[ "\${labels}" == *"Movies"* ]]; then
|
||||||
|
reported_dir="/downloads/Movies"
|
||||||
|
elif [[ "\${labels}" == *"TV"* ]]; then
|
||||||
|
reported_dir="/downloads/TV"
|
||||||
|
elif [[ "\${labels}" == *"Games"* ]]; then
|
||||||
|
reported_dir="/downloads/Games"
|
||||||
|
elif [[ "\${labels}" == *"Apps"* ]]; then
|
||||||
|
reported_dir="/downloads/Apps"
|
||||||
|
elif [[ "\${labels}" == *"Music"* ]]; then
|
||||||
|
reported_dir="/downloads/Music"
|
||||||
|
else
|
||||||
|
# Default to Other if we can't determine
|
||||||
|
reported_dir="/downloads/Other"
|
||||||
|
fi
|
||||||
|
log_debug "Derived directory from labels: '\${reported_dir}'"
|
||||||
|
fi
|
||||||
|
|
||||||
|
local dir
|
||||||
|
dir=\$(translate_source "\${reported_dir}")
|
||||||
|
log_info "Torrent source directory: '\${reported_dir}' translated to '\${dir}'"
|
||||||
|
|
||||||
|
# Initialize empty directory mapping if needed
|
||||||
|
if [[ -z "\$dir" ]]; then
|
||||||
|
log_warn "Empty directory path detected, using default"
|
||||||
|
dir="\${LOCAL_PATH_PREFIX}/Other"
|
||||||
|
fi
|
||||||
|
|
||||||
|
local dst
|
||||||
|
dst=\$(get_destination "\${dir}")
|
||||||
|
|
||||||
|
# Process the torrent
|
||||||
|
if (( \$(bc <<< "\${percent_done} >= 100") )) && ! is_processed "\${hash}"; then
|
||||||
|
log_info "Processing completed torrent \${id} (\${percent_done}% done)"
|
||||||
|
process_copy "\${id}" "\${hash}" "\${dir}" "\${dst}"
|
||||||
|
else
|
||||||
|
log_info "Torrent \${id} already processed or not complete"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check seed ratio/time criteria
|
||||||
|
if (( \$(bc <<< "\${ratio} >= \${SEED_RATIO}") )) || (( \$(bc <<< "\${time} >= \${SEED_TIME}") )); then
|
||||||
|
log_info "Removing torrent \${id} (Ratio: \${ratio}, Time: \${time})"
|
||||||
|
process_removal "\${id}"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
# Main function
|
||||||
|
process_single_torrent "$torrent_id"
|
||||||
|
EOF
|
||||||
|
|
||||||
|
chmod +x "$TMP_SCRIPT"
|
||||||
|
"$TMP_SCRIPT"
|
||||||
|
rm -f "$TMP_SCRIPT"
|
||||||
|
elif [ -n "$CATEGORY" ]; then
|
||||||
|
echo "Processing category: $CATEGORY"
|
||||||
|
|
||||||
|
# Set category-specific filter
|
||||||
|
CATEGORY_PATH=""
|
||||||
|
PATTERN=""
|
||||||
|
case $CATEGORY in
|
||||||
|
books)
|
||||||
|
echo "Looking for book torrents..."
|
||||||
|
CATEGORY_PATH="/downloads/Books"
|
||||||
|
PATTERN="*books*|*ebook*|*epub*|*pdf*"
|
||||||
|
;;
|
||||||
|
movies)
|
||||||
|
echo "Looking for movie torrents..."
|
||||||
|
CATEGORY_PATH="/downloads/Movies"
|
||||||
|
PATTERN="*movies*|*film*|*video*"
|
||||||
|
;;
|
||||||
|
tv)
|
||||||
|
echo "Looking for TV show torrents..."
|
||||||
|
CATEGORY_PATH="/downloads/TV"
|
||||||
|
PATTERN="*tv*|*series*|*episode*"
|
||||||
|
;;
|
||||||
|
apps)
|
||||||
|
echo "Looking for application torrents..."
|
||||||
|
CATEGORY_PATH="/downloads/Apps"
|
||||||
|
PATTERN="*apps*|*applications*|*programs*|*software*"
|
||||||
|
;;
|
||||||
|
games)
|
||||||
|
echo "Looking for game torrents..."
|
||||||
|
CATEGORY_PATH="/downloads/Games"
|
||||||
|
PATTERN="*games*"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# Create a script to process just this category
|
||||||
|
TMP_SCRIPT=$(mktemp)
|
||||||
|
cat > "$TMP_SCRIPT" << EOF
|
||||||
|
#!/bin/bash
|
||||||
|
source /etc/torrent/mover.conf
|
||||||
|
source /usr/local/lib/torrent-mover/common.sh
|
||||||
|
source /usr/local/lib/torrent-mover/file_operations.sh
|
||||||
|
source /usr/local/lib/torrent-mover/transmission_handler.sh
|
||||||
|
source /usr/local/lib/torrent-mover/archive_handler.sh
|
||||||
|
|
||||||
|
# Set debug mode
|
||||||
|
DEBUG=1
|
||||||
|
|
||||||
|
# Get all torrents
|
||||||
|
get_torrent_ids() {
|
||||||
|
local cmd="transmission-remote \${TRANSMISSION_IP}:\${TRANSMISSION_PORT} -n \${TRANSMISSION_USER}:\${TRANSMISSION_PASSWORD} -l"
|
||||||
|
local output
|
||||||
|
output=\$(retry_command "\$cmd" 3 20)
|
||||||
|
echo "\$output" | awk 'NR>1 && NF>1 {gsub(/^[ ]+/, "", \$1); if (\$1 ~ /^[0-9]+\$/) print \$1}'
|
||||||
|
}
|
||||||
|
|
||||||
|
# Process category torrents
|
||||||
|
process_category_torrents() {
|
||||||
|
local category_path="$CATEGORY_PATH"
|
||||||
|
local pattern="$PATTERN"
|
||||||
|
log_debug "Processing category: $CATEGORY with path \$category_path and pattern '\$pattern'"
|
||||||
|
|
||||||
|
# Get list of all torrents
|
||||||
|
local torrent_ids=\$(get_torrent_ids)
|
||||||
|
|
||||||
|
# Process each torrent
|
||||||
|
for id in \$torrent_ids; do
|
||||||
|
# Get torrent info
|
||||||
|
local info cmd
|
||||||
|
cmd="transmission-remote \${TRANSMISSION_IP}:\${TRANSMISSION_PORT} -n \${TRANSMISSION_USER}:\${TRANSMISSION_PASSWORD} -t \${id} -i"
|
||||||
|
info=\$(eval "\$cmd")
|
||||||
|
|
||||||
|
if [[ -z "\$info" ]]; then
|
||||||
|
log_warn "Failed to get info for torrent \$id, skipping"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Extract name and location
|
||||||
|
local name=\$(grep -i "Name:" <<< "\$info" | awk -F": " '{print \$2}' | xargs)
|
||||||
|
local reported_dir=\$(grep -i "Location:" <<< "\$info" | awk -F": " '{print \$2}' | xargs)
|
||||||
|
local labels=\$(grep -i "Labels:" <<< "\$info" | awk -F": " '{print \$2}' | xargs)
|
||||||
|
|
||||||
|
# Check if this torrent matches our category
|
||||||
|
if [[ "\$reported_dir" == "\$category_path" ]] ||
|
||||||
|
[[ "\$labels" == *"$CATEGORY"* ]] ||
|
||||||
|
[[ "\$name" =~ \$pattern ]]; then
|
||||||
|
|
||||||
|
log_info "Found matching torrent: \$id - \$name"
|
||||||
|
|
||||||
|
# Process torrent info
|
||||||
|
local hash=\$(grep "Hash:" <<< "\$info" | awk '{print \$2}')
|
||||||
|
local ratio=\$(grep "Ratio:" <<< "\$info" | awk '{print \$2 == "None" ? 0 : \$2}' | tr -cd '0-9.')
|
||||||
|
ratio=\${ratio:-0}
|
||||||
|
local time=\$(grep "Seeding Time:" <<< "\$info" | awk '{print \$3 == "None" ? 0 : \$3}' | tr -cd '0-9.')
|
||||||
|
time=\${time:-0}
|
||||||
|
local percent_done=\$(grep "Percent Done:" <<< "\$info" | awk '{gsub(/%/, ""); print \$3 == "None" ? 0 : \$3}')
|
||||||
|
percent_done=\${percent_done:-0}
|
||||||
|
|
||||||
|
# If the reported directory is empty, derive it
|
||||||
|
if [[ -z "\$reported_dir" ]]; then
|
||||||
|
reported_dir="\$category_path"
|
||||||
|
log_debug "Using derived directory: '\$reported_dir'"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Process the torrent
|
||||||
|
local dir=\$(translate_source "\$reported_dir")
|
||||||
|
log_info "Torrent source directory: '\$reported_dir' translated to '\$dir'"
|
||||||
|
|
||||||
|
# Initialize empty directory mapping if needed
|
||||||
|
if [[ -z "\$dir" ]]; then
|
||||||
|
log_warn "Empty directory path detected, using default"
|
||||||
|
dir="\${LOCAL_PATH_PREFIX}/$CATEGORY"
|
||||||
|
fi
|
||||||
|
|
||||||
|
local dst=\$(get_destination "\$dir")
|
||||||
|
|
||||||
|
# Process the torrent
|
||||||
|
if (( \$(bc <<< "\${percent_done} >= 100") )) && ! is_processed "\${hash}"; then
|
||||||
|
log_info "Processing completed torrent \${id} (\${percent_done}% done)"
|
||||||
|
process_copy "\${id}" "\${hash}" "\${dir}" "\${dst}"
|
||||||
|
else
|
||||||
|
log_info "Torrent \${id} already processed or not complete"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check seed ratio/time criteria
|
||||||
|
if (( \$(bc <<< "\${ratio} >= \${SEED_RATIO}") )) || (( \$(bc <<< "\${time} >= \${SEED_TIME}") )); then
|
||||||
|
log_info "Removing torrent \${id} (Ratio: \${ratio}, Time: \${time})"
|
||||||
|
process_removal "\${id}"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# Main function
|
||||||
|
process_category_torrents
|
||||||
|
EOF
|
||||||
|
|
||||||
|
chmod +x "$TMP_SCRIPT"
|
||||||
|
"$TMP_SCRIPT"
|
||||||
|
rm -f "$TMP_SCRIPT"
|
||||||
|
else
|
||||||
|
echo "Processing all torrents"
|
||||||
|
# Run the main torrent-mover script directly
|
||||||
|
/usr/local/bin/torrent-mover --debug
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "Processing complete\!"
|
||||||
|
echo "Check /var/log/torrent_mover.log for details"
|
@ -18,23 +18,31 @@ declare -A PATH_CACHE
|
|||||||
log_debug() {
|
log_debug() {
|
||||||
if [[ "${DEBUG}" -eq 1 ]]; then
|
if [[ "${DEBUG}" -eq 1 ]]; then
|
||||||
echo -e "[DEBUG] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
echo -e "[DEBUG] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
||||||
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[DEBUG] $*"
|
if [[ "${USE_SYSLOG}" == "true" ]]; then
|
||||||
|
logger -t torrent-mover "[DEBUG] $*" || true
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
log_info() {
|
log_info() {
|
||||||
echo -e "[INFO] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
echo -e "[INFO] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
||||||
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[INFO] $*"
|
if [[ "${USE_SYSLOG}" == "true" ]]; then
|
||||||
|
logger -t torrent-mover "[INFO] $*" || true
|
||||||
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
log_warn() {
|
log_warn() {
|
||||||
echo -e "[WARN] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
echo -e "[WARN] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
||||||
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[WARN] $*"
|
if [[ "${USE_SYSLOG}" == "true" ]]; then
|
||||||
|
logger -t torrent-mover "[WARN] $*" || true
|
||||||
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
log_error() {
|
log_error() {
|
||||||
echo -e "[ERROR] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
echo -e "[ERROR] $(date '+%F %T') - $*" | tee -a "${LOG_FILE}" >&2
|
||||||
[[ "${USE_SYSLOG}" == "true" ]] && logger -t torrent-mover "[ERROR] $*"
|
if [[ "${USE_SYSLOG}" == "true" ]]; then
|
||||||
|
logger -t torrent-mover "[ERROR] $*" || true
|
||||||
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
# Error Handling & Notifications
|
# Error Handling & Notifications
|
||||||
@ -93,20 +101,42 @@ check_dependencies() {
|
|||||||
check_disk_usage() {
|
check_disk_usage() {
|
||||||
local dir="$1"
|
local dir="$1"
|
||||||
[[ -z "${dir}" ]] && return
|
[[ -z "${dir}" ]] && return
|
||||||
|
|
||||||
|
log_debug "Checking disk usage for directory: ${dir}"
|
||||||
|
|
||||||
if ! df -P "${dir}" &>/dev/null; then
|
if ! df -P "${dir}" &>/dev/null; then
|
||||||
log_warn "Directory not found: ${dir}"
|
log_warn "Directory not found: ${dir}"
|
||||||
return
|
return
|
||||||
fi
|
fi
|
||||||
|
|
||||||
local mount_point
|
local mount_point
|
||||||
mount_point=$(df -P "${dir}" | awk 'NR==2 {print $6}')
|
mount_point=$(df -P "${dir}" | awk 'NR==2 {print $6}')
|
||||||
[[ -z "${mount_point}" ]] && return
|
|
||||||
if [[ -z "${CHECKED_MOUNTS["${mount_point}"]+x}" ]]; then
|
if [[ -z "${mount_point}" ]]; then
|
||||||
|
log_warn "Could not determine mount point for: ${dir}"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
log_debug "Mount point for ${dir} is ${mount_point}"
|
||||||
|
|
||||||
|
# Initialize CHECKED_MOUNTS as an empty array if not already done
|
||||||
|
if [[ -z "${CHECKED_MOUNTS+x}" ]]; then
|
||||||
|
declare -A CHECKED_MOUNTS
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check if we've already checked this mount point
|
||||||
|
if [[ -z "${CHECKED_MOUNTS[${mount_point}]+x}" ]]; then
|
||||||
local usage
|
local usage
|
||||||
usage=$(df -P "${dir}" | awk 'NR==2 {sub(/%/, "", $5); print $5}')
|
usage=$(df -P "${dir}" | awk 'NR==2 {sub(/%/, "", $5); print $5}')
|
||||||
|
log_debug "Usage for ${mount_point}: ${usage}%"
|
||||||
|
|
||||||
if (( usage >= 90 )); then
|
if (( usage >= 90 )); then
|
||||||
log_warn "Storage warning: ${mount_point} at ${usage}% capacity"
|
log_warn "Storage warning: ${mount_point} at ${usage}% capacity"
|
||||||
fi
|
fi
|
||||||
CHECKED_MOUNTS["${mount_point}"]=1
|
|
||||||
|
CHECKED_MOUNTS[${mount_point}]=1
|
||||||
|
else
|
||||||
|
log_debug "Mount point ${mount_point} already checked"
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -131,44 +161,59 @@ retry_command() {
|
|||||||
local wait_time="${3:-10}" # Default to 10 seconds wait between attempts
|
local wait_time="${3:-10}" # Default to 10 seconds wait between attempts
|
||||||
local attempt=1
|
local attempt=1
|
||||||
local exit_code=0
|
local exit_code=0
|
||||||
local error_output=""
|
local command_output=""
|
||||||
|
|
||||||
# Create a temporary file for capturing error output
|
# Create a temporary file for capturing output
|
||||||
local error_file
|
local output_file
|
||||||
error_file=$(mktemp)
|
output_file=$(mktemp)
|
||||||
|
|
||||||
|
# Use a more verbose logging for this command - always log, not just in debug mode
|
||||||
|
log_info "Executing command: $cmd"
|
||||||
|
|
||||||
while (( attempt <= max_attempts )); do
|
while (( attempt <= max_attempts )); do
|
||||||
log_debug "Attempt $attempt of $max_attempts: $cmd"
|
log_info "Attempt $attempt of $max_attempts: $cmd"
|
||||||
|
|
||||||
# Execute command and capture both exit code and stderr
|
# Execute command directly and capture output and exit code
|
||||||
error_output=$( { eval "$cmd"; exit_code=$?; } 2>&1 > >(tee /dev/stderr) )
|
command_output=$(eval "$cmd" 2>&1)
|
||||||
|
exit_code=$?
|
||||||
|
echo "$command_output" > "${output_file}"
|
||||||
|
|
||||||
|
# Always log the first 10 lines of output
|
||||||
|
log_info "Command output (first 10 lines):"
|
||||||
|
head -n 10 "${output_file}" | while IFS= read -r line; do
|
||||||
|
log_info " > $line"
|
||||||
|
done
|
||||||
|
|
||||||
if [[ ${exit_code} -eq 0 ]]; then
|
if [[ ${exit_code} -eq 0 ]]; then
|
||||||
log_debug "Command succeeded on attempt $attempt"
|
log_info "Command succeeded on attempt $attempt"
|
||||||
rm -f "${error_file}"
|
rm -f "${output_file}"
|
||||||
|
echo "$command_output"
|
||||||
return 0
|
return 0
|
||||||
else
|
else
|
||||||
# Log detailed error information
|
# Log detailed error information
|
||||||
echo "${error_output}" > "${error_file}"
|
|
||||||
log_warn "Command failed (attempt $attempt, exit code: ${exit_code})"
|
log_warn "Command failed (attempt $attempt, exit code: ${exit_code})"
|
||||||
log_debug "Error details: $(head -n 5 "${error_file}")"
|
|
||||||
|
|
||||||
if (( attempt == max_attempts )); then
|
if (( attempt == max_attempts )); then
|
||||||
log_error "Maximum attempts reached for command, last exit code: ${exit_code}"
|
log_error "Maximum attempts reached for command, last exit code: ${exit_code}"
|
||||||
log_error "Last error output: $(head -n 10 "${error_file}")"
|
log_error "Last error output (first 10 lines):"
|
||||||
rm -f "${error_file}"
|
head -n 10 "${output_file}" | while IFS= read -r line; do
|
||||||
|
log_error " > $line"
|
||||||
|
done
|
||||||
|
rm -f "${output_file}"
|
||||||
|
echo "$command_output"
|
||||||
return ${exit_code}
|
return ${exit_code}
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Exponential backoff - wait longer for each successive attempt
|
# Exponential backoff - wait longer for each successive attempt
|
||||||
local adjusted_wait=$((wait_time * attempt))
|
local adjusted_wait=$((wait_time * attempt))
|
||||||
log_debug "Waiting ${adjusted_wait} seconds before retry"
|
log_info "Waiting ${adjusted_wait} seconds before retry"
|
||||||
sleep ${adjusted_wait}
|
sleep ${adjusted_wait}
|
||||||
(( attempt++ ))
|
(( attempt++ ))
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
|
|
||||||
rm -f "${error_file}"
|
rm -f "${output_file}"
|
||||||
|
echo "$command_output"
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@ -215,6 +215,13 @@ process_copy() {
|
|||||||
local id="$1" hash="$2" src="$3" dst="$4"
|
local id="$1" hash="$2" src="$3" dst="$4"
|
||||||
local operation_result=0
|
local operation_result=0
|
||||||
|
|
||||||
|
# Check if source and destination are the same or if we've already processed this
|
||||||
|
if [[ "${src}" == "${dst}" ]]; then
|
||||||
|
log_info "Source and destination are the same - skipping: ${src}"
|
||||||
|
mark_processed "${hash}"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
if [[ ! -d "${src}" ]]; then
|
if [[ ! -d "${src}" ]]; then
|
||||||
log_error "Source directory missing: ${src}"
|
log_error "Source directory missing: ${src}"
|
||||||
return 1
|
return 1
|
||||||
|
@ -4,11 +4,45 @@
|
|||||||
# get_destination: Maps a source directory to a destination directory based on keywords and patterns
|
# get_destination: Maps a source directory to a destination directory based on keywords and patterns
|
||||||
get_destination() {
|
get_destination() {
|
||||||
local source_path="$1"
|
local source_path="$1"
|
||||||
|
|
||||||
|
# Check if source_path is valid before accessing the array
|
||||||
|
if [[ -z "${source_path}" ]]; then
|
||||||
|
log_warn "Empty source path provided to get_destination"
|
||||||
|
return "${DEFAULT_DST}"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check if path is already in the cache
|
||||||
if [[ -n "${PATH_CACHE["${source_path}"]+x}" ]]; then
|
if [[ -n "${PATH_CACHE["${source_path}"]+x}" ]]; then
|
||||||
echo "${PATH_CACHE["${source_path}"]}"
|
local cached_destination="${PATH_CACHE["${source_path}"]}"
|
||||||
|
log_debug "Using cached destination for ${source_path}: ${cached_destination}"
|
||||||
|
echo "${cached_destination}"
|
||||||
return
|
return
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# Skip recursive path analysis - only log once
|
||||||
|
if [[ "${source_path}" =~ ^/mnt/dsnas1/ ]]; then
|
||||||
|
# Already in destination format, return as is
|
||||||
|
log_debug "Path already in destination format: ${source_path}"
|
||||||
|
PATH_CACHE["${source_path}"]="${source_path}"
|
||||||
|
echo "${source_path}"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
# For paths in dsnas2, check if they map to same structure in dsnas1
|
||||||
|
if [[ "${source_path}" =~ ^/mnt/dsnas2/ ]]; then
|
||||||
|
local dir_suffix="${source_path#/mnt/dsnas2/}"
|
||||||
|
local potential_dest="/mnt/dsnas1/${dir_suffix}"
|
||||||
|
|
||||||
|
# If the directories match exactly in structure, only on different mounts,
|
||||||
|
# return the source to avoid needless copying
|
||||||
|
if [[ -d "${potential_dest}" ]]; then
|
||||||
|
log_debug "Path maps to same structure on different mount: ${source_path} -> ${source_path}"
|
||||||
|
PATH_CACHE["${source_path}"]="${source_path}"
|
||||||
|
echo "${source_path}"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
log_info "Analyzing path: ${source_path}"
|
log_info "Analyzing path: ${source_path}"
|
||||||
local destination="${DEFAULT_DST}"
|
local destination="${DEFAULT_DST}"
|
||||||
|
|
||||||
@ -56,7 +90,12 @@ get_destination() {
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
log_info "Mapped to: ${destination}"
|
log_info "Mapped to: ${destination}"
|
||||||
|
|
||||||
|
# Only set in cache if source_path is not empty
|
||||||
|
if [[ -n "${source_path}" ]]; then
|
||||||
PATH_CACHE["${source_path}"]="${destination}"
|
PATH_CACHE["${source_path}"]="${destination}"
|
||||||
|
fi
|
||||||
|
|
||||||
echo "${destination}"
|
echo "${destination}"
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -68,17 +107,95 @@ process_removal() {
|
|||||||
return
|
return
|
||||||
fi
|
fi
|
||||||
|
|
||||||
retry_command "transmission-remote \"${TRANSMISSION_IP}:${TRANSMISSION_PORT}\" -n \"${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}\" -t \"${id}\" --remove-and-delete" 3 15
|
local cmd="transmission-remote ${TRANSMISSION_IP}:${TRANSMISSION_PORT} -n ${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD} -t ${id} --remove-and-delete"
|
||||||
|
retry_command "$cmd" 3 15
|
||||||
}
|
}
|
||||||
|
|
||||||
# get_torrents: Retrieves a list of torrents from Transmission
|
# get_torrents: Retrieves a list of torrents from Transmission
|
||||||
get_torrents() {
|
get_torrents() {
|
||||||
retry_command "transmission-remote \"${TRANSMISSION_IP}:${TRANSMISSION_PORT}\" -n \"${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}\" -l" 3 20 |
|
# Log connection parameters (redacted password)
|
||||||
awk 'NR>1 && $1 ~ /^[0-9]+$/ {print $1}'
|
log_info "Transmission connection parameters:"
|
||||||
|
log_info " IP: ${TRANSMISSION_IP}:${TRANSMISSION_PORT}"
|
||||||
|
log_info " Username: ${TRANSMISSION_USER}"
|
||||||
|
log_info " Password: [redacted]"
|
||||||
|
|
||||||
|
# Try a direct command without using retry_command to get clearer error messages
|
||||||
|
log_info "Direct transmission-remote access test:"
|
||||||
|
local test_output
|
||||||
|
test_output=$(transmission-remote "${TRANSMISSION_IP}:${TRANSMISSION_PORT}" -n "${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}" -l 2>&1)
|
||||||
|
local test_exit=$?
|
||||||
|
if [[ $test_exit -ne 0 ]]; then
|
||||||
|
log_error "Direct transmission-remote test failed with exit code: $test_exit"
|
||||||
|
log_error "Error output: $test_output"
|
||||||
|
# Continue anyway to see retry attempt logs
|
||||||
|
else
|
||||||
|
log_info "Direct transmission-remote test succeeded"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Execute the actual command with retries
|
||||||
|
local real_cmd="transmission-remote ${TRANSMISSION_IP}:${TRANSMISSION_PORT} -n ${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD} -l"
|
||||||
|
local output
|
||||||
|
output=$(retry_command "$real_cmd" 3 20)
|
||||||
|
|
||||||
|
# Line-by-line raw output inspection (debugging)
|
||||||
|
log_info "Raw command output detailed analysis:"
|
||||||
|
if [[ -z "$output" ]]; then
|
||||||
|
log_error "Command produced EMPTY output"
|
||||||
|
else
|
||||||
|
log_info "Output length: $(echo "$output" | wc -l) lines"
|
||||||
|
echo "$output" | while IFS= read -r line; do
|
||||||
|
log_info " LINE: '$line'"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Extract IDs directly using awk with detailed debugging
|
||||||
|
log_info "Extracting torrent IDs from output..."
|
||||||
|
local line_num=0
|
||||||
|
local found_ids=0
|
||||||
|
echo "$output" | while IFS= read -r line; do
|
||||||
|
line_num=$((line_num + 1))
|
||||||
|
# Skip header line
|
||||||
|
if [[ $line_num -eq 1 ]]; then
|
||||||
|
log_info " Skipping header: '$line'"
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
# Check for torrent ID in first column
|
||||||
|
local potential_id
|
||||||
|
potential_id=$(echo "$line" | awk '{gsub(/^[ ]+/, "", $1); print $1}')
|
||||||
|
log_info " Line $line_num: potential ID '$potential_id'"
|
||||||
|
if [[ "$potential_id" =~ ^[0-9]+$ ]]; then
|
||||||
|
log_info " Found valid ID: $potential_id"
|
||||||
|
found_ids=$((found_ids + 1))
|
||||||
|
echo "$potential_id"
|
||||||
|
else
|
||||||
|
log_info " Not a valid ID: '$potential_id'"
|
||||||
|
fi
|
||||||
|
done | tee /tmp/torrent_ids.txt
|
||||||
|
|
||||||
|
# Read back the file to get around pipe subshell issues
|
||||||
|
local torrent_ids
|
||||||
|
torrent_ids=$(cat /tmp/torrent_ids.txt)
|
||||||
|
rm -f /tmp/torrent_ids.txt
|
||||||
|
|
||||||
|
# Check if we found any torrents
|
||||||
|
if [[ -z "$torrent_ids" ]]; then
|
||||||
|
log_error "NO TORRENT IDs FOUND in transmission output"
|
||||||
|
else
|
||||||
|
log_info "Found torrent IDs: $torrent_ids"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Fallback to hardcoded ID for testing if nothing found
|
||||||
|
if [[ -z "$torrent_ids" && "${DRY_RUN}" -eq 1 ]]; then
|
||||||
|
log_info "DRY RUN MODE: Adding test torrent ID 1 for debugging"
|
||||||
|
echo "1"
|
||||||
|
else
|
||||||
|
echo "$torrent_ids"
|
||||||
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
# get_torrent_info: Gets detailed info for a specific torrent
|
# get_torrent_info: Gets detailed info for a specific torrent
|
||||||
get_torrent_info() {
|
get_torrent_info() {
|
||||||
local id="$1"
|
local id="$1"
|
||||||
retry_command "transmission-remote \"${TRANSMISSION_IP}:${TRANSMISSION_PORT}\" -n \"${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD}\" -t \"${id}\" -i" 3 15
|
local cmd="transmission-remote ${TRANSMISSION_IP}:${TRANSMISSION_PORT} -n ${TRANSMISSION_USER}:${TRANSMISSION_PASSWORD} -t ${id} -i"
|
||||||
|
retry_command "$cmd" 3 15
|
||||||
}
|
}
|
Loading…
x
Reference in New Issue
Block a user