Site Tools


wiki:shell-file-operations

Shell File Operations

File Tests

Use conditional tests to check file properties. These tests are most readable within [[ ]] conditionals.

# Test existence and type
[[ -f /etc/passwd ]]     && echo "Regular file exists"
[[ -d /home ]]           && echo "Directory exists"
[[ -L /etc/default ]]    && echo "Symbolic link exists"
[[ -b /dev/sda ]]        && echo "Block device exists"
 
# Test permissions and attributes
[[ -r /etc/passwd ]]     && echo "File is readable"
[[ -w /tmp ]]            && echo "Directory is writable"
[[ -x /usr/bin/bash ]]   && echo "File is executable"
[[ -s /var/log/syslog ]] && echo "File exists and is non-empty"
 
# Test file relationships
[[ file1 -nt file2 ]]    && echo "file1 is newer than file2"
[[ file1 -ot file2 ]]    && echo "file1 is older than file2"
[[ file1 -ef file2 ]]    && echo "file1 and file2 are the same inode"

Combine tests with logical operators:

# Check if file exists and is readable
if [[ -f "$file" && -r "$file" ]]; then
    echo "Can read $file"
fi
 
# Check if file is NOT executable
if [[ ! -x "$script" ]]; then
    chmod +x "$script"
fi
 
# Check for directory AND it's writable
if [[ -d "$dir" && -w "$dir" ]]; then
    touch "$dir/test.txt"
fi

Path Operations

Work with file paths without spawning external commands using basename, dirname, and parameter expansion.

# Extract filename from path
path="/home/user/documents/report.pdf"
basename "$path"         # Output: report.pdf
basename "$path" .pdf    # Output: report (strip suffix)
 
# Extract directory from path
dirname "$path"          # Output: /home/user/documents
 
# Resolve symlinks to absolute path
readlink -f /etc/default  # Follows symlinks and prints absolute path
 
# Get current working directory
pwd
cd /some/path && echo $PWD

Combine operations to manipulate paths:

# Get absolute path of a file
file="script.sh"
absolute_path="$(cd "$(dirname "$file")" && pwd -P)/$(basename "$file")"
 
# Check if path is absolute
if [[ "$path" =~ ^/ ]]; then
    echo "Absolute path"
else
    echo "Relative path"
fi
 
# Remove trailing slash
path="/home/user/"
path=${path%/}   # Output: /home/user

Working with Text Files

Extract and process content from files using standard text utilities.

# Print first and last lines
head -n 5 myfile.txt     # First 5 lines
tail -n 5 myfile.txt     # Last 5 lines
 
# Follow file updates in real-time (useful for logs)
tail -f /var/log/syslog
 
# Extract specific field (colon-delimited)
cut -d: -f1 /etc/passwd  # List all usernames
 
# Extract multiple fields
cut -d: -f1,3 /etc/passwd  # Username and UID
 
# Sort lines
sort myfile.txt
sort -n numbers.txt      # Numeric sort
sort -r myfile.txt       # Reverse sort
 
# Remove duplicate lines (must be sorted)
sort myfile.txt | uniq
uniq -c myfile.txt       # Print count of duplicates
 
# Translate or delete characters
tr 'a-z' 'A-Z' < input.txt   # Uppercase
tr -d '[:space:]' < input.txt # Remove whitespace
 
# Search for lines matching pattern
grep "error" logfile.txt
grep -i "ERROR" logfile.txt  # Case-insensitive
grep -v "debug" logfile.txt  # Invert match (exclude pattern)
grep -c "error" logfile.txt  # Count matching lines

Combine tools in a pipeline for complex processing:

# Find lines, count unique values, sort by count
grep "GET" access.log | cut -d' ' -f7 | sort | uniq -c | sort -rn
 
# Process configuration file: remove comments and empty lines
grep -v '^#' /etc/config.conf | grep -v '^$'
 
# Extract and count field values
cut -d: -f3 /etc/passwd | sort -n | uniq -c | sort -rn
wiki/shell-file-operations.md · Last modified: by 127.0.0.1