“I need to update the API endpoint in all our configuration files.” This seemingly simple task can become a significant challenge when dealing with dozens or hundreds of files across multiple directories. Manual editing is tedious and error-prone, while specialized tools might not be available in all environments.

Enter sed, the powerful stream editor available on virtually every Unix-like system. While many developers know sed can replace text in a single file, its real power emerges when applied to multiple files systematically.

After years of DevOps and system administration work, I’ve refined a set of reliable patterns for multi-file text replacement using sed. In this comprehensive guide, I’ll share these techniques, from basic replacements to complex pattern matching across large codebases.

Understanding the Fundamentals

Before diving into multi-file operations, let’s ensure we understand how sed handles single-file replacements. As Robert Love explains in “Linux System Programming”, sed processes text line by line, applying specified operations to each line in sequence.

The basic syntax for text replacement is:

sed 's/old_text/new_text/' file.txt 

However, this only prints the modified content to standard output without changing the file. For in-place editing, we need the -i flag:

sed -i 's/old_text/new_text/' file.txt 

According to Brian Ward’s “How Linux Works”, the -i option is one of sed’s most powerful features for system administration tasks, as it enables direct modification of configuration files.

Method 1: Using Shell Globbing for Multiple Files

The simplest approach to multi-file operations uses shell globbing patterns to select files.

Basic Globbing Example

# Replace "old_api.example.com" with "new_api.example.com" in all .json files sed -i 's/old_api\.example\.com/new_api.example.com/g' *.json 

This command:

1. Selects all files with .json extension in the current directory
2. Replaces all occurrences of old_api.example.com with new_api.example.com
3. The g flag ensures all occurrences in each line are replaced, not just the first

Using Globbing with Directories

For nested directories, you can use extended globbing features if your shell supports them:

# In Bash with globstar option enabled shopt -s globstar sed -i 's/old_api\.example\.com/new_api.example.com/g' **/*.json 

This matches .json files in the current directory and all subdirectories.

Backup Before Replacing

“Unix Power Tools” emphasizes the importance of backups before bulk operations. The -i flag can create backups by appending a suffix:

# Create .bak backups before making changes sed -i.bak 's/old_api\.example\.com/new_api.example.com/g' *.json 

This generates a backup file (e.g., config.json.bak) for each modified file.

Method 2: Using find with sed for Precise Control

Shell globbing works for simple cases, but for more complex file selection, combining find with sed provides greater flexibility and power.

Basic find and sed Example

# Replace text in all .js files modified in the last 7 days find . -name "*.js" -mtime -7 -exec sed -i 's/const API_URL = ".*"/const API_URL = "https:\/\/new-api.example.com"/g' {} \; 

This pattern:

1. Uses find to locate .js files modified within the last 7 days
2. Executes sed on each matching file to update the API URL

Handling Spaces in Filenames

File paths with spaces can cause issues with simple approaches. Here’s a robust solution:

# Safely handle filenames with spaces find . -name "*.config" -print0 | xargs -0 sed -i 's/debug=false/debug=true/g' 

The -print0 and -0 flags ensure filenames are separated by null characters, safely handling spaces and special characters.

Filtering with grep Before Replacing

Sometimes you only want to modify files containing specific content. Combine find, grep, and sed:

# Only replace in files that contain the old text find . -name "*.yaml" -type f -exec grep -l "oldValue" {} \; | xargs sed -i 's/oldValue/newValue/g' 

This approach:

1. Uses find to locate all .yaml files
2. Filters with grep -l to get only files containing “oldValue”
3. Passes those filenames to sed for replacement

According to “Shell Scripting” guides, this pre-filtering significantly improves performance for large file sets by avoiding unnecessary processing.

Method 3: Using sed with Process Substitution

For more complex scenarios, you can combine sed with process substitution:

# Replace in all files listed in a manifest sed -i 's/VERSION=.*/VERSION="2.0.1"/' $(cat files_to_update.txt) 

This reads filenames from files_to_update.txt and applies the replacement to each.

You can also dynamically generate the file list:

# Replace in all non-binary files containing a specific string sed -i 's/api\.v1\./api.v2./g' $(grep -l -r --include="*.php" "api.v1." .) 

This searches recursively for .php files containing “api.v1.” and replaces the text in all matches.

Advanced Sed Techniques for Multi-File Operations

With the basics covered, let’s explore more powerful techniques for complex scenarios.

Regular Expression Replacements

For pattern-based replacements, use sed’s regular expression capabilities:

# Replace all phone numbers with a specific format find . -name "*.html" -exec sed -i -E 's/\(([0-9]{3})\) ([0-9]{3})-([0-9]{4})/\1-\2-\3/g' {} \; 

This converts phone numbers from (123) 456-7890 format to 123-456-7890 format.

The -E flag enables extended regular expressions, making the pattern more readable.

Replacing Across Line Boundaries

Sometimes the text you need to replace spans multiple lines. Here’s how to handle that:

# Replace a multi-line XML tag find . -name "*.xml" -exec sed -i '/<settings>/,/<\/settings>/{ s/<debug>false<\/debug>/<debug>true<\/debug>/g }' {} \; 

This only applies the replacement within the <settings> tags.

Conditional Replacements

For more targeted changes, use conditional logic:

# Replace only in production configuration files find . -name "*.conf" -exec sed -i '/\[production\]/,/\[/{ s/debug=true/debug=false/g }' {} \; 

This only changes debug=true to debug=false within the [production] section of configuration files.

Real-World Examples

Let’s apply these techniques to common scenarios developers and system administrators face.

Example 1: Updating Dependencies in a Project

#!/bin/bash # Update a dependency version across a project  OLD_VERSION="1.2.3" NEW_VERSION="2.0.0" DEPENDENCY_NAME="example-lib"  echo "Updating $DEPENDENCY_NAME from v$OLD_VERSION to v$NEW_VERSION..."  # Create a timestamped backup directory BACKUP_DIR="backup_$(date +%Y%m%d_%H%M%S)" mkdir -p "$BACKUP_DIR"  # Update package.json files find . -name "package.json" -exec grep -l "\"$DEPENDENCY_NAME\"" {} \; | while read file; do  # Create backup  cp "$file" "$BACKUP_DIR/$(basename "$file").$(date +%s)"   # Update the dependency version  sed -i "s/\"$DEPENDENCY_NAME\": \".*$OLD_VERSION.*\"/\"$DEPENDENCY_NAME\": \"^$NEW_VERSION\"/g" "$file"   echo "Updated $file" done  # Update import statements in code find . -name "*.js" -o -name "*.ts" | xargs grep -l "$DEPENDENCY_NAME" | while read file; do  # Create backup  cp "$file" "$BACKUP_DIR/$(basename "$file").$(date +%s)"   # Update require statements  sed -i "s/require(['\"]$DEPENDENCY_NAME\/v$OLD_VERSION['\"])/require('$DEPENDENCY_NAME\/v$NEW_VERSION')/g" "$file"   # Update import statements  sed -i "s/from ['\"]$DEPENDENCY_NAME\/v$OLD_VERSION['\"])/from '$DEPENDENCY_NAME\/v$NEW_VERSION'/g" "$file"   echo "Updated $file" done  echo "Update complete. Backups saved to $BACKUP_DIR/" 

This script:

1. Creates a timestamped backup directory
2. Updates version references in package.json files
3. Updates import/require statements in JavaScript and TypeScript files
4. Provides feedback on each updated file

Example 2: Changing Configuration Across Environments

#!/bin/bash # Update database connection strings across multiple environments  OLD_DB_HOST="db-old.internal" NEW_DB_HOST="db-new.internal"  # Files to check (using find for flexibility) CONFIG_FILES=$(find ./config -type f -name "*.properties" -o -name "*.xml" -o -name "*.yaml")  # Check if any files were found if [ -z "$CONFIG_FILES" ]; then  echo "No configuration files found!"  exit 1 fi  echo "The following files will be modified:" echo "$CONFIG_FILES" echo  # Prompt for confirmation read -p "Do you want to proceed? (y/n): " confirm if [[ "$confirm" != "y" && "$confirm" != "Y" ]]; then  echo "Operation cancelled."  exit 0 fi  # Create backup directory BACKUP_DIR="./config_backup_$(date +%Y%m%d)" mkdir -p "$BACKUP_DIR" echo "Creating backups in $BACKUP_DIR"  # Process each file echo "$CONFIG_FILES" | while read -r file; do  # Skip if file doesn't exist (might have been moved/deleted)  [ ! -f "$file" ] && continue   # Create backup  cp "$file" "$BACKUP_DIR/$(basename "$file").bak"   # Check file type and use appropriate replacement strategy  case "$file" in  *.properties)  # Handle Java properties files  sed -i "s/$OLD_DB_HOST/$NEW_DB_HOST/g" "$file"  ;;  *.xml)  # Handle XML files, being careful with escaping  sed -i "s|<url>jdbc:.*$OLD_DB_HOST|<url>jdbc:mysql://$NEW_DB_HOST|g" "$file"  sed -i "s|<host>$OLD_DB_HOST</host>|<host>$NEW_DB_HOST</host>|g" "$file"  ;;  *.yaml|*.yml)  # Handle YAML files  sed -i "s/host: $OLD_DB_HOST/host: $NEW_DB_HOST/g" "$file"  ;;  esac   echo "Updated $file" done  echo "Operation completed successfully." 

This script:

1. Finds configuration files of different types
2. Asks for confirmation before proceeding
3. Creates a dated backup of all files
4. Uses different sed patterns based on file type
5. Provides feedback for each modified file

Example 3: Code Refactoring Across a Codebase

#!/bin/bash # Refactor deprecated API calls across a codebase  OLD_API_PATTERN="getUserData\([^)]*\)" NEW_API_PATTERN="fetchUserInfo\(\1\)"  # Find all JavaScript and TypeScript files CODE_FILES=$(find ./src -type f -name "*.js" -o -name "*.ts" | grep -v "node_modules" | grep -v "dist")  echo "Scanning $(echo "$CODE_FILES" | wc -l) files for deprecated API usage..."  # First, create a report of all occurrences echo "Files containing deprecated API calls:" for file in $CODE_FILES; do  count=$(grep -c "$OLD_API_PATTERN" "$file" || true)  if [ "$count" -gt 0 ]; then  echo " $file: $count occurrences"  fi done  # Prompt for confirmation read -p "Do you want to update these files? (y/n): " confirm if [[ "$confirm" != "y" && "$confirm" != "Y" ]]; then  echo "Operation cancelled."  exit 0 fi  # Create backup directory BACKUP_DIR="./refactor_backup_$(date +%Y%m%d_%H%M%S)" mkdir -p "$BACKUP_DIR"  # Process each file for file in $CODE_FILES; do  if grep -q "$OLD_API_PATTERN" "$file"; then  # Create backup  mkdir -p "$BACKUP_DIR/$(dirname "$file")"  cp "$file" "$BACKUP_DIR/$file"   # Perform replacement with extended regex  sed -i -E "s/$OLD_API_PATTERN/$NEW_API_PATTERN/g" "$file"   echo "Updated $file"  fi done  echo "Refactoring complete. Backups saved to $BACKUP_DIR/" 

This script:

1. Scans a codebase for deprecated API usage
2. Creates a report of all occurrences
3. Asks for confirmation before proceeding
4. Creates a backup of all modified files
5. Uses extended regex for complex pattern matching
6. Provides feedback on each updated file

Best Practices for Multi-File Text Replacement

Based on my experience and principles from “How Linux Works” and “Shell Scripting” guides, here are essential best practices for safe and effective multi-file operations:

1. Always Create Backups

Never perform multi-file replacements without a backup strategy:

# Create backups with the -i option sed -i.bak 's/old/new/g' *.conf  # Or create a backup directory first mkdir -p backups find . -name "*.conf" -exec cp {} backups/ \; -exec sed -i 's/old/new/g' {} \; 

2. Test on a Subset First

Before processing all files, test your pattern on a representative sample:

# Test on one file and display the result without changing it sed 's/old/new/g' example.conf  # Test on a small subset find . -name "*.conf" | head -n 2 | xargs sed -i.test 's/old/new/g' 

3. Use Version Control

If your files are in a version control system, commit before making bulk changes:

git add . git commit -m "Commit before bulk replacement" # Perform replacements git diff # Review the changes 

4. Verify After Replacement

Always verify the results after performing replacements:

# Count occurrences before and after to verify all were replaced before=$(grep -r "old_text" --include="*.conf" . | wc -l) sed -i 's/old_text/new_text/g' *.conf after=$(grep -r "old_text" --include="*.conf" . | wc -l) echo "Replaced $((before - after)) occurrences" 

5. Use Appropriate Delimiters

When your search or replacement text contains slashes, use alternative delimiters:

# Using | as delimiter for paths sed -i 's|/var/www/old/|/var/www/new/|g' *.conf  # Using @ for URLs sed -i 's@http://old.com@https://new.com@g' *.conf 

6. Consider Line Endings

Be aware of different line endings when working across platforms:

# Convert Windows line endings to Unix before processing find . -name "*.txt" -exec dos2unix {} \; -exec sed -i 's/old/new/g' {} \; 

Troubleshooting Common Issues

Even with careful planning, you may encounter issues. Here’s how to address common problems:

Issue 1: Changes Not Being Applied

Symptoms: sed runs without errors, but files aren’t changed.

Potential causes and solutions:

1. File permissions: Ensure you have write permissions.

   # Check file permissions ls -la *.conf # Update if needed chmod u+w *.conf 

2. Pattern not matching: Your regex might not match any content.

   # Verify your pattern matches something grep "old_pattern" *.conf # If not matching, adjust your pattern 

3. Escaping issues: Special characters need proper escaping.

   # For patterns with special characters, use -E and proper escaping sed -i -E 's/value=\$\{old\}/value=${new}/g' *.conf 

Issue 2: Corrupted Output Files

Symptoms: Files are modified but contain corrupted text or binary data.

Potential causes and solutions:

1. Binary files: sed doesn’t handle binary files well.

   # Exclude binary files find . -type f -name "*.conf" -exec grep -Iq . {} \; -print | xargs sed -i 's/old/new/g' 

2. Encoding issues: File encoding might be affected.

   # Preserve encoding with iconv for file in *.conf; do  iconv -f UTF-8 -t UTF-8 "$file" | sed 's/old/new/g' > "$file.new"  mv "$file.new" "$file" done 

Issue 3: Performance Problems with Large Files

Symptoms: Operations on large files are extremely slow or cause high system load.

Potential causes and solutions:

1. File size: Very large files can slow down processing.

   # Process large files in chunks split -b 100m large_file.txt chunk_ for chunk in chunk_*; do  sed -i 's/old/new/g' "$chunk" done cat chunk_* > large_file_new.txt 

2. Inefficient patterns: Complex patterns can cause performance issues.

   # Pre-filter files to reduce processing grep -l "old" *.txt | xargs sed -i 's/old/new/g' 

Advanced Topic: Combining sed with awk and perl

For even more powerful text processing, combine sed with other tools:

Using sed with awk

# Use awk for more complex conditional replacements find . -name "*.csv" -exec awk '{  if ($1 == "id" && $2 == "old") {  gsub("old", "new", $0)  }  print }' {} > {}.new \; -exec mv {}.new {} \; 

Using sed with perl

# Use perl for multi-line pattern replacement find . -name "*.html" -exec perl -i -0pe 's/<div class="old">\s*<p>(.*?)<\/p>\s*<\/div>/<div class="new">\n <p>\1<\/p>\n<\/div>/gs' {} \; 

According to “Unix Power Tools”, these combinations leverage the strengths of each tool for more sophisticated text processing.

Conclusion: Mastering Multi-File Text Replacement

Throughout this guide, we’ve explored a comprehensive set of techniques for replacing text across multiple files using sed. From basic shell globbing to complex find commands and advanced pattern matching, you now have a toolkit for handling virtually any text replacement scenario.

Remember these key principles:

1. Always create backups before performing batch operations
2. Test your patterns on a small sample before applying widely
3. Use appropriate file selection methods based on your needs
4. Consider the specific requirements of different file types
5. Verify your changes after performing replacements

With these techniques and best practices, you can confidently perform even complex text replacements across large codebases, saving countless hours of manual editing and reducing the risk of errors.

What multi-file text replacement challenges have you encountered? I’d love to hear about your experiences and additional techniques in the comments below!