How to replace multiple strings in a file using PowerShell

Question

I am writing a script for customising a configuration file. I want to replace multiple instances of strings within this file, and I tried using PowerShell to do the job.

It works fine for a single replace, but doing multiple replaces is very slow because each time it has to parse the whole file again, and this file is very large. The script looks like this:

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'
(Get-Content $original_file) | Foreach-Object {
    $_ -replace 'something1', 'something1new'
    } | Set-Content $destination_file

I want something like this, but I don't know how to write it:

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'
(Get-Content $original_file) | Foreach-Object {
    $_ -replace 'something1', 'something1aa'
    $_ -replace 'something2', 'something2bb'
    $_ -replace 'something3', 'something3cc'
    $_ -replace 'something4', 'something4dd'
    $_ -replace 'something5', 'something5dsf'
    $_ -replace 'something6', 'something6dfsfds'
    } | Set-Content $destination_file

KyleMit · Accepted Answer · 2015-07-13 18:48:56Z

197

One option is to chain the -replace operations together. The ` at the end of each line escapes the newline, causing PowerShell to continue parsing the expression on the next line:

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'
(Get-Content $original_file) | Foreach-Object {
    $_ -replace 'something1', 'something1aa' `
       -replace 'something2', 'something2bb' `
       -replace 'something3', 'something3cc' `
       -replace 'something4', 'something4dd' `
       -replace 'something5', 'something5dsf' `
       -replace 'something6', 'something6dfsfds'
    } | Set-Content $destination_file

Another option would be to assign an intermediate variable:

$x = $_ -replace 'something1', 'something1aa'
$x = $x -replace 'something2', 'something2bb'
...
$x

edited Jul 13, 2015 at 18:48

KyleMit♦

31.4k74 gold badges517 silver badges712 bronze badges

answered Aug 4, 2010 at 7:10

dahlbyk

78k8 gold badges105 silver badges126 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

cquadrini Over a year ago

Can $original_file == $destination_file? As in I am modifying the same file as my source?

dahlbyk Over a year ago

Because of the way PowerShell cmdlets stream their input/ouput, I don't believe it would work to write out to the same file in the same pipeline. However, you could do something like $c = Get-Content $original_file; $c | ... | Set-Content $original_file.

Kiquenet Over a year ago

Have you problems about file encoding using Set-Content that not mantains the original encoding? UTF-8 or ANSI encodings for example.

dahlbyk Over a year ago

Yeah PowerShell is...unhelpful like that. You have to detect encoding yourself, e.g. github.com/dahlbyk/posh-git/blob/…

SouthSun Over a year ago

This solution has failed for me. It took a longer time to process and it generated a very big file instead of a file with 30 short lines that is in the existing file.

Peter Mortensen · Accepted Answer · 2017-09-28 22:22:55Z

32

To get the post by George Howarth working properly with more than one replacement you need to remove the break, assign the output to a variable ($line) and then output the variable:

$lookupTable = @{
    'something1' = 'something1aa'
    'something2' = 'something2bb'
    'something3' = 'something3cc'
    'something4' = 'something4dd'
    'something5' = 'something5dsf'
    'something6' = 'something6dfsfds'
}

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'

Get-Content -Path $original_file | ForEach-Object {
    $line = $_

    $lookupTable.GetEnumerator() | ForEach-Object {
        if ($line -match $_.Key)
        {
            $line = $line -replace $_.Key, $_.Value
        }
    }
   $line
} | Set-Content -Path $destination_file

edited Sep 28, 2017 at 22:22

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Feb 17, 2012 at 10:42

TroyBramley

6045 silver badges7 bronze badges

2 Comments

angularsen Over a year ago

This is by far the best approach I've seen so far. The only issue is that I had to read the entire file contents to a variable first in order to use the same source/destination file paths.

user705185 Over a year ago

this looks like the best answer, though I've seen some weird behavior with it matching incorrectly. i.e. in the case where you have a hash table with hex values as strings (0x0, 0x1, 0x100, 0x10000) and 0x10000 will match 0x1.

Peter Mortensen · Accepted Answer · 2017-09-28 22:23:35Z

21

With version 3 of PowerShell you can chain the replace calls together:

 (Get-Content $sourceFile) | ForEach-Object {
    $_.replace('something1', 'something1').replace('somethingElse1', 'somethingElse2')
 } | Set-Content $destinationFile

edited Sep 28, 2017 at 22:23

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Jan 10, 2016 at 11:54

Ian Robertson

2,8323 gold badges30 silver badges37 bronze badges

4 Comments

hdoghmen Over a year ago

Works fine + fluent flavor

nloewen Over a year ago

As long as you don't need RegEx

Ian Robertson Over a year ago

Where does the question mention regex?

Evan Avanti Over a year ago

regex the tool we use your solution to avoid using. Chaining replace statements feels wrong but I am doing it!

Peter Mortensen · Accepted Answer · 2017-09-28 22:21:31Z

14

Assuming you can only have one 'something1' or 'something2', etc. per line, you can use a lookup table:

$lookupTable = @{
    'something1' = 'something1aa'
    'something2' = 'something2bb'
    'something3' = 'something3cc'
    'something4' = 'something4dd'
    'something5' = 'something5dsf'
    'something6' = 'something6dfsfds'
}

$original_file = 'path\filename.abc'
$destination_file =  'path\filename.abc.new'

Get-Content -Path $original_file | ForEach-Object {
    $line = $_

    $lookupTable.GetEnumerator() | ForEach-Object {
        if ($line -match $_.Key)
        {
            $line -replace $_.Key, $_.Value
            break
        }
    }
} | Set-Content -Path $destination_file

If you can have more than one of those, just remove the break in the if statement.

edited Sep 28, 2017 at 22:21

Peter Mortensen

31.4k22 gold badges110 silver badges134 bronze badges

answered Aug 4, 2010 at 7:20

George Howarth

2,92321 silver badges19 bronze badges

1 Comment

cliffclof Over a year ago

I see TroyBramley added $line just before the last line to write any line that had no changes in it. Okay. In my case I only changed every line needing replacements.

user1624688 · Accepted Answer · 2014-01-04 04:38:48Z

13

A third option, for a pipelined one-liner is to nest the -replaces:

PS> ("ABC" -replace "B","C") -replace "C","D"
ADD

And:

PS> ("ABC" -replace "C","D") -replace "B","C"
ACD

This preserves execution order, is easy to read, and fits neatly into a pipeline. I prefer to use parentheses for explicit control, self-documentation, etc. It works without them, but how far do you trust that?

-Replace is a Comparison Operator, which accepts an object and returns a presumably modified object. This is why you can stack or nest them as shown above.

Please see:

help about_operators

answered Jan 4, 2014 at 4:38

user1624688

Comments

iRon · Accepted Answer · 2021-08-05 10:17:08Z

1

Just a general reusable solution:

function Replace-String {
    [CmdletBinding()][OutputType([string])] param(
        [Parameter(Mandatory = $True, ValueFromPipeLine = $True)]$InputObject,
        [Parameter(Mandatory = $True, Position = 0)][Array]$Pair,
        [Alias('CaseSensitive')][switch]$MatchCase
    )
    for ($i = 0; $i -lt $Pair.get_Count()) {
        if ($Pair[$i] -is [Array]) {
            $InputObject = $InputObject |Replace-String -MatchCase:$MatchCase $Pair[$i++]
        }
        else {
            $Regex = $Pair[$i++]
            $Substitute = if ($i -lt $Pair.get_Count() -and $Pair[$i] -isnot [Array]) { $Pair[$i++] }
            if ($MatchCase) { $InputObject = $InputObject -cReplace $Regex, $Substitute }
            else            { $InputObject = $InputObject -iReplace $Regex, $Substitute }
        }
    }
    $InputObject
}; Set-Alias Replace Replace-String

Usage:

$lookupTable |Replace 'something1', 'something1aa', 'something2', 'something2bb', 'something3', 'something3cc'

or:

$lookupTable |Replace ('something1', 'something1aa'), ('something2', 'something2bb'), ('something3', 'something3cc')

Example:

'hello world' |Replace ('h','H'), ' ', ('w','W')
HelloWorld

edited Aug 5, 2021 at 10:17

answered Aug 5, 2021 at 9:38

iRon

24.4k10 gold badges60 silver badges107 bronze badges

1 Comment

iRon Over a year ago

I have created a formal PowerShell request for this: #15876 Make -Replace operator support multiple Regex/Substitution pairs

Collectives™ on Stack Overflow

How to replace multiple strings in a file using PowerShell

6 Answers 6

5 Comments

2 Comments

4 Comments

1 Comment

Comments

1 Comment

Linked

Hot Network Questions

Collectives™ on Stack Overflow

6 Answers 6

5 Comments

2 Comments

4 Comments

1 Comment

Comments

1 Comment

Linked

Related