Replace CRLF using powershell

Question

^{Editor's note: Judging by later comments by the OP, the gist of this question is:
How can you convert a file with CRLF (Windows-style) line endings to a LF-only (Unix-style) file in PowerShell?}

Here is my powershell script:

 $original_file ='C:\Users\abc\Desktop\File\abc.txt'
 (Get-Content $original_file) | Foreach-Object {
 $_ -replace "'", "2"`
-replace '2', '3'`
-replace '1', '7'`
-replace '9', ''`
-replace "`r`n",'`n'
} | Set-Content "C:\Users\abc\Desktop\File\abc.txt" -Force

With this code i am able to replace 2 with 3, 1 with 7 and 9 with an empty string. I am unable to replace the carriage return line feed with just the line feed. But this doesnt work.

Set-Content writes from the pipeline to a file. Each item from the pipeline is written on a new line. — Andrew Savinykh
– Andrew Savinykh, Commented Oct 1, 2013 at 23:55

mklement0 · Accepted Answer · 2025-11-12 14:25:23Z

This is a state-of-the-union answer as of Windows PowerShell v5.1 / PowerShell (Core) 7 v7.5.x:

Andrew Savinykh's ill-fated answer, despite being the accepted one, is, as of this writing, fundamentally flawed (I do hope it gets fixed - there's enough information in the comments - and in the edit history - to do so).
Ansgar Wiecher's helpful answer works well, but requires direct use of the .NET Framework (and reads the entire file into memory, though that could be changed). Direct use of the .NET Framework is not a problem per se, but is harder to master for novices and hard to remember in general.
A future version of PowerShell (Core) 7 may introduce a way to control the newline format when writing files, though as of this writing the prospects aren't good:
- GitHub issue #2872 is an - unfortunately stalled - proposal to add a -NewLine parameter to the Set-Content cmdlet.
- GitHub issue #17 is an - equally stalled - proposal to introduce a Convert-LineEnding cmdlet for in-place updating of text files with a specific newline format; unfortunately, the module it was meant to be added to is no longer being maintained.

In PSv5+, PowerShell-native solutions are now possible, because Set-Content now supports the -NoNewline switch, which prevents undesired appending of a platform-native newline^[1] :

# Convert CRLFs to LFs only.
# Note:
#  * (...) around Get-Content ensures that $file is read *in full*
#    up front, so that it is possible to write back the transformed content
#    to the same file.
#  * + "`n" ensures that the file has a *trailing LF*, which Unix platforms
#     expect.
((Get-Content $file) -join "`n") + "`n" | Set-Content -NoNewline $file

The above relies on Get-Content's ability to read a text file that uses any combination of CR-only, CRLF, and LF-only newlines line by line.

Caveats:

You need to specify the output encoding to match the input file's in order to recreate it with the same encoding. The command above does NOT specify an output encoding; to do so, use -Encoding;
By default, without -Encoding:
- In Windows PowerShell, you'll get "ANSI" encoding, your system's single-byte, 8-bit legacy encoding, such as Windows-1252 on US-English systems.
- In PowerShell (Core) 7, you'll get UTF-8 encoding without a BOM.
- The input file's content as well as its transformed copy must fit into memory as a whole, which can be problematic with large input files, though is rarely a concern with text files.
- There's a small risk of file corruption, if the process of writing back to the input file gets interrupted.

^{[1] In fact, if there are multiple strings to write, -NoNewline also doesn't place a newline between them; in the case at hand, however, this is irrelevant, because only one string is written.}

Andrew Savinykh · Accepted Answer · 2020-11-25 20:44:45Z

52

You have not specified the version, I'm assuming you are using Powershell v3.

Try this:

$path = "C:\Users\abc\Desktop\File\abc.txt"
(Get-Content $path -Raw).Replace("`r`n","`n") | Set-Content $path -Force

^{Editor's note: As mike z points out in the comments, Set-Content appends a trailing CRLF, which is undesired. Verify with: 'hi' > t.txt; (Get-Content -Raw t.txt).Replace("`r`n","`n") | Set-Content t.txt; (Get-Content -Raw t.txt).EndsWith("`r`n"), which yields $True.}

Note this loads the whole file in memory, so you might want a different solution if you want to process huge files.

UPDATE

This might work for v2 (sorry nowhere to test):

$in = "C:\Users\abc\Desktop\File\abc.txt"
$out = "C:\Users\abc\Desktop\File\abc-out.txt"
(Get-Content $in) -join "`n" > $out

^{Editor's note: Note that this solution (now) writes to a different file and is therefore not equivalent to the (still flawed) v3 solution. (A different file is targeted to avoid the pitfall Ansgar Wiechers points out in the comments: using > truncates the target file before execution begins). More importantly, though: this solution too appends a trailing CRLF, which may be undesired. Verify with 'hi' > t.txt; (Get-Content t.txt) -join "`n" > t.NEW.txt; [io.file]::ReadAllText((Convert-Path t.NEW.txt)).endswith("`r`n"), which yields $True.}

Same reservation about being loaded to memory though.

edited Nov 25, 2020 at 20:44

answered Oct 2, 2013 at 0:06

Andrew Savinykh

26.7k20 gold badges110 silver badges168 bronze badges

22 Comments

Mike Zboray Over a year ago

That will almost work. Set-Content will still insert an extra CR/LF at the end.

Angel_Boy Over a year ago

Great i updated to powershell v3 and your code worked, but it still leaves CR/LF at the end like mike mentioned. I just want all LF's and no CR/LF's

Ansgar Wiechers Over a year ago

Your suggestion for PowerShell v2 will erase the files content, because the redirection will create a new empty file before the subshell can read it. Please remove it.

Ansgar Wiechers Over a year ago

The behavior is identical in PowerShell v2 and v3. Using the redirection operator truncates the file before it's read by Get-Content.

mklement0 Over a year ago

PSv5+ offers a solution to the trailing CRLF problem: Set-Content -NoNewline. The truncation of the output file with > can be avoided by using | Out-File … (or | Set-Content …) instead.

|

Ansgar Wiechers · Accepted Answer · 2013-10-02 08:09:07Z

34

Alternative solution that won't append a spurious CR-LF:

$original_file ='C:\Users\abc\Desktop\File\abc.txt'
$text = [IO.File]::ReadAllText($original_file) -replace "`r`n", "`n"
[IO.File]::WriteAllText($original_file, $text)

answered Oct 2, 2013 at 8:09

Ansgar Wiechers

202k27 gold badges286 silver badges363 bronze badges

3 Comments

mklement0 Over a year ago

Nicely done (works in v2 too). A tip re use of relative paths: Use (Convert-Path $original_file) to convert relative paths to full paths first, because the .NET framework's idea of what the current directory is usually differs from PS's.

Seth Over a year ago

What would the replace clause look like if you wanted to switch Unix to Windows, but it was possible that it was already Windows.

Ansgar Wiechers Over a year ago

@Seth Use a negative lookbehind assertion: '(?<!\r)\n', "`r`n" (replace LF with CR-LF only if LF is not preceded by CR).

Rod · Accepted Answer · 2017-05-18 05:30:09Z

Adding another version based on example above by @ricky89 and @mklement0 with few improvements:

Script to process:

*.txt files in the current folder
replace LF with CRLF (Unix to Windows line-endings)
save resulting files to CR-to-CRLF subfolder
tested on 100MB+ files, PS v5;

LF-to-CRLF.ps1:

# get current dir
$currentDirectory = Split-Path $MyInvocation.MyCommand.Path -Parent

# create subdir CR-to-CRLF for new files
$outDir = $(Join-Path $currentDirectory "CR-to-CRLF")
New-Item -ItemType Directory -Force -Path $outDir | Out-Null

# get all .txt files
Get-ChildItem $currentDirectory -Force | Where-Object {$_.extension -eq ".txt"} | ForEach-Object {
  $file = New-Object System.IO.StreamReader -Arg $_.FullName
  # Resulting file will be in CR-to-CRLF subdir
  $outstream = [System.IO.StreamWriter] $(Join-Path  $outDir $($_.BaseName + $_.Extension))
  $count = 0 
  # read line by line, replace CR with CRLF in each by saving it with $outstream.WriteLine
  while ($line = $file.ReadLine()) {
        $count += 1
        $outstream.WriteLine($line)
    }
  $file.close()
  $outstream.close()
  Write-Host ("$_`: " + $count + ' lines processed.')
}

GeekyMonkey · Accepted Answer · 2020-04-22 10:47:05Z

3

Below is my script for converting all files recursively. You can specify folders or files to exclude.

$excludeFolders = "node_modules|dist|.vs";
$excludeFiles = ".*\.map.*|.*\.zip|.*\.png|.*\.ps1"

Function Dos2Unix {
    [CmdletBinding()]
    Param([Parameter(ValueFromPipeline)] $fileName)

    Write-Host -Nonewline "."

    $fileContents = Get-Content -raw $fileName
    $containsCrLf = $fileContents | %{$_ -match "\r\n"}
    If($containsCrLf -contains $true)
    {
        Write-Host "`r`nCleaing file: $fileName"
        set-content -Nonewline -Encoding utf8 $fileName ($fileContents -replace "`r`n","`n")
    }
}

Get-Childitem -File "." -Recurse |
Where-Object {$_.PSParentPath -notmatch $excludeFolders} |
Where-Object {$_.PSPath -notmatch $excludeFiles} |
foreach { $_.PSPath | Dos2Unix }

answered Apr 22, 2020 at 10:47

GeekyMonkey

13.1k6 gold badges36 silver badges40 bronze badges

2 Comments

geisterfurz007 Over a year ago

Heads up: This is opinionated to use utf8 as encoding and not add a new line at the end. I used this after accidentally pushing an entire project with crlf to VCS which murdered the gradle build.

geisterfurz007 Over a year ago

Digging a little more, this is caused by Powershell adding the BOM to the start of the file. For ways around that either check here or don't use Powershell to rewrite your filed :')

wolfrevo · Accepted Answer · 2024-06-10 05:50:36Z

3

The following makes sure that the output file has Unix-style line endings (LF) no matter what line endings the input has:

Get-Content input.txt | % { $_ + "`n"} | Set-Content output.txt -NoNewline

Conversely the following makes that the output file has Windows-style line endings (CR LF) no matter what line endings the input has:

Get-Content input.txt| % { $_ + "`r`n"} | Set-Content output.txt -NoNewline

answered Jun 10, 2024 at 5:50

wolfrevo

7,3792 gold badges34 silver badges46 bronze badges

Comments

santiago · Accepted Answer · 2020-12-14 17:41:07Z

2

For CMD one line LF-only:

powershell -NoProfile -command "((Get-Content 'prueba1.txt') -join \"`n\") + \"`n\" | Set-Content -NoNewline 'prueba1.txt'"

so you can create a .bat

edited Dec 14, 2020 at 17:41

answered Dec 14, 2020 at 17:06

santiago

212 bronze badges

1 Comment

kyrlon Nov 6 at 14:39

This worked on first try no issues!

mklement0 · Accepted Answer · 2016-12-21 21:01:42Z

0

The following will be able to process very large files quickly.

$file = New-Object System.IO.StreamReader -Arg "file1.txt"
$outstream = [System.IO.StreamWriter] "file2.txt"
$count = 0 

while ($line = $file.ReadLine()) {
      $count += 1
      $s = $line -replace "`n", "`r`n"
      $outstream.WriteLine($s)
  }

$file.close()
$outstream.close()

Write-Host ([string] $count + ' lines have been processed.')

edited Dec 21, 2016 at 21:01

mklement0

452k68 gold badges727 silver badges986 bronze badges

answered Sep 6, 2015 at 14:39

ricky89

1,4067 gold badges27 silver badges37 bronze badges

1 Comment

mklement0 Over a year ago

On Windows, this works for LF -> CRLF conversion (the opposite of what the OP wanted), but only accidentally so: System.IO.StreamReader can also read LF-only files, and .ReadLine() returns a line without its original line ending (whether it was LF or CRLF), so the -replace operation does nothing. On Windows, System.IO.StreamReader appends CRLF when using .WriteLine(), so that's how the CRLF line breaks end up in the output file.

Collectives™ on Stack Overflow

Replace CRLF using powershell

8 Answers 8

Comments

22 Comments

3 Comments

Comments

2 Comments

Comments

1 Comment

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

8 Answers 8

Comments

22 Comments

3 Comments

Comments

2 Comments

Comments

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related