Converting text file to UTF-8 on Windows command prompt

Question

I need to convert a text file to UTF-8 format via Windows command prompt. This needs to be done on another machine and I do not have rights to install software on that machine. I need something like:

c:\notepad   source-file target-file --encoding option

Is there a Windows command prompt utility which can do it?

score 48 · Accepted Answer · answered Jan 05 '17 at 14:38

48

I need to convert a text file to utf-8 format via windows command prompt

You can easily do this with PowerShell:

Get-Content .\test.txt | Set-Content -Encoding utf8 test-utf8.txt

Additionally, you need to switch to powershell in order to execute the command and then exit it to go back to windows command prompt. – user1107888 Jan 05 '17 at 16:09
Indeed. Or run it directly in a PowerShell shell. – DavidPostill Jan 05 '17 at 16:44
@DavidPostill any idea how we can convert more than one file i.e batch file processing with the same command line ? – vibs2006 Jan 28 '19 at 09:49
1

@vibs2006 Use a `ForEach` loop. See [ForEach - PowerShell - SS64.com](https://ss64.com/ps/foreach.html). – DavidPostill Jan 28 '19 at 18:32
2

Will this method convert to plain UTF-8 or UTF-8-BOM? – Cale Sweeney Oct 23 '19 at 22:28
`Get-Content -ReadCount 5000 -Raw .\test.txt | Set-Content -Encoding utf8 test-utf8.txt` can convert huge files (over GB) very fast. – endo64 Apr 10 '21 at 21:33
Unfortunately, you have to define a different name for the ouptut file. Otherwise you get a "used by another process" error message. – PeterCo Feb 20 '22 at 07:05
1

but if I want to convert multiples .txt files, from a folder+subfolders, what is the command? – Just Me Aug 01 '22 at 14:15
1

@JustMe See my comment above - use a `ForEach` loop – DavidPostill Aug 01 '22 at 16:51
1

yes, but is much easy to put the entire formula here – Just Me Aug 02 '22 at 16:25
1

@JustMe I'm not going to do that for you. – DavidPostill Aug 02 '22 at 17:01
1

please, thanks. Not just for me, for all those that need the solution. – Just Me Aug 02 '22 at 20:47
Strangely, it put spaces in-between characters when I ran this (I was using it on a .tex file, which is just a text file). Seems to be because it's converting from UTF16-LE (related to using the latexdiff tool on Windows). – Sterling Nov 09 '22 at 19:46

score 7 · Answer 2 · edited Feb 21 '18 at 15:56

7

Use iconv from GNUWin32 pack. It is much faster, especially if your files are about or more than 1 Gb.

"C:\Program Files (x86)\GnuWin32\bin\iconv.exe" -f cp1251 -t utf-8 source.txt > result.txt

edited Feb 21 '18 at 15:56

Kamil Maciorowski

69,815
22
136
202

answered Feb 21 '18 at 15:09

Raul N-k

71
1
1

1

what does the `cp1251` flag do ? – lacostenycoder Mar 11 '19 at 16:50
2

it's the input encoding. https://en.wikipedia.org/wiki/Windows-1251 – nadre May 17 '19 at 08:57
3

Git bash works also: iconv -f gb2312 -t utf-8 ./s.txt > t.txt – Shawn Xue May 30 '20 at 04:47

score 3 · Answer 3 · answered May 20 '19 at 10:20

3

Here is for each convert *.text file to *.sql file:

foreach ($file in get-ChildItem *.txt) {
    Echo $file.name
    Get-Content $file | Set-Content -Encoding utf8 ("$file.name" +".sql")
 }

answered May 20 '19 at 10:20

nobjta_9x_tq

131
2

after run some sql file, sometime error, I decided to using linux with iconv command =)) – nobjta_9x_tq May 28 '19 at 15:46

score 1 · Answer 4 · answered Sep 30 '20 at 20:49

1

You can do this from the command prompt as follows:

powershell -command "Get-Content .\test.txt" > test-utf8.txt

It turns out that piping the output to a file from the command prompt saves as utf-8.

answered Sep 30 '20 at 20:49

Gord Hooker

11
1

but if I want to convert multiples .txt files, from a folder+subfolders, what is the command? – Just Me Aug 01 '22 at 14:15

score 0 · Answer 5 · answered Aug 01 '22 at 14:19

POWERSHELL: # Assumes Windows PowerShell, use -Encoding utf8BOM with PowerShell Core. For multiple files:

FIRST SOLUTION:

$files = Get-ChildItem c:\Folder1\ -Filter *.txt 

foreach ($file in $files) {

    Get-Content $file.FullName | Set-Content "E:\Temp\Destination\$($file.Name)" -Encoding utf8BOM

}

OR, SECOND SOLUTION (for multiple files):

get-item C:\Folder1*.* | foreach-object {get-content -Encoding utf8BOM $_ | out-file ("C:\Folder1" + $_.Name) -encoding default}

OR, THE THIRD SOLUTION: (only for 2 files)

$a = "C:/Folder1/TEST_ro.txt"
 $b = "C:/Folder1/TEST_ro-2.txt"
 (Get-Content -path $a) | Set-Content -Encoding UTF8BOM -Path $b

score 0 · Answer 6 · answered Apr 12 '23 at 13:46

For those who want to batch convert several files (e.g.: all *.txt files in folder and sub-folders):

dir *.txt -Recurse | foreach {
  # May remove the line below if you are confident
  Copy-Item $_ $_.bkp
  
  # Note that since we are reading and saving to the same file,
  # we need to enclose the command in parenthesis so it fully executes 
  # (reading all content and closing the file) before proceeding
  (Get-Content $_) | Set-Content -Encoding utf8 $_
}

Converting text file to UTF-8 on Windows command prompt

6 Answers6

I need to convert a text file to utf-8 format via windows command prompt

Further Reading

Linked