The iconv command is a file encoding conversion tool running on the linux / unix platform. When we view the text file in the linux/ unix system shell, we often find that the Chinese of the file is garbled. This is caused by the difference between the encoding of the text file and the encoding set by the current operating system. At this time, iconv can be used for encoding conversion. , so as to solve the garbled problem.
There are 3 steps to solve the problem of garbled text files: 1. Determine the file encoding, 2. Determine whether iconv supports the conversion of this encoding, 3. Determine the Linux /Unix operating system encoding, 4. Convert the file encoding to be consistent with the system encoding; test.txt file for example.
1. Use the file command to determine the file encoding:
file test.txt ISO-8859 text, with CRLF line terminators
Also started using the following commands to get more precise encoding:
file -bi test.txt | sed -e 's/.*[ ]charset=//' |tr '[a-z]' '[A-Z' ISO-8859-1
It can be seen that the test.txt file is encoded as ISO-8859-1.
2. Use iconv -l to determine whether iconv supports conversion of this encoding:
$ iconv -l | grep ISO-8859-1 ISO-8859-1// ISO-8859-10// ISO-8859-11// ISO-8859-13// ISO-8859-14// ISO-8859-15// ISO-8859-16//
3. Determine the Linux /Unix operating system encoding:
$ echo $LONG zh_CN.UTF-8
The current operating system environment encoding is "UTF-8"
4. Conversion coding
Note: Since the file command often misjudges the encoding, if you find that the converted encoding is still garbled, you can change the input encoding of iconv -f to other common encodings and try: GBK, BIG5, HZ, GB2312, GB18030, ASCII.
In addition: When encoding conversion, if your source format is set to GB2312, and when converting to UTF-8, it is found that the program will report an error of "illegal input sequence at position xxxx". This is due to a problem with the assumptions you made earlier. GB2312 is the smallest and earliest Chinese coding standard in the national standard. Among them, only 6,763 Chinese characters are covered. So the original format of the file you need to convert may not be GB2312 encoding. At this time, you can use GB18030 as the source format for conversion. GB18030 is the latest national standard, including 27,564 Chinese characters, and is backward compatible with GB2312 and GBK.
The iconv command is used to convert the encoding of the specified file, output to the standard output device by default, or specify the output file.
Usage: iconv [options...] [file...]
The following options are available:
Input/Output Format Specification:
-f, --from-code=name raw text encoding
-t, --to-code=Name output encoding
information:
-l, --list list all known character sets
Output control:
-c ignore invalid characters from output
-o, --output=FILE output file
-s, --silent turn off warnings
--verbose print progress information
-?, --help give a list of help for this system
--usage give brief usage information
-V, --version print program version number
example:
iconv -f utf-8 -t gb2312 aaa.txt >bbb.txt
这个命令读取aaa.txt文件，从utf-8编码转换为gb2312编码,其输出定向到bbb.txt文件。
附：
文件名转换
因为现在用linux,原来在windows里的文件都是用GBK编码的。所以copy到linux下是乱码，文件内容可以用iconv来转换可是好多中文的文件名还是乱码，找到个可以转换文件名编码的命令，就是convmv。
convmv命令详细参数
例如
convmv -f GBK -t UTF-8 *.mp3
不过这个命令不会直正的转换，你可以看到转换前后的对比。如果要直正的转换要加上参数 --notest
convmv -f GBK -t UTF-8 --notest *.mp3
-f 参数是指出转换前的编码，-t 是转换后的编码。这个千万不要弄错了。不然可能还是乱码哦。还有一个参数很有用。就是 -r 这个表示递归转换当前目录下的所有子目录。
三、更好的傻瓜型命令行工具enca，它不但能智能的识别文件的编码，而且还支持成批转换。
1.安装
$sudo apt-get install enca
2.查看当前文件编码
enca -L zh_CN ip.txt
Simplified Chinese National Standard; GB2312
Surrounded by/intermixed with non-text data
3.转换
命令格式如下
$enca -L 当前语言 -x 目标编码 文件名
例如要把当前目录下的所有文件都转成utf-8
enca -L zh_CN -x utf-8 *
enca -L zh_CN file 检查文件的编码
enca -L zh_CN -x UTF-8 file 将文件编码转换为"UTF-8"编码
enca -L zh_CN -x UTF-8 < file1 > file2 如果不想覆盖原文件可以这样，很简单吧。
In fact, you can also use notepad++ to convert, notepad++ conversion is very simple, in Windows , after using Notepad++ to open the file format , select the "Format" toolbar, and then you can choose to convert to a certain format