Sql server bulk insert

Bulk Insert

Sql server table of a bulk insert statements can be efficiently introduced into a large amount of data flat file (txt, csv files) into the database, which is used as follows:

bulk insert test
from 'f:\test.txt'
with
(fieldterminator=',',
rowterminator='\n')

Where "test" is the name of the database table, "f: \ test.txt" is the address import flat files, specify what fieldterminator flat file column separator character is, what rowterminator specify the flat file rows terminator yes.

You can also use FIRSTROW and LASTROW limit the number of rows. COPY The first three lines are as follows:

bulk insert test
from 'f:\test.txt'
with
(fieldterminator=',',
rowterminator='\n',
FIRSTROW =1,
LASTROW=3)

Make a flat file data into a database table, flat file only three fields, database table has seven fields,
how the field corresponds to a flat file fields to the table, how to use bulk insert to achieve?

Database table userinfo

CREATE TABLE userinfo
(
    id INT identity,
    userName varchar(20),
    pass varchar(20),
    address varchar(100),
    phone varchar(20),
    email varchar(128),
    registerTime datetime
)

Flat file data is F: \ test.txt

userName, address, Phone 
hua, Hunan, 5971898 
jan, Chongqing, 2334512 
wang, Beijing, 100201

G first stored in a disk formatted xml file
G: \ format.xml

<?xml version="1.0"?>
<BCPFORMAT xmlns="http://schemas.microsoft.com/sqlserver/2004/bulkload/format" 

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
 <RECORD>
  <FIELD ID="1" xsi:type="CharTerm" TERMINATOR="," MAX_LENGTH="20" COLLATION="Chinese_PRC_CI_AS"/>
  <FIELD ID="2" xsi:type="CharTerm" TERMINATOR="," MAX_LENGTH="100" COLLATION="Chinese_PRC_CI_AS"/>
  <FIELD ID="3" xsi:type="CharTerm" TERMINATOR="\r\n" MAX_LENGTH="20" COLLATION="Chinese_PRC_CI_AS"/>
 </RECORD>
 <ROW>
  <COLUMN SOURCE="1" NAME="userName" xsi:type="SQLVARYCHAR"/>
  <COLUMN SOURCE="2" NAME="address" xsi:type="SQLVARYCHAR"/>
  <COLUMN SOURCE="3" NAME="phone" xsi:type="SQLVARYCHAR"/>
 </ROW>
</BCPFORMAT>

Then use bulk insert statement, using FORMATFILE parameter specifies the file format.xml

BULK INSERT userinfo
    FROM 'F:\test.txt'
WITH
(
    FORMATFILE = 'G:\format.xml',
    FIELDTERMINATOR=',',
    ROWTERMINATOR='\n',
    FIRSTROW = 2
)

Such bulk insert statement will follow format.xml column mapping file, flat file F: \ test.txt three columns are sequentially inserted into the table userinfo userName, address, phone on the three fields.

Troubleshooting

1. the best use hexadecimal ASCII code to declare ROWTERMINATOR

Sometimes when we file header txt or csv file without encoding declaration file (such as txt or csv file is a 936-GBK, or UTF-8 encoding, etc., but not declared in its header), it will lead to bulk insert ROWTERMINATOR parameters statement fails, such as when I import a file test.txt is the time 936-GBK encoding, I found the file in line breaks is obviously '\ n', but when I declare ROWTERMINATOR = '\ n' in time does not work, bulk insert always find the file in line breaks and thus error.

Then I discovered through online information, the original ROWTERMINATOR parameter bulk insert, you can use ASCII hexadecimal code to indicate what line breaks yes. For example, '\ n' ASCII code 12 decimal, hexadecimal 0A, then we can insert statement declared in bulk 0x0a expressed as ROWTERMINATOR newline '\ n', as follows:

BULK INSERT userinfo
    FROM 'F:\test.txt'
WITH
(
    FIELDTERMINATOR='|',
    ROWTERMINATOR='0x0a',
    FIRSTROW =1,
    LASTROW=1000,
    CODEPAGE='936'
)

It was found that when I use the ROWTERMINATOR parameters hexadecimal ASCII code of the bulk insert statement, sql server successfully identified the end of each line of the file location. Similarly, if a newline character is '\ r', then we can declare ROWTERMINATOR hexadecimal ASCII code 0x0d. So when you use the regular characters assigned to ROWTERMINATOR parameter does not work when the bulk insert, you can try ( '\ n', '\ r', etc. for example) use ASCII-character hexadecimal code assigned to the parameters ROWTERMINATOR , ROWTERMINATOR bulk insert parameter can be accurately identified in the data file according to the hexadecimal ASCII code line breaks.

2. Use the encoding parameter declaration files CODEPAGE

CODEPAGE parameter can declare what encoding txt or csv file is sometimes unable to identify the bulk insert encoding of the import file, cause data imported from files created at this time if we knew encoding files, you can use CODEPAGE parameter tells bulk insert file is in what encoding to avoid import the data into the database becomes corrupted. For example, in the above sql statement we use CODEPAGE parameter encoding of the import file is 936 (GBK)

Also here there is a very detailed article summarizes some problems using bulk insert line breaks may appear, you can refer to the following:

SQL Server Bulk Insert Row Terminator Issues