SQLite는의 운영 성능을 개선하기 위해 (재 인쇄)

원본 주소 :

https://blog.devart.com/increasing-sqlite-performance.html

자신의 응용 프로그램에서 SQLite는 DBMS를 사용하면 개발자가 발견 한 주요 문제는 성능 문제입니다.

아마, 고전적인 사례 모두가 처음 SQLite는 사용시에 도착하는 다중의 매우 느린 실행이다 INSERT / UPDATE / DELETE 작업을. 실제로, 테이블에조차 수천하지만 INSERT들 수백의 순차적 인 실행이 너무 오래 걸릴 수 있습니다. 문제의 기원은 SQLite는에 거래를 사용하는 특이성에있다. SQLite는 어떤 DML 문을 실행하기 전에 트랜잭션을 때마다 자동으로 시작 및 실행 후을 얻어냅니다. 여러 결과의 문을 실행할 때 따라서, 새로운 트랜잭션이 시작하고 각 문에 대해 최선을 다하고됩니다.

이 문제의 해결책은 매우 간단합니다 - DML 문의 블록에 동봉 할 수있다 ... END의 BEGIN 사업자 블록 ( https://www.sqlite.org/lang_transaction.html ). 이 경우 각 DML 문은 별도의 트랜잭션에서 실행되지 않지만 하나의 트랜잭션은 전체 블록 실행 전에 시작하고 모든 수정 후 최선을 다하고됩니다.

이러한 접근 방식은 배 SQLite는 데이터 수정 성능이 향상됩니다. SQLite는 문서 (에 대한 자세한 내용을 참조하십시오 https://www.sqlite.org/faq.html#q19을 ).

그러나이 방법은 SQLite는 성능을 높일 수있는 유일한 방법은 아니다. DBMS에의 파라미터는 또한 소위 PRAGMA (하여 구성 될 수있다 https://www.sqlite.org/pragma.html ). 사실은 SQLite는 매개 변수는 기본적으로 높은 성능을하지 지향하는 것입니다 만, 최대 데이터 안전성과 무결성. 이러한 매개 변수의 수정은 데이터 손상 위험이 너무 증가, 그러나 성능, 메모를 증가시킬 수있다.

의 사용 다른하는 pragma하여 성능 삽입에 미치는 영향 분석하자 LiteDAC을 .

우리는 우리의 프로젝트에 테스트 테이블 SPEED_TEST를 사용합니다 :

CREATE TABLE SPEED_TEST (
  ID INTEGER,
  F_INTEGER INTEGER,
  F_FLOAT FLOAT,
  F_STRING VARCHAR(250),
  F_DATE DATETIME,
  CONSTRAINT PK_BATCH_TEST PRIMARY KEY (ID)
)

CREATE TABLE SPEED_TEST (

ID INTEGER,

F_INTEGER INTEGER,

F_FLOAT FLOAT,

F_STRING VARCHAR(250),

F_DATE DATETIME,

CONSTRAINT PK_BATCH_TEST PRIMARY KEY (ID)

)

각 테스트에서, 우리는 데이터베이스를 삭제하고 다시 만든 다음 다음과 같이 SPEED_TEST 테이블에 10,000 레코드를 삽입 :

var
  LiteQuery:  TLiteQuery;
  i: Integer;
  ParamID, ParamF_INTEGER, ParamF_FLOAT, ParamF_STRING, ParamF_DATE: TDAParam;
begin
   ...
  LiteQuery.SQL.Text := 'INSERT INTO SPEED_TEST VALUES (:ID, :F_INTEGER, :F_FLOAT, :F_STRING, :F_DATE)';  ParamID := LiteQuery.Params.ParamByName('ID');
  ParamID.DataType := ftInteger;
  ParamF_INTEGER := LiteQuery.Params.ParamByName('F_INTEGER');
  ParamF_INTEGER.DataType := ftInteger;
  ParamF_FLOAT := LiteQuery.Params.ParamByName('F_FLOAT');
  ParamF_FLOAT.DataType := ftFloat;
  ParamF_STRING := LiteQuery.Params.ParamByName('F_STRING');
  ParamF_STRING.DataType := ftWideString;
  ParamF_DATE := LiteQuery.Params.ParamByName('F_DATE');
  ParamF_DATE.DataType := ftDateTime;

  for i := 0 to 10000 - 1 do begin
    ParamID.AsInteger := i + 1;
    ParamF_INTEGER.AsInteger := i + 5000 + 1;
    ParamF_FLOAT.AsFloat := (i + 1) / 15;
    ParamF_STRING.AsString := 'Values ' + IntToStr(i + 1);
    ParamF_DATE.AsDateTime := Now;
    LiteQuery.Execute;
  end;
  ...
end;

var

LiteQuery: TLiteQuery;

i: Integer;

ParamID, ParamF_INTEGER, ParamF_FLOAT, ParamF_STRING, ParamF_DATE: TDAParam;

begin

...

LiteQuery.SQL.Text := 'INSERT INTO SPEED_TEST VALUES (:ID, :F_INTEGER, :F_FLOAT, :F_STRING, :F_DATE)'; ParamID := LiteQuery.Params.ParamByName('ID');

ParamID.DataType := ftInteger;

ParamF_INTEGER := LiteQuery.Params.ParamByName('F_INTEGER');

ParamF_INTEGER.DataType := ftInteger;

ParamF_FLOAT := LiteQuery.Params.ParamByName('F_FLOAT');

ParamF_FLOAT.DataType := ftFloat;

ParamF_STRING := LiteQuery.Params.ParamByName('F_STRING');

ParamF_STRING.DataType := ftWideString;

ParamF_DATE := LiteQuery.Params.ParamByName('F_DATE');

ParamF_DATE.DataType := ftDateTime;

for i := 0 to 10000 - 1 do begin

ParamID.AsInteger := i + 1;

ParamF_INTEGER.AsInteger := i + 5000 + 1;

ParamF_FLOAT.AsFloat := (i + 1) / 15;

ParamF_STRING.AsString := 'Values ' + IntToStr(i + 1);

ParamF_DATE.AsDateTime := Now;

LiteQuery.Execute;

end;

...

end;

마이크로 소프트 윈도우 7 x86 및 맥 OS X 10.9 매버릭스 : 우리는이 개 플랫폼에서 테스트 프로젝트를 실행하겠습니다.

PRAGMA TEMP_STORE

This parameter allows to specify location of temporary objects in the database: tables, indexes, triggers, views, etc. PRAGMA TEMP_STORE accepts 3 values:

0 | DEFAULT — the default value, when location of temporary objects storage is defined by the option, which is set during SQLite compilation;
1 | FILE — temporary objects are stored in a file (its location depends on the used OS);
2 | MEMORY — temporary objects are stored in memory.

When the TEMP_STORE parameter is changed, all the temporary tables, indexes, triggers, views are deleted.

Time: sec

Microsoft Windows 7 x86
DEFAULT	FILE	MEMORY
235	225	215

MacOS X 10.9 Mavericks
DEFAULT	FILE	MEMORY
34	33	32

According to the retrieved results, making RAM a storage for temporary DB objects increases performance a little.

PRAGMA JOURNAL_MODE

The parameter sets the database log working mode (rollback journal file used on transaction processing).

PRAGMA JOURNAL_MODE accepts the following values:

DELETE (the default value) — in this mode the log file is deleted after transaction is committed.
TRUNCATE — after transaction is committed, the log file is truncated to 0 size. This works faster than log deleting on a number of platforms, since the catalog containing the log file is not modified at this.
PERSIST — instead of deleting and truncating, the log file header is filled in with zeros. In this case, the log file size doesn’t change and it can require quite much space. However, such an operation may be executed faster than DELETE or TRUNCATE.
MEMORY — the rollback journal is kept in RAM and doesn’t use the disk subsystem. Such mode provides more significant performance increase when working with log. However, in case of any failures within a transaction, data in the DB will be corrupted with high probability due to a lack of saved data copy on the disk.
OFF — not using log. In this mode, transaction rollback is impossible. In case of a crash, the database will be likely corrupted.

Time: sec

Microsoft Windows 7 x86
DELETE	TRUNCATE	PERSIST	MEMORY	OFF
235	210	220	65	63

MacOS X 10.9 Mavericks
DELETE	TRUNCATE	PERSIST	MEMORY	OFF
34	4	3	2	1

Changing this parameter significantly increases performance when inserting data on both platforms. Note, that at using MEMORY or OFF values, the risk of data loss is maximal too.

PRAGMA SYNCHRONOUS

Defines the mode of rollback journal synchronization with the data.

2 | FULL — the default value. SQLite pauses working in critical cases, in order to guarantee 100% data integrity when saving to the database file — even on system crash or power failure. This synchronization mode is absolutely safe, but the most slow.
1 | NORMAL — SQLite will pause in critical cases as well, but less often in the FULL mode. At this value, there is a (quite small) data corruption possibility. The NORMAL mode is a compromise between reliability and performance.
0 | OFF — database synchronization is not used. I.e., SQLite takes no breaks when transmitting data to the operating system. Such mode can substantially increase performance. The database will meet the integrity conditions after the SQLite crash, however, data will be corrupted in case of system crash or power off.

Time: sec

Microsoft Windows 7 x86
FULL	NORMAL	OFF
235	175	43

MacOS X 10.9 Mavericks
FULL	NORMAL	OFF
34	32	32

The test demonstrated significant performance increase on Windows. It is highly recommended to speed up performance this way only with assurance of the operating system stability and power quality.

PRAGMA LOCKING_MODE

Defines the locking mode.

NORMAL — the default value. The database file is locked at the moment of record reading or writing. After the operation completes, the lock is released.
EXCLUSIVE — the database file is used in exclusive mode. The number of system calls to implement file operations decreases in this case, which may increase database performance.

Time: sec

Microsoft Windows 7 x86
NORMAL	EXCLUSIVE
235	155

MacOS X 10.9 Mavericks
NORMAL	EXCLUSIVE
34	3

The most secure approach to increase performance. Though, EXCLUSIVE MODE allows the database to serve only one connection.

PRAGMA CACHE_SIZE

Defines the number of pages from the database file for storing in RAM, i.e., the cache size. Increasing this parameter may increase performance of the database on high load, since the greater its value is, the more modifications a session can perform before retrieving exclusive lock.

Time: sec

Microsoft Windows 7 x86
0	500	1000	2000	4000	8000	10000
222	185	230	230	230	230	250

MacOS X 10.9 Mavericks
0	500	1000	2000	4000	8000	10000
–	34	34	34	34	34	34

Performance increase is observed only on Windows. Changing this parameter is almost secure, yet the performance growth is minor.

PRAGMA PAGE_SIZE

Defines the database page size.

The page size is set by default depending on some computer and OS specifications. They include disk sector size and the used encoding. SQLite supports page size range from 512 to 65536 bytes.

Time: sec

Windows 7 x86
512	1024	2048	4096	8192	16384	32768	65535
240	235	227	225	255	295	450	295

MacOS X 10.9 Mavericks
512	1024	2048	4096	8192	16384	32768	65535
34	34	33	33	34	40	50	65

Like the previous parameter, PRAGMA PAGE_SIZE is almost safe with respect to risk of data corruption, however, tuning of this parameter doesn’t lead to significant performance increase and requires search for values depending on the used sector size.

Summary

There are ways to increase SQLite data insert performance in addition to using transactions.
Usage of PRAGMA may significantly increase performance of SQLite.
SQLite default settings don’t provide optimal performance, yet guarantee absolute data integrity and safety.
Changing SQLite parameter values in favor of performance increases the risk of data corruption.
Values of some parameters (PRAGMA PAGE_SIZE, PRAGMA CACHE_SIZE) should be found depending on specifications of the used environment.