Postgres special connection cpu resources caused by consumption of PostgreSQL performance tuning considerations

As the development phase, and there is no configuration parameters postgres, you are using the default configuration during installation,
before running it would not have what is not normal, but a few days ago my cpu sudden increase in resource consumption.
View process, found postgres of a process CPU is more than 80%, but remains high;

At first I thought it was the need to modify the configuration, but in fact, the default configuration is basically a very optimized, but also the development phase, the amount of data is not large.
Later, through the analysis, we concluded that the problem should be solved one by one to consider the following aspects:

1, SQL query terms
to check whether to establish an index of data retrieval, all fields need to look as far as possible indexing, and even joint index;
create indexes, including expression and partial indexes;
using the COPY statement instead of multiple Insert statements;
multiple SQL statements a transaction to reduce commit transaction overhead;
used when extracting a plurality of records from one index the CLUSTER;
a LIMIT partial record is removed from a query result;
precompiled formula query (Prepared query);
using ANALYZE to maintain accurate optimization statistics;
Regular use of VACUUM or pg_autovacuum
large amounts of data changes when first delete the index (and then rebuilding the index)


2, the program experience
checkers, using a connection pool, if not used, use it as soon as possible;
keep checking procedure, after connection, if returned to the connection pool;


3, the server configuration parameters
lot of settings can affect the performance of the configuration file postgres.conf,
shared_buffers: This is the most important parameter, postgresql deal by shared_buffers and core / disk.
Therefore, it should be sufficiently large so that more data is cached in shared_buffers usually set at 10% of the actual RAM is reasonable, such as 50000 (400M)
work_mem: sort_mem is called before pgsql 8.0. postgresql when sorting is done,
will decide whether a large result set is split into several small temporary files and check work_mem small size according to the size of work_mem.
Apparently the split result is to reduce the sort of speed. Thus increasing work_mem help improve the sorting speed. Is typically set to 2-4% of real RAM, depending on the size of the result set to be sorted may be, for example 81920 (80M)
effective_cache_size: postgresql is the maximum cache can be used,
this number should be large enough for independent pgsql server, For example 4G of memory, can be set to 3.5G (437.5 thousand)
maintence_work_mem: defined here only memory used when CREATE INDEX, VACUUM, etc., is used so infrequently, but often these instructions consume more resources,
and should therefore these instructions as soon as possible so that fast is finished: to maintence_work_mem large memory, such as 512M (524288)
max_connections: in general, the aim is to prevent max_connections max_connections * work_mem beyond the actual memory size.
For example, if the set work_mem actual memory size of 2%, then in the extreme case, if the query are sorted in claim 50, and 2% of all memory, generates the swap will result, system performance will be greatly reduce.
Of course, if there is 4G of memory, while the emergence of such a large probability of 50 queries should be small. However, to understand the relationship of max_connections and work_mem.
Interpretation of the relevant parameters visible: http://www.varlena.com/varlena/GeneralBits/Tidbits/annotated_conf_e.html and http://www.varlena.com/varlena/GeneralBits/Tidbits/perf.html .


4, hardware choices
because most computer hardware is compatible, people tend to believe that all computer hardware quality is the same.
Indeed not, ECC RAM (memory with parity), SCSI (hard drive) and the quality of the motherboard to be more reliable than some bargains and have a better performance.
PostgreSQL will run on almost any hardware, but if reliability and performance are important to your system, you need a comprehensive look at your hardware configured.
The impact on the performance of computer hardware can browse http://candle.pha.pa.us/main/writings/pgsql/hw_performance/index.html and http://www.powerpostgresql.com/PerfList/ .


5,为什么在试图连接时收到“Sorry, too many clients”消息?
这表示你已达到缺省100个并发后台进程数的限制,
你需要通过修改postgresql.conf文件中的max_connections值来 增加postmaster的后台并发处理数,修改后需重新启动postmaster。

转载于:https://www.cnblogs.com/kungfupanda/archive/2009/10/15/1583746.html

Guess you like

Origin blog.csdn.net/weixin_33901641/article/details/94493915