Several trade-offs of ES to improve your indexing performance

Scientific Test PerformanceEdit

Performance testing is always complex, so be as scientific as possible in your approach. Randomly fiddling with knobs and writing switches is not a good way to do performance tuning. If there are too many possibilities , we can't tell which one has the best effect . A reasonable test method is as follows:    

1. On a single node, test performance for a single shard, no replica scenario.
2. Record performance results with 100% default configuration so you have a baseline to compare against.
3. Make sure the performance tests are run long enough (30+ minutes) so you can evaluate long-term performance, not short-term spikes or delays. Some events (eg segment merging, GC) do not happen immediately, so the performance profile will continue to change over time.
4. Start modifying the default values ​​one by one on the baseline. Test them rigorously, and if the performance improvement is acceptable, keep this configuration item and start the next one.

Use bulk requests and resize themEdit

Obviously, to optimize performance, batch requests should be used. The batch size depends on your data, analytics, and cluster configuration, but 5–15 MB per batch is a good starting point. Note that the size in physical bytes is mentioned here. Document count is not a good metric for batch size. For example, if you are indexing batches of 1000 documents at a time, keep in mind the following facts:

• 1000 1 KB documents add up to 1 MB.
• 1000 documents of 100 KB size add up to 100 MB.

This is a completely different batch size. Batch requests need to be loaded into memory on the coordinator node, so the physical size of the batch request is much more important than the document count.

Start testing batch request sizes at 5–15 MB and slowly increase this number until you see no performance improvement. Then start increasing the concurrency of your batch writes (multi-threading, etc.).

Monitor your nodes with Marvel and tools such as , and see when resources are bottlenecked. If you start receiving , your cluster can't continue: at least one resource is bottlenecked. Either reduce the number of concurrency, or provide more constrained resources (such as switching from mechanical disks to SSDs), or add more nodes. iostat  top  ps  EsRejectedExecutionException 

When writing data, make sure that batch requests are sent to all your data nodes in a round-robin fashion. Do not send all requests to a single node, as the node will need to keep all batch requests in memory while processing.

save edit

Disks are often the bottleneck on modern servers. Elasticsearch is disk-heavy, and the more throughput your disk can handle, the more stable your nodes will be. Here are some tips for optimizing disk I/O:

• Use SSD. As mentioned elsewhere, They are much better than mechanical disks.
• Use RAID 0. Striped RAID increases disk I/O, at the cost of obviously failing when one hard drive fails. Do not use mirroring or parity RAID as replicas already provide this functionality.
• Also, use multiple hard drives and allow Elasticsearch to stripe data across them through multiple directory configurations. path.data 
• Do not use remotely mounted storage such as NFS or SMB/CIFS. This introduced latency is totally counterproductive for performance.
• If you're using EC2, beware of EBS. Even SSD-based EBS is generally slower than local instance storage.

Segment and merge editing

Segment merging is computationally expensive, And also eat a lot of disk I/O. Merges operate periodically in the background because they can take a long time to complete, especially for larger segments. This is usually no problem, because the probability of large-scale segment merging is very small.

不过有时候合并会拖累写入速率。如果这个真的发生了，Elasticsearch 会自动限制索引请求到单个线程里。这个可以防止出现 段爆炸 问题，即数以百计的段在被合并之前就生成出来。如果 Elasticsearch 发现合并拖累索引了，它会会记录一个声明有 now throttling indexing 的 INFO 级别信息。

Elasticsearch 默认设置在这块比较保守：不希望搜索性能被后台合并影响。不过有时候（尤其是 SSD，或者日志场景）限流阈值太低了。

默认值是 20 MB/s，对机械磁盘应该是个不错的设置。如果你用的是 SSD，可以考虑提高到 100–200 MB/s。测试验证对你的系统哪个值合适：

PUT /_cluster/settings
{"persistent":{"indices.store.throttle.max_bytes_per_sec":"100mb"}}

如果你在做批量导入，完全不在意搜索，你可以彻底关掉合并限流。这样让你的索引速度跑到你磁盘允许的极限：

PUT /_cluster/settings
{"transient":{"indices.store.throttle.type":"none"}}

设置限流类型为 none 彻底关闭合并限流。等你完成了导入，记得改回 merge 重新打开限流。

如果你使用的是机械磁盘而非 SSD，你需要添加下面这个配置到你的 elasticsearch.yml 里：

index.merge.scheduler.max_thread_count: 1

机械磁盘在并发 I/O 支持方面比较差，所以我们需要降低每个索引并发访问磁盘的线程数。这个设置允许 max_thread_count + 2 个线程同时进行磁盘操作，也就是设置为 1 允许三个线程。

对于 SSD，你可以忽略这个设置，默认是 Math.min(3, Runtime.getRuntime().availableProcessors() / 2) ，对 SSD 来说运行的很好。

最后，你可以增加 index.translog.flush_threshold_size 设置，从默认的 512 MB 到更大一些的值，比如 1 GB。这可以在一次清空触发的时候在事务日志里积累出更大的段。而通过构建更大的段，清空的频率变低，大段合并的频率也变低。这一切合起来导致更少的磁盘 I/O 开销和更好的索引速率。当然，你会需要对应量级的 heap 内存用以积累更大的缓冲空间，调整这个设置的时候请记住这点。

其他编辑

Finally, there are a few other things worth keeping in mind:

• If your search results do not require near real-time accuracy, consider adding each index's index.refresh_interval change to . If you are doing a large batch import, you can turn off refresh by setting this value during import. Don't forget to turn it back on when you're done. 30s -1 

• If you are doing large batch imports, consider setting the index.number_of_replicas: 0Close the copy. When a document is replicated, the entire document content is sent to the replica node, and the indexing process is repeated verbatim. This means that each replica also performs analysis, indexing, and possibly a merge process.

  Conversely, if your index is zero-copy, and then start the copy after the write is complete, the recovery process is essentially just a byte-to-byte network transfer. This is quite efficient compared to repeating the indexing process.

• If you don't have an ID for each document, use Elasticsearch's automatic ID feature. This is optimized to avoid version lookups, since the auto-generated IDs are unique.
• If you are using your own ID, try a Lucene friendly ID.  Includes zero-padded sequence IDs, UUID-1s, and nanoseconds; these IDs all have a consistent, well-compressed sequence pattern. Conversely, IDs like UUID-4, which are inherently random and have a low compression ratio, will significantly slow down Lucene.