Lucene在做大量term值查询时, 如果这值过多, 超1024个term的话, 会出现
TooManyClauses[maxClauseCount is set to 1024] 的异常,因此建议在term过多的情况下采用filter, 而不是query。
以下是该情形在ES中的测试。
Settings defaultSettings = ImmutableSettings.settingsBuilder().put("client.transport.sniff", true).build(); Settings finalSettings = ImmutableSettings.settingsBuilder().put(defaultSettings) .put("name", NetworkUtils.getLocalAddress().getHostName()).build(); TransportClient tmp = new TransportClient(finalSettings); Client client = tmp.addTransportAddress(new InetSocketTransportAddress("127.0.0.1", 9300)); //demo 100万数据 for (int i = 0; i < 1000000; i++) { client.prepareIndex("test2", "book",String.valueOf(i)).setSource("bookid", String.valueOf(i), "booktype", String.valueOf(i%10000)).execute() .actionGet(); } //demo 近1万个term String[] values = new String[10000]; for (int i = 1; i < 10000; i++) { values[i] = String.valueOf(i); } //terms query //TermsQueryBuilder termQueryBuilder = new TermsQueryBuilder("booktype", values); TermsFilterBuilder termsFilterBuilder = new TermsFilterBuilder("booktype", values); // SearchResponse searchResponse = client.prepareSearch().setIndices("test2").setQuery(termQueryBuilder) // .setFrom(0).setSize(100).execute().actionGet(); //terms filter SearchResponse searchResponse = client.prepareSearch().setIndices("test2").setQuery(QueryBuilders.matchAllQuery()).setFilter(termsFilterBuilder) .setFrom(0).setSize(100).execute().actionGet(); SearchHits hits = searchResponse.getHits(); System.out.println(hits.totalHits()); for (SearchHit searchHit : hits) { System.out.println(searchHit.getId() + ":" + searchHit.getSource().get("booktype")); }
上述结果会发现, 用TermsQueryBuilder查询的话, 会出现TooManyClauses的异常, 因为设置了9999个term值。因此,当term过多时,建议采用filter, 而不是query.