------------------------------------------------------------------------------------------------------------------------------------
Defining a custom field type for microblog text
------------------------------------------------------------------------------------------------------------------------------------
Advanced text analysis
How do you select the right text analyzer during indexing? Assuming you want to index all your documents regardless of language in the same index, a simple solution would be to use a unique field for each language. Suppose we want to index French tweets in our microblog search application. We could define the following field:
<field name="text_fr" type="text_microblog_fr" indexed="true" stored="true" />
-------------------------------------------------------------------------------------------------------------------------------------
Integrate jcseg with solr to deal with chinese tokenizer
1.cp jcseg-core-1.9.5.jar and jcseg-solr-1.9.5.jar to solr-4.7.0/example/solr-webapp/webapp/WEB-INF/lib/
2. cp lexicon dir to solr-4.7.0/example/solr-webapp/webapp/WEB-INF/lib/
3. alter schema.xml add fildtype
<fieldtype name="textComplex" class="solr.TextField">
<analyzer>
<tokenizer class="org.lionsoul.jcseg.solr.JcsegTokenizerFactory" mode="complex"/>
<filter class="solr.StopFilterFactory" ignoreCase="true" words="lang/stopwords_ch.txt"/>
</analyzer>
</fieldtype>