Package org.apache.lucene.analysis.cjk
Class CJKAnalyzer
- java.lang.Object
-
- org.apache.lucene.analysis.Analyzer
-
- org.apache.lucene.analysis.ReusableAnalyzerBase
-
- org.apache.lucene.analysis.StopwordAnalyzerBase
-
- org.apache.lucene.analysis.cjk.CJKAnalyzer
-
- All Implemented Interfaces:
Closeable
,AutoCloseable
public final class CJKAnalyzer extends StopwordAnalyzerBase
AnAnalyzer
that tokenizes text withStandardTokenizer
, normalizes content withCJKWidthFilter
, folds case withLowerCaseFilter
, forms bigrams of CJK withCJKBigramFilter
, and filters stopwords withStopFilter
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.analysis.ReusableAnalyzerBase
ReusableAnalyzerBase.TokenStreamComponents
-
-
Field Summary
Fields Modifier and Type Field Description static String[]
STOP_WORDS
Deprecated.usegetDefaultStopSet()
instead-
Fields inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase
matchVersion, stopwords
-
-
Constructor Summary
Constructors Constructor Description CJKAnalyzer(Version matchVersion)
Builds an analyzer which removes words ingetDefaultStopSet()
.CJKAnalyzer(Version matchVersion, String... stopWords)
Deprecated.useCJKAnalyzer(Version, Set)
insteadCJKAnalyzer(Version matchVersion, Set<?> stopwords)
Builds an analyzer with the given stop words
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description protected ReusableAnalyzerBase.TokenStreamComponents
createComponents(String fieldName, Reader reader)
Creates a newReusableAnalyzerBase.TokenStreamComponents
instance for this analyzer.static Set<?>
getDefaultStopSet()
Returns an unmodifiable instance of the default stop-words set.-
Methods inherited from class org.apache.lucene.analysis.StopwordAnalyzerBase
getStopwordSet, loadStopwordSet, loadStopwordSet, loadStopwordSet
-
Methods inherited from class org.apache.lucene.analysis.ReusableAnalyzerBase
initReader, reusableTokenStream, tokenStream
-
Methods inherited from class org.apache.lucene.analysis.Analyzer
close, getOffsetGap, getPositionIncrementGap, getPreviousTokenStream, setPreviousTokenStream
-
-
-
-
Field Detail
-
STOP_WORDS
@Deprecated public static final String[] STOP_WORDS
Deprecated.usegetDefaultStopSet()
insteadAn array containing some common English words that are not usually useful for searching and some double-byte interpunctions.
-
-
Constructor Detail
-
CJKAnalyzer
public CJKAnalyzer(Version matchVersion)
Builds an analyzer which removes words ingetDefaultStopSet()
.
-
CJKAnalyzer
public CJKAnalyzer(Version matchVersion, Set<?> stopwords)
Builds an analyzer with the given stop words- Parameters:
matchVersion
- lucene compatibility versionstopwords
- a stopword set
-
CJKAnalyzer
@Deprecated public CJKAnalyzer(Version matchVersion, String... stopWords)
Deprecated.useCJKAnalyzer(Version, Set)
insteadBuilds an analyzer which removes words in the provided array.- Parameters:
stopWords
- stop word array
-
-
Method Detail
-
getDefaultStopSet
public static Set<?> getDefaultStopSet()
Returns an unmodifiable instance of the default stop-words set.- Returns:
- an unmodifiable instance of the default stop-words set.
-
createComponents
protected ReusableAnalyzerBase.TokenStreamComponents createComponents(String fieldName, Reader reader)
Description copied from class:ReusableAnalyzerBase
Creates a newReusableAnalyzerBase.TokenStreamComponents
instance for this analyzer.- Specified by:
createComponents
in classReusableAnalyzerBase
- Parameters:
fieldName
- the name of the fields content passed to theReusableAnalyzerBase.TokenStreamComponents
sink as a readerreader
- the reader passed to theTokenizer
constructor- Returns:
- the
ReusableAnalyzerBase.TokenStreamComponents
for this analyzer.
-
-