The PerFieldAnalyzer is for use when you want to analyze different fields with different analyzers. With the PerFieldAnalyzer you can specify how you want each field analyzed.
# Create a new PerFieldAnalyzer which uses StandardAnalyzer by default pfa = PerFieldAnalyzer.new(StandardAnalyzer.new()) # Use the WhiteSpaceAnalyzer with no lowercasing on the :title field pfa[:title] = WhiteSpaceAnalyzer.new(false) # Use a custom analyzer on the :created_at field pfa[:created_at] = DateAnalyzer.new
Create a new PerFieldAnalyzer specifying the default analyzer to use on all fields that are set specifically.
analyzer to be used on fields that aren't otherwise specified
static VALUE frb_per_field_analyzer_init(VALUE self, VALUE ranalyzer) { Analyzer *def = frb_get_cwrapped_analyzer(ranalyzer); Analyzer *a = per_field_analyzer_new(def); Frt_Wrap_Struct(self, &frb_pfa_mark, &frb_analyzer_free, a); object_add(a, self); return self; }
Set the analyzer to be used on field field_name
. Note that
field_name should be a symbol.
field we wish to set the analyzer for
analyzer to be used on field_name
static VALUE frb_per_field_analyzer_add_field(VALUE self, VALUE rfield, VALUE ranalyzer) { Analyzer *pfa, *a; Data_Get_Struct(self, Analyzer, pfa); a = frb_get_cwrapped_analyzer(ranalyzer); pfa_add_field(pfa, frb_field(rfield), a); return self; }
Set the analyzer to be used on field field_name
. Note that
field_name should be a symbol.
field we wish to set the analyzer for
analyzer to be used on field_name
static VALUE frb_per_field_analyzer_add_field(VALUE self, VALUE rfield, VALUE ranalyzer) { Analyzer *pfa, *a; Data_Get_Struct(self, Analyzer, pfa); a = frb_get_cwrapped_analyzer(ranalyzer); pfa_add_field(pfa, frb_field(rfield), a); return self; }
Create a new TokenStream to tokenize
input
. The TokenStream created
will also depend on the field_name
in the case of the PerFieldAnalyzer.
name of the field to be tokenized
data from the field to be tokenized
static VALUE frb_pfa_analyzer_token_stream(VALUE self, VALUE rfield, VALUE rstring) { Analyzer *pfa, *a; Symbol field = frb_field(rfield); GET_A(pfa, self); StringValue(rstring); a = (Analyzer *)h_get(PFA(pfa)->dict, field); if (a == NULL) { a = PFA(pfa)->default_a; } if (a->get_ts == cwa_get_ts) { return rb_funcall(CWA(a)->ranalyzer, id_token_stream, 2, FSYM2SYM(field), rb_str_new2(rs2s(rstring))); } else { return get_rb_ts_from_a(a, rfield, rstring); } }