How to search number full size japanese in couchbase - full-text-search

Question

I'm getting error when try full text search number full size "９" in couchbase 6.0.3. Exception throws : err: bleve: QueryBleve validating request, err: parse error: error parsing number: strconv.ParseFloat: parsing.
If i searching with some string "９abc" , searching successfull so i think , lib of couchbase search regconize "９" is number and parse failed. I dont know to to resolve problem. Please help me!
Couchbase 6.0.3
ConjunctionQuery fts = SearchQuery.conjuncts(SearchQuery.queryString(source));
fts = fts.and(SearchQuery.matchPhrase("123").field("tm"));
fts = fts.and(SearchQuery.booleanField(true).field("active"));
SearchQuery query = new SearchQuery("segmentIndex"), fts);
SearchQueryResult result = bucket.query(query);
Exception throws : err: bleve: QueryBleve validating request, err: parse error: error parsing number: strconv.ParseFloat: parsing.

Answer 1

{
"name": "tmSegmentIndex",
"type": "fulltext-index",
"params": {
"doc_config": {
"docid_prefix_delim": "",
"docid_regexp": "",
"mode": "type_field",
"type_field": "type"
},
"mapping": {
"analysis": {
"analyzers": {
"remove_fullsize_number": {
"char_filters": [
"remove_fullsize_number"
],
"token_filters": [
"cjk_bigram",
"cjk_width"
],
"tokenizer": "whitespace",
"type": "custom"
}
},
"char_filters": {
"remove_fullsize_number": {
"regexp": "９",
"replace": "9",
"type": "regexp"
}
}
},
"default_analyzer": "cjk",
"default_datetime_parser": "dateTimeOptional",
"default_field": "_all",
"default_mapping": {
"default_analyzer": "cjk",
"dynamic": true,
"enabled": true
},
"default_type": "_default",
"docvalues_dynamic": true,
"index_dynamic": true,
"store_dynamic": false,
"type_field": "_type"
},
"store": {
"indexType": "scorch",
"kvStoreName": "mossStore"
}
},
"sourceType": "couchbase",
"sourceName": "tm-segment",
"sourceUUID": "973fdbffc567cdfe8f423289b9700f19",
"sourceParams": {},
"planParams": {
"maxPartitionsPerPIndex": 171,
"numReplicas": 0
},
"uuid": "1265a6bedbfd027c"
}

Answer 2

can you just try a custom analyser with an asciifolding character filter like below.
Also, when you directly search from the UI box without a field name, its getting searched in the "_all" field which won't get the right/intended analyser used for parsing the query text.
You may field scope the query there like => field:"９"
{
"type": "fulltext-index",
"name": "FTS",
"uuid": "401ee8132818cee3",
"sourceType": "couchbase",
"sourceName": "sample",
"sourceUUID": "6bd6d0b1c714fcd7697a349ff8166bf8",
"planParams": {
"maxPartitionsPerPIndex": 171,
"indexPartitions": 6
},
"params": {
"doc_config": {
"docid_prefix_delim": "",
"docid_regexp": "",
"mode": "type_field",
"type_field": "type"
},
"mapping": {
"analysis": {
"analyzers": {
"custom": {
"char_filters": [
"asciifolding"
],
"tokenizer": "unicode",
"type": "custom"
}
}
},
"default_analyzer": "standard",
"default_datetime_parser": "dateTimeOptional",
"default_field": "_all",
"default_mapping": {
"dynamic": false,
"enabled": true,
"properties": {
"id": {
"dynamic": false,
"enabled": true,
"fields": [
{
"analyzer": "custom",
"docvalues": true,
"include_in_all": true,
"include_term_vectors": true,
"index": true,
"name": "id",
"type": "text"
}
]
}
}
},
"default_type": "_default",
"docvalues_dynamic": true,
"index_dynamic": true,
"store_dynamic": false,
"type_field": "_type"
},
"store": {
"indexType": "scorch"
}
},
"sourceParams": {}
}
Asciifolding filters are a part of 6.5.0 Couchbase release. Its available in beta for trials.