pass array or slice into go-elasticsearch search query - go

I need to pass array or slice in my go-elasticsearch query for search.
for example:
filters := make([]json.RawMessage, 0)
channelids := []string{"2", "3", "4"}
termFilter, _ := json.Marshal(map[string]interface{}{"term": map[string]interface{}{"channelids": channelids}})
filters = append(filters, termFilter)
termFilter1, _ := json.Marshal(map[string]interface{}{"term": map[string]interface{}{"user": "kimchy"}})
filters = append(filters, termFilter1)
query, _ := json.Marshal(map[string]interface{}{
"query": map[string]interface{}{
"bool": map[string]interface{}{
"must": filters,
},
},
})
querybody1: {"query":{"bool":{"must":[{"term":{"channelids":["2","3","4"]}},{"term":{"user":"kimchy"}}]}}}
when I pass this query it through error:
{
"error" : {
"root_cause" : [
{
"type" : "x_content_parse_exception",
"reason" : "[1:49] [bool] failed to parse field [must]"
}
],
"type" : "x_content_parse_exception",
"reason" : "[1:49] [bool] failed to parse field [must]",
"caused_by" : {
"type" : "illegal_state_exception",
"reason" : "Can't get text on a START_ARRAY at 1:49"
}
},
"status" : 400
}
According to my findings it is due to {"channelids":["2","3","4"]}} in querybody1.
So,How can I manage it? or pass array in search query.

Related

How to query two values from index? Appsync Query Dynamodb

My resolver template
{
"version" : "2017-02-28",
"operation" : "Query",
"query" : {
"expression" : "receiverusername = :receiverusername and createdat > :createdat",
"expressionValues" : {
":receiverusername" : $util.dynamodb.toDynamoDBJson($context.identity.username),
":createdat" : $util.dynamodb.toDynamoDBJson($ctx.args.input.lastDate),
}
},
"index" : "receiverusername-createdat-index",
}
I am trying to query the date and username but I keep on getting this error
{
"data": {
"listMyMessages": null
},
"errors": [
{
"path": [
"listMyMessages"
],
"data": null,
"errorType": "DynamoDB:DynamoDbException",
"errorInfo": null,
"locations": [
{
"line": 2,
"column": 3,
"sourceName": null
}
],
"message": "One or more parameter values were invalid: Condition parameter type does not match schema type (Service: DynamoDb, Status Code: 400, Request ID: 7I18ODEK46H52NMBSF99OTNSQ7VV4KQNSO5AEMVJF66Q9ASUAAJG)"
}
]
How do I query this ?
I tried to learn from documentations but nothing gives me any clue!
Querying 2 values at the same time is not possible for DynamoDB, since every GSI creates a duplicate table with different index. A good solution I can think of is querying indexes in parallel and returning their intersaction.
I just did this and it worked.
And yes I am not touching or investigating why.
{
"version" : "2017-02-28",
"operation" : "Query",
"index" : "receiverusername-createdat-index",
"query" : {
## Provide a query expression. **
"expression": "receiverusername = :receiverusername and createdat > :createdat",
"expressionValues" : {
":receiverusername" : {
"S" : "${context.identity.username}"
},
":createdat": {
"S": "${ctx.args.lastDate}"
}
}
}
}
I did not use sort key but just index keys

Does Elasticsearch accept 1 or 0 for searching on boolean field?

this is part of my elasticsearch query:
{
"match": {
"value": {
"query": "1",
}
}
}
value is a boolean field in my index, does elasticsearch accept 1 as true and 0 as false in search on the boolean field?
as specified in the doc ES accept true/"true"/false/"false" as boolean values.
Other values like 0/1 will throw error in recent versions
error: failed to create query: Can't parse boolean value 1, expected
[true] or [false]
NB: you should use a term query to filter in boolean field
This is easy to test...
First create the index:
PUT test
{
"mappings": {
"properties": {
"bool_field": {
"type": "boolean"
}
}
}
}
Then index a document:
PUT test/_doc/1
{
"bool_field": true
}
Try to query using 0/1 instead of a boolean
POST test/_search
{
"query": {
"term": {
"bool_field": "1"
}
}
}
Response: Can't parse boolean value [1], expected [true] or [false]
{
"error" : {
"root_cause" : [
{
"type" : "query_shard_exception",
"reason" : "failed to create query: Can't parse boolean value [1], expected [true] or [false]",
"index_uuid" : "bZpN3j1kT9KtMBnGkpOmKQ",
"index" : "test"
}
],
"type" : "search_phase_execution_exception",
"reason" : "all shards failed",
"phase" : "query",
"grouped" : true,
"failed_shards" : [
{
"shard" : 0,
"index" : "test",
"node" : "CyVrqrOtR0CP3RfZtdBTag",
"reason" : {
"type" : "query_shard_exception",
"reason" : "failed to create query: Can't parse boolean value [1], expected [true] or [false]",
"index_uuid" : "bZpN3j1kT9KtMBnGkpOmKQ",
"index" : "test",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "Can't parse boolean value [1], expected [true] or [false]"
}
}
}
]
},
"status" : 400
}
PS: It used to be possible up to ES 5. As of ES 6, only true/false are accepted values for boolean fields.

Getting a timestamp exception when I try to update an unrelated field using painless in elasticsearch

Im trying to run the following script
POST /data_hip/_update/1638643727.0
{
"script":{
"source":"ctx._source.avgmer=4;"
}
}
But I am getting the following error.
{
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "failed to parse field [#timestamp] of type [date] in document with id '1638643727.0'. Preview of field's value: '1.638642742E12'"
}
],
"type" : "mapper_parsing_exception",
"reason" : "failed to parse field [#timestamp] of type [date] in document with id '1638643727.0'. Preview of field's value: '1.638642742E12'",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : "failed to parse date field [1.638642742E12] with format [epoch_millis]",
"caused_by" : {
"type" : "date_time_parse_exception",
"reason" : "Failed to parse with all enclosed parsers"
}
}
},
"status" : 400
}
this is strange because on queries (not updates) the date time is parsed fine.
The timestamp field mapping is as follows
"#timestamp": {
"type":"date",
"format":"epoch_millis"
},
I am running elasticsearch 7+
EDIT:
Adding my index settings
{
"data_hip" : {
"settings" : {
"index" : {
"routing" : {
"allocation" : {
"include" : {
"_tier_preference" : "data_content"
}
}
},
"number_of_shards" : "1",
"provided_name" : "data_hip",
"creation_date" : "1638559533343",
"number_of_replicas" : "1",
"uuid" : "CHjkvSdhSgySLioCju9NqQ",
"version" : {
"created" : "7150199"
}
}
}
}
}
Im not running an ingest pipeline
The problem is the scientific notation, the 'E12' suffix, being in a field that ES is expecting to be an integer.
Using this reprex:
PUT so_test
{
"mappings": {
"properties": {
"ts": {
"type": "date",
"format": "epoch_millis"
}
}
}
}
# this works
POST so_test/_doc/
{
"ts" : "123456789"
}
# this does not, throws the same error you have IRL
POST so_test/_doc/
{
"ts" : "123456789E12"
}
I'm not sure how/where those values are creeping in, but they are there in the document you are passing to ES.

Is it possible to combine _msearch and a suggester in Elasticsearch?

I'm trying to bundle multi search api with a term suggester.
When I try to use a suggester on the search endpoint, it works:
POST my-index-000001/_search
{
"query" : {
"match": {
"message": "tring out Elasticsearch"
}
},
"suggest" : {
"my-suggestion" : {
"text" : "tring out Elasticsearch",
"term" : {
"field" : "message"
}
}
}
}
Here's how to try to do the same with multi search:
GET my-index-000001/_msearch
{ }
{
"query" : {
"match": {
"message": "tring out Elasticsearch"
}
},
"suggest" : {
"my-suggestion" : {
"text" : "tring out Elasticsearch",
"term" : {
"field" : "message"
}
}
}
}
{"index": "my-index-000002"}
{"query" : {"match_all" : {}}}
As a result, I get an error:
{
"error" : {
"root_cause" : [
{
"type" : "json_e_o_f_exception",
"reason" : "Unexpected end-of-input: expected close marker for Object (start marker at [Source: (org.elasticsearch.common.bytes.AbstractBytesReference$MarkSupportingStreamInputWrapper); line: 1, column: 1])\n at [Source: (org.elasticsearch.common.bytes.AbstractBytesReference$MarkSupportingStreamInputWrapper); line: 1, column: 2]"
}
],
"type" : "json_e_o_f_exception",
"reason" : "Unexpected end-of-input: expected close marker for Object (start marker at [Source: (org.elasticsearch.common.bytes.AbstractBytesReference$MarkSupportingStreamInputWrapper); line: 1, column: 1])\n at [Source: (org.elasticsearch.common.bytes.AbstractBytesReference$MarkSupportingStreamInputWrapper); line: 1, column: 2]"
},
"status" : 400
}
So my question is, is it even possible?
I found a similarly named question (Using Nest Phrase Suggester on MultiSearch query), but the question itself appears to be about multi-match queries, not multi search.
The version I'm using is 7.13.
Your queries are syntactically correct but the Multi Search API (_msearch) only supports newline-delimited json payloads (ndjson).
As such, line breaks (\n) can only occur when separating the payload header from the payload body:
header\n
body\n
header\n
body\n
...
So, adjust your line breaks to conform with the required structure:
GET my-index-000001/_msearch
{}
{"query":{"match":{"message":"tring out Elasticsearch"}},"suggest":{"my-suggestion":{"text":"tring out Elasticsearch","term":{"field":"message"}}}}
{"index":"my-index-000002"}
{"query":{"match_all":{}}}
Tip: if you're using Kibana (and it looks like you are), you can toggle between the expanded and the _msearch-valid mode with ⌘+i / ctrl+i.

extract data from csv failed

so i tried to extract data from csv, but its seems i failed
i have already tried, but it seems i always got it wrong
this is my message data
"message" : """42307;"FX2CHTPEKAFB";"PACKING CYL COP JUPITER Z FUBORU";"PCS";"";"";"";"";;"""""
this is my pattern
"patterns": ["""%{DATA:id_product};"%{DATA:code_product}";"%{DATA:name_product}";"%{DATA:satuan_product}";"%{DATA:merek_vehicle}";"%{DATA:jenis_vehicle}";"%{DATA:merek_product}";"%{DATA:part_number}";%{DATA:weight:float};"%{DATA:unit_weight}""""]"patterns": ["""%{DATA:id_product};"%{DATA:code_product}";"%{DATA:name_product}";"%{DATA:satuan_product}";"%{DATA:merek_vehicle}";"%{DATA:jenis_vehicle}";"%{DATA:merek_product}";"%{DATA:part_number}";%{DATA:weight:float};"%{DATA:unit_weight}""""]
my result
"docs" : [
{
"error" : {
"root_cause" : [
{
"type" : "exception",
"reason" : """java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [42305;"FX4PER000501I";"PER DPN F-50 DH-0005-01 48110-87624-01 MITS";"PCS";"DAIHATSU";"";"INDOSPRING";"";;]""",
"header" : {
"processor_type" : "grok"
}
}
],
"type" : "exception",
"reason" : """java.lang.IllegalArgumentException: java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [42305;"FX4PER000501I";"PER DPN F-50 DH-0005-01 48110-87624-01 MITS";"PCS";"DAIHATSU";"";"INDOSPRING";"";;]""",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : """java.lang.IllegalArgumentException: Provided Grok expressions do not match field value: [42305;"FX4PER000501I";"PER DPN F-50 DH-0005-01 48110-87624-01 MITS";"PCS";"DAIHATSU";"";"INDOSPRING";"";;]""",
"caused_by" : {
"type" : "illegal_argument_exception",
"reason" : """Provided Grok expressions do not match field value: [42305;"FX4PER000501I";"PER DPN F-50 DH-0005-01 48110-87624-01 MITS";"PCS";"DAIHATSU";"";"INDOSPRING";"";;]"""
}
},
"header" : {
"processor_type" : "grok"
}
}
It doesn't like %{DATA:weight:float}.
If you remove :float, giving:
%{DATA:id_product};"%{DATA:code_product}";"%{DATA:name_product}";"%{DATA:satuan_product}";"%{DATA:merek_vehicle}";"%{DATA:jenis_vehicle}";"%{DATA:merek_product}";"%{DATA:part_number}";%{DATA:weight};"%{DATA:unit_weight}
You will get:
{
"name_product": "PACKING CYL COP JUPITER Z FUBORU",
"jenis_vehicle": "",
"satuan_product": "PCS",
"weight": "",
"id_product": "42307",
"merek_vehicle": "",
"code_product": "FX2CHTPEKAFB",
"merek_product": "",
"part_number": "",
"unit_weight": ""
}

Resources