JS - client.bulk() results in not mapped documents - elasticsearch

I have a template with a mapping like this:
"template" : "database*",
"mappings": {
"_default_" : {
"dynamic_templates" : [
{
// default string mapping
"string" : {
"match" : "*",
"match_mapping_type": "string",
"mapping": {
"type": "text",
"fields": {
"raw" : {
"type": "keyword"
}
}
}
}
},
]
}
}
The idea behind this template is to generate for every new field a "type : keyword" field for exact searches.
When adding documents (in an empty index using this template) with JS API client.index() everything works fine, this way i am able to query like:
{
"query": {
"match": {
"fooBar": "bla"
}
}
}
Or for exact searches like:
{
"query": {
"match": {
"fooBar.raw": "bla"
}
}
}
But adding documents (again empty index) in bulk with client.bulk() like this:
client.bulk({
body: [
// FIRST DOCUMENT
{ index: { _index: 'myindex', _type: 'mytype', _id: 1 } },
// the first document to index
{ fooBar: 'foo' },
// SECOND DOCUMENT
{ index: { _index: 'myindex', _type: 'mytype', _id: 2 } },
// the second document to index
{ fooBar: 'bar' },
(...)
]
}, function (err, resp) {
// ...
});
The same "raw" request results in an error:
"No mapping found for [fooBar.raw] (...)"
while:
{
"query": {
"match": {
"fooBar": "bla"
}
}
}
delivers results. This brings me to the conclusion, that the document is indeed indexed but the field "raw" has not been created.
Is that right? Is in fact bulk Indexing not using mapping?
How can I use bulk import and map the documents?
Thanks! :)

Related

ElasticSearch sorting based with wildcard in key

I have a data structure something like this from query. I want to apply a sort based on the date in the object values.
{
users: {
"1234": {
name: "User 1",
joining_date: "2022-12-28T11:37:00.000Z"
},
"3456": {
name: "User 2",
joining_date: "2022-12-18T11:37:00.000Z"
}
}
}
This is my query so far.
GET /_search
{
"sort" : [ {
"users.*.joining_date": {
"order": "desc",
"format": "date",
"unmapped_type": "long"
} }
],
"query": {
"query_string": {
"query": "_schema:users"
}
}
}
The problem is with using a wildcard in the key. I have tried multiple combinations from the documentation but nothing worked so far. I will be grateful for any help.

How to formulate an and condition for a nested object in Opensearch?

Opensearch ingests documents similar to this example (its just a minimal example):
PUT nested_test/_doc/4
{
"log": "This is a fourth log message",
"function": "4 test function",
"related_objects": [
{ "type": "user", "id": "10" },
{ "type": "offer", "id": "120" }
]
}
PUT nested_test/_doc/5
{
"log": "This is a fifth log message",
"function": "5 test function",
"related_objects": [
{ "type": "user", "id": "120" },
{ "type": "offer", "id": "90" }
]
}
With many of these documents, I'd like to filter those which have a specific related object (e.g. type=user and id=120). With the example data above, this should only return the document with id 5. Using simple filters (DQL syntax) as follows does not work:
related_objects.type:user and related_objects.id:120
As this would also match a document 5, as there is a related_object with type user and a related object with id 120, although its not the related user object with id 120, its the related offer.
If Array[object] is used, the field type is nested, The document reference
Elasticsearch query example:
{
"query" : {
"nested" : {
"path" : "related_objects",
"query" : {
"bool" : {
"must" : [
{
"term" : {"related_objects.type" : "MYTYPE"}
},
{
"term" : {"related_objects.id" : "MYID"}
}
]
}
}
}
}
}
Basically just go into a nested query and specify all your AND conditions as MUST clauses inside a bool query.
As soon as the field is declared as nested field, it is possible to run a simple DQL query to get the desired information:
related_objects:{type:"user" and id:120}
This requires that the field has been defined as nested before:
PUT my-index-000001
{
"mappings": {
"properties": {
"related_objects": {
"type": "nested"
}
}
}
}

Search for documents with exactly different fields values

I'm adding documents with the following strutucte
{
"proposta": {
"matriculaIndicacao": 654321,
"filial": 100,
"cpf": "12345678901",
"idStatus": "3",
"status": "Reprovada",
"dadosPessoais": {
"nome": "John Five",
"dataNascimento": "1980-12-01",
"email": "fulanodasilva#fulano.com.br",
"emailValidado": true,
"telefoneCelular": "11 99876-9999",
"telefoneCelularValidado": true,
"telefoneResidencial": "11 2211-1122",
"idGenero": "1",
"genero": "M"
}
}
}
I'm trying to perform a search with multiple field values.
I can successfull search for a document with a specific cpf atribute with the following search
{
"query": {
"term" : {
"proposta.cpf" : "23798770823"
}
}
}
But now I need to add an AND clause, like
{
"query": {
"term" : {
"proposta.cpf" : "23798770823"
,"proposta.dadosPessoais.dataNascimento": "1980-12-01"
}
}
}
but it's returning an error message.
P.S: If possible I would like to perform a search where if the field doesn't exist, it returns the document that matches only the proposta.cpf field.
I really appreciate any help.
The idea is to combine your constraints within a bool/should query
{
"query": {
"bool": {
"should": [
{
"term": {
"proposta.cpf": "23798770823"
}
},
{
"term": {
"proposta.dadosPessoais.dataNascimento": "1980-12-01"
}
}
]
}
}
}

ElasticSearch filtering for a tag in array

I've got a bunch of events that are tagged for their audience:
{ id = 123, audiences = ["Public", "Lecture"], ... }
I've trying to do an ElasticSearch query with filtering, so that the search will only return events that have the an exact entry of "Public" in that audiences array (and won't return events that a "Not Public").
How do I do that?
This is what I have so far, but it's returning zero results, even though I definitely have "Public" events:
curl -XGET 'http://localhost:9200/events/event/_search' -d '
{
"query" : {
"filtered" : {
"filter" : {
"term" : {
"audiences": "Public"
}
},
"query" : {
"match" : {
"title" : "[searchterm]"
}
}
}
}
}'
You could use this mapping for you content type
{
"your_index": {
"mappings": {
"your_type": {
"properties": {
"audiences": {
"type": "string",
"index": "not_analyzed"
},
}
}
}
}
}
not_analyzed
Index this field, so it is searchable, but index the
value exactly as specified. Do not analyze it.
And use lowercase term value in search query

Elasticsearch boost with Wildcardsearch on _all

Im trying to search documents with wildcard and _all. But It does not seem like it's possible to get boosted result with wildcard on _all ?
MappingRequest:
"theboostingclass": {
"properties": {
"Important": {
"boost": 2.0,
"type": "string"
},
"LessImportant": {
"type": "string"
},
"Garbage": {
"type": "string"
}
}
}
}
Indexing:
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"bomb",
"LessImportant":"kruka",
"Garbage":"kalkon"
}
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"kalkon",
"LessImportant":"bomb",
"Garbage":"bomber"
}
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"kruka",
"LessImportant":"bomber",
"Garbage":"bomb"
}
Query
"query": {
"wildcard": {
"_all": {
"value": "*bomb*"
}
}
}
The result returs all hits with a Score of 1 and a seemingly random order. Which is not really what Im after. I want the hit on "Important"field to yield a higher score.
If I do a wildcard search on all 3 fields the scoring seems correct. However I want to use it on _all. Any ideas?
Please see documentation here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-term-rewrite.html. Note that the reason it works with a constant scoring by default is for performance.
I believe you need to modify your query as follows:
"query": {
"wildcard": {
"_all": {
"value": "*bomb*",
"rewrite": "scoring_boolean"
}
}
}

Resources