Index Mapping for nested json for schema registry

Index Mapping for nested json for schema registry - elasticsearch

I have mapped and successfully indexed json files of different formats. But this particular json file which is created from schema registry is weird and i am unable to crack the mapping for it. I have also referenced a lot of articles, but of no help.
Please find below mapping and json file which i am trying to index. It does get indexed, i am not getting the output as i want it to look like.
Mapping:
"mappings":
{
"_doc":
{
"dynamic" : "true",
"properties":
{
"schema":
{
"type": "nested",
"properties":
{
"type":
{
"type": "text"
},
"name":
{
"type": "text"
},
"namespace":
{
"type": "text"
},
"fields":
{
"type": "nested",
"properties":
{
"name":
{
"type": "text"
},
"type":
{
"type": "text"
}
}
}
}
},
"deleted":
{
"type": "boolean"
},
"subject":
{
"type": "text"
},
"id":
{
"type": "integer"
},
"version":
{
"type": "integer"
}
}
}
}
the json file looks like.
{
"schema": "{\"type\":\"record\",\"name\":\"F3253\",\"namespace\":\"ADMIN\",\"fields\":[{\"name\":\"table\",\"type\":[\"null\",\"string\"]},{\"name\":\"op_type\",\"type\":[\"null\",\"string\"]},{\"name\":\"op_ts\",\"type\":[\"null\",\"string\"]},{\"name\":\"current_ts\",\"type\":[\"null\",\"string\"]},{\"name\":\"pos\",\"type\":[\"null\",\"string\"]},{\"name\":\"primary_keys\",\"type\":[\"null\",{\"type\":\"array\",\"items\":\"string\"}]},{\"name\":\"before\",\"type\":[\"null\",{\"type\":\"record\",\"name\":\"row\",\"namespace\":\"\",\"fields\":[{\"name\":\"C1\",\"type\":[\"null\",\"string\"]},{\"name\":\"C2\",\"type\":[\"null\",\"string\"]},{\"name\":\"C3\",\"type\":[\"null\",\"long\"]},{\"name\":\"C4\",\"type\":[\"null\",\"string\"]},{\"name\":\"C5\",\"type\":[\"null\",\"string\"]},{\"name\":\"C6\",\"type\":[\"null\",\"long\"]},{\"name\":\"C7\",\"type\":[\"null\",\"long\"]},{\"name\":\"C8\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940003025\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940003026\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940003027\",\"type\":[\"null\",\"string\"]},{\"name\":\"C800502536\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005001\",\"type\":[\"null\",\"string\"]},{\"name\":\"C536870914\",\"type\":[\"null\",\"string\"]},{\"name\":\"C536870918\",\"type\":[\"null\",\"long\"]},{\"name\":\"C536870919\",\"type\":[\"null\",\"long\"]},{\"name\":\"C536870920\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005060\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005061\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005063\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005064\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005082\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005083\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005085\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005089\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005092\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005093\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005094\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005095\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005096\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005097\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005099\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005100\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005101\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005102\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005104\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005105\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005106\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005107\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005108\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005109\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005110\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005111\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005112\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005113\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005114\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005115\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005122\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005123\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005124\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005125\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005126\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005127\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005128\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005129\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005130\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005141\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005142\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940005143\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005144\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005145\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005509\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005510\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005511\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005512\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005513\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005514\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005515\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005518\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005519\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005520\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005521\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940005522\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940007513\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008202\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008203\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940006953\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008052\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008053\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008054\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008055\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008056\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008057\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008058\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008060\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008061\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940008062\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008063\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008064\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008065\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008066\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940008067\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008068\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008069\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008070\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008071\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008073\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940008075\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008076\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940008352\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009041\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009042\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009043\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009044\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009045\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009046\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009047\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009048\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009049\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009050\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009051\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009052\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940009053\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009054\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009055\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009056\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009057\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009058\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009059\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009062\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009277\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009278\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009279\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009280\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009281\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940009282\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940009295\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009296\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009297\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009760\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940009761\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009762\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009763\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940009764\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009765\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009766\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009767\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009772\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010169\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940007906\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940007907\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940007909\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009512\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009513\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009514\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009516\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009517\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009519\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009520\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009521\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009528\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009530\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009532\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940009533\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010354\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010355\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010356\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010357\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010358\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010359\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010360\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010361\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010362\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010363\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010364\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010365\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010366\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940010367\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010370\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010375\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010376\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010377\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010378\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940010380\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010382\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940010390\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010391\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010392\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940010394\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940010395\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010396\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010404\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010878\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940010879\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010880\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010881\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010882\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940010883\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010884\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940010885\",\"type\":[\"null\",\"string\"]},{\"name\":\"C800502452\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011556\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011557\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011558\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940011559\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940011560\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011561\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011562\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011563\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011564\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011565\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011566\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940011571\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011572\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011573\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940011574\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940011575\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011576\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011577\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011578\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011579\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011580\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011581\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011582\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011583\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011584\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011585\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011586\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011587\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011621\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940011623\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940011629\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011634\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011635\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011636\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011637\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011638\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011639\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011640\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011641\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011642\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011665\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011666\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011672\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011815\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011816\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011821\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011822\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011823\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011824\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011825\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011826\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011827\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940011828\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012160\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012161\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012162\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012163\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940012164\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940012165\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012169\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012170\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012171\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012172\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012648\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940012649\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940012650\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940012651\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940013752\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940013753\",\"type\":[\"null\",\"long\"]},{\"name\":\"F940013754\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940013755\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940013756\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940013757\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940013758\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940014875\",\"type\":[\"null\",\"string\"]},{\"name\":\"F940014876\",\"type\":[\"null\",\"long\"]}],\"connect.name\":\"row\"}]},{\"name\":\"after\",\"type\":[\"null\",\"row\"]}],\"connect.name\":\"ADMIN.F3253\"}",
"deleted": false,
"subject": "app-goldenGate-ass-ADMIN_F3253-value",
"id": 157,
"version": 1
}
I want something where the index shows
metadata.schema.namespace : ....
metadata.schema.name : ....
Not like the below as i am currently getting -
metadata.schema {"type":"record","name":"Envelope","namespace":Bla

Related

Is there a way to know datatype of key in Elasticsearch?

I am facing issue like datatype of key getting changed. On creating index I have datatype as nested but for some reason, it gets changed to object. I make CRUD operations through the painless script but that seems to be fine.
Elastic version 7.3.0
Initial template:
"settings": {
"number_of_shards": 1,
},
"mappings" : {
"properties": {
"deleted_at": { "type": "date" },
"updated_at": { "type": "date" },
"id": { "type": "integer" },
"user_id": { "type": "integer" },
... some more keys
"user_tags": {
"type": "nested"
},
"user_files": {
"type": "nested"
},
}
}
Mapping After some bulk insert/update
"mappings" : {
"properties": {
"deleted_at": { "type": "date" },
"updated_at": { "type": "date" },
"id": { "type": "integer" },
"user_id": { "type": "integer" },
"user_tags": {
"properties": {
...some properties
}
},
"user_files": {
"properties": {
...some properties
}
},
}
}
I have to reindex to fix this issue but it's happening very often. Also is there any way to know the datatype of key whether it is nested or object?
Thanks in advance.

By setting "dynamic": "strict" your mapping won't be changed and unsuitable documents would not be inserted. To solve this problem you need to define all fields that you want to be inside of your nested field. For example:
{
"user_tags": {
"type": "nested",
"properties": {
"code": {
"type": "keyword",
"store": true
},
"score": {
"type": "float",
"store": true
}
}
}
}
If you want to just store a list you can use mapping as below:
{
"user_tags": {
"type": "keyword",
"store": true
}
}
In the second mapping you can store user_tags with this result ["tag1", "tag2", ...]

Root mapping definition has unsupported parameters issue

I am posting to my nearly created Elasticsearch service, but unable to post the mapping. I can not identify which part is unsupported. Any suggestions?
{
"mappings": {
"employees": {
"properties": {
"FirstName": {
"type": "text"
},
"LastName": {
"type": "text"
},
"Designation": {
"type": "text"
},
"Salary": {
"type": "integer"
},
"DateOfJoining": {
"type": "date",
"format": "yyyy-MM-dd"
},
"Address": {
"type": "text"
},
"Gender": {
"type": "text"
},
"Age": {
"type": "integer"
},
"MaritalStatus": {
"type": "text"
},
"Interests": {
"type": "text"
}
}
}
}
}
I consistently get this error returned.
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [employees : {properties={Designation={type=text}, Salary={type=integer}, MaritalStatus={type=text}, DateOfJoining={format=yyyy-MM-dd, type=date}, Address={type=text}, FirstName={type=text}, LastName={type=text}, Gender={type=text}, Age={type=integer}, Interests={type=text}}}]"
}
Using Version 7.6.2

As you didn't specify your Elasticsearch version, which I also asked in my comment,
I have multiple versions of Elasticsearch and as you are using text, I started with Elasticsearch 7 as, you also specified employees in your mapping, which created suspect as this is normally defined in older versions, where types are defined.
Please see removal of types official doc for more info and not it would be completely removed in coming major version of 8.X
I was able to reproduce the issue with Elasticsearch 7.6 and got the same error:
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [employees : {properties={Designation={type=text}, Salary={type=integer}, MaritalStatus={type=text}, DateOfJoining={format=yyyy-MM-dd, type=date}, Address={type=text}, FirstName={type=text}, LastName={type=text}, Gender={type=text}, Age={type=integer}, Interests={type=text}}}]"
}
],
"type": "mapper_parsing_exception",
"reason": "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters: [employees : {properties={Designation={type=text}, Salary={type=integer}, MaritalStatus={type=text}, DateOfJoining={format=yyyy-MM-dd, type=date}, Address={type=text}, FirstName={type=text}, LastName={type=text}, Gender={type=text}, Age={type=integer}, Interests={type=text}}}]",
"caused_by": {
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [employees : {properties={Designation={type=text}, Salary={type=integer}, MaritalStatus={type=text}, DateOfJoining={format=yyyy-MM-dd, type=date}, Address={type=text}, FirstName={type=text}, LastName={type=text}, Gender={type=text}, Age={type=integer}, Interests={type=text}}}]"
}
},
"status": 400
}
Solution
Simply remove the employee from your mapping and use below mapping.
{
"mappings": {
"properties": { --> note `employees` removed.
"FirstName": {
"type": "text"
},
"LastName": {
"type": "text"
},
"Designation": {
"type": "text"
},
"Salary": {
"type": "integer"
},
"DateOfJoining": {
"type": "date",
"format": "yyyy-MM-dd"
},
"Address": {
"type": "text"
},
"Gender": {
"type": "text"
},
"Age": {
"type": "integer"
},
"MaritalStatus": {
"type": "text"
},
"Interests": {
"type": "text"
}
}
}
}
Above create index successfully
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "mustnot"
}

You seem to be using version 7.x of Elasticsearch, which doesn't support mapping types anymore. You simply need to remove employees, like this:
{
"mappings": {
"properties": {
"FirstName": {
"type": "text"
},
"LastName": {
"type": "text"
},
"Designation": {
"type": "text"
},
"Salary": {
"type": "integer"
},
"DateOfJoining": {
"type": "date",
"format": "yyyy-MM-dd"
},
"Address": {
"type": "text"
},
"Gender": {
"type": "text"
},
"Age": {
"type": "integer"
},
"MaritalStatus": {
"type": "text"
},
"Interests": {
"type": "text"
}
}
}
}

In MS Flow, how do I loop through an array and extract values from array?

I have http rest result in this format:
{
"type": "object",
"properties": {
"page": {
"type": "object",
"properties": {
"total": {
"type": "integer"
}
}
},
"list": {
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"type": {
"type": "string"
},
"status": {
"type": "string"
}
},
"required": [
"id",
"type",
"status"
]
}
}
}
}
I am trying to loop through each item in "list" and extract the id, type, status. How do I do this in MS Flow? Here is what I got:
As you can see the variables are not in the dynamic content picker, how do I get it to show up?

"mapper_parsing_exception" error in postman

i'm using postman to communicate with elastic search server and i am receiving an error in postman when i am trying to connect with my elastic search server. Where could i have gone wrong?
Here is my code.
{
"mappings": {
"post": {
"properties": {
"city": {
"type": "text"
},
"contact_email": {
"type": "text"
},
"country": {
"type": "text"
},
"description": {
"type": "text"
},
"image": {
"type": "text"
},
"post_id": {
"type": "text"
},
"state_province": {
"type": "text"
},
"title": {
"type": "text"
}
}
}
}
}
i've tried communicating with my server but i keep receiving this error
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "Root mapping definition has unsupported parameters: [post : {properties={country={type=text}, image={type=text}, post_id={type=text}, city={type=text}, description={type=text}, state_province={type=text}, title={type=text}, contact_email={type=text}}}]"
}

Seems like you are using elasticsearch version 7.0. Since elasticsearch no more support more than one mapping type per index, mapping name is no more required and should not be provided in this version.
Remove the mapping name post from the json input. Use as below:
{
"mappings": {
"properties": {
"city": {
"type": "text"
},
"contact_email": {
"type": "text"
},
"country": {
"type": "text"
},
"description": {
"type": "text"
},
"image": {
"type": "text"
},
"post_id": {
"type": "text"
},
"state_province": {
"type": "text"
},
"title": {
"type": "text"
}
}
}
}

How to debug elastic search error number_format_exception with reason "empty String" but no field name

I just deployed a small application that loads a few thousand docs into an index and when working with production data i get an error in my search request.
http code is 400 and the error is
{
"error": {
"root_cause": [
{
"type": "number_format_exception",
"reason": "empty String"
}
],
"type": "number_format_exception",
"reason": "empty String"
},
"status": 400
}
Okay i kind of get it that my mapping defines some numeric field which i oviously dont store correctly, but how am i supposed to find that field?
each doc contains hundereds of fields.... i mean, really?
I tried looking in /var/log/elasticsearch but nothing useful there...
Please help me i need to get the thing going
I defined my fields as integer which should hold arrays and might be empty. Could that be a problem?
My ES Version is 6.6.0
Update:
The error does occur while searching, during index all is fine
My mapping for that index:
{
"development-object-1551202425": {
"mappings": {
"_doc": {
"dynamic": "false",
"properties": {
"accommodation": {
"properties": {
"badges": {
"properties": {
"maskedProp1": {
"type": "boolean"
},
"maskedProp2": {
"type": "boolean"
},
"maskedProp3": {
"type": "boolean"
},
"maskedProp4": {
"type": "boolean"
},
"maskedProp5": {
"type": "boolean"
},
"maskedProp6": {
"type": "boolean"
}
}
},
"businessTypes": {
"type": "integer"
},
"classification": {
"properties": {
"classification": {
"type": "keyword"
},
"classificationValue": {
"type": "short"
}
}
},
"endowments": {
"type": "integer"
},
"hasPrice": {
"type": "boolean"
},
"lowestPrice": {
"type": "float"
},
"metascore": {
"type": "short"
},
"rating": {
"type": "short"
},
"regionscore": {
"type": "short"
}
}
},
"certificates": {
"type": "integer"
},
"geoLocation": {
"type": "geo_point"
},
"id": {
"type": "text"
},
"isAccommodation": {
"type": "boolean"
},
"location": {
"properties": {
"maskedProp1": {
"type": "integer"
},
"maskedProp2": {
"type": "integer"
},
"id": {
"type": "integer"
},
"name": {
"type": "text",
"fielddata": true
},
"zipcodes": {
"type": "integer"
}
}
},
"maskedProp1": {
"type": "integer"
},
"maskedProp2": {
"type": "integer"
},
"description": {
"type": "text"
},
"sortTitle": {
"type": "keyword"
},
"title": {
"type": "text"
}
}
}
}
}
}
The name consists of an environment string (development) and a timestamp appended (i work with automatic index switching and query for the alias, that does is called {env}-{name}-current.

In my case the error was an empty "size" parameter in the query, i tried to find the error in my filters and did not see that...
A more verbose error message (at least at what property or setting the error occured) could save thousands of hours of debugging all around the world i guess xD.
For now you would have to take apart your dsl section by section to find the issue.

Develop Reference

ruby bash windows laravel spring algorithm oracle macos go visual-studio

Index Mapping for nested json for schema registry - elasticsearch

Related

Is there a way to know datatype of key in Elasticsearch?

Root mapping definition has unsupported parameters issue

In MS Flow, how do I loop through an array and extract values from array?

"mapper_parsing_exception" error in postman

How to debug elastic search error number_format_exception with reason "empty String" but no field name

Categories

Resources