Can anyone assist me on how to sum of two value from two field( ts + duration) and compare if SUMVALUE is under a RANGE between startDate and endDate. And finally do a sorting under aggregation. Thank You
{
"_index" : "conn",
"_type" : "logs",
"_id" : "AVa8ThhDvPxUqjjShMXt",
"_score" : 0.0,
"_source" : {
"ts" : "2016-08-24T11:26:26.066429Z",
"uid" : "CBnx2VffmLzE5CvE7",
"proto" : "udp",
"duration" : 3.003767,
"orig_bytes" : 696,
"resp_bytes" : 0,
"conn_state" : "S0",
"local_orig" : false,
"local_resp" : false,
"missed_bytes" : 0,
"history" : "D",
"orig_pkts" : 4,
"orig_ip_bytes" : 808,
"resp_pkts" : 0,
"resp_ip_bytes" : 0,
"tunnel_parents" : [ ],
"#version" : "1",
"#timestamp" : "2016-08-24T11:26:26.066Z",
"path" : "/usr/local/logs/current/conn.log",
"host" : "oracle",
"tags" : [ "conn" ],
"id_orig_h" : "10.3.28.100",
"id_orig_p" : 58548,
"id_resp_h" : "239.255.255.250",
"id_resp_p" : 1900
}
} ]
},
Related
I am encountering a problem while updating. The problem I'm having is, for example, I want to update the Title field, but I see that a new field has been created for the Title field in the document. (A title field that starts with a lowercase letter). I'm doing the update process with NEST, can anyone share an idea with me? Thank you in advance for your help.
The state of the document before the update:
{
"_index" : "my_test_index",
"_type" : "_doc",
"_id" : "uPggFnoBChFNLIc8qdjW",
"_score" : 31.908756,
"_source" : {
"RelatedPassiveCompanyId" : "0d075c1681106286cfe9f31999f8247c",
"CreateTime" : "2021-06-16T21:41:17.2697847+03:00",
"Title" : "FE NEW CENTURY INDUSTRY(SINGAPORE)PTE LTD",
"IsBannedFromOpenCorpCompanies" : false,
"CreatedBy" : 1,
"IsActivated" : false,
"IsCancelled" : false,
"IsMembershipTypeBought" : false
}
}
The state of the document after the update:
{
"_index" : "my_test_index",
"_type" : "_doc",
"_id" : "uPggFnoBChFNLIc8qdjW",
"_score" : 26.380388,
"_source" : {
"RelatedPassiveCompanyId" : "0d075c1681106286cfe9f31999f8247c",
"CreateTime" : "2021-06-16T21:41:17.2697847+03:00",
"Title" : "FE NEW CENTURY INDUSTRY(SINGAPORE)PTE LTD",
"IsBannedFromOpenCorpCompanies" : false,
"CreatedBy" : 1,
"IsActivated" : false,
"IsCancelled" : false,
"IsMembershipTypeBought" : false,
"isBannedFromOpenCorpCompanies" : false,
"contactInformations" : {
"contactPerson" : { },
"phones" : [ ]
},
"isCancelled" : false,
"dnbInformation" : {
"processId" : "fba921ee-493d-4f12-aa0a-0a432b9e8b3a",
"requestLogs" : [
{
"requestTime" : "2021-11-23T10:03:09.8302661+03:00",
"message" : "Company not found on Dnb",
"resultType" : 2
}
]
},
"createTime" : "2021-06-16T21:41:17.2697847+03:00",
"createdBy" : 1,
"isMembershipTypeBought" : false,
"isActivated" : false,
"title" : "FE NEW CENTURY INDUSTRY(SINGAPORE)PTE LTD",
"relatedPassiveCompanyId" : "0d075c1681106286cfe9f31999f8247c"
}
},
my update function:
public bool UpdateDocuments(IHit<MyESModel> documentHitItem)
{
var response = elasticClient.Update<MyESModel, object>(DocumentPath<MyESModel>
.Id(documentHitItem.Id), u => u
.Index("my_test_index")
.Doc(documentHitItem.Source)
.DocAsUpsert(true)
.RetryOnConflict(8)
);
return response.IsValid;
}
I would be very happy if anyone has any idea what the problem could be.
Using Hibernate Search 5.9 and elastic server 5.6.10.
I'm trying to persist data from 3 fields into a single field with #Transient annotation. But though the fields shows up in the index structure, the same does not show up when I query the index with curl/chrome. Its not present on the index and the data is lost this way.
Code:
#Transient
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.YES)
private String fullAgentNumber = "";
public String getFullAgentNumber() {
return this.fillr1 +""+ this.rpt0agt0nr +""+ this.fillr2;
}
Result on Index:
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [
{
"_index" : "master_policy_index",
"_type" : "com.csc.pt.svc.data.to.Basclt1400TO",
"_id" : "00,0004087,WCV,05,00",
"_score" : 1.0,
"_source" : {
"id" : "00,0004087,WCV,05,00",
"location" : "00",
"symbol" : "WCV",
"module" : "00",
"policy0num" : "0004087",
"master0co" : "05",
"cltseqnum" : 277,
"addrseqnum" : "1",
"policies" : [
{
"location" : "00",
"symbol" : "WCV",
"module" : "00",
"policy0num" : "0004087",
"master0co" : "05",
"trans0stat" : "P",
"id02" : "02",
"eff0yr" : "118",
"eff0mo" : "03",
"eff0da" : "15",
"exp0yr" : "119",
"exp0mo" : "03",
"exp0da" : "15",
"fillr1" : "000",
"rpt0agt0nr" : "0",
"fillr2" : "358",
"tot0ag0prm" : "0.00",
"line0bus" : "WCV",
"issue0code" : "N",
"type0act" : "NB"
}
]
}
}
]
}
}
Expect the transient field to contain the data am trying to persist while creating the index.
Also I believe once the field has the data, it will too update if the fields its referencing is updated?
You added the #Field annotation on an object field which is, apparently, always empty. Hibernate Search will thus always index an empty string.
You don't need an object field for transient methods. Try this:
#Transient
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.YES)
public String getFullAgentNumber() {
return this.fillr1 +""+ this.rpt0agt0nr +""+ this.fillr2;
}
We are using Elasticsearch v5.6.12 for our database. We update this frequently using the bulk REST api. Some of the time the individual requests won't change anything (i.e. the value of the document that Elasticsearch is already up to date). How can I detect these instances?
I saw this (https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update.html) but I'm not sure it's applicable in our situation.
You can use the noop detection when checking the result of your bulk queries.
When the bulk query returns, you can iterate over each update result and check if the result field has a value of noop (vs updated)
# Say the document is indexed
PUT test/doc/1
{
"test": "123"
}
# Now you want to bulk update it
POST test/doc/_bulk
{"update":{"_id": "1"}}
{"doc":{"test":"123"}} <-- this will yield `result: noop`
{"update":{"_id": "1"}}
{"doc":{"test":"1234"}} <-- this will yield `result: updated`
{"update":{"_id": "2"}}
{"doc":{"test":"3456"}, "doc_as_upsert": true} <-- this will yield `result: created`
Result:
{
"took" : 6,
"errors" : false,
"items" : [
{
"update" : {
"_index" : "test",
"_type" : "doc",
"_id" : "1",
"_version" : 2,
"result" : "noop", <-- see "noop"
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"status" : 200
}
},
{
"update" : {
"_index" : "test",
"_type" : "doc",
"_id" : "1",
"_version" : 3,
"result" : "updated", <-- see "updated"
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 2,
"_primary_term" : 1,
"status" : 200
}
},
{
"_index" : "test",
"_type" : "doc",
"_id" : "2",
"_version" : 1,
"result" : "created", <-- see "created"
"_shards" : {
"total" : 2,
"successful" : 1,
"failed" : 0
},
"_seq_no" : 0,
"_primary_term" : 1
}
]
}
As you can see, when specifying doc_as_upsert: true for document with id 2, the document will be created and the result field value will be created
I'm facing up with a trouble related with document updatings.
I'm able to index(create) documents and they are correctly added on index.
Nevertheless, when I'm trying to update one of them, the operation is not made, the document is not updated.
When I first time add the document it's like:
{
"user" : "user4",
"timestamp" : "2016-12-16T15:00:22.645Z",
"startTimestamp" : "2016-12-16T15:00:22.645Z",
"dueTimestamp" : null,
"closingTimestamp" : null,
"matter" : "F1",
"comment" : null,
"status" : 0,
"backlogStatus" : 20,
"metainfos" : {
"ceeaceaaaceeaceaaaceeaceaaaceeaaceaaaceeabceaaa" : [ "FZ11" ]
},
"resources" : [ ],
"notes" : null
}
This is the code I'm using in order to build UpdateRequest:
this.elasticsearchResources.getElasticsearchClient()
.prepareUpdate()
.setIndex(this.user.getMe().getUser())
.setType(type)
.setId(id.toString())
.setDoc(source)
.setUpsert(source)
.setDetectNoop(true);
I've also been able to debug which's the content of this request begore sending it to elasticsearch. The document is:
{
"user":"user4",
"timestamp":"2016-12-16T15:00:22.645Z",
"startTimestamp":"2016-12-16T15:00:22.645Z",
"dueTimestamp":null,
"closingTimestamp":null,
"matter":"F1",
"comment":null,
"status":0,
"backlogStatus":20,
"metainfos":{
},
"resources":[
],
"notes":null
}
As you can see the only difference is metainfos is empty when I try to update the document.
After having performed this update request the document is not updated. I mean the content of metainfos keeps as before:
#curl -XGET 'http://localhost:9200/user4/fuas/_search?pretty'
{
"took" : 2,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "living_v1",
"_type" : "fuas",
"_id" : "327c9435-c394-11e6-aa90-02420a011808",
"_score" : 1.0,
"_routing" : "user4",
"_source" : {
"user" : "user4",
"timestamp" : "2016-12-16T15:00:22.645Z",
"startTimestamp" : "2016-12-16T15:00:22.645Z",
"dueTimestamp" : null,
"closingTimestamp" : null,
"matter" : "F1",
"comment" : null,
"status" : 0,
"backlogStatus" : 20,
"metainfos" : {
>>>>>>>> "ceeaceaaaceeaceaaaceeaceaaaceeaaceaaaceeabceaaa" : [ "FZ11" ]
},
"resources" : [ ],
"notes" : null
}
} ]
}
}
I don't quite figure out what's wrong. Any ideas?
ElasticSearch will not update an empty object. You can try with:
null "metainfos":null
or
"metainfos":"ceeaceaaaceeaceaaaceeaceaaaceeaaceaaaceeabceaaa":[]
to clean the field.
I'm using ElasticSearch as primary backend infrastructure.
Currently, I need to know whether a field is informed, so, I need to know #documents with a field 'x' > 0.
Imagine a collection with these two documents:
[ {
"_index" : "living_v1",
"_type" : "fuas",
"_id" : "58fb4509-9452-11e6-a361-02420a016207",
"_score" : 1.0,
"_routing" : "living_team",
"_source" : {
"user" : "living_team",
"timestamp" : "2016-10-17T10:29:27.037Z",
"startTimestamp" : "2016-10-17T10:29:27.037Z",
"dueTimestamp" : null,
"closingTimestamp" : null,
"matter" : "FUA1",
"comment" : null,
"status" : 0,
"backlogStatus" : 20,
"metainfos" : {
"cabeaacaceaacadeaacaeeaacafeaa" : [ "s11" ],
"cdbccaeacdbccaeacdbccaeacdbccaeacdbccaea_ldate" : [ "2016-10-19T07:08:23.130Z" ]
},
"resources" : [ ],
"notes" : null
}
}, {
"_index" : "living_v1",
"_type" : "fuas",
"_id" : "2298eab3-9a8a-11e6-8f4a-02420a010a07",
"_score" : 1.0,
"_routing" : "living_team",
"_source" : {
"user" : "living_team",
"timestamp" : "2016-10-25T09:53:23.078Z",
"startTimestamp" : "2016-10-25T09:53:23.078Z",
"dueTimestamp" : null,
"closingTimestamp" : null,
"matter" : "FUA2",
"comment" : null,
"status" : 0,
"backlogStatus" : 20,
"metainfos" : {
"aecfacebfaaecfcebfaaecfcebfaaecfcebfaaecfcebfa" : [ "s22" ]
},
"resources" : [ ],
"notes" : null
}
} ]
I'd like to know how many documents have a field status informed. For example, if I want to know if a field named exfield is informed, the response will be NO, nevertheless, if I want to know if a field named matter is informed the response will be YES.
Does exist anyway to do that?