I find myself having to write two db queries and I would like to know whether it's possible to reduce to one.
I am updating a document and then finding the document again in the database. Is it possible to update and receive the full updated document in the response of the update query? This would save a call to the db.
I should explain that the document I'm saving won't be a complete document which is why I'm having to retrieve it from the database to get all the fields.
Yes, it is possible. Please have a look at the documentation of the Query.Apply method, which runs the findAndModify MongoDB command.
Straight from the documentation, this example increments a counter and prints its new value:
change := mgo.Change{
Update: bson.M{"$inc": bson.M{"n": 1}},
ReturnNew: true,
}
info, err = col.Find(M{"_id": id}).Apply(change, &doc)
fmt.Println(doc.N)
Related
Example:
My documents:
{"_id":"1", "data_sent":"100"}
{"_id":"2", "data_sent":"110"}
{"_id":"3", "data_sent":"120"}
I would like to get value of 'data_sent' for every new document and sum it up to another index, lets say
index_name: 'data_sum'
field: 'total_data_sent'='330'
Bonus: I would like to create new indexes automatically for specified time period (for example /week)
I know that aggregations can be used here, but as I understand they are performed when the request is sent and for big data it could last for a while. I need to receive those data very fast when its needed.
Is there anything in Elastic that could help in my case?
I have figured it out by diving deeper into documentation.
'Transforms' was that I was looking for.
https://www.elastic.co/guide/en/elasticsearch/reference/7.9/transform-overview.html
I'm working on an application we're writing in Go, and the application needs to pull some statistics over time. I want to pull some data from a monitoring index we have and group it first by time, and then by the different statuses that the monitor can return at a given point in time.
E.g., for something that's logging the response code when hitting a particular URL, I'd like to get an aggregation first by time slice and then with summary counts for 200, 400, 403, etc. status codes.
How do I do that using the Elastic package?
There are a few things to know here. First, I found it easiest to work out how the package works by looking at the test code in the package. That's how I found the two aggregations I needed, NewAutoDateHistogramAggregation and NewTermsAggregation.
For the nested aggregation, you can achieve that by chaining the sub aggregation onto the first one.
We first create the statusAgg object (the sub-aggregation), and then chain that on as we set up the parent aggregation against time. Note passing in the statusAgg to the SubAggregation chained onto the date histogram aggregation.
Then to execute the aggregation, only the parent aggregation is actually passed to the search context, and the return has the results nested as you would expect.
statusAgg := elastic.NewTermsAggregation().Field("statuscode")
timeAgg := elastic.NewAutoDateHistogramAggregation().
Field("#timestamp").
SubAggregation("status", statusAgg)
result, err := es.Search().
Index(index).
Query(boolQuery).
Aggregation("time", timeAgg).
Size(0).
Pretty(true).
Do(ctx)
I'm quite sure that I want to be notified with the inserted document by each insertion in the couch db.
something like this:
http://localhost:5058/db-name/_chnages/_view/inserted-document
And I like the response to be something like the following:
{
"id":"0552065465",
"name":"james"
.
.
.
}
Reconnecting to the database for giving the actual document by each notification can cause performance issues.
Can I define a view that return the actual document by each change?
There are 3 possible way to define if a document was just added:
You add a status field to your document with a specific status for new documents.
If the revision starts with a 1- but it's not 100% accurate according to this if you do replication.
In the changes response, check if the number of revision of the document is equal to one. If so, it means it was just added(best solution IMO)
If you want to query the _changes endpoint and directly get the newly inserted documents, you can use the approach #1 and use a filter function that only returns documents with status="new".
Otherwise, you should go with approach #3 and filter the _changes responses locally. Eg: your application would receive all changes and only handle documents with revisions array count equal to 1.
And as you mentioned, you want to receive the document, not only the _id and the _rev. To do so, you can simply add the query parameter: include_docs=true
I have an elasticsearch server which i'm accessing via a java server using the Jest client and i was looking for the best way to update multiple fields of a document each time.
I have looked to the documentation so far, and i have found that there are two way for doing it :
Partial update via a script : i don't think it is suitable for multiple field update (because i don't know the modified fields).
Whole document update: via re-indexing the whole document.
My question is how could i update the whole document knowing that Jest provide only update via a script?
Is it the best way to delete a document and indexing the updated version?
Already answered this in the github issue you also opened but again:
You should use the second way you linked (Whole document update) and there is no special API for it, it's just a regular index request. So you can do it simply by sending your Index request against the id of the document you want to update.
For example assuming you have below document already indexed in Elasticsearch within index people, type food, id 9:
{"user": "kramer", "fav_food": "jello"}
Then you would do:
String source = "{\"user\": \"kramer\", \"fav_food\": \"pizza\"}";
JestResult result = client.execute(
new Index.Builder(source)
.index("people")
.type("food")
.id(9)
.build()
);
I'm looking to search for a particular JSON document in a bucket and I don't know its document ID, all I know is the value of one of the sub-keys. I've looked through the API documentation but still confused when it comes to my particular use case:
In mongo I can do a dynamic query like:
bucket.get({ "name" : "some-arbritrary-name-here" })
With couchbase I'm under the impression that you need to create an index (for example on the name property) and use startKey / endKey but this feels wrong - could you still end up with multiple documents being returned? Would be nice to be able to pass a parameter to the view that an exact match could be performed on. Also how would we handle multi-dimensional searches? i.e. name and category.
I'd like to do as much of the filtering as possible on the couchbase instance and ideally narrow it down to one record rather than having to filter when it comes back to the App Tier. Something like passing a dynamic value to the mapping function and only emitting documents that match.
I know you can use LINQ with couchbase to filter but if I've read the docs correctly this filtering is still done client-side but at least if we could narrow down the returned dataset to a sensible subset, client-side filtering wouldn't be such a big deal.
Cheers
So you are correct on one point, you need to create a view (an index indeed) to be able to query on on the content of the JSON document.
So in you case you have to create a view with this kind of code:
function (doc, meta) {
if (doc.type == "youtype") { // just a good practice to type the doc
emit(doc.name);
}
}
So this will create a index - distributed on all the nodes of your cluster - that you can now use in your application. You can point to a specific value using the "key" parameter