I'm new to Couchdb, i don't know their design.
I know we have to use emit() to make keys for searching.
How to make a request on view with these conditions (Like SQL)?
Where A = 'b' && Date >= 2010 && Date <= 2020
You need to emit a multi-variate key, something like
# map
function(doc) {
if (doc.A && doc.Date) {
emit([doc.A, doc.Date], 1)
}
}
# reduce
_count
You can then query it like
curl 'https://host.com/DB/_design/DDOC/_view/VIEW?reduce=false&include_docs=true&inclusive_end=true&startkey=\["a",2010\]&endkey=\["a",2020\]'
You can try it yourself on a database I made, which is open for reading.
curl 'https://skruger.cloudant.com/stackex/_design/examples/_view/by-a-date?reduce=false&include_docs=true&inclusive_end=true&startkey=\["a",2010\]&endkey=\["a",2020\]'
If you're using CouchDB 2.X, you can use Mango as an alternative. I find it easier to use and understand.
{
"A":"b",
"$and":[
{
"Date": {
"$gte":2010
}
},
{
"Date":{
"$lte":2020
}
}
]
}
You probably want to build an index for this query to be has fast as the other alternative response.
Related
I'm using Contentful's GraphQL API. What I want to do is to query all the events that haven't past yet.
I tried using lt, but that doesn't seem to be working. I also found out that the date is a string, so what options do I have?
eventCollection(where: {eventEndDate: {lt: "2022-10-27T00:00:00.000-06:00"}}){
items {
slug
eventEndDate
}
}
A normal query (without the where condition) gives you:
"eventCollection": {
"items": [
{
"slug": "black-friday",
"eventEndDate": "2022-11-27T12:00:00.000-07:00"
}
]
}
You should have an eventEndDate_gte filter available. On every field, there will be type dependent filter available. It's best to use GraphiQL or the GraphQL Playground to discover available filter options.
The following filter works fine for my space.
query {
tilPostCollection(where: {date_gte: "2022-09-05T00:00:00.000+02:00"}) {
items {
title
date
}
}
}
As per documentation it is possible to provide a hint to an update.
Now I'm using the java mongo client and mongo collection to do an update.
For this update I cannot find any way to provide a hint which index to use.
I see for the update I'm doing a COLSCAN in the logs, so wanting to provide the hint.
this.collection.updateOne(
or(eq("_id", "someId"), eq("array1.id", "someId")),
and(
addToSet("array1", new Document()),
addToSet("array2", new Document())
)
);
Indexes are available for both _id and array1.id
I found out in the logs the query for this update is using a COLSCAN to find the document.
Anyone who can point me in the right direction?
Using AWS DocumentDB, which is MongoDB v3.6
Lets consider a document with an array of embedded documents:
{ _id: 1, arr: [ { fld1: "x", fld2: 43 }, { fld1: "r", fld2: 80 } ] }
I created an index on arr.fld1; this is a Multikey index (indexes on arrays are called as so). The _id field already has the default unique index.
The following query uses the indexes on both fields - arr.fld1 and the _id. The query plan generated using explain() on the query showed an index scan (IXSCAN) for both fields.
db.test.find( { $or: [ { _id: 2 }, { "arr.fld1": "m" } ] } )
Now the same query filter is used for the update operation also. So, the update where we add two sub-documents to the array:
db.test.update(
{ $or: [ { _id: 1 }, { "arr.fld1": "m" } ] },
{ $addToSet: { arr: { $each: [ { "fld1": "xx" }, { "fld1": "zz" } ] } } }
)
Again, the query plan showed that both the indexes are used for the update operation. Note, I have not used the hint for the find or the update query.
I cannot come to conclusion about what the issue is with your code or indexes (see point Notes: 1, below).
NOTES:
The above observations are based on queries run on a MongoDB server
version 4.0 (valid for version 3.6 also, as I know).
The
explain
method is used as follows for find and update:
db.collection.explain().find( ... ) and
db.collection.explain().update( ... ).
Note that you cannot generate a query plan using explain() for
updateOne method; it is only available for findAndModify() and
update() methods. You can get a list of methods that can generate a
query plan by using the command at mongo shell:
db.collection.explain().help().
Note on Java Code:
The Java code to update an array field with multiple sub-document add, is as follows:
collection.updateOne(
or(eq("_id", new Integer(1)), eq("arr.fld1", "m")),
addEachToSet("arr", Arrays.asList(new Document("fld1", "value-1"), new Document("fld1", "value-2"))
);
I have documents like:
{
MyProp: ["lorem", "ipsum", "dolor"]
... lots of stuff here ...
}
My documents can be quite big (but these MyProp fields are not), and expensive to generate from scratch.
Sometimes I need to update batches of these - it would therefore be beneficial to do a partial update (to save "indexing client" processing power and bandwidth, and thus time) and replace the MyProp values with new values.
Example of original document:
{
MyProp: ["lorem", "ipsum", "dolor"]
... lots of stuff here ...
}
Example of updated document (or rather how it should look):
{
MyProp: ["dolor", "sit"]
... lots of stuff here ...
}
From what I have seen, this includes scripting.
Can anyone enlighten me with the remaining bits of the puzzle?
Bounty added:
I'd like to also have some instructions of how to make these in a batch statement, if possible.
You can use the update by query API in order to do batch updates. This works since ES 2.3 onwards, otherwise you need to install a plugin.
POST index/_update_by_query
{
"script": {
"inline": "ctx._source.myProp += newProp",
"params": {
"newProp": "sit"
}
},
"query": {
"match_all": {}
}
}
You can of course use whatever query you want in order to select the documents on which MyProp needs to be updated. For instance, you could have a query to select documents having some specific MyProp values to be replaced.
The above will only add a new value to the existing array. If you need to completely replace the MyProp array, then you can also change the script to this:
POST index/_update_by_query
{
"script": {
"inline": "ctx._source.myProp = newProps",
"params": {
"newProps": ["dolor", "sit"]
}
},
"query": {
"match_all": {}
}
}
Note that you also need to enable dynamic scripting in order for this to work.
UPDATE
If you simply want to update a single document you can use the partial document update API, like this:
POST test/type1/1/_update
{
"doc" : {
"MyProp" : ["dolor", "sit"]
}
}
This will effectively replace the MyProp array in the specified document.
If you want to go the bulk route, you don't need scripting to achieve what you want:
POST index/type/_bulk
{ "update" : {"_id" : "1"} }
{ "doc" : {"MyProp" : ["dolor", "sit"] } }
{ "update" : {"_id" : "2"} }
{ "doc" : {"MyProp" : ["dolor", "sit"] } }
Would a _bulk update work for you?
POST test/type1/_bulk
{"update":{"_id":1}}
{"script":{"inline":"ctx._source.MyProp += new_param","params":{"new_param":"bla"},"lang":"groovy"}}
{"update":{"_id":2}}
{"script":{"inline":"ctx._source.MyProp += new_param","params":{"new_param":"bla"},"lang":"groovy"}}
{"update":{"_id":3}}
{"script":{"inline":"ctx._source.MyProp += new_param","params":{"new_param":"bla"},"lang":"groovy"}}
....
And you would also need to enable inline scripting for groovy. What the above would do is to add a bla value to the listed documents in MyProp field. Of course, depending on your requirements many other changes can be performed in that script.
In ElasticSearch I'm using upsert to update a document that may not exist:
POST /website/pageviews/1/_update
{
"script" : "ctx._source.online+=1",
"upsert": {
"online": 1
}
}
Since my data are going to change frequently I want to remove my document if online == 0.
It would be useless to use update if I need to get the document and check online value every time, and I don't want to accumulate a lot of trash documents.
Which is the best way to remove my document when online == 0? Something like:
POST /website/pageviews/1/_update
{
"script" : "ctx._source.online-=1",
"remove_doc": "ctx._source.online == 0"
}
You can use the delete operation like this:
POST /website/pageviews/1/_update
{
"script" : "if (online == 0) { ctx.op = 'delete' } else { ctx._source.online += 1 }",
"upsert": {
"online": 1
}
}
So, since I'm too dumb obviously to figure this out myself, I'll ask you better folks here on SO instead.
Basically i have a datastructure that looks like the following:
....,
{
"id": 12345
....
"policy_subjects": [
{
"compiled": "^(user|max|anonymous)$",
"template": "<user|max|anonymous>"
},
{
"compiled": "^max$",
"template": "max"
}
]
....
}
compiled is a "compiled" regex
template is the same regex without regex-modifiers
What I want is to do a simple query in RethinkDB using the "compiled" value and matching that against a string, say "max".
Basically
r.table("regex_policies").filter(function(policy_row) {
return "max".match("(?i)"+policy_row("policy_subjects")("compiled"))
}
Is what i want to do (+case-insensitive search)
There are of course lots of policy_subjects in the database so in this example the result should be the whole dataset (1 result) that matches "max". Since "max" exists twice in this case and it matches both regexes (once would have been enough).
"foobar" would likewise in this example yield 0 results, since any of the compiled regexes does not match "foobar".
Does anyone know how to do this relatively simple query?
You definitely want to use r.expr here and I got this example to work:
r.expr([{
"id": 12345,
"policy_subjects": [
{
compiled: "^(user|max|anonymous)$",
template: "<user|max|anonymous>"
},
{
compiled: "^max$",
template: "max"
}
]
}]).merge(function(policy_row) {
return {
"policy_subjects": policy_row("policy_subjects").filter(function(item){
return r.expr("max").match(r.expr("(?i)").add(item("compiled"))).ne(null);
})
}
})
Changing max to something else that does not match, returns the document with no elements inside policy_subjects.
For example, changing max => to wat (my favorite test string of all time) looks like this:
.merge(function(policy_row) {
return {
"policy_subjects": policy_row("policy_subjects").filter(function(item){
return r.expr("wat").match(r.expr("(?i)").add(item("compiled"))).ne(null);
})
}
})
And results in this:
[
{
"id": 12345 ,
"policy_subjects": [ ]
}
]
I think your logic for reducing to the one policy_subject document you want might be a little subjective to your use case so I'm not sure what the right answer is but you can use .reduce(...) to just return the right-most value.