I am wanting to know how to create an index in rethinkdb that will return rows in the order they were added, to use it as a kind of log.
You will want to set a datetime field of some sort in your documents like so:
# Shorthand for table
test = r.db("test").table("test")
# Create index
test.createIndex("datetime", r.row("datetime"))
# Insert document with datetime field
test.insert({
datetime: r.now(),
})
# To get all documents in sorted order
test.order_by(index="datetime")
# To get documents after a certain point
test.between(<some sort of datetime object>, r.maxval, index="datetime")
https://www.rethinkdb.com/api/python/order_by/
https://rethinkdb.com/api/python/between/
Related
I have clear code for ordering a DynamoDB scan by ascending or descending using the
response = table.query(
ScanIndexForward=False # true = ascending, false = descending
)
argument. Likewise I have the boto paginator for paginating responses using the following:
paginator = dynamodb.get_paginator('scan')
response_iterator = paginator.paginate(
TableName=table.table_name,
PaginationConfig={"MaxItems": 25, "PageSize": 1}
)
But I am unable to find an optional argument or method to do both. The order returned by the paginator class appears to be random.
Is there a way to order the notifications by ascending or descending and then split into paginated shards?
I have investigated the optional arguments passed to the paginator scan in the documentation but ScanIndexForward is not an optional argument on SCAN and there is no ASC or DESC option in the conditions that can be passed to ScanFilter.
The table is created within the python CDK with the following partition and sort keys:
dynamodb.Table(
self,
"NotificationsTable",
partition_key=dynamodb.Attribute(
name="_id", type=dynamodb.AttributeType.STRING
),
sort_key=dynamodb.Attribute(
name="Date", type=dynamodb.AttributeType.NUMBER
)
)
You cannot order a Scan operation as it spans multiple partition keys, each pk will return at random, but in order by its accompanying sort-key. Your original request was a Query with ScanIndexForward, this only returns a single partition key ordered by the sort key.
I'm learning Elasticsearch API while practicing I'm facing the issue is unable to fetch documents between two dates those documents match two fields but without date range it's working fine
BoolQueryBuilder filter = new BoolQueryBuilder();
BoolQueryBuilder query = QueryBuilders.boolQuery();
for (String q : list) {
// both the fields must exists
query = QueryBuilders.boolQuery().must(QueryBuilders.matchQuery("field1", q))
.must(QueryBuilders.matchQuery("field2", val));
filter.should(query);
}
filter.must(QueryBuilders.rangeQuery("datetime").gte(from).lte(to);
searchSourceBuilder.query(filter);
Where,
list contains the list of words for the field1 field.
Both field1 & field2 must match such document I want to retrieve
datetime is a custom datetime field & the value looks like 2022-06-09 12:32:36
Can anyone help me to resolve this issue
I think you need, Date format to convert date values in the query. To format your dates, either use the built-in formats provided by ES - https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html
or, you can try customised format - https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html
After formatting, gte and lte should work as expected.
I have an index with 2 index pattern using alias.
Example:
Index Name: my_index
Fields: sender_name, receiver_name, item_name
Alias: my_index_alias_1, my_index_alias_2
Index Patterns: my_index_alias_1, my_index_alias_2
I have a dashboard with two data tables using my_index_alias_1 and my_index_alias_2.
Same person can also be sender and receiver but there should be only one filter to select the user.
Example:
If a user named Bob is filtered.
my_index_alias_1 Data Table should filter by received_name
my_index_alias_2 Data Table should filter by sender_name
I don't want do have duplicate index, so I think scripted field is the better option.
But scripted field can solve this only when I can access the alias name using doc_value, so then I can write condition like the below Pseudocode
if doc['_alias'].value=='my_index_alias_1' then doc['received_name'].value
if doc['_alias'].value=='my_index_alias_2 ' then doc['sender_name'].value
Here is requirement I am working on
There are multiple indexes with name content_ssc, content_teal, content_mmy.
These indexes can have common data (co_code is one of the field in the documents of these indexes)
a. content_ssc can have documents with co_code = teal/ssc/mmy
b. content_mmy can have documents with co_code = ssc/mmy
I need to get the data using below condition (this is one of the approach to get the unique data from these indexes)
a. (Index = content_ssc and site_code = ssc) OR (Index = content_mmy and site_code = mmy)
Basically I am getting a duplicate data from these indexes currently so I need any solution which should fetch unique data from these indexes using the above condition.
I have tried using boolean query with multiple indices from this link but it didn't produce unique result.
Please suggest.
You can use distinct query , and you will get unique result
From the docs:
Default value is "logstash-%{+YYYY.MM.dd}"
I'm wondering where logstash gets the information for YYYY.MM.dd? Is it the #timestamp field? And if so, can it be told to use a different field (#mydate, for example)?
The index YYYY.MM.dd is according to the #timestamp time.
You can refer to elasticsearch.rb about the 'event.sprintf' to print the logstash index.
index = event.sprintf(#index)
and then you can study event.rb to see what the sprintf do.
t = #data["#timestamp"]
formatter = org.joda.time.format.DateTimeFormat.forPattern(key[1 .. -1])\
.withZone(org.joda.time.DateTimeZone::UTC)
#next org.joda.time.Instant.new(t.tv_sec * 1000 + t.tv_usec / 1000).toDateTime.toString(formatter)
# Invoke a specific Instant constructor to avoid this warning in JRuby
# > ambiguous Java methods found, using org.joda.time.Instant(long)
org.joda.time.Instant.java_class.constructor(Java::long).new_instance(
t.tv_sec * 1000 + t.tv_usec / 1000
).to_java.toDateTime.toString(formatter)
So, If you want the index follow your own field, you have to modify the event.rb to use your own field instead of timestamp. Or you can change the timestamp value to your own field time.