How to fetch all objects in s3 buckets based on dates? - bash

Here is my usecase - objects in my buckets are suffixed with dates eg:
file-2018-10-10 , file-2018-10-15 etc.
If someone were to enter 2018-10-12, then my tool should download the nearest file in future, which in this case would be file-2018-10-15.
For that I first plan to list the bucket and loop through all the S3-Keys.
Is it possible to list the keys 'reverse-sorted'
Is there any shell script example to list the 'reverse-sorted' list?

No you can't list keys in reverse order, but you don't need to. You can specify the start key when you list a bucket's objects. In your case you would specify file-2018-10-12 as --start-after parameter for the list-object-v2 api.
https://docs.aws.amazon.com/cli/latest/reference/s3api/list-objects-v2.html

Basically, it is the answer to your first question.
You use the simple PHP reverse array function to reverse the array after fetching the result array.
$files_name = $s3->listObjects(array('Bucket' => $bucketName, 'Prefix' => $prefix));
$object2 = $files_name['Contents'];
$object2 = array_reverse($object2);

Related

How can i sort a laravel's elequent list by another ids

I have a challenge that if I want to sort the records when getting using Laravel's ORM based on a list of IDs, how should I do it?!!!!!!!
I mean :
Suppose we have a table called users, which contains 100 records and each record has a unique ID.
We also have an array of IDs.
$ids = [4,1,2,3]
Now I want to get the list of users, but only the users who are first in the ids array and secondly according to the same order as they are listed in this array.
User::whereIn('id' , $Ids)->sortBy('id',$ids)->get();
Can you think of a solution to do this?
User::whereIn('id' , $Ids)->sortBy('id',$ids)->get();
The collections sortBy() function can take a custom call back this way:
$users = User::whereIn('id', $Ids)->get()
->sortBy(function($user, $key) use($ids) {
return array_search($user->id, $ids);
});
This will sort your collection according to the given array.
You can also reference the docs for more information.
Note that the sortBy() function must act upon a collection, which means that the get() function must come before it.

Using Spring MongoTemplate to update nested arrays in MongoDB

Can anyone help with a MongoTemplate question?
I have got a record structure which has nested arrays and I want to update a specific entry in a 2nd level array. I can find the appropriate entry easier enough by the Set path needs the indexes of both array entries & the '$' only refers to the leaf item. For example if I had an array of teams which contained an array of player I need to generate an update path like :
val query = Query(Criteria.where( "teams.players.playerId").`is`(playerId))
val update = Update()
with(update) {
set("teams.$.players.$.name", player.name)
This fails as the '$' can only be used once to refer to the index in the players array, I need a way to generate the equivalent '$' for the index in the teams array.
I am thinking that I need to use a separate Aggregate query using the something like this but I can't get it to work.
project().and(ArrayOperators.arrayOf( "markets").indexOf("")).`as`("index")
Any ideas for this Mongo newbie?
For others who is facing similar issue, One option is to use arrayFilters in UpdateOptions. But looks like mongotemplate in spring does not yet support the use of UpdateOptions directly. Hence, what can be done is:
Sample for document which contain object with arrays of arrayObj (which contain another arrays of arrayObj).
Bson filter = eq("arrayObj.arrayObj.id", "12345");
UpdateResult result = mongoTemplate.getDb().getCollection(collectionName)
.updateOne(filter,
new Document("$set", new Document("arrayObj.$[].arrayObj.$[x].someField"), "someValueToUpdate"),
new UpdateOptions().arrayFilters(
Arrays.asList(Filters.eq("x.id, "12345))
));

Difference between ListResultDto and List

In my application service, why should I return ListResultDto instead of returning List?
It seems like extra work.
On the frontend, it means I need to access the items property instead of treating the results as an array e.g.
Using ListResultDto
.subscribe((next) => this.entities = next.items);
Using List
.subscrube((next) => this.entities = next);
Using ListResultDto allows you to add additional information about your items in the future without breaking your existing API. If you return items as an array, you are restricting your API.
An example is TotalCount in PagedResultDto, if you want to implement pagination of items.

Is there a way to merge multiple properties and then sort via LINQ?

I'm working on a class that has URL and FileName fields. An object can either have a URL or a FileName, but can't have both at the same time.
Is there any way to merge these two fields via LINQ and then sort them? I know I can't use
OrderBy(i => item.URL).ThenBy(i => item.FileName);
because it would just sort the items via URL first and then by their respective FileNames. I need to sort it as if I'm sorting only one field.
Thank you :)
var sorted = list.OrderBy(x => x.URL + x.FileName);
You can pad the URL if desired, or do just about any other operation you need.

multiple levels of associated db objects to YAML

I need to create a 'List' object from the following db tables. I've already done this in a rails/datamapper application, but now I have a need to get specific lists into and out of a db through YAML.
List
Categories
Items
Item choices
e.g. given a list identifier, pull the initial list, the categories for that list, the items for those categories, and the choices for those items into some object, then output as a yaml file.
My first step is output a specific list to yaml, this shouldn't be a unique situation and I'm sure others have solved it before. From reading I'm guessing I need a multilevel hash of some sort, but all I've been able to do so far is get list and category...i.e. this is a bit out of my range right now, and I'm only working from the command line.
I'm asking for two things really to assist in sharpening my skill set:
guidance on working with a multiple level, nested hash situation to properly serialize an object for yaml, given a series of associated db tables
if there is an easier way that someone has already solved.
The included to_json (doc) method already allows you to easily nest related records, and choose what you want to output :
List.all.to_json(:only => {}, :include => {
:categories => { :only => {}, :include => {
:items => { :only => :your_attribute_name }
}
})
The next step is to convert it to yaml :
ActiveSupport::JSON.decode(your_json).to_yaml
Hope this helps

Resources