Does MongoRepository utilize MongoDB cursor? - spring

Taken from this SO answer:
#Query("{}")
Stream<Alarm> findAllByCustomQueryAndStream();
I'm trying to understand how would that work for a large collection? Does this utilize MongoDB cursor?

All find queries use cursors..

Related

Using SOQL queries in Spring JPA

I have to use SOQL queries within a Spring Data Repository, Is there a way to do so by using #Query annotation ? If not is there a alternative way ?
As far as I know salesforce doesn't expose the table structures. Rather they expose their objects and you can write queries on them. spring-data-jpa is used on top of an entity framework like hibernate. Unless you have entity objects mapped to actual database tables, spring-data-jpa is not useful.
The best way would be to use a query builder like jooq and construct SOQL queries easily using query builders.

Is aggregating outside of Hive a better choice?

I have more of a conceptual question. I'm using Hive to pull data and then I want to insert all the retrieved values into IBM BigSQL (basically DB2) so that aggregating data would be easier/faster. So I want to create a view in Hive that I will use nightly perform CTAS so that I can take the table and migrate it to db2 and do the rest of the aggregation.
Is there a better practice?
I wanted to do everything including aggregation in Hive but it is extremely slow.
Thanks for your suggestions!
Considering that you are using Cloudera, is there a reason why you don't perform the aggregations in Impala? convert the json data to parquet (I would recommend this if there is not a lot of nested structure) shouldn't be really expensive. Another alternative depending the kind of aggregations that you are doing is use Spark to convert the data (also will depend a lot of your cluster size). I would like to give you more specific hints but without know what aggregations you are doing is be complicated

Performance and Security with Laravel Query Builder vs Stored Procedures

Im new to Laravel and have been reading up on its query builder, however haven't been able to find a definite answer. I know that stored procedures are essentially like functions that are stored inside the database to increase performance and security, but does Laravel's query builder work as a replacement for that?
Essentially, would it be necessary to use both stored procedures AND the Laravel query builder, or would using the query builder be enough since the functions are already built into Laravel?
Query builder is not a replacement of stored procedures. It works using PHP PDO. Stored procedures are a set of SQL queries in the compiled form. It helps to reduce network traffic and improve performance.

Use MyBatis cache to load entire table and query

This is what is want to achieve using MyBatis cache(or combining with ehCache/others):
-load the entire result set for an aggregate query into cache
-ability to query this result set and apply sql based filter(between start and end dates)
I searched around the web but could not find an answer to this. Please help.
Suggestions welcome.
Ehcache has search API, you can load entries to cache and afterwords search it based on whatever criteria you like, including dates.
Of course this means implementing caching mechanism yourself, maybe by extending EhcacheCache or since you're using Spring maybe extending AbstractCacheManager or EhCacheCacheManager could be an option.
But there should be performance conciderations, since cache is mean not for querying, but for caching, especially standalone Ehcache version.

NHibernate Criteria query on in-memory collection of entities

I would like to apply a Criteria query to an in-memory collection
of entities, instead of on the database. Is this possible?
To have Criteria API work like LINQ? Or alternatively, convert
Criteria query to LINQ query.
Thanks!
I don't believe you can use Criteria to query against an in-memory collection and come to think about it it doesn't seem to make much sense. If I'm understanding everything correctly you've already queried against your database. I'd suggest to either tune your original query (whichever method you choose) to include all of your filters. Or you could use LINQ (as you suggested) to refine your results.
Also, what's your reasoning for wanting to query from memory?
It sounds like you're rolling your own caching mechanism. I would highly recommend checking out NHibernate's 2nd level cache. It handles many complex scenarios gracefully such as invalidating query results on updates to the underlying tables.
http://ayende.com/Blog/archive/2009/04/24/nhibernate-2nd-level-cache.aspx

Resources