I am new to springBatch, and I am looking how to do a SQL query Limit.
In the reading process, I am getting my ResultSet from a DB entry. I tried setMaxItemCount to limit the number of rows but it did not work. It still reading all DB entries.
public class PeopleReader extends RepositoryItemReader<Product> {
public People read () throws Exception {
DataSourceContextHolder.set("people");
return this.doRead();
}
public PeopleReader(final PeopleRepository peopleRepository) {
DataSourceContextHolder.set("people");
this.setRepository(peopleRepository);
this.setMaxItemCount(100);
this.setSort(new HashMap<>());
this.setPageSize(100);
}
}
#Repository
public interface PeopleRepository extends JpaRepository<People, String> {
}
How to limit the whole resultset from the DataSource to 100 for example ( as SQL queyr Limit 100) ?
Related
I'm having some trouble making MongoDB Date queries using #Query annotation on SpringDataMongoDB on a project created using JHipster.
Since JHipster was used to create the project most of the queries were created using Spring Data query builder mechanism and for more refined queries, instead of using Type-safe Query methods I decided to stick with JHipster's standard configuration and make personalized queries using #Query annotation that allows the creation of MongoDBJSON queries.
However, I can't reference in my Json queries any entity field of type Date or LocalDate.
I tried to adopt as a solution the answer from this thread without success.
Query attempts
#Repository
public interface CourseClassRepository extends MongoRepository<CourseClass, String> {
// WORKS - query with `endDate` directly constructed by Spring Data
// This sollution however isn't enought, since 'experience_enrollments.device_id' cannot be used as a parameter
List<CourseClass> findAllByInstitutionIdAndEndDateIsGreaterThanEqual(Long institutionId, LocalDate dateLimit);
// Using #Query to create a JSON query doesn't work.
// apparently data parameter cannot be found. This is weird, considering that in any other #Query created the parameter is found just fine.
// ERROR: org.bson.json.JsonParseException: Invalid JSON input. Position: 124. Character: '?'
#Query(" { 'experience_enrollments.device_id' : ?0, 'institution_id': ?1, 'end_date': { $gte: { $date: ?2 } } } ")
List<CourseClass> findAllByExperienceDeviceAndInstitutionIdAndEndDate(String deviceId, Long institutionId, Date dateLimit);
// Adopting the stackoverflow answer mentioned above also throws an error. I belive that this error is related to the fact that '?2' is being interpreted as a String value and not as reference to a parameter
// ERROR: org.bson.json.JsonParseException: Failed to parse string as a date
#Query(" { 'experience_enrollments.device_id' : ?0, 'institution_id': ?1, 'end_date': { $gte: { $date: '?2' } } } ")
List<CourseClass> findAllByExperienceDeviceAndInstitutionIdAndEndDate(String deviceId, Long institutionId, Date dateLimit);
// Even hardcoding the date parameter, the query throws an error
// ERROR: org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class java.time.ZonedDateTime.
#Query(" { 'experience_enrollments.device_id' : ?0, 'institution_id': ?1, 'end_date': { '$gte': { '$date': '2015-05-16T07:55:23.257Z' } } }")
List<CourseClass> findAllByExperienceDeviceAndInstitutionIdAndEndDate(String deviceId, Long institutionId);
}
Database Configurations
#Configuration
#EnableMongoRepositories("br.com.pixinside.lms.course.repository")
#Profile("!" + JHipsterConstants.SPRING_PROFILE_CLOUD)
#Import(value = MongoAutoConfiguration.class)
#EnableMongoAuditing(auditorAwareRef = "springSecurityAuditorAware")
public class DatabaseConfiguration {
#Bean
public MongoCustomConversions customConversions() {
List<Converter<?, ?>> converters = new ArrayList<>();
converters.add(DateToZonedDateTimeConverter.INSTANCE);
converters.add(ZonedDateTimeToDateConverter.INSTANCE);
return new MongoCustomConversions(converters);
}
}
Date converters
public static class DateToZonedDateTimeConverter implements Converter<Date, ZonedDateTime> {
public static final DateToZonedDateTimeConverter INSTANCE = new DateToZonedDateTimeConverter();
private DateToZonedDateTimeConverter() {
}
#Override
public ZonedDateTime convert(Date source) {
return source == null ? null : ZonedDateTime.ofInstant(source.toInstant(), ZoneId.systemDefault());
}
}
public static class ZonedDateTimeToDateConverter implements Converter<ZonedDateTime, Date> {
public static final ZonedDateTimeToDateConverter INSTANCE = new ZonedDateTimeToDateConverter();
private ZonedDateTimeToDateConverter() {
}
#Override
public Date convert(ZonedDateTime source) {
return source == null ? null : Date.from(source.toInstant());
}
}
Turns out that, as mentioned by Christoph Strobl, the behavior was, in fact, a bug. So it won't be necessary to worry about that in a future version of Spring Data MongoDB. Until there, I'm sharing my solution.
Since I was unable to use MongoDBJSon to create the query, I used the MongoTemplate and everything was just fine.
import org.springframework.data.mongodb.core.MongoTemplate;
import static org.springframework.data.mongodb.core.query.Criteria.where;
import static org.springframework.data.mongodb.core.query.Query.query;
#Autowired
public MongoTemplate mongoTemplate;
public List<CourseClass> findEnrolledOnExperienceDeviceWithMaxEndDateAndInstitutionId(String deviceId, LocalDate endDate, Long institutionId) {
return mongoTemplate.find(query(
where("experience_enrollments.device_id").is(deviceId)
.and("institution_id").is(institutionId)
.and("end_date").gte(endDate)), CourseClass.class);
}
i have a problem with my transaction.
i have 4 entities - entityA, entityB, entityC and entityD - all are different objects (for different tables in my DB).
for each of them i have a repository that looks like this:
#Repository
public interface entityXRepository extends JPARepository<entityX, Long> {}
(entityX is any of the above)
and i also have a service for every of the entityB, entityC, entityD, which looks like:
#Service
public class entityXService {
#Autowired
entityXRepository repository;
public entityX save(entityX toSave) {
return repository.save(toSave);
}
}
as you can see i don't use #transactional on the save.
and my service for entityA looks like this:
#Service
public class entityAService {
#Autowired
entityARepository repository;
public entityA save(entityA toSave) {
return repository.save(toSave);
}
public void doSomeStuff(...) throws Exception {
// do a lot of stuff and create entities for A,B,C,D and i throw exception
//if something happens here
saveAllToDB(entityAToSave, entityBToSave, entityCToSave, entityDToSave);
}
#Transactional(rollbackFor = "Exception.class")
public void saveAllToDB(entityA a, entityB b, entityC c, entityD d) {
serviceA.save(a);
if(b != null) {
serviceB.save(b);
}
if(c != null) {
serviceC.save(c);
}
if(d != null) {
serviceD.save(d);
}
}
}
in doSomeStuff i get input from the client and create the entities, and if the input violate some rule i throw exception with message to the user.
as you can see only the saveAllToDB is Transactional.
when there is an exception in saving one of the entities like entityD there is no rollback only an exception being thrown to the error controller and entity A to B are saved in the DB. (to check this, i entered to D a number with 12 digits, and the db column is 10 digits)
i tired putting #Transactional on all of the save functions in the services but it didn't work either.
can anyone help me?
NOTE: Go down in order to see the edited message.
I'm trying to imitate this query:
db.sentiments.aggregate([
{"$group" : {_id:{theme_id:"$theme",sentiment_id:"$sentiment"}, count:{$sum:1}}},
{"$sort":{"_id.theme_id":1}} ])
This is the code that I had generated in order to imitate it:
#RepositoryRestResource(collectionResourceRel = "sentiments", path = "sentiments")
public interface SentimentsRepository extends MongoRepository<Sentiments, String> {
Long countByTheme(#Param("theme") String theme);
#Query(value ="[\n" +
" {\"$group\" : {_id:{theme_id:\"$theme\",sentiment_id:\"$sentiment\"}, count:{$sum:1}}},\n" +
"\t{\"$sort\":{\"_id.theme_id\":1}}\n" +
"]",count = true)
List<Object> comptarSentiments();
}
Well this code is returning me this error:
"exception": "org.springframework.data.mongodb.UncategorizedMongoDbException",
"message": "Can't canonicalize query: BadValue unknown operator: $group; nested exception is com.mongodb.MongoException: Can't canonicalize query: BadValue unknown operator: $group",
Actually I'm a begginer in what refers to the use of Spring so I'm very lost, does any one know what should I do?
Thanks and sorry for my bad english, not my native language.
[EDIT]----------------------------------------
Just as the comment wrote by Shawn Clark It's not possible to do it this way, in order to achieve that you will need to create a customRepository.
What's the difference between Spring Data's MongoTemplate and MongoRepository?
I have been trying to do it this way but something doesn't seem to be correct, here is my new code:
#RepositoryRestResource(collectionResourceRel = "sentiments", path = "sentiments")
public interface SentimentsRepository extends CrudRepository<Sentiments, String>, CustomSentimentsRepository {
//Other methods...
}
public interface CustomSentimentsRepository {
List<CountResult> yourCustomMethod();
class CountResult{
String theme;
String sentiment;
int total;
}
}
public class SentimentsRepositoryImpl implements CustomSentimentsRepository {
private final MongoOperations operations;
#Autowired
public SentimentsRepositoryImpl(MongoOperations operations) {
Assert.notNull(operations, "MongoOperations must not be null!");
this.operations = operations;
}
#Override
public List<CountResult> yourCustomMethod(){
Aggregation agg = Aggregation.newAggregation(
Aggregation.group("theme","sentiment").count().as("total"),
Aggregation.project("theme","sentiment").and("total").previousOperation(),
Aggregation.sort(Sort.Direction.DESC, "theme")
);
//Convert the aggregation result into a List
AggregationResults<CountResult> groupResults
= operations.aggregate(agg,"sentiments", CountResult.class);
//List<CountResult> result = groupResults.getMappedResults();
return groupResults.getMappedResults();
}
}
I'm not even able to debbug this code and I'm always getting a 404.
Based on the information I have found you can't do that complex of a #Query on a MongoRepository method. In this case you would want to create a class and implement your comptarSentiments() method using the mongoTemplate to query the data store with your aggregate function. Then create a controller class that exposes a REST endpoint and have it call the repository.
Once you get to doing complex queries in Mongo you lose the ease of #RepositoryRestResource and have to go back to wiring the REST endpoint to the repository yourself.
Spring Data REST : custom query for MongoDB repository
Implementing custom methods of Spring Data repository and exposing them through REST
I finally managed to solve the problem, seems like it was related with the controller and the type of the atribute "total" from the innerClass CountResult, it needs to be a String (this is very important, otherwise the Aggregation.project will fail). Here goes the final code:
public interface CustomSentimentsRepository {
List<CountResult> myCountGroupByThemeAndSentiment();
class CountResult{
public String theme;
public String sentiment;
public String total;
}
}
public class SentimentsRepositoryImpl implements CustomSentimentsRepository {
private final MongoTemplate mongoTemplate;
#Autowired
public SentimentsRepositoryImpl(MongoTemplate mongoTemplate) {
this.mongoTemplate = mongoTemplate;
}
#Override
public List<CountResult> myCountGroupByThemeAndSentiment(){
Aggregation agg = Aggregation.newAggregation(
Aggregation.group("theme","sentiment").count().as("total"),
Aggregation.project("theme","sentiment").andInclude("total"),
Aggregation.sort(Sort.Direction.ASC,"theme","sentiment")
);
AggregationResults<CountResult> groupResults
= mongoTemplate.aggregate(agg,"sentiments", CountResult.class);
return groupResults.getMappedResults();
}
}
#RepositoryRestResource(collectionResourceRel = "sentiments", path = "sentiments")
public interface SentimentsRepository extends CrudRepository<Sentiments, String>, CustomSentimentsRepository {
//Other methods
}
#RestController
#RequestMapping(value = "sentiments/search")
public class ChartsController {
#Autowired
private SentimentsRepository sentimentsRepository;
#RequestMapping(value = "myCountGroupByThemeAndSentiment", method = RequestMethod.GET)
public ResponseEntity<?> yourCustomMethod() {
List<?> count=sentimentsRepository.myCountGroupByThemeAndSentiment();
return new ResponseEntity(count, HttpStatus.OK);
}
}
You can use #Aggrgation available in spring data mongodb 2.2.X versions:
#Aggregation(pipeline = {"{ '$group': { '_id' : '$lastname', names : { $addToSet : '$?0' } } }", "{ '$sort' : { 'lastname' : -1 } }"}) List<PersonAggregate> groupByLastnameAnd(String property);
I'm working on a web application using angular js, spring mvc and spring jpa data.
I'm wondering if there is something similar to criteria and detachedcriteria(hibernate) to build advanced queries with spring jpa data.
Nothing stops you from still using Criteria
#Repository
public interface FooRepository extends JpaRepository<Foo, Long>, FooRepositoryCustom {
}
interface FooRepositoryCustom {
public List<Foo> findByBar(Bar bar);
}
class FooRepositoryImpl implements FooRepositoryCustom {
#PersistenceContext
protected EntityManager em;
#Transactional(readOnly = true)
public List<Foo> findByBar(Bar bar) {
Criteria crit = em.unwrap(Session.class).createCriteria(Foo.class);
crit.add(Restrictions.eq("name", bar.getName()));
...
crit.setResultTransformer(DetachedCriteria.DISTINCT_ROOT_ENTITY);
List<Foo> foos = crit.list();
return foos;
}
}
Yes, you can use Specifications, which basically uses the Criteria API (obviously, since Spring Data JPA is just a wrapper around JPA).
you can use Query Dsl
, it is less verbose than Specification, here is a blog containing both Specification and QueryDsl.
You can use Criteria with Spring Data, you don't need a Custom Repository, You could use JpaSpecificationExecutor, here an example:
Your repository:
#Repository("yourRepository")
public interface YourRepository extends JpaRepository, JpaSpecificationExecutor
{
}
Your Service
#Override
public List<YourModel> yourDataModel getAllEntitiesByAttr(String attrValue){
List<YourModel> yourDataModel = null;
try {
Specification specification=getAndSpecByAttribute("attribute",attrValue);
List list = userRepository.findAll(specification);
yourDataModel =orikaMapper.mapAsList(list, YourModel.class);
} catch (Exception e) {
throw e;
}
return yourDataModel;
}
private Specification getAndSpecByAttribute(String attribute, String valueAttribute){
return new Specification() {
#Override public Predicate toPredicate(Root root, CriteriaQuery query, CriteriaBuilder cb) {
Path path = root.get(attribute);
return cb.equal(path, valueAttribute);
};
};
}
It is enough.
I'm using Spring data jpa & hibernate for data access along with Spring boot. All the repository beans are singleton by default. I want to define the scope of all my repositories to Prototype. How can I do that?
#Repository
public interface CustomerRepository extends CrudRepository<Customer, Long> {
List<Customer> findByLastName(String lastName);
}
Edit 1
The problem is related to domain object being shared in 2 different transactions which is causing my code to fail. I thought it is happening because repository beans are singleton. That's the reason I asked the question. Here is the detailed explanation of the scenario.
I have 2 entities User and UserSkill. User has 1-* relationship with UserSkills with lazy loading enabled on UserSkill relation.
In a UserAggregationService, I first make a call to fetch an individual user skill by id 123 which belongs to user with id 1.
public class UserAggregationService {
public List<Object> getAggregatedResults() {
resultList.add(userSkillService.getUserSkill(123));
//Throws Null Pointer Exception. See below for more details.
resultList.add(userService.get(1));
}
}
Implementation of UserSkillService method looks like
#Override
public UserSkillDTO getUserSkill(String id) {
UserSkill userSkill = userSkillService.get(id);
//Skills set to null avoid recursive DTO mapping. Dozer mapper is used
//for mapping.
userSkill.getUser().setSkills(null);
UserSkillDTO result = mapper.map(userSkill, UserSkillDTO.class);
return result;
}
In the call of user aggregation service, I call UserService to fetch userDetails. UserService code looks like
#Override
public UserDTO getById(String id) {
User user = userService.getByGuid(id);
List<UserSkillDTO> userSkillList = Lists.newArrayList();
//user.getSkills throws null pointer exception.
for (UserSkill uSkill : user.getSkills()) {
//Code emitted
}
....
//code removed for conciseness
return userDTO;
}
UserSkillService method implementation
public class UserSkillService {
#Override
#Transactional(propagation = Propagation.SUPPORTS)
public UserSkill get(String guid) throws PostNotFoundException {
UserSkill skill = userSkillRepository.findByGuid(guid);
if (skill == null) {
throw new SkillNotFoundException(guid);
}
return skill;
}
}
UserService method implementation:
public class UserService {
#Override
#Transactional(readOnly = true)
public User getByGuid(String guid) throws UserNotFoundException {
User user = userRepo.findByGuid(guid);
if (user == null) {
throw new UserNotFoundException(guid);
}
return user;
}
}
Spring boot auto configuration is used to instantiate entity manager factory and transaction manager. In the configuration file spring.jpa.* keys are used to connect to the database.
If I comment the below line of code, then I do not get the exception. I am unable to understand why change in the domain object is being affecting the object fetch in a different transaction.
userSkill.getUser().setSkills(null);
Please suggest If I have missed something.