I have found that using a very common query like this:
#Query(nativeQuery=true, value="select * from questions where id in (22,45,65,777,444)") results in a list of objects instead of a list of questions object.
[[Ljava.lang.Object;#49fd8d8f, [Ljava.lang.Object;#2558cd3c, [Ljava.lang.Object;#690a314b]
How is this situation going to be fixed?
How can I manage this right now?
Thanks!
Edit:
#RooJpaRepository(domainType = Challenge.class)
public interface ChallengeRepository extends ChallengeRepositoryCustom
{
#Query(nativeQuery=true, value="select * from questions where id in (22,45,65,777,444)")
List<Questions> chooseRandomOldQuestions();
}
I strongly feel like this has nothing to do with the in clause but the types you use.
Your repository is typed to the Challenge domain class. Question (I guess you mean that rather than Questions) is probably not a sub type of Challenge right? If this is the case we do not hand the domain type to the EntityManager to accommodate scenarios where people execute projections to Long etc. and the query execution would fail if we handed a non-managed type to the query execution.
The workaround is to create a dedicated repository managing Question(s) instances (still not sure this is a typo or a separate class) should do the trick.
Related
I want many to many relation mapping with explicit join table.
My java spring project is providing REST api in HAL format and now it has only two types of classed:
entities defined and
"empty" interfaces for repositories (annotated by #RepositoryRestResource).
Sidenote, dependencies are circa these:
spring-boot-starter-parent
spring-boot-starter-data-jpa
spring-boot-starter-data-rest
spring-data-rest-hal-browser
postgresql / h2
spring-boot-starter-hateoas
Relations between tables (I mean _links of rest resources. have a look at sample hal+json document and look for ea:basket for example.) are working as expected and it is working "for free", because of magic powder from spring auto-configuration and other magic included.
I am struggling now when adding new many to many dependency.
I have entities A, B and Tag. I want to have any number of Tag entities to be associated with A an B entities. I have no need to have any list/set in A and B entities (I will use jooq if I need anything more than crud).
First problem/question:
I see at least three approaches to model many to many relation with explicit join table (to be able to model my association needs) and I do not know differences. What are differences of these approaches?:
use embeddable composite key in join table as Vlad suggest? https://vladmihalcea.com/the-best-way-to-map-a-many-to-many-association-with-extra-columns-when-using-jpa-and-hibernate/
use #IdClass approach as stated here: https://stackoverflow.com/a/3588400/11152683
use multiple #Id in join table as shown here: https://hellokoding.com/jpa-many-to-many-extra-columns-relationship-mapping-example-with-spring-boot-hsql/
Second question:
What approach is needed in my case to easily model associations and make magic powder do its work when providing HAL format in crud repositories. I mean that links of relations will be generated automagically.
To make many2many association work nicely with spring data rest things and provide HAL representation with correct things, you have first know a bit of JPA/Hibernate. There are two approaches (1 and 2 from question, as third is only shortcut for second one and working in Hibernate only.).
Both approaches are shown in proof of concept repository in tags branch. I use given repository to test various set-ups of project.
Approach 1, EmbeddedId. It does use hackish thing in FixConfig class in BackendIdConverter bean, where it uses bookRepository to get Book entity when it parses request id from url onto embedable id class.
Approach 2, IdClass. It is using plain Integers int its IdClass and does seem to be correct solution.
I think that first approach can be modified to work similarly as second, but I am not able to do it for now.
I would mark as solution any answer providing some insight for "why" it is like this.
I read that getOne() is lazy loaded and findOne() fetches the whole entity right away. I've checked the debugging log and I even enabled monitoring on my sql server to see what statements gets executed, I found that both getOne() and findOne() generates and executes the same query. However when I use getOne() the values are initially null (except for the id of course).
So could anyone please tell me, if both methods executes the same query on the database, why should I use one over the other? I'm basically looking for a way to fetch an entity without getting all of its children/attributes.
EDIT1:
Entity code
Dao code:
#Repository
public interface FlightDao extends JpaRepository<Flight, Long> {
}
Debugging log findOne() vs getOne()
EDIT2:
Thanks to Chlebik I was able to identify the problem. Like Chlebik stated, if you try to access any property of the entity fetched by getOne() the full query will be executed. In my case, I was checking the behavior while debugging, moving one line at a time, I totally forgot that while debugging the IDE tries to access object properties for debugging purposes (or at least that's what I think is happening), so debugging triggers the full query execution. I stopped debugging and then checked the logs and everything appears to be normal.
getOne() vs findOne() (This log is taken from MySQL general_log and not hibernate.
Debugging log
No debugging log
It is just a guess but in 'pure JPA' there is a method of EntityManager called getReference. And it is designed to retrieve entity with only ID in it. Its use was mostly for indicating reference existed without the need to retrieve whole entity. Maybe the code will tell more:
// em is EntityManager
Department dept = em.getReference(Department.class, 30); // Gets only entity with ID property, rest is null
Employee emp = new Employee();
emp.setId(53);
emp.setName("Peter");
emp.setDepartment(dept);
dept.getEmployees().add(emp);
em.persist(emp);
I assume then getOne serves the same purpose. Why the queries generated are the same you ask? Well, AFAIR in JPA bible - Pro JPA2 by Mike Keith and Merrick Schincariol - almost every paragraph contains something like 'the behaviour depends on the vendor'.
EDIT:
I've set my own setup. Finally I came to conclusion that if You in any way interfere with entity fetched with getOne (even go for entity.getId()) it causes SQL to be executed. Although if You are using it only to create proxy (eg. for relationship indicator like shown in a code above), nothing happens and there is no additional SQL executed. So I assume in your service class You do something with this entity (use getter, log something) and that is why the output of these two methods looks the same.
ChlebikGitHub with example code
SO helpful question #1
SO helpful question #2
Suppose you want to remove an Entity by id. In SQL you can execute a query like this :
"delete form TABLE_NAME where id = ?".
And in Hibernate, first you have to get a managed instance of your Entity and then pass it to EntityManager.remove method.
Entity a = em.find(Entity.class, id);
em.remove(a);
But this way, You have to fetch the Entity you want to delete from database before deletion. Is that really necessary ?
The method EntityManager.getReference returns a Hibernate proxy without querying the database and setting the properties of your entity. Unless you try to get properties of the returned proxy yourself.
Method JpaRepository.getOne uses EntityManager.getReference method instead of EntityManager.find method. so whenever you need a managed object but you don't really need to query database for that, it's better to use JpaRepostory.getOne method to eliminate the unnecessary query.
If data is not found the table for particular ID, findOne will return null, whereas getOne will throw javax.persistence.EntityNotFoundException.
Both have their own pros and cons. Please see example below:
If data not found is not failure case for you (eg. You are just
verifying if data the data is deleted and success will be data to be
null), you can use findOne.
In another case, you can use getOne.
This can be updated as per your requirements, if you know outcomes.
TL;DR: How do I implement fine-grained access control in the flattened REST api approach that Spring-Data-Rest gives us?
So - I'm making an API using Spring-Data-Rest where there's three main access levels:
1) The admin - can see/update all groups
2) An owner of a group - can see/update the group and everything under it
3) An owner of a sub-group - can see/update only his group. No recursive nesting, just one sub-level allowed.
And 'group' is exposed as a resource (has a crud repository).
So far so good - and I've implemented some access control for modification using a Repository Event Handler - so on the create/write/delete side I think I'm fine.
Now I need to get to the point of limiting visibility of some of the items. This is ok for getting a single item since I can use Pre/Post Authorize annotations and reference the principal.
The problem lies in the findAll() methods - I don't have an easy hook to filter out the specific instances I don't want exposed based on the current principal. For example - a sub-group owner could see all groups by doing GET /groups. They should ideally have the items they don't have access to not even be visible at all.
To me this sounds like writing custom #Query() annotations on the repository interfaces, but that doesn't seem doable because:
I need to reference the principal in the query. SPeL is supposed to be supported, but doesn't seem to work at all with ?# expressions (despite this blog post suggesting otherwise: https://spring.io/blog/2014/07/15/spel-support-in-spring-data-jpa-query-definitions). I am using spring-boot with 1.1.8.RELEASE and the Evans-RELEASE train for spring-data generally.
The kind of query I need to write is going to be different depending on the access level, which can't realistically be encompassed in a single JPQL statement (if admin select all groups, else get all (sub)groups associated with the principal's user).
Therefore it sounds like I need to write some custom repository implementations for that and just reference the principal in code. Well that's ok - but it seems like a lot of work for each repository that I need to control the access to (I think this will be almost all of them). This applies to findAll and various custom search methods.
Am I approaching this wrong? Is there another approach to dynamically limiting item visibility based on the currently logged-in user that would work better? In a flat namespace like spring-data-rest exposes, I would imagine this would be a common problem.
In a prior design I just solved it by exposing everything under /api/groups/{groupId}/... and had a sub-resource locator act as a single pinch-point to control access to anything under it. No such luck in spring-data-rest.
Update: now stumbling with a custom method overriding findAll() (this works for other methods defined on my custom interface). Though this might be a separate question - I'm blocked right now. Spring-data is just not calling this when I do a GET /groups, but calling the original. Oddly enough it does use my query if I define one on the interface and mark it with #Query (perhaps custom overrides of built-in methods aren't supported anymore?).
public interface GroupRepository extends JpaRepository<Group, Long>, GroupCustomRepository {}
public interface GroupCustomRepository {
Page<Group> findAll(Pageable pageable);
}
public class GroupCustomRepositoryImpl extends SimpleJpaRepository<Group, Long> implements GroupCustomRepository {
#Inject
public GroupCustomRepositoryImpl(EntityManager em) {
super(Group.class, em);
}
#Override
public Page<Group> findAll(Pageable pageable) {
MyPrincipal principal = (MyPrincipal) SecurityContextHolder.getContext().getAuthentication().getPrincipal();
Page<Group> result;
if (principal.isAdmin()) {
result = findAll(pageable);
} else {
Specification<Group> spec = (root, query, cb) -> cb.or(
cb.equal(root, principal.getGroup()),
cb.and(cb.isNotNull(root.get(Group_.parentGroup)), cb.equal(root.get(Group_.parentGroup), principal.getGroup()))
);
result = findAll(spec, pageable);
}
return result;
}
}
Update 2: Since I can't access the principal in the #Query, and I can't override it with a custom method, I'm at a brick wall. #PostFilter doesn't work either because the return object is a Page rather than a collection.
I've decided to just wall-off the /groups to admins only, and have everyone else use different approaches (/groups/search/somethingSpecific) with #PostFilters/#PostAuthorizations.
This doesn't seem like it meshes very well with the HAL approach though. Interested in how other people are solving these kinds of issues with Spring-data-rest.
We ended up approaching this as follows:
We created a custom aspect which sits in front of the CRUD methods on a repository. It then looks up and calls an associated 'authorization handler' which is annotated on the repository that dynamically manages authorization details.
We had to be pretty heavy-handed when it came to limiting results in a findAll() query (eg: looking at /users) - essentially, only admins could list all of anything sensitive. Otherwise limited users had to use query methods for specific items.
We created some reusable authorization-related classes, and use those in certain scenarios - particularly custom queries, eg:
#PreAuthorize("#authorizations.systemAdminRead()")
#Query("select u FROM User r where ...")
List findAll();
#PostAuthorize("#otherAuthorizationHandler.readAllowed(returnObject)") ResponseObject someQuery();
All in all, it works - but it feels very clunky, and it's easy to miss things. I do wish this was baked-in to the framework more, even being able to dynamically adjust the default queries would be useful (when I was attempting this, I wasn't able to have the queries updated appropriately with #Query).
We happen to be using PostgreSQL, so the upcoming row level security (http://michael.otacoo.com/postgresql-2/postgres-9-5-feature-highlight-row-level-security/) would have fit the bill nicely, assuming we could feed it the proper authorization details via the DB connection.
Hi all patient developers using spring data graph. Since there is so less documentation and pretty poor test coverage it is sometimes very difficult to understand what is the expected behavior of the underlying framework how the framework is supposed to work. Currently i have some questions related to new fetching approach introduced in SDG 1.1. As opposite to SDG 1.1 write\read through in 2.0 only relations and related object annotated with #Fetch annotation are eagerly fetched others are supposed to be fetched lazily .. and now my first question:
Is it possible to configure SDG so that if the loading of entity and
invoking getter on lazy relation takes place in the same transaction,
requested collection is fetch automatically? Kind of Persistence
Context in transaction scope, or maybe it is planned for the feature
releases.
How can I fetch lazy collection at once for #RelatedTo annotation ? fetch() method on from Neo4jOperation allows to fetch only one entity. Do i have to iterate through whole list and fetch entity for each object? What would be the best way to check if given object is already fetched / initialized or not?
As suggestion i think it would be more intuitive if there will be kind of lazy loading exception thrown instead of getting NPE when working with not initialized objects. Moreover the behavior is misleading since when object is not initialized and all member properties are null apart from id, equals method can provide true for different objects which has not been initialized, which is quite serious issues considering for example appliance of sets
Another issue which i noticed when working with SDG 2.0.0.RC1 is following: when i add new object to not fetched collection sometimes is properly added and persisted,however sometimes is not. I wrote test for this case and it works in non deterministic way. Sometimes it fails sometimes end with success. Here is the use case:
Group groupFromDb = neoTemplate.findOne(group.getId(), Group.class);
assertNotNull(groupFromDb);
assertEquals("Number of members must be equals to 1", 1, groupFromDb.getMembers().size());
User secondMember = UserMappingTest.createUser("secondMember");
groupFromDb.addMember(secondMember);
neoTemplate.save(groupFromDb);
Group groupAfterChange = neoTemplate.findOne(groupFromDb.getId(), Group.class);
assertNotNull(groupAfterChange);
assertEquals("Number of members must be equals to saved entity", groupFromDb.getMembers().size(), groupAfterChange.getMembers().size());
assertEquals("Number of members must be equals to 2", 2, groupAfterChange.getMembers().size());
This test fails sometimes on the last assert, which would mean that sometimes member is added to the set and sometimes not. I guess that the problem lies somewhere in the ManagedFieldAccessorSet, but it is difficult to say since this is non deterministic. I run the test with mvn2 and mvn3 with java 1.6_22 and 1.6_27 and i got always the same result: sometimes is Ok sometimes test fails. Implementation of User equals seems as follows:
#Override
public boolean equals(final Object other) {
if ( !(other instanceof User) ) {
return false;
}
User castOther = (User) other;
if(castOther.getId() == this.getId()) {
return true;
}
return new EqualsBuilder().append(username, castOther.username).isEquals();
}
- I find it also a bit problematic that for objects annotated with #Fetch java HashSet is used which is serializable, while using for lazy loaded fields ManagedFieldAccessorSet is used which is not serializable and causes not serializable exception.
Any help or advice are welcome. Thanks in advance!
I put together a quick code sample showing how to use the fetch() technique Michael describes:
http://springinpractice.com/2011/12/28/initializing-lazy-loaded-collections-with-spring-data-neo4j/
The simple mapping approach was only added to Spring Data Neo4j 2.0, so it is not as mature as the advanced AspectJ mapping. We're currently working on documenting it more extensively.
The lazy loading option was also added lately. So your feedback is very welcome.
Right now SDN doesn't employ a proxy approach for the lazily loaded objects. So the automatic "fetch on access" is not (yet) supported. That's why also no exception is thrown when accessing non-loaded fields and there is no means of "discovering" if an entity was not fully loaded.
In the current snapshot there is the template.fetch() operation to fully load lazy loaded objects and collections.
We'll look into the HashSet vs. ManagedSet issue, it is correct that this is not a good solution.
For the test-case. is the getId() returning a Long object or a long primitive? It might be sensible to use getId().equals(castOther.getId()) here as reference equality is not guaranteed for Number objects.
I have a semi complicated question regarding Entity Framework4, Lambda expressions, and Data Transfer Objects (DTO).
So I have a small EF4 project, and following established OO principles, I have a DTO to provide a layer of abstraction between the data consumers (GUI) and the data model.
VideoDTO = DTO with getters/setters, used by the GUI
VideoEntity = Entity generated by EF4
My question revolves around the use of the DTO by the GUI (and not having the GUI use the Entity at all), combined with a need to pass a lambda to the data layer. My data layer is a basic repository pattern with Add. Change, Delete, Get, GetList, etc.
Trying to implement a Find method with a signature like so:
public IEnumerable<VideoDTO> Find(Expression<Func<VideoEntity, bool>> exp)
...
_dataModel.Videos.Where(exp).ToList<Video>()
---
My problem/concern is the "exp" needing to be of type VideoEntity instead of VideoDTO. I want to preserve the separation of concerns so that the GUI does not know about the Entity objects. But if I try to pass in
Func<VideoDTO, bool>
I cannot then do a LINQ Where on that expression using the actual data model.
Is there a way to convert a Func<VideoDTO,bool> to a Func<VideoEntity, bool>
Ideally my method signature would accept Func<VideoDTO, bool> and that way the GUI would have no reference to the underlying data entity.
Is this clear enough? Thanks for your help
Thanks for the repliesto both of you.
I'll try the idea of defining the search criteria in an object and using that in the LINQ expression. Just starting out with both EF4 and L2S, using this as a learning project.
Thanks again!
In architectures like CQRS there isn't need for such a conversion at all cause read & write sides of app are separated.
But in Your case, You can't runaway from translation.
First of all - You should be more specific when defining repositories. Repository signature is thing You want to keep explicit instead of generic.
Common example to show this idea - can You tell what indexes You need in Your database when You look at Your repository signature (maybe looking at repository implementation, but certainly w/o looking at client code)? You can't. Cause it's too generic and client side can search by anything.
In Your example it's a bit better cause expression genericness is tied with dto instead of entity.
This is what I do (using NHibernate.Linq, but the idea remains)
public class Application{
public Project Project {get;set;}
}
public class ApplicationRepository{
public IEnumerable<Application> Search(SearchCriteria inp){
var c=Session.Linq<Application>();
var q=c.AsQueryable();
if(!string.IsNullOrEmpty(inp.Acronym))
q=q.Where(a=>a.Project.Acronym.Contains(inp.Acronym));
/*~20 lines of similar code snipped*/
return q.AsQueryable();
}
}
//used by client
public class SearchCriteria{
public string Acronym{get;set;}
/*some more fields that defines how we can search Applications*/
}
If You do want to keep Your expressions, one way would be to define dictionary manually like this:
var d=new Dictionary<Expression<Func<VideoDTO,object>>,
Expression<Func<VideoEntity,object>>{
{x=>x.DtoPropNumberOne,x=>x.EntityPropNumberOne} /*, {2}, {3}, etc.*/
};
And use it later:
//can You spot it?
//client does not know explicitly what expressions dictionary contains
_dataModel.Videos.Where(d[exp]).ToList<Video>();
//and I'm not 100% sure checking expression equality would actually work
If You don't want to write mapping dictionary manually, You will need some advanced techniques. One idea would be to translate dto expression to string and then back to entity expression. Here are some ideas (sorting related though) that might help. Expressions are quite complicated beasts.
Anyway - as I said, You should avoid this. Otherwise - You will produce really fragile code.
Perhaps your design goal is to prevent propagation of the data model entities to the client tier rather than to prevent a dependency between the presentation layer and data model. If viewed that way then there would be nothing wrong with the query being formed the way you state.
To go further you could expose the searchable fields from VideoEntity via an interface (IVideoEntityQueryFields) and use that as the type in the expression.
If you don't want to add an interface to your entities then the more complicated option is to use a VideoEntityQuery object and something that translates an Expression<Func<VideoEntityQuery,bool>> to an Expression<Func<VideoEntity,bool>>.